Git log: commit 183acc8f706658da4ed11393c7babbc5817cb3b9
Merge: c4f90fe9 ec362ce1
Author: Colin Watson <cjwatson@debian.org>
Date: Mon Sep 16 17:57:13 2024 +0000
Merge branch 'piuparts-base-tgz-lookup' into 'devel'
Apply some environment constraints to the piuparts base_tgz lookup
See merge request freexian-team/debusine!1110
commit c4f90fe902744dd041167d688a8b50e9c9d63dfb
Merge: f9e55ed5 d6284aa9
Author: Carles Pina i Estany <carles@pina.cat>
Date: Mon Sep 16 17:28:21 2024 +0000
Merge branch 'workflow-coordination-populate' into 'devel'
scheduler(): call populate_workflow() only for root workflows (related #481)
See merge request freexian-team/debusine!1115
commit d6284aa9b41f793aff3686d17b7ac5bc4892c9aa
Author: Carles Pina i Estany <carles@pina.cat>
Date: Mon Sep 16 17:28:21 2024 +0000
scheduler(): call populate_workflow() only for root workflows
commit f9e55ed5e50a636e8d37f4a90a0211a7afb19354
Merge: ec75d448 63822b7d
Author: Stefano Rivera <stefanor@debian.org>
Date: Mon Sep 16 15:51:57 2024 +0000
Merge branch 'build-profiles' into 'devel'
Sbuild: Support build_profiles
See merge request freexian-team/debusine!1107
commit ec75d44898ccd14340c170bb8ab058d24777f23c
Merge: a9198b93 8a66a3b4
Author: Colin Watson <cjwatson@debian.org>
Date: Mon Sep 16 15:23:34 2024 +0000
Merge branch 'ui-external-debsign' into 'devel'
Show "Waiting for signature" card on blocked ExternalDebsign requests
See merge request freexian-team/debusine!1114
commit ec362ce1ddeeb44452a1fa67f2373d5be72208ae
Author: Colin Watson <cjwatson@debian.org>
Date: Mon Sep 16 15:50:49 2024 +0100
Simplify environment lookup interface
It now does the lookup directly rather than just constructing the lookup
string. Suggested by Jochen Sprickerhof.
commit 2ff9c8b7291bb68263910be13f3845902632010f
Author: Colin Watson <cjwatson@debian.org>
Date: Fri Sep 13 14:39:42 2024 +0100
Apply some environment constraints to the piuparts base_tgz lookup
It's easy to make the mistake of passing the same simplified environment
lookup (e.g. `debian/match:codename=sid`) for both `environment` and
`base_tgz` in the Piuparts task. However, unlike `environment`,
`base_tgz` doesn't currently apply additional constraints such as the
host architecture, and so this can easily result in a completely
inappropriate artifact such as one for a different architecture.
It doesn't necessarily make sense to apply a backend constraint, but we
can at least try to pick the right architecture and format.
commit de1f683c9dfc52bd81cbbf396a4dd2c90f6b1fde
Author: Colin Watson <cjwatson@debian.org>
Date: Mon Sep 16 15:59:36 2024 +0100
Add missing constraint to MakeSourcePackageUploadData.environment
`MakeSourcePackageUpload.compute_dynamic_data` already indirectly
asserted that this is non-None, and its dynamic sibling was already
constrained to be non-None in `MakeSourcePackageUploadDynamicData`.
commit a9198b93601e50849ceb3e4c597d1046632fa337
Merge: d87910fb 9ccc4e8c
Author: Carles Pina i Estany <carles@pina.cat>
Date: Mon Sep 16 14:49:31 2024 +0000
Merge branch 'workflow-coordination-artifact-replace-bare-data-or-another-artifact' into 'devel'
Sub-workflow coordination: WorkflowInternalManager.do_add_artifact an artifact to replace either bare data or another artifact (related #481)
See merge request freexian-team/debusine!1113
commit d87910fb4fc249d2650fdebb56c80040bcb000b5
Merge: 2b528fab cd1d88c4
Author: Colin Watson <cjwatson@debian.org>
Date: Mon Sep 16 14:19:49 2024 +0000
Merge branch 'task-package-upload' into 'devel'
Add PackageUpload task
See merge request freexian-team/debusine!1108
commit 8a66a3b43e4ebf63f05e7a526111e0df5ab04a77
Author: Colin Watson <cjwatson@debian.org>
Date: Mon Sep 16 13:49:12 2024 +0100
Show "Waiting for signature" card on blocked ExternalDebsign requests
Part of #364.
commit 9ccc4e8c5b09e54cf8df4e98e312c5a036173a5b
Author: Carles Pina i Estany <carles@pina.cat>
Date: Mon Sep 16 10:11:39 2024 +0100
WorkflowInternalManager.do_add_artifact: replace bare data with artifact
commit 2b528fab07d4dfa3c754ca7e47866abb89291aec
Merge: d1eb5bcc 4e277528
Author: Jochen Sprickerhof <jspricke@debian.org>
Date: Mon Sep 16 12:53:21 2024 +0000
Merge branch 'cleanup' into 'devel'
Drop unused attribute
See merge request freexian-team/debusine!1112
commit d1eb5bcc6c82d01ea1db0ed8aa7235494e5fef42
Merge: 8fa91be7 dd73f319
Author: Colin Watson <cjwatson@debian.org>
Date: Mon Sep 16 11:39:45 2024 +0000
Merge branch 'client-provide-signature' into 'devel'
Add "debusine provide-signature" command
See merge request freexian-team/debusine!1103
commit 8fa91be79a659c5d6ac847fc7ca844d797bdf873
Merge: 63132a6d 16ca9806
Author: Colin Watson <cjwatson@debian.org>
Date: Mon Sep 16 11:39:28 2024 +0000
Merge branch 'api-external-debsign' into 'devel'
Add work-request-external-debsign view
See merge request freexian-team/debusine!1102
commit cd1d88c4efb367482c4e786fd07e46b02af68771
Author: Colin Watson <cjwatson@debian.org>
Date: Thu Sep 12 22:18:43 2024 +0100
Add PackageUpload task
The hardest bit of this was writing useful tests. I decided it would be
best to have something resembling real FTP and SFTP servers in unit
tests, so wrote basic ones using `pyftpdlib` and `paramiko`
respectively.
While the design called for a server test, I concluded that it would be
better to put it on a worker for resilience: if an upload hangs for some
reason, then it will only take out one external worker, not one of the
much more limited pool of Celery workers. The main point of that part
of the design was to avoid needing an environment to run `dput`, which
is still fulfilled here: we use `ftplib` or `fabric` to perform the
uploads.
There are some new dependencies here, but they're quite light: around 2
MiB of installed size.
Part of #364.
commit c860c349cd31c5070edd388cdceee91f1ec9610a
Author: Carles Pina i Estany <carles@pina.cat>
Date: Mon Sep 16 07:43:16 2024 +0100
DebianEnvironmentsManagerTests, DebianSuiteSigningKeysManagerTests, WorkflowInternalManagerTests: use more Playground
commit 63132a6db70ff4248c2c7707258287fdb4bfb882
Merge: 9015122a 574bd11f
Author: Colin Watson <cjwatson@debian.org>
Date: Mon Sep 16 09:38:29 2024 +0000
Merge branch 'translation-es' into 'devel'
Add Spanish translation
See merge request freexian-team/debusine!1111
commit 9015122a55ee5a935c65b33ad38c66baed5f13b4
Merge: bd55c140 c0c7c381
Author: Carles Pina i Estany <carles@pina.cat>
Date: Mon Sep 16 09:27:10 2024 +0000
Merge branch 'workflow-coordination-add-artifact-pass-workflow' into 'devel'
Workflow coordination add artifact pass workflow (related #481)
See merge request freexian-team/debusine!1109
commit c0c7c381d3ca723d59a7f374bb716bc5f90bf8be
Author: Carles Pina i Estany <carles@pina.cat>
Date: Mon Sep 16 09:27:10 2024 +0000
*Manager.do_add_artifact: pass workflow if replacing the artifact
commit 4e277528e99eb7e9a2bf6198e0e1b0aa4fc409d7
Author: Jochen Sprickerhof <git@jochen.sprickerhof.de>
Date: Mon Sep 16 10:21:08 2024 +0200
Drop unused attribute
commit 574bd11f14bb4cd0315bed073b57debce76bd44f
Author: Colin Watson <cjwatson@debian.org>
Date: Sun Sep 15 22:07:14 2024 +0100
Add Spanish translation
Thanks, Camaleón.
Closes: #1081762
commit 63822b7d20493ae45140bf910718ab91073bf35f
Author: Stefano Rivera <stefano@freexian.com>
Date: Thu Sep 12 19:29:12 2024 +0200
Sbuild: Support build_profiles
This doesn't do anything to taint the built package artifacts, we don't
have a mechanism for that yet.
commit dd73f31901727bd3ded69d6340978401a6646b06
Author: Colin Watson <cjwatson@debian.org>
Date: Thu Sep 12 13:03:55 2024 +0100
Simplify Upload validation
There was previously some validation complexity because the constructor
validated the class before adding files to it. Since the artifact data
has already been validated by a separate model before `Upload.create`
gets as far as constructing the instance, it's safe to skip validation
by using `cls.construct` instead, allowing us to avoid some strange
checks in the validators.
commit 38c242ecdcc13bc8094aa570f69f0d537b2f262d
Author: Colin Watson <cjwatson@debian.org>
Date: Wed Sep 11 18:21:57 2024 +0100
Add "debusine provide-signature" command
This completes an `ExternalDebsign` task by running `debsign` on the
corresponding unsigned upload and submitting the signed upload back to
debusine.
Part of #364.
commit 16ca980650f4fea885fb898386c2da5865087f62
Author: Colin Watson <cjwatson@debian.org>
Date: Wed Sep 11 11:03:17 2024 +0100
Add work-request-external-debsign view
This allows a client to run `debsign` and submit the results back to
debusine.
Part of #364.
commit 37ff01164edbacbc4044b5ff56e12c0d01cd7e89
Author: Colin Watson <cjwatson@debian.org>
Date: Wed Sep 11 10:52:49 2024 +0100
Add a WorkRequest.scheduling_disabled context manager
commit bd55c140574c7f997bc8789a1dc35462b8f9ffd5
Merge: ead16315 d7c8341a
Author: Sylvain Beucler <beuc@beuc.net>
Date: Fri Sep 13 12:13:04 2024 +0000
Merge branch 'issue364-makesourcepackageupload' into 'devel'
Add MakeSourcePackageUpload Task
See merge request freexian-team/debusine!1099
commit ead16315d61c77c621f2ac15264ce203a553166d
Merge: 9bc7485a e2e749ae
Author: Jochen Sprickerhof <jspricke@debian.org>
Date: Fri Sep 13 12:12:57 2024 +0000
Merge branch 'vulture' into 'devel'
Vulture
See merge request freexian-team/debusine!1106
commit e2e749aec5d9702fbcebb6d304ba96234b770e35
Author: Jochen Sprickerhof <git@jochen.sprickerhof.de>
Date: Wed Sep 11 15:45:02 2024 +0200
Drop unused classes
commit 89cd949c86c88b73742caaa7571fa663495e1904
Author: Jochen Sprickerhof <git@jochen.sprickerhof.de>
Date: Wed Sep 11 16:50:08 2024 +0200
Use vulture to find dead code
commit 4df58bd0e91f73cad5225d732355cd15db98dd66
Author: Jochen Sprickerhof <git@jochen.sprickerhof.de>
Date: Wed Sep 11 17:06:25 2024 +0200
Fix unused variables for vulture
commit d7c8341a1e71c6b87dd7627b0fe5952ecd3b87ee
Author: Sylvain Beucler <beuc@beuc.net>
Date: Fri Sep 13 11:14:18 2024 +0000
doc: makesourcepackageupload is meant to be used with unshare backend
commit 9bc7485a074cb35a9d61694e92bd22b502e56efd
Merge: 2477da2f 65692f4b
Author: Stefano Rivera <stefanor@debian.org>
Date: Fri Sep 13 08:14:11 2024 +0000
Merge branch 'binpkg-label' into 'devel'
Improve DebianBinaryPackage label
Closes #468
See merge request freexian-team/debusine!1105
commit 07e708b6c063b11a2f023c8c38ab7fdf016582c2
Author: Sylvain Beucler <beuc@beuc.net>
Date: Thu Sep 12 18:44:35 2024 +0200
Fixes from review
commit 65692f4bcbd1d7037806f2683ec0f3f5db360e49
Author: Stefano Rivera <stefano@freexian.com>
Date: Thu Sep 12 14:52:43 2024 +0200
Improve DebianBinaryPackage label
BinaryPackage used a label that was copied from BinaryPackages, a
collection of all the binary packages built together. Labelling those
after their source package made sence, but we can do much better for
binary packages.
Fixes: #468
commit 592ee0af245d5926252a872f0025188894149367
Author: Colin Watson <cjwatson@debian.org>
Date: Wed Sep 11 18:19:35 2024 +0100
Allow uploading an artifact with some files that exist only remotely
This is useful when uploading a signed version of a `debian:upload`
artifact, where we don't want the client to have to download the files
that would remain unchanged by the signing process.
commit 2477da2ff8e7d0c2a7675a0b61970d8faf92f862
Merge: fbc015df 8bd4e8f2
Author: Carles Pina i Estany <carles@pina.cat>
Date: Thu Sep 12 08:56:39 2024 +0000
Merge branch 'workflow-coordination-by-workflow-fields' into 'devel'
CollectionItem: add created_by_workflow and removed_by_workflow (related #481)
See merge request freexian-team/debusine!1090
commit 8bd4e8f263629ecb1570c1672a7d0ca4caf6d446
Author: Carles Pina i Estany <carles@pina.cat>
Date: Thu Sep 5 10:29:05 2024 +0100
CollectionItem: add created_by_workflow and removed_by_workflow
Pass down from WorkRequest.process_update_collection_*
Adjust delete_expired command
commit fbc015dfb6414c6f8d15f14269f43536db066fcc
Merge: 02bb03d7 49149a0f
Author: Carles Pina i Estany <carles@pina.cat>
Date: Wed Sep 11 13:25:37 2024 +0000
Merge branch 'workflow-coordination-part-1' into 'devel'
Workflow coordination part 1 (related #481)
See merge request freexian-team/debusine!1089
commit 49149a0fd789baa5924228091bac73ab8e4f15a3
Author: Carles Pina i Estany <carles@pina.cat>
Date: Wed Sep 11 13:25:36 2024 +0000
Add different workflow coordination functionality
-`debusine:promise` bare data: add category name to `debusine.artifacts.models.BareDataCategory`
-validate bare data in collections against pydantic models
-change `WorkflowInternalManager.do_add_bare_data` to forbid names starting with `promise_` unless the category is `debusine:promise`
-change `WorkflowInternalManager.do_add_artifact` to copy `variables` into per-item data, and to forbid names starting with `promise_`
-change `WorkRequest._retry_supersede` to update the work request ID in any promises associated with the work request being retried
Related: #481
commit 02bb03d73d8edf79dbe3fa562cd1b82b20fc0a52
Merge: 9cffb17d 8e8740a0
Author: Colin Watson <cjwatson@debian.org>
Date: Wed Sep 11 09:46:12 2024 +0000
Merge branch 'task-external-debsign' into 'devel'
Add ExternalDebsign task
See merge request freexian-team/debusine!1095
commit 9cffb17d402704b73f051e980f3db620c28b8545
Merge: 7e04c561 1d119ad3
Author: Sylvain Beucler <beuc@beuc.net>
Date: Tue Sep 10 09:46:06 2024 +0000
Merge branch 'blhc-cleanup' into 'devel'
blhc: drop obsolete fields
See merge request freexian-team/debusine!1098
commit 7e04c56115ee5ce8ef1a896d207d69adfaaa22ca
Merge: 6b7ae3d9 0e45dc8a
Author: Sylvain Beucler <beuc@beuc.net>
Date: Tue Sep 10 09:45:39 2024 +0000
Merge branch 'docfix-piuparts' into 'devel'
piuparts: drop obsolete comments
See merge request freexian-team/debusine!1097
commit 6b7ae3d9ef3b252d9edabc243026d0b0245d1d09
Merge: 74338df5 252cdd71
Author: Enrico Zini <enrico@debian.org>
Date: Tue Sep 10 06:29:23 2024 +0000
Merge branch 'fix-permissions-rst' into 'devel'
Fixed syntax of permissions.rst
See merge request freexian-team/debusine!1100
commit 252cdd712b428e31eb1624d4de0d861a97550a3f
Author: Enrico Zini <enrico@enricozini.org>
Date: Mon Sep 9 20:11:03 2024 +0200
Fixed syntax of permissions.rst
commit 74338df5bc42e1d16aeb76c0c225c0740df4b5f7
Merge: d2891c63 f0a2dbf0
Author: Colin Watson <cjwatson@debian.org>
Date: Mon Sep 9 17:15:28 2024 +0000
Merge branch 'design-revdep-autopkgtests' into 'devel'
Add design for reverse-dependencies-autopkgtest workflow
See merge request freexian-team/debusine!1083
commit f0a2dbf013db5d08a8e676c32faba05046112a4e
Author: Colin Watson <cjwatson@debian.org>
Date: Fri Aug 16 10:34:20 2024 +0100
Add design for reverse-dependencies-autopkgtest workflow
Part of #397.
commit d2891c6369f40f2b8fc1ff0b4c583e33d9241216
Merge: 83dc60db 86b330da
Author: Enrico Zini <enrico@debian.org>
Date: Mon Sep 9 13:37:30 2024 +0000
Merge branch 'permissions' into 'devel'
Permission management design (#395)
See merge request freexian-team/debusine!1068
commit 86b330da1f98e3916b00a9a45019c6c90ea2a199
Author: Enrico Zini <enrico@enricozini.org>
Date: Thu Aug 22 09:42:44 2024 +0200
Added blueprint for the initial iteratio of permission management design
commit 83dc60dbdec103e6ed33c90fed3079cc315f7b6d
Merge: 9f39b8ab 104b92cd
Author: Colin Watson <cjwatson@debian.org>
Date: Mon Sep 9 09:29:36 2024 +0000
Merge branch 'translation-de' into 'devel'
Add German translation
See merge request freexian-team/debusine!1093
commit 477a043c453f83f78dba00a6d914891351807848
Author: Sylvain Beucler <beuc@beuc.net>
Date: Mon Sep 9 09:01:21 2024 +0200
Add MakeSourcePackageUpload Task
commit 1d119ad38370f41aa570aed37cf67eebe6923d71
Author: Sylvain Beucler <beuc@beuc.net>
Date: Mon Sep 9 08:58:22 2024 +0200
blhc: drop obsolete fields
Those were probably copy/pasted from lintian and are neither documented nor used.
commit 0e45dc8afa644298105b74dfd8ae921b12ec20f3
Author: Sylvain Beucler <beuc@beuc.net>
Date: Mon Sep 9 08:46:55 2024 +0200
piuparts: drop obsolete comments
commit 8e8740a0d11ed8372371c944d6278ac6ca511d36
Author: Colin Watson <cjwatson@debian.org>
Date: Sun Sep 8 15:14:27 2024 +0100
Add ExternalDebsign task
The actual work will be done in an API view that doesn't exist yet, but
this at least gives us the basic task model.
Part of #364.
commit 9f39b8ab21cabf112c64934c11c27311e774d493
Merge: 5cf603d2 2f3564ca
Author: Colin Watson <cjwatson@debian.org>
Date: Sun Sep 8 13:01:06 2024 +0000
Merge branch 'task-type-wait' into 'devel'
Implement Wait task type and Delay task
See merge request freexian-team/debusine!1092
commit 104b92cd6bdd20ec3378d16aa70a1ca2cfaeb705
Author: Colin Watson <cjwatson@debian.org>
Date: Sun Sep 8 13:34:40 2024 +0100
Add German translation
Thanks, Christoph Brinkhaus.
Closes: #1081125
commit 2f3564caf1a9537cab8ad610fe0e64eb180f2fae
Author: Colin Watson <cjwatson@debian.org>
Date: Fri Sep 6 14:12:28 2024 +0100
Implement Wait task type and Delay task
Part of #364 and #462.
commit 8fcd7ee4d16368d58bc5dca4362efe0f3c58b1af
Author: Colin Watson <cjwatson@debian.org>
Date: Tue Sep 3 15:29:59 2024 +0100
Document signing workers and tasks
commit 5cf603d24870eb877efdcab2dad09d979dffe014
Merge: 50b5db9b 222fecd6
Author: Colin Watson <cjwatson@debian.org>
Date: Fri Sep 6 13:49:06 2024 +0000
Merge branch 'design-dep-wait' into 'devel'
Add design for retrying dependency-wait tasks
See merge request freexian-team/debusine!1086
commit 222fecd649dcf3a33336ce6afc7b1417863fe642
Author: Colin Watson <cjwatson@debian.org>
Date: Fri Aug 30 00:03:01 2024 +0100
Add design for retrying dependency-wait tasks
Part of #462.
commit 50b5db9b97923642da50a3f7ae281408674b3795
Merge: a048f9ab aeae1425
Author: Carles Pina i Estany <carles@pina.cat>
Date: Fri Sep 6 10:06:18 2024 +0000
Merge branch 'fix-return-annotation' into 'devel'
Fix typing annotation
See merge request freexian-team/debusine!1091
commit aeae142531ddb7b3ea8fb9d511ec7ef1df555086
Author: Carles Pina i Estany <carles@pina.cat>
Date: Fri Sep 6 07:55:44 2024 +0100
Fix typing annotation
get_stream() returns a IO[bytes] or urllib3.response.HTTPResponse which
is compatible as IO[bytes]
commit a048f9ab0f1baae2fce1c4a9ee64b5150d7e19a1
Merge: 88e08796 d44e356c
Author: Colin Watson <cjwatson@debian.org>
Date: Wed Sep 4 14:45:31 2024 +0000
Merge branch 'improve-is-file-in-workspace' into 'devel'
Tighten up handling of creating artifacts with files that already exist
See merge request freexian-team/debusine!1081
commit d44e356cb1987939d6d4380722af45689b42a3e0
Author: Colin Watson <cjwatson@debian.org>
Date: Thu Aug 29 17:31:58 2024 +0100
When creating an artifact, copy references to existing visible files
For example, this allows a user to upload a signed version of a package
without having to reupload the whole thing (see !1061).
commit f11d31fcbc19703969b212d87a2f9ee00a3bbc76
Author: Colin Watson <cjwatson@debian.org>
Date: Thu Aug 29 16:39:39 2024 +0100
Tighten Workspace.is_file_in_workspace
`Workspace.is_file_in_workspace` is used to check whether a file needs
to be reuploaded when creating a new artifact, so it's important that it
doesn't allow making copies of files that the user wouldn't normally be
able to see. However, just checking that the file is in one of the file
backends used by the workspace doesn't seem very robust, since the same
file backend may be shared by multiple workspaces.
Add a check that the file is part of an artifact that's in the workspace
in question, and that it is complete (not waiting to be uploaded).
commit cc236224aa9747e1ce3c03a0eba7e6b8895a33bf
Author: Colin Watson <cjwatson@debian.org>
Date: Fri Aug 30 17:06:05 2024 +0100
Add FileInArtifact.complete
It's possible to create an artifact referring to files that already
exist in one of the workspace's file backends, which can save upload
bandwidth in some cases; but to do that safely we need to be able to
check whether the user has access to the file contents. Checking
whether the file is in the backend isn't sufficient, because a backend
might be shared among multiple workspaces and so file presence doesn't
imply visibility.
Add a new `FileInArtifact.complete` column so that we can check this
safely. For `FileInArtifact` rows created before this change, it's set
to True if and only if the file is in a store that's only used by the
artifact's workspace.
commit bdf8d41b968a306fb16a2868fedcb06a39f4ec46
Author: Colin Watson <cjwatson@debian.org>
Date: Thu Aug 29 16:07:03 2024 +0100
Be more careful about contents mismatches in file backends
If a file backend is shared between multiple workspaces, then it may be
necessary for users to reupload files that the backend already has in
order to prove that they know their contents. If somebody manages to
construct a SHA-256 hash collision, then this should not allow them to
replace existing files in a backend; the attempt should result in an
error instead.
