Sync projects on repository push via webhooks

[fernandorocagonzalez]

Summary

Projects can now be updated by a webhook. A push to the repository on GitHub, GitLab or Bitbucket Data Center triggers the same update the Sync button does, so the local copy of the project follows the repo the moment it changes, without polling schedules and without paying the "Update Revision on Launch" penalty on every job launch.

• New Enable Webhook option in the project form, same workflow as job templates: pick the service, save, and configure the generated URL and key in the repository
• Only push and tag push events start a sync (GitHub push, GitLab Push Hook/Tag Push Hook, Bitbucket DC repo:refs_changed/mirror:repo_synchronized), anything else is acknowledged with a 200 and ignored
• Optional Webhook Ref Filter: an fnmatch pattern (refs/heads/main, refs/heads/release-*) so only pushes to matching refs sync the project. Empty (the default) syncs on any push
• Duplicate deliveries are dropped: project updates record webhook_service and webhook_guid, the same dedupe jobs already do
• There is no webhook credential for projects, a sync has no status to post back, so projects only take the service/key part of the mixin

Bring your own webhook key

While wiring the UI for this it became obvious that the always-server-generated key does not play well with managing the controller as code: every time the resource is re-applied the key changes and someone has to update the repository webhook by hand.

So webhook_key is now also writable (write only) on projects, job templates and workflow job templates:

• Supply a key and it is kept as is, so the same secret can live in a vault and be applied to both the repository and the controller by automation
• Leave it blank and the behavior is exactly what it was: a new key is generated whenever the webhook service is set or changed
• Blanking the key of an active webhook generates a fresh one
• The key is never returned on the resource itself, reading it still goes through the webhook_key endpoint and its permission check, and copies of a resource always get their own key
• The key field in the UI is now an editable input with the same semantics

Design notes

• WebhookKeyTemplateMixin (service + key + rotation) extracted from WebhookTemplateMixin, which now inherits from it and adds the credential. No behavior change for templates
• The existing receivers gained a projects model kwarg. Project requests short circuit into handle_project_sync(), which fires create_project_update() with launch_type: webhook instead of building a unified job with payload extra_vars, so the sync honors the project's configured branch/refspec exactly like the Sync button
• get_event_ref_name() resolves the symbolic pushed ref per service for the filter, next to the existing get_event_ref() commit hash lookup
• Validation: manual projects cannot enable webhooks, and a webhook key cannot be set without a webhook service
• One migration (0201). Add conditional workflow connectors based on job artifacts (#505) #561 also carries a 0201, whichever lands second gets renumbered
• Docs: user guide section and the webhook_key endpoint help text
• Tests: functional coverage for the new receiver paths (signed GitHub HMAC and GitLab token deliveries, dedupe by guid, ref filter matrix, permission checks on the key endpoint, custom/blank key semantics, copy gets its own key) plus UI jest tests for the project subform and the editable key

This PR is independent from #561 and #562 and mergeable in any order relative to them.

Screenshots

The webhook options in the project form:

Project details with the webhook configuration:

[Copilot]

Pull request overview

Adds webhook-driven project sync support (GitHub/GitLab/Bitbucket DC) and extends existing webhook key handling so keys can be user-supplied (write-only) and preserved for “controller-as-code” workflows, while keeping secrets retrievable only via the dedicated webhook_key endpoint.

Changes:

• Add project webhook receivers that trigger SCM sync on push/tag events with optional ref filtering and best-effort deduplication.
• Extract WebhookKeyTemplateMixin to share service/key/rotation semantics across projects and templates; make webhook_key writable (write-only) for projects, job templates, and workflow job templates.
• Extend UI + docs + functional/UI tests to support editable webhook keys and project webhook configuration (including ref filter).

Reviewed changes

Copilot reviewed 29 out of 29 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
docs/docsite/rst/userguide/webhooks.rst	Documents project webhooks, ref filtering, and BYO webhook key behavior.
awx/ui/src/screens/Template/WorkflowJobTemplateEdit/WorkflowJobTemplateEdit.js	Adds webhook_key into WFJT update payload (currently only when truthy).
awx/ui/src/screens/Template/WorkflowJobTemplateAdd/WorkflowJobTemplateAdd.js	Adds webhook_key into WFJT create payload (currently only when truthy).
awx/ui/src/screens/Template/shared/WorkflowJobTemplateForm.test.js	Updates expectation: webhook key input is editable (not readonly).
awx/ui/src/screens/Template/shared/WebhookSubForm.test.js	Updates key/service behavior tests; adds project webhook subform tests.
awx/ui/src/screens/Template/shared/WebhookSubForm.js	Makes webhook key input editable; supports project mode + ref filter + project key rotation API.
awx/ui/src/screens/Template/shared/JobTemplateForm.test.js	Updates expectations for editable key input and empty initial key behavior.
awx/ui/src/screens/Template/shared/JobTemplateForm.js	Sets initial webhook_key to empty string instead of placeholder text.
awx/ui/src/screens/Template/JobTemplateEdit/JobTemplateEdit.js	Adds webhook_key into JT update payload (currently only when truthy).
awx/ui/src/screens/Template/JobTemplateAdd/JobTemplateAdd.js	Adds webhook_key into JT create payload (currently only when truthy).
awx/ui/src/screens/Project/shared/ProjectSubForms/SharedFields.js	Adds “Enable Webhook” toggle and mounts webhook subform for projects.
awx/ui/src/screens/Project/shared/ProjectForm.test.js	Adds project form tests for webhook subform visibility and fields.
awx/ui/src/screens/Project/shared/ProjectForm.js	Adds project webhook fields to initial form values.
awx/ui/src/screens/Project/shared/Project.helptext.js	Adds help text for enabling and configuring project webhooks.
awx/ui/src/screens/Project/ProjectEdit/ProjectEdit.js	Enables sending webhook_key in project update payload (currently only when truthy).
awx/ui/src/screens/Project/ProjectDetail/ProjectDetail.js	Displays webhook service/URL and ref filter on the project details page.
awx/ui/src/screens/Project/ProjectAdd/ProjectAdd.js	Enables sending webhook_key in project create payload (currently only when truthy).
awx/ui/src/screens/Project/Project.js	Fetches project webhook key via related.webhook_key endpoint when permitted.
awx/ui/src/api/models/Projects.js	Adds readWebhookKey() / updateWebhookKey() client methods for projects.
awx/main/tests/functional/api/test_webhooks.py	Adds functional coverage for project webhook receivers, permissions, key semantics, and ref filter.
awx/main/models/workflow.py	Ensures WFJT copies discard webhook_key.
awx/main/models/projects.py	Adds webhook support to Project (mixin + ref filter) and records webhook metadata on ProjectUpdate.
awx/main/models/mixins.py	Extracts WebhookKeyTemplateMixin; updates key rotation semantics; keeps credential logic in WebhookTemplateMixin.
awx/main/models/jobs.py	Ensures JT copies discard webhook_key.
awx/main/migrations/0201_project_webhooks.py	Adds DB fields for project webhooks and webhook metadata on project updates.
awx/api/views/webhooks.py	Extends webhook receivers to include projects and adds project sync handler + ref filter logic.
awx/api/urls/project.py	Exposes webhook endpoints under /api/v2/projects/<id>/....
awx/api/templates/api/webhook_key_view.md	Updates endpoint docs to include projects and writable webhook_key field behavior.
awx/api/serializers.py	Adds write-only webhook_key fields + validation; exposes project webhook receiver/key related URLs; includes webhook fields on ProjectUpdate serializer.

[cigamit]

Do we need the webhook ref filter field? Since we already have a branch field, seems like we could check and see if there is a branch set. And if so then only sync if scm_refspec == os.path.basename(ref_name);

[cigamit]

Need to fix the migration file number now that #561 was merged

Scratch that, need to rebase on main, as it contains the changes from the other PRs.

[fernandorocagonzalez]

Rebased on main (now includes #561 and #562), should be clean.

About the ref filter question, good point. Using scm_branch would cover the most common case and I thought about that initially. The reason I went with a separate filter is that the project sync always pulls all remote branches regardless of scm_branch, so scm_branch is really about which branch gets checked out at job launch time, not about what the sync fetches.

Where a dedicated filter adds value is CI/testing workflows: say you have a project with scm_branch set to dev, but you want the project to sync whenever someone pushes a feature branch (feature/*) so you can then launch a job template with branch override pointing to that feature branch for testing. With scm_branch alone you can only trigger on pushes to dev, and you lose the ability to react to pushes on other branches that you might want to test via override.

That said, it's a more advanced use case and I'm fine either way. If you'd rather keep it simple with scm_branch for now we can always add the pattern filter later in a follow up. Let me know what you think.

[cigamit]

I am fine with the ref filter, as I had thought about several other use cases after posting that.

[cigamit]

Oh, we do need to fix the migration file, its named 201 also, and relies on the same previous 0200_add_list_ordering as #561.

[cigamit]

Great work.