Skip to content

Skip related endpoints without NATURAL_KEY during awxkit export#573

Merged
cigamit merged 1 commit into
ctrliq:mainfrom
blaipr:fix/ansible-collections-path-singular
Jul 10, 2026
Merged

Skip related endpoints without NATURAL_KEY during awxkit export#573
cigamit merged 1 commit into
ctrliq:mainfrom
blaipr:fix/ansible-collections-path-singular

Conversation

@blaipr

@blaipr blaipr commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Summary

When exporting resources (e.g. job_templates), the awxkit export code iterates over POST fields from the OPTIONS response. For write_only fields like webhook_key (added to projects in #563, and already present on job templates and workflow job templates), the key appears in both post_fields and _page.related, but the resolved endpoint (e.g. /api/v2/job_templates/N/webhook_key/) is served by a view whose page class (Base) has no NATURAL_KEY defined. This caused the awxkit export to log an ERROR, set _has_error=True, and return None for every affected resource, producing empty asset lists (for example zero job templates in an awxkit collection export).

Fix

When the resolved related endpoint has no NATURAL_KEY attribute, log a warning and skip the field instead of erroring out and dropping the whole resource. A missing NATURAL_KEY means the page class does not participate in natural key resolution at all, which is a different situation from a failed natural key construction, and it should not be fatal.

This fixes the test_export_simple failure in ctrliq/ascender-collection#165, where every JobTemplate was silently dropped from the awxkit export output.

Verification

I ran the ascender-collection unit test suite against this branch (ascender_devel image, pytest tests/unit/): 149 passed, including test_export_simple.

@blaipr blaipr force-pushed the fix/ansible-collections-path-singular branch 2 times, most recently from fe76fc0 to 54b6372 Compare July 8, 2026 16:29
@cigamit cigamit requested a review from Copilot July 8, 2026 16:50
@cigamit cigamit self-assigned this Jul 8, 2026
@cigamit cigamit added the bug Something isn't working label Jul 8, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes awxkit’s asset export flow so that related endpoints which don’t define a NATURAL_KEY (e.g., write-only related endpoints like webhook_key) are skipped instead of causing the entire resource export to fail and drop assets.

Changes:

  • Skip related endpoints that lack NATURAL_KEY during _export(), logging a warning instead of erroring out.
  • Add webhook_key entries to DEPENDENT_NONEXPORT for Project, JobTemplate, and WorkflowJobTemplate.

Comment thread awxkit/awxkit/api/pages/api.py
blaipr added a commit to blaipr/ascender-collection that referenced this pull request Jul 9, 2026
…filter

- Move the new API fields (webhook_key, webhook_ref_filter,
  webhook_service, max_retries) from no_api_parameter_ok to
  needs_param_development. no_api_parameter_ok only applies when the
  module has a parameter the API lacks; these are the opposite case
  (new API fields the modules do not support yet), which is exactly
  what needs_param_development is for. Also add job_slice_pinned_hosts,
  which was still failing.
- Drop the IgnoreNaturalKeyFilter from export.py. Filtering the log
  message cannot fix the export (awxkit still drops the assets), and it
  would hide the genuine natural-key errors that mean the export is
  incomplete. The real fix is in awxkit (ctrliq/ascender#573).
…ad of failing

When exporting resources (e.g. job_templates), the awxkit export code
iterates over POST fields from the OPTIONS response. For write_only
fields like webhook_key, the key appears in both post_fields and
_page.related (added by the serializer's get_related), but the resolved
endpoint (e.g. /api/v2/job_templates/N/webhook_key/) is served by a
view whose page class (Base) has no NATURAL_KEY defined. Previously this
caused the export to log an ERROR, set _has_error=True, and return None
for every affected resource, producing empty asset lists.

Fix:
1. Phase A (direct fields): When the resolved related endpoint has no
   NATURAL_KEY attribute, log a warning and skip the field instead of
   erroring out.
2. Documentation: Add webhook_key to DEPENDENT_NONEXPORT for Project,
   JobTemplate, and WorkflowJobTemplate.
@blaipr blaipr force-pushed the fix/ansible-collections-path-singular branch from 54b6372 to e299adf Compare July 9, 2026 17:52
@blaipr

blaipr commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

I updated the PR:

  • Removed the three DEPENDENT_NONEXPORT entries. That list is only consulted in the related-objects loop, which skips any key already present in post_fields before reaching the check, and webhook_key is a post field. So the entries were never evaluated and the generic NATURAL_KEY skip handles the case on its own.
  • Rebased on current main.

Verification: I ran the ascender-collection unit test suite against this branch (ascender_devel image, pytest tests/unit/): 149 passed, including test_export_simple, which fails against main because every JobTemplate gets dropped from the export. This also unblocks ctrliq/ascender-collection#165, whose CI installs awxkit from this repo's default branch, so it would be good to merge this one first.

@cigamit cigamit merged commit 4269129 into ctrliq:main Jul 10, 2026
cigamit pushed a commit to ctrliq/ascender-collection that referenced this pull request Jul 10, 2026
* Fix unit test failures in CI

* Fix pep8 E303: too many blank lines after removing test_no_templated_values

* Rework unit test fixes: use needs_param_development, drop export log filter

- Move the new API fields (webhook_key, webhook_ref_filter,
  webhook_service, max_retries) from no_api_parameter_ok to
  needs_param_development. no_api_parameter_ok only applies when the
  module has a parameter the API lacks; these are the opposite case
  (new API fields the modules do not support yet), which is exactly
  what needs_param_development is for. Also add job_slice_pinned_hosts,
  which was still failing.
- Drop the IgnoreNaturalKeyFilter from export.py. Filtering the log
  message cannot fix the export (awxkit still drops the assets), and it
  would hide the genuine natural-key errors that mean the export is
  incomplete. The real fix is in awxkit (ctrliq/ascender#573).

* Replace removed version-invariant unit test and refresh tools/README

test_no_templated_values used to catch a mismatch between the hardcoded
_COLLECTION_VERSION and galaxy.yml; it was dropped along with the other
unit test fixes. Add test_collection_version_matches_galaxy_yml to keep
that guarantee: it loads controller_api.py through the collection_import
fixture and compares _COLLECTION_VERSION against galaxy.yml's version.

Also update tools/README.md: the source no longer carries a
0.0.1-devel placeholder, it ships the real released version, so
template_galaxy.yml is now only needed for non-default namespace
builds or to stamp a different version at build time.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Development

Successfully merging this pull request may close these issues.

3 participants