make pulling images faster by removing files more efficiently

[rochdev]

Motivation

The current cleanup strategy takes anywhere from 1 to 2 minutes. We tried larger machines before which worked perfectly, but is more costly and made configuration more difficult.

Changes

Make pulling images faster by removing files more efficiently and focusing on less larger files instead of more smaller ones.

Before: >1 minute for 10gb
After: <10 seconds for 13gb

See https://github.com/endersonmenezes/free-disk-space for the action that was used for this.

See https://github.com/DataDog/dd-trace-js/actions/runs/25087471665/job/73506075117 for a sample run (had to do it in another repo since the action is not allowed yet).

Workflow

1. ⚠️ Create your PR as draft ⚠️
2. Work on you PR until the CI passes
3. Mark it as ready for review
   • Test logic is modified? -> Get a review from RFC owner.
   • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

• Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
• A docker base image is modified?
  • the relevant build-XXX-image label is present
• A scenario is added, removed or renamed?
  • Get a review from R&P team

[github-actions]

CODEOWNERS have been resolved as:

.github/actions/pull_images/action.yml                                  @DataDog/system-tests-core

[datadog-datadog-prod-us1]

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: bb6ef55 | Docs | Datadog PR Page | Give us feedback!}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bb6ef5514d

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Avoid wildcard path in remove_folders configuration

remove_folders includes /usr/share/az*, but this action passes each folder argument quoted (run_rm -f "${FOLDER}" in the pinned endersonmenezes/free-disk-space script), so globbing does not occur and the literal * path is not removed. On runners where Azure directories are present, this silently reclaims less space than expected and can leave docker compose pull vulnerable to running out of disk.

Useful? React with 👍 / 👎.