GH-4186: Add license and NOTICE validation for distribution and Spark bundles

[Kaustav-Sarkar]

This PR introduces automated CI checks to enforce LICENSE and NOTICE compliance for Polaris distributions and Spark bundles.
Target Issue: #4186
Description:

• Adds a dedicated license-checks CI job in GitHub Actions to validate distribution LICENSE/NOTICE merging and Quarkus license reports independently of other checks.
• Introduces BundleJarLicenseNoticeValidation: A Gradle task that ensures shadow bundle JARs (like the Spark plugin) contain top-level LICENSE and NOTICE files.
• Introduces BundleLicenseValidation: A Gradle task that automatically verifies that every direct runtime dependency (in group:artifactId form) is correctly attributed in the bundle's BUNDLE-LICENSE file. This prevents dependency updates from silently slipping past license attribution.
• Applies these checks to the Spark plugin (checkBundleLicense, checkBundleJarLicenseNotice), running them automatically as part of the standard check task.
• Updates the Spark BUNDLE-LICENSE to correctly attribute the iceberg-spark-runtime dependencies.

[snazy]

Thanks for working on this, but I don’t think this actually addresses #4186 in a meaningful way.

The hard part here is not checking whether the bundle jar contains top-level LICENSE/NOTICE files, or whether the only two direct runtimeClasspath dependency Maven coordinates are mentioned. The hard part is validating the actual contents of bundled fat artifacts, including both the obvious dependencies with embedded pom.xml metadata and the non-obvious ones that only show up as relocated classes without Maven metadata.

As far as I can tell, this PR validate the one relevant direct runtimeClasspath artifact only. That does not catch missing, stale, or incorrect attribution for the transitive contents inside iceberg-spark-runtime, which is the main case we need to solve. The CHANGELOG entry also explicitly says that transitive attribution inside fat runtime JARs remains manually maintained, so I don’t see what this buys us for the difficult part of the issue.

Before this can move forward, I think we need a concrete investigation:

1. Which existing checks already cover distribution LICENSE/NOTICE and Quarkus license reports?
2. Which specific missing/stale/incorrect license cases does this PR catch?
3. Which cases from #4186 does it intentionally not catch?
4. How did you inspect iceberg-spark-runtime to identify both Maven-metadata-backed and relocated/no-pom dependencies?
5. Can you include at least one failing example that demonstrates this check catches a real problem that current checks miss?

Also, if significant AI assistance was used to prepare this change, please disclose that in the PR description as encouraged by CONTRIBUTING.md.

[Kaustav-Sarkar]

@snazy thanks for the review, I will investigate over the weekend and get back on this

[github-actions]

This PR is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.