Add object-storage-mock test utility#4570
Conversation
This PR donates the polaris-object-storage-mock module to Polaris. The module provides an embeddable object storage mock server for tests. It exposes provider-facing HTTP surfaces for S3, GCS, ADLS Gen2, and STS, while keeping the backing bucket behavior controllable by tests. This lets Polaris tests exercise real SDK/client behavior without depending on external cloud services. See tools/object-storage-mock/README.md for more detail on the module’s purpose, how it differs from a simple file-system or in-memory object store, and how it can be used. This PR is intentionally scoped to the initial donation/import of the object-storage-mock into Polaris. Follow-up changes, cleanup, feature work, broader integration, or behavior changes should be handled in separate PRs.
| @@ -0,0 +1,33 @@ | |||
| /* | |||
| * Copyright (C) 2024 Dremio | |||
There was a problem hiding this comment.
Shouldn't we change the headers to ASF headers + add CODE_COPIED_TO_POLARIS?
There was a problem hiding this comment.
Once a file/code has substantially changed.
There was a problem hiding this comment.
CODE_COPIED_TO_POLARIS still makes sense, I think
There was a problem hiding this comment.
Not sure about CODE_COPIED_TO_POLARIS - it felt like a good idea, but IDK 🤷
There was a problem hiding this comment.
but wasn't CODE_COPIED_TO_POLARIS added specifically to track copied/contributed files?
There was a problem hiding this comment.
It's not really enforced and a lot of contributed files already do not have it.
github-project-automation
Bot
moved this from PRs In Progress
to Ready to merge
in Basic Kanban Board
|
Hi @snazy, thanks so much for putting this together! Donating the polaris-object-storage-mock utility is a massive contribution. Given that this is a substantial new module donation to the project, could we start a quick thread on the dev mailing list to discuss the motivation, the integration plan and the header/licensing tracking. Since it's a large import, bringing it to the dev list ensures the broader community has visibility. Appreciate your hard work on this! |
Agree with @flyrain . This warrants a good discussion |
|
This is huge😃! At a glance, I wonder if this is better to go with polaris main repo or should go separately? I feel a dev thread can definitely get the folks up to speed on the rationale, motivation etc. |
|
This PR is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days. |
github-project-automation
Bot
moved this from Ready to merge
to Done
in Basic Kanban Board
6 participants
This PR donates the polaris-object-storage-mock module to Polaris.
The module provides an embeddable object storage mock server for tests. It exposes provider-facing HTTP surfaces for S3, GCS, ADLS Gen2, and STS, while keeping the backing bucket behavior controllable by tests. This lets Polaris tests exercise real SDK/client behavior without depending on external cloud services.
See tools/object-storage-mock/README.md for more detail on the module’s purpose, how it differs from a simple file-system or in-memory object store, and how it can be used.
This PR is intentionally scoped to the initial donation/import of the object-storage-mock into Polaris. Follow-up changes, cleanup, feature work, broader integration, or behavior changes should be handled in separate PRs.