In the scope of the https://dandiarchive.org (https://github.com/dandi/) project we are researching on improving our management and access to Zarrs we store. Given that zarrs are quite special in requiring to provide access to thousands of files, special provisions to ensure adequate performance might be desired. We wondered if there is some overview of the potential/existing authorized access schemes on S3 and elsewhere. Anyone could point to such a resource or individual implementations/approaches?
In the scope of the https://dandiarchive.org (https://github.com/dandi/) project we are researching on improving our management and access to Zarrs we store. Given that zarrs are quite special in requiring to provide access to thousands of files, special provisions to ensure adequate performance might be desired. We wondered if there is some overview of the potential/existing authorized access schemes on S3 and elsewhere. Anyone could point to such a resource or individual implementations/approaches?