This demonstrate a neat trick, whereby we store several large files in Gzip format somewhere (in a database for example), and if we need to retrieve several of them at once, we create a Zip archive on the fly without decompressing the Gzip files, but instead by extracting the compressed section of each Gzip file and sticking it as-is in the Zip archive.
This allows:
- saving storage space in the database
- sending individual gzip file compressed with the
Content-EncodingHTTP header set togzip, causing it to uncompressed transparently by the remote browser - sending several files at once as a Zip archive, without needing to decompress and recompress them server side
We provide a ZipFileFromGzip class that derives from Zipfile, and patches internal methods to hook a custom compressor that extracts the compressed data from the provided gzip'ed file, instead of a normal compression step.
This means using Zipfile directly works completely normally, it's only when using ZipFileFromGzip that files added to the archive are expected to be gzip'ed.
Technically, we could also compress files using DEFLATE, as Content-Encoding also supports deflate, and since we use ZIP_DEFLATE in the Zip archive, maybe this would be more straight forward.
Sadly no: the deflate content encoding actually means a DEFLATE compressed stream, inside a zlib data format (https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Content-Encoding#deflate), whereas Zip expects the raw DEFLATE stream with no encoding. So we would have to do the same as for gzip: extract the compressed section and skip the header and footer. So an equivalent amount of work (thought the Zlib format is simpler than Gzip).
But even then, we'd run into another issue: the Zip archive format wants to know the size of the data before compression, which is not retained in the Zlib format, but is in the gzip format, which also uses the same CRC32 as Zip, so we can save those when extracting the compressed data, and patch them into the Zip archive as needed.