Skip to content

Implement support for dictionary compression (draft-ietf-httpbis-compression-dictionary-19) #83

Description

@CAFxX

Implement support for dictionary compression (draft-ietf-httpbis-compression-dictionary-19)

Agent Context [ "### Implement Dictionary-Compressed Brotli (`dcb`)\n\n1. **Create `contrib/andybalholm/brotli/dcb.go`:**\n * Define a new `dcbCompressor` that embeds the existing `brotli.compressor`.\n * This new compressor will take a dictionary as a parameter.\n * Implement the `Get` method to write the `dcb` header (magic number and dictionary SHA-256 hash) before writing the compressed data.\n\n2. **Create `dcb.go`:**\n * Define the `DcbCompressor` option, similar to `BrotliCompressor`.\n * This option will register the \"dcb\" content encoding with a new `dcbCompressorProvider`.\n\n### Implement Dictionary-Compressed Zstandard (`dcz`)\n\n3. **Create `contrib/klauspost/zstd/dcz.go`:**\n * Define a new `dczCompressor` that embeds the existing `zstd.compressor`.\n * This new compressor will take a dictionary as a parameter.\n * Implement the `Get` method to write the `dcz` header (magic number and dictionary SHA-256 hash) before writing the compressed data.\n\n4. **Create `dcz.go`:**\n * Define the `DczCompressor` option, similar to `ZstandardCompressor`.\n * This option will register the \"dcz\" content encoding with a new `dczCompressorProvider`.\n\n### Integrate Dictionary Handling into the Middleware\n\n5. **Modify `adapter.go`:**\n * In the `Adapter` function, add logic to parse the `Available-Dictionary` header from the request.\n * If the header is present, extract the dictionary hash.\n * *This step will require a mechanism to look up the dictionary content based on the hash. For this implementation, we will assume a simple in-memory map for dictionaries. A more robust solution would involve a persistent cache.*\n * If a dictionary is found, instantiate the `dcb` and `dcz` compressors with the dictionary.\n * Add the `dcb` and `dcz` encodings to the list of supported encodings for the request.\n * Add the `Vary: Available-Dictionary` header to the response.\n\n6. **Modify `response_writer.go`:**\n * In the `Write` method of `compressWriter`, when initializing the compressor, check if the selected encoding is `dcb` or `dcz`.\n * If so, pass the dictionary to the compressor.\n\n7. **Modify `prefer.go`:**\n * In `preferredEncoding`, ensure that `dcb` and `dcz` are only considered if a dictionary is available for the request. This can be done by checking for the presence of the dictionary hash from the `Available-Dictionary` header.\n\n### Add Dictionary Management (Server-side)\n\n8. **Create `dictionary.go`:**\n * Implement a simple in-memory dictionary cache (a map of hash to dictionary content).\n * Add functions to add dictionaries to the cache.\n * Implement a function to set the `Use-As-Dictionary` header on a response, which will add the response body to the dictionary cache and set the appropriate header fields (`match`, `id`, etc.).\n\n### Example and Testing\n\n9. **Update `test/server.go`:**\n * Add a new endpoint that serves a resource that can be used as a dictionary. This endpoint should use the new dictionary management functions to set the `Use-As-Dictionary` header.\n * Add another endpoint that can be compressed using the dictionary.\n * Add client-side logic to the test server to:\n 1. Fetch the dictionary.\n 2. Make a request to the compressible endpoint with the `Available-Dictionary` header.\n 3. Verify that the response is compressed with `dcb` or `dcz`.", "Implement support for dictionary compression (draft-ietf-httpbis-compression-dictionary-19)" ]

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions