Skip to content

I/O: Improve processing and error handling #774

@amotl

Description

@amotl

Problem

How the I/O subsystem is currently doing its processing is, well, a bit ad hoc, and doesn't provide much guidance when it comes to informing callers about progress/status, and final outcome. /cc @hampsterx

Proposal

I would like to introduce a first-citizen model entity that keeps track of all metadata in a typical I/O conversation in the spirit of a context object, let's just call it JobInfo. That context object would be used to serve multiple purposes:

  • Track any job metadata and metrics.
  • Emit progress/status information while the job is running.
  • Be returned when a job finshes successfully.
  • Be slotted into a corresponding first-citizen exception object 1:1 when an error happens during an I/O operation, or used as a callback argument to possible error handling hook functions, see I/O: Unify on_error configuration across Kinesis relay and MongoDB CDC #759.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions