Skip to content

Design: value-preview extractor registry for pformat (numpy/pandas structural previews) #86

Description

@pierreolivierbonin

Motivation

Sec 3.2 of the paper names bounded-preview coverage as open work: "Finding even better formats that are obvious to LLMs, and supporting more types, remains open work." Today numpy arrays and pandas frames — the types a data-touching CodeAct agent handles most — hit _pformat's truncated-repr fallback, which slices repr text mid-escape and hides shape, dtype, and column schema:

ndarray(repr_len=10250, [:100]='array([[   0. ,    1.5,    3. ,    4.5],\n       [   6. , ...')

Proposed design (implementation ready)

A value-preview extractor registry mirroring the existing doc() adapter architecture:

  • spec.define_preview(SomeType) registers (instance, PreviewBudget) -> str | None; MRO-aware lookup, thread-safe, cleared by clear_registry().
  • _pformat consults it just before the repr fallback (and at the max_depth shallow path, which today writes full untruncated reprs). None or an exception declines → existing behavior. Zero change for unregistered types.
  • numpy/pandas adapters ship in agentdoc/adapters/, registered via register_all(); neither library becomes a dependency. Formats stay in the truncation 3.0 marker family:
ndarray(shape=(250, 4), dtype=float64, [:3]=[[0., 1.5, 3., 4.5], ...], [-3:]=[...])
DataFrame(shape=(500, 3), columns={'price': 'float64', 'qty': 'int64', 'label': 'str'}, [:3]=[{...}], [-2:]=[{...}])
  • Completeness triggers use total element counts (arr.size, df.size) because numpy/pandas elide their own reprs with ... — a short repr is not evidence of a complete value.
  • CodeAct registers adapters at execution start; previously register_all() ran only on the opaque-return-type path, so schemable-return methods never activated adapters (found via a live agent run; regression-tested).

Design questions for maintainers

  1. Separate define_preview registry vs. overloading define_doc extractors with a preview capability?
  2. Is the two-field PreviewBudget (max_length, max_string) the right contract, or should previews see the full truncation config?
  3. Preview format opinions welcome — e.g. whether DataFrame row samples should be records dicts (current) or a columnar form.

PR with the full implementation, 40 tests, and live-run evidence: #87

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions