Release: durable investigations, analytics correctness, and platform hardening - #541
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: ASSERTIVE Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
The latest updates on your projects. Learn more about Unkey Deploy
|
Greptile SummaryThis PR centralizes AI query trait-filter resolution into a new
Confidence Score: 4/5The core refactor is sound — trait resolution is cleanly centralized, index bookkeeping in executeBatch is correct, and the session guard fix prevents accidental sign-outs. The two flagged items are edge-case inefficiencies that do not break the happy path. The unknown-type guard in resolveRequestTraitFilters lets an unnecessary identity-service round-trip happen before an unknown query type fails downstream, and the raw ClickHouse error strings now reach the LLM context without any sanitization pass. Neither causes incorrect data or broken flows on valid inputs, but both deserve a follow-up before the code sees heavy use with novel query types or noisy ClickHouse errors. packages/ai/src/query/trait-filters.ts (unknown-type guard) and packages/ai/src/ai/mcp/agent-tools.ts (raw error forwarding) are worth a second look. Important Files Changed
Sequence Diagram%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant AI as AI Agent / Tool
participant MCP as MCP buildBatchQueryRequests
participant EX as executeBatch
participant TF as resolveRequestTraitFilters
participant ID as Identity Service (resolveTraitSegment)
participant CH as ClickHouse
AI->>MCP: "queries[] with trait:<key> filters"
MCP->>MCP: invalidFilterFieldError() per query
MCP-->>AI: invalid[] (bad field) + requests[] (ok)
AI->>EX: executeBatch(requests)
loop per request (parallel)
EX->>TF: resolveRequestTraitFilters(req)
alt has trait filters
TF->>ID: resolveTraitSegment(projectId, traitFilters)
ID-->>TF: profile_id[]
TF-->>EX: req with profile_id filter
else no trait filters
TF-->>EX: original req (no-op)
end
end
EX->>CH: union query (successful requests)
CH-->>EX: rows
EX-->>AI: BatchResult[] (with errors for failures)
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant AI as AI Agent / Tool
participant MCP as MCP buildBatchQueryRequests
participant EX as executeBatch
participant TF as resolveRequestTraitFilters
participant ID as Identity Service (resolveTraitSegment)
participant CH as ClickHouse
AI->>MCP: "queries[] with trait:<key> filters"
MCP->>MCP: invalidFilterFieldError() per query
MCP-->>AI: invalid[] (bad field) + requests[] (ok)
AI->>EX: executeBatch(requests)
loop per request (parallel)
EX->>TF: resolveRequestTraitFilters(req)
alt has trait filters
TF->>ID: resolveTraitSegment(projectId, traitFilters)
ID-->>TF: profile_id[]
TF-->>EX: req with profile_id filter
else no trait filters
TF-->>EX: original req (no-op)
end
end
EX->>CH: union query (successful requests)
CH-->>EX: rows
EX-->>AI: BatchResult[] (with errors for failures)
Reviews (1): Last reviewed commit: "chore(staging): merge main" | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
All reported issues were addressed across 521 files
Confidence score: 5/5
- Safe to merge after the addressed issues were fixed.
Note: This PR contains a large number of files. cubic only reviews up to 200 files per PR, so some files may not have been reviewed. cubic prioritizes the most important files to review.
Shadow auto-approve: would not auto-approve because issues were found.
Re-trigger cubic
There was a problem hiding this comment.
0 issues found across 7 files (changes from recent commits).
Shadow auto-approve: would not auto-approve. Auto-approval blocked by 51 unresolved issues from previous reviews.
Re-trigger cubic
There was a problem hiding this comment.
1 issue found across 5 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/rpc/src/routers/insights.ts">
<violation number="1" location="packages/rpc/src/routers/insights.ts:337">
P2: The `.catch(() => false)` wrapper around `withWorkspace` in `getById` and `related` swallows every rejection, including database and infrastructure errors. After reviewing `resolveWorkspace` in `packages/rpc/src/procedures/with-workspace.ts`, it calls `getMemberRole` and `context.getBilling()` which can throw on DB/connectivity failures. When that happens the endpoint will return `{ success: true, insight: null }` (or an empty array) instead of a real error, masking outages and bypassing telemetry.
Preserve the anti-enumeration behavior for actual permission denials, but rethrow unexpected/infrastructure errors so they surface properly. A simple approach is to inspect the rejection and only swallow the expected `rpcError.forbidden` / `rpcError.unauthorized` kinds.</violation>
</file>
Shadow auto-approve: would not auto-approve because issues were found.
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
There was a problem hiding this comment.
0 issues found across 1 file (changes from recent commits).
Shadow auto-approve: would not auto-approve. Auto-approval blocked by 46 unresolved issues from previous reviews.
Re-trigger cubic
There was a problem hiding this comment.
All reported issues were addressed across 22 files (changes from recent commits).
Shadow auto-approve: would not auto-approve because issues were found.
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
There was a problem hiding this comment.
0 issues found across 1 file (changes from recent commits).
Shadow auto-approve: would not auto-approve. Auto-approval blocked by 22 unresolved issues from previous reviews.
Re-trigger cubic
There was a problem hiding this comment.
All reported issues were addressed across 91 files (changes from recent commits).
Shadow auto-approve: would not auto-approve because issues were found.
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
* feat(insights): restore bounded investigation agent * fix(insights): keep investigations time-scoped * docs(insights): define investigation product contract * fix(ai): correct page and session query semantics * refactor(ai): consolidate investigation tools * refactor(insights): simplify investigation persistence model * feat(insights): run durable agent investigations * feat(rpc): add investigation timelines and replies * feat(dashboard): replace insight cards with investigations * fix(ai): add Terra usage pricing * test(evals): replace synthetic insight evals with production shadow * chore(deps): prune obsolete investigation package surface * feat(insights): simplify durable investigations * refactor(ai): expose native investigation tools * feat(slack): continue delivered investigations * feat(dashboard): surface investigations * feat(investigations): add business and deploy context * chore(repo): remove obsolete investigation evals * docs(investigations): align product language * docs(agent): update investigation guidance * fix(ai): keep dry-run tools read-only * fix(docs): build tracker before copy tests * style(repo): fix baseline lint * fix(ai): make shared mocks order-safe
* fix(tracker): reject unsafe URLs and align docs * fix(ai): isolate ClickHouse test doubles * fix(platform): harden external and client trust boundaries * docs(repo): fix API test command * fix(billing): make Autumn ingestion replay-safe * feat(revenue): normalize payment events and reporting * fix(analytics): correct identity funnels links and uptime * fix(db): enforce ClickHouse schema parity * fix(revenue): scope payment diagnostics to filters * test(revenue): stabilize payment scope integration * test(revenue): stabilize invoice reconciliation integration * fix(notifications): bound Slack payload delivery * fix(insights): preserve concurrent schedule edits * refactor(config): simplify runtime and deployment setup * fix(dashboard): clear insight caches on organization change * fix(analytics): resolve session identities by event time * fix(basket): bound graceful shutdown * fix(dashboard): preserve revenue detail * fix(revenue): retain sparse refund attribution * fix(api): drain webhook replay before shutdown * fix(env): respect explicit production URLs * fix(api): stop replay claims during shutdown * fix(dashboard): polish investigations layout
|
Too many files changed for review. ( |
1 participant
Release overview
Promotes staging at 96110dda to main.
This is the full release train since main, not only the final investigations hardening slice: 138 commits, 731 files, +41,530 / -35,908. The final net behavior is summarized below; superseded deterministic-investigation and synthetic-eval iterations are intentionally not presented as shipped architecture.
What ships
Durable investigations
Revenue, billing, and identity
Slack, MCP, agent, and feedback
Analytics and product experience
SDK, tracker, privacy, and flags
Data and platform
Production rollout
1. PostgreSQL requires a coordinated maintenance-window migration
This is a destructive, non-expand/contract schema change. Do not let an automatic application deploy race the database push: current main code reads columns the new schema drops, while new code immediately needs the replacement tables and columns.
Before the production push:
Snapshot PostgreSQL and preserve any historical insight/config data that may still be needed.
Pause/disable Insights and coordinate the API, Insights, Dashboard, and Slack rollout from the same SHA.
Run both preflight queries and reconcile any returned rows:
SELECT organization_id, count()
FROM insight_generation_configs
GROUP BY organization_id
HAVING count() > 1;
SELECT organization_id, count()
FROM insight_runs
WHERE status IN ('queued', 'running')
GROUP BY organization_id
HAVING count() > 1;
Run bun run db:push during the deployment window.
Deploy API, Insights, Dashboard, and Slack together, then re-enable and verify scheduling, replies, and delivery.
The push drops persisted per-site/custom investigation controls including website, cron, depth, model/tool/budget/limit/lookback/cooldown, and last-run fields. It also removes legacy insight action/evidence/period/run fields and replaces rollups with observations, effects, and replies. Existing enabled per-site rows need an explicit canonical organization-level decision before migration; new configurations default disabled.
2. Production environment preflight
3. External and separately published surfaces
Verification
Fresh evidence on the release head:
Coverage limits are explicit: CI does not call the real AI Gateway or grade investigation usefulness, there is no Railway Insights staging deployment check on this head, and the live ClickHouse verification step had no read-only URL and was skipped.
Known follow-up
One in-flight Autumn replay can exceed the API's 10-second hard shutdown budget and remain leased until retry. Provider idempotency prevents duplicate email delivery, but persistence before process exit is not guaranteed for that slow in-flight claim.
Merge gate