Teams connector: one message-retrieval failure aborts the entire team (0 documents indexed), the HTTP error is never logged, and a blank team list silently no-ops

[dev-jamal25]

Summary

The Microsoft Teams connector successfully authenticates and enumerates a team,
spends ~22 seconds retrieving from it, then indexes zero documents and
records a single failure. The index attempt finishes as completed with 1 failures.

The underlying HTTPError is never written to the logs — not at INFO, not at
DEBUG. It exists only as a generic string in the index_attempt_errors table,
carrying no HTTP status code and no Graph error payload. That makes the failure
undiagnosable from logs alone, which is the main reason for this report.

Three distinct defects are described below. They are independent and can be
fixed separately.

Environment

Image	onyxdotapp/onyx-backend:latest, image ID 64ca3768a71a, pulled ~2 weeks before this report
Code running	unmodified published image — not a local build, not a fork
Deployment	docker compose, deployment/docker_compose/docker-compose.yml
Host	Windows, Docker Desktop, Docker 29.7.2
Document index	OpenSearch 3.6.0
Log level	LOG_LEVEL=debug, confirmed resolved via docker compose config and verified in the running container
Connector	Microsoft Teams

Setup

Followed the official documentation at
https://docs.onyx.app/admin/connectors/official/teams exactly:

1. Azure Portal → App registrations → New registration.

2. Copied the Application (client) ID and Directory (tenant) ID.

3. Certificates & secrets → New client secret, 24 month expiry, copied the value.

4. API Permissions → Microsoft Graph → Application permissions, added:

   • Team.ReadBasic.All
   • TeamSettings.ReadWrite.All
   • Channel.ReadBasic.All
   • ChannelSettings.ReadWrite.All
   • ChannelMessage.Read.All
   • ChannelMember.Read.All

   Then Grant admin consent → Confirm. Consent shows as granted.

5. Onyx Admin Panel → Teams connector tile → entered client ID, directory ID and
   client secret.

6. Selected the team by name and clicked Connect.

Data being indexed

A single team containing 3 standard channels (no private or shared channels)
with ordinary user messages and threaded replies, posted by licensed users in
the same tenant. Content is seeded business-style conversation.

Reproduced on two separate teams — one private, one org-wide public — with
two separate connectors and two separate credentials. Identical result in every
case.

---

Defect 1 — the underlying HTTP error is never logged

The entire background container log for the failing attempt, at DEBUG level,
contains no traceback and no reference to the failure. The only traceback in 424
lines is an unrelated cleanup_idle_sandboxes task failing on a missing
kube-config.

The failure surfaces only in the index_attempt_errors table and the admin UI
dialog:

Retrieval of message and its replies failed; channel.id='<CHANNEL_ID>' <MESSAGE_ID>
error_type: HTTPError
document_id: null
document_link: null
is_resolved: false

No status code, no Graph error.code, no error.message. An operator cannot
distinguish a 401, 403, 404 or 429 — each of which has a completely different
remedy. Raising the log level does not help, because the exception is caught and
recorded without ever being logged.

Suggested fix: log the caught exception with its HTTP status code and the
Graph error body before recording it, and include the status code in
failure_message.

Defect 2 — one failed message aborts the whole team

Log sequence from the failing attempt, in order:

connector.py       556: Using basic /teams endpoint with pagination. Teams: ['<TEAM_NAME>']
connector.py       645: Found all requested teams after 1 pages
run_docfetching.py 638: Running 'teams' connector with checkpoint: {"has_more":true,"todo_team_ids":["<TEAM_ID>"]}
connector.py       266: Processed team with id <TEAM_ID>; 0 team(s) left to process
connector_runner.py 223: Connector took 22.527535516999706 seconds to get to the next checkpoint.
run_docfetching.py 838: Document extraction completed: batches_queued=0 elapsed=25.66s
indexing_coordination.py 150: Set total batches: attempt=<N> total=0
tasks.py           592: Index attempt <N> completed with 1 failures

Final counters: total_docs=0, total_chunks=0, total_failures=1,
batches_queued=0.

The team contains multiple channels with many messages. One failure produced
zero documents — not "all messages but one". The exception appears to
propagate up and terminate the generator for the entire team, discarding
everything already retrieved during those 22 seconds.

Notably, the failing message is the same message ID in the same channel
across every attempt we have run, spanning more than 24 hours, two connectors,
two credentials and a team rename. One unindexable message permanently prevents
the entire team from indexing.

Suggested fix: catch per-message retrieval failures, record them as entity
failures, and continue to the next message so partial progress is preserved.

Defect 3 — a blank team list silently indexes nothing and reports success

The documentation states that the team field may be left blank to pull
everything:

Select a list of teams to pull from or leave blank and click Connect to pull
everything.
— https://docs.onyx.app/admin/connectors/official/teams

Observed behaviour with the field left blank:

run_docfetching.py 352: Docfetching starting: config='{'teams': [], 'authority_host': ..., 'graph_api_host': ...}'
connector.py       145: Requested team count: 0, Has special chars: False
connector.py       157: Teams validation successful - Access to teams endpoint confirmed
connector.py       549: No specific teams requested - returning empty list
run_docfetching.py 838: Document extraction completed: batches_queued=0 elapsed=2.96s
tasks.py           589: Index attempt <N> completed successfully

The connector returns an empty list rather than enumerating accessible teams.
The attempt is reported as succeeded in the admin UI, with a green status and
zero documents and zero failures — so nothing signals to the user that anything
is wrong.

Suggested fix: enumerate all accessible teams when none are specified, as the
documentation describes. If that is not the intended behaviour, fail validation
with a clear message, and correct the documentation — reporting success for a
zero-document run is misleading either way.

---

Steps to reproduce

1. Create an Entra ID app registration and grant the six Microsoft Graph
   application permissions listed in the documentation, with admin consent.
2. In Onyx, create a Teams credential with the client ID, directory ID and client
   secret.
3. Create a Teams connector naming one team that contains standard channels with
   messages and threaded replies.
4. Run an index attempt → Defects 1 and 2.
5. Create a second Teams connector leaving the team field blank.
6. Run an index attempt → Defect 3.

Expected

Channel messages and their replies are indexed. An individual message that
cannot be retrieved is logged with its HTTP status and skipped, leaving the
remaining messages indexed. A blank team list indexes all accessible teams, as
documented.

Actual

Zero documents in every configuration tried. One opaque HTTPError recorded in
the errors table and nowhere in the logs. A blank team list reports success
having done nothing.

What we ruled out

• Log verbosity. LOG_LEVEL=debug confirmed applied via docker compose config and verified inside the running container. The error appears nowhere
  in any container's logs.
• Authentication and team lookup. The connector logs Teams validation successful - Access to teams endpoint confirmed and Found all requested teams after 1 pages, and resolves the team to its ID.
• Special characters in the team name. The original team name contained an
  ampersand, which routes through a client-side filtering branch
  (connector.py:556). We renamed the team to remove it and re-ran; identical
  failure. Not the cause.
• Team privacy. Reproduced on both a private team and an org-wide public
  team.
• Connector or credential state. Reproduced across two separate connectors
  and two separate credentials.
• Channel type. All channels are standard. No private or shared channels
  exist in either team.
• Throttling. The failure is deterministic on the same message ID across many
  attempts over more than 24 hours. A 429 pattern would be intermittent and would
  move between messages.

Additional context

We are happy to test patches against our tenant — we have a Teams tenant with
seeded channel data and can re-run index attempts quickly.

Defect 3 looks like the smallest and most clear-cut of the three, since it is a
direct mismatch with the documented behaviour. Defect 1 is the one that matters
most operationally, because it is the reason we cannot tell you what the
underlying HTTP error actually is.