upload_files: pending resources from a failed transfer block retries for ~2 days

[Caushi]

Problem

A failed tus transfer leaves its resources in pending (no hook ever rules on them). Retrying the same files then fails at create-resources, because the duplicate check matches those pending resources and on_duplicate="link" tries to link to them:

create-resources REST call failed (400):
{"error_description":"Resource 3166570 is not available and cannot be linked!","error":"bad_request"}

Retrying identical files after a transfer failure is completely ordinary — a cron-driven uploader does it automatically. But every retry fails this way until B-Fabric's orphan cleanup removes the pending resources (~2 days), so one transient failure can stall an instrument's data for up to 48h. It then heals spontaneously, which makes it confusing to diagnose: the error names a resource id rather than the actual cause.

Reproduction

Observed with BioBeamer's new tus backend against a local tusd + storage service:

1. Run an upload with on_duplicate="link". Have the transfer fail after create-resources — in our case the storage service's pre-create hook returned 503 ("Cannot verify resource in B-Fabric"), so no bytes were sent. Resources 3166570–3166574 are left pending.
2. Re-run with the same files. check-duplicates matches the pending resources by MD5, and create-resources rejects the link with the 400 above.
3. Only a change of file content (new MD5) or the 2-day cleanup unblocks it.

Why pending is the wrong thing to link to

The status machine already distinguishes real outcomes — the storage service sets available on success and failed on virus-scan/MD5 failure or post-terminate. So pending does not mean "failed"; it means nothing conclusive happened yet: either bytes are still in flight, or the attempt died before any hook could rule on it. Neither is linkable.

Suggested fix

Treat a non-available duplicate as uploadable rather than an error:

• available → linkable (today's behaviour)
• pending → not linkable; the bytes may not exist. Return it as a file to upload.
• failed → not linkable. Return it as a file to upload.

This makes retry-after-failure work on the next run instead of in two days, and needs no new state — only the status that check-duplicates / create-resources already sees. Relevant client-side call site: bfabric/src/bfabric/transfer/upload.py:103 (linkFromResourceId), though the authoritative check is server-side.

A stricter alternative — reusing the pending resource and resuming into it — is more valuable but much bigger, and needs the tus upload URL, which create-resources has no knowledge of. Tracked separately.