v2.0.0 alpha#779
Open
ajslater wants to merge 335 commits into
Open
Conversation
…sockets The v3→v4 cutover left two unrouted views behind and built the composite /session endpoint by copy-pasting from the endpoints it replaced. - Remove dead AdminFlagsView (public.py) and TimezoneView (timezone.py), both unrouted since the cutover — /session and /auth/profile cover their payloads now. Drop the orphaned AuthAdminFlagsSerializer, TimezoneSerializer, and TimezoneSerializerMixin. - Extract version_payload() and user_payload() so SessionView, VersionView, and ProfileView stop carrying byte-for-byte copies. - Collapse the identical _MTIME_TYPES/_SCOPE_TYPES frozensets into one _ENRICHED_TYPES. - Rename the unknown-notification wire key v3 → raw (payloads.py + notify.js) and fix notify.js's stale v4_messages.py reference. Behavior-preserving. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… in place NestedSettings hands out user_settings by reference — it is the same dict as settings.REST_REGISTRATION — so writing keys onto it clobbered the canonical Django setting in place (and it never invalidated the attr cache, so a runtime change could silently no-op once a flag had been read). Copy the baseline, layer the runtime values onto the copy, swap it in, and reset the attr cache instead. This also removes an order-dependent test failure: the in-place mutation outlived TestCase rollback, so a test that saved an email host left the reset-password flow enabled for later tests asserting the disabled default 404s. The new conftest fixture resets the singleton between tests (the same reset rest-registration's setting_changed handler runs), and a regression test pins the no-mutation invariant. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
With the API now v4-only, the client comments no longer need to contrast against v3. Pure comment edits in the admin/base API clients, the app boot hook, and the metadata store — no behavior change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Consolidate the 12 development migrations 0043-0054 into one pristine 0043_comicbox_tagging_defaults. The data-migration functions are inlined (squashmigrations cannot serialize functions defined in migration modules), created-then-removed fields and redundant choice-only alters are dropped, and load-bearing data-migration ordering is preserved. Validated end-to-end against a v1.12.7 (0042) baseline with representative data. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Stream every backup straight into lzma (single write, no temp file) and treat the user-data sidecar as a backup alongside the DB backups. - DB backup: Connection.serialize() image -> lzma. The write-lock now spans only the ~ms snapshot, not the (slow) compression. serialize() keeps the FTS5 index byte-for-byte intact; iterdump() would drop it. - Sidecar: dumped into an in-memory SQLite, then iterdump() streamed into user_data.<date>.sql.xz in the backups dir (no uncompressed sidecar at rest). Legacy binary user_data.sqlite is removed once a snapshot exists. - Both are dated YYYY-MM-DD, nightly, newest 7 retained; before-upgrade DB backups are compressed (.bak.xz) and exempt from pruning. - Restore reads .sql.xz / .sql / legacy binary (decompress-on-read), defaulting to the newest backup. Admin Restore tab gains a backup picker (GET /admin/user-data/backups); the restore endpoint accepts a filename. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Match the existing sidecar-test convention (test_user_data_restore.py): annotate setUp-assigned instance vars with the pyright ignore. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- foreign_keys.py: drop two `ty: ignore[invalid-argument-type]` directives ty now reports as unused. - conftest.py: annotate the autouse fixture for pyright's reportUnusedFunction false positive. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…dule Pick both the xz preset and the DB-backup method from the cgroup-aware memory budget so a low-RAM host with a large DB can't OOM: - xz_preset() scales 0..9 by available memory; a 1 GiB host lands ~preset 6, an 8 GiB+ host gets the full preset 9. - backup_db() serialize()s a small DB in RAM (fast, short lock), but a DB larger than ~half the budget falls back to VACUUM INTO a temp file + chunked compression on a dedicated autocommit connection — bounded memory, FTS-safe, immune to the caller's transaction state. - Split a pure read_mem_limit() out of get_mem_limit() (which sets RLIMIT_AS) so the admin request worker can size backups without pinning its own AS. - Rename codex/compression.py -> codex/xz.py to avoid shadowing the new Python 3.14 stdlib `compression` package. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Consolidate the admin area's drifted styles into reusable primitives and a single spec (frontend/DESIGN.md), then rewire every tab onto them. - New primitives in components/admin/tabs/: AdminSection, AdminActionBar, AdminExpandToggle, AdminKeyValueTable; design tokens in design.scss. - One reading-column / full-bleed layout law and fixed text-colour roles; a single Save/Revert bar, disclosure toggle, and key/value table replace the per-tab copies. Removes the bespoke stats-table.vue. Net -151 lines across 16 tabs; lint, 76 unit tests, and the production build all pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Follow-ups to the admin design-language refactor (e836e93): - Group-tab help: use margin-block so the `margin: 2em 0` shorthand no longer zeroes the inline margins and cancel .adminReadingColumn's margin-inline:auto centering on the same element. - Key/value tables: add an inter-column gap so a long label (e.g. "Authorization Groups") no longer butts into the right-aligned value, and stripe alternating rows for readability, replacing the per-row border. - Drop the stale stats-table.vue reference from status-helpers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pure behavior-preserving extractions so each flagged function/class lands under the gate thresholds. - vacuum.py: split backup_db into _resolve_backup_path / _write_backup / _prune_dated_backups (complexipy 16 -> <15) - restore.py: extract _build_user_defaults and _build_tagging_defaults; the tagging builder keeps the null-coalescing ors in one comprehension (radon cc C13/C11 -> A) - admin/user.py: split AdminUserBulkView.post into _parse_delete_ids and _delete_users; radon scores a class by its method-CC average, so this drops the class C12 -> A5 - browser/group_mtime.py: extract page-mtime cache get/set and the aggregate query out of get_group_mtime (radon cc C12 -> A5) - test_admin_custom_cover.py: extract _assert_covers_enqueued (C11 -> B9) - move the two self-contained unit classes out of test_browser_table_response.py into test_browser_table_columns.py (radon mi B 18.30 -> A 23.21) Verified: complexipy clean (pathless), radon cc --min C and mi --min B both empty, ruff/basedpyright/ty clean, 127 affected tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The email tab's Test Send recipient field lived in the settings <v-form>, so pressing Save Settings ran form.validate() over it and warned "Recipient is required" on the usually-empty field. Move Test Send into its own form and validate the recipient in runTest() instead. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Save could stay disabled for a valid change because it was gated on the auth-form-mixin's async submitButtonEnabled flag, and the mixin also re-validated the whole form on every keystroke (flashing "required" on untouched fields). Opt out of the mixin's credentials watcher and gate Save on a synchronous isValid computed; fields keep their inline validate-on=input rules. Adds a focused unit test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Serve the SPA for /auth/reset-password/ so Vue Router renders the reset screen; the catch-all was redirecting the emailed link to the home page. - Stop Django autoescaping the URL in the plain-text email (it turned the query separators into &, corrupting user_id/timestamp/signature). - Open the login dialog after a successful reset. - Show the username on the reset screen, carried by the link (display-only; the reset stays gated by the signed user_id/timestamp/signature). Adds backend routing + email tests and a frontend component test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- validate-on=submit on the login form so diverting to "Forgot password?" no longer trips the empty-username required rule on blur. - Close the reset-request dialog on send; the green "Reset link sent" banner already shown on the login screen is the only confirmation needed. Drops the redundant in-dialog message and the now-dead resetPasswordRequestSent store flag. Adds/updates unit tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Point RESET_PASSWORD_VERIFICATION_EMAIL_TEMPLATES at both a text_body and a new html_body so rest-registration sends a multipart message: the text template is the plain-text part (text-only readers fall back to it) and the HTML is attached as an alternative. The HTML renders the reset link as a styled codex-orange "Reset Password" button with a copy-paste fallback link, and carries the same &username= param so the reset screen still shows it. djlint H021 (no inline styles) is disabled for the template since email clients strip <style>/external CSS. Adds a multipart-assertion test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The edit tags panel could set Issue Count and Volume Count but had no
field for the comic's own issue number or suffix. Add Issue Number and
Issue Suffix inputs, regrouping the Publishing section to mirror the
read-only header (Volume + Volume Count, Issue + Issue Count).
buildPatch emits the comicbox `issue: {number, suffix}` object; comicbox
computes the combined `issue.name` written to <Number> for both
ComicInfo and MetronInfo. Enable both fields for the two formats in
format-field-support.json.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The SPA boots every visitor through GET /api/v4/session, but the view was gated behind IsAuthenticatedOrEnabledNonUsers. When a visitor was logged out AND the non_users flag was off, the request 401'd, so adminFlags never loaded, isAuthChecked stayed false, and the browser spun on the placeholder forever. Make SessionView AllowAny -- the payload is anonymous-safe by construction (user is null, permissions all-false). Split the admin flags into a public subset (registration, non_users, banner_text, register_verification, email_enabled) that the logged-out shell needs and a private subset (lazy_import_metadata, remote_user_enabled) that only the authenticated UI reads, so an unauthenticated boot discloses nothing beyond the logged-out screen. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Tag writes resolved a selected group (e.g. a publisher) to *every* comic under it via a bare relation filter, ignoring the active browser filters. Writing the "Image" publisher tag to a CBR+unread-filtered publisher leaked onto an unread CBZ (Monstress #62) the filters should have excluded. The shared `_resolve_comic_pks` helper backed all three archive-write entry points (bulk tag-write, its preflight, and online tag start), so every one of them ignored file_type, read/unread, ACL, favorite, and search filters alike. Replace the helper with a `FilteredComicPksView` mixin that resolves group+pks through `get_filtered_queryset(Comic, group, pks)` — the same browser filter pipeline `ForceUpdateView` and `BookmarkView` already use, reading the user's persisted filters via a load-only `params` override. The admin envelope renderer and browser base share `EnvelopeJSONRenderer`, so the wire format is unchanged and the Vue callers need no edits. Add a regression test covering the reported file_type case plus read/unread, with a no-filter control. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The browser empty-state mapped adminFlags.registration into a computed but never referenced it in the template or any other computed. Dead mapping, removed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two bugs left the hover-to-lazy-import feature dead code: - lazyImportEnabled read this.stateLazyImportMetadata, but the mapped auth-store state is named stateLazyImportEnabled, so the guard was always undefined (falsy). - the @mouseenter binding was a ternary (lazyImportEnabled ? onMouseEnter : null) which evaluates to a function reference but never invokes it. onMouseEnter already guards on lazyImportEnabled internally, so bind it directly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Lock in the behavior that cd283fd restored. Mounts the real Vuetify button and dispatches a DOM mouseenter so the @mouseenter binding and the lazyImportEnabled guard are exercised end-to-end: - fires lazyImport({ group, ids }) only when the lazyImportMetadata admin flag is on and the book is an un-imported comic - stays inert when the flag is off, the comic already has metadata, or the group is not "c" - prefers book.ids over book.pk and imports at most once Reintroducing the stateLazyImportMetadata/stateLazyImportEnabled typo fails the two firing assertions, so this guards the exact regression. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
compute_collection_intersections read the page pks via
values_list('pk'), a clone that re-executed the entire annotated/
grouped/sorted collection query; the serializer then ran the original
queryset again. Iterating the queryset itself primes its result cache,
which the serializer reuses — the heaviest table-mode query now runs
once per request.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…le mode - _get_zero_pad re-ran the whole ordered/annotated book query as a MAX subquery (~20% of warm-request SQL) to derive one digit count. Materialize the final page once (priming the result cache the serializer reuses) and take the max issue_number off the page rows in Python. The OPDS path keeps the aggregate variant. - Table mode serialized ~100 cards through BrowserCardSerializer and then popped them from the result. Pop the card fields from self.fields before to_representation instead (and symmetrically the rows field in card mode) — same wire output, none of the per-card work. Fixed the stale 'mobile fallback' comment claiming cards stay populated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
cover_mtime was a .values('updated_at') Subquery over the identical
correlated cover queryset that already computes cover_pk — two full
executions of the most expensive expression per card (measured 40-49%
of the card query across publisher/folder/search pages).
Drop the annotation; resolve the representative comics' updated_at
after pagination with one indexed pk__in batch over the page's
cover_pks and attach it to the cached instances (the serializer
iterates the same objects). Custom-cover mtimes keep their cheap rowid
subquery and still win the coalesce. _CoverMtimeCoalesce is gone with
the annotation that needed it. Responses verified byte-identical.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tched once Every browse GET re-fetched the settings row (two uncachable django_session probes) and synchronously rewrote SettingsBrowserLastRoute on navigation — a full-row UPDATE (rewriting created_at) that evicted cachalot's settings cache for everyone and stalled 1.85s behind the WAL writer lock during imports. - Navigation now queues a LastRouteUpdateTask to the librarian bookmark thread (same machinery as page-turn bookmarks): the aggregator keys on the settings pk, so rapid navigation collapses to one filter().update() per flood window. Measured: navigation under a held write transaction 1,850ms -> 67ms; same-route repeats are fully write-free at 2 queries. - The settings instance is memoized per request keyed on (model, client) — the save path reuses the loaded row, dropping the second fetch and the second session probe. - The remaining synchronous saves (real settings changes via params, PATCH) use update_fields so created_at is never rewritten. The persisted route now lags navigation by the writer's flood window (<=5s) — the same freshness contract page-turn bookmarks already have. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The raw comicfts MATCH re-executed the FTS5 scan in every statement carrying the search filter — counts, mtime probe, pagination pk query, main query, intersections: 5-6 scans per request, measured ~870ms of a 1.3s search request at 17k matches. Browse/mtime requests now materialize the match pks once per request (cached_property) and bind a pk IN-list instead; cold-cachalot browse drops to exactly one MATCH execution. Scoped deliberately: - Rank-ordered requests (order_by=search_score) keep the raw MATCH — ComicFTSRank needs it active in the scored statement. - choices/metadata keep the constant-size MATCH/subquery forms: their statements repeat the filter per probe arm, and an inlined list binds one variable per pk (SQLite caps statements at 32,766). - Match sets over 15,000 pks keep raw MATCH for the same reason — the list may appear twice per statement (main query + cover subquery). The cover subquery consumes the swapped set directly instead of re-wrapping it in a membership sub-SELECT. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… TTL key Two fixes to the collection mtime probe paid on every browse/head: - The probe annotated a per-row Greatest, GROUP BYed every column, DISTINCTed and sorted all rows just to read one global MAX. Keep the filtered queryset (demoted joins + FTS MATCH intact) as a pk-subquery and aggregate over a fresh queryset (23.8ms -> 7.9ms at 18k, identical value). A plain .aggregate() on the filtered qs is NOT equivalent — join demotion doesn't survive Django's aggregation rewrite and FTS5 raises 'unable to use function MATCH'. - The 5s TTL cache key included the page number and order params, which never affect the probed value — every page flip and order toggle re-paid the probe (~20% of a bookmark-filtered page-flip request). Key on user/model/collection/pks/filters only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The lazy parent_folder walk issued one SELECT per ancestor level — a depth-12 folder paid 11 round-trips per browse with cold cachalot, and every import invalidates the Folder table. Fetch all ancestors at once via the indexed materialized path (PurePath.parents prefixes scoped by library_id; verified equal to the FK chain on the real corpus, 5.6ms -> 1.7ms at depth 12). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… loop bookmark_updated_ats built a sorted, deduped JSON_GROUP_ARRAY of every matching bookmark timestamp per card, which the serializer parsed string-by-string with fromisoformat just to take the max (~50 cards x up to ~50 timestamps per browse response for heavy readers). A Max aggregate produces the same value with no JSON construction, per-group sort, transfer, or Python loop. The alias is distinct from bookmark_updated_at: when the primary sort is bookmark_updated_at ascending that alias already holds a Min aggregate, and the table-view collection branch annotates it independently. Responses verified byte-identical, incl. OPDS and bookmark-filtered listings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
DateTimeColumn snapshotted its Date in data(), which runs once per component instance. When LIBRARY_CHANGED refetched the libraries table, Vue reused the row's component and only the dttm prop changed, so a new library's Last Poll stayed at the epoch (new Date(null)) until a full page reload. Derive the Date as a computed instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
_handle_pending_polls called qs.only(*_LIBRARY_ONLY) without reassigning, so full Library rows were fetched. With the projection applied, the library.save() after setting last_poll writes only loaded fields, which also avoids clobbering update_in_progress set concurrently by the importer. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per tasks/comicbox-handoff.md: - Bump comicbox pin to ~=4.0.0a3 (constructor no longer accepts or needs logger=; comicbox logs through the host's loguru sinks). - Drop the logger= kwarg from every Comicbox() call site. - Catch comicbox.exceptions.ComicboxError instead of broad Exception where the intent is "comicbox failed on this file": the reader page endpoint 404s only on comicbox errors now, and tag-by-id fetch failures surface in the admin Tagging-tab error panel instead of a generic thread-crash log. - No workarounds to remove: TagWriter already drains bulk_write with cancel=abort_event (which now actually works), the explicit-id root-unwrap remains valid, and tests already build events with keyword args. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Merge implicit f-string concatenations, add missing @OverRide on LibrarianStatusCursorPagination.get_ordering, and replace a stale ty: ignore in SeeOtherRedirectError._get_query_params with a real Mapping isinstance narrowing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 27c7647f0ffdcec1a5ca178e47539c4ad060de05
Author: AJ Slater <aj@slater.net>
Date: Sat Jun 13 01:54:29 2026 -0700
refactor(complexity): clear radon cc rank-C findings
bin/lint-complexity.sh reported three rank-C functions; extract helpers
to drop each below the gate (radon cc --min C now empty).
- AdminTagByIdView.post: pull primary-identifier parsing and merge
extra-id resolution into _resolve_primary / _resolve_extra_ids.
- OnlineTagSessionManager._apply_resolution: extract _handle_unresolved
and _enqueue_resolved_write.
- test_update_protagonist: extract repeated protagonist assertions into
a helper (radon counts each assert as a branch).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
commit 55bb659fcc8c02e73c20a34823652a4f81be1d56
Author: AJ Slater <aj@slater.net>
Date: Sat Jun 13 01:46:04 2026 -0700
refactor(migrations): squash 0044 into 0043
Fold the merge_all_sources field into the ComicboxTaggingDefaults
CreateModel in 0043 and drop the standalone 0044. makemigrations --check
reports no drift.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
commit 7a988737a63286cc85396346579e124e5fff8e5b
Author: AJ Slater <aj@slater.net>
Date: Sat Jun 13 01:44:59 2026 -0700
feat(onlinetag): merge metadata from all sources
Add an opt-in "merge all sources" mode that queries every enabled online
source per comic and merges the results (comicbox first_wins=False) for
maximum tag completeness, instead of stopping at the first match.
- Admin tagging defaults: merge_all_sources boolean (default off),
settable as the default and overridable per scan.
- Search dialog: per-scan toggle, gated on >=2 enabled sources; estimate
+ rate-limit warning multiply calls by source count (kept in sync with
estimate.py) so the doubled API cost is visible.
- By-ID: chips input accepting multiple URLs/ids; merging fetches two
explicit ids (one per source) and merges them. Source select shows only
when a token can't be auto-identified (bare number).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
commit 8416c668c041cdbc56a16e56e357ce99aa3d9dae
Author: AJ Slater <aj@slater.net>
Date: Sat Jun 13 00:34:56 2026 -0700
update deps
commit 473221d7a09403b1848f202d55902b71e31319c2
Author: AJ Slater <aj@slater.net>
Date: Sat Jun 13 00:22:58 2026 -0700
fix(lint): clear ruff/pyright findings; fix benchmark import path
- tests: add docstrings, name magic estimate values, use pytest.raises
and out-of-class exception message
- onlinetag: collapse implicit f-string concatenation in log call
- benchmark-import: correct mock_comics import (fixtures.* -> benchmark.*)
which was a real runtime ImportError in library generation
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
commit 45a9fde6076a5b0b15be889769e9a55af4b0678a
Author: AJ Slater <aj@slater.net>
Date: Sat Jun 13 00:13:49 2026 -0700
refactor(migrations): squash 0044 + 0045 into 0043
Fold the comic-leading composite Bookmark indexes (former 0044) and the
LibrarianStatus eta/retry_at fields + JFR status-type choice (former
0045) into the consolidated 0043 migration; delete both files.
makemigrations --check reports no changes against the model state.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
commit 22da6b9cd85615f3d7b8e7043818babac313d24e
Author: AJ Slater <aj@slater.net>
Date: Sat Jun 13 00:04:35 2026 -0700
update deps
commit 6f27454623e43cccd95253d5dfe66471820c9349
Author: AJ Slater <aj@slater.net>
Date: Sat Jun 13 00:04:15 2026 -0700
fix(onlinetag): persist mid-scan prompt answers; live rate-limit countdowns
Two online-tagging bugs plus a status-UX overhaul.
Prompt answers given while a scan was running were lost on refresh: the
response task sat in the queue behind the busy OnlineTagThread, so the
cache kept the prompt. Now the drain loop removes answered prompts from
the cache inline (race-free, single-threaded) and marks the fingerprint
so the running scan stops re-persisting it; the network re-fetch + write
is deferred until the scan releases the thread.
A rate-limited lookup looked hung: collect_results could raise mid-pass
(budget exhausted, network error) without ever calling finish(), leaving
the status row frozen forever. finish() now runs in a finally.
The rate-limit status was a static "rate limited ~57s" — a second
subtitle writer clobbered the parseable one, and nothing refreshed it
during comicbox's blocking sleep. Replaced with two absolute-timestamp
countdowns the admin UI ticks down to live:
- retry_at: time until the next online request, re-anchored per attempt.
- eta: total time remaining, carrying forward the launcher dialog's
estimate (ported verbatim to estimate.py), re-estimated on each comic
completion and pushed out by rate-limit waits.
Adds eta/retry_at to LibrarianStatus (migration 0045, which also folds in
the pending status_type choices drift), plumbed through Status and
StatusController.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
commit eccf41bc603b118bdbd63f4e4918a30da87140ae
Author: AJ Slater <aj@slater.net>
Date: Fri Jun 12 22:39:15 2026 -0700
feat(tagging): admin-orderable online source priority
defaultSources order is now run priority end to end. comicbox runs
sources in the given order under first-wins, so the admin can choose
which source is primary and which is fallback:
- Tagging tab renders the source rows in defaultSources order with
per-row up/down arrows; enabling a source appends it at lowest
priority. A hint explains that the first matching source wins.
- The launcher dialog normalizes its checkbox selection to the
admin's priority order before starting a session (checkbox v-model
records click order, not priority).
- Serializers validate defaultSources / sources: known names only,
deduped, order preserved (unknown source → 400).
- explicit_id / session settings pass ordered tuples to comicbox's
retyped OnlineLookupSettings.sources.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 02869ddf4d027c325637dd41081199c59f1141ad
Author: AJ Slater <aj@slater.net>
Date: Fri Jun 12 17:06:25 2026 -0700
add METRON_INFO to default formats to write
commit a214c9c93cf679194beeb5c42eb66deea933a3da
Author: AJ Slater <aj@slater.net>
Date: Fri Jun 12 14:58:36 2026 -0700
update deps
commit 65a294362171bded71547430dd51b322ea83dcef
Author: AJ Slater <aj@slater.net>
Date: Fri Jun 12 14:58:08 2026 -0700
don't make abort button red
commit de49b5dd9529446ab34a7468bda694fae39c7a00
Author: AJ Slater <aj@slater.net>
Date: Fri Jun 12 14:52:14 2026 -0700
fix(frontend): remove duplicated Tag Write Errors sidebar item
admin-menu.vue contained the same tag-write-errors CodexListItem twice
(re-added in a later squash while the original stayed), so any error
rendered two identical sidebar entries. One item per error kind; the
counts live in the admin tables.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit e0bd853d9ed1991859849e11d85d487409dc1194
Author: AJ Slater <aj@slater.net>
Date: Fri Jun 12 13:43:19 2026 -0700
fix(onlinetag): stop spurious rewrites and prompt loss in online tagging
Three interacting bugs broke a single-comic tagging run:
- Pass-1 and prompt-resolution writes now require result.matched
(new comicbox flag): unmatched results carry the comic's merged
existing metadata, so gating on truthy tags re-wrote files with
no new information — which also triggered a re-import mid-session.
- Pending prompts and tag-write errors move to a dedicated 'tagging'
cache (codex.cache.tagging_cache). The session_cache docstring
claimed key namespacing protected them from cache.clear(), but
Django's file-based clear deletes everything: every import that
changed anything (importer finish), Library/Group CRUD, and startup
wiped prompts mid-answer, causing 'resolve_prompt: unknown prompt'.
- Prompt resolution builds its replay session with defer_prompts=True
so the preloaded resolution is actually consulted (comicbox only
installs its cache-reading selector with defer or a handler;
without it an ambiguous re-search fell through to comicbox's
interactive CLI prompt inside the daemon). A fingerprint miss —
the re-search returned different candidates — now re-queues the
fresh deferred prompt instead of dying silently.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 9df7837bd3e3480e4fefd382f09db6d326cfadf4
Author: AJ Slater <aj@slater.net>
Date: Fri Jun 12 11:45:22 2026 -0700
feat(admin): auto-enable a tagging source when its credentials are first saved
Saving credentials for a source that previously had none now adds it to
default_sources via the auto-saving draft, so a freshly configured source
works without a second checkbox click. Re-saving credentials for an
already-configured source leaves the enable checkbox untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit fd5b9c0b6fb5fa0da079bae268e9e7553098b853
Author: AJ Slater <aj@slater.net>
Date: Fri Jun 12 11:38:11 2026 -0700
Squashed commit of the following:
commit 9543008f24441b6b2992f0c818d019f42135f7df
Author: AJ Slater <aj@slater.net>
Date: Fri Jun 12 10:21:24 2026 -0700
dockerignore benchmark dir
commit 00134a40fe1c37d96e64066735cb42cc03f463c8
Author: AJ Slater <aj@slater.net>
Date: Fri Jun 12 10:20:24 2026 -0700
mv fixutres to benchmark dir
commit 451c5b7c1df5bffdbed68bdcfeaa96cdb6b47550
Author: AJ Slater <aj@slater.net>
Date: Fri Jun 12 10:13:54 2026 -0700
change import path for mock comics
commit 21d9af8e54f4fb1bfe015d1cf96e8ad0c415d434
Author: AJ Slater <aj@slater.net>
Date: Fri Jun 12 10:13:05 2026 -0700
ignore fixtures dir
commit 5c589f69d2a146ff1e839b095c1b563f73ed54ba
Author: AJ Slater <aj@slater.net>
Date: Fri Jun 12 10:11:25 2026 -0700
common subdir for mock & benchmark
commit cd768965a544069fb623f9798a2ef7f472016586
Author: AJ Slater <aj@slater.net>
Date: Fri Jun 12 10:01:01 2026 -0700
move benchmark import out of ci
commit d1a3a213d514317acfcb0dcf19b3a9163e2880b8
Author: AJ Slater <aj@slater.net>
Date: Fri Jun 12 09:52:32 2026 -0700
v2.0.0a10
commit 43b6042bca9c546d49268a52002efa4db69651c4
Author: AJ Slater <aj@slater.net>
Date: Fri Jun 12 09:51:47 2026 -0700
update deps
commit 02709b3d43119451ff177ecc6d0d3b9d76cfc634
Author: AJ Slater <aj@slater.net>
Date: Fri Jun 12 02:39:57 2026 -0700
fix typecheck and ty warnings
Merge implicit f-string concatenations, add missing @override on
LibrarianStatusCursorPagination.get_ordering, and replace a stale
ty: ignore in SeeOtherRedirectError._get_query_params with a real
Mapping isinstance narrowing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 68c1d9fcbf2a5a2d7b4b11b94148151cb74a4555
Author: AJ Slater <aj@slater.net>
Date: Fri Jun 12 02:28:58 2026 -0700
remove bad news line
commit 4b92f77b022b7f8392edb695bf4b5eb1871529ac
Author: AJ Slater <aj@slater.net>
Date: Fri Jun 12 02:18:54 2026 -0700
adapt to comicbox 4.0.0a3 API
Per tasks/comicbox-handoff.md:
- Bump comicbox pin to ~=4.0.0a3 (constructor no longer accepts or
needs logger=; comicbox logs through the host's loguru sinks).
- Drop the logger= kwarg from every Comicbox() call site.
- Catch comicbox.exceptions.ComicboxError instead of broad Exception
where the intent is "comicbox failed on this file": the reader page
endpoint 404s only on comicbox errors now, and tag-by-id fetch
failures surface in the admin Tagging-tab error panel instead of a
generic thread-crash log.
- No workarounds to remove: TagWriter already drains bulk_write with
cancel=abort_event (which now actually works), the explicit-id
root-unwrap remains valid, and tests already build events with
keyword args.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 027fa1d8b2f79866dd0e37e00dfa45617db7b251
Author: AJ Slater <aj@slater.net>
Date: Thu Jun 11 21:55:25 2026 -0700
update deps
commit 919fad63cf5790b82275e5f20207e38f65934a6c
Author: AJ Slater <aj@slater.net>
Date: Thu Jun 11 21:48:25 2026 -0700
update deps
commit 147510b43c0d860f1d35e3b4d3420b48b62f7a3e
Author: AJ Slater <aj@slater.net>
Date: Thu Jun 11 14:00:48 2026 -0700
poller: apply discarded .only() projection in manual poll path
_handle_pending_polls called qs.only(*_LIBRARY_ONLY) without reassigning,
so full Library rows were fetched. With the projection applied, the
library.save() after setting last_poll writes only loaded fields, which
also avoids clobbering update_in_progress set concurrently by the importer.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit cc629e5baa28fc3e19da9ae40676a8df4b6248f4
Author: AJ Slater <aj@slater.net>
Date: Thu Jun 11 07:52:01 2026 -0700
admin: fix stale Last Poll column after websocket table refetch
DateTimeColumn snapshotted its Date in data(), which runs once per
component instance. When LIBRARY_CHANGED refetched the libraries
table, Vue reused the row's component and only the dttm prop changed,
so a new library's Last Poll stayed at the epoch (new Date(null))
until a full page reload. Derive the Date as a computed instead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 188dd22775d6beb284b6ddafb3629dcb0affbccb
Author: AJ Slater <aj@slater.net>
Date: Thu Jun 11 00:18:16 2026 -0700
edit news for brevity
commit abad1fe2f68c2ba6899c056e9075d6d8e6555b1a
Author: AJ Slater <aj@slater.net>
Date: Thu Jun 11 00:17:05 2026 -0700
NEWS: browser performance entry for 2.0.0
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 994d6a586c249376827c1d72494071b4281528fa
Author: AJ Slater <aj@slater.net>
Date: Thu Jun 11 00:15:27 2026 -0700
news: v2.0.0 performance section for the import speedups
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit cbb07c980bd5bb317e7a506bec81ba8f66415163
Author: AJ Slater <aj@slater.net>
Date: Thu Jun 11 00:06:03 2026 -0700
cards: Max bookmark timestamp aggregate instead of JSON array + parse loop
bookmark_updated_ats built a sorted, deduped JSON_GROUP_ARRAY of every
matching bookmark timestamp per card, which the serializer parsed
string-by-string with fromisoformat just to take the max (~50 cards x
up to ~50 timestamps per browse response for heavy readers). A Max
aggregate produces the same value with no JSON construction, per-group
sort, transfer, or Python loop.
The alias is distinct from bookmark_updated_at: when the primary sort
is bookmark_updated_at ascending that alias already holds a Min
aggregate, and the table-view collection branch annotates it
independently. Responses verified byte-identical, incl. OPDS and
bookmark-filtered listings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 01a9394d8ecaaa22e72885b4257545a82220b3d4
Author: AJ Slater <aj@slater.net>
Date: Thu Jun 11 00:06:03 2026 -0700
breadcrumbs: batch the folder ancestor chain in one query
The lazy parent_folder walk issued one SELECT per ancestor level — a
depth-12 folder paid 11 round-trips per browse with cold cachalot, and
every import invalidates the Folder table. Fetch all ancestors at once
via the indexed materialized path (PurePath.parents prefixes scoped by
library_id; verified equal to the FK chain on the real corpus, 5.6ms
-> 1.7ms at depth 12).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit d0c2f64c9dbb9cc796fe17a9408e0aaff4788733
Author: AJ Slater <aj@slater.net>
Date: Thu Jun 11 00:06:03 2026 -0700
mtime probe: plain aggregate over a pk-subquery; stop fragmenting the TTL key
Two fixes to the collection mtime probe paid on every browse/head:
- The probe annotated a per-row Greatest, GROUP BYed every column,
DISTINCTed and sorted all rows just to read one global MAX. Keep the
filtered queryset (demoted joins + FTS MATCH intact) as a
pk-subquery and aggregate over a fresh queryset (23.8ms -> 7.9ms at
18k, identical value). A plain .aggregate() on the filtered qs is
NOT equivalent — join demotion doesn't survive Django's aggregation
rewrite and FTS5 raises 'unable to use function MATCH'.
- The 5s TTL cache key included the page number and order params,
which never affect the probed value — every page flip and order
toggle re-paid the probe (~20% of a bookmark-filtered page-flip
request). Key on user/model/collection/pks/filters only.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 578807ab0294391095e2f6b6fe3d1b1540f643a4
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 23:55:07 2026 -0700
search: materialize the FTS match set once per browse request
The raw comicfts MATCH re-executed the FTS5 scan in every statement
carrying the search filter — counts, mtime probe, pagination pk query,
main query, intersections: 5-6 scans per request, measured ~870ms of a
1.3s search request at 17k matches.
Browse/mtime requests now materialize the match pks once per request
(cached_property) and bind a pk IN-list instead; cold-cachalot browse
drops to exactly one MATCH execution. Scoped deliberately:
- Rank-ordered requests (order_by=search_score) keep the raw MATCH —
ComicFTSRank needs it active in the scored statement.
- choices/metadata keep the constant-size MATCH/subquery forms: their
statements repeat the filter per probe arm, and an inlined list
binds one variable per pk (SQLite caps statements at 32,766).
- Match sets over 15,000 pks keep raw MATCH for the same reason — the
list may appear twice per statement (main query + cover subquery).
The cover subquery consumes the swapped set directly instead of
re-wrapping it in a membership sub-SELECT.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit e56b7eb2dd54cab402041de418832f865269540c
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 23:46:46 2026 -0700
lint: format follow-ups for the order/cover perf commits
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit ca67645e4bf757568021e19781d9179ae599bda2
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 23:46:46 2026 -0700
browser: move the last-route write off the read path; settings row fetched once
Every browse GET re-fetched the settings row (two uncachable
django_session probes) and synchronously rewrote
SettingsBrowserLastRoute on navigation — a full-row UPDATE (rewriting
created_at) that evicted cachalot's settings cache for everyone and
stalled 1.85s behind the WAL writer lock during imports.
- Navigation now queues a LastRouteUpdateTask to the librarian
bookmark thread (same machinery as page-turn bookmarks): the
aggregator keys on the settings pk, so rapid navigation collapses
to one filter().update() per flood window. Measured: navigation
under a held write transaction 1,850ms -> 67ms; same-route repeats
are fully write-free at 2 queries.
- The settings instance is memoized per request keyed on
(model, client) — the save path reuses the loaded row, dropping the
second fetch and the second session probe.
- The remaining synchronous saves (real settings changes via params,
PATCH) use update_fields so created_at is never rewritten.
The persisted route now lags navigation by the writer's flood window
(<=5s) — the same freshness contract page-turn bookmarks already have.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 94ef67e41554869cfe4c19494cea7e422ff5d30b
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 21:10:00 2026 -0700
covers: batch cover mtimes post-pagination instead of a second subquery
cover_mtime was a .values('updated_at') Subquery over the identical
correlated cover queryset that already computes cover_pk — two full
executions of the most expensive expression per card (measured 40-49%
of the card query across publisher/folder/search pages).
Drop the annotation; resolve the representative comics' updated_at
after pagination with one indexed pk__in batch over the page's
cover_pks and attach it to the cached instances (the serializer
iterates the same objects). Custom-cover mtimes keep their cheap rowid
subquery and still win the coalesce. _CoverMtimeCoalesce is gone with
the annotation that needed it. Responses verified byte-identical.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 0a0546e2c92de8dd96773a37790980ea2bcb16b2
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 21:06:50 2026 -0700
browser: zero-pad from page rows; skip dead card serialization in table mode
- _get_zero_pad re-ran the whole ordered/annotated book query as a MAX
subquery (~20% of warm-request SQL) to derive one digit count.
Materialize the final page once (priming the result cache the
serializer reuses) and take the max issue_number off the page rows
in Python. The OPDS path keeps the aggregate variant.
- Table mode serialized ~100 cards through BrowserCardSerializer and
then popped them from the result. Pop the card fields from
self.fields before to_representation instead (and symmetrically the
rows field in card mode) — same wire output, none of the per-card
work. Fixed the stale 'mobile fallback' comment claiming cards stay
populated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit d1789eda2ac5a4351c3039ee42d4677db1c669c6
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 21:06:50 2026 -0700
table mode: evaluate the collection page query once, not twice
compute_collection_intersections read the page pks via
values_list('pk'), a clone that re-executed the entire annotated/
grouped/sorted collection query; the serializer then ran the original
queryset again. Iterating the queryset itself primes its result cache,
which the serializer reuses — the heaviest table-mode query now runs
once per request.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 820bbc5617c1ac0e1b1a50d5d7c54a461b3b7c28
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 21:06:50 2026 -0700
comic listings: stop aggregating order aliases; GROUP BY pk only
Two GROUP BY-width fixes for Comic-model querysets:
- _alias_filename aggregated Min/Max over the comic's own single path,
which turned the alias into an aggregate and dragged every selected
column into GROUP BY inside the listing and both cover subqueries.
A comic is its own file; use the bare expression (as
annotate_comic_extra_specials already did). A filename-ordered
folder browse measured 135s before, 0.6s after, identical rows.
- The ids JsonGroupArray made Django compute GROUP BY over all ~50
selected Comic columns including TEXT blobs. Force GROUP BY id —
every other column is functionally dependent on the pk, so groups
are identical and SQLite sorts a one-column key (25-34% off the
books query). Not applied to the cover path, where the forced
literal column doesn't survive nested-subquery aliasing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 14ecafa0781ea291822a0be894cf0a1cc390d4bd
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 20:56:40 2026 -0700
browser: UNREAD filter must not hide comics other users finished
The UNREAD arm was Q(bookmark=None) | (mine & unfinished):
bookmark=None matches only comics with no bookmark from ANY user or
session, so a comic finished by someone else (or a stale anonymous
session) matched neither arm and silently vanished from my unread
listing (reproduced: 50 foreign finished bookmarks shrank my unread
count by 50).
UNREAD now probes 'I have no finished bookmark on this comic' with a
correlated NOT EXISTS scoped to the requesting user/session. It
correlates on the queryset's comic pk, so on collection querysets it
binds to the same joined comic row as the other filters in the
combined .filter() call — group rows still require ONE comic that
satisfies every condition (pinned by test).
New comic-leading composite indexes (comic,user)/(comic,session) keep
the probe off the user index: under stale sqlite_stat1 the planner
flipped a user-scoped probe onto it — a measured 17s vs 27ms plan.
EXPLAIN confirms 'SEARCH ... USING INDEX bookmark_comic_user
(comic_id=? AND user_id=?)'. READ/IN_PROGRESS semantics unchanged;
single-user responses verified byte-identical.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit cad52d79e8242e3b69e4cbd604757f2c7faab728
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 20:56:40 2026 -0700
metadata: stop GET writing m2m through tables and corrupting tags
_copy_m2m_intersections called ManyRelatedManager.set(qs, clear=True)
for every real m2m field on the Comic path — obj is a live saved
instance (the 'values dicts' comment was stale), so a GET /metadata
DELETE+INSERTed up to 12 through tables, and a multi-comic selection
permanently rewrote the first comic's tags to the selection's
intersection (reproduced: 339 characters silently became 43).
Serve the intersection queryset through the instance's prefetch cache
instead: RelatedManager.get_queryset consults it keyed by field name,
so the serializer reads the already-optimized queryset query-free.
Response verified byte-identical; the 38-query identifier N+1 (the
post-write re-read discarded select_related) disappears with it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 71b52749a3b24a8493c6803f0c3cda04466280d4
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 20:19:11 2026 -0700
browse: bind favorite collection code as str so cachalot can cache
Binding the Collection StrEnum member itself made cachalot's
parameter-type check raise UncachableQuery for every query the
favorite subquery landed in — silently uncaching all browse queries.
Repeat table-mode browses re-paid ~10s of SQL every time; with .value
they serve from cache (16-22ms, 2-3 queries). The bound SQL value is
identical either way.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit a200e1ba3b261508cbe2c22cce6712ad19a6e0ea
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 20:19:11 2026 -0700
metadata: pk subquery instead of inlined IN lists; count the filtered relation
Two fixes to the intersection queries behind GET /metadata:
1. _get_comic_pks returned a materialized frozenset, inlining one bound
SQL variable per comic per union arm. With 12 m2m arms, any
collection over ~2,730 comics exceeded SQLite's 32,766-variable
limit and the endpoint returned 500 (reproduced at 4.2k, 8.5k and
17.8k comics). Passing the distinct .values(pk) queryset keeps the
statement constant-size (36 params); the same union measured 101ms
at 8,576 comics. num_comics is computed once via COUNT and threaded
to both consumers.
2. _get_fk_intersection_query counted the hardcoded 'comic' reverse
m2m even when the filter traversed the main_*_in_comics FK
back-relations — a semantically wrong HAVING over the unfiltered
relation that also forced a separate unindexed join (main_team
measured 2,069ms -> 12ms counting the filtered relation).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 5beb474835c7375f4b726eca63aa05f2ef8b9966
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 20:18:54 2026 -0700
arc browse: demote story-arc chain joins to INNER for StoryArc queries
With only codex_comic demoted, SQLite planned arc search browses from
the arc side and built a cartesian product against the FTS match set:
arcs root with a common search term measured 156s (main query 150.8s).
Demoting codex_storyarcnumber and the M2M through table lets the plan
start from the FTS/comic side (36ms). Scoped to StoryArc-model
querysets: there the chain IS the collection relation, so NULL-extended
rows are exactly the empty collections force_inner_joins exists to
drop. On other models these tables appear as table-view annotation
LEFT JOINs, where demotion would drop rows whose comics have no arcs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 1434a5a771fc7445472cb52726f4e86c645df999
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 20:18:54 2026 -0700
cover subqueries: join FTS rank only when ordering by search_score
The direct fts_q join exists solely to populate codex_comicfts.rank
for the cover subquery's search_score ORDER BY, but it was applied on
every search regardless of order key — and the MATCH re-executed once
per correlated cover evaluation. The pre-materialized fts_sq membership
test already filters correctly. A name-sorted search browse measured
135s before, 6.5s after (byte-identical rows); rank-ordered searches
keep the join and pick the same top-ranked cover as before.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 5bdc927cc0db62775913f0e9b3ca5a6d86d18192
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 20:18:40 2026 -0700
table mode: keep intersection-sort subqueries out of GROUP BY
RawSQL.get_group_by_cols() returns [self], so the correlated
intersection-sort subqueries landed in the GROUP BY of aggregated
collection queries. Folder is the one collection model without a
forced-GROUP-BY entry, so SQLite evaluated the subquery once per
pre-aggregation joined row — at folders root the ancestor M2M fans out
to every descendant comic (366s request; each of the two queries
measured ~256s alone, 85ms after). Same mechanism as the
_CoverMtimeCoalesce fix, recurring through a different expression type.
_IntersectionSortRawSQL overrides get_group_by_cols() to return []:
the subquery's only outer reference is the collection pk, which is
always grouped, so grouping is unchanged (verified row-identical).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 9f61f262b69fadc4cac463af5aafe88a9d248c9c
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 20:14:18 2026 -0700
fix librarian status feed order: cursor pagination discarded the order_by
The sidebar progress feed showed import phases scrambled — read next
to the search statii, query below create. The viewset ordered active
rows by (preactive, active, pk), but DRF CursorPagination re-orders
the queryset with its own ordering, so the v4 admin refactor's plain
pk default (545fc2aa9) silently replaced the registration-stamp order
with row-creation order.
LibrarianStatusCursorPagination orders the active poll by
(preactive, active, pk) — matching start_many's padded registration
stamps, with NULL preactive sorting first for rows started without
pre-registration (the cover thread case). The Jobs tab history view
keeps the pk default. The active feed is ~20 rows against a 200-row
page, so the cursor filter never engages the nullable fields.
Regression test creates status rows with pk order deliberately
opposite the registration stamps and asserts both routes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 396312c6fe9964b36f895dd6a2c8bb84757191c3
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 19:55:39 2026 -0700
import timing: don't double-count sub-steps in the phase share total
_run_phases reuses the timed_step helper instead of an inline
perf_counter copy, and _log_phase_times sums only top-level phases —
dotted "phase.step" entries already accrue inside their parents, so
including them inflated the total and shrank every phase's share.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 693d9dbb84efd6846c0a3941b1f3ca35de650afc
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 19:47:39 2026 -0700
metadata edit: full-size Protagonist label, live menu, stale-pick clear
The Protagonist row inherited the old Main sub-row styling (smaller
label, dimmed row) — it is a first-class row now, styled like its tag
siblings. The choice menu already tracked the entered character/team
names live (it is name-based, no pks involved); what was missing is
that removing the picked name left the selection dangling — a watcher
now clears it. A protagonist seeded from the DB but absent from the
entered lists still survives load untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 042c42db2fa0f1cce5deddb7ad3a3c2e8eea3c0a
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 19:29:28 2026 -0700
metadata: unify main_character/main_team as a single Protagonist field
Display: the starred main-chip machinery (MAIN_TAGS/markTagMain) is
replaced by one "Protagonist" row above the tag table showing the
mainCharacter and/or mainTeam chips — both only in the
should-never-happen case where both are set. Each chip carries its own
browser filter key (characters vs teams) via a per-item filter
override in MetadataTags. En route this fixes mapTag's sticky filter:
the loop reassigned the filter param, so every row after the first
reused the first row's filter key.
Edit: the two Main Character / Main Team sub-selects collapse into one
Protagonist select offering the entered characters and teams; the pick
is sent as the comicbox "protagonist" patch field on write, and the
importer resolves it back to main_character or main_team (nulling the
other) on re-import.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 632a238590f188ee249c4a8164a6936bcdfc779c
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 19:29:16 2026 -0700
importer: stop clearing an unchanged protagonist on update imports
The fk link phase set main_character/main_team to None whenever a
comic's remaining LINK_FKS dict lacked a "protagonist" key. The query
prune pops that key when the protagonist already matches the DB, so
re-importing a comic with any *other* FK change silently NULLed its
protagonist. Updates now only touch the protagonist fields when the
metadata explicitly carried one (set-or-clear), matching the
absent-means-untouched semantics of every other simple FK tag; creates
keep both fields defaulted. Also: a name that is both a Character and
a Team now links only main_character — both fields filled is invalid.
Regression test drives query → prune → link with only scan_info
changed and asserts the protagonist survives (it NULLed before).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit b2e6423badc0d49e4f0d2cca8f7c59bbbc634ead
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 16:16:30 2026 -0700
import perf: drop the timestamp updater's count aggregate (1.5s -> 0.1s)
TimestampUpdater filtered every collection model through a
Count(DISTINCT comic) alias to exclude empty collections — a per-row
GROUP BY join aggregate costing ~1.5s per import even with zero
deletions, in every import scenario.
The count was redundant for two of the three OR branches: a
comic__updated_at join match implies the collection has children, and
force-updated pks were exempted from the child filter anyway. Only
the custom-cover branch can match an empty collection, so it now
carries its own join-presence test (comic__isnull=False) instead.
Also replaces the fetch-instances + CASE-WHEN bulk_update round trip
with a batched .update(updated_at=Now()) over the distinct pks.
Also corrects the stale IMPORTER_FILTER_BATCH_SIZE comment: the
binding constraint on OR chains is SQLITE_LIMIT_EXPR_DEPTH (1000),
not the old 999-variable cap (modern SQLite allows 32766), so 900
remains correct for the remaining chain users.
Benchmark, 1000 mock comics: delete phase 1.46s -> 0.10s on fresh and
below 0.1s on reimport (5.9s total). Re-stamp semantics covered by
tests/importer/test_move_timestamps.py.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit bac8ee8af974bec78781f5c764e543f8429540a1
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 15:51:29 2026 -0700
import perf: bulk_update only changed comic fields (update step 8x)
update_comics passed all ~40 BULK_UPDATE_COMIC_FIELDS to bulk_update
regardless of what changed — CASE-WHEN compilation and parse scale
with rows x fields, so a force-reimport whose comics needed nothing
but a stat refresh paid the full-width rewrite: 4.4s of the 7s
remaining reimport time, almost all inside the two bulk_update
manager calls.
Collect the union of fields actually applied per batch (proposed md
diffs + FK link fields) and update that union plus the
presave-derived/timestamp set (stat, size, date, decade,
age_rating_metron_index, updated_at). Fields outside
BULK_UPDATE_COMIC_FIELDS are dropped as before.
Benchmark, 1000 mock comics: forced reimport 10.0s -> 7.0s,
update_comics sub-step 4.4s -> 0.55s. Fresh and noop unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 55401e6d372da3a9bf1c8b1cedeecbfd6fa06a27
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 15:46:01 2026 -0700
import perf: narrow existing-row queries with a selector IN, not Q-OR chains
Multi-key existing-row lookups (query phase query_existing_mds, link
and create pk-map builders) built per-key AND/OR Q chains — one
Django filter resolution per leaf and a giant OR for SQLite to parse,
per 500-900-key batch. Profiling a fresh import showed ~40% of its
time in this filter machinery.
Both consumers already match exact key tuples in Python from the
fetched values, so the chain only ever bounded the fetch. Replace it
with a single indexed IN on the model's most selective non-null key
rel (MODEL_SELECTOR_REL_MAP: imprint/series name, volume series name,
credit person, identifier key, story-arc name, folder path). Superset
rows — the same selector value under a different parent — are
harmless extra map entries. Keys with a None selector value fall back
to the old chain (selector columns are non-null, so normally none).
Also: read phase sub-instrumented (extract vs aggregate) and the
benchmark gained --profile (cProfile per scenario).
Benchmark, 1000 mock comics: fresh 12.1s -> 9.4s (query phase 2.6s ->
1.0s, link 2.5s -> 1.7s); reimport 11.8s -> 10.0s (query 3.4s ->
1.6s). Tag/link counts byte-identical.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 610c51b67580006567b168e0e260c6c6097433d4
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 14:11:28 2026 -0700
import perf: skip archive opens when the fs stat snapshot is unchanged
The fs prefilter compared disk mtime against the stored *embedded*
metadata mtime, which is older than the file's mtime for any archive
copied into the library after tagging — so the common at-rest comic
never skipped and every modified-event re-poll opened its archive
(an unrar spawn per CBR) just for comicbox to no-op.
Reuse the stat call to also compare disk (mtime, size) against the
stored Comic.stat snapshot — the poller's own modified test — and
skip the worker entirely on a match. Paths with no stored embedded
mtime still always survive, so a library imported with the
import-metadata flag off picks up tags after the flag turns on.
Benchmark, 1000 mock comics: noop poll 1.0s -> 0.2s (7.6s at
baseline); zero archive opens. Reimport unaffected (force path never
populates the prefilter's mtime map).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 06bec468cd68b423efb8f2b9da01be0f250c9d34
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 14:06:59 2026 -0700
import perf: skip true no-op stat-only updates (noop poll 7.6s -> 1s)
_envelope_deltas included metadata_mtime unconditionally, so every
file whose tags comicbox skipped still routed through the stat-only
update: the comic row rewritten, its covers purged and re-queued for
generation, and the import counted as changed -> cache.clear() +
LIBRARY_CHANGED broadcast. A modified-event burst over an unchanged
library rewrote every row and regenerated every cover.
Compare metadata_mtime against the stored value, and skip the
stat-only routing entirely when the envelope is unchanged AND the
on-disk (mtime, size) still matches Comic.stat — the same equality
the poller's SnapshotDiff uses, so a skipped file is exactly one the
poller would not re-emit (no stale-stat re-poll loop). Touched files
(stat moved, content same) keep today's full stat-only path, which
deliberately refreshes Comic.stat via presave.
Benchmark, 1000 mock comics: noop poll 7.6s -> 1.0s with "No updates
necessary" (the remaining ~0.9s is archive opens for the comicbox
mtime check). Fresh and reimport unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 488af595b62d643f130098f38eb2d0d8760230cd
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 13:40:39 2026 -0700
import perf: batch-resolve comic fk links (fresh import 3x)
Comic creation resolved every FK link with objects.get() per
(comic, field) — folder, two protagonist lookups and ~one per FK
field, ~14 single-row SELECTs per created comic. create_comics was
59% of a fresh import.
prepare_fk_link_instance_maps now runs once per chunk after the FK
create/update steps: one batched query per referenced field over the
distinct key tuples (reusing the link phase's pk-map builders), plus
in_bulk for instances. get_comic_fk_links stitches from the maps.
Instances, not pks, so Comic(**md), the update path's
setattr/default handling and presave's age_rating dereference are
unchanged. Missing rows still raise DoesNotExist and skip the comic.
The parent-folder map is also library-scoped now; the old
Folder.objects.get(path=...) could MultipleObjectsReturned when two
libraries shared a directory path.
Also fixes a latent crash exposed by the new path:
_build_pk_map_multi_key sorted key tuples that mix None with values
(volume year/number_to, story-arc numbers) — TypeError when two keys
first differ at a None position. Sort key now orders None last.
Benchmark, 1000 mock comics: fresh 35.1s -> 12.1s; create_comics
sub-step 21.7s -> 0.25s (map building 0.5s). Cumulative since
baseline: fresh 2.9x, forced reimport 131.6s -> 11.8s (11x).
Sub-step timing for create_and_update phases included
(timed_step helper; dotted names excluded from share totals).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 94dd32a4ab9a74e2fd250aea22340c41ff35b5e5
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 08:59:44 2026 -0700
import perf: compare fk prune links as values tuples (query 12s -> 3.7s)
The fk prune select_related'd only the directly referenced FK fields,
then walked each key rel via getattr chains — every hop past depth one
(volume -> series -> imprint -> publisher) lazy-loaded a single-row
SELECT per comic, and the protagonist check lazy-loaded main_character
and main_team likewise.
Query every referenced field's key rels in one batched values_list and
compare row slices against the proposed key tuples. Also makes the
protagonist prunable when it's the only referenced field (it was
skipped before because it isn't a concrete FK for select_related).
Benchmark, 1000 mock comics: reimport 23.9s -> 13.2s, query phase
12.2s -> 3.7s. With the m2m prune rewrite, forced reimport is 10x
faster than the 131.6s baseline. Fresh and noop unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 55ceae11efd87c421f59b616ec65bf64882d4a8e
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 08:53:01 2026 -0700
import perf: scan m2m prune off the through tables (10x)
The m2m prune walked prefetched related objects per comic and read
key attrs through instance getattr chains — for credits, identifiers
and story-arc-numbers each FK in the key tuple lazy-loaded with a
single-row SELECT per through row. A forced reimport of 1000 comics
spent 121s (92%) in this phase.
Scan the through tables directly instead: one values_list query per
(field, batch) with the key attrs joined in SQL, pruned via set ops
against the proposed links. No comic or related-model instances, no
lazy FK loads. get_through_model moves from the link phase to const
so the query phase can share it.
Same pruning semantics (kept/deleted/FTS-existing/moved-source
capture verified by tests/importer fixture diffs). Benchmark, 1000
mock comics: reimport 131.6s -> 23.9s; prune phase 121.0s -> 12.2s.
Fresh and noop unchanged (nothing to prune).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 56144211fdeff41b084302bb2e7d566d9d2f16c0
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 10 02:21:36 2026 -0700
import perf phase 0: per-phase timing + repeatable import benchmark
- Importer accumulates wall time per phase across chunks and logs a
share table at finish (DEBUG).
- bin/benchmark-import.py: deterministic mock library (seeded
mock_comics) + scratch CODEX_CONFIG_DIR, times fresh / noop /
force-reimport scenarios and prints per-phase tables.
- mock_comics: _hex_path adler32 collisions silently overwrote ~2/3 of
requested files; keep checksum dirs, name files by index.
- ruff: bin/* scripts may print (replaces per-line noqas in roman.py).
Baseline (1000 mock comics, ~77 tags each): reimport spends 92% in the
query/prune phase (121s, superlinear); fresh spends 69% in
create_and_update (24s, superlinear); noop rewrites every comic row via
an unconditional metadata_mtime envelope delta. Plan re-ranked in
tasks/todo.md; pipeline-overlap idea dropped (read is ~8% of fresh).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 3d7c290e0cba1bb4883a7a3bbdd5379d2e5fc820
Author: AJ Slater <aj@slater.net>
Date: Tue Jun 9 22:51:28 2026 -0700
remove unused css
commit 9e7df00b27a2074563482f4d788e3a19017b2c59
Author: AJ Slater <aj@slater.net>
Date: Tue Jun 9 22:49:12 2026 -0700
age rating As-tagged tab: standardized column first, no parens
Swap the two columns: the dim standardized equivalent now leads in a
fixed-width left column (10ch, so the raw tags align), the raw tagged
value follows it, and the parentheses are gone. The column renders
empty for unmapped tags to preserve alignment.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit a6b9de4e63192d1c13a264b8650a9cec3cb98c6e
Author: AJ Slater <aj@slater.net>
Date: Tue Jun 9 22:48:00 2026 -0700
update comicbox
commit 09237388499591adf5c044dcf2384cdaf693d27f
Author: AJ Slater <aj@slater.net>
Date: Tue Jun 9 22:47:01 2026 -0700
0043: relink AgeRating.metron FKs with the current comicbox mapping
AgeRating.metron is derived once, in presave at row creation, so
mappings comicbox gains later (e.g. Rating Pending -> Unknown in
4.0.0a2) never reach existing rows. Add a data migration that
recomputes every FK with the live comicbox lookup and heals the two
denormalizations that derive from it: Comic.age_rating_metron_index,
and the FTS age_rating_metron column by bumping affected comics'
updated_at past the FTS watermark so the next search sync refreshes
their entries.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 93475e7104e97b180202124353a9680856ba097f
Author: AJ Slater <aj@slater.net>
Date: Tue Jun 9 22:44:49 2026 -0700
fix quadratic folder browse: keep cover_mtime coalesce out of GROUP BY
Django adds non-aggregate Func annotations to the GROUP BY wholesale, so
the Coalesce(custom cover, comic cover) mtime subqueries were evaluated
once per pre-aggregation joined row. In folder mode the root folder's
ancestors M2M fans out to every descendant comic, and each evaluation
re-scanned that same set: an 18k-comic folder took minutes to browse.
Delegate get_group_by_cols to the source subqueries (their correlation
column is already in the GROUP BY), restoring bare-Subquery behavior:
0.7s.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit f19552423ab4e009ba6bbb812cddeec570b6d1ab
Author: AJ Slater <aj@slater.net>
Date: Tue Jun 9 22:31:16 2026 -0700
age rating As-tagged tab: right-justify the standardized column
Move the parenthetical standardized equivalent out of the row title
into a right-aligned append-slot column, so the As-tagged list reads
as two columns: raw tag left, "(Standardized)" right.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 73088c6ff2ecd4145d9970ff2d8a2f02d79bac43
Author: AJ Slater <aj@slater.net>
Date: Tue Jun 9 22:19:34 2026 -0700
age rating As-tagged tab: show all tagged values with standardized name
The As-tagged tab previously hid every tagged value that had a
Metron mapping, leaving it nearly empty (often just Rating Pending).
Now it lists every raw tagged age rating, annotating each with its
standardized equivalent in parentheses when they differ, e.g.
"MA15+ (Teen Plus)". The tab is hidden only when the library has
no tagged values at all. The old right-aligned metronName column,
its None-row header hack, and dead taggedColumnHeader styles are
removed in favor of the inline parenthetical.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 2de448a635b5918843a785f008bacc6828868275
Author: AJ Slater <aj@slater.net>
Date: Tue Jun 9 22:01:03 2026 -0700
remove out of date help
commit 09484cc5970fb7116846befe384ebe2e5f525ce0
Author: AJ Slater <aj@slater.net>
Date: Tue Jun 9 21:46:41 2026 -0700
age rating filter sub-menu: replace expansion panels with tabs
The Standardized / As-tagged split now renders as compact tabs
instead of accordion panels. When no unstandardized tagged values
exist the tab bar is hidden entirely and only the Standardized list
shows. A primary checkbox icon on a tab marks filters with active
selections, and the shared search switches tabs when matches are
exclusive to the other tab.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit d44b81697389ebed8330c7591ac08607a14b06d8
Author: AJ Slater <aj@slater.net>
Date: Tue Jun 9 21:12:34 2026 -0700
fix importer tests leaking an open class-level transaction
BaseTestImporter.tearDownClass never called super(), so TestCase's
class atomics were never rolled back. The import run's open
transaction (and its SQLite shared-cache table locks) leaked into
every subsequent test, making fts_rebuild's PRAGMA wal_checkpoint
fail with 'database table is locked' in full-suite runs.
Also run FtsRebuildTests as TransactionTestCase: fts_rebuild
checkpoints the WAL, which is incompatible with any wrapping
transaction that has already executed statements. Production calls
it from the janitor in autocommit, so the unwrapped test matches
real semantics and is robust against future fixtures touching the
DB first.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 835b6d020116c03ae409413726d42abd6d9d52fa
Author: AJ Slater <aj@slater.net>
Date: Tue Jun 9 20:34:23 2026 -0700
remove duplicate status
commit 82c1128f3ce8583ebcd632377acab2696d8f82e1
Author: AJ Slater <aj@slater.net>
Date: Tue Jun 9 20:32:40 2026 -0700
fix basedpyright deprecation warnings: Iterator -> Generator for contextmanagers
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
commit 66ab9f913fe2bc0fc08fa3d785b2b6baea6bb5fd
Author: AJ Slater <aj@slater.net>
Date: Tue Jun 9 11:54:21 2026 -0700
Squashed commit of the following:
commit 3a339a4a016f0f16055cf086ee085b6e02a23ac2
Author: AJ Slater <aj@slater.net>
Date: Tue Jun 9 11:54:06 2026 -0700
update deps and version to 2.0.0a9
commit 2923da8cfec40ca9c56bd1a28ed6bcb6b1cac756
Author: AJ Slater <aj@slater.net>
Date: Tue Jun 9 01:37:03 2026 -0700
refactor(librarian): log FTS rebuild from the shared fn for every caller
fts_rebuild() was the only integrity helper that logged nothing itself,
relying on JanitorDBFTSRebuildStatus.log_success to surface a generic
status line. Its three siblings (integrity_check, fts_integrity_check,
fix_foreign_keys) each log a specific success inside the shared function
so any caller benefits. Match them: log the rebuild success in the shared
fn and drop the status's log_success, so the wrapper emits one generic
INFO line plus the specific SUCCESS line like the other integrity tasks.
No live caller bypassed the wrapper -- this is a symmetry/defensive change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
commit 099d4ba1eafccf1a399782d3c431da3ba868ced4
Author: AJ Slater <aj@slater.net>
Date: Tue Jun 9 01:31:45 2026 -0700
fix(librarian): log snapshot/restore and failed-import re-queue for every caller
Completion logs for these librarian operations lived in a single caller's
wrapper, so other entry points ran silently:
- User-data snapshot logged only from the nightly janitor wrapper, so the
admin "Snapshot now" button produced a backup with no log. Move the log
into the shared snapshot_sidecar(); restore() now logs a completion line
(covering the admin view and the restore_user_data CLI command). Drop the
now-redundant janitor log.
- force_update_all_failed_imports (admin "Failed Imports" button) had no
status and no log, running silently -- especially on the zero-failed
no-op. Log the queued total once, mirroring force_update's sibling.
Add regression tests asserting both shared paths emit a summary line.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
commit dd3d57bdc241bb8b3c1e2630bf95792c34dd9fe7
Author: AJ Slater <aj@slater.net>
Date: Tue Jun 9 00:41:55 2026 -0700
Squashed commit of the following:
commit c009b02c3147ebf30bd0c0f86b1f46d6dbab4f5a
Author: AJ Slater <aj@slater.net>
Date: Tue Jun 9 00:41:35 2026 -0700
format md
commit bd4bf37372f32d6026260771636b94764681ea61
Author: AJ Slater <aj@slater.net>
Date: Tue Jun 9 00:41:29 2026 -0700
update deps
commit f40076874f7fa1886fbad1f0046c73079df1eaaa
Author: AJ Slater <aj@slater.net>
Date: Tue Jun 9 00:30:40 2026 -0700
update version to 2.0.0a8
commit d394064c062bc562640bcdda7c5201c9c911ec91
Author: AJ Slater <aj@slater.net>
Date: Tue Jun 9 00:30:18 2026 -0700
docs: require a scribe priority for new librarian jobs
New ScribeTask/JanitorTask classes must be registered in _SCRIBE_TASK_PRIORITY (and janitor jobs in _JANITOR_METHOD_MAP + _NIGHTLY_TASK_CLASSES). Notes the ValueError symptom when omitted and the test_scribe_priority.py guard.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
commit c7429fc42ad1667d2aeeca823496cc20509244b5
Author: AJ Slater <aj@slater.net>
Date: Tue Jun 9 00:30:11 2026 -0700
fix(librarian): stop nightly janitor crashes from unranked task and stale show-flags
JanitorFolderRelationsCheckTask was dispatched via _JANITOR_METHOD_MAP / _NIGHTLY_TASK_CLASSES but absent from _SCRIBE_TASK_PRIORITY, so get_task_priority's tuple.index raised ValueError when it was queued and the folder-relations repair never ran.
The user_data sidecar dump/restore still read SettingsBrowserShow.{p,i,s,v}, renamed to {publishers,imprints,series,volumes} in migration 0044, so every SettingsBrowser row failed to serialize and was silently dropped from the nightly snapshot (and would fail on restore). The sidecar SQL columns stay show_{p,i,s,v} (storage keys, no schema migration).
Add regression tests for both, each proven to fail pre-fix: test_scribe_priority.py enforces _JANITOR_METHOD_MAP keys are a subset of _SCRIBE_TASK_PRIORITY; test_user_data_dump/restore now create a real SettingsBrowser+Show and round-trip the show flags.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
commit 0356e309401d8ec78b5e7c108c7d60ec2c9e7d0f
Author: AJ Slater <aj@slater.net>
Date: Mon Jun 8 20:40:20 2026 -0700
fix(types): clear basedpyright and ty errors in folder-relations code
make typecheck (basedpyright) reported 10 errors and make ty 1; both
now pass cleanly.
- foreign_keys.py: django-stubs types Field.remote_field as Field, which
hides the M2M .through attribute. Cast remote_field to the real
ManyToManyRel so both checkers resolve it, dropping the old ty:ignore.
- test_janitor_folder_relations.py: ignore reportUninitializedInstanceVariable
on setUp fixtures (matching test_metadata_path.py); None-narrow the
optional parent_folder FK before .path; use parent_folder.pk instead of
the plugin-only parent_folder_id companion attr basedpyright can't see.
- test_metadata_path.py: annotate _get_path -> str and coalesce the
optional .get("path") so .startswith() no longer trips Optional access.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
commit c3a7ac6c857baa8c94dca5e81c1ad81229b06d69
Author: AJ Slater <aj@slater.net>
Date: Mon Jun 8 20:24:01 2026 -0700
refactor(librarian): drop folder repair from startup; fix job descs
Startup queued JanitorAdoptOrphanFoldersTask alongside the search-index
sync, but structural folder repair doesn't belong at boot: it's an
idempotent no-op when the DB is consistent, only drifts from botched
imports (not offline filesystem changes), and logically follows a
re-poll rather than preceding it. Adopt-folders also already queues its
own SearchIndexSyncTask, which startup runs anyway. Both folder repairs
(adopt + folder-relations) still run nightly and on demand.
Startup now reconciles only the search index against the DB.
Also corrects two Jobs-menu descriptions that overstated startup:
- cleanup_tagging_state never ran the full task at startup (only the
online-tag thread clears its scan marker); now "Runs nightly."
- adopt_folders updated to match the behavior change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
commit 2378794c7526c36c77b2cec481a32938ad94dd7d
Author: AJ Slater <aj@slater.net>
Date: Mon Jun 8 19:57:35 2026 -0700
format
commit 3c2e0b22d8e24d6f4d374b2ee7eeb8b4ff1a699e
Author: AJ Slater <aj@slater.net>
Date: Mon Jun 8 19:56:26 2026 -0700
bump version 2.0.0a7
commit b8226af7eadc386ece5b04c1fa66156b5ae21d87
Author: AJ Slater <aj@slater.net>
Date: Mon Jun 8 19:55:12 2026 -0700
more spacing for button
commit 08833e9c2cff4b1e64917ce8b9ca04ccf83ee8ae
Author: AJ Slater <aj@slater.net>
Date: Mon Jun 8 19:51:08 2026 -0700
refactor(admin): move Database jobs section above Search Index
Reorder the ADMIN_JOBS menu groups so Database precedes Search Index.
Pure reorder of the two group dicts; no behavior change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
commit 8e4eae3702531996be49a1fefa9e2169420e78b0
Author: AJ Slater <aj@slater.net>
Date: Mon Jun 8 19:49:04 2026 -0700
feat(admin): add Remove Orphan Settings cleanup job
JanitorCleanupSettingsTask (status JAS) ran only via nightly
maintenance — unlike every sibling cleanup, it had no standalone admin
Jobs button. Wire it into the Jobs menu:
- Map cleanup_settings -> JanitorCleanupSettingsTask in the dispatch map
- Add the "Remove Orphan Settings" button to the Cleanup group
- Add JAS to the nightly-job status list so Run Nightly Maintenance
surfaces its progress
The serializer's valid-choice set and the frontend menu both derive
from ADMIN_JOBS, so the button alone makes the task POST-able.
Also trims the db_folder_relations_check job description.
Co-Author…
The bookmark composite indexes and librarianstatus fields were folded into 0043 during the merge-fix commit (c31c61e), but the orphaned 0044_bookmark_comic_indexes.py was left behind with identical AddIndex operations. Re-running them aborted test-DB creation with "index bookmark_comic_user already exists" (415 test errors). makemigrations --check reports no changes; full suite 558 passed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.