harness enhance

[TXT0Law]

No description provided.

[github-actions]

Coverage Summary

Service	Coverage
Frontend coverage	unavailable
Backend coverage	unavailable
OSINT coverage	unavailable

Overall status: unavailable

[Copilot]

Pull request overview

This PR strengthens OrbiCheck’s “harness” (CI quality gates + contracts) and runtime hardening across the frontend, backend, scan-service, and scanner. It adds stricter validation/coverage enforcement, introduces internal-service auth, and improves operational/infra safeguards so regressions and unsafe defaults are caught earlier.

Changes:

• Enforce coverage floors (Vitest + Pytest) and add new harness checks (docs drift, bundle budgets, structured dependency/boundary validation).
• Add internal HMAC auth for service-to-service calls plus outbound URL safety/pinning (including Playwright routing hardening).
• Harden containers (non-root, safer runtime env defaults) and add ops scripts/docs (backup/restore drill, container contract checks).

Reviewed changes

Copilot reviewed 155 out of 159 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
vitest.config.ts	Adds Vitest coverage thresholds to enforce frontend coverage floors.
tests/scripts/bundle-budget.test.ts	Unit-tests the new per-route bundle budget checker.
tests/pages/scan-trend-page.test.tsx	Makes trend chart test await async rendering.
tests/pages/scan-detail-pages.test.tsx	Makes scan summary test await async sections.
tests/pages/reports-pages.test.tsx	Makes report preview test await async content.
tests/lib/utils/export-json.test.ts	Updates downloadJson import to new API download module.
tests/lib/utils/export-download.test.ts	Updates module import path for download helper.
tests/lib/utils/export-csv.test.ts	Updates downloadCsv import to new API download module.
tests/lib/export-json.test.ts	Splits imports between API download + export pickers.
tests/lib/api/core-response-validation.test.ts	Adds tests asserting API response shape validation failures.
tests/lib/api-scans.test.ts	Updates apiClient mocking and makes scan detail expectation deep-equal.
tests/connected/accessibility.connected.spec.ts	Adds connected Playwright + Axe accessibility checks.
tests/components/scan/scan-header.test.tsx	Updates mocks for download functions now in lib/api/download.
tests/components/scan/details/ssl-detail.test.tsx	Makes SSL detail test await dynamic chart content.
tests/components/monitor/monitor-change-timeline.test.tsx	Asserts new content-visibility rendering behavior.
shared/schemas/scan.ts	Expands scan-related Zod schemas and tolerates legacy/snake_case fields.
shared/schemas/report.ts	Adds Zod schemas for report list/record/preview payload parsing.
shared/schemas/monitor.ts	Adds Zod schema for alert events.
scripts/test-phase1-container-network.sh	Adds container-network security smoke checks for phase 1.
scripts/test-p2-container-hardening.sh	Adds runtime hardening assertions + scanner smoke checks.
scripts/ops/backup-restore-drill.sh	Adds encrypted pg_dump backup + isolated restore drill script.
scripts/dev/start-linked-backend.py	Reworks linked backend DB reset to be Alembic-only.
scripts/ci/test.sh	Enforces backend coverage floor via --cov-fail-under.
scripts/ci/test-require-tests.sh	Adds self-tests for new require-tests gate scenarios.
scripts/ci/test-harness-checks.py	Adds fixture self-tests for structured harness checks.
scripts/ci/require-tests.sh	Improves base/head handling; expands test requirements across subsystems.
scripts/ci/check-harness.sh	Adds harness checker self-tests and docs drift check to harness job.
scripts/ci/check-dependency-direction.sh	Switches dependency direction gate to structured Python checker.
scripts/ci/check-bundle-budget.mjs	Implements per-route gzip bundle budget enforcement.
scripts/ci/check-boundary-validation.sh	Switches boundary validation gate to structured Python checker.
quickstart/quickStart.md	Updates scan-service install guidance and container notes.
package.json	Adds bundle budget script, adds sharp, adds axe-core/playwright dev dep.
next.config.mjs	Makes persistent webpack cache opt-in via env flag.
Makefile	Adds new targets (bundle budget, docs checks, scanner tests, hardening, backup drill).
lib/utils/export-json.ts	Removes browser download side-effect helper from utils layer.
lib/utils/export-csv.ts	Removes browser download side-effect helper from utils layer; updates comment.
lib/api/reports.ts	Adds Zod parsing for report responses + uses new download helper.
lib/api/download.ts	Centralizes browser download helpers in API layer (csv/json/blob + API GET).
lib/api/alerts.ts	Adds Zod parsing for alert responses and metadata.
docs/scan-modules.md	Updates scan module source-of-truth and internal auth notes; clarifies exclusions.
docs/operations.md	Adds operational docs for hardening, backups, and trace correlation.
docs/inventory.json	Adds generated inventory of services/routes/modules.
docs/harness/file-ownership.md	Fixes references to agent responsibility docs.
docs/harness/done-definition.md	Updates done-definition to include migrations + scanner and docs checks.
docs/harness/dependency-direction.md	Updates dependency direction docs to reference structured checker + fixtures.
docs/harness/boundary-validation.md	Updates boundary validation docs to reference structured checker + fixtures.
docs/api/report-schedules.md	Adds report schedules API doc stub pointing to OpenAPI.
docs/api/README.md	Expands API docs index and notes single-admin model.
docs/api/maintenance-windows.md	Adds maintenance windows API doc stub pointing to OpenAPI.
docker/scanner/test_app.py	Adds FastAPI scanner security/readiness unit tests.
docker/scanner/Dockerfile	Hardens scanner image (non-root, HOME/tmp, chown).
docker/scan/Dockerfile	Hardens scan-service image (non-root, HOME/tmp, chown).
docker/init/init-db.sql	Removes table bootstrap; establishes Alembic as sole schema authority.
docker/frontend/Dockerfile	Hardens frontend runtime image (non-root, HOME/tmp, chown).
docker/celery/entrypoint.sh	Runs celery directly and adds beat schedule file location.
docker/backend/entrypoint.sh	Uses Alembic upgrade only; removes create_all/stamp bootstrap logic.
docker/backend/Dockerfile	Hardens backend image, uses uv.lock + uv sync, non-root user, HOME/tmp.
docker-compose.prod.yml	Adds required production env vars and internal auth requirements.
components/scan/scan-header.tsx	Switches download functions to lib/api/download.
components/scan/details/ssl-detail.tsx	Lazy-loads SSL charts client-side to reduce initial bundle cost.
components/monitor/monitor-change-timeline.tsx	Adds content-visibility optimizations; switches download imports.
backend/tests/unit/test_report_service.py	Updates docstring to remove prompt_dev reference.
backend/tests/unit/test_outbound_http.py	Adds unit tests for outbound HTTP pinning, redirect revalidation, and auth signing.
backend/tests/unit/test_monitor_schema_fetch_mode.py	Updates docstring language to remove prompt_dev reference.
backend/tests/unit/test_monitor_execute_check_expanded.py	Switches tests to monkeypatch request_safely, aligning with new outbound HTTP path.
backend/tests/unit/test_migration_bootstrap.py	Adds regression tests ensuring Alembic-only bootstrap + secrets enforcement.
backend/tests/unit/test_docs_drift.py	Adds regression tests for generated docs inventory and local link checks.
backend/tests/unit/test_container_hardening.py	Adds static regression tests for container hardening and compose policies.
backend/tests/unit/test_config.py	Adds production config validation tests (placeholders, HTTPS-only constraints).
backend/tests/conftest.py	Sets default env to disable DNS pinning in tests.
backend/scan/tls.js	Pins outbound TLS probe to validated address while preserving SNI hostname.
backend/scan/tech-stack.js	Adds gated Wappalyzer browser behavior for isolated egress environments.
backend/scan/status.js	Updates history comment wording.
backend/scan/ssl.js	Pins TLS connect host to validated address while preserving servername.
backend/scan/server.js	Adds internal auth middleware, raw body capture, lifecycle start/stop and shutdown handling.
backend/scan/security-txt.js	Updates history comment wording.
backend/scan/runner.js	Updates timeout comment wording.
backend/scan/registry.js	Excludes jest.live.config.js from module registry.
backend/scan/ports.js	Adds internal-auth headers to scanner calls and pins native scan target to validated address.
backend/scan/page-source.js	Switches to shared httpWith client with size/time bounds and TLS relax control.
backend/scan/package.json	Adds live test script, detects open handles; removes cors dependency usage.
backend/scan/package-lock.json	Removes cors and transitive deps from lockfile.
backend/scan/jest.live.config.js	Adds separate Jest config for live/integration suite.
backend/scan/jest.config.js	Adds coverage thresholds and excludes live tests from hermetic runs.
backend/scan/hsts.js	Pins DNS resolution for HTTPS requests and keeps HEAD/GET fallback.
backend/scan/associated-hosts.js	Pins certificate probing to validated IP and threads request context through handler.
backend/scan/AGENTS.md	Updates module envelope field naming convention to durationMs.
backend/scan/.env.example	Documents internal service auth env vars.
backend/scan/_common/tech-stack-fallback.js	Injects request function for testability and uses shared HTTP client by default.
backend/scan/_common/playwright-browser.js	Adds safe routing for Playwright requests with size bounds and URL safety enforcement.
backend/scan/_common/middleware.js	Adds runtime URL safety resolution and stores resolved target in request context.
backend/scan/_common/internal-auth.js	Implements internal HMAC auth helpers + middleware and production config validation.
backend/scan/_common/http.js	Uses safe HTTP adapter to enforce URL safety/pinning for axios clients.
backend/scan/_common/host-limiter.js	Updates rationale comment wording.
backend/scan/_common/circuit-breaker.js	Updates rationale comment wording.
backend/scan/tests/url-safety.test.js	Adds tests for URL safety policy and safe axios adapter behavior.
backend/scan/tests/tech-stack-fallback.test.js	Updates tests to use injected request function rather than global fetch mocking.
backend/scan/tests/server-lifecycle.test.js	Adds lifecycle test for start/stop server behavior.
backend/scan/tests/result.test.js	Updates comment wording around envelope migration.
backend/scan/tests/ports.test.js	Updates expectation for new scanner authorization_acknowledged payload.
backend/scan/tests/internal-auth.test.js	Adds tests for internal auth middleware behavior and CORS header absence.
backend/scan/tests/batch.live.test.js	Moves live test to dedicated config (not skipped in-file).
backend/app/utils/url_safety.py	Adds resolved-url return type and stronger public-only resolution policy.
backend/app/services/url_group_run_service.py	Extracts terminal-state check helper for reuse.
backend/app/services/scan_trend.py	Updates module docstring wording.
backend/app/services/scan_client.py	Adds InternalServiceAuth to scan-service HTTPX clients.
backend/app/services/report_service.py	Updates comment wording around SSL chain preview.
backend/app/services/report_charts.py	Updates module docstring wording.
backend/app/services/recommendations.py	Updates module docstring wording.
backend/app/services/notification_test_service.py	Extracts synthetic monitor builder into reusable service helper.
backend/app/services/notification_channels/webhook.py	Uses shared post_json helper and adds blocked-target error handling.
backend/app/services/notification_channels/lifecycle.py	Uses shared post_json helper for lifecycle webhooks.
backend/app/services/notification_channels/_helpers.py	Routes outbound requests via request_safely + enforces URL public/HTTPS policy.
backend/app/services/monitor_service.py	Switches outbound monitor fetches to request_safely with size bounds and block handling.
backend/app/services/maintenance_window_service.py	Adds serializer helper used by API endpoint responses.
backend/app/services/content_rendered_fetch.py	Adds InternalServiceAuth to rendered DOM fetch client.
backend/app/main.py	Removes create_all on startup; relies on Alembic/migrations.
backend/app/db/migrations/versions/module_status_retrying_uppercase.py	Adds enum value migration for module status.
backend/app/db/migrations/versions/initial_scan_schema.py	Adds explicit initial scan schema migration.
backend/app/db/migrations/versions/celery_task_id_on_scans.py	Corrects down_revision to chain from initial schema.
backend/app/core/internal_auth.py	Adds internal-service HMAC signing for backend-to-service calls.
backend/app/core/config.py	Adds production safety validation and internal-service auth config settings.
backend/app/api/v1/schemas/monitor.py	Updates browser fetch mode comment wording.
backend/app/api/v1/endpoints/url_groups.py	Reuses terminal-run helper and adds explicit return typing.
backend/app/api/v1/endpoints/scans.py	Adds explicit return typing to scan endpoints.
backend/app/api/v1/endpoints/reports.py	Adds explicit return typing to download endpoint.
backend/app/api/v1/endpoints/monitors.py	Adds explicit return typing to SSE/download endpoints.
backend/app/api/v1/endpoints/me.py	Uses extracted synthetic monitor helper and removes inline helper.
backend/app/api/v1/endpoints/maintenance_windows.py	Uses serializer-based response construction and simplifies imports.
backend/.env.example	Adds dev INTERNAL_SERVICE_SECRET and production-related env placeholders.
app/login/page.tsx	Adds main-content anchor target and improves heading semantics.
app/layout.tsx	Adds a global “Skip to main content” link.
app/globals.css	Adds reduced-motion overrides for accessibility.
app/dashboard/scan/page.tsx	Improves heading semantics in scan list view.
app/dashboard/scan/[scanId]/trend/page.tsx	Lazy-loads trend chart client-side.
app/dashboard/scan/[scanId]/page.tsx	Lazy-loads heavy security chart section client-side.
app/dashboard/scan/[scanId]/layout.tsx	Adds main-content anchor target in scan layout (but currently duplicates parent main).
app/dashboard/reports/[reportId]/page.tsx	Lazy-loads Markdown preview client-side.
app/dashboard/monitor/[monitorId]/uptime/page.tsx	Lazy-loads charts client-side.
app/dashboard/monitor/[monitorId]/ssl/page.tsx	Lazy-loads SSL expiry chart client-side.
app/dashboard/monitor/[monitorId]/layout.tsx	Adds main-content anchor target in monitor layout (but currently duplicates parent main).
app/dashboard/layout.tsx	Adds main-content anchor target for dashboard routes.
AGENTS.md	Updates global agent docs with new bundle/docs verification commands.
.gitignore	Ignores backups/ directory created by backup drill.
.github/workflows/production.yml	Switches scan-service install to npm ci.
.github/workflows/osint-live.yml	Adds scheduled/manual workflow for live scan-service tests.
.github/workflows/deploy.yml	Switches scan-service install to npm ci.
.github/workflows/ci.yml	Adds migrations + scanner jobs, installs backend deps for harness, enforces build+bundle budgets.
.do/app.yaml	Adds health checks and removes secret placeholders from deployable spec; documents required secrets.

Files not reviewed (2)

• backend/scan/package-lock.json: Generated file
• pnpm-lock.yaml: Generated file

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.