Skip to content

test-stubs.md

Latest commit

 

History

History
77 lines (56 loc) · 7.68 KB

test-stubs.md

File metadata and controls

77 lines (56 loc) · 7.68 KB
Raw
domain software-craft
tags
test-stubs
traceability
pytest-beehave
scenario-outline
hypothesis
last-updated 2026-05-20

Test Stubs

Key Takeaways

  • Test stubs are auto-generated by beehave generate <feature_title> from the feature file; no manual stub creation is needed. pytest-beehave uses title-based mapping: each Example title becomes a test function named test_<example_title_slug>.
  • beehave check verifies structural traceability (every Example has a test, no orphan tests, placeholders present, literals present). Scenario Outline produces parameterized stubs with Hypothesis @given decorators (inferred strategies) and @example decorators (one per Examples table row).
  • Literals from Given/When/Then steps (quoted strings, bare numbers) must appear verbatim in test function bodies — beehave check enforces this. Stubs (functions with ... body) are exempt from literal and placeholder checks.
  • beehave status reports development stage per feature across 6 stages (ok, broken, needs scenarios, needs tests, needs bodies, needs fixes) with tree or --json output. Use beehave status --json for project-wide overview and beehave status for per-feature tree view including inline violation codes.
  • beehave list lists feature slugs and titles for features with Examples; list -v adds path, scenario count, stub/impl counts. beehave clean <feature> --force removes unmapped test functions from feature-paired test directories.
  • Feature file stem MUST match the Feature title slug (e.g., Feature "CLI Entrypoint" → cli_entrypoint.feature). Title violations anywhere in the project block beehave generate (pre-flight validates all titles project-wide). beehave check <single-feature> skips global title validation — only beehave check (no argument) runs validate_all_titles.

Concepts

Title-Based Mapping and Auto-Generated Stubs. pytest-beehave maps each Example or Scenario Outline in the feature file to a test function by title. The function name is derived from the Example or Scenario Outline title as a slug (e.g., Example: "VAT is applied at the correct rate" → test_vat_is_applied_at_the_correct_rate). Titles must be unique within a feature file and 2–6 words per [[requirements/gherkin#concepts]]. When beehave generate <feature_title> runs, pytest-beehave reads the feature files and creates test stubs automatically. Each stub has an ... (Ellipsis) body. During pytest collection, pytest-beehave auto-skips any test function with an ... body — no @pytest.mark.skip decorator is needed. The SE replaces the ... body with the test implementation during the RED phase.

Traceability Verification and Scenario Outline Stubs. beehave check enforces structural traceability with 6 violation types: unmapped-scenario (Example with no test), unmapped-test (test with no Example), misplaced-test (test in wrong file), missing-placeholder (placeholder not in test body), missing-literal (literal not in test body), example-mismatch (Examples table row lacking @example() decorator). Scenario Outline stubs include @given(placeholder_name=strategy) decorators with inferred Hypothesis strategies plus @example() for each Examples table row. Strategy is inferred from column values: all integers → st.integers(), all floats → st.floats(), all booleans → st.booleans(), otherwise st.text(). Override by defining a strategy variable in the test file.

Literal and Placeholder Verification. beehave check extracts quoted strings ("value") and bare numbers (42, -3) from Given/When/Then steps and verifies they appear in the test function body. Per Spec Value Fidelity ([[software-craft/test-design#concepts]]), every literal and placeholder must carry domain meaning in the test — identifiers identify entities, boundaries bound, configurations configure. Never satisfy traceability with noise: assigning to _, stuffing strings into assert messages, or helper functions whose sole purpose is consuming a literal. Placeholder names become Python function parameters and must be valid Python identifiers (not keywords, not builtins like sum, list).

Development Stage Tracking with beehave status. beehave status computes a development stage for each feature file. JSON output (--json) provides per-feature stages, per-scenario status with violation types, summary counts, unmapped_directories, and collisions. Tree output shows Rule → Scenario hierarchy with inline violation codes. --include-unmapped finds orphan test directories. A feature is ok even with mixed stubs and implementations — needs bodies fires only when ALL scenarios are stubs.

Project Overview with beehave list and Cleanup with beehave clean. beehave list shows feature slugs and titles for features with at least one Example. beehave list -v adds: path, scenario count (total + top-level vs rule breakdown), stub/impl counts (e.g., "stubs: 1/2 (1 implemented)"). beehave clean <feature> [--force] removes unmapped test functions from that feature's test directory and reports what was removed.

Feature File Stem and Title Consistency. The feature filename stem MUST match the Feature title slug. Feature "CLI Entrypoint" (slug cli_entrypoint) → file cli_entrypoint.feature → test directory tests/features/cli_entrypoint/. Mismatch causes unmapped directories and test mapping failures. beehave generate runs a project-wide validate_all_titles pre-flight that blocks all generation if ANY feature has bad titles. beehave check <feature> skips global title validation — only beehave check with no argument runs validate_all_titles. When a Gherkin parse error exists, validate_all_titles raises an exception rather than handling gracefully.

Content

Development Stages (beehave status)

Stage Condition
ok All Examples have implemented tests with no violations
broken Feature file has Gherkin parse errors
needs scenarios Has Rules but no Examples
needs tests Has Examples but some lack test functions
needs bodies All Examples have test functions but all bodies are ... stubs
needs fixes Tests exist with bodies but have violations (missing literals, missing placeholders, example mismatch, etc.)

beehave status --json Structure

Key Content
features Per-feature: slug, stage, scenarios[] with title, status, violations[]
summary Counts per stage: ok, broken, needs_scenarios, needs_tests, needs_bodies, needs_fixes
unmapped_directories Test directories with no matching feature file
collisions Test function names appearing in multiple feature test dirs

beehave list -v Output

Field Description
slug Feature title slugified
title Feature title
path Path to the .feature file
scenarios Total scenario count
top_level_scenarios Scenarios not under a Rule
rules Count of Rule blocks
rule_scenarios Scenarios under Rules
stubs Stub count (including ... body count)
implemented Implemented scenario count

Test File Layout

pytest-beehave organizes tests as: Feature title → directory, Rule → test file, Example/Scenario Outline → function name. Test files are placed in tests/features/<feature_slug>/<rule_slug>_test.py.

Related

  • [[requirements/gherkin]]: Example format, title conventions, Scenario Outline syntax, placeholder and literal rules
  • [[software-craft/source-stubs]]: typed source stubs and creation order
  • [[software-craft/test-design]]: semantic depth and test location convention
  • [[software-craft/external-fixtures]]: real data fixtures for external adapter mocking