Test Filter: Intelligent Test Selection for Clojure

Table of Contents

Overview
- Key Features
- How It Works
  - Workflow
Installation
Usage
Advanced Features
Architecture
Project Status
Contributing
License
Support
Acknowledgments

Overview

Test Filter is an intelligent test selection tool for Clojure projects that dramatically reduces test execution time by running only tests affected by your code changes. Instead of running your entire test suite on every change, Test Filter analyzes your source code dependencies and git history to determine the minimum set of tests that need to run.

Important

This project source and documentation was completely written by Claude Code. It wrote more than I wanted and I’ve not had time to verify the documentation. Feel free to play with it, but I make ZERO guarantees that anything works as advertised.

Key Features

Dependency-aware: Builds a complete symbol dependency graph using static analysis
Content-hash based change detection: Uses SHA256 hashes of normalized code to detect semantic changes
Formatting-immune: Ignores whitespace, indentation, and docstring changes - only logic changes trigger tests
Git-integrated: Tracks file changes between revisions to identify affected code
Intelligent caching: Incrementally updates analysis with content hashes, avoiding full re-scans
Multiple test frameworks: Supports clojure.test, fulcro-spec, and custom test macros
Integration test handling: Special handling for integration tests with broad dependencies
CLJC support: Properly handles Clojure Common files with reader conditionals
Flexible output: Multiple formats for integration with various test runners

How It Works

Test Filter uses a two-cache architecture to track which tests need to run:

Analysis Cache (.test-filter-cache.edn): Ephemeral snapshot of current codebase
- Contains dependency graph and current content hashes
- Completely overwritten on each analyze! command
- Only used to communicate between CLI steps
Success Cache (.test-filter-success.edn): Persistent baseline of verified code
- Contains content hashes of successfully tested symbols
- Only updated when you explicitly mark tests as verified
- Persists across sessions and builds

Workflow

Analyze: Build dependency graph and generate current content hashes
- Parses all source files with clj-kondo
- Creates SHA256 hashes of normalized function bodies (ignoring docstrings/formatting)
- Writes snapshot to analysis cache
Select Tests: Compare current state vs. verified baseline
- Loads current hashes from analysis cache
- Loads verified hashes from success cache
- Identifies symbols where current hash ≠ verified hash
- Walks dependency graph to find affected tests
Run Tests: Execute the selected tests (user action)
Mark Verified: Record that tests passed
- Updates success cache with hashes from current analysis
- Only done after tests successfully pass
- Creates new baseline for future comparisons

Example: Formatting Change (No Tests Triggered)

1. Analyze: generates hash "abc123" for `foo/bar` (docstring + formatting)
2. User reformats docstring
3. Analyze: generates hash "abc123" for `foo/bar` (still same - normalized)
4. Select Tests: current hash = success hash → NO tests needed

Example: Logic Change (Tests Triggered)

1. Success cache has: foo/bar → "abc123" (verified)
2. User changes `foo/bar` logic: (* x 2) becomes (* x 3)
3. Analyze: generates new hash "def456" for `foo/bar`
4. Select Tests: current hash ≠ success hash → change detected
5. Walk Graph: find tests depending on `foo/bar`
   - `baz/qux` uses `foo/bar`
   - `app/handler` uses `baz/qux`
   - `app-test/handler-test` tests `app/handler`
6. Return Selection: {:tests [app-test/handler-test]
                      :changed-symbols #{foo/bar}
                      :changed-hashes {foo/bar "def456"}}
7. User runs tests, they pass
8. Mark Verified: updates success cache with foo/bar → "def456"

Installation

Add to your deps.edn:

{:deps {com.fulcrologic/test-filter {:mvn/version "1.0.0" }}

 ;; optional
 :aliases
 {:cli {:main-opts ["-m" "com.fulcrologic.test-filter.cli"]}}}

Usage

Command Line Interface

Analyze Command

Analyze the current codebase and update the analysis cache:

clojure -M:cli analyze

This command:

Runs clj-kondo analysis on your source code
Builds a complete symbol dependency graph
Generates content hashes for all symbols
Overwrites .test-filter-cache.edn with current state
NOTE: Does NOT update the success cache - that’s done with mark-verified

Select Command

Find tests affected by changes:

# Basic usage
clojure -M:cli select

# With verbose output showing statistics
clojure -M:cli select -v

# Get all tests (ignore changes)
clojure -M:cli select --all

Output Formats

# Fully-qualified test vars (default)
clojure -M:cli select -o vars

# Test namespaces only
clojure -M:cli select -o namespaces

# Kaocha command-line format
clojure -M:cli select -o kaocha

Status Command

Check cache status:

clojure -M:cli status

Shows:

Whether analysis cache exists
Whether success cache exists
Cache file sizes and timestamps
Number of verified symbols (with -v flag)

Mark Verified Command

Mark tests as successfully verified (updates success cache):

# Mark all selected tests as verified
clojure -M:cli mark-verified

# Mark specific tests as verified
clojure -M:cli mark-verified -t my.app.core-test/foo-test

This command:

Updates .test-filter-success.edn with verified symbol hashes
Should only be run after tests pass
Creates the baseline for future test selection

Mark All Verified Command

Initialize the success cache by marking all symbols in the current analysis as verified:

# Mark all symbols in the graph as verified
clojure -M:cli mark-all-verified

This command:

Marks ALL symbols from the analysis cache as verified
Useful for initializing test-filter on an existing large codebase
Requires that you run analyze first
After this, only new changes will trigger tests
Creates the initial baseline without running any tests

Clear Command

Invalidate the caches:

# Clear analysis cache only
clojure -M:cli clear

# Clear both analysis and success caches
clojure -M:cli clear --all

Programmatic API

Use Test Filter from the REPL or your code:

(require '[com.fulcrologic.test-filter.core :as tf])

;; 1. Analyze the codebase (generates current state)
(tf/analyze! :paths ["src/main" "src/test"])

;; 2. Select tests based on changes
(def selection (tf/select-tests :verbose true))

;; The selection contains everything needed for verification
selection
;; => {:tests [my.app-test/foo-test my.app-test/bar-test]
;;     :changed-symbols #{my.app/foo my.app/bar}
;;     :changed-hashes {my.app/foo "abc123..." my.app/bar "def456..."}
;;     :graph {...}
;;     :stats {...}}

;; 3. Show affected tests
(tf/print-tests (:tests selection) :format :namespaces)

;; 4. Run the tests (external - use your test runner)
;; ... run tests ...

;; 5. Mark verified after tests pass
(tf/mark-verified! selection)  ; Mark all selected tests
;; or
(tf/mark-verified! selection [my.app-test/foo-test])  ; Mark subset

;; Alternative: Initialize on existing codebase (skip testing)
(def graph (tf/analyze! :paths ["src/main" "src/test"]))
(tf/mark-all-verified! graph)  ; Mark everything as verified
;; => 141  (returns count of verified symbols)

Integration with Test Runners

Kaocha

# Run only affected tests with Kaocha
clojure -M:cli select -o kaocha | xargs clojure -M:kaocha

Cognitect test-runner

# Get affected test namespaces
TESTS=$(clojure -M:cli select -o namespaces)

# Run with test-runner
if [ -n "$TESTS" ]; then
  clojure -M:test -n $TESTS
fi

Initializing on a Large Existing Codebase

For large projects where you want to start using test-filter immediately:

# 1. Analyze the entire codebase
clojure -M:cli analyze

# 2. Mark everything as verified (creates initial baseline)
clojure -M:cli mark-all-verified

# 3. Now only new changes will trigger tests
clojure -M:cli select -v
# => No tests need to be run.

# 4. Make a change to any file
# ... edit file ...

# 5. Analyze and select - only affected tests will run
clojure -M:cli analyze
clojure -M:cli select -v
# => Only tests affected by your change

Typical Workflow

# 1. Analyze current codebase
clojure -M:cli analyze

# 2. Make code changes
# ... edit files ...

# 3. Analyze again to capture changes
clojure -M:cli analyze

# 4. Select affected tests
clojure -M:cli select -v

# 5. Run only affected tests
clojure -M:cli select -o kaocha | xargs clojure -M:kaocha

# 6. If tests pass, mark as verified
clojure -M:cli mark-verified

# 7. Continue development
# ... edit more files ...

# 8. Next iteration - analyze and select again
clojure -M:cli analyze
clojure -M:cli select -v
# ... only new changes will trigger tests ...

CI/CD Integration

#!/bin/bash
# In your CI pipeline

# Analyze current PR branch
clojure -M:cli analyze

# Select affected tests (compares against success cache from main)
TESTS=$(clojure -M:cli select -o namespaces)

if [ -n "$TESTS" ]; then
  echo "Running affected tests: $TESTS"
  clojure -M:kaocha --focus $TESTS

  # If tests pass, update success cache
  if [ $? -eq 0 ]; then
    clojure -M:cli mark-verified
    # Commit updated success cache to track verified state
    git add .test-filter-success.edn
    git commit -m "Update verified test baseline"
  fi
else
  echo "No tests affected by changes"
fi

Note	The `.test-filter-success.edn` file should be committed to your repository to track the verified baseline across CI runs. The `.test-filter-cache.edn` file is ephemeral and should be in `.gitignore`.

Advanced Features

Macro-Based Test Detection

Test Filter supports test frameworks that use macros instead of deftest:

(ns my-app.spec-test
  (:require [fulcro-spec.core :refer [specification assertions]]))

(specification "User registration"
  (assertions
    "creates a new user"
    (register-user {:name "Alice"}) => {:id 1 :name "Alice"}))

Detected test frameworks:

fulcro-spec.core/specification
Custom macros (configurable)

Integration Test Handling

Integration tests often have broad dependencies that are difficult to track with static analysis. Test Filter provides flexible options for handling them.

Detection Methods

Test Filter identifies integration tests in two ways:

Namespace Pattern: Any test in a namespace containing .integration. as a segment
Explicit Metadata: Tests marked with :integration true metadata

;; Method 1: Namespace pattern (automatic detection)
(ns my-app.integration.api-test
  (:require [clojure.test :refer [deftest is]]
            [my-app.system :as system]))

(deftest test-user-api
  (let [sys (system/start)]
    ;; Integration test
    (is (= 200 (:status (api-call sys))))))

;; Method 2: Explicit metadata (any namespace)
(deftest ^{:integration true} test-full-workflow
  (let [sys (system/start)]
    ;; Integration test in a regular namespace
    (is (= :success (run-full-workflow sys)))))

Behavior Modes

Integration tests behave differently based on their metadata:

Conservative Mode (default): Run the test whenever uncertain about dependencies
- Used when test is marked as integration but has no :test-targets
- Safest option: ensures integration tests run when needed
- May run more often than strictly necessary
Targeted Mode: Run only when specific symbols change
- Used when test has :test-targets metadata
- Precise control over when integration tests run
- Reduces unnecessary test execution

Using `:test-targets` Metadata

The :test-targets (or singular :test-target) metadata allows you to specify exactly which symbols an integration test depends on. The test will only run if one of those target symbols changes.

IMPORTANT: The :test-targets metadata works independently of actual function calls in your test code. You don’t need to call the target functions for the metadata to work - Test Filter uses the metadata alone to determine dependencies. This is powerful for integration tests where dependencies might be:

Dynamic or indirect (loaded at runtime)
Hidden behind macros or protocols
External system interactions (databases, APIs)
Not statically analyzable by clj-kondo

Example: A test can have {:test-targets my.app/process-payment} without ever calling process-payment directly, and it will still run when process-payment changes.

Supported Formats

The metadata accepts multiple formats:

;; Single symbol (fully-qualified)
(deftest ^{:test-target my.app/process-order} test-order-processing
  (is (= :processed (:status (process-order {:id 123})))))

;; Single symbol (syntax-quoted with alias)
(ns my-app.integration.orders-test
  (:require [my.app :as app]))

(deftest ^{:test-targets `app/process-order} test-order-processing
  (is (= :processed (:status (app/process-order {:id 123})))))

;; Multiple symbols as a set
(deftest ^{:test-targets #{my.app/send-notification
                           my.app/handle-refund}}
  test-notification-and-refund
  (is (= :sent (send-notification "customer-1" "message")))
  (is (= :refunded (handle-refund 456 25.50))))

;; Multiple symbols as a vector (normalized to set)
(deftest ^{:test-targets [my.app/foo my.app/bar]} test-both
  (is (= :ok (foo)))
  (is (= :ok (bar))))

With fulcro-spec

The specification macro also accepts metadata:

(ns my-app.integration.payment-test
  (:require [my.app :as app]
            [fulcro-spec.core :refer [specification assertions]]))

;; Single target
(specification {:test-targets `app/validate-payment}
  "Payment Validation"
  (assertions
    "validates valid payment"
    (app/validate-payment {:card "1234" :amount 50}) => true))

;; Multiple targets
(specification {:test-targets #{my.app/process-order
                                my.app/send-confirmation}}
  "Order Processing Flow"
  (assertions
    "processes and confirms"
    (app/process-order {:id 123}) => {:status :processed}
    (app/send-confirmation 123) => {:sent true}))

;; Singular form also works
(specification {:test-target my.app/validate-payment}
  "Payment Validation - Singular"
  (assertions
    (app/validate-payment {:card "1234" :amount 50}) => true))

Combining with `:integration`

You can combine both metadata fields:

;; Explicit integration marker + specific targets
(deftest ^{:integration true
           :test-targets #{my.app/critical-fn}}
  test-critical-integration
  (is (= :success (critical-fn))))

Decision Tree

Test Filter uses this logic to determine when integration tests run:

Has :test-targets or :test-target metadata?
- YES → Run only if one of the target symbols changed
- NO → Continue to step 2
Is marked :integration? or in .integration. namespace?
- YES → Run conservatively (always run when any code changes)
- NO → Use normal transitive dependency analysis

Examples

;; Example 1: Metadata-only targeting (no function calls required)
(ns my-app.integration.payment-test
  (:require [clojure.test :refer [deftest is]]
            [my-app.payment :as payment]))

(deftest ^{:test-targets #{my.app.payment/process-payment
                           my.app.payment/validate-card}}
  test-payment-flow-end-to-end
  ;; This test doesn't directly call process-payment or validate-card
  ;; They're called indirectly through the full system
  ;; But it will run when either of them changes
  (is (= :success (start-system-and-test-payment))))

;; Example 2: Conservative integration test (always runs)
(ns my-app.integration.full-system-test
  (:require [clojure.test :refer [deftest is]]))

(deftest test-entire-system
  ;; No :test-targets specified
  ;; Runs whenever ANY change is detected
  (is (= :success (start-and-test-full-system))))

;; Example 3: Targeted integration test (runs only when order functions change)
(ns my-app.integration.orders-test
  (:require [clojure.test :refer [deftest is]]
            [my.app.orders :as orders]))

(deftest ^{:test-targets #{my.app.orders/create-order
                           my.app.orders/validate-order}}
  test-order-workflow
  ;; Only runs if create-order or validate-order changed
  (is (= :valid (orders/validate-order {:id 1})))
  (is (= :created (orders/create-order {:id 1}))))

;; Example 4: Mix of conservative and targeted
(ns my-app.integration.mixed-test
  (:require [clojure.test :refer [deftest is]]
            [my.app.core :as core]))

;; This one always runs (conservative)
(deftest test-critical-path
  (is (= :ok (core/critical-operation))))

;; This one only runs when payment-fn changes
(deftest ^{:test-targets my.app.payment/process-payment}
  test-payment-integration
  (is (= :processed (core/process-payment {:amount 100}))))

Best Practices

Start Conservative: Use namespace pattern or :integration metadata initially
Add Targets Gradually: As you understand dependencies, add :test-targets to reduce test time
Be Explicit: Prefer fully-qualified symbols in :test-targets for clarity
Document Intent: Use comments to explain why specific targets were chosen
Regular Review: Periodically verify that targeted tests still cover the right dependencies

CLJC File Support

Test Filter properly handles Clojure Common (.cljc) files with reader conditionals:

(ns my-app.utils
  #?(:clj (:import [java.nio.file Paths])))

(defn normalize-path [path]
  #?(:clj  (-> (Paths/get path (into-array String []))
               (.normalize)
               (.toString))
     :cljs (.normalize js/path path)))

Analyzes only the :clj side of CLJC files
Ignores pure .cljs files
Tracks dependencies correctly across platforms

Architecture

Content-Hash Based Change Detection

Test Filter uses a sophisticated approach to detect which code changes actually require testing:

Parse with EDN reader: Uses clojure.tools.reader to parse code as data structures
Strip docstrings: Removes documentation strings from function definitions
Normalize formatting: Uses pr-str to get consistent formatting regardless of whitespace/indentation
Generate SHA256 hash: Creates a unique hash representing the function’s logic
Cache hashes: Stores hashes alongside the dependency graph
Compare on change: When files change, re-analyze and compare new hashes to cached ones
Identify real changes: Only symbols with different hashes are considered "changed"

This means:

✓ Adding/changing docstrings doesn’t trigger tests
✓ Reformatting code doesn’t trigger tests
✓ Reordering functions doesn’t trigger tests
✓ Adding comments doesn’t trigger tests
✓ Only actual logic changes trigger the appropriate tests

Components

Component	Description
Analyzer (`analyzer.clj`)	Uses clj-kondo to extract var definitions, namespace definitions, and usage relationships
Graph (`graph.clj`)	Builds directed dependency graph using Loom library; provides traversal operations
Content (`content.clj`)	Extracts function bodies, normalizes them (strips docstrings/whitespace), and generates SHA256 hashes for semantic comparison
Git (`git.clj`)	Wraps git commands to detect which files changed between revisions
Cache (`cache.clj`)	Persists graph and content hashes to EDN format; handles incremental updates and cache invalidation
Core (`core.clj`)	Main test selection algorithm; coordinates all components
CLI (`cli.clj`)	Command-line interface with multiple output formats

Data Model

Symbol Node

{:symbol 'my.ns/foo
 :type :var
 :file "src/my/ns.clj"
 :line 42
 :end-line 47
 :defined-by 'defn
 :metadata {:private false
            :macro false
            :test false}}

Dependency Edge

{:from 'my.ns/foo
 :to 'other.ns/bar
 :context 'my.ns/foo}

Analysis Cache Structure

;; .test-filter-cache.edn (ephemeral, not committed to git)
{:analyzed-at "2025-01-09T10:30:00Z"
 :paths ["src/main" "src/test"]
 :nodes {symbol -> node-data}
 :edges [{:from :to :context}]
 :files {"src/my/ns.clj" {:symbols [...]}}
 :content-hashes {my.ns/foo "sha256..."
                  my.ns/bar "sha256..."}}

Success Cache Structure

;; .test-filter-success.edn (committed to git)
{my.ns/foo "sha256-of-verified-version..."
 my.ns/bar "sha256-of-verified-version..."
 my.ns-test/foo-test "sha256-of-test-when-it-passed..."}

Selection Object

;; Returned by select-tests
{:tests [my.ns-test/foo-test my.ns-test/bar-test]
 :changed-symbols #{my.ns/foo my.ns/baz}
 :changed-hashes {my.ns/foo "new-sha256..."
                  my.ns/baz "new-sha256..."}
 :trace {my.ns-test/foo-test {my.ns/foo [my.ns-test/foo-test my.ns/foo]}}
 :graph {...}  ; Full dependency graph
 :stats {:total-tests 12
         :selected-tests 2
         :changed-symbols 2
         :selection-rate "16.7%"}}

Project Status

✅ Completed Features

All planned phases (1-9) are complete:

✓ Foundation and project setup
✓ clj-kondo integration
✓ Graph operations with Loom
✓ Git integration and change detection
✓ Cache persistence and incremental updates
✓ Test selection algorithm
✓ Command-line interface
✓ Real-world testing and bug fixes
✓ Macro-based test detection (fulcro-spec)
✓ Integration test handling
✓ CLJC file support
✓ Content-hash based change detection (ignores formatting/docstrings)

Known Limitations

Testing scope: Needs validation on larger codebases (>100k LOC)
Dynamic requires: Conservative handling (assumes dependency)
Circular dependencies: Not yet optimized
ClojureScript: Not supported (by design, focuses on CLJ/CLJC)

Future Enhancements

Support for test.check generative tests
Parallel test execution planning
Coverage-based refinement
Watch mode for continuous testing
Configuration file for custom patterns

Contributing

Contributions are welcome! Please:

Fork the repository
Create a feature branch
Add tests for new functionality
Ensure all tests pass
Submit a pull request

License

MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Support

For issues, questions, or suggestions:

Open an issue on GitHub
Check existing documentation in PLAN.md and STATUS.md
Review code examples in namespace docstrings

Acknowledgments

Built with:

clj-kondo - Static analysis
Loom - Graph algorithms
tools.reader - EDN parsing for content hashing
Clojure - The language that makes this possible

Name		Name	Last commit message	Last commit date
Latest commit History 45 Commits
.circleci		.circleci
docs/ai		docs/ai
src		src
.gitignore		.gitignore
.mcp.json		.mcp.json
.test-filter-success.edn		.test-filter-success.edn
CLAUDE.md		CLAUDE.md
PLAN.md		PLAN.md
README-gitlab.adoc		README-gitlab.adoc
README.adoc		README.adoc
STATUS.md		STATUS.md
deps.edn		deps.edn
package.json		package.json
pom.xml		pom.xml
tests.edn		tests.edn

Folders and files

Latest commit

History

Repository files navigation

Test Filter: Intelligent Test Selection for Clojure

Overview

Key Features

How It Works

Workflow

Installation

Usage

Command Line Interface

Analyze Command

Select Command

Output Formats

Status Command

Mark Verified Command

Mark All Verified Command

Clear Command

Programmatic API

Integration with Test Runners

Kaocha

Cognitect test-runner

Initializing on a Large Existing Codebase

Typical Workflow

CI/CD Integration

Advanced Features

Macro-Based Test Detection

Integration Test Handling

Detection Methods

Behavior Modes

Using :test-targets Metadata

Supported Formats

With fulcro-spec

Combining with :integration

Decision Tree

Examples

Best Practices

CLJC File Support

Architecture

Content-Hash Based Change Detection

Components

Data Model

Symbol Node

Dependency Edge

Analysis Cache Structure

Success Cache Structure

Selection Object

Project Status

✅ Completed Features

Known Limitations

Future Enhancements

Contributing

License

Support

Acknowledgments

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Using `:test-targets` Metadata

Combining with `:integration`

Packages