Generate realistic synthetic security logs for cybersecurity threat hunting training and research.
For background on the project and why we built it, read our announcement: Introducing EvidenceForge: synthetic security logs that don't look (as) fake.
EvidenceForge creates multi-format security log datasets from YAML scenario definitions. You describe an environment—users, systems, network topology, and normal activity—and an optional attack storyline. EvidenceForge then generates temporally consistent evidence across all selected formats, complete with cross-referenced identities, sessions, processes, timestamps, and network connections.
Every generated bundle includes human-readable GROUND_TRUTH.md and machine-readable
GROUND_TRUTH.json answer keys. Attack scenarios document what happened, when, and where, while
baseline-only scenarios explicitly state that no malicious events were generated.
- Guided scenario authoring — Agent skills turn exercise ideas into validated scenario definitions, environment briefings, reusable packs, and configuration changes.
- Multi-source evidence generation — Produce Windows, Linux, EDR, network, IDS, firewall, web, proxy, and email evidence from one scenario.
- Baseline and storyline modeling — Combine ordinary user and system activity, benign red herrings, and typed attack events in the same dataset.
- Repeatable generation at scale — Deterministic seeds, resource forecasts, progress reporting, and resumable checkpoints support complex simulations and huge datasets.
- Validation and quality measurement — Catch schema, cross-reference, topology, and capacity problems before generation, then evaluate the resulting evidence across four quality pillars.
- Reusable environment modeling — Split scenarios with YAML includes, compose versioned industry or organization packs, and apply project-local configuration overlays.
Most synthetic log generators create independent rows or replay isolated templates. EvidenceForge models activities first, then renders the evidence those activities would leave across different systems and sensors.
- Correlated evidence, not independent rows. A logon, process, file operation, or connection retains the same identities and relationships everywhere it is observed, enabling realistic pivots between endpoint, identity, network, and application sources.
- Causal activity, not keyword matching. DNS lookups precede connections, Kerberos tickets precede domain logons, and lifecycle endings follow their beginnings. Required supporting evidence is generated automatically instead of being hand-authored as disconnected events.
- Behavioral and temporal realism. Bursty user activity, periodic system traffic with jitter, day-of-week variation, role-aware services, and benign anomalies create the texture analysts expect from real environments.
- Observation-aware output. Sensor placement, network direction, collection profiles, source clocks, and coherent visibility gaps determine what each source can actually observe.
- Source-native evidence. Each output uses the identities, fields, ordering, and lifecycle conventions of the source it represents rather than projecting one generic event schema into every format.
- Creative authoring, deterministic rendering. Agent skills help research and design scenarios, while the generation engine makes no LLM calls. The same scenario, seed, formats, and version reproduce the same dataset without API costs or model variability.
-
Greater realism across sources. Endpoint, identity, network, application, and IDS evidence now agrees more closely on actors, credentials, processes, timing, network sessions, artifacts, and lifecycle boundaries. Analysts can follow investigative pivots with fewer synthetic contradictions and more source-native behavior.
-
Full cross-platform SMB2/3 activity. Model stateful file-share activity across Windows and Linux clients and Windows or Samba servers, including storage topology, authentication, mappings, mounts, access controls, persistent sessions, and file operations. Correlated Windows Security, EDR, Samba audit, network traffic, and Zeek
smb_mapping,smb_files, andfileslogs provide complete SMB investigations instead of inferred port 445 activity. -
Resumable large-scale generation. Automatic incremental checkpoints, graceful suspension, integrity verification, and compatibility-aware recovery make long or multi-week runs easier to operate. Interrupted jobs can resume safely without starting over, while improved progress reporting, resource forecasting, and performance make large datasets more practical.
-
Reusable environment packs. Industry packs capture sector-specific applications, roles, traffic, and activity patterns, while organization packs define a consistent environment, including its identities, assets, services, storage, and background noise. Teams can share these packs, and reusing an organization pack across scenarios makes the resulting datasets look like different incidents collected from the same real environment.
-
Composable Scenario 2.0 environments. Scenarios can combine reusable packs with scenario-specific content and nested YAML includes. Users can separate stable organizational context from individual storylines, customize only what an exercise requires, and avoid rebuilding the same environment for every dataset.
-
Self-describing output bundles. Every generated bundle includes its authoritative resolved scenario and generation manifest, recording the effective composition, formats, seed, digests, and provenance. Bundles can be evaluated without separately locating the original scenario and retain the information needed to understand or reproduce the run.
-
Faster, clearer scenario authoring. Focused
eforge schemacommands provide exact installed-version field definitions and minimal examples, while runtime inventories expose valid roles, personas, formats, and IDS signatures. Grouped validation diagnostics and dedicated agent skills help authors find and repair the relevant object without navigating the entire scenario schema. -
Earlier detection of invisible or impossible behavior. Validation now considers deployed sources, host roles, sensor placement, observation settings, collection windows, and selected output formats before generation begins. Blocking evidence gaps fail early, while intentionally valid but entirely invisible activity receives an actionable warning before users commit time and resources to a run.
See the complete changelog for detailed release history.
| Format | Description |
|---|---|
| Windows Security Events | 30 event IDs covering authentication, process activity, Kerberos, persistence, account and group management, permitted connections, and log clearing |
| Windows Sysmon | Events 1, 3, 5, 7, 8, 10, 11, 12, 13, and 22 for process, network, module, injection, file, registry, and DNS activity |
| Zeek (16 log types) | conn, dhcp, dns, files, http, ntp, ocsp, packet_filter, pe, reporter, smb_files, smb_mapping, smtp, ssl, weird, and x509 |
| eCAR | Simulated EDR/XDR telemetry for processes, files, flows, registry, modules, threads, user sessions, and services |
| Linux syslog | Authentication, session, service, package, scheduler, maintenance, firewall, Samba, and other role-aware system activity |
| Bash history | Per-user timestamped command history |
| Snort/Suricata alerts | Fast-format IDS alerts with sensor-aware filtering and correlation to network evidence |
| Cisco ASA | Connection, teardown, deny, NAT, and threat-detection syslog from modeled firewall control points |
| Web access | Apache/Nginx combined text or Splunk-compatible JSON, depending on the output target |
| HTTP proxy | Extended Apache/Nginx combined text, SOF-ELK®-compatible combined text, or Splunk-compatible JSON, depending on the output target |
The default target uses SIEM-neutral output. --target sof-elk produces layouts and source-native
variants suitable for SOF-ELK, including Snare Windows events and year-partitioned RFC3164
syslog. --target splunk produces Splunk-friendly Windows event streams and JSON variants for web
and proxy access logs. Formats whose representation does not need to change remain identical
across targets.
See the Evidence Formats Reference for field-level details and the Output Target Ingest Guides for target-specific ingestion and parser support.
# Install EvidenceForge from the source checkout
git clone https://github.com/Cisco-Talos/EvidenceForge.git
cd EvidenceForge
uv sync
# Install skills. You can choose either project- or user-level skills, or both
# Install the project-local skills for Claude Code and ChatGPT/Codex
# (for the current directory/project only)
uv run eforge install-skills
# Install the user-level skills for Claude Code and ChatGPT/Codex
# (for all user projects)
uv run eforge install-skills --globalIn Claude Code, create a new exercise or try the bundled branch-office scenario:
/eforge scenario
/eforge generate scenarios/branch-office-example/scenario.yaml to ./output
/eforge evaluate ./output
In ChatGPT or Codex, use the corresponding eforge-scenario, eforge-generate, and
eforge-evaluate skills.
Checkpoint-enabled runs can be inspected, stopped safely after the current simulated hour, and resumed from another terminal:
uv run eforge checkpoint status ./output
uv run eforge checkpoint verify ./output
uv run eforge checkpoint suspend ./output
uv run eforge generate --output ./output --resumeEvidenceForge provides skills for the creative and interactive parts of the workflow. They guide scenario and pack authoring, invoke the deterministic CLI when appropriate, interpret results, and help repair problems without adding LLM calls to generation itself.
| Workflow | Claude Code | ChatGPT/Codex | Purpose |
|---|---|---|---|
| Scenario authoring | /eforge scenario |
eforge-scenario |
Create or revise a validated exercise and its environment briefing |
| Scenario validation | /eforge validate |
eforge-validate |
Explain validation failures and repair authored scenarios when requested |
| Log generation | /eforge generate |
eforge-generate |
Generate, monitor, verify, and troubleshoot an existing scenario |
| Quality evaluation | /eforge evaluate |
eforge-evaluate |
Score generated evidence, interpret results, and review realism |
| Pack discovery and lifecycle | /eforge pack |
eforge-pack |
Find, inspect, validate, initialize, and copy reusable packs |
| Industry-pack authoring | /eforge industry-pack |
eforge-industry-pack |
Create reusable sector-specific personas, applications, traffic, and storage vocabulary |
| Organization-pack authoring | /eforge organization-pack |
eforge-organization-pack |
Create reusable users, systems, topology, services, and baseline activity |
| Pack releases | /eforge pack-release |
eforge-pack-release |
Build, inspect, import, hydrate, and verify portable .efpack releases |
| Configuration | /eforge config |
eforge-config |
Inspect or tailor project-local personas, applications, traffic, and other generator data |
By default, uv run eforge install-skills installs both integrations for the current project under
.claude/commands/eforge/ and .agents/skills/eforge-*. Use --global for user-wide installation,
or select one integration with --agent claude or --agent chatgpt; --agent codex remains an
alias for --agent chatgpt.
For scripted or non-interactive use:
| Command | Description |
|---|---|
eforge generate <scenario.yaml> -o <dir> [--seed N] |
Forecast resources, then generate logs with 24-hour checkpoints; --seed overrides the scenario seed |
eforge checkpoint status <bundle-root> [--verbose|--json] |
Thoroughly inspect recovery health, compatibility, cursor, and managed storage without resuming |
eforge checkpoint verify <bundle-root> [--verbose|--json] |
Read-only full hydration with phased progress and behavior/runtime drift diagnostics |
eforge checkpoint suspend <bundle-root> |
Ask an active checkpoint-enabled generator to stop safely after its current simulated hour |
eforge validate <scenario.yaml> |
Validate schema and cross-references, and always print a machine-aware memory and disk forecast |
eforge resolve <scenario.yaml> -o <resolved.yaml> [--explain-composition] |
Compile an authoritative, self-contained scenario without generating logs |
eforge pack <command> |
Discover, author, lock, validate, package, inspect, import, or hydrate industry and organization packs |
eforge eval <output_dir> [-s <scenario.yaml>] [--allow-large-evaluation] |
Evaluate quality; new bundles use their adjacent resolved scenario, while legacy bundles require --scenario |
eforge info [field] |
Show installation info, config paths, and data inventories. Pass a dot-path field for a specific value (e.g., eforge info personas). Use --fields to list available fields, --json for machine output. |
eforge schema <selector> [--json] |
Show one focused installed-version authored-scenario contract, such as environment.network_identities or event.email_read. |
eforge validate-config |
Validate config files for cross-reference integrity. Use --json for machine output. |
eforge install-skills [--agent all|claude|chatgpt|codex] [--global] |
Install project-local or user-wide agent skills; defaults to all agents (codex aliases chatgpt) |
eforge version |
Show version |
Useful generate flags include --verbose / --debug, --formats / -F,
--target default|sof-elk|splunk, --resume, --overwrite, and --checkpoint-hours N. The
default checkpoint cadence is 24 simulated hours; 0 disables new checkpoints. validate accepts
the same checkpoint-cadence option so its resource forecast reflects the intended run.
Resume defaults to --resume-policy compatible. It attempts environment drift such as Python,
dependency, OS, and architecture changes, while preserving hard integrity, state-schema, immutable
run-input, and fresh OOB-authorization boundaries. Material or unknown EvidenceForge behavior
changes require interactive confirmation (default no) or explicit --resume-policy attempt after
read-only verification; exact policy retains the byte-equivalent fingerprint requirement.
See Generation Checkpoints and Resume for recovery and filesystem-safety details, and the Output Target Ingest Guides for target-specific layouts and parser support.
All commands accept --help and -h for usage information.
EvidenceForge uses a large data-driven configuration catalog for DNS, applications, personas,
traffic profiles, source behavior, timing, and more. Customize it through a project-local overlay
at .eforge/config/; project changes remain separate from the installed defaults and survive
package upgrades.
The recommended approach is the agent skill (/eforge config in Claude Code or eforge-config in
ChatGPT/Codex):
/eforge config add a nurse persona for a healthcare scenario
For the overlay workflow, manual editing, and cross-file dependencies, see Customizing Configuration.
Scenarios can compose exact-version industry or organization packs while still supporting monolithic authoring. Industry packs provide reusable sector-specific behavior and vocabulary; organization packs can provide a concrete environment and baseline activity. The skills are the recommended way to discover, select, author, and release packs.
Bundled industry packs:
financev1.0.0healthcarev1.0.0technologyv1.0.0
Bundled fictional organization packs:
metrolink-specialty-carev1.0.0northstar-healthv1.0.0 and v1.1.0; v1.1.0 adds cross-platform SMB storage
Use /eforge pack or eforge-pack to inspect the available inventory. For the underlying
composition and lifecycle contract, see
Reusable scenario packs.
EvidenceForge can evaluate a generated bundle across four complementary quality pillars:
| Pillar | Weight | What it measures |
|---|---|---|
| Parseability | 30% | Source conformance and format constraints |
| Plausibility | 25% | Values, cross-source agreement, distributions, diversity, and anomaly rates |
| Causality | 25% | Event presence, ordering, authored-intent reconciliation, and investigative pivots |
| Timing | 20% | Attack-chain timing, burstiness, regularity, diurnal patterns, and event rates |
Applicable hard gates must pass; aspirational targets show where quality can improve without turning every shortfall into a failure. Measures that do not apply to a dataset are reported as unavailable rather than receiving an automatic perfect score.
uv run eforge eval ./outputuv sync --all-extras
uv run pytest
uv run ruff check .
uv run ruff format --check .See Contributing for the complete development workflow, extended test tiers, coverage gate, coding conventions, and external-parser validation requirements.
- Scenario Reference — Scenario fields, includes, typed events, and validation rules
- Evidence Formats Reference — Output layout, log types, field details, and known limitations
- Reusable Scenario Packs — Industry and organization pack composition and lifecycle
- Customizing Configuration — Project-local configuration overlays and data catalogs
- Generation Checkpoints and Resume — Safe suspension, recovery, status, storage, and filesystem behavior
- Output Target Ingest Guides — Default, SOF-ELK, and Splunk layouts, parsing, and ingestion
- Adversarial Payload Testing — Safe synthetic payload and callback-testing workflow
- Credential Spillage Modeling — Synthetic credential leakage and evidence-surface behavior
- Configuration Compatibility — Legacy configuration normalization and compatibility rules
- External Parser Validation — SOF-ELK and Splunk validation harnesses
- Architecture — Generation architecture and ownership contracts
- Changelog — Release history
- Contributing, Security, and Code of Conduct — Project contribution and security policies
- Agent Development Conventions — Repository conventions for coding agents
Before opening a pull request, please open an issue describing the problem or proposed change and wait for the approach to be discussed with the maintainers. This helps avoid work on changes that do not fit the project direction; pull requests submitted without prior agreement may be closed. Once an approach is agreed, follow CONTRIBUTING.md for development, testing, and submission requirements.
SOF-ELK® is a registered trademark of Lewes Technology Consulting, LLC. Used with permission.
MIT License - Copyright (c) 2026 Cisco Systems, Inc.