Security Scanning - USPTO Final Petition Decisions MCP

This document describes the comprehensive security scanning infrastructure for the USPTO Final Petition Decisions (FPD) MCP, designed to prevent accidental exposure of sensitive credentials and protect against sophisticated prompt injection attacks including Unicode steganography.

Overview

The FPD MCP uses detect-secrets and custom prompt injection detection to scan for accidentally committed secrets and malicious prompt patterns. This provides three layers of protection:

Pre-commit hooks - Scans files locally before they're committed to git
GitHub Actions CI/CD - Scans all code pushed to GitHub on every push/PR
Unicode Steganography Detection - Protects against emoji-based prompt injection attacks

🚨 NEW: Unicode Steganography Protection

Following the discovery of Unicode Variation Selector attacks (as detailed in the Repello.ai article), the FPD MCP now includes advanced detection for:

Emoji-based steganography - Malicious instructions hidden in innocent text using invisible Unicode characters
Variation Selector encoding - Binary patterns using VS0 (U+FE00) and VS1 (U+FE01) to hide messages
High invisible character ratios - Suspicious concentrations of invisible Unicode characters
Binary pattern detection - Recognition of steganographic encoding schemes

Example Attack: Text like "Hello!" can contain hidden malicious instructions encoded in invisible Variation Selectors that are completely undetectable to normal security scanning.

What Gets Scanned

1. Secret Detection (20+ types)

Uses detect-secrets to find accidentally committed credentials

2. Prompt Injection Detection (80+ patterns)

Scans for malicious prompt patterns targeting Final Petition Decisions system, including Unicode steganography attacks

Secret Types Detected (20+ types)

The scanner detects the following types of secrets:

API Keys & Authentication:

AWS/Azure/GCP cloud credentials
GitHub/GitLab tokens
Mistral API keys (used for OCR)
OpenAI API keys
JWT tokens
Basic authentication credentials

Infrastructure Secrets:

Private SSH keys
SSL/TLS certificates
Database passwords
NPM tokens
PyPI tokens

High-Entropy Strings:

Base64-encoded secrets (limit: 4.5 entropy)
Hexadecimal secrets (limit: 3.0 entropy)

Communication Services:

Slack tokens
Discord bot tokens
Telegram bot tokens
Twilio API keys
SendGrid API keys
Mailchimp API keys

Prompt Injection Attack Patterns (70+ patterns)

Attack Categories Detected:

Instruction override attempts ("ignore previous instructions")
System prompt extraction ("show me your instructions")
AI behavior manipulation ("you are now a different AI")
Output format manipulation ("encode in hex")
Social engineering ("we became friends")

Final Petition Decisions Specific:

Petition data extraction ("extract all petition numbers")
Decision reasoning disclosure ("reveal director decisions")
USPTO API bypass attempts ("ignore API restrictions")
CFR rule manipulation ("override 37 CFR requirements")
Petitioner information exfiltration ("dump petitioner data")

🚨 Unicode Steganography Attacks (NEW):

Variation Selector steganography - Malicious instructions hidden using VS0/VS1 binary encoding
Invisible character injection - Zero-width spaces and other invisible Unicode characters
Emoji-based hiding - Messages concealed in emoji variation selectors
Binary steganography patterns - Systematic use of invisible characters for encoding
High steganography ratios - Suspicious concentrations of invisible vs visible characters

Prompt Injection Baseline System

The prompt injection scanner uses a baseline system to track known findings and only flag NEW patterns that are not in the baseline. This solves the problem of false positives from legitimate code and documentation while maintaining protection against malicious prompt injection attacks.

Why a Baseline System?

The scanner was flagging legitimate patterns as "attacks":

Variable Names:

prompt = """Some prompt text"""        # Flagged as attack
system = "some value"               # Flagged as attack
class PromptTemplate:               # Flagged as attack
def system_init(self):              # Flagged as attack

Documentation:

Instructions for using the system  # Flagged as attack
System health check                   # Flagged as attack

Legitimate Unicode Characters:

Emoji variation selectors in documentation
Unicode characters in prompt templates
Special characters in logging

The baseline system allows legitimate code without compromising security - only NEW findings not in baseline cause failures.

How It Works

Baseline File: .prompt_injections.baseline stores known findings
Fingerprinting: Each finding gets a unique SHA256 hash fingerprint
Comparison: Scanner checks if each finding is in the baseline
Exit Codes:
- 0 - No NEW findings (all findings in baseline)
- 1 - NEW findings detected (not in baseline)
- 2 - Error occurred

Baseline File Format

{
  "src/fpd_mcp/shared/health_check.py": {
    "abc123def456": {
      "line": 2,
      "match": "system"
    },
    "def789ghi012": {
      "line": 4,
      "match": "system"
    }
  },
  "src/fpd_mcp/prompts/art_unit_quality_assessment.py": {
    "ghi345jkl678": {
      "line": 6,
      "match": "prompt"
    }
  }
}

Baseline Commands

Option	Purpose
`--baseline`	Use existing baseline (only NEW findings fail)
`--update-baseline`	Add new findings to baseline
`--force-baseline`	Create new baseline (overwrite existing)
`--verbose, -v`	Show detailed output with full matches
`--quiet, -q`	Only show summary (suppress individual findings)
`--include-security-files`	Check security documentation files

Usage Examples

First Run - Create Baseline:

# Scan and create baseline (will NOT fail if findings exist)
uv run python .security/check_prompt_injections.py --update-baseline src/ tests/ *.md *.yml *.yaml *.json *.py

Normal Run - Check Against Baseline:

# Check for NEW findings only
uv run python .security/check_prompt_injections.py --baseline src/ tests/ *.yml *.yaml *.json *.py

Add New Legitimate Finding to Baseline:

# Update baseline to include new legitimate finding
uv run python .security/check_prompt_injections.py --update-baseline src/ tests/ *.md *.yml *.yaml *.json *.py

Force New Baseline:

# Overwrite existing baseline
uv run python .security/check_prompt_injections.py --force-baseline src/ tests/ *.md *.yml *.yaml *.json *.py

When to Update Baseline

DO Update Baseline When:

New legitimate code is flagged
- New variable/function names with "prompt" or "system"
- Documentation with these words
- Legitimate Unicode characters
Approved refactoring changes line numbers
- Code moves (findings appear at new lines)
- File restructuring
Baseline is outdated
- Many findings no longer exist in current code

DON'T Update Baseline When:

Malicious pattern detected - Remove the malicious code instead
You're unsure - Ask for review first
Security-related finding - Even if it looks "innocent", review carefully

GitHub Actions Integration

The workflow automatically handles baselines:

- name: Run prompt injection detection
  run: |
    # Create baseline if it doesn't exist, then check against it
    if [ ! -f .prompt_injections.baseline ]; then
      echo "📋 Creating baseline for first time..."
      uv run python .security/check_prompt_injections.py --update-baseline src/ tests/ *.md *.yml *.yaml *.json *.py
    else
      echo "📋 Using existing baseline..."
      uv run python .security/check_prompt_injections.py --baseline src/ tests/ *.yml *.yaml *.json *.py
    fi

Workflow Behavior:

Scenario	GitHub Actions Result
First run (no baseline)	Creates baseline, passes
Subsequent runs (no NEW findings)	Passes (findings in baseline)
Subsequent runs (NEW findings)	Fails (requires review)
Review finds legitimate false positive	Update baseline locally, push

Managing the Baseline File

Add Baseline to Repository:

git add .prompt_injections.baseline
git commit -m "Add prompt injection baseline"
git push

Reviewing Baseline Changes:

# See what changed in baseline
git diff .prompt_injections.baseline

Regenerating Baseline:

# Regenerate from scratch (re-run full scan)
uv run python .security/check_prompt_injections.py --force-baseline src/ tests/ *.md *.yml *.yaml *.json *.py

# Review and commit
git add .prompt_injections.baseline
git commit -m "Regenerate prompt injection baseline"

Files Excluded from Scanning

The following file patterns are excluded to reduce false positives:

configs/**/*.json - Example configuration files with placeholder keys
*.md - Markdown documentation files (except actual secrets)
package-lock.json - NPM lock files
*.lock - Other lock files (Cargo.lock, poetry.lock, etc.)

Important:

Prompts directory IS scanned - The baseline system allows legitimate prompt code while catching malicious injections
CLAUDE.md is in .gitignore and not scanned
All other files (including test files) are scanned to prevent accidental exposure

Installation

Prerequisites

You need Python 3.11+ and either:

uv (recommended, already used by FPD MCP)
pip (standard Python package manager)

Option 1: Using uv (Recommended)

# Install detect-secrets as a tool
uv tool install detect-secrets

# Verify installation
uv tool run detect-secrets --version

Option 2: Using pip

# Install detect-secrets
pip install detect-secrets

# Verify installation
detect-secrets --version

Install Pre-commit Hooks

# Install pre-commit package
pip install pre-commit

# Install git hooks (run from project root)
cd C:\Users\JohnWalkoe\uspto_fpd_mcp
pre-commit install

# Verify installation
pre-commit --version

Usage

Prompt Injection Detection (Including Unicode Steganography)

🚨 CRITICAL: The scanner now detects Unicode steganography attacks as described in the Repello.ai article.

📋 BASELINE SYSTEM: The scanner uses a baseline file to track known legitimate findings. See the Prompt Injection Baseline System section for complete details.

Manual Scanning:

# Check against existing baseline (only NEW findings fail)
uv run python .security/check_prompt_injections.py --baseline src/ tests/ *.md *.py

# Update baseline with new legitimate findings
uv run python .security/check_prompt_injections.py --update-baseline src/ tests/ *.md *.yml *.yaml *.json *.py

# Scan specific directories for all attack patterns
uv run python .security/check_prompt_injections.py --baseline src/fpd_mcp/

# Include security files in scan (normally excluded)
uv run python .security/check_prompt_injections.py --include-security-files src/ tests/

# Run via pre-commit hook (automatic Unicode steganography detection)
uv run pre-commit run prompt-injection-check --all-files

# Verbose output (shows full attack patterns detected)
uv run python .security/check_prompt_injections.py --verbose src/ tests/

# Quiet mode (only shows summary and Unicode steganography alerts)
uv run python .security/check_prompt_injections.py --quiet src/ tests/

🔍 Unicode Steganography Examples:

# Test with known steganography patterns
echo 'Hello!' > test_file.txt
# If test_file.txt contains hidden Variation Selectors, scanner will detect them

# Check specific file for invisible characters
uv run python .security/check_prompt_injections.py --baseline suspicious_file.md

# Emergency scan when Unicode steganography is suspected
uv run python .security/check_prompt_injections.py --verbose --include-security-files .

Pre-commit Hooks (Local Development)

Once installed, pre-commit hooks run automatically on git commit:

# Make changes to files
git add src/fpd_mcp/config/settings.py

# Commit (hooks run automatically)
git commit -m "Update settings"

# If secrets detected:
# - Commit is blocked
# - Files with secrets are listed
# - Fix the issues and try again

Bypass hooks (NOT RECOMMENDED):

# Only use in emergencies (requires justification in commit message)
git commit --no-verify -m "Emergency fix - bypassing hooks"

Manual Scanning

Run secret scanning manually anytime:

# Scan all files against baseline (uv method)
uv tool run detect-secrets scan --baseline .secrets.baseline

# Scan all files against baseline (pip method)
detect-secrets scan --baseline .secrets.baseline

# Scan specific file
uv tool run detect-secrets scan --baseline .secrets.baseline src/fpd_mcp/main.py

# Scan git history (last 100 commits)
git log --all --pretty=format: -p -100 | uv tool run detect-secrets scan --stdin

Updating the Baseline

When you add new test placeholders or example configurations, update the baseline:

# Update baseline with new findings (uv method)
uv tool run detect-secrets scan --baseline .secrets.baseline

# Update baseline with new findings (pip method)
detect-secrets scan --baseline .secrets.baseline

# Review changes
git diff .secrets.baseline

# Commit updated baseline
git add .secrets.baseline
git commit -m "Update secrets baseline for new test placeholders"

When to update the baseline:

Adding new test files with placeholder keys like "test_key_for_unit_tests"
Adding example configuration files with dummy credentials
After verifying a detection is a false positive (e.g., UUID, documentation example)

When NOT to update the baseline:

When a real secret is detected (remove the secret instead!)
When you're unsure if it's a real secret (ask for review first)

GitHub Actions Integration

The FPD MCP automatically scans code on every push and pull request to main, master, or develop branches.

Workflow Configuration

File: .github/workflows/secret-scan.yaml

Triggers:

Push to main, master, or develop
Pull requests targeting those branches

What it does:

Checks out code with full git history
Installs detect-secrets
Scans current codebase against .secrets.baseline
Scans last 100 commits of git history
Fails build if new secrets detected (not in baseline)

Viewing Scan Results

On GitHub:

Go to your repository
Click Actions tab
Select Secret Scanning workflow
View latest run results

Build failures:

If scan finds new secrets, the build fails
Check the workflow logs for detected secrets
Remove secrets and update baseline if needed
Push again to re-run checks

Troubleshooting

False Positives

Problem: Scanner detects a UUID or other non-secret as a secret

Solution:

# Add to baseline (after verifying it's not a real secret)
uv tool run detect-secrets scan --baseline .secrets.baseline

# Review the addition
git diff .secrets.baseline

# Commit the updated baseline
git add .secrets.baseline
git commit -m "Add false positive UUID to baseline"

Pre-commit Hook Not Running

Problem: Hooks don't run on git commit

Solution:

# Verify hooks are installed
ls -la .git/hooks/pre-commit

# If missing, reinstall
pre-commit install

# Test manually
pre-commit run --all-files

Baseline File Conflicts

Problem: Merge conflicts in .secrets.baseline after updating from main

Solution:

# Regenerate baseline from scratch
uv tool run detect-secrets scan --baseline .secrets.baseline

# Review changes carefully
git diff .secrets.baseline

# Commit regenerated baseline
git add .secrets.baseline
git commit -m "Regenerate secrets baseline after merge"

Real Secret Committed by Mistake

Problem: You accidentally committed a real secret

Solution:

1. Immediately rotate the compromised credential:

USPTO API key: Generate new key at https://developer.uspto.gov/
Mistral API key: Generate new key at https://console.mistral.ai/api-keys/

2. Remove from git history:

# Use git filter-repo (recommended) or BFG Repo Cleaner
# See: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/removing-sensitive-data-from-a-repository

# Example with git filter-repo (install first: pip install git-filter-repo)
git filter-repo --invert-paths --path path/to/file/with/secret.py

3. Force push (if already pushed to GitHub):

git push --force origin main

4. Update baseline and verify:

uv tool run detect-secrets scan --baseline .secrets.baseline
git add .secrets.baseline
git commit -m "Remove compromised secret from baseline"
git push origin main

Scanner Hangs or Times Out

Problem: Scanning very large files or repos takes too long

Solution:

# Increase timeout for large files (not typically needed for FPD MCP)
uv tool run detect-secrets scan --baseline .secrets.baseline --timeout 300

# Or exclude large files
uv tool run detect-secrets scan --baseline .secrets.baseline --exclude-files 'large_file.json'

Best Practices

For Developers

✅ DO:

Run pre-commit run --all-files before pushing
Use environment variables for all secrets (never hardcode)
Store secrets in CLAUDE.md (already in .gitignore)
Use placeholder keys in tests (e.g., "test_key_for_unit_tests")
Update baseline when adding legitimate test placeholders
Review baseline changes carefully before committing

❌ DON'T:

Commit files with real API keys, tokens, or passwords
Use --no-verify to bypass hooks (except emergencies)
Update baseline to hide real secrets (rotate the secret instead!)
Share API keys in chat, email, or documentation
Reuse the same API key across multiple projects

For API Key Management

USPTO API Key:

Store in Claude Desktop config: %APPDATA%\Claude\claude_desktop_config.json
Or use environment variable: $env:USPTO_API_KEY="your_key"
Rotate quarterly or after any security incident

Mistral API Key (optional):

Store in same Claude Desktop config file
Or use environment variable: $env:MISTRAL_API_KEY="your_key"
Monitor usage costs ($0.001/page for OCR)

For Code Reviews

Check for:

Files with high-entropy strings (not in baseline)
Hardcoded credentials or tokens
Configuration files with real secrets (should use env vars)
Test files with real API keys (should use placeholders)

Verify:

Baseline updates are justified (false positives or test placeholders)
No --no-verify commits without good reason
Environment variables are documented in README

Integration with FPD MCP Architecture

Protected Files

The following FPD MCP files are particularly sensitive and actively scanned:

Configuration:

src/fpd_mcp/config/settings.py - Environment variable handling
field_configs.yaml - Field configuration (no secrets, but scanned)

Test Files:

tests/test_basic.py - Unit tests with placeholder keys (baseline tracked)
tests/test_integration.py - Integration tests (baseline tracked)
tests/test_extraction.py - Document extraction tests (CONTAINS REAL KEYS - NOT COMMITTED)

API Client:

src/fpd_mcp/api/fpd_client.py - API authentication logic

Documentation:

CLAUDE.md - IN .GITIGNORE - Contains real keys, never committed
README.md - Public documentation (scanned to ensure no real keys)

Excluded Files

Automatically excluded:

CLAUDE.md - In .gitignore (contains real keys for development)
configs/**/*.json - Example configurations with placeholder keys
*.md files - Markdown documentation (except real secrets)

Cross-MCP Consistency

The FPD MCP secret scanning matches the PTAB MCP pattern:

Same detect-secrets version (v1.5.0)
Same exclusion patterns
Same pre-commit hooks
Same GitHub Actions workflow structure
Consistent with Patent File Wrapper MCP security practices

Additional Resources

detect-secrets documentation: https://github.com/Yelp/detect-secrets
USPTO API key management: https://developer.uspto.gov/
Mistral API key management: https://console.mistral.ai/api-keys/
GitHub secret scanning: https://docs.github.com/en/code-security/secret-scanning
FPD MCP Security Guidelines: SECURITY_GUIDELINES.md

Support

Found a security issue?

Create a private security advisory on GitHub
Or email: [contact information]

Questions about secret scanning?

Review this documentation
Check troubleshooting section
Open an issue on GitHub (without revealing secrets!)

Last Updated: 2025-10-12 Version: 1.0 detect-secrets Version: 1.5.0

FilesExpand file tree

SECURITY_SCANNING.md

Latest commit

History

SECURITY_SCANNING.md

File metadata and controls

Security Scanning - USPTO Final Petition Decisions MCP

Overview

🚨 NEW: Unicode Steganography Protection

What Gets Scanned

1. Secret Detection (20+ types)

2. Prompt Injection Detection (80+ patterns)

Secret Types Detected (20+ types)

Prompt Injection Attack Patterns (70+ patterns)

Prompt Injection Baseline System

Why a Baseline System?

How It Works

Baseline File Format

Baseline Commands

Usage Examples

When to Update Baseline

GitHub Actions Integration

Managing the Baseline File

Files Excluded from Scanning

Installation

Prerequisites

Option 1: Using uv (Recommended)

Option 2: Using pip

Install Pre-commit Hooks

Usage

Prompt Injection Detection (Including Unicode Steganography)

Pre-commit Hooks (Local Development)

Manual Scanning

Updating the Baseline

GitHub Actions Integration

Workflow Configuration

Viewing Scan Results

Troubleshooting

False Positives

Pre-commit Hook Not Running

Baseline File Conflicts

Real Secret Committed by Mistake

Scanner Hangs or Times Out

Best Practices

For Developers

For API Key Management

For Code Reviews

Integration with FPD MCP Architecture

Protected Files

Excluded Files

Cross-MCP Consistency

Additional Resources

Support