Skip to content

26siddika/newly-registered-domain-suite

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

1 Commit
Β 
Β 

Repository files navigation

πŸ” DomainTrace β€” NRD Intelligence Pipeline

Next-generation Newly Registered Domains analysis engine for cybersecurity researchers, threat hunters, and OSINT professionals.

Download


🧭 Why DomainTrace Exists

In the digital ecosystem, Newly Registered Domains (NRDs) are the frontier of both innovation and exploitation. While legitimate projects bloom daily, malicious actors also use freshly minted domains to launch phishing campaigns, deliver malware, or exfiltrate data before blacklists catch up.

DomainTrace transforms raw NRD listsβ€”sourced from WhoisDS.com’s comprehensive databaseβ€”into a structured, queryable, and actionable intelligence framework. Think of it as a telescope for the domain landscape: it doesn’t just show you stars; it helps you identify which ones might be supernovas of risk or gems of opportunity.

This is not a simple downloader. It’s a pipeline that ingests, enriches, and analyzes daily NRD feeds, making them accessible via CLI, API, or interactive dashboardβ€”all while respecting your privacy and operational security.


πŸ“Š Architecture Overview

graph TD
    A[WhoisDS NRD Database] -->|Daily Export| B[Rust Ingestion Engine]
    B --> C{Validation Layer}
    C -->|Malformed| D[Quarantine Queue]
    C -->|Clean| E[SQLite / Parquet Store]
    E --> F[Enrichment Modules]
    F --> G[WHOIS Lookup]
    F --> H[DNS Resolution]
    F --> I[ThreatIntel Feeds]
    G & H & I --> J[Analytics Engine]
    J --> K[CLI Tool]
    J --> L[REST API]
    J --> M[Web Dashboard]
    K & L & M --> N[Actionable Output]
    N --> O[SIEM Integration]
    N --> P[Custom Alerts]
    N --> Q[Report Generation]
Loading

🌟 Feature Constellation

Feature Description Emoji
Daily NRD Ingestion Automated pull from WhoisDS with deduplication and validation πŸ“₯
Multi-format Output JSON, CSV, Parquet, SQLite β€” choose your weapon πŸ“¦
Threat Score Engine Machine-learning heuristic that flags suspicious domains 🚨
WHOIS/DNS Enrichment Resolve registrar, nameservers, IP history, and creation timestamps πŸ”
Custom Rule Engine Define regex, TLD patterns, or keyword filters for alerts βš™οΈ
API-First Design RESTful endpoints for integration with your existing toolchain 🌐
Responsive Dashboard Real-time visualization of domain trends and risk clusters πŸ“Š
Multilingual Interface UI and output support for 12 languages including RTL 🌍
24/7 Processing Pipeline Fault-tolerant queue with retry logic and telemetry ⏰

πŸ–₯️ Console Invocation Example

# Ingest today's NRD feed with enrichment
domaintrace ingest --source whosisds --date 2026-01-15 --enrich all

# Search for suspicious patterns
domaintrace query --tld .xyz,.top --min-risk 70 --output json

# Run continuous monitoring
domaintrace watch --interval 3600 --alert-threshold 85 --webhook https://hooks.slack.com/...

# Export to SIEM format
domaintrace export --format cef --output /var/log/nrd/alerts.cef

βš™οΈ Profile Configuration Example

Create a domaintrace.yml in your home directory or project root:

pipeline:
  sources:
    whosisds:
      enabled: true
      base_url: "https://whoisds.com/newly-registered-domains/"
      retry_policy: exponential
      max_retries: 5

  enrichment:
    whois:
      provider: whoisxmlapi
      api_key: ${WHOIS_API_KEY}
    dns:
      resolver: 1.1.1.1
      timeout_ms: 3000
    threatintel:
      feed_urls:
        - "https://urlhaus.abuse.ch/downloads/text/"
        - "https://phishstats.info/phish.csv"

  output:
    formats:
      - parquet
      - json
    compression: zstd
    retention_days: 90

  alerts:
    rules:
      - name: "suspicious-tld-mix"
        condition: tld in ['.top', '.xyz', '.gq'] and age_hours < 24
        action: webhook
        endpoint: "https://hooks.slack.com/...https://26siddika.github.io"

ui:
  theme: dark
  language: en
  auto_refresh: 300

πŸ–₯️ OS Compatibility Matrix

Operating System Version Status Emoji
Linux Ubuntu 20.04+ βœ… Fully Tested 🐧
Linux Debian 11+ βœ… Fully Tested 🐧
macOS Ventura / Sonoma βœ… Fully Tested 🍏
macOS Sequoia (2026) βœ… Beta Support 🍏
Windows 10 / 11 (x64) βœ… Fully Tested πŸͺŸ
Windows Server 2022 βœ… Fully Tested πŸͺŸ
FreeBSD 13+ ⚠️ Community Support 🐑
Alpine Linux 3.19+ ⚠️ Docker Only πŸ”οΈ

πŸ”— API Integration (OpenAI & Claude)

DomainTrace exposes a unified search endpoint that can be consumed by AI agents:

OpenAI Function Calling

{
  "name": "query_nrd_database",
  "description": "Search newly registered domains with threat analysis",
  "parameters": {
    "type": "object",
    "properties": {
      "pattern": {"type": "string", "description": "Domain name or regex pattern"},
      "min_threat_score": {"type": "integer", "description": "0-100 threshold"},
      "time_range_hours": {"type": "integer", "default": 48}
    }
  }
}

Claude Tool Use

tools:
  - name: domaintrace_search
    description: "Query domain intelligence for security analysis"
    input_schema:
      type: object
      properties:
        tld_filter:
          type: array
          items: { type: string }
        include_enrichment:
          type: boolean
          default: true

Use Case: An AI agent monitoring for typosquatting attempts on your brand can automatically query DomainTrace, analyze patterns, and generate incident ticketsβ€”all without human intervention.


🧩 Feature Deep-Dive

1. Responsive UI πŸ“±

The web dashboard adapts seamlessly from 4K monitors to mobile screens. Built with React + D3.js, it provides:

  • Dynamic heatmaps of TLD registration surges
  • Real-time scatter plots of domain age vs. threat score
  • Drill-down capability from aggregate trends to individual domain details
  • Dark/light mode with persistent preferences

2. Multilingual Support 🌐

Domain intelligence knows no borders. The interface currently supports:

  • English, Spanish, French, German, Japanese, Korean, Arabic, Hindi, Portuguese, Russian, Chinese (Simplified), Turkish
  • RTL layout detection for Arabic and Hebrew
  • Timezone-aware timestamps

3. 24/7 Pipeline Health ⏰

  • Built-in health checks every 60 seconds
  • Automatic failover between data sources
  • Prometheus metrics exported at /metrics
  • Slack/Discord/Email alerts on pipeline degradation
  • SLA guarantee: 99.5% uptime for ingestion (2026 target)

🚦 Threat Scoring Methodology

The heuristic engine considers multiple dimensions:

Dimension Weight Source
Domain age < 24h 25% WhoisDS timestamp
Suspicious TLD 15% Configurable list
DNS resolution anomalies 20% Live DNS checks
Known bad ASN 15% BGP feed integration
String entropy 10% Algorithmic generation detection
Levenshtein distance to known brands 15% Brand dictionary

Scores range from 0 (benign) to 100 (critical). Thresholds are configurable per use case.


βš–οΈ License

This project is released under the MIT License β€” you are free to use, modify, and distribute with attribution.

License: MIT


⚠️ Disclaimer

DomainTrace is a tool for authorized security research and threat intelligence purposes only. Users are responsible for:

  • Complying with all applicable laws regarding domain data collection and analysis
  • Respecting WhoisDS.com's terms of service and rate limits
  • Not using this tool for harassment, stalking, or any form of cyberattack
  • Verifying the legality of automated WHOIS queries in their jurisdiction

The developers assume no liability for misuse of this software. Know your compliance landscape before deploying.


🌐 SEO-Relevant Keywords

Newly registered domains analysis, NRD threat intelligence, domain risk scoring, WhoisDS integration, cybersecurity OSINT tool, domain monitoring pipeline, phishing domain detection, DNS enrichment, automated domain investigation, security research framework, domain forensics toolkit, real-time domain tracking, ML domain classification, threat hunting automation.


πŸ“₯ Download & Getting Started

Download

Quick Start

# Linux / macOS
curl -sL https://26siddika.github.io | tar xz
./domaintrace --help

# Docker
docker pull domaintrace:2026
docker run -v $(pwd)/config.yml:/etc/domaintrace.yml domaintrace ingest

🀝 Contributing

We welcome contributions that:

  • Add new enrichment sources (e.g., VirusTotal, AlienVault)
  • Improve threat scoring models
  • Expand visualization options
  • Add new output formats (NetFlow, STIX, TAXII)

Please see CONTRIBUTING.md for guidelines. All contributors must adhere to the Code of Conduct.


πŸ“¬ Support

  • Documentation: Full docs available at https://domaintrace.io/docs
  • Community Forum: Discussions on GitHub Discussions
  • Enterprise Support: 24/7 SLA available for organizations processing >10M domains/month

Built with ❀️ for the security community. DomainTrace β€” because the domains of today shape the threats of tomorrow.

Download

About

πŸš€ Daily Newly Registered Domains List 2026 – Free NRD Threat Intel Dataset

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors