Research Reports

Analyzing Nuclei

7 min read

Source: projectdiscovery/nuclei @ 397ffaa

How It Works

Nuclei is a template-driven vulnerability scanner: you write YAML files that describe what to send and how to recognize a vulnerability, and the engine handles target enumeration, concurrency, deduplication, and result reporting. A template specifies one or more protocol requests (HTTP, DNS, TCP, SSL, WebSocket, WHOIS, JavaScript, or Code) along with matchers and extractors that evaluate responses. The engine compiles templates into executable request objects, clusters identical requests across templates to avoid redundant network I/O, and fans out execution across configurable work pools.

The design philosophy is “zero false positives through real-world simulation.” Templates don’t just pattern-match signatures — they can chain multiple requests, use conditional logic via flow expressions, inject OAST (Out-of-band Application Security Testing) payloads through interactsh, and run arbitrary JavaScript. This makes a nuclei template closer to a scripted exploit than a static signature.

At its core, the scanner follows a pipeline: load templates → cluster → execute per strategy → match/extract → report. Templates live in a separate nuclei-templates repository (thousands of community-contributed templates), but users can write and load their own. The engine supports two scan strategies — template-spray (iterate targets per template) and host-spray (iterate templates per target) — chosen based on the --scan-strategy flag.

Architecture

cmd/nuclei/main.go          # CLI entry, flag parsing, signal handling
     │
     ▼
internal/runner/runner.go   # Orchestrator: init, template loading, execution
     │
     ├── pkg/catalog/        # Template discovery from disk/remote
     ├── pkg/catalog/loader/ # Template store: filtering by tags/severity/author
     ├── pkg/core/           # Execution engine, work pools, clustering
     │      ├── engine.go
     │      ├── execute_options.go   # Scan strategies (template/host spray)
     │      ├── executors.go         # Per-target template execution
     │      ├── workpool.go          # Adaptive wait groups
     │      └── workflow_execute.go  # Recursive workflow execution
     │
     ├── pkg/templates/      # Template struct, compilation, clustering
     ├── pkg/protocols/      # Protocol implementations
     │      ├── http/
     │      ├── dns/
     │      ├── network/     # Raw TCP
     │      ├── ssl/
     │      ├── websocket/
     │      ├── whois/
     │      ├── headless/    # Browser-based (via go-rod)
     │      ├── javascript/  # JS runtime
     │      ├── code/        # Code protocol templates
     │      └── file/        # Local file scanning
     │
     ├── pkg/operators/      # Matchers + extractors (shared across protocols)
     │      ├── matchers/    # status, size, word, regex, binary, DSL
     │      └── extractors/  # regex, xpath, json, dsl, kval
     │
     ├── pkg/output/         # Result formatting (JSON, JSONL, SARIF, Markdown, PDF)
     ├── pkg/reporting/      # Issue tracker integrations (Jira, GitHub, etc.)
     ├── pkg/fuzz/           # DAST / fuzzing engine
     ├── pkg/js/             # JavaScript runtime for code protocol
     └── lib/                # Public SDK (NucleiEngine) for embedding

Data flow: The Runner initializes all subsystems (rate limiter, interactsh client, headless browser, output writer), creates an ExecutorOptions struct that carries shared state, then hands templates and targets to core.Engine.ExecuteScanWithOpts(). The engine clusters templates by hashing their request signatures, then dispatches via the chosen strategy. Each protocol request implements ExecuteWithResults() which returns []*output.ResultEvent through a callback. Matchers and extractors evaluate responses using compiled regex/patterns or a DSL expression engine (govaluate).

The Spine

1. Entry point — cmd/nuclei/main.go parses flags via goflags, creates a Runner, and calls RunEnumeration().

2. Runner initialization — Runner.New() sets up:

  • Template catalog (disk-based, pointing at ~/.nuclei-templates)
  • Output writer (file, screen, or both)
  • Interactsh client for OAST callbacks
  • Rate limiter, headless browser, reporting client
  • Resume state from config

3. RunEnumeration() — The main scan loop:

executorOpts := &protocols.ExecutorOptions{...}  // shared execution config
executorEngine := core.New(r.options)
store, _ := loader.New(loaderConfig)              // load + filter templates
store.Load()
results := engine.ExecuteScanWithOpts(ctx, finalTemplates, r.inputProvider, r.options.DisableClustering)

4. Engine execution — ExecuteScanWithOpts():

  • Clusters templates via templates.ClusterTemplates() (hashes request signatures)
  • Separates self-contained templates (no target needed)
  • Dispatches via strategy: executeTemplateSpray() or executeHostSpray()
  • Each goroutine calls template.Executer.ExecuteWithResults(scanContext)

5. Protocol execution — Each protocol’s Request struct (e.g., http.Request) implements:

  • Compile() — pre-compiles matchers, extractors, payloads
  • ExecuteWithResults() — sends the request, evaluates operators, invokes the result callback

6. Result writing — output.Writer formats and writes findings. Each ResultEvent contains template ID, matched host, extracted data, severity, request/response pairs, and the raw template content.

Key Patterns

Protocol interface uniformity. Every protocol implements the same Executer interface (Compile, Execute, ExecuteWithResults). This means adding a new protocol requires implementing one struct with three methods, plus embedding Operators for matchers/extractors. The pattern is clean and consistent across all nine protocol types.

Template clustering. Templates with a single identical request (same path, headers, method) are grouped into a ClusterExecuter. One HTTP request is sent; all cluster members’ matchers evaluate the same response. This can reduce network traffic by orders of magnitude for large template sets targeting the same endpoints.

Adaptive work pools. The WorkPool uses syncutil.AdaptiveWaitGroup with separate pools for headless (browser) and standard protocols. Pool sizes are configurable at runtime and can be resized mid-scan via RefreshWithConfig().

Interactsh integration. OAST testing is deeply woven into the engine. The interactsh.Client maintains a URL-to-request cache and polls an interactsh server for out-of-band interactions. When a template uses {{interactsh-url}}, the engine substitutes a unique subdomain and later correlates any DNS/HTTP interactions back to the originating request.

Operator DSL. Matchers and extractors support a DSL powered by govaluate with custom functions (len, contains, sha256, base64, etc.). This allows complex conditional matching without writing code. The DSL function registry is in pkg/operators/common/dsl/.

Scan context pattern. scan.ScanContext wraps a context.Context with an input target, error/warning accumulators, and event/result callbacks. This provides a clean boundary for per-request state without polluting the protocol implementations.

Non-Obvious Details

Template signing and verification

Nuclei supports cryptographic signing of templates via Ed25519 keys. The --sign flag signs templates with a private key, and --disable-unsigned-templates (-dut) refuses to run unsigned ones. This is a supply-chain security measure for enterprise deployments where template provenance matters.

Honeypot detection. The engine has a built-in honeypot detector (pkg/protocols/common/honeypotdetector/) that tracks how many distinct template IDs match a given host. If a host triggers more than --honeypot-threshold (default: 15) different templates, it’s flagged as a likely honeypot and results can be suppressed with --suppress-honeypot.

Two execution modes beyond CLI. Besides the CLI, nuclei can run as:

  1. A DAST server (--dast-server) that accepts HTTP requests with payloads and fuzzes them in real-time
  2. An embedded SDK (lib/) exposing NucleiEngine with a functional-options API for Go programs

Global matchers. Templates marked with global-matchers register operators that evaluate every response across the scan — not just their own protocol requests. This is useful for passive detection rules that should run on all traffic.

Flow-based templates. The flow field in a template accepts JavaScript code that controls request execution order and conditional logic. This enables multi-step attack chains where the output of one request determines whether subsequent requests run. Flow templates are excluded from clustering.

Resume mechanism. Nuclei tracks in-flight requests per template ID in ResumeCfg. On Ctrl+C, it serializes state to a resume file; on --resume, it skips already-completed template/target pairs. This is critical for long-running scans against large target lists.

Assessment

Strengths:

  • Extensibility is first-class. The template system is the product — the engine is the platform. This separation means the community can grow the detection library without touching Go code.
  • Clustering is a genuine performance differentiator. Deduplicating identical requests across thousands of templates turns a O(n×m) problem into something tractable.
  • Dual scan strategies (template-spray vs host-spray) let operators optimize for their scenario — few targets/many templates vs many targets/few templates.
  • The SDK (lib/) makes nuclei embeddable in CI/CD pipelines and other tools, which is unusual for scanners this capable.

Concerns:

  • The Runner is monolithic. internal/runner/runner.go is ~1,060 lines with deep initialization chains. It holds 15+ struct fields covering unrelated concerns (fuzzing, DAST server, honeypot detection, PDCP auth). This makes testing individual subsystems harder.
  • YAML v2 parsing. The codebase uses gopkg.in/yaml.v2, which is in maintenance mode. Custom YAML handling in pkg/utils/yaml/ adds strict syntax checking, but the underlying parser has known limitations with complex nested structures.
  • Error handling is inconsistent. Some paths use errors.Wrap, others use errkit.Wrap, and many initialization steps silently swallow errors with _ = ... assignments. The errkit package appears to be a newer addition that hasn’t fully displaced pkg/errors.

Recommendations:

  • Break Runner into smaller, composable initialization stages — template loading, network setup, output configuration, and scan execution could each be separate structs with clear interfaces.
  • Consider migrating to a protocol registry pattern where protocols self-register (e.g., via init()) rather than being hardcoded in templates.go imports. This would make the protocol set more pluggable.
  • The ExecutorOptions struct has grown to ~40 fields. Grouping related fields (network config, output config, scan state) into sub-structs would improve readability.

Related reports: analyzing-bloodhound, analyzing-pacu, analyzing-pentagi, analyzing-misp, analyzing-opencti

Graph View

Backlinks