The Complete NATS Developer Guide

A deep, practical reference for building production systems with NATS — covering Core NATS, JetStream, security, clustering, and battle-tested design patterns.

What is NATS
Core Concepts
Core NATS Messaging
Subjects & Wildcards
Queue Groups
Request-Reply
JetStream Overview
Streams
Consumers
Key/Value Store
Object Store
Security
Clustering & Superclusters
Monitoring & Observability
Client Libraries
Design Patterns
Best Practices Checklist
Common Pitfalls
Deployment Recipes
Resources

1. What is NATS

NATS is a lightweight, high-performance messaging system written in Go, designed around simplicity and speed. It provides:

Core NATS: fire-and-forget pub/sub, request-reply, queueing — at-most-once delivery, extremely low latency (microseconds), no persistence.
JetStream: a built-in persistence layer on top of Core NATS providing at-least-once and exactly-once delivery, streaming, replay, and durable consumers — comparable to Kafka but far simpler to operate.
NATS.io ecosystem: NATS Server, NGS (global managed service), leaf nodes, superclusters, NATS CLI, NATS Surveyor, NACK (Kubernetes controller).

Why NATS over Kafka/RabbitMQ?

Aspect	NATS	Kafka	RabbitMQ
Operational complexity	Very low (single binary)	High (ZK/KRaft, brokers)	Medium
Latency	Sub-millisecond	Low-ms	Low-ms
Persistence	Optional (JetStream)	Always	Optional
Protocol	Simple text-based	Custom binary	AMQP
Multi-tenancy	Accounts (native)	ACLs	vhosts
Edge / IoT friendly	Yes (leaf nodes, tiny footprint)	No	Partial
Exactly-once	Yes (JetStream + dedup)	Yes (transactions)	No (native)

2. Core Concepts

Subject: the addressing mechanism (like a topic), e.g. orders.created.eu. Hierarchical, dot-separated.
Publisher/Subscriber: publish/subscribe to subjects.
Connection: a client’s TCP connection to a NATS server, can multiplex many subs/pubs.
Server / Cluster / Supercluster: single node → clustered nodes (same region) → gateways connecting clusters (multi-region).
Account: isolated namespace with its own subject space, security context, and JetStream limits — the core multi-tenancy primitive.
Stream: a JetStream-managed, durable, ordered log of messages captured from one or more subjects.
Consumer: a cursor/view over a stream, either push- or pull-based, tracking delivery state.

3. Core NATS Messaging

nc, _ := nats.Connect(nats.DefaultURL)
defer nc.Close()

sub, _ := nc.Subscribe("orders.created", func(msg *nats.Msg) {
    fmt.Printf("Received: %s\n", string(msg.Data))
})
defer sub.Unsubscribe()

nc.Publish("orders.created", []byte(`{"id": 123}`))

Core NATS is at-most-once: if no subscriber is listening, the message is lost. There is no persistence, no acknowledgment, no replay. This is by design — use it for ephemeral signals, telemetry, service discovery, and low-latency RPC. Use JetStream when you need durability.

Connection Lifecycle & Reconnection

Always configure reconnection behavior explicitly in production:

nc, err := nats.Connect(
    "nats://server1:4222,nats://server2:4222,nats://server3:4222",
    nats.MaxReconnects(-1),          // retry forever
    nats.ReconnectWait(2*time.Second),
    nats.Timeout(5*time.Second),
    nats.DisconnectErrHandler(func(nc *nats.Conn, err error) {
        log.Printf("disconnected: %v", err)
    }),
    nats.ReconnectHandler(func(nc *nats.Conn) {
        log.Printf("reconnected to %s", nc.ConnectedUrl())
    }),
    nats.ClosedHandler(func(nc *nats.Conn) {
        log.Printf("connection closed")
    }),
)

Always pass a list of seed URLs (not just one) so the client can fail over during initial connect.

4. Subjects & Wildcards

Subjects are dot-delimited tokens: region.service.event, e.g. eu.billing.invoice_created.

* matches exactly one token: eu.*.invoice_created matches eu.billing.invoice_created but not eu.billing.sub.invoice_created.
> matches one or more trailing tokens (must be last): eu.> matches eu.billing.invoice_created and eu.billing.sub.x.

Subject Design Best Practices

Design subjects hierarchically from broad to narrow: <domain>.<entity>.<event>.<region> — this lets consumers subscribe at whatever granularity they need.
Never put dynamic, high-cardinality data as the leading token — e.g. don’t do <user_id>.events; prefer events.user.<user_id> so wildcard subscriptions on events.> still work sanely and ACLs stay manageable.
Keep subjects short and predictable — long subjects cost more CPU on matching at scale.
Version your subjects when contracts change: orders.v2.created instead of breaking orders.created silently.
Reserve a dedicated namespace per service/team, e.g. prefix with the owning service: billing.invoice.created.

5. Queue Groups

Queue groups implement load-balanced pub/sub — multiple subscribers in the same queue group, only one receives each message (round-robin distribution), enabling horizontal scaling of workers.

nc.QueueSubscribe("work.jobs", "workers", func(msg *nats.Msg) {
    process(msg)
})

Run N instances of this worker with the same queue name — NATS distributes messages across them automatically. Combine with subject wildcards for topic-based worker pools:

nc.QueueSubscribe("orders.*.process", "order-workers", handler)

Pattern: use queue groups for Core NATS scale-out workers when you can tolerate message loss on crash (no redelivery); use JetStream pull consumers when you need guaranteed processing.

6. Request-Reply

Core NATS has native RPC semantics: the client generates a unique inbox subject, publishes with a reply-to, and waits for a response.

// Responder
nc.Subscribe("math.add", func(msg *nats.Msg) {
    result := add(msg.Data)
    nc.Publish(msg.Reply, result)
})

// Requester
resp, err := nc.Request("math.add", []byte(`{"a":1,"b":2}`), 2*time.Second)

Under the hood, this uses an ephemeral inbox subject (_INBOX.<uuid>) — the client library manages subscribing/unsubscribing transparently. Always set a timeout; a request with no responder will otherwise hang until timeout anyway, so make it explicit and short (typically 1-5s for intra-DC calls).

Scatter-Gather: publish a request, collect multiple replies within a window using SubscribeSync + a timer, useful for service discovery or “ask all shards” patterns.

7. JetStream Overview

JetStream adds a persistence and streaming layer. Key primitives:

Stream: durable, ordered, append-only log capturing messages from subjects.
Consumer: stateful cursor over a stream (push or pull), tracks acked/unacked messages.
Storage backends: file (disk, durable across restarts) or memory (fast, volatile).
Delivery guarantees: at-least-once by default; exactly-once achievable via message deduplication (Nats-Msg-Id header) + idempotent consumers.

js, _ := nc.JetStream()

js.AddStream(&nats.StreamConfig{
    Name:     "ORDERS",
    Subjects: []string{"orders.>"},
    Storage:  nats.FileStorage,
    Retention: nats.LimitsPolicy,
    MaxAge:   7 * 24 * time.Hour,
    Replicas: 3,
})

js.Publish("orders.created", []byte(`{"id":123}`))

8. Streams

Retention Policies

Policy	Behavior
`LimitsPolicy`	Keep messages until limits (age/size/count) are hit — classic log retention
`InterestPolicy`	Delete messages once all known consumers have acked them — good for work queues
`WorkQueuePolicy`	Message removed as soon as any consumer acks it (single-consumer-semantics guarantee: only one consumer group can attach)

Key Stream Config Fields

&nats.StreamConfig{
    Name:              "EVENTS",
    Subjects:          []string{"events.>"},
    Retention:         nats.LimitsPolicy,
    MaxConsumers:      -1,
    MaxMsgs:           1_000_000,
    MaxBytes:          10 * 1024 * 1024 * 1024, // 10GB
    MaxAge:            30 * 24 * time.Hour,
    MaxMsgSize:        1024 * 1024,
    Storage:           nats.FileStorage,
    Replicas:          3,             // RAFT-replicated for HA
    Discard:           nats.DiscardOld,
    Duplicates:        2 * time.Minute, // dedup window
    AllowRollup:       true,
    DenyDelete:        true,          // compliance: no manual deletes
    DenyPurge:         false,
}

Message Deduplication (Exactly-Once Publish)

js.Publish("orders.created", data, nats.MsgId("order-123-v1"))

If a message with the same Nats-Msg-Id arrives again within the Duplicates window, JetStream silently drops it — this gives you idempotent publishing for free (essential for retry-safe producers).

Stream Mirroring & Sourcing

Mirror: exact 1:1 replica of another stream (same subjects, same messages) — great for cross-region read replicas or DR.
Source: aggregate messages from multiple streams into one, optionally re-subject-mapping — great for building materialized/aggregate streams.

js.AddStream(&nats.StreamConfig{
    Name: "ORDERS_EU_MIRROR",
    Mirror: &nats.StreamSource{Name: "ORDERS_EU"},
})

9. Consumers

Push vs Pull

Push consumers: server pushes messages to a subscription subject. Simple, but harder to control backpressure precisely — mostly superseded by pull in modern usage.
Pull consumers: client explicitly requests batches of messages (Fetch). Recommended default for almost all workloads — gives full control over concurrency, backpressure, and horizontal scaling.

sub, _ := js.PullSubscribe("orders.>", "order-processor",
    nats.ManualAck(),
    nats.AckWait(30*time.Second),
    nats.MaxDeliver(5),
)

for {
    msgs, _ := sub.Fetch(10, nats.MaxWait(5*time.Second))
    for _, m := range msgs {
        if err := process(m); err != nil {
            m.Nak()          // negative ack: redeliver
            continue
        }
        m.Ack()
    }
}

Durable vs Ephemeral

Durable consumer (has a Durable name): survives client disconnects; resumes where it left off. Use for anything that must not lose its cursor position.
Ephemeral consumer (no durable name): deleted when the last subscription closes — good for temporary/interactive queries or debugging.

Ack Strategies

Mode	Meaning
`AckExplicit`	Client must Ack/Nak each message individually (default, safest)
`AckAll`	Acking message N also acks all prior unacked messages — higher throughput, less granular
`AckNone`	Fire-and-forget, no redelivery tracking

Ack patterns:

msg.Ack() — success.
msg.Nak() — failure, redeliver (subject to MaxDeliver).
msg.NakWithDelay(d) — redeliver after backoff.
msg.Term() — poison message, do not redeliver, mark terminally failed.
msg.InProgress() — extend ack wait for long-running processing (heartbeat).

Consumer Config Essentials

&nats.ConsumerConfig{
    Durable:       "order-processor",
    AckPolicy:     nats.AckExplicitPolicy,
    AckWait:       30 * time.Second,
    MaxDeliver:    5,
    MaxAckPending: 1000,             // backpressure cap
    DeliverPolicy: nats.DeliverAllPolicy, // or DeliverNewPolicy / DeliverByStartTimePolicy
    ReplayPolicy:  nats.ReplayInstantPolicy, // or ReplayOriginalPolicy (respect original timing)
    FilterSubject: "orders.created.*",
}

MaxAckPending is your primary backpressure knob — it caps how many messages can be outstanding (delivered but unacked) at once, protecting slow consumers from being overwhelmed.

10. Key/Value Store

JetStream-backed KV store, ideal for config, feature flags, service discovery, distributed locks, and small state.

kv, _ := js.CreateKeyValue(&nats.KeyValueConfig{
    Bucket:  "config",
    History: 5,
    TTL:     0, // no expiry
})

kv.Put("feature.dark_mode", []byte("true"))
entry, _ := kv.Get("feature.dark_mode")

// Watch for changes (real-time config propagation)
watcher, _ := kv.Watch("feature.>")
for update := range watcher.Updates() {
    if update != nil {
        fmt.Println(update.Key(), string(update.Value()))
    }
}

Under the hood, KV is just a stream named KV_<bucket> with one subject per key.
History controls how many past revisions are retained per key.
Use kv.Update(key, val, revision) for optimistic-concurrency-controlled writes (compare-and-swap) — essential for distributed coordination.

11. Object Store

For blobs too large for regular messages (files, images, backups) — chunks large objects automatically over a stream.

os, _ := js.CreateObjectStore(&nats.ObjectStoreConfig{Bucket: "uploads"})
os.PutFile("/local/path/report.pdf")
os.GetFile("report.pdf", "/tmp/report.pdf")

12. Security

Accounts & Multi-Tenancy

Accounts provide hard isolation: each account has its own subject namespace (no cross-account leakage unless explicitly exported/imported), its own JetStream resource limits, and independent auth. This is NATS’s core multi-tenancy primitive — think of accounts as separate organizations sharing infrastructure.

Decentralized Auth: NKeys & JWTs

NKeys: Ed25519 key pairs (like SSH keys) used for identity — no shared secrets transmitted over the wire.
JWTs: describe an account or user’s permissions (subject-level pub/sub allow/deny lists, limits) and are signed by an operator/account key.

nsc add operator MyOperator
nsc add account MyApp
nsc add user --account MyApp MyUser
nsc edit user MyUser --allow-pub "orders.>" --allow-sub "orders.>,_INBOX.>"

TLS

Always enable TLS between clients and servers, and between cluster/gateway routes, in any production or multi-host deployment:

tls {
  cert_file: "/etc/nats/server-cert.pem"
  key_file:  "/etc/nats/server-key.pem"
  ca_file:   "/etc/nats/ca.pem"
  verify:    true
}

Authorization Best Practices

Use least-privilege subject permissions — grant pub/sub only on the exact subject patterns a service needs, never > blanket access in production.
Isolate tenants with separate accounts, not just subject prefixes — prefixes are convention, accounts are enforced isolation.
Rotate credentials via short-lived JWTs where possible; avoid static long-lived tokens.
Use exports/imports for deliberate, explicit cross-account communication rather than opening broad access.

13. Clustering & Superclusters

Cluster: 3+ NATS servers (odd number, RAFT quorum) in the same region/DC, routed together for HA. 3 or 5 nodes typical.
Gateway: connects clusters across regions into a supercluster, providing global subject visibility with locality-aware routing.
Leaf nodes: lightweight edge connections (e.g., IoT devices, branch offices) that extend the subject space into a central cluster without joining full mesh routing — ideal for edge computing and low-bandwidth links.

JetStream Replication

Set Replicas: 3 on streams for RAFT-based replication — tolerates 1 node failure with automatic leader election and no data loss (as long as quorum is intact). Always deploy JetStream clusters with odd replica counts (1, 3, 5) to maintain quorum math.

Sizing Guidance

3-node cluster: tolerates 1 failure.
5-node cluster: tolerates 2 failures, higher write latency (more replication round trips) — use for critical streams only.
Don’t over-replicate: replicating every stream to 5 nodes for “safety” tanks throughput unnecessarily.

14. Monitoring & Observability

/varz, /connz, /subz, /routez, /jsz HTTP monitoring endpoints exposed by nats-server -m 8222.
NATS CLI: nats stream info, nats consumer info, nats stream report, nats bench for load testing.
NATS Surveyor: Prometheus exporter + Grafana dashboards for fleet-wide observability.
Server events: subscribe to $SYS.> subjects for connection/disconnection/auth-violation events (requires system account access).

Key metrics to alert on:

Consumer num_pending / num_ack_pending growth (backlog building up).
MaxAckPending saturation (slow consumers).
JetStream storage utilization vs MaxBytes.
Cluster RAFT leader elections frequency (instability signal).
Slow consumer disconnects (nats.ErrSlowConsumer) — indicates a subscriber not draining fast enough.

15. Client Libraries

Official/first-class clients: Go (nats.go), Python (nats.py), Node.js (nats.js), Java, C#/.NET, Rust, Ruby, Elixir (gnat), C.

General client best practices:

Reuse a single connection per process; multiplex subjects over it rather than opening many connections.
Always handle ErrSlowConsumer — it means your callback isn’t keeping up and messages are being dropped from the pending buffer.
Set explicit PendingLimits for subscriptions handling high-volume subjects.
Use async publishing (js.PublishAsync) with a completion callback for high-throughput producers, but track/flush with js.PublishAsyncComplete() before shutdown.

16. Design Patterns

Event-Driven Microservices

Services publish domain events on well-namespaced subjects (<service>.<entity>.<event>); other services subscribe to the events they care about via wildcards. Decouples producers from consumers entirely.

CQRS + Event Sourcing

Use a LimitsPolicy stream as the durable event log (source of truth); build read-model projections by consuming the stream with a durable consumer and applying events to a database/cache. Replay the stream (DeliverPolicy: DeliverAllPolicy) to rebuild projections from scratch.

Work Queue

Use WorkQueuePolicy streams + pull consumers with a shared durable name across worker instances — NATS guarantees each message goes to exactly one worker in the group, removed once acked.

Saga / Choreography

Chain services via events: Service A publishes order.created → Service B (queue-group subscriber) processes payment, publishes payment.completed or payment.failed → Service C reacts accordingly. Use correlation IDs in headers for tracing the whole saga.

Request-Reply Microservices (Synchronous RPC)

Use Core NATS request-reply for low-latency internal RPC calls (auth checks, lookups) where you don’t need durability — much simpler and faster than HTTP for service-to-service calls inside a cluster.

Fan-Out / Fan-In

Fan-out: one publisher, many independent subscribers (each gets a copy) via plain subscribe (not queue group). Fan-in: many publishers into one subject, single consumer aggregates — common in telemetry/logging pipelines.

Outbox Pattern

When publishing must be transactionally consistent with a DB write, write the event to an “outbox” table in the same DB transaction, then a separate relay process publishes to JetStream using the DB row ID as Nats-Msg-Id for dedup-safe, exactly-once-effective delivery.

Distributed Locks via KV

Use kv.Create() (fails if key exists) as a compare-and-swap primitive for lightweight distributed locking; release by deleting the key or letting TTL expire.

17. Best Practices Checklist

18. Common Pitfalls

Using Core NATS for anything that must not be lost. No subscriber = message gone. Always use JetStream for durability-critical data.
Forgetting to Ack. Unacked messages redeliver forever (up to MaxDeliver), causing duplicate processing loops if consumers don’t handle idempotency.
Using > in ACLs for convenience. This is a security anti-pattern; always scope subjects tightly.
Not setting MaxAckPending. Unbounded pending messages can overwhelm slow consumers and blow memory.
Single seed URL in connection string. If that one node is down during a reconnect attempt, the client can’t discover the rest of the cluster.
Ignoring slow consumer errors. They silently drop messages from the client-side pending buffer — treat as a critical alert, not a warning.
Over-replicating streams. Unnecessary R5 replication on non-critical data adds write latency for no benefit.
Not testing failover. Kill a JetStream leader node in staging regularly to validate your app handles reconnect/re-election gracefully.
Conflating “queue group” (Core NATS load balancing) with “work queue stream” (JetStream durable work distribution) — they solve similar problems but have very different durability guarantees.

19. Deployment Recipes

Minimal Single-Node (dev)

nats-server -js -sd /data/jetstream

3-Node Cluster (docker-compose sketch)

services:
  nats1:
    image: nats:2.10-alpine
    command: "-js -sd /data -cluster_name NATS -cluster nats://0.0.0.0:6222 -routes nats://nats2:6222,nats://nats3:6222"
  nats2:
    image: nats:2.10-alpine
    command: "-js -sd /data -cluster_name NATS -cluster nats://0.0.0.0:6222 -routes nats://nats1:6222,nats://nats3:6222"
  nats3:
    image: nats:2.10-alpine
    command: "-js -sd /data -cluster_name NATS -cluster nats://0.0.0.0:6222 -routes nats://nats1:6222,nats://nats2:6222"

Kubernetes

Use the official NATS Helm chart or the NACK (NATS controller for Kubernetes) to manage Streams/Consumers as CRDs declaratively:

apiVersion: jetstream.nats.io/v1beta2
kind: Stream
metadata:
  name: orders
spec:
  name: ORDERS
  subjects: ["orders.>"]
  storage: file
  replicas: 3
  maxAge: "168h"

20. Resources

Official docs: https://docs.nats.io
NATS by Example (runnable snippets): https://natsbyexample.com
GitHub: https://github.com/nats-io
Slack community: via docs.nats.io
nats CLI: brew install nats-io/nats-tools/nats or via GitHub releases

This guide reflects NATS Server 2.10+ semantics. Always cross-check against the latest official docs for API changes.

Table of Contents