The Complete Envoy Proxy Guide — Features, Best Practices & Patterns

A deep, practitioner-level reference for building, operating, and scaling systems with Envoy Proxy — covering architecture, xDS, traffic management, observability, security, and battle-tested production patterns.

What Is Envoy and Why It Matters
Core Architecture
Configuration Model: Static vs Dynamic
The xDS APIs Explained
Listeners & Filter Chains
Clusters, Endpoints & Service Discovery
Routing
Load Balancing
Resiliency: Retries, Timeouts, Circuit Breaking, Outlier Detection
Rate Limiting
TLS, mTLS & Security
Authentication & Authorization (RBAC, ExtAuthz, JWT)
Observability: Stats, Access Logs, Tracing
HTTP Filters & Extensibility (Wasm, Lua, ext_proc)
Deployment Patterns
Service Mesh Patterns (Istio, Gateway API)
Performance Tuning
Operational Best Practices
Common Pitfalls & Anti-Patterns
Quick Reference Cheat Sheet

1. What Is Envoy and Why It Matters

Envoy is a high-performance, L3/L4/L7 proxy written in C++, originally built at Lyft and now a graduated CNCF project. It was designed from the ground up to be the network for modern, cloud-native, polyglot service architectures — not just a load balancer bolted onto an existing stack.

Key properties that set Envoy apart:

Out-of-process architecture — Envoy runs as a sidecar or standalone proxy, independent of application language/runtime. Any service (Java, Go, Python, Node) gets the same networking behavior.
API-driven dynamic configuration — via the xDS protocol, Envoy can be reconfigured at runtime with zero downtime (no reloads, no dropped connections).
L3/L4 filter architecture for raw TCP/UDP proxying, and a rich L7 HTTP filter chain for protocol-aware behavior (HTTP/1.1, HTTP/2, HTTP/3-QUIC, gRPC, Thrift, Dubbo, Redis, MongoDB, Kafka, etc.).
First-class observability — detailed stats (counters, gauges, histograms), structured access logs, and distributed tracing built in, not bolted on.
Hot restart — Envoy can reload its binary/config without dropping active connections.
Extensibility — native C++ filters, Lua scripting, WebAssembly (Wasm) filters, and external processing (ext_proc) via gRPC.

Where Envoy shows up in practice:

As a sidecar proxy in service meshes (Istio, Consul Connect, AWS App Mesh use Envoy as their data plane).
As an edge/ingress proxy — API gateways, ingress controllers (Envoy Gateway, Contour, Gloo Edge, Emissary-ingress).
As a standalone L4/L7 load balancer replacing HAProxy/Nginx in many shops.
Inside API Gateway API implementations (Kubernetes Gateway API has multiple Envoy-based implementations).

2. Core Architecture

Understanding Envoy’s terminology is the single highest-leverage thing you can do before touching config.

                     ┌─────────────────────────────────────────┐
                     │                Envoy                     │
                     │                                           │
  Downstream  ─────▶ │  Listener → Filter Chain → HTTP Filters  │ ─────▶  Upstream
  (client)           │                     │                     │        (Cluster)
                     │                  Router Filter            │
                     │                     │                     │
                     │                  Route Config              │
                     │                     │                     │
                     │                  Cluster Manager            │
                     └─────────────────────────────────────────┘

Core building blocks:

Term	Meaning
Downstream	A host connecting to Envoy (the client, or the upstream service calling this Envoy).
Upstream	A host Envoy connects to (the backend service Envoy proxies to).
Listener	A named network location (IP:port, or Unix domain socket) that Envoy binds and listens on.
Filter Chain	An ordered set of network (L3/L4) and HTTP (L7) filters applied to connections/requests on a listener.
Cluster	A logical group of upstream hosts (endpoints) that Envoy load balances across — analogous to a Kubernetes Service or an upstream block in Nginx.
Endpoint	A single upstream host/instance inside a cluster.
Route	Maps incoming request attributes (path, headers, host) to a target cluster.
Runtime	Dynamic feature-flagging / percentage-based config values Envoy can read at runtime (via RTDS).

Envoy is fundamentally event-driven and non-blocking, running a fixed-size pool of worker threads (--concurrency), each running its own event loop, each capable of independently accepting and fully processing connections. There’s a separate main thread for configuration processing, stats flushing, and admin.

3. Configuration Model: Static vs Dynamic

Envoy configuration is fundamentally protobuf-based, usually authored in YAML or JSON.

Static configuration

Everything defined directly in the bootstrap config file. Simple, good for learning, small/single-purpose proxies, or when config genuinely never changes.

static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address: { address: 0.0.0.0, port_value: 10000 }
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          route_config:
            name: local_route
            virtual_hosts:
            - name: backend
              domains: ["*"]
              routes:
              - match: { prefix: "/" }
                route: { cluster: backend_service }
          http_filters:
          - name: envoy.filters.http.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
  clusters:
  - name: backend_service
    connect_timeout: 5s
    type: STRICT_DNS
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: backend_service
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address: { address: backend.internal, port_value: 8080 }

Dynamic configuration (xDS)

Production Envoy deployments almost always use dynamic configuration, where a control plane (Istio Pilot, go-control-plane based servers, Contour, etc.) pushes config to Envoy over gRPC streams. This is what enables zero-downtime, fleet-wide updates.

dynamic_resources:
  lds_config:
    resource_api_version: V3
    ads: {}
  cds_config:
    resource_api_version: V3
    ads: {}
  ads_config:
    api_type: GRPC
    transport_api_version: V3
    grpc_services:
    - envoy_grpc: { cluster_name: xds_cluster }

Best practice: Even in dynamic setups, the bootstrap config (which defines how Envoy finds its control plane) is static — this is the one piece of config you manage by hand/config-management, everything else flows from xDS.

4. The xDS APIs Explained

xDS = “discovery service” family. Each letter maps to a resource type:

API	Full Name	Discovers
LDS	Listener Discovery Service	Listeners
RDS	Route Discovery Service	Route configurations
CDS	Cluster Discovery Service	Clusters
EDS	Endpoint Discovery Service	Cluster membership (endpoints/IPs)
SDS	Secret Discovery Service	TLS certs/keys, delivered securely at runtime
VHDS	Virtual Host Discovery Service	Individual virtual hosts (fine-grained RDS)
RTDS	Runtime Discovery Service	Runtime feature flags
ECDS	Extension Config Discovery Service	Filter/extension configuration

ADS (Aggregated Discovery Service) multiplexes all of the above over a single gRPC stream — strongly recommended in production to avoid update-ordering races between separate LDS/RDS/CDS/EDS streams.

Update ordering matters

Envoy’s control plane implementations must respect a specific update sequence to avoid blackholing traffic:

CDS (clusters) → EDS (endpoints for those clusters) → LDS (listeners) → RDS (routes referencing clusters)

Adding a new route to a cluster that doesn’t exist yet = requests fail. Good control planes (Istio, go-control-plane’s SnapshotCache) handle this ordering for you, but it’s essential to understand when debugging blackholes during rollout.

State-of-the-World vs Incremental (Delta) xDS

SotW (State of the World): each update sends the full list of resources of a type. Simple, but expensive at scale (thousands of clusters resent on every tiny change).
Delta xDS: only sends the diff (added/updated/removed resources). Essential for very large fleets (Istio uses Delta xDS by default since 1.12+).

Best practice: Use ADS + Delta xDS in any environment with more than a few hundred clusters/routes to control control-plane and data-plane resource usage.

5. Listeners & Filter Chains

A Listener binds to an address and applies a filter chain to every accepted connection.

Filter chain matching

A single listener can have multiple filter chains, selected based on characteristics of the incoming connection — SNI, destination port, source IP range, ALPN, transport protocol (e.g., detecting TLS via tls_inspector).

listeners:
- name: multiplexed_listener
  address:
    socket_address: { address: 0.0.0.0, port_value: 443 }
  listener_filters:
  - name: envoy.filters.listener.tls_inspector
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.filters.listener.tls_inspector.v3.TlsInspector
  filter_chains:
  - filter_chain_match:
      server_names: ["api.example.com"]
    filters: [ ... http_connection_manager for api ... ]
  - filter_chain_match:
      server_names: ["admin.example.com"]
    filters: [ ... http_connection_manager for admin ... ]

This is the mechanism behind SNI-based routing at L4 — Envoy doesn’t even need to terminate TLS to route to different backends by hostname (see TLS passthrough patterns).

Network filters vs HTTP filters

Network filters operate on raw connection bytes (L3/L4): tcp_proxy, http_connection_manager (which bridges L4→L7), redis_proxy, mongo_proxy, RBAC (network-level), rate limiting (network-level).
HTTP filters operate inside the http_connection_manager, once bytes are parsed into HTTP requests/responses: router, cors, jwt_authn, ext_authz, rate_limit, fault, lua, wasm, grpc_web, health_check, compressor, etc.

Order matters. HTTP filters execute in the order listed for the request path, and reverse order for the response path. The router filter (which actually forwards to upstream) must always be last.

http_filters:
- name: envoy.filters.http.jwt_authn        # 1. authenticate first
- name: envoy.filters.http.rbac              # 2. then authorize
- name: envoy.filters.http.ext_authz        # 3. optional external auth check
- name: envoy.filters.http.fault             # 4. fault injection (testing)
- name: envoy.filters.http.cors              # 5. CORS handling
- name: envoy.filters.http.router            # 6. ALWAYS LAST — does the actual proxying

6. Clusters, Endpoints & Service Discovery

A Cluster is Envoy’s abstraction for “a named group of backends I can load balance across.” Envoy supports multiple discovery types:

Type	Description	Use case
`STATIC`	Fixed IP list in config	Testing, truly static infra
`STRICT_DNS`	Resolves DNS, refreshes on TTL, tracks all returned IPs	Traditional DNS-based service discovery
`LOGICAL_DNS`	Resolves DNS but only uses first IP returned; re-resolves per new connection	Large/rotating DNS pools (e.g., cloud LBs)
`EDS`	Dynamic, pushed by control plane via Endpoint Discovery Service	Kubernetes, service mesh — the production default
`ORIGINAL_DST`	Routes to the connection’s original destination (pre-NAT/redirect)	Transparent proxying, sidecar interception

clusters:
- name: payments_service
  connect_timeout: 2s
  type: EDS
  eds_cluster_config:
    eds_config: { ads: {} }
  lb_policy: ROUND_ROBIN
  health_checks:
  - timeout: 1s
    interval: 5s
    unhealthy_threshold: 3
    healthy_threshold: 2
    http_health_check:
      path: /healthz
  circuit_breakers:
    thresholds:
    - priority: DEFAULT
      max_connections: 1000
      max_pending_requests: 1000
      max_requests: 1000
      max_retries: 3

Best practice: Prefer EDS (fed by a real control plane) over DNS-based discovery in Kubernetes — DNS-based discovery has caching/TTL lag issues and doesn’t give you per-endpoint health signals or weighted routing as cleanly.

Health checking: active vs passive

Active health checks — Envoy proactively probes endpoints (http_health_check, tcp_health_check, grpc_health_check) on an interval.
Passive health checks (Outlier Detection) — Envoy watches real traffic and ejects hosts that return errors/timeouts, without extra probe traffic. See Section 9.

Best practice: Run both. Active checks catch dead hosts before traffic hits them; outlier detection catches “gray failure” (host responds, but badly) that active checks might miss.

7. Routing

Route configuration maps requests to clusters, based on virtual hosts (matched by Host/:authority header) and route match rules (path, headers, query params, gRPC method, etc.).

route_config:
  name: main_routes
  virtual_hosts:
  - name: api
    domains: ["api.example.com"]
    routes:
    - match:
        prefix: "/v2/"
        headers:
        - name: "x-canary"
          string_match: { exact: "true" }
      route:
        cluster: api_service_canary
    - match: { prefix: "/v2/" }
      route:
        cluster: api_service_v2
        timeout: 15s
        retry_policy:
          retry_on: "5xx,reset,connect-failure"
          num_retries: 2
    - match: { prefix: "/" }
      route: { cluster: api_service_v1 }

Traffic splitting / weighted clusters

Canary releases and blue/green deploys are done with weighted_clusters:

route:
  weighted_clusters:
    clusters:
    - name: api_service_v1
      weight: 90
    - name: api_service_v2
      weight: 10
  total_weight: 100

Header-based routing patterns

Common production patterns:

Canary by header (x-canary: true) — for internal testers/QA to hit new versions.
A/B testing by cookie or user-id hash — using request_headers_to_add combined with hash-based routing.
Traffic mirroring (shadowing) — send a copy of traffic to a new service without affecting the response to the client:

route:
  cluster: api_service_v1
  request_mirror_policies:
  - cluster: api_service_v2_shadow
    runtime_fraction:
      default_value: { numerator: 10, denominator: HUNDRED }

Mirroring is the safest way to validate a new service version under real production load before it ever serves real responses.

Redirects, rewrites, and direct responses

- match: { prefix: "/old-path" }
  redirect: { path_redirect: "/new-path", response_code: MOVED_PERMANENTLY }
- match: { prefix: "/health" }
  direct_response: { status: 200, body: { inline_string: "OK" } }
- match: { prefix: "/api/" }
  route:
    cluster: backend
    prefix_rewrite: "/"

8. Load Balancing

Envoy supports several LB policies at the cluster level:

Policy	Behavior	Best for
`ROUND_ROBIN`	Cycles through healthy hosts	Simple, uniform backends
`LEAST_REQUEST`	Picks host with fewest active requests (uses power-of-two-choices by default)	Variable request cost/duration — usually the best default
`RANDOM`	Random selection	High-throughput, stateless, simple
`RING_HASH`	Consistent hashing onto a ring	Session affinity / cache-friendly routing
`MAGLEV`	Google’s consistent hashing algorithm, faster table build than ring hash	Large clusters needing consistent hashing at scale
`CLUSTER_PROVIDED`	Delegates to a custom LB implemented at cluster level	Custom LB logic

Best practice: LEAST_REQUEST is the generally recommended default over ROUND_ROBIN for HTTP services with variable latency, because round robin can send requests to an already-overloaded host purely by rotation order.

Locality-aware & zone-aware load balancing

Envoy can prefer routing within the same zone/region as the caller, falling back to other zones only if the local zone lacks capacity — critical for reducing cross-AZ data transfer costs and latency.

cluster:
  common_lb_config:
    locality_weighted_lb_config: {}
    zone_aware_lb_config:
      routing_enabled: { value: 100 }
      min_cluster_size: 6

Session affinity

For stateful backends, RING_HASH or MAGLEV combined with a hash policy (cookie, header, or source IP) gives consistent routing per client:

route:
  cluster: sticky_backend
  hash_policy:
  - cookie:
      name: "session-id"
      ttl: 3600s

9. Resiliency: Retries, Timeouts, Circuit Breaking, Outlier Detection

This is where Envoy earns its reputation as a resilience layer, not just a router.

Timeouts

Two timeout concepts matter and are frequently confused:

Route/request timeout (timeout:) — total time allowed for the entire request/response, including retries.
per_try_timeout — time allowed per individual retry attempt. Must be ≤ overall timeout.

route:
  cluster: backend
  timeout: 10s
  retry_policy:
    retry_on: "5xx,reconnect,connect-failure,refused-stream"
    num_retries: 3
    per_try_timeout: 2s
    retry_back_off:
      base_interval: 0.1s
      max_interval: 1s

Best practice: Always set explicit timeouts. Envoy’s default request timeout is 15s, often too long for latency-sensitive APIs and too short for long-polling/streaming — tune per route.

Retries — the double-edged sword

Retries improve tail latency and reliability for transient errors, but naive retries under load can create retry storms that amplify an outage.

Mitigate with:

retry_budget — caps retries as a percentage of active requests rather than a fixed count, adaptive under load:

retry_policy:
  retry_on: "5xx"
  retry_back_off:
    base_interval: 0.1s
retry_budget:
  budget_percent: { value: 20 }
  min_retry_concurrency: 3

Retrying only idempotent operations (GET, PUT with idempotency keys) — never blindly retry POST unless you know it’s safe.
Combining with circuit breakers so retries stop hammering an already-failing cluster.

Circuit Breaking

Unlike classic “trip after N failures” circuit breakers (e.g., Hystrix), Envoy’s circuit breakers are resource limit gates — they cap concurrent connections/requests/retries to a cluster to protect both Envoy and the upstream from being overwhelmed:

circuit_breakers:
  thresholds:
  - priority: DEFAULT
    max_connections: 1024
    max_pending_requests: 1024
    max_requests: 1024
    max_retries: 3
    track_remaining: true

When a threshold is hit, new requests fail fast (503) instead of queueing indefinitely — a critical protection against cascading failure.

Outlier Detection (passive health checking)

Ejects hosts from the load balancing pool based on observed error rates, without needing an active health-check endpoint:

outlier_detection:
  consecutive_5xx: 5
  interval: 10s
  base_ejection_time: 30s
  max_ejection_percent: 50
  split_external_local_origin_errors: true

Best practice combo: timeouts + bounded retries with retry budgets + circuit breakers + outlier detection together form Envoy’s resilience stack — using only one of these leaves gaps (e.g., retries without circuit breakers can turn a partial outage into a total one).

10. Rate Limiting

Envoy supports both local (per-Envoy-instance, no external dependency) and global (via gRPC rate limit service, shared state across the fleet) rate limiting.

Local rate limiting (token bucket, per-instance)

http_filters:
- name: envoy.filters.http.local_ratelimit
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
    stat_prefix: http_local_rate_limiter
    token_bucket:
      max_tokens: 100
      tokens_per_fill: 100
      fill_interval: 60s
    filter_enabled:
      runtime_key: local_rate_limit_enabled
      default_value: { numerator: 100, denominator: HUNDRED }
    filter_enforced:
      runtime_key: local_rate_limit_enforced
      default_value: { numerator: 100, denominator: HUNDRED }
    response_headers_to_add:
    - append_action: OVERWRITE_IF_EXISTS_OR_ADD
      header: { key: "x-local-rate-limit", value: "true" }

Cheap, fast, no network hop — but each Envoy instance enforces its own bucket, so effective limits scale with replica count.

Global rate limiting (via ext_ratelimit gRPC service)

http_filters:
- name: envoy.filters.http.ratelimit
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
    domain: "api_ratelimit"
    rate_limit_service:
      grpc_service:
        envoy_grpc: { cluster_name: ratelimit_service }
      transport_api_version: V3

Requires deploying a rate-limit service (e.g., envoyproxy/ratelimit backed by Redis) that shares state across the whole fleet.

Best practice: Use local rate limiting as a cheap first line of defense (DoS protection per-instance) and global rate limiting for actual business-logic quota enforcement (e.g., “100 req/min per API key”).

11. TLS, mTLS & Security

Terminating TLS (downstream)

filter_chains:
- transport_socket:
    name: envoy.transport_sockets.tls
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
      common_tls_context:
        tls_certificate_sds_secret_configs:
        - name: server_cert
          sds_config: { ads: {} }
        tls_params:
          tls_minimum_protocol_version: TLSv1_2

Originating TLS to upstream

clusters:
- name: secure_backend
  transport_socket:
    name: envoy.transport_sockets.tls
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
      sni: backend.internal

mTLS (mutual TLS) between services

Requiring client certs enables strong service-to-service identity — the foundation of zero-trust service meshes:

common_tls_context:
  tls_certificate_sds_secret_configs: [...]
  validation_context_sds_secret_config:
    name: validation_context
    sds_config: { ads: {} }
require_client_certificate: true

Best practice: Always deliver TLS certs via SDS (Secret Discovery Service), not inline file paths in static config — SDS allows rotation without restarts, and avoids secrets sitting in plaintext config files/git repos.

Common TLS pitfalls

Forgetting tls_minimum_protocol_version (defaults can be permissive; pin to TLS 1.2+).
SNI mismatches when multiple filter chains share a listener — always test with openssl s_client -servername.
Not rotating SDS certs before expiry — monitor cluster.<name>.ssl.* and cert-expiration stats.

12. Authentication & Authorization (RBAC, ExtAuthz, JWT)

JWT authentication

http_filters:
- name: envoy.filters.http.jwt_authn
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.filters.http.jwt_authn.v3.JwtAuthentication
    providers:
      auth0:
        issuer: "https://example.auth0.com/"
        remote_jwks:
          http_uri:
            uri: "https://example.auth0.com/.well-known/jwks.json"
            cluster: auth0_jwks
            timeout: 5s
          cache_duration: 300s
        forward: true
    rules:
    - match: { prefix: "/api/" }
      requires: { provider_name: "auth0" }

RBAC (role-based access control)

Fine-grained allow/deny rules based on principal (source IP, mTLS identity, JWT claims) and permission (path, method, headers):

http_filters:
- name: envoy.filters.http.rbac
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.filters.http.rbac.v3.RBAC
    rules:
      action: ALLOW
      policies:
        "admin-access":
          permissions:
          - and_rules:
              rules:
              - url_path: { path: { prefix: "/admin" } }
          principals:
          - authenticated:
              principal_name: { exact: "spiffe://cluster.local/ns/default/sa/admin" }

External authorization (ext_authz)

Delegates the allow/deny decision to an external gRPC or HTTP service — useful for complex policy (OPA/Open Policy Agent is a very common pairing):

http_filters:
- name: envoy.filters.http.ext_authz
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthz
    grpc_service:
      envoy_grpc: { cluster_name: opa_authz }
      timeout: 0.25s
    failure_mode_allow: false

Best practice: failure_mode_allow: false is the secure default — if the authz service is unreachable, deny by default rather than fail open. Only set true in explicitly non-critical paths.

13. Observability: Stats, Access Logs, Tracing

Envoy is exceptionally observable out of the box — this is one of its defining strengths.

Stats

Three kinds: counters, gauges, histograms — exposed via /stats, /stats/prometheus, or pushed via StatsD/dogstatsd sinks.

Key stats to alert on in production:

cluster.<name>.upstream_rq_5xx / upstream_rq_timeout — upstream failures
cluster.<name>.upstream_cx_connect_fail — connection failures to backends
cluster.<name>.circuit_breakers.default.rq_open — circuit breaker trips
cluster.<name>.outlier_detection.ejections_active — hosts currently ejected
listener.<addr>.downstream_cx_overflow — listener backlog/overload
server.memory_allocated, server.concurrency — resource pressure

stats_sinks:
- name: envoy.stat_sinks.metrics_service
  typed_config:
    "@type": type.googleapis.com/envoy.config.metrics.v3.MetricsServiceConfig
    grpc_service:
      envoy_grpc: { cluster_name: stats_sink }

Best practice: Use stats_matcher to filter which stats are actually emitted at high cluster/route counts — unrestricted per-endpoint stats can become a real memory/cardinality problem at scale.

Access logging

Structured, configurable per listener/filter chain:

access_log:
- name: envoy.access_loggers.file
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
    path: "/dev/stdout"
    log_format:
      json_format:
        start_time: "%START_TIME%"
        method: "%REQ(:METHOD)%"
        path: "%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%"
        response_code: "%RESPONSE_CODE%"
        upstream_cluster: "%UPSTREAM_CLUSTER%"
        duration_ms: "%DURATION%"
        upstream_host: "%UPSTREAM_HOST%"
        response_flags: "%RESPONSE_FLAGS%"

%RESPONSE_FLAGS% is gold for debugging — it tells you why a request failed (UO = upstream overflow, UF = upstream connection failure, UT = upstream timeout, NR = no route, RL = rate limited, etc.) without needing to correlate with upstream logs.

Distributed tracing

Envoy supports Zipkin, Jaeger, Datadog, OpenTelemetry, and AWS X-Ray natively:

tracing:
  provider:
    name: envoy.tracers.opentelemetry
    typed_config:
      "@type": type.googleapis.com/envoy.config.trace.v3.OpenTelemetryConfig
      grpc_service:
        envoy_grpc: { cluster_name: otel_collector }
      service_name: "checkout-service"

Envoy will propagate/generate trace headers (traceparent, x-b3-*) automatically, but applications must forward incoming trace headers on their own outbound calls — Envoy can’t stitch spans across your app’s business logic.

14. HTTP Filters & Extensibility (Wasm, Lua, ext_proc)

When built-in filters aren’t enough, Envoy offers three extensibility mechanisms, in increasing order of flexibility/complexity:

Lua filter (fast to write, in-process)

http_filters:
- name: envoy.filters.http.lua
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua
    default_source_code:
      inline_string: |
        function envoy_on_request(request_handle)
          request_handle:headers():add("x-custom-header", "hello")
        end

Good for lightweight header manipulation, simple custom logic, prototyping.

WebAssembly (Wasm) filter

Compile filters in Rust, C++, AssemblyScript, TinyGo — sandboxed, portable across Envoy versions, hot-swappable without rebuilding Envoy itself.

http_filters:
- name: envoy.filters.http.wasm
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
    config:
      name: "my_filter"
      vm_config:
        runtime: "envoy.wasm.runtime.v8"
        code:
          local: { filename: "/etc/envoy/filters/my_filter.wasm" }

Best practice: Wasm is the right choice for reusable, versioned, org-wide filters (e.g., a standard auth-header-injection filter shared across 50 teams) — it decouples filter logic from the Envoy binary release cycle.

External Processing (ext_proc)

Streams request/response data to an out-of-process gRPC service for arbitrary transformation — the most powerful and highest-latency option, ideal for complex business logic that shouldn’t live in the proxy itself:

http_filters:
- name: envoy.filters.http.ext_proc
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.filters.http.ext_proc.v3.ExternalProcessor
    grpc_service:
      envoy_grpc: { cluster_name: ext_proc_service }
    processing_mode:
      request_header_mode: SEND
      response_header_mode: SEND

Choosing between them: Lua for quick in-process tweaks → Wasm for portable/reusable filters shared across teams → ext_proc for heavy/complex logic best kept out of the data plane’s hot path (accepting network-hop latency in exchange for full language/library freedom).

15. Deployment Patterns

1. Sidecar proxy (service mesh)

One Envoy instance per application pod/instance, intercepting all inbound/outbound traffic transparently (usually via iptables redirect to ORIGINAL_DST cluster). This is the Istio/Consul Connect/App Mesh model. Gives per-service mTLS, retries, observability without app code changes.

2. Edge/gateway proxy

A shared fleet of Envoy instances at the perimeter, handling TLS termination, routing to internal services, auth, rate limiting. This is the “API Gateway” pattern (Envoy Gateway, Contour, Gloo, Emissary-ingress, Istio Ingress Gateway).

3. Front proxy + TLS passthrough (double proxy / SNI routing)

An edge Envoy that doesn’t terminate TLS at all, just routes based on SNI (via tcp_proxy + tls_inspector) to internal Envoy instances that do terminate TLS — used when the edge shouldn’t hold private keys, or clients need true end-to-end TLS.

4. Standalone L4/L7 load balancer

Replacing HAProxy/Nginx for internal load balancing — often simpler because config generation tooling (control planes) can drive updates without reloads.

5. Database/protocol proxy

Envoy’s redis_proxy, mongo_proxy, thrift_proxy, dubbo_proxy, and kafka_broker filters let it proxy non-HTTP protocols too — giving connection pooling, observability, and auth for these protocols the same way it does for HTTP.

Best practice for sidecar meshes: Watch resource overhead — each sidecar consumes CPU/memory per pod. Tune concurrency, use LEAST_REQUEST LB, and disable unused stats/filters (unnecessary Wasm/Lua execution on every request adds real per-request latency).

16. Service Mesh Patterns (Istio, Gateway API)

Most people’s first encounter with Envoy today is through Istio, which uses Envoy exclusively as its data plane, with istiod as the control plane pushing xDS config.

Istio-specific concepts layered on top of raw Envoy

VirtualService → compiles down to Envoy RouteConfiguration (RDS).
DestinationRule → compiles down to Envoy Cluster config (LB policy, circuit breakers, TLS settings, subsets).
Gateway → compiles down to Envoy Listener config at mesh ingress/egress.
PeerAuthentication / AuthorizationPolicy → compiles down to Envoy RBAC filter config + mTLS settings.
EnvoyFilter → an escape hatch to patch raw Envoy config directly when Istio’s abstractions don’t expose something you need — powerful but fragile across Istio upgrades; use sparingly and pin carefully.

Kubernetes Gateway API

The newer, vendor-neutral standard (superseding Ingress) — several implementations use Envoy as the underlying data plane (Envoy Gateway is the reference implementation maintained by the Envoy project itself, plus Istio’s Gateway API support, GKE Gateway, etc.). It brings role-oriented resources (GatewayClass, Gateway, HTTPRoute, TCPRoute) that map cleanly onto Envoy’s Listener/Route model.

Best practice: If you’re adopting a mesh purely for mTLS + observability and don’t need advanced traffic shifting, consider whether a simpler sidecar-less mesh (ambient mode in Istio, using per-node Envoy “ztunnel” + waypoint proxies) meets your needs with less per-pod overhead than classic sidecars.

17. Performance Tuning

--concurrency N — set to match available CPU cores (or slightly under, leaving headroom for control-plane/admin work). Too many worker threads causes context-switch overhead; too few underutilizes CPU.
Connection pooling — tune max_connections, max_requests_per_connection on clusters; HTTP/2 multiplexing (http2_protocol_options) reduces connection churn to upstreams that support it.
Buffer limits — per_connection_buffer_limit_bytes controls backpressure; too high risks memory bloat under load, too low causes premature connection resets on bursty traffic.
Disable unused stats — high-cardinality per-endpoint/per-route stats can dominate memory at scale; use stats_matcher exclusion lists.
Avoid excessive Lua/Wasm in the hot path — every filter adds latency; benchmark with wrk/nighthawk (Envoy’s own load-testing tool) before and after adding custom filters.
HTTP/2 & HTTP/3 (QUIC) — enabling http2_protocol_options upstream and codec_type: HTTP3 downstream (with UDP listener) can significantly cut connection-setup latency for mobile/high-latency clients.
Hot restart / draining — configure drain_time and use SIGTERM-based graceful shutdown so in-flight requests complete during rolling deploys instead of being cut off.

Use Nighthawk (Envoy’s companion load generator) for realistic Envoy-aware benchmarking rather than generic tools when you need Envoy-specific metrics correlation.

18. Operational Best Practices

Always use ADS with a single gRPC stream in production, not separate LDS/RDS/CDS/EDS connections — avoids config skew/ordering races.
Version and canary your control plane changes — a bad xDS push can be fleet-wide instantly; roll control-plane config changes progressively just like app deploys.
Set explicit timeouts everywhere — never rely on defaults for production routes.
Pair retries with circuit breakers and retry budgets — never ship “num_retries” alone.
Use SDS for all certificates — never bake certs into static config or images.
Alert on RESPONSE_FLAGS, circuit breaker trips, and outlier ejections, not just raw 5xx counts — flags tell you why.
Keep failure_mode_allow: false for ext_authz on security-critical paths.
Use stats_matcher to control cardinality once you cross a few hundred clusters/routes.
Drain connections gracefully on shutdown/deploy (drain_time, listener drain, healthcheck-fail-before-SIGTERM patterns).
Pin Envoy/Istio versions carefully and read release notes — deprecated/removed extensions (Envoy has a documented deprecation policy, typically 2 minor versions) can silently break configs on upgrade.
Test config changes with envoy --mode validate before rolling out, and use the admin /config_dump endpoint to verify what’s actually active.
Use the admin interface’s /clusters, /listeners, /stats, /server_info endpoints as your first debugging stop — don’t guess, inspect actual runtime state.

19. Common Pitfalls & Anti-Patterns

Pitfall	Why It Hurts	Fix
Retries without circuit breakers	Amplifies partial outages into total outages (“retry storm”)	Add `retry_budget` + `circuit_breakers`
Static TLS certs in config files	Can’t rotate without restart; secrets in git/config-management	Use SDS
No explicit route timeouts	Defaults may be wrong for your latency profile; hung requests pile up	Set `timeout` and `per_try_timeout` per route
Separate LDS/RDS/CDS/EDS streams	Update-ordering races cause transient 503s during rollout	Use ADS (single aggregated stream)
Unbounded per-endpoint stats at scale	Memory/cardinality explosion with large fleets	`stats_matcher` exclusions
`failure_mode_allow: true` on auth paths	Fails open when authz service is down — security hole	Set `false` on critical paths
Ignoring `%RESPONSE_FLAGS%` in logs	Miss the actual root cause of failures (UO/UF/UT/NR/etc.)	Add to access log format, alert on them
Blindly retrying non-idempotent POSTs	Can cause duplicate side effects (double charges, etc.)	Restrict `retry_on`, use idempotency keys
One giant filter chain instead of chain matching	Hard to reason about, hard to reuse across domains	Use `filter_chain_match` for SNI/port-based splits
Treating EnvoyFilter (Istio) as a first resort	Fragile across Istio upgrades, hard to audit	Prefer native Istio CRDs; use EnvoyFilter sparingly

20. Quick Reference Cheat Sheet

# Validate config without running
envoy --mode validate -c envoy.yaml

# Run with N worker threads
envoy -c envoy.yaml --concurrency 4

# Admin endpoints (default port 9901)
GET /stats                  → all stats (text)
GET /stats/prometheus       → Prometheus-formatted stats
GET /clusters               → cluster health/membership
GET /listeners              → active listeners
GET /config_dump            → currently active config (post-xDS)
GET /server_info            → version, uptime, state
POST /healthcheck/fail      → mark this Envoy as unhealthy (graceful drain)
POST /drain_listeners       → begin draining connections

# Key response flags (access logs)
UO  = Upstream Overflow (circuit breaker tripped)
UF  = Upstream connection Failure
UT  = Upstream request Timeout
NR  = No Route configured
RL  = Rate Limited
UAEX = Unauthorized (ext_authz denied)
LR  = Connection Local Reset
DC  = Downstream connection termination

Recommended default resilience stack for any production route

route:
  timeout: 10s
  retry_policy:
    retry_on: "5xx,reset,connect-failure,refused-stream"
    num_retries: 2
    per_try_timeout: 3s
    retry_back_off: { base_interval: 0.1s, max_interval: 1s }
    retry_budget:
      budget_percent: { value: 20 }
      min_retry_concurrency: 3
cluster:
  circuit_breakers:
    thresholds:
    - priority: DEFAULT
      max_connections: 1000
      max_pending_requests: 1000
      max_requests: 1000
      max_retries: 3
  outlier_detection:
    consecutive_5xx: 5
    interval: 10s
    base_ejection_time: 30s
    max_ejection_percent: 50
  health_checks:
  - timeout: 1s
    interval: 5s
    unhealthy_threshold: 3
    healthy_threshold: 2
    http_health_check: { path: /healthz }

Table of Contents