A reverse proxy sits between clients and one or more upstream services. Instead of clients communicating directly with your application, every request first passes through the proxy before being forwarded to an upstream.
Mature reverse proxies such as Nginx, Envoy, and HAProxy do much more than simply forward requests. They perform tasks such as load balancing, health checks, rate limiting, metrics collection, and much more.
I wanted to better understand how some of these concepts work in practice, so I built a reverse proxy in Go. Along the way I implemented request forwarding, multiple load-balancing strategies, health checks, circuit breakers, rate limiting, request logging, metrics, and graceful shutdown.
If you’d like to explore Proxify as we go, you can find the project here:
https://github.com/Rahmannugar/proxify
Table of Contents
- Request Lifecycle
- Project Structure
- Configuration
- Reverse Proxy
- Load Balancing
- Health Checks
- Circuit Breakers
- Middleware
- Graceful Shutdown
- Running Proxify with Docker
1. Request Lifecycle
At a high level, every request follows the same path through the reverse proxy.
A client sends an HTTP request to Proxify instead of communicating directly with an upstream service. Proxify receives the request, selects a healthy upstream using the configured load-balancing strategy, forwards the request, waits for the upstream’s response, and finally returns that response to the client.
Client
│
▼
+---------------+
| Proxify |
+---------------+
│
Select Healthy Upstream
│
┌───────┴────────┐
▼ ▼
Upstream A Upstream B
│
▼
HTTP Response
│
▼
Client
Enter fullscreen mode Exit fullscreen mode
Although the overall flow is straightforward, every step introduces additional considerations.
- Which upstream should receive the next request?
- What happens when an upstream becomes unhealthy?
- How can requests be distributed efficiently across multiple upstreams?
- How do we prevent a failing upstream from continuing to receive traffic?
The remainder of this article answers those questions by gradually building each component of the reverse proxy.
2. Project Structure
The project is organized into small, focused packages, with each package responsible for a single part of the reverse proxy.
.
├── cmd/
│ └── proxify/
├── configs/
├── examples/
│ └── upstream/
├── internal/
│ ├── circuitbreaker/
│ ├── config/
│ ├── exporter/
│ ├── health/
│ ├── loadbalancer/
│ ├── metrics/
│ ├── middleware/
│ ├── proxy/
│ ├── ratelimiter/
│ ├── registry/
│ ├── requestid/
│ ├── server/
│ ├── transport/
│ └── upstream/
└── Dockerfile
Enter fullscreen mode Exit fullscreen mode
-
cmd/contains the application entry point. -
config/loads and validates configuration. -
registry/manages the configured upstream services. -
proxy/forwards incoming requests. -
loadbalancer/selects which upstream should receive a request. -
health/periodically checks upstream availability. -
circuitbreaker/prevents unhealthy upstreams from receiving continuous traffic. -
middleware/contains reusable HTTP middleware such as request logging and rate limiting. -
metrics/collects runtime metrics. -
server/configures and starts the HTTP server.
3. Configuration
Rather than hardcoding values into the application, Proxify is configured using YAML. This makes it easy to change the proxy’s behavior without modifying the code.
server:
port: 8080
loadBalancer:
strategy: round-robin
health:
endpoint: /health
interval: 10s
timeout: 2s
retries: 3
expectedStatus: 200
timeouts:
read: 10s
readHeader: 5s
write: 30s
idle: 60s
transport:
dial: 5s
responseHeader: 10s
idleConn: 90s
tlsHandshake: 5s
rateLimit:
requestsPerSecond: 10
burst: 20
upstreams:
- id: api-1
url: http://localhost:9001
- id: api-2
url: http://localhost:9002
Enter fullscreen mode Exit fullscreen mode
On startup, Proxify loads the configuration file, validates it, and uses it to initialize the reverse proxy before the server begins accepting requests.
4. Reverse Proxy
The core responsibility of a reverse proxy is straightforward: receive an incoming request, forward it to an upstream service, wait for the response, and return that response to the client.
Go’s standard library already provides a reverse proxy implementation through httputil.ReverseProxy, so rather than implementing HTTP forwarding from scratch, Proxify builds on top of it.
Each configured upstream owns its own reverse proxy instance, allowing requests to be forwarded while reusing persistent connections through a shared HTTP transport.
type Upstream struct {
ID string
URL string
ReverseProxy *httputil.ReverseProxy
...
}
Enter fullscreen mode Exit fullscreen mode
When a request reaches Proxify, the proxy first asks the configured load balancer to select an upstream. If no healthy upstream is available, the request fails with a 503 Service Unavailable.
Otherwise, the selected upstream’s reverse proxy forwards the request.
func (p *Proxy) ServeHTTP(w http.ResponseWriter, r *http.Request) {
upstream, err := p.loadBalancer.Next()
if err != nil {
http.Error(
w,
err.Error(),
http.StatusServiceUnavailable,
)
return
}
upstream.ReverseProxy.ServeHTTP(w, r)
}
Enter fullscreen mode Exit fullscreen mode
Although forwarding a request is relatively simple, deciding which upstream should receive that request is where things become more interesting.
5. Load Balancing
If every request were forwarded to the same upstream, the remaining upstreams would sit idle. Load balancing solves this problem by distributing requests across multiple upstream services.
Proxify supports two load-balancing strategies:
- Round Robin
- Least Outstanding Requests
The strategy is selected through configuration.
loadBalancer:
strategy: round-robin
Enter fullscreen mode Exit fullscreen mode
Round Robin
Round Robin is one of the simplest load-balancing algorithms.
Each incoming request is forwarded to the next healthy upstream in sequence. Once the last upstream has been selected, the algorithm starts again from the beginning.
For example, with three upstreams:
Request 1 → API 1
Request 2 → API 2
Request 3 → API 3
Request 4 → API 1
Request 5 → API 2
Enter fullscreen mode Exit fullscreen mode
This approach distributes traffic evenly and works well when upstreams have similar capacity and requests take roughly the same amount of time to complete.
Least Outstanding Requests
Not every request has the same execution time.
Some requests may complete in a few milliseconds while others may take several seconds. In those situations, simply rotating through upstreams may result in one server handling significantly more work than another.
The Least Outstanding Requests strategy selects the healthy upstream currently handling the fewest active requests.
API 1 → 4 active requests
API 2 → 1 active request
API 3 → 2 active requests
Next request → API 2
Enter fullscreen mode Exit fullscreen mode
As requests begin and complete, each upstream maintains a count of its active requests, allowing the load balancer to always choose the least busy healthy upstream.
Both strategies skip unhealthy upstreams. If no healthy upstream is available, the proxy returns a 503 Service Unavailable response.
6. Health Checks
A load balancer is only as good as the information it has about the services behind it. If it continues routing requests to an unavailable upstream, clients will continue receiving failed requests.
To avoid this, Proxify periodically performs health checks against every configured upstream.
Every 10 Seconds
+-------------------+
| Proxify |
+-------------------+
│ │
GET /health GET /health
│ │
▼ ▼
API 1 API 2
Enter fullscreen mode Exit fullscreen mode
Each health check sends an HTTP request to the configured health endpoint. If the upstream responds with the expected status code, it is marked as healthy. Otherwise, it is marked as unhealthy.
health:
endpoint: /health
interval: 10s
timeout: 2s
expectedStatus: 200
Enter fullscreen mode Exit fullscreen mode
The load balancer only considers healthy upstreams when selecting where to forward requests. If every upstream becomes unhealthy, the proxy returns a 503 Service Unavailable response until at least one upstream recovers.
Separating health checks from the load-balancing logic keeps each component focused on a single responsibility. The health checker determines which upstreams are available, while the load balancer simply routes requests using that information.
7. Circuit Breakers
Health checks periodically determine whether an upstream is available, but failures can still occur between health check intervals.
For example, an upstream might become overloaded or begin returning errors immediately after a successful health check. Continuing to send requests to that upstream only increases the number of failed requests.
Circuit breakers help prevent this.
Each upstream owns its own circuit breaker that tracks request failures and successes.
Request
│
▼
Circuit Breaker
│
┌──────┴──────┐
▼ ▼
Allow Reject
│
▼
Upstream
Enter fullscreen mode Exit fullscreen mode
Proxify implements the three standard circuit breaker states:
- Closed — Requests are forwarded normally.
- Open — Requests are rejected immediately after the failure threshold has been reached.
- Half-Open — After a recovery timeout, a small number of requests are allowed through to determine whether the upstream has recovered.
Closed ── failures ──► Open
▲ │
│ │ recovery timeout
│ ▼
└──── successes ◄── Half-Open
Enter fullscreen mode Exit fullscreen mode
If the trial requests succeed, the circuit closes and traffic resumes normally. If they fail, the circuit immediately returns to the open state.
This prevents continuously sending requests to an upstream that is already failing, giving it time to recover before it begins receiving traffic again.
8. Middleware
Before a request reaches the reverse proxy, it passes through a chain of HTTP middleware.
Each middleware is responsible for a single concern, allowing cross-cutting functionality to be added without complicating the proxy itself.
Incoming Request
│
▼
Request ID Middleware
│
▼
Logging Middleware
│
▼
Rate Limiting Middleware
│
▼
Reverse Proxy
Enter fullscreen mode Exit fullscreen mode
Request IDs
Every incoming request is assigned a unique request ID.
The request ID is attached to the response and can also be included in logs, making it easier to trace a request as it passes through the system.
Logging
Request logging provides visibility into how the proxy is being used.
For every request, Proxify records information such as the request method, path, response status, duration, and the upstream that handled the request.
This makes it significantly easier to debug issues and understand traffic flowing through the proxy.
Rate Limiting
A reverse proxy often serves as the first point of contact for incoming traffic, making it a natural place to apply rate limiting.
Proxify uses a token bucket rate limiter to control how many requests a client can make over time.
rateLimit:
requestsPerSecond: 10
burst: 20
Enter fullscreen mode Exit fullscreen mode
If a client exceeds the configured rate limit, the proxy immediately responds with 429 Too Many Requests instead of forwarding the request to an upstream.
Applying these concerns as middleware keeps the reverse proxy focused on request forwarding while allowing additional behavior to be composed around it.
9. Graceful Shutdown
A reverse proxy should be able to stop accepting traffic without abruptly terminating requests that are already in progress.
To achieve this, Proxify performs a graceful shutdown when it receives a termination signal.
Rather than immediately exiting, the server stops accepting new connections while allowing existing requests to complete before shutting down.
SIGINT / SIGTERM
│
▼
Stop Accepting Requests
│
▼
Finish Active Requests
│
▼
Shutdown
Enter fullscreen mode Exit fullscreen mode
This allows Proxify to exit cleanly without unnecessarily interrupting requests that are already being processed.
10. Running Proxify with Docker
Although Proxify can be run directly with Go, I also wanted an easy way to run the entire project locally.
Docker Compose starts the reverse proxy alongside two upstream services, allowing the proxy to distribute traffic immediately after startup.
Client
│
▼
+---------------+
| Proxify |
+---------------+
│ │
▼ ▼
Upstream 1 Upstream 2
Enter fullscreen mode Exit fullscreen mode
To start everything:
docker compose up --build
Enter fullscreen mode Exit fullscreen mode
Once the containers are running, requests can be sent to the proxy on port 8080.
curl localhost:8080
Enter fullscreen mode Exit fullscreen mode
Successive requests are distributed across the configured upstreams according to the selected load-balancing strategy.
Switching between strategies is as simple as changing the configuration and restarting the application.
loadBalancer:
strategy: round-robin
Enter fullscreen mode Exit fullscreen mode
or
loadBalancer:
strategy: least-outstanding
Enter fullscreen mode Exit fullscreen mode
답글 남기기