Load Balancing on AWS and GCP: A Practical Guide

Choosing and configuring the right managed load balancer for your cloud architecture

Introduction

Both AWS and GCP offer mature, fully managed load balancing services — removing the need to run and patch your own HAProxy or NGINX fleet. But each cloud has its own naming, tiers, and quirks, and picking the wrong one can mean paying for capabilities you don’t need or missing ones you do. This guide walks through the load balancing options on both platforms, when to use each, and how to configure them.

1. The Load Balancer Landscape

Layer AWS GCP Layer 7 (HTTP/HTTPS) Application Load Balancer (ALB) External/Internal HTTP(S) Load Balancer Layer 4 (TCP/UDP) Network Load Balancer (NLB) External/Internal TCP/UDP Network Load Balancer Legacy/basic Classic Load Balancer (CLB) — legacy, avoid for new builds (none — GCP moved fully to the above two) Global anycast Global Accelerator Global External HTTP(S) Load Balancer (native) Service mesh / internal AWS App Mesh + NLB/ALB Internal HTTP(S) LB + Traffic Director

Key distinction: GCP’s HTTP(S) Load Balancer is global by default — a single anycast IP can front backends in multiple regions. AWS’s ALB and NLB are regional; global reach requires layering Global Accelerator or Route 53 on top.

2. AWS Load Balancing Options

Application Load Balancer (ALB)

Layer 7, HTTP/HTTPS/gRPC-aware. Best for microservices, path-based routing, and containerized workloads (ECS/EKS).

aws elbv2 create-load-balancer \
  --name my-app-alb \
  --subnets subnet-0123abcd subnet-0456efgh \
  --security-groups sg-0789ijkl \
  --scheme internet-facing \
  --type application

Enter fullscreen mode Exit fullscreen mode

Routing rules let you send traffic to different target groups based on path or host:

aws elbv2 create-rule \
  --listener-arn <listener-arn> \
  --priority 10 \
  --conditions Field=path-pattern,Values='/api/*' \
  --actions Type=forward,TargetGroupArn=<api-target-group-arn>

Enter fullscreen mode Exit fullscreen mode

Network Load Balancer (NLB)

Layer 4, ultra-low latency, handles millions of requests per second, preserves client source IP. Best for TCP/UDP workloads, gaming, or when raw throughput matters more than content-aware routing.

aws elbv2 create-load-balancer \
  --name my-tcp-nlb \
  --subnets subnet-0123abcd subnet-0456efgh \
  --type network \
  --scheme internet-facing

Enter fullscreen mode Exit fullscreen mode

Global Accelerator

Uses AWS’s global network backbone and anycast IPs to route users to the nearest healthy regional endpoint (which can itself be an ALB or NLB). Useful for multi-region failover and reducing latency for globally distributed users.

aws globalaccelerator create-accelerator \
  --name my-global-app \
  --ip-address-type IPV4 \
  --enabled

Enter fullscreen mode Exit fullscreen mode

Health Checks (ALB/NLB)

aws elbv2 create-target-group \
  --name my-targets \
  --protocol HTTP \
  --port 80 \
  --vpc-id vpc-0abc123 \
  --health-check-path /health \
  --health-check-interval-seconds 15 \
  --healthy-threshold-count 2 \
  --unhealthy-threshold-count 3

Enter fullscreen mode Exit fullscreen mode

3. GCP Load Balancing Options

External HTTP(S) Load Balancer (Global)

Layer 7, globally distributed via a single anycast IP, backed by Google’s edge network. Best default choice for public-facing web apps and APIs.

# Create a health check
gcloud compute health-checks create http my-health-check \
  --port 80 --request-path=/health

# Create a backend service
gcloud compute backend-services create my-backend-service \
  --protocol=HTTP \
  --health-checks=my-health-check \
  --global

# Add an instance group or NEG as a backend
gcloud compute backend-services add-backend my-backend-service \
  --instance-group=my-instance-group \
  --instance-group-zone=us-central1-a \
  --global

# Create URL map, proxy, and forwarding rule
gcloud compute url-maps create my-url-map \
  --default-service=my-backend-service

gcloud compute target-http-proxies create my-http-proxy \
  --url-map=my-url-map

gcloud compute forwarding-rules create my-http-rule \
  --global \
  --target-http-proxy=my-http-proxy \
  --ports=80

Enter fullscreen mode Exit fullscreen mode

Internal HTTP(S) Load Balancer

Same Layer 7 capabilities as above, but scoped to a VPC — ideal for internal microservice-to-microservice traffic within GKE or Compute Engine.

External/Internal TCP/UDP Network Load Balancer

Layer 4 pass-through balancing, preserves source IP, minimal latency overhead. Choose this for non-HTTP protocols or when you need raw packet forwarding performance.

gcloud compute forwarding-rules create my-tcp-rule \
  --region=us-central1 \
  --ports=443 \
  --target-pool=my-target-pool

Enter fullscreen mode Exit fullscreen mode

GKE-Native Load Balancing

On GKE, a standard Service of type LoadBalancer automatically provisions a GCP Network Load Balancer:

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  type: LoadBalancer
  selector:
    app: my-app
  ports:
    - port: 80
      targetPort: 8080

Enter fullscreen mode Exit fullscreen mode

For Layer 7 routing on GKE, an Ingress resource provisions a Global External HTTP(S) Load Balancer automatically:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
  annotations:
    kubernetes.io/ingress.class: "gce"
spec:
  rules:
    - http:
        paths:
          - path: /api/*
            pathType: ImplementationSpecific
            backend:
              service:
                name: api-service
                port:
                  number: 80

Enter fullscreen mode Exit fullscreen mode

4. Choosing Between AWS and GCP Options

Your Need AWS GCP Public HTTP(S) web app, single region ALB External HTTP(S) LB (regional backend, still global IP) Public HTTP(S) app, multi-region, one IP ALB + Global Accelerator External HTTP(S) LB (global by default) Raw TCP/UDP, high throughput, preserve client IP NLB External/Internal TCP/UDP LB Internal service-to-service traffic ALB/NLB with internal scheme Internal HTTP(S) or TCP/UDP LB Kubernetes workloads ALB/NLB via AWS Load Balancer Controller (EKS) Native via GKE Ingress/Service gRPC ALB (native support) External/Internal HTTP(S) LB (native support)

5. Health Checks and Failover

Both platforms let you tune failure detection sensitivity — keep this in mind so a single blip doesn’t remove a healthy backend, but real failures are caught quickly.

• AWS: --healthy-threshold-count / --unhealthy-threshold-count on target groups, plus --health-check-interval-seconds.
• GCP: --check-interval and --unhealthy-threshold on gcloud compute health-checks.

# GCP: tune check sensitivity
gcloud compute health-checks update http my-health-check \
  --check-interval=10s \
  --unhealthy-threshold=3 \
  --healthy-threshold=2

Enter fullscreen mode Exit fullscreen mode

Recommendation: start with a 2-failure threshold to mark unhealthy and a 2-success threshold to mark healthy again — aggressive enough to react fast, conservative enough to avoid flapping.

6. TLS Termination

Both clouds support terminating TLS at the load balancer, offloading certificate management from your application servers.

AWS (ALB with ACM certificate):

aws elbv2 create-listener \
  --load-balancer-arn <alb-arn> \
  --protocol HTTPS \
  --port 443 \
  --certificates CertificateArn=<acm-cert-arn> \
  --default-actions Type=forward,TargetGroupArn=<target-group-arn>

Enter fullscreen mode Exit fullscreen mode

GCP (HTTP(S) LB with Google-managed certificate):

gcloud compute ssl-certificates create my-cert \
  --domains=example.com \
  --global

gcloud compute target-https-proxies create my-https-proxy \
  --url-map=my-url-map \
  --ssl-certificates=my-cert

Enter fullscreen mode Exit fullscreen mode

Both AWS Certificate Manager and Google-managed SSL certificates auto-renew, so once configured correctly this is largely maintenance-free.

Closing Thoughts

AWS and GCP take slightly different philosophies — AWS gives you regional building blocks (ALB, NLB) that you compose into a global architecture with Global Accelerator, while GCP’s HTTP(S) Load Balancer is global by default. Neither is strictly better; the right choice depends on whether your traffic is HTTP-aware or raw TCP/UDP, whether you need global anycast reach, and how your workloads are deployed. Once you match the load balancer type to the traffic pattern, both platforms make the operational side — health checks, TLS, scaling — largely hands-off.