API Gateway vs Load Balancer: What Is the Difference?

Ask five engineers to explain the difference between an API gateway and a load balancer and you will get five overlapping answers, most of them partly right and partly muddled. The confusion is understandable. Both components live at the edge of your system, in front of the servers that actually do the work. Both receive an incoming request and decide what happens next. Both can appear as a single rectangle labelled "entry point" on a whiteboard. But they solve fundamentally different problems, they operate at different layers of the network stack, and treating one as a substitute for the other is a reliable way to build something fragile. This article draws the line clearly.

The short version: a load balancer spreads traffic across a pool of identical servers so no single one is overwhelmed. An API gateway is a smart front door for your APIs that inspects each request and applies application-level policy such as authentication, rate limiting and routing to different backend services. A load balancer answers "which server should handle this?" An API gateway answers "what is this request, is it allowed, and where in my system does it belong?" This distinction is one piece of a larger picture. If you want the full landscape of how services connect, start with the enterprise system integration pillar.

1. Why the two get confused

The confusion has real roots, so it is worth naming them rather than dismissing them. First, both components sit in the same physical position in a diagram: between the client and the servers. When you draw the request flow, the load balancer and the API gateway occupy the same slot on the page, and it is natural to assume two things in the same position must do the same job.

Second, their feature sets overlap at the edges. A modern load balancer can inspect a URL path and route based on it. A modern API gateway almost always distributes traffic across multiple backend instances, which looks a lot like load balancing. Vendors add features across the boundary until the marketing pages read as if the two products are interchangeable. They are not, but the overlap is real and it fuels the muddle.

Third, the same piece of software can wear both hats. Products such as NGINX, HAProxy and Envoy can be configured as a plain load balancer, as an API gateway, or as something in between. When one binary can play both roles, people stop distinguishing the roles. The cleanest way to cut through all of this is to stop thinking about products and start thinking about jobs. What is the problem each one exists to solve? Once you frame it that way, the boundary becomes obvious. For a grounding in the request-and-response model that both of these sit on top of, the REST API explained pillar is a useful companion.

2. What a load balancer does

A load balancer solves one problem: you have more traffic than a single server can handle reliably, so you run several identical servers and you need something to spread the requests across them evenly. That something is the load balancer. It receives every incoming connection and forwards it to one of the servers in a pool, choosing which one according to an algorithm such as round robin, least connections or weighted distribution.

The servers behind a load balancer are, in the classic case, interchangeable. Each one runs the same application and can serve the same request. The load balancer does not care what the request contains; it cares only about keeping the pool balanced and the servers healthy. That health check is the second core job. The load balancer continuously probes each server, and when one stops responding it removes that server from rotation and sends traffic only to the healthy remainder. When the server recovers, it returns to the pool. This is what gives you high availability: a single server can fail without the users noticing.

A third common job is session persistence, sometimes called sticky sessions, where the load balancer sends a given client back to the same server it used before, which matters when the server holds state for that user. And a fourth, when the load balancer sits at the TLS boundary, is SSL termination, where it decrypts inbound HTTPS so the backend servers can work in plain HTTP behind it.

The diagram below shows the mental model. On the left, a load balancer takes one stream of traffic and fans it out across identical servers. On the right, for contrast, an API gateway takes requests and routes each one to a different specialised service while handling API concerns along the way. Hold both pictures side by side and the difference in purpose becomes clear.

The key phrase for a load balancer is "identical servers." It fans one kind of traffic across many copies of the same thing. It is content-agnostic and application-unaware in its classic form, and that simplicity is a strength: it is fast, predictable and battle-tested.

3. What an API gateway does

An API gateway solves a different problem. When you expose APIs, whether to your own front-end applications, to partners, or to the public, every one of those APIs shares a set of cross-cutting concerns. Every request needs to be authenticated. Every caller needs to be held within a rate limit so one client cannot exhaust your capacity. Requests need to be routed to whichever backend service owns that particular endpoint. Responses may need shaping, logging, caching or transformation. You could build all of that into each service individually, but that means duplicating the same logic everywhere and keeping it consistent by hope. The API gateway exists so you build it once, at the edge, and every API behind it inherits it.

Concretely, an API gateway typically handles the following. It performs authentication and authorisation, validating API keys, JSON Web Tokens or OAuth access tokens before a request is allowed through. It enforces rate limiting and quotas per client, protecting the backend from overload and abuse. It routes by path, method or header, sending a request for /orders to the orders service and /users to the users service, which is fundamentally different from a load balancer sending traffic to identical copies. It can transform requests and responses, rewrite headers, aggregate calls, and translate between protocols. It centralises observability, logging every call and emitting metrics. And it often manages API versioning, so that /v1 and /v2 of an endpoint can coexist and route to different implementations.

The gateway is deeply application-aware. It reads and understands the request as an API call, not merely as bytes to be forwarded. That awareness is the whole point. In a microservices architecture the gateway becomes the single front door that hides the internal service topology from clients: callers see one clean, versioned API surface, while behind the gateway the system may be a dozen independently deployed services. For a fuller treatment of the gateway pattern on its own, see the what is an API gateway pillar.

A caution on scope creep: because an API gateway sits in a privileged position and can do so much, it is tempting to push business logic into it. Resist that. The gateway should handle cross-cutting concerns that genuinely belong at the edge, such as authentication, rate limiting and routing. When it starts making business decisions or transforming payloads in ways that encode domain rules, it becomes a bottleneck that every team has to coordinate around, and a single point of failure that is painful to change. Keep it thin. The wider set of trade-offs is covered in the enterprise system integration pillar.

4. Head to head

Laying the two side by side across the dimensions that actually matter makes the boundary unmistakable. Note in particular the rows for OSI layer, awareness of the API, and the application-level concerns of authentication and rate limiting, which is where the two products diverge most sharply.

Read the table top to bottom and a pattern appears. The load balancer is a network component that happens to touch traffic. The API gateway is an application component that happens to sit on the network path. That distinction, network job versus application job, is the cleanest one-line summary of the whole comparison.

5. OSI layers: Layer 4 versus Layer 7

The OSI model gives us the precise vocabulary to describe the difference, and it is worth a moment because it explains why the two components can do what they do. The two layers that matter here are Layer 4, the transport layer, and Layer 7, the application layer.

Layer 4 works with TCP and UDP. A Layer 4 load balancer sees connections and packets: source and destination IP addresses and ports. It does not open the packet to read what is inside. It cannot tell whether the traffic is an API call, a web page request or a database connection, because at Layer 4 that information is simply not visible to it. This is why a pure Layer 4 load balancer is fast and content-agnostic: it makes its decision on the connection metadata alone, without the cost of parsing the payload.

Layer 7 works with the application protocol itself, most commonly HTTP. A Layer 7 component reads the full request: the URL path, the HTTP method, the headers, the cookies, and if needed the body. Because it understands HTTP, it can make decisions based on what the request actually is. It can route /api/orders to one place and /api/users to another, read an Authorization header, or apply a rate limit keyed to an API token.

This is where the layers explain the roles. A load balancer can operate at Layer 4 or Layer 7; many run at Layer 4 for raw speed, and Layer 7 load balancers exist when you want path-based distribution. An API gateway, by contrast, is inherently a Layer 7 component. Everything it does, authentication, path routing, rate limiting, transformation, requires reading and understanding the application-layer request. There is no such thing as a Layer 4 API gateway, because you cannot manage an API you cannot read. When someone tells you a gateway is "just a load balancer with extra features," the accurate correction is that it is a Layer 7 component built for application concerns, not a Layer 4 component with add-ons.

The insight worth keeping: the layer tells you the ceiling of what a component can do. Layer 4 sees connections, so it can distribute but not understand. Layer 7 sees requests, so it can understand and therefore govern. A load balancer can live at either layer; an API gateway must live at Layer 7. Once you internalise that, you never confuse the two again.

6. Do you need both?

Often, yes, and they sit in a natural order. In a typical production system the load balancer is the outermost layer, the first thing internet traffic reaches. It terminates TLS, spreads incoming connections across multiple instances, and provides the raw availability and scale. Behind it sits the API gateway, or more precisely a pool of gateway instances, which then reads each request, authenticates it, applies rate limits, and routes it to the correct backend service.

Notice something important in that arrangement: the load balancer is not only in front of the gateway, it is also frequently behind it. The gateway routes a request to the orders service, but the orders service itself runs as several identical instances, so there is a load balancer distributing across those instances too. In other words, load balancing appears in multiple places, wherever you have identical copies to spread traffic across, while the API gateway appears once, as the governed front door to the API surface. They are not competitors fighting for the same slot; they are complementary and they nest.

There are cases where you need only one. If you run a single monolithic web application scaled across identical servers with no separate API surface to govern, a Layer 7 load balancer alone may be entirely sufficient; adding an API gateway would be complexity with no payoff. Conversely, a small internal API with modest traffic served by one instance might use a lightweight API gateway for authentication and rate limiting and have no need for a load balancer at all, because there is nothing to balance. The rule is simple: reach for a load balancer when you have multiple identical instances and need availability and scale, and reach for an API gateway when you have API concerns such as authentication, rate limiting and multi-service routing to centralise. Many real systems have both because they have both problems.

If your APIs are being consumed by other business systems rather than by browsers, the gateway's role in enforcing a stable, versioned contract becomes even more valuable. This is the territory covered in the what is system integration pillar, and in the specific context of connecting a business platform to the outside world, the Business Central APIs and integrations pillar.

7. Common architectures

A handful of arrangements cover most of what you will encounter in practice. Recognising which one you are looking at tells you immediately what each box is doing.

• Load balancer only. A monolithic application scaled horizontally. Traffic hits a Layer 7 load balancer that spreads it across identical app servers and checks their health. There is no separate API layer to govern. Simple, robust, and common for traditional web applications.
• API gateway only. A small or internal set of APIs where authentication, rate limiting and path routing matter but traffic volume does not demand a dedicated balancing tier. The gateway is the single front door and each backend runs as a single instance. Common in early-stage products and internal tooling.
• Load balancer in front of the API gateway. The standard production shape for anything at scale. The load balancer terminates TLS and distributes across a pool of gateway instances for availability; the gateway then handles API policy and routes to services. This is the arrangement most enterprise systems converge on.
• Gateway with load balancing behind it. The gateway routes to a service, and that service runs as multiple identical instances with a load balancer, often the platform's built-in service load balancer, spreading across them. Layered load balancing is the norm inside a microservices cluster, so you see balancers both in front of and behind the gateway.
• Gateway plus service mesh. In mature microservices platforms the gateway handles north-south traffic, meaning traffic entering and leaving the cluster, while a service mesh handles east-west traffic between internal services, doing its own load balancing, retries and mutual TLS. The gateway governs the external edge; the mesh governs the internal fabric.

Across all of these, the same principle holds. Wherever you see identical copies that traffic must be spread across, that is a load balancer, and it can appear many times. Wherever you see the single governed entry point that authenticates, throttles and routes API calls to distinct services, that is the API gateway, and it usually appears once per API surface. Keep those two questions in mind, "is this spreading traffic across copies?" and "is this governing and routing API calls?", and any architecture diagram becomes readable.

8. References

The following are stable, authoritative sources for the concepts in this article. They are worth consulting directly when you need the precise, vendor-neutral definitions.

• International Organization for Standardization, ISO/IEC 7498-1, the OSI reference model, which defines the seven-layer model including the transport (Layer 4) and application (Layer 7) layers referenced throughout.
• Internet Engineering Task Force, RFC 9110, HTTP Semantics, the current specification for the HTTP application protocol that Layer 7 components read and act upon.
• Internet Engineering Task Force, RFC 6749, The OAuth 2.0 Authorization Framework, and RFC 7519, JSON Web Token (JWT), which underpin the authentication that API gateways enforce.
• Chris Richardson, Microservices Patterns, which documents the API Gateway pattern and the surrounding service architecture in detail.
• Vendor documentation for NGINX, HAProxy, Envoy and cloud provider load balancers and API gateways, useful for seeing how a single product can be configured for either role.

Final thoughts

The reason an API gateway and a load balancer get mistaken for each other is that they share a position, share some features, and can even share a binary. But they answer different questions. A load balancer asks "which of my identical servers should take this connection, and are they healthy?" and it can do that at Layer 4 without ever reading the request. An API gateway asks "what is this API call, who is making it, are they allowed, and which of my different services owns it?" and it can only do that at Layer 7, because governing an API means understanding it. One is a network component for scale and availability. The other is an application component for policy and routing.

In most real systems of any size you end up with both, nested rather than competing: a load balancer at the outer edge for raw distribution, an API gateway inside it for API governance, and more load balancing again behind the gateway wherever services run as identical copies. Once you stop trying to pick one over the other and start seeing them as two tools for two jobs, the whole edge of your architecture becomes far easier to reason about. If you want to see where these pieces fit in the broader connectivity picture, the enterprise system integration pillar ties the gateway, the load balancer, APIs and the systems behind them into a single map.

Related reading: Enterprise system integration explained, What is an API gateway, REST API explained, What is system integration, Business Central APIs and integrations.