← All posts

Every per-IP control we shipped was bypassable with one header

Authagonal·July 11, 2026
securityrate-limitinginfraaks

Our account-lockout counter capped failed logins at five. Our rate limiter throttled abusive clients. Both keyed off the client's IP address, which is the obvious thing to key them off. Both were completely bypassable, and the bypass was a single HTTP header you could set yourself.

Here is why. When your app runs behind a reverse proxy, and ours runs behind nginx on Kubernetes, the TCP connection the app sees does not come from the user. It comes from the proxy. Every request arrives with the same socket peer address: the ingress. So the real client IP has to be carried in a header, X-Forwarded-For, which the proxy sets and the app reads. ASP.NET Core has middleware for exactly this: it reads X-Forwarded-For and rewrites the connection's remote IP to match, so the rest of your code, including your rate limiter, sees the true client.

The problem is that X-Forwarded-For is just a header. Anyone can send one. If the app trusts it unconditionally, and ours did, trusting it from any source at all, then the value is attacker-controlled. So the attack is not clever, it is one line:

curl -H "X-Forwarded-For: 10.0.0.1" https://.../login
curl -H "X-Forwarded-For: 10.0.0.2" https://.../login
curl -H "X-Forwarded-For: 10.0.0.3" https://.../login

Change the header every request and the app thinks each attempt comes from a brand new client. The lockout counter, keyed on that IP, never reaches five for any single value. The rate limiter's bucket is fresh every time. You can brute force a password all day, and every per-IP defense we had counts happily to one. A control that keys off a value the attacker sets is not a control.

The fix sounds trivial: only trust X-Forwarded-For from your own proxy. It is trivial in principle and fiddly in practice, and Kubernetes is where the fiddliness lives.

You cannot just stop reading the header. If you do, every request looks like it comes from the ingress, so now all your clients share one IP and the per-IP controls break in the other direction: one abuser trips the lockout for everybody. You have to read X-Forwarded-For, but only believe it when the request actually came from a proxy you trust.

ASP.NET's forwarded-headers middleware trusts by source: a list of known proxy IPs, or known proxy networks given as CIDRs. The naive move is to pin the ingress pod's IP. On AKS that IP is not stable. Ingress pods get rescheduled and rescaled, and their addresses churn with them, so a pinned IP goes stale and either breaks legitimate traffic or, if you leave an old fallback in, reopens the hole. What is stable is the subnet the pods draw from. So you trust the node-subnet CIDR, not any single address. Every hop that can legitimately set the header lives in that range, and nothing outside it is believed.

Then there is the count. The middleware walks the X-Forwarded-For list from the right, peeling one trusted hop at a time, and whatever is left after the trusted hops is taken as the client. Walk one hop too far and you step off your proxy and into the part of the header the attacker wrote. So the number of hops you peel has to equal the number of proxies that actually append to the header, exactly. In our path there is one: nginx. The Azure load balancer in front of it is layer 4. It forwards TCP, it does not parse HTTP, and it adds no X-Forwarded-For entry. So the correct forward limit is one. Not the default, not "a few to be safe." One, because one proxy touches that header.

That pair, trust the node subnet and peel exactly one hop, is the whole fix. The client IP the middleware hands you is now the value nginx wrote from the real socket, and the attacker's injected entries sit to the left of it, ignored, because we stop walking before we reach them.

The transferable part is not the numbers, it is that they are numbers about your specific topology. X-Forwarded-For is not data, it is trust delegation: reading it is saying "I believe whoever set this." That belief has to be pinned to the exact hops in your request path, no wider and no narrower. Trust too broadly and it is spoofable. Trust too narrowly and you collapse all your clients into the proxy. And the count is coupled to your infrastructure, so the day you put a CDN in front of everything, you have added a hop, and the forward limit that was correct yesterday is now one short. The fix is not a magic constant. It is: count the proxies in your path, trust those and only those, and re-count every time the path changes.

Counting proxy hops correctly is unglamorous, easy to get wrong, and it decides whether your rate limits mean anything at all. Authagonal does that bookkeeping for you, so your lockout counts the client and not a header.