Resolve Nginx 503 errors caused by rate limiting. Learn to diagnose, adjust limit_req_zone, and optimize Nginx configuration on Ubuntu 20.04 LTS.
When your Nginx web server starts returning HTTP 503 Service Unavailable errors, especially during periods of high traffic or suspicious activity, it often indicates that Nginx's built-in rate limiting mechanism has been triggered. This guide delves into diagnosing and resolving "Nginx rate limit exceeded" issues, specifically on Ubuntu 20.04 LTS, ensuring your services remain available and performant under load.
Symptom & Error Signature
Users attempting to access your web application will receive an HTTP 503 Service Unavailable response. The browser might display a generic Nginx 503 page or your custom error page, stating that the service is temporarily unavailable.
Upon inspecting Nginx access logs (typically /var/log/nginx/access.log), you will observe numerous entries with a 503 status code for affected requests:
192.168.1.1 - - [18/Jul/2026:10:30:05 +0000] "GET /api/data HTTP/1.1" 503 197 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.75 Safari/537.36"
192.168.1.2 - - [18/Jul/2026:10:30:06 +0000] "POST /submit HTTP/1.1" 503 197 "-" "User-Agent: MyBot/1.0"
Crucially, the Nginx error log (typically /var/log/nginx/error.log) will contain specific messages indicating rate limiting:
2026/07/18 10:30:05 [error] 12345#12345: *67890 limiting requests, excess: 5.000 by zone "api_rate_limit", client: 192.168.1.1, server: example.com, request: "GET /api/data HTTP/1.1", host: "example.com"
2026/07/18 10:30:06 [error] 12346#12346: *67891 limiting requests, excess: 3.500 by zone "api_rate_limit", client: 192.168.1.2, server: example.com, request: "POST /submit HTTP/1.1", host: "example.com", referrer: "http://example.com/"
The key phrase here is limiting requests, excess: X.XXX by zone "yourzonename", which confirms that the Nginx limit_req module is actively rejecting or delaying requests.
Root Cause Analysis
The HTTP 503 Service Unavailable status, accompanied by "limiting requests" messages in the error log, directly points to Nginx's limit_req module. This module is designed to control the rate of requests for specific keys (typically client IP addresses) and is configured using two primary directives:
-
limitreqzone: Defined in thehttpcontext, this directive creates a shared memory zone to store request states. It specifies: - * Key: The variable to track (e.g.,
$binaryremoteaddrfor client IP). - * Zone Name & Size: A unique name for the zone and its memory allocation (e.g.,
zone=apiratelimit:10m). - * Rate: The maximum average request rate allowed (e.g.,
rate=5r/sfor 5 requests per second, orrate=300r/mfor 300 requests per minute). -
limitreq: Applied withinhttp,server, orlocationcontexts, this directive references alimitreq_zoneand specifies: - * Zone Name: Which shared memory zone to use.
- * Burst: How many requests a client can make in excess of the defined rate without being immediately rejected. These excess requests are queued and processed with a delay.
- Nodelay: An optional parameter that, if present, tells Nginx to process requests without delay once the burst limit is reached, potentially leading to server overload but avoiding client delays. If
nodelayis not* used, requests exceeding the rate but within the burst limit are delayed. Ifburstitself is exceeded (ornodelayis present and the rate is exceeded), Nginx returns a 503 (or 500 depending onlimitreqstatus).
Common Scenarios Leading to Rate Limit Exceedance:
- Legitimate High Traffic: A sudden surge in user activity, viral content, or successful marketing campaigns can push legitimate traffic beyond the configured limits.
- Malicious Activity/DDoS: Bots, scrapers, or Distributed Denial of Service (DDoS) attacks can generate an extremely high volume of requests, intentionally triggering rate limits.
- Misconfigured Clients/Scripts: Automated scripts, monitoring tools, or even poorly implemented AJAX calls can make excessive requests, especially if there's a bug causing a loop.
- Overly Aggressive Configuration: The
limitreqzoneandlimit_reqparameters might be set too low for the actual expected traffic patterns of your application.
Step-by-Step Resolution
1. Identify the Active Nginx Rate Limit Configuration
Your first step is to locate where limitreqzone and limit_req are defined in your Nginx configuration. Nginx configurations on Ubuntu 20.04 LTS are typically found in /etc/nginx/nginx.conf and files included from conf.d or sites-enabled.
# Search for limit_req_zone and limit_req directives across Nginx configuration files
grep -R "limit_req_zone" /etc/nginx/
grep -R "limit_req" /etc/nginx/
You'll likely find something similar to this structure:
/etc/nginx/nginx.conf (or an included file like /etc/nginx/conf.d/ratelimit.conf):
http {
# ... other http settings ...
# Defines a 10MB zone named 'api_rate_limit' tracking client IPs, allowing 5 requests/second
limit_req_zone $binary_remote_addr zone=api_rate_limit:10m rate=5r/s;
# ...
}
/etc/nginx/sites-enabled/your_site.conf (or within a location block inside a server config):
server {
listen 80;
location /api/ { # Applies the 'apiratelimit' zone, allowing a burst of 10 requests, with no delay limitreq zone=apirate_limit burst=10 nodelay; # … proxy_pass or root directives … }
location /admin/ {
# Potentially a different limit for admin panel
limitreq zone=apirate_limit burst=5 nodelay;
# …
}
}
`
[!IMPORTANT] Note the
zone=name (e.g.,apiratelimit). This name is crucial as it links thelimitreqzonedefinition (where the rate and zone size are set) to thelimit_reqapplication (where burst and nodelay are set).
2. Analyze Nginx Error Logs for Specifics
Re-examine /var/log/nginx/error.log for recent limiting requests messages. This will confirm the specific zone being hit, the client IP addresses that are exceeding the limit, and the excess value.
sudo tail -f /var/log/nginx/error.log | grep "limiting requests"
Look for the excess: X.XXX value. A high excess value indicates that requests are coming in much faster than the configured rate and burst limits. This helps you understand the magnitude of the problem.
3. Adjust limitreqzone Parameters (Rate and Burst)
This is the most common and direct resolution: increasing the allowed request rate or burst capacity to accommodate legitimate traffic.
Edit the Nginx configuration file where your limitreqzone is defined (e.g., /etc/nginx/nginx.conf or a custom .conf in conf.d).
sudo nano /etc/nginx/nginx.conf
Understanding the Parameters to Adjust:
-
rate: Defines the average request processing rate Nginx attempts to maintain. - *
rate=5r/s: Allows 5 requests per second. - *
rate=300r/m: Allows 300 requests per minute (equivalent to 5r/s). - * Recommendation: If you suspect legitimate traffic is being blocked, consider increasing the rate. A good starting point is to double the current rate (e.g., from
5r/sto10r/s) and monitor. -
burst: Specifies how many requests a client can send in excess of the definedratebefore Nginx returns a 503. These excess requests are put into a queue and processed as capacity becomes available. - *
burst=10: Allows 10 requests to be queued. - * Recommendation: Increase this value to accommodate momentary spikes in traffic. A higher
burstallows more flexibility and reduces 503 errors during brief peaks but consumes more shared memory for the queue. -
zonesize: The memory allocated for the shared memory zone (e.g.,zone=apiratelimit:10m). Each state for an IP address takes about 32 bytes.10mcan store ~320,000 active IP addresses. If you significantly increase the burst or expect many unique IPs, you might need to increase the zone size.
Example Modification:
Original (potentially too restrictive):
# In http block:
In server/location:
limitreq zone=apirate_limit burst=10 nodelay;
`
Modified (more permissive):
# In http block:
# Increased rate to 10 requests/sec and zone size to 20MB
In server/location:
# Increased burst to 20 requests
limitreq zone=apirate_limit burst=20 nodelay;
`
[!WARNING] Increasing
rateandbursttoo aggressively without understanding your server's underlying capacity (CPU, memory, backend application limits) can lead to backend overload and system instability. Always monitor server resource usage after making changes.
4. Consider Whitelisting Specific IPs
If you have specific IP addresses (e.g., your own monitoring systems, trusted partners, or internal tools) that legitimately need to make high volumes of requests without being rate-limited, you can whitelist them using the geo or map directives.
Using geo directive (in http block):
http {
# ...
# Define a variable '$whitelist' that is 0 for whitelisted IPs, and 1 for others
geo $whitelist {
default 1; # By default, enable rate limiting
127.0.0.1 0; # Whitelist localhost
192.168.1.0/24 0; # Whitelist a subnet (e.g., internal network)
203.0.113.10 0; # Whitelist a specific public IP
Now, define your rate limit zone, potentially referencing the whitelist variable
limitreqzone $binaryremoteaddr zone=apiratelimit:10m rate=5r/s;
}
`
Then, in your server or location block, apply the limit_req only if the client is not whitelisted:
server {
# ...
location /api/ {
if ($whitelist = 1) { # Only apply rate limit if client is not whitelisted
limit_req zone=api_rate_limit burst=10 nodelay;
}
# ... Other directives for /api/
}
}
5. Test Nginx Configuration and Reload
After making any changes to Nginx configuration files, it is critical to test the syntax before reloading. This prevents Nginx from failing to start due to misconfigurations.
sudo nginx -t
If the test is successful, you'll see messages like:
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
Then, reload Nginx to apply the changes gracefully:
sudo systemctl reload nginx
[!IMPORTANT] Always use
sudo systemctl reload nginxinstead ofsudo systemctl restart nginxin production environments.reloadallows Nginx to gracefully shut down old worker processes and start new ones with the updated configuration, minimizing downtime by keeping existing connections active during the transition.
6. Monitor and Iterate
After applying the changes, it's crucial to continuously monitor your Nginx error logs, access logs, and overall server resource utilization.
- Check
error.log: Ensurelimiting requestsmessages have significantly reduced or disappeared for legitimate traffic patterns. - Check
access.log: Verify that503responses are no longer being served by Nginx for requests that should be allowed. - System Monitoring: Use tools like
htop,netdata,atop, or integrated solutions likePrometheus/Grafanato observe CPU, memory, network I/O, and active connection counts. If these resources spike dangerously after increasing limits, your backend application or database might be the actual bottleneck, not just Nginx's rate limit.
You may need to iterate on these changes, gradually adjusting rate and burst until you find a balanced configuration that protects your server resources while allowing legitimate traffic to flow freely.
7. Consider External Solutions (Advanced)
For high-volume websites or those frequently targeted by sophisticated DDoS attacks, Nginx's built-in rate limiting might not be sufficient on its own. Consider implementing additional layers of protection:
- Content Delivery Networks (CDNs) with WAF: Services like Cloudflare, Akamai, or AWS CloudFront with Web Application Firewall (WAF) capabilities can absorb and filter malicious traffic and bot requests before they ever reach your Nginx server, significantly reducing the load.
- Dedicated Web Application Firewalls (WAFs): Standalone or cloud-based WAFs can provide more sophisticated rule sets, behavioral analysis, and threat intelligence for advanced attack detection and mitigation.
- Fail2Ban: While Nginx rate limiting deals with requests per second, Fail2Ban can be configured to parse Nginx logs and automatically ban (at the firewall level) IP addresses that repeatedly trigger rate limit errors or other suspicious activities, effectively blocking persistent attackers.
By carefully diagnosing the cause and making informed adjustments to your Nginx rate limiting configuration, you can effectively manage traffic spikes and protect your web services from overload on Ubuntu 20.04 LTS.
Leave a Reply