When hosting an application on Heroku, managing logs efficiently is crucial for maintaining system health and keeping costs down. Heroku provides built-in logging for all incoming requests, but by default, Gunicorn, the Python HTTP server often used in Heroku deployments, also logs incoming requests. This duplication can clutter your logs, making them harder to parse and more expensive to store. Let’s explore why this redundancy exists and how to fix it.
Heroku Router Logs
Heroku’s Router automatically logs all incoming HTTP requests, providing a wealth of data for monitoring and debugging your application. These logs are always enabled and include detailed information, such as the HTTP method and URL path of the request, the response status code, the client’s IP address, and the request processing time (see Heroku Router Log Format).
Because these logs are highly informative, they serve as a robust tool for observability, making other access logs superfluous.
Gunicorn Access Logs
Gunicorn includes an option to log incoming requests using the accesslog
setting. By default, this is set to None
, meaning no access logs are generated. However, when deploying on Heroku, the Python buildpack explicitly sets it to '-'
using the GUNICORN_CMD_ARGS
environment variable (see source code), which logs to stdout.
This results in both Heroku Router logs and Gunicorn access logs being written to the same log stream, creating unnecessary duplication.
Reducing Log Noise and Costs
You have a few options to remove the excess Gunicorn logs: discard them before they’re ingested by your logging service or disable them entirely.
Filter Logs at the Logging Service Level
If you use a logging service, you can filter out logs before they’re processed or stored. Most services support filtering based on string or regex patterns:
- Datadog: Exclusion Filters
- New Relic: Drop Filter Rules
- Sumo Logic: Processing Rules
- PaperTrail: Log Filtering
This approach is useful if you have a specific reason to keep some Gunicorn access logs while excluding others. However, in most cases, you’ll want to remove all Gunicorn access logs, which is best done at the source.
Disable Gunicorn Access Logs
If you want to completely eliminate Gunicorn access logs on Heroku, the most efficient solution is to disable them at the command line. This works because command-line arguments override the GUNICORN_CMD_ARGS
environment variable, as stated in the Gunicorn documentation. In practice, this is done by updating your Procfile to explicitly disable the access log:
web: gunicorn myapp.wsgi --accesslog None
With this change, Gunicorn will stop logging incoming requests.
May your logs be clean and cost-effective.