Company
Date Published
Author
Ben Weintraub
Word count
2281
Language
English
Hacker News points
None

Summary

New Relic, a company that provides infrastructure monitoring services, has implemented various techniques to limit API overloading and protect itself from intentional or inadvertent abuse. One such technique is the separation of API and UI worker pools, which helps prevent API users from consuming all available worker time. Another technique is the use of an internal tool called API Overload Protection, which tracks worker time usage for each API key and restricts access if excessive usage is detected. However, New Relic recently encountered a puzzling issue where API requests were being reported as having used less time than they actually had, leading to inaccurate overload protection. The investigation revealed that the issue was caused by clients setting an aggressive timeout value on their requests, which caused nginx to give up on waiting for Unicorn's response before writing it back to the client. To fix this issue, New Relic set a configuration option in nginx to defer post-processing work until the upstream server had actually sent a response, ensuring that the API Overload Protection tool gets an accurate accounting of worker time consumed by each account.