(C)Python is slow, relative to almost any other language. But that hardly means that every task in Python will take 3 seconds to perform or something. If you do something small, Python will do it quickly. Another language may do it more quickly, but if it's fast enough, it's fast enough.
Where I get really frustrated with the 90s-era dynamic scripting languages is not when you are "filtering one list" or "serving JSON out of the DB", but when you're trying to do all kinds of CPU-type work in one page, with numerous (unavoidable) DB accesses and filtering and transforming etc etc. You can get into "seconds" surprisingly quickly, since it is, indeed, a fairly slow language.
It is likely that rewriting this code in C++ or something could get another 5-10x out of it even over PyPy, probably even more if we really start going crazy with optimizations, but at much larger investment of developer time. By contrast, twiddling with a couple of details and switching to PyPy is the sort of thing you can prototype in an afternoon and deploy in a week or two, even with solid testing procedures. At scale that's probably worth it (using 5-10 times fewer machines is generally a significant cost savings even in this "cloud" era), but they may not be to that scale yet. Perhaps they never will be. Who knows? Not me.
We are not using PyPy in the company I work (https://www.demonware.net) at the moment and we are handling way more requests per second, with also very small response times (less than 10ms in most cases, but it depends).
The bottleneck is usually DB work, so PyPy is not a great help there.
Bottomline: Number of requests (scalability) and response times (performance) are more a work of architecture than using a particular language. And Python can be actually quite fast (fast enough)
Hi, I'm the author of the post. Our 95% latency is just shy of 10ms, and max latency around 100ms. Our monitoring tool pre-calculates the percentiles, so I don't have 99% or 99.99%, but my guess is that they're under or around 50ms. Too much more than that and we'd be hearing from our partners about timeout rates. We haven't thoroughly profiled the difference between "most" and "all" in terms of latency sources, but I'd guess that GC pauses account for some of it, and some requests are simply much more expensive for us to process than others.
Hi, work for one of the partners. From what I can tell, your account manager should be able to get you this info, including 99 percentile, if you are interested. This is for round trip from our point of view of course.
Spot checking, you've done better than you think :)
Nice one. Yes the GC pauses are usually the root cause of higher p99+ latency. Anyways, the 10ms-100ms range is pretty amazing by itself. With such a huge throughput it is hard to measure latency accurately you are kind of forced to use sampling, but it can be a good representation of reality still.
