Rate limit exceeded keeps failing in production

Started by Private User on Saturday, December 7, 2019
Problem with this page?

Participants:

  • Private User
    Geni member
  • Private User
    Geni Pro

Related Projects:

Showing all 6 posts
Private User
12/7/2019 at 4:42 AM

This conversation is linked to this one: https://www.geni.com/discussions/204253

It really looks there is some problem with the Rate limit. I am using access token to call geni API so (ref https://www.geni.com/platform/developer/help/rate_limits?version=1), it should support 40 requests per 10 seconds, so, the limit is 1 request each 0.25 seconds.

I have created an small script in python to test the proble. Essentially, I created this piece of code for running the url calls:

data = None
continue_iteration = True
i=1
while continue_iteration:
print(i,datetime.datetime.now())
data = requests.get(url)
i += 1
value_error = data.json().get("error", {}).get("message", None)
if not ( value_error and ("Rate limit exceeded." in value_error) ): continue_iteration = False

So, my expectatino of output is that all print outs will start with 1, given that the difference between the first call and the next is at least 0.25 seconds.

But this is the result I get:

1 2019-12-07 13:37:34.182276
1 2019-12-07 13:37:35.331122
1 2019-12-07 13:37:36.359418
1 2019-12-07 13:37:36.967325
1 2019-12-07 13:37:37.889597
1 2019-12-07 13:37:38.284527
1 2019-12-07 13:37:38.916835
1 2019-12-07 13:37:39.313022
1 2019-12-07 13:37:40.463302
1 2019-12-07 13:37:40.960288
1 2019-12-07 13:37:41.983683
2 2019-12-07 13:37:42.503397
3 2019-12-07 13:37:43.007764
4 2019-12-07 13:37:43.394448
5 2019-12-07 13:37:43.785502
6 2019-12-07 13:37:44.544111
1 2019-12-07 13:37:45.574507
1 2019-12-07 13:37:45.942273
1 2019-12-07 13:37:47.104166
1 2019-12-07 13:37:48.133591
1 2019-12-07 13:37:48.641903
1 2019-12-07 13:37:49.041802
1 2019-12-07 13:37:49.465901
1 2019-12-07 13:37:50.178529
1 2019-12-07 13:37:50.441913
1 2019-12-07 13:37:50.785818
2 2019-12-07 13:37:51.199413
3 2019-12-07 13:37:51.712037
4 2019-12-07 13:37:52.095940
5 2019-12-07 13:37:52.429748
6 2019-12-07 13:37:52.745671
7 2019-12-07 13:37:53.150062
8 2019-12-07 13:37:53.759839
9 2019-12-07 13:37:54.271167
1 2019-12-07 13:37:54.634139
1 2019-12-07 13:37:55.298050

As you can see, it gives an error of Rate Limit around 10 calls (not always 10), notice that all the calls are done between 0.4-1 second, so below the 0.25 limit

Private User
12/7/2019 at 5:43 AM

in production (www.geni.com) our rate limit is 10-requests every 10-seconds.

Private User
12/8/2019 at 1:49 PM

That changed recently, isn't it? This script was working ok until the last weeks.

Private User
12/8/2019 at 1:51 PM

Or... the computational power has increased driving to faster answers from the server and allowing more requests in 10 seconds and making my software to fail (as not considering this limit)

Private User
12/9/2019 at 5:40 AM

We added more servers a few weeks ago.

Private User
12/9/2019 at 7:36 AM

So it does matches... as the computational capacity was lower, the answer I was receiving before was likely taking more time and making the same script to go slower but below the rate limit...
It looks that now, with a bigger capacity is goes quite faster and I hit the rate limit. Btw, I hit it by just 2 seconds :)

Showing all 6 posts

Create a free account or login to participate in this discussion