« Back to Project Discussions

API Authentication Error

Started by Craig Andrew Miles on Thursday, March 9, 2023
Problem with this page?

Participants:

Related Projects:

Showing all 23 posts

I am experiencing an issue with test code that up until relatively recently has been working for a number of years.

I initiate the authentication process via the API and am directed to the Login page. I enter the credentials and then instead of being redirected to the page to click the Authorise button, I end up back on the home page, with an error message "The security code provided was incorrect. Please try again."

What "security code"? Surely this isnt referring to the "access token"?

And the strange thing is that it only happens on the second iteration - it always works the first time.

There's nothing I can debug at my end because control is not returned to my app between hitting the Login button and expecting to see the Geni authorization page.

Can someone please help?

Because you missed this one: www.geni.com/media/proxy?media_id=6000000192606710825&size=large

Trying to automate the login? Forget it. If you have enabled Multi factor Authentication you have to manually open another App to get the code.

You could of course try to disable your Two-Factor Authentication in your
Email & Password Setting, but...

Even if you have succeeded before, Geni recently changed so that a regular login does not give you access to the api anymore. To demonstrate it: Opening this in a browser did give you the JSON of your own profile before that change: https://www.geni.com/api/profile

Two-factor authentication is supported for web authorization. Is that what you're using, Craig, or are you trying to do mobile authorization or the Trusted or App login where the grant_type is "password" or 'client_credentials" ? It's possible those aren't working properly with 2FA but I'd need to know how you're trying to authenticate.

Thanks for the responses Private User and Mike Stangel

I wasnt aware that 2FA had been implemented in Geni - I havent been using it in my regular interactions with the site and I dont believe that it is enabled on my account given that the option "Enable Google Authenticator" is active.

My app is using the server side authentication flow documented here https://www.geni.com/platform/developer/help/oauth_server_side?vers... using the access token returned

I am confused because some executions using this flow are working and then subsequent ones are redirecting to the page which says "The security code provided was incorrect".

At least I know what the security code is now but it seems that some interactions are expecting a security code even though I haven't opted in to use 2FA?

Surely any use of 2FA in automated authentication requests via the API is impossible?

Craig Andrew Miles - I have used 2FA the whole time since Geni started beta-testing it and it cause no problem for using the Api. If you check-mark "Trust this computer" you will not be asked for it in a while.

Let me guess: You get the same problem if you enter a wrong login?
Hint: Do not close the login window unless you get a cancel or AccessToken.

So you are saying that I should enable it, authenticate with Geni once using 2FA and Trust this computer manually, and then the API should be untroubled?

Regardless, it still doesnt make sense that the API is rejecting some calls due to 2FA (or lack of) when 2FA is not currently even enabled for my account.

So Private User following your suggestion I finally did a test with 2FA enabled and as expected it didnt work when authenticating from a script running in a virtual machine as I am still redirected to a page to provide the security code.

I guess Geni doesnt recognise the VM as the "trusted computer".

So I am back to square one Mike Stangel - with 2FA once again disabled, regardless Geni still seems to be getting itself confused and expecting a security code in some instances when authenticating via the server side authentication flow which I am unable to provide via the scripted API calls.

Did you get a chance to look into this at all?

Craig Andrew Miles it looks like this may be related to Captcha ("I am not a robot") -- not sure why that would have been necessary unless you had multiple failed login attempts, but we're going to follow that thread to see if there's something there that needs fixing.

Craig - I am still not understanding what you try to do. The API with 2FA enabled is working fine, and of course you will be redirected to a page to provide the security code. If your app don't catch that your app will of course fail.
Are you trying to script parsing the regular web pages, not using the API at all?

I notice you write using "Server Side Authorization Flow" - why? Your app is a client, isn't it?

Thanks for your response Private User

I am running a Linux Virtual Machine on my windows computer which runs the web-server for my app. It is the web-server that executes the API calls which is the reason why I need to execute the "Server Side Authorization Flow", I think.

In reality it is barely different from the Client side version which states "Use this flow if your application needs to call Geni's API from a client, such as JavaScript in a Web browser or from a native mobile or desktop application." which I am certainly not doing.

There is no issue if I authenticate with Geni using 2FA from a browser running on the Windows host computer, but if I then access Geni with the same credentials from a browser running on the VM, this is now perceived as a different computer so doesnt honour the "trust my computer" setting of the Windows host.

The thing is, I never actually interact with Geni directly from a browser running on the VM, only in headless browser mode with automated scripting. It is possible that if I fire up a browser directly in the VM and authenticate with Geni using 2FA that the "trust my computer" setting will then be honoured by the API running on the webserver on the VM when in headless mode but I'm not entirely sure how all this will work and it is:

a) An inconvenience that I didnt previously have
b) Something I wasnt expecting to be forced into because what was previously working without 2FA is no longer working for some unknown reason (looks like in some circumstances a security code is being expected when it shouldnt be)

Thanks also for your response Mike Stangel

Has something changed recently regarding Captcha then?

Each automated interaction with Geni starts with authentication (scripted logging in without 2FA via the Geni login dialogue) followed by authorising usage of the app via the Geni authorisation dialogue.

It seems that each initial interaction works as expected but a subsequent interaction fails at the login stage and redirects to the home page with an error message "The security code provided was incorrect. Please try again." instead of redirecting to the authorisation page.

In theory this could be because of captcha but why would this then generate the security code message? I'll try to capture some additional screenshots around the login process.

Mike Stangel

I cant capture any more screenshots other than the Login page when it is presented and then what is presented immediately after clicking Login.

The scripted interactions are:

1. Ensure that any saved access token is removed
2. Ensure logged out of Geni
3. Visit the Geni login page
4. Fill in the username
5. Fill in the password
6. Click the Login button
7. Check that the authorisation page is presented
8. Click the Authorize button
9. Proceed

This flow previously worked reliably over repeated iterations.

Maybe there is a recently introduced timing issue now - a second iteration occurring immediately afterwards fails at Step #7 where instead of the authorisation page being presented the Welcome Page with the security code error message is shown.

I could send you screenshots (I cant seem to attach them here) that show
- the error message at the top in red
- Welcome to Geni
- Login dialogue at left with:
+ My email prefilled but *** out
+ Password blank
+ Security code label displayed with I'm not a Robot Captcha checkbox immediately to the right
- New to Geni pane to the right

As a follow up, I did try authenticating with Geni using 2FA via a browser session launched within the VM with the "trust this computer" option checked..

Subsequent attempts to authenticate with Geni via the automated scripting for reasons that I dont really understand still redirected to the security code dialogue so this avenue seems closed.

We are back again to why I should ever even be prompted for a security code if 2FA is not enabled in the associated Geni account settings? There is something clearly wrong I am sure you would agree.

To add further intrigue I have also experienced this on occasions with the Geni UI during normal browser interaction when the system decides that I am no longer logged in and redirects to a version of the Welcome page that displays the Security code label with the I'm not a Robot Captcha checkbox (as described previously).

I just experienced this while trying to authenticate to Geni on my phone for example.

I havent been able to detect a pattern to this and certainly cant repeat at will but unfortunately something that had been working for years has been broken Mike Stangel by a relatively recent change. Sigh....

Interestingly I have noticed that Geni seems to manage sessions in an unexpected way, for me at least, as compared to other web applications.

If I log in to Geni in a Chrome session, and then open Geni in a Firefox session, I am required to log in again (as I would expect).

If I then logout of the Firefox session and return to the Chrome session, I am now logged out of Chrome and required to re-authenticate.

This also happens between devices - if I am logged into Genu using Chrome on my laptop and phone at the same time, if I log out of one device my session on the other device is also terminated.

This is definitely NOT what happens with any other web app I have engaged with where a session in each device/browser combination is managed 100% independently.

I dont know if this is deliberate but perhaps the logout action is being unknowingly and unnecessarily over-exuberant in logging the user out of ALL current sessions.

I am wondering if this could be related to my authentication woes from the VM.

Craig Andrew Miles the logout is indeed "over-exuberant" and we will probably add an option to "Log out of this device" in the future.

I'm still not convinced this is 2FA, it could be Captcha required due to failed login attempts. So today we released two code changes:

1. We changed the language on the "security" error message to better-distinguish between 2FA failures and Captcha failures; Captcha will now report "The security check failed. Please try again." while 2FA will report "The Authenticator code provided was incorrect. Please try again." -- if you get the error again, please take note and let me know which one you're seeing;

2. We added Captcha support to the API authorization logon pages, where it was mistakenly overlooked. I suspect this will be the fix you need.

Mike

Thanks Mike Stangel

I'm not sure if the update has actually been applied yet as I am still seeing "The Security code provided was incorrect. Please try again." as opposed to the message you say I should be seeing in the event of Authentication failure "The Authenticator code provided was incorrect. Please try again."
.
Assuming the update hasnt been finalised on the servers yet I'll check again tomorrow.. Otherwise I'm still seeing a 2FA-like error without 2FA being activated.on the account.

And the Login page presented via API flow is not showing a Captcha, yet at least.

But are you saying that I should expect to have to complete the captcha each time on API Authorization flow? Unless this is a consistently presented checkbox (and not identifying buses or some such) it will be impossible to complete via an automated interaction - and strictly speaking I would be lying to claim to be human in this case!

Still getting weird behaviour though:
- Logged into Geni via Chrome
- Automated test running in the background via VM
- Go to post this comment via my Chrome browser session but now not logged in (as previously discussed) due to over-exuberant logout
Login again successfully but unexpectedly get the "The Security code provided was incorrect. Please try again." at the top of the welcome page. And yet I am logged in. And not using 2FA. I have a screenshot!

Craig Andrew Miles I see the failed authorization attempt about 10 minutes to your message above. It was a Captcha failure, not 2FA. I would have bet money that the error message shown was "The security check failed. Please try again." but you're saying it said Security code ? That's what your screenshot shows?

About Captcha: since 2012 we have required users to complete a Captcha if they failed the login password three times in a row. In January of this year we extended this to also include failing the login by not providing the correct MFA code. We expect this to be relatively uncommon. In either event, you should see the Captcha prompt below the password field (and below the MFA field, if present) wherever you encounter a login page, IF a Captcha is required. A week ago I said this:

it looks like this may be related to Captcha ("I am not a robot") -- not sure why that would have been necessary unless you had multiple failed login attempts, but we're going to follow that thread to see if there's something there that needs fixing.

And indeed, I discovered we needed to add the Captcha prompt to the pages that serve up login forms for API authentication. I did that and we released it, however reviewing the flow of events this morning I think we have another fix to make: your app passed the client_id with to the Oauth authorization URL, which forwarded that client_id to the login page. That login page did NOT show a Captcha because it doesn't know which user is going to log in. You submitted your email address and password, which failed because it was looking for a Captcha for your account [we should figure out why this is, if you didn't fail a login 3 times in a row] so what SHOULD have happened is that you get the error message and another attempt at logging in, this time WITH the Captcha. But what appears to be the case is that the second attempt has lost track of the client_id, meaning you're now logging into Geni's web interface instead of completing the Oauth flow. I'm going to work on that tomorrow, and I'll test it accordingly.

But are you saying that I should expect to have to complete the captcha each time on API Authorization flow?

Yes, but ideally the authorization flow is done only once by a user because you are storing the refresh token. If that makes sense then don't bother reading the following explanation:

When a user is required to authorize your application, here's how it should work:

  1. User decides "hey I'd like to use Craig's Cool App, lemme click this link he gave me" (to the /oauth/authorize URL)
  2. If the user is not logged in, Geni prompts them for their Email and Password with the message "Login to use your Geni account with Craig's Cool App." They enter their email address and password and complete the MFA if necessary. If a Captcha was necessary, they might have to repeat it and also complete a Captcha. Once logged in, the user us returned to the Oauth authorization URL
  3. Geni displays the Oauth authorization dialog saying "Craig's Cool App would like permission to access your Geni account" [Authorize] / [Cancel]
  4. User clicks Authorize
  5. Geni returns the user to your app's callback URL with an authorization token and a refresh token
  6. Your app uses the authorization token in API calls in accordance with the API docs
  7. Once the access token expires, your app can use the request_token call to get a new access token and refresh token, which you can repeat ad infinitum if the user doesn't de-authorize your app
  8. If the refresh token is being returned to something ephemeral like a web page JavaScript application, you can store the refresh token in cookies, in browser storage, or in a call to your backend servers.

Note: the user clicking Logout in the web interface should not cause access tokens or refresh tokens to fail.

Thanks Mike Stangel for the detailed response. I'm glad you are making progress.

I couldnt think of a better way of sharing screenshots with you so here are some links to Google Drive for images:

https://drive.google.com/file/d/1s7XDW6FvPyT0uWae9es97bXA4Osy-RGm/v... - original (23/03) authentication error captured from automated flow as screenshot on fail

https://drive.google.com/file/d/1WDREqohIfPqkyuTRhStM_7yeVc_y6I9D/v... - today (30/03) authentication error captured from automated flow as screenshot on fail. Looks identical to previous to me.

https://drive.google.com/file/d/1-Z7pWNBuMUZAUIMJCD3tgLJJ0YGW8xUr/v... - screenshot of error described yesterday as "weird behaviour" when utilising a regular Chrome session with Geni

I hope that these will be of some use to you - the automated auth flow is entirely reproducible but in a way I dont understand.

If I do 2 consecutive iterations of the same test then the first one ALWAYS succeeds and the second one ALWAYS fails with the auth error.

If I execute the second iteration without the first then it ALWAYS passes. So there is a dependency between the 2 iterations that didnt previously exist.

From the app side each iteration commences from an identical, reproducible start point as the previous access token is always removed as a first step.

I tried introducing a wait of 2 seconds between both iterations thinking it could be some kind of timing issue, but this didnt change anything.

If I run a full test suite then I get 13 initial failures.

If I rerun just the 13 failures, 5 of them then fail, then 2, then 1 until the last test runs and passes on its own.

This shows that each isolated interaction behaves as expected but something unpredictable is happening now when they are run in sequence. Previously the full test suite would run 100% successfully at the first attempt with no failures

The captcha thing is still a mystery to me as I dont have any automated tests that are coded to expect a login fail and only ever authenticate as part of a flow exactly as you have described in your last message.

However I do start each automated test with a request for a new access token (apart from the tests specifically testing the use of a saved token).

In normal interaction with my app the access and refresh tokens are stored and utilised as intended.

If there is going to be a need to respond to a Captcha in a normal, non-failure API flow, then I am going to be stumped to automate this unless a) I know when to expect the Captcha and b) I can automate the interaction by clicking a checkbox

An additional thought Mike Stangel

Since I have experienced the error message both with the API and without (as shown by the screenshots in my previous message), the thing that both situations have in common is a "logout".

In fact when running the automated scenarios, the only difference between the first scenario that completes successfully, and the second, that fails, is that there has been a logout in the intervening period.

Is it possible that the logout is messing things up?

Craig Andrew Miles evidently our deploy on 27 March was incomplete -- we've resolved that issue and also fixed one other potential issue. Can you give it another go and let me know the outcome? (and if it fails, be sure to post right after so I can use the timestamp of your discussion message to find the activity in our logs)

The logout shouldn't be messing things up, but there are a few points where the API authentication code intersects the cookie-based login session so I suppose anything is possible.

Hi Mike Stangel

Thanks for your ongoing efforts.

I started a test run at Mon Apr 3 10:47:56 AEST 203l so the first authentication attempt would occur sometime shortly after that, and obviously a while before this message (it is now 12:41 AEST).

Whatever you have done in the latest deployment seems to have definitely changed something. It seems that you are correct that this is a recaptcha issue, albeit one that I wasnt previously encountering, and not a 2FA issue.

Sometimes the login page seems to be requiring verification of the Captcha, and sometimes not. Here is an example where it is.

https://drive.google.com/file/d/1yejKnuGXlw1t6wYGUo559JVFVrL2Vwg6/v...

Regardless, when it is required, as I am not catering for it in my test flow, I now seem to be getting an error message that refers to security check failed as shown:

https://drive.google.com/file/d/17IuyO3nvaBeq8slUk2kpqoAgNJbkpvIP/v...

But another things is that the overall behaviour has now changed. As compared to before, on the first full run:
Some tests that were previously passing are now failing
Some tests that were previously failing are now passing

I am definitely getting more test failures on the first pass now (20 v 13) as I think that most logins to Geni are now failing (because of the recaptcha) although some are succeeding.

The set of failing tests still decreases when I rerun only the failures (=>14, then 5, then 2) and as before I can eventually get this down to zero with sufficient reruns.

In what circumstances should I expect to be challenged with a Captcha? This doesnt seem to be entirely predictable. The only login fails I should be getting now are as a result of the Captcha so I dont know how I got in this login failure cycle.

Unpredictability is a problem in automated testing as the Gherkin "language" doesnt cater for conditional actions so I would then have to implement "if there is a captcha visible then click it" in the underlying login code. But this is not possible as the whole purpose of a Captcha is to prevent this automation.

So that leaves me with either predictably being able to avoid the Captcha completely, or else having to continue with the "number of iterations" approach to get a full set of tests to pass. Am I able to avoid the Captcha completely during in automated tests Mike? If so, how?

Craig Andrew Miles it's not clear to me why you are automating the authorization step? That's explicitly how NOT to do things... granting access to a person's Geni account should require that person's permission as communicated to us by personally approving the authorization page. What exactly is your use case here? Why not get approval ONCE by a real person and then use refresh tokens to keep the authorization current?

Hi Mike Stangel

I thought you understood that I am only talking about automation of authorisation in the context of automated testing of the functionality of the app as part of the development lifecycle. There's no other way to do it. I cant manually intervene to authorise as part of an automated test suite, and furthermore I am actually testing the authorisation flow.

In normal usage of course the authorisation is manual. It's not really an issue anyway as I am currently the only use of the app but the principle still holds I know and it is not being violated.

Craig Andrew Miles when we test our software's behavior with external APIs we mock the API interface so as not to require banging on another site's production API while we run tests.

Showing all 23 posts

Create a free account or login to participate in this discussion

English (US)   eesti   Svenska   Español (España)   Français   עברית   Norsk (bokmål)   dansk   Nederlands   Deutsch   »
To enable the proper functioning and security of the website, we collect information via cookies as specified in our Cookie Policy. Geni does not use any third-party cookies.