API Gateway, Cognito and Python

This post is about working with Cognito and API Gateway from Python. It also briefly explains JSON Web Tokens in the process.

I’ll go through setting up an API that calls a Lambda function and a Cognito user pool that is used to authorize calls to that API. I created a user, signed in to Cognito, then did a POST to the API. The parameters sent to Lambda are registerd in a DynamoDB table. All code in Python. There are plenty of instructions for JavaScript, Python not so much.


My goal was to understand how things work. Security settings were kept to a minimum in order to keep it simple. Not recommended for real world applications.

Preparations - setting up Cognito, API Gateway, Lambda and DynamoDB

Set up Cognito

Cognito has two major components, user pools and identity pools.

A Cognito user pool is a user directory. That is, a list of users with their associated password, email address and other configurable attributes. It is used for authentication. In simple words, when a user attempts to log in, the username and password are checked against the directory. It confirms that they are correct, but doesn’t give any information about what resources the user has access to.

A Cognito identity pool on the other hand deals with authorization. It can be used to check if a user has access to a certain resource or not, but it doesn’t know anything about a user’s credentials. A Cognito identity pool is used to give access to AWS resources (S3, DynamoDB tables, etc.).

An identity pool is configured with a ‘provider’ that deals with authorization. One of the possible providers is a Cognito user pool. I didn’t use identity pools.

  1. Create a user pool. The defaults are acceptable for my purpose.
  2. Set up an app client.
    • Uncheck the box to generate a secret. if a secret is present it must be sent as a hash involving secret + username + client_ID. One less thing to deal with.
    • Enable username-password (NON-SRP) flow. This is a form of authentication where the username and password are sent with no encryption. Other than HTTPS, that is. The least secure, but also the most obvious and easiest to work with. Not for real systems.
  3. Make a note of the pool ID, found under General Settings, and the app client ID, found under App clients.

Set up DynamoDB

We’ll use a DynamoDB table to record the information that is passed on by the API Gateway when a call is made with Cognito credentials.

Create a simple table named tst_logins with a primary key called user_id. I did not accept defaults, but disabled auto scaling to avoid surprise costs.

Set up Lambda

I created a Lambda function to update the DynamoDB table with the user name, login time, and the values of “event” and “context” parameters received from API gateway.

  1. Create an IAM role for Lambda. Give it write access to DynamoDB.
  2. Create the Lambda function.
    • I named it tst_insert_logins.
    • Python 3.6 at least as runtime.
    • Give it the role created above.
  3. Function code. The format of event in this case is described in the official docs.
import json
import boto3
from datetime import datetime

def lambda_handler(event, context):
    ddb = boto3.client('dynamodb')
    ddb_put = ddb.put_item(
                'user_id': {
                    'S': 'id-of-user'
                'event': {
                    'S': json.dumps(event)
                'context': {
                    'S': str(vars(context))
                'context_identity': {
                    'S': str(dir(context.identity))
                'insert_time': {
                    'S': str(datetime.now())
    return {
        'statusCode': 200,
        'body': json.dumps(ddb_put)

Set up API Gateway

The final piece, which connects the outside world to our Lambda function. We’ll call the API from Python, with a Cognito token. The token will get passed to our Lambda function and be recorded in DynamoDB.

  1. Create a new REST API. The final URL will be something like https://4a48x6598i.execute-api.eu-central-1.amazonaws.com/prod/insert-login.
  2. Create the insert-login resource.
    • Click on “Actions” button, then “Create Resource”.
    • Put “insert-login” for Resource Name, then click “Create Resource”.
  3. Create a POST method for insert-login which will call our Lambda function.
    • With “insert-login” (not “/”) selected click on “Actions” again, then “Create Method”. Should open a list box under “insert-login”. Choose “POST” from the list and click the checkbox.
    • For “Integration Type” choose “Lambda Function”.
    • Check the checkbox next to “Use Lambda Proxy integration”. With proxy integration details about the request received by the API, like the Cognito username, will be passed to Lambda inside the event parameter.
    • Fill in the Lambda Function and Save.
  4. Configure the API to use the Cognito user pool for authorization.
    • Go to “Authorizers” on the left navigation bar and click on “Create New Authorizer”.
    • Choose “Cognito” as Type, choose the user pool and put “Authorization” in the Token Source field. Leave “Token Validation” empty. With this setup the ID token from Cognito will be used for authorization. It is also possible to use the access token. Save.
    • Go back to “Resources”, choose the POST method under insert-login. Click on Method Request in the right panel.
    • Under “Settings”, click on the pencil next to Authorization. Select the authorizer that was just created from the list and click on the check sign to save the setting. Might need to force a page refresh if the authorizer doesn’t show in the list.
  5. Finally, deploy the API.
    • Click on “Actions”, then “Deploy API”
    • Choose “New stage” and fill in “prod” (or something else) for Stage name. Click “Deploy”.
    • Select “Stages” in the left nav bar, expand “prod” and select POST. Note the Invoke URL.

The code

Operations on Cognito user pools can be done from boto3 using the CognitoidentityProvider service. Which makes sense if you remember that Cognito user pools are providers for Cognito identity pools.

This is example code, actual working code at the end.

Create a new user

To have a user register themselves use the sign_up() method.

import boto3

cidp = boto3.client('cognito-idp')
r = cidp.sign_up(
        UserAttributes=[{'Name': 'email',
                         'Value': 'cognito-py-demo@notmydomain.ro'}])

The ClientId parameter must be the ID of the app that we registered with the user pool. If the client app also has a secret associated with it that has to be sent also.

The response (r):

         "date":"Tue, 14 May 2019 15:47:34 GMT",

UserSub is a unique user ID generated by Cognito. “Sub” stands for Subject, which is one of the fields that can appear inside the claims set of a JWT token. More on that in the login section.

Note that it says UserConfirmed is False. A user can be confirmed either by email/phone, or by an administrator. For brevity this user is confirmed using admin powers:

r = cidp.admin_confirm_sign_up(


         "date":"Tue, 14 May 2019 15:47:35 GMT",

The sign up procedure is complete now, the user can log in.

Logging in

The login process follows an authentication flow that can ask the user to answer to multiple challenges. A challenge can be an MFA code, for example. It starts with calling InitiateAuth API with the chosen authentication flow passed as a parameters. The server will answer with the next challenge, or with a set of tokens if all challenges have been met. If there’s another challenge, the client will answer to it by calling RespondToAuthChallenge.

The standard flow for clients is USER_SRP_AUTH. It uses the Secure Remote Protocol. The password is never sent over the wire. Instead, some keys are generated and exchanged based on the password, such that the server can verify that the client does know the password without actually receiving it.

The SDKs for mobile and JavaScript have methods built in to easily deal with SRP. The Python SDK doesn’t though, so I avoided it.

There is another flow, USER_PASSWORD_AUTH, which is much less secure as the password is simply sent to the server. It’s intended for migrating users from another authentication system to Cognito user pools, and not for production. It’s also easier to use in Python. The app client associated with the pool has to be configured to allow it.

import boto3

cidp = boto3.client('cognito-idp')
r = cidp.initiate_auth(
            'USERNAME': 'cognito-py-demo',
            'PASSWORD': 'D0lphins!'},

And the response is


         "date":"Thu, 16 May 2019 14:54:31 GMT",

The response contains three tokens. And ID Token, an Access Token and a Refresh Token. The above are real tokens and they can be decoded here.

The ID Token contains information about the identity of the user. Things like email and username. It is usually used to call APIs set up in API gateway.

The Access Token is used to grant access to resources. It can also be used in API Gateway if OAuth Scopes are specified.

The Refresh Token is used to get a new set of tokens. The ones provided by Cognito expire in one hour and that setting is not configurable at this time. A new set can be obtained by calling InitiateAuth API again with REFRESH_TOKEN_AUTH as AuthFlow.

Decoding and checking tokens

A JSON Web Token is made up of three parts that are Base64url encoded and separated by a dot. It’s basically


We can take each of those parts, decode them, and use the contents.

The header is a JSON document that contains information about the algorithm used for the signature.

The payload is where the information we need is. It’s also in JSON format and the fields are called “claims”.

The signature is used to validate the token. In Cognito’s case the signature consists of header+payload, encrypted (signed) with one of two private keys associated with the user pool. The public keys that can be used to decrypt and check the signature can be found at https://cognito-idp.{region}.amazonaws.com/{userPoolId}/.well-known/jwks.json.

To decode these tokens I used python-jose library. JOSE stands for JavaScript Object Signing and Encryption. This library has a simple way to decode tokens and check the signature and some of the claims at the same time, jose.jwt.decode():

import requests
from jose import jwt
from pprint import pprint

def decode_token(token):
    # build the URL where the public keys are
    jwks_url = 'https://cognito-idp.{}.amazonaws.com/{}/' \
    # get the keys
    jwks = requests.get(jwks_url).json()
    pprint(jwt.decode(token, jwks))

Result when passing it the access token:


If the signature is wrong, or the token expired, or some of the claims don’t check, errors will be raised.

This is the easy way. For the purpose of learning, here’s how it can be done the semi-hard way, where each part of the token is verified separately. The same library is used. Code is commented.

import requests
from jose import jwt
from pprint import pprint

 def check_token(token):
    # https://amzn.to/2vUwFx7
    # Decode the headers and payload without verifying signature
    access_headers = jwt.get_unverified_header(token)
    print('Token headers:')
    access_claims = jwt.get_unverified_claims(token)
    print('Token claims:')
    # Now let's check the signature, step by step.
    # As seen in https://bit.ly/2E3fAFP
    print('Checking key manually')
    # Retrieve JSON Web Key Set, which contains two public keys
    # corresponding to the two private keys that could be used
    # to sign the token.
    jwks_url = 'https://cognito-idp.{}.amazonaws.com/{}/' \
    r = requests.get(jwks_url)
    if r.status_code == 200:
        jwks = r.json()
        raise 'Did not retrieve JWKS, got {}'.format(r._status_code)
    # The token header contains a field 'kid', which stands for Key ID.
    # The JWKS also contains two 'kid' fields, one for each key. The
    # 'kid' in the header tells us which public key must be used
    # to verify the signature.
    kid = access_headers['kid']
    # get the public key that corresponds to the key id from headers
    key_index = -1
    for i in range(len(jwks['keys'])):
        if kid == jwks['keys'][i]['kid']:
            key_index = i
    if key_index == -1:
        print('Public key not found, can not verify token')
        # convert public key to the proper format
        public_key = jwk.construct(jwks['keys'][key_index])
        # get claims and signature from token
        claims, encoded_signature = token.rsplit('.', 1)
        # decrypted signature must match header and payload
        decoded_signature = base64url_decode(
        if not public_key.verify(claims.encode("utf8"),
            print('Signature verification failed')
            print('Signature successfully verified')

To verify the token: - Decode header and payload, which are base64url encoded - Get the JWKS containing the public key by accessing the Cognito URL - Decode the signature, also base64url encoded - Decrypt the signature using the public key with a kid that matches the one in the header - Compare that the decrypted signature equals the base64url encoded header and payload, concatenated with a ‘.’ between them

Calling the API with a Cognito token

Finally. This one is easy. Just POST to the API URL, passing the ID token in the Authorization header. The URL can be seen in the API Gateway console, noted above.

import requests

headers = {'Authorization': token}
url = 'https://4a48x6598i.execute-api.eu-central-1.amazonaws.com/' \
r = requests.post(url, headers=headers)

Assuming everything went well, here’s the event field in DynamoDB:

      "Accept-Encoding":"gzip, deflate",
         "gzip, deflate"
            "exp":"Thu May 16 15:54:30 UTC 2019",
            "iat":"Thu May 16 14:54:30 UTC 2019",
      "requestTime":"16/May/2019:14:54:31 +0000",

That’s what I was looking for.

I put the entire Python code in a gist on Github. When executed it will go through all the steps detailed here.