How to Query a DynamoDB Table
in an Authenticated and
Cost Effective Way?

Nowadays, when multi-tenant applications appear more and more, we see growing needs for some mechanisms permitting querying data per customer or tenant in shared databases. In this blog we want to propose a solution for querying a DynamoDB table by clients/applications while also assuring the privacy of each client request. Each client request should get access to its specific data only. Of course, each client would be authenticated with some IdP or some other sort of custom authentication method.

Let’s take a use case that is: Clients, authenticated with Cognito, need to query their personal data from a DynamoDB table in a way that each client can only access its own data.

This scenario can be solved in several ways, like, for example, using a Lambda function querying the DynamoDB table:

The problem is that when we speak about lots of clients and queries, we can quickly be faced with cost and concurrency problems induced by the Lambda usage. We’ll propose a solution that should be scalable as well as cost-effective.

 

We’ll use an AWS REST APIGW direct integration with the DynamoDB service. This direct integration permits avoiding Lambda usage.

REST APIGW also permits managing the authentication part.

Note: We use Cognito here, but you can use any IdP or custom authentication, as those are also integrated with REST APIGW through Lambda Authorizer.

Here is the diagram:

The flow is the following:

  1. The client performs authentication with Cognito and gets an identity token.
  2. The client sends a request to the REST APIGW for getting personal data—an identity token is attached to the request.
  3. REST APIGW validates the identity token.
  4. REST APIGW formats the request to include the username taken from the Identity Token as a parameter for the query to the DynamoDB table.
  5. REST APIGW passes the formatted request to the DynamoDB table.
  6. DynamoDB retrieves the corresponding results—personal data.
  7. REST APIGW passes the response to the client.

DynamoDB Table

For the sake of simplicity, we’ll define a DynamoDB table with a primary Key=username,- email, phone.

We’ll add some attributes for our users, which would be considered personal data

We want to perform queries based on the username only, so the primary key will fill the bill.

We’ll keep the default settings for the DynamoDB table.

Let’s create some items in the table (users’ personal data):

Cognito

Let’s create a Cognito User Pool. We’ll use basic settings:

For the rest, we’ll use default settings and no MFA. We won’t enable self-registration for simplicity.

We’ll define the OIDC attribute name as well as a custom attribute, mygroup. This will permit showing their appearance in the Identity Token later:

We’ll use Cognito Email Delivery for simplicity. We won’t use Cognito Hosted UI. We’ll use the basic app client—we’ll use ALLOW_ADMIN_USER_PASSWORD_AUTH.

It’s important to set permissions on the client application for the custom attribute to permit its usage:

Once the user pool is created, we’ll create user for our venerable Joe (we’ll use AWS CLI):

aws cognito-idp admin-create-user \
    --user-pool-id {User Pool ID} \
    --username joe \
    --user-attributes Name=name,Value="Joe Dalton" Name=custom:mygroup,Value="architects" \
    --message-action SUPPRESS
Answer:
{
    "User": {
        "Username": "joe",
        "Attributes": [
            {
                "Name": "name",
                "Value": "Joe Dalton"
            },
            {
                "Name": "custom:mygroup",
                "Value": "architects"
            },
            {
                "Name": "sub",
                "Value": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"
            }
        ],
        "UserCreateDate": "2024-08-29T13:59:40.759000+03:00",
        "UserLastModifiedDate": "2024-08-29T13:59:40.759000+03:00",
        "Enabled": true,
        "UserStatus": "FORCE_CHANGE_PASSWORD"
    }
}

Let’s reset his password:

aws cognito-idp admin-set-user-password \
--user-pool-id {User Pool ID} \
--username joe \
--password {Joe’s Password} \
--permanent

Let’s retrieve a token:

aws cognito-idp admin-initiate-auth --cli-input-json file://auth.json

Where auth.json is:

{
    "UserPoolId": "{User Pool ID}",
    "ClientId": "{Client ID}",
    "AuthFlow": "ADMIN_USER_PASSWORD_AUTH",
    "AuthParameters": {
        "USERNAME": "joe",
        "PASSWORD": "{Joe’s Password}"
    }
}

Answer:
{
    "ChallengeParameters": {},
    "AuthenticationResult": {
        "AccessToken": "eyJ–—----------------",
        "ExpiresIn": 3600,
        "TokenType": "Bearer",
        "RefreshToken": "eyJ—-----------------",
        "IdToken": "eyJ—------------------"
    }
}

When decoding the ID token, here is the payload:

{
  "sub": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee",
  "iss": "https://cognito-idp.eu-west-1.amazonaws.com/{User Pool ID}",
  "cognito:username": "joe",
  "origin_jti": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee",
  "aud": "xxxxxxxxxxxxexample",
  "event_id": "6e4021b2-28eb-469e-b0fc-97e1c9860146",
  "token_use": "id",
  "auth_time": 1724929571,
  "name": "Joe Dalton",
  "exp": 1724933171,
  "custom:mygroup": "architects",
  "iat": 1724929571,
  "jti": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"

We’ll use the cognito:username claim as the key for the queries but we could use any of the claims. For example, here we defined a custom claim custom:mygroup.

REST APIGW

We’ll create an authorizer for our Cognito User Pool:

We’ll define an IAM role that will be used by the REST API Gateway to query the DynamoDB table. Following best practices, this IAM role should be limited to the specific request sent to the DynamoDB table.

We’ll then create a regional APIGW with a GET method using the authorizer we just defined:

On the integration request, we’ll define DynamoDB with the pre-defined IAM role:

Please pay attention to the mapping template, which is where the magic happens. It includes the details of the request to the DynamoDB table:

As you can see, we use the cognito:username claim taken from the Identity Token as a key on the query. By doing this, we restrict the query to specific client item(s).

We’ll then deploy the APIGW.

We’ll now run the command for testing the APIGW, using our identity token:

curl -H “Authorization: eyJ—————-" https://{My APIGW ID}.execute-api.eu-west-1.amazonaws.com/prod

Answer:

{"Count":1,"Items":[{"email":{"S":"joe@email.com"},"phone":{"S":"5555-5555"},"username":{"S":"joe"}}],"ScannedCount":1}%  

As you can see, the request did not include any information about the client; the whole client’s identity was taken from the Identity Token, and thus the key used in the query was based on this identity. The data received by the client was then filtered, and only his personal data was delivered.

By using this solution, we succeeded in implementing filtering logic based on authentication at the APIGW level. This permits avoiding implementing Lambda with its cost and limitations. As a general idea, it’s worth investigating APIGW capabilities with AWS-integrated services to avoid additional processing as much as possible.

Hope this will help you in your future designs.

AlllCloud’s experts are always here to assist. Contact us today

Mikael Fassi
Solutions Architect