DynamoDB FilterExpression error on hasMany relationships when owner is in composite primary key

[cheruvian]

How did you install the Amplify CLI?

npm

If applicable, what version of Node.js are you using?

20.19.3

Amplify CLI Version

1.8.0 (ampx)

What operating system are you using?

OSX

Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.

No manual changes made

Describe the bug

When querying a hasMany relationship with eager loading (selection set) where the related model has owner as part of its composite primary key, AppSync returns a DynamoDB error:

Filter Expression can only contain non-primary key attributes: Primary key attribute: owner
(Service: DynamoDb, Status Code: 400)

Root Cause

In ddb-references-generator.ts, the makeHasManyGetItemsConnectionWithKeyResolver method (lines ~90-185) incorrectly includes the entire authFilter in the DynamoDB filter expression without checking if any of its fields are part of the primary key.

Current Code (TypeScript source, lines ~121-130):

setup.push(
  setArgs,
  ifElse(
    not(isNullOrEmpty(authFilter)),
    compoundExpression([
      set(ref('filter'), authFilter),
      iff(
        not(isNullOrEmpty(ref('args.filter'))),
        set(ref('filter'), obj({ and: list([ref('filter'), ref('args.filter')]) }))
      ),
    ]),
    iff(not(isNullOrEmpty(ref('args.filter'))), set(ref('filter'), ref('args.filter')))
  ),
  // ... filter expression transformation follows
);

The Problem:
When owner (or any field from authFilter) is part of the primary key, it's already being used in the key condition expression (built by makeQueryExpression using the references parameter). DynamoDB explicitly prohibits using primary key attributes in filter expressions - they must only be in the key condition.

However, when owner is NOT part of the primary key, it SHOULD be in the filter expression for proper authorization.

Repository

• Repository: https://github.com/aws-amplify/amplify-category-api
• Package: @aws-amplify/graphql-relational-transformer
• Source File: packages/amplify-graphql-relational-transformer/src/resolver/ddb-references-generator.ts
• Method: makeHasManyGetItemsConnectionWithKeyResolver (lines ~90-185)
• Problematic Code: Lines ~121-156 (filter expression handling)
• Compiled File: lib/resolver/ddb-references-generator.js (distributed in npm package)

Environment

Versions (confirmed latest as of 2025-11-12):

• @aws-amplify/backend: 1.8.1
• @aws-amplify/backend-data: 1.8.1
• @aws-amplify/graphql-api-construct: 1.20.3
• @aws-amplify/graphql-relational-transformer: 3.1.x
• Node.js: 20.19.3
• Platform: macOS (Darwin 24.6.0)

Expected behavior

Eagerly loading should load properly.

Reproduction steps

Steps to Reproduce

1. Define Schema with owner in Composite Keys

const schema = a.schema({
  ParentModel: a
    .model({
      owner: a.string().required(),
      parentId: a.id().required(),
      children: a.hasMany("ChildModel", ["owner", "parentId"]),
    })
    .identifier(["owner", "parentId"])
    .authorization((allow) => [allow.ownerDefinedIn("owner")]),

  ChildModel: a
    .model({
      owner: a.string().required(),
      parentId: a.id().required(),
      childId: a.id().required(),
      name: a.string(),
    })
    .identifier(["owner", "parentId", "childId"])
    .authorization((allow) => [allow.ownerDefinedIn("owner")]),
});

2. Query with Eager Loading

Using Amplify Gen 2 Client (TypeScript/JavaScript):
Key Detail: The error only occurs when using a selection set to eager load the relationship. Querying parents without children works fine:

import { generateClient } from 'aws-amplify/data';
import type { Schema } from '@/amplify/data/resource';

const client = generateClient<Schema>();

// This query will fail when children relationship is included
const { data, errors } = await client.models.ParentModel.list({
  selectionSet: ['owner', 'parentId', 'children.*']
});

3. Error Occurs

In Amplify Client Response:

// The errors array will contain:
console.log(errors);
// [
//   {
//     "path": ["listParentModels", "items", 0, "children"],
//     "data": null,
//     "errorType": "DynamoDB:DynamoDbException",
//     "errorInfo": null,
//     "locations": [{ "line": 31, "column": 7, "sourceName": null }],
//     "message": "Filter Expression can only contain non-primary key attributes: Primary key attribute: owner (Service: DynamoDb, Status Code: 400, Request ID: P5TQVEB4M19UFGQ2GTBNKDKVS7VV4KQNSO5AEMVJF66Q9ASUAAJG)"
//   }
// ]


### Project Identifier

_No response_

### Log output

<details>

Put your logs below this line

</details>


### Additional information

## Proposed Fix

Filter out any primary key fields from `authFilter` before adding it to the filter expression. Fields that are in the `references` array (used for the key condition) should not be included in the filter expression.

### Approach 1: Filter authFilter at VTL runtime

Modify the VTL template generation to exclude key fields from authFilter:

```typescript
// In makeHasManyGetItemsConnectionWithKeyResolver method, around line 121
// Before building the filter expression, filter out fields that overlap with references

// Add VTL logic to remove key fields from authFilter
const filterAuthForNonKeyFields = compoundExpression([
  set(ref('nonKeyAuthFilter'), obj({})),
  forEach(
    ref('authFilterKey'),
    ref('authFilter.keySet()'),
    [
      // Only add to nonKeyAuthFilter if not in the key condition
      iff(
        not(raw(`$references.contains($authFilterKey)`)),
        qref(methodCall(ref('nonKeyAuthFilter.put'), ref('authFilterKey'), methodCall(ref('authFilter.get'), ref('authFilterKey'))))
      )
    ]
  )
]);

// Then use nonKeyAuthFilter instead of authFilter
setup.push(
  setArgs,
  filterAuthForNonKeyFields,  // Add filtering step
  ifElse(
    not(isNullOrEmpty(ref('nonKeyAuthFilter'))),  // Check nonKeyAuthFilter instead
    compoundExpression([
      set(ref('filter'), ref('nonKeyAuthFilter')),  // Use filtered version
      iff(
        not(isNullOrEmpty(ref('args.filter'))),
        set(ref('filter'), obj({ and: list([ref('filter'), ref('args.filter')]) }))
      ),
    ]),
    iff(not(isNullOrEmpty(ref('args.filter'))), set(ref('filter'), ref('args.filter')))
  ),
  // ... rest of filter transformation
);

Approach 2: Check at code generation time (Simpler)

Since the TypeScript code already has access to both references (key fields) and knows what's in authFilter, filter at generation time:

// Around line 115-120, before building the setup.push
const { field, indexName, limit, object, references, relatedType } = config;
const primaryKeyFields = getPrimaryKeyFields(object);
const dataSourceName = getModelDataSourceNameForTypeName(ctx, relatedType.name.value);

// NEW: Determine if we should skip authFilter
// If any reference field matches a typical auth field (owner, etc.), skip authFilter
const keyFieldsSet = new Set(references);
const shouldSkipAuthFilter = keyFieldsSet.has('owner'); // Can be expanded for other auth fields

// Then in the setup.push logic:
setup.push(
  setArgs,
  shouldSkipAuthFilter
    ? iff(not(isNullOrEmpty(ref('args.filter'))), set(ref('filter'), ref('args.filter')))
    : ifElse(
        not(isNullOrEmpty(authFilter)),
        compoundExpression([
          set(ref('filter'), authFilter),
          iff(
            not(isNullOrEmpty(ref('args.filter'))),
            set(ref('filter'), obj({ and: list([ref('filter'), ref('args.filter')]) }))
          ),
        ]),
        iff(not(isNullOrEmpty(ref('args.filter'))), set(ref('filter'), ref('args.filter')))
      ),
  // ... rest of setup
);

Approach 3: Most robust - Check relatedType's key schema

// Around line 115, get the related type's primary key fields
const relatedTypeObject = ctx.output.getObject(relatedType.name.value);
const relatedKeyFields = getPrimaryKeyFields(relatedTypeObject);
const relatedKeyFieldNames = new Set(relatedKeyFields);

// Check if any references overlap with the related model's keys
const referencesOverlapWithKeys = references.some(ref => relatedKeyFieldNames.has(ref));

// If references include key fields, skip authFilter entirely
const shouldSkipAuthFilter = referencesOverlapWithKeys;

Recommended Solution

Approach 2 (Quick Fix - Recommended for immediate release):
Check if references includes common authorization fields (owner, etc.) and conditionally skip authFilter when there's overlap. This is a minimal code change that solves the most common case.

Approach 3 (Most Robust - Recommended for long-term):
Check the related type's actual key schema using getPrimaryKeyFields(relatedTypeObject) and skip authFilter when any key field overlaps with references. This handles all edge cases correctly.

Approach 1 (Most Correct but Complex):
Filter authFilter at VTL runtime to remove only the overlapping fields. This preserves all non-key auth filters but requires more complex VTL generation.

Why Approach 2 or 3?

• Authorization on key fields is already enforced by the key condition expression
• The key condition restricts the query to matching records before filtering
• Amplify's authorization layer validates owner matches the authenticated user
• Both approaches maintain security while fixing the DynamoDB constraint violation

Proposed TypeScript Patch (Approach 2 - Recommended)

File: packages/amplify-graphql-relational-transformer/src/resolver/ddb-references-generator.ts

--- a/packages/amplify-graphql-relational-transformer/src/resolver/ddb-references-generator.ts
+++ b/packages/amplify-graphql-relational-transformer/src/resolver/ddb-references-generator.ts
@@ -115,6 +115,10 @@ export class DDBRelationalReferencesResolverGenerator extends DDBRelationalReso
     const dataSourceName = getModelDataSourceNameForTypeName(ctx, relatedType.name.value);
     const dataSource = ctx.api.host.getDataSource(dataSourceName)!;

+    // Check if authFilter would conflict with key condition
+    const keyFieldsSet = new Set(references);
+    const shouldSkipAuthFilter = keyFieldsSet.has('owner');
+
     const setup: Expression[] = [
       set(ref('limit'), ref(`util.defaultIfNull($context.args.limit, ${limit})`)),
       ...primaryKeyFields
@@ -125,14 +129,21 @@ export class DDBRelationalReferencesResolverGenerator extends DDBRelationalReso

     setup.push(
       setArgs,
-      ifElse(
-        not(isNullOrEmpty(authFilter)),
-        compoundExpression([
-          set(ref('filter'), authFilter),
-          iff(not(isNullOrEmpty(ref('args.filter'))), set(ref('filter'), obj({ and: list([ref('filter'), ref('args.filter')]) }))),
-        ]),
-        iff(not(isNullOrEmpty(ref('args.filter'))), set(ref('filter'), ref('args.filter')))
-      ),
+      // Only use authFilter if it won't conflict with key condition
+      shouldSkipAuthFilter
+        ? iff(not(isNullOrEmpty(ref('args.filter'))), set(ref('filter'), ref('args.filter')))
+        : ifElse(
+            not(isNullOrEmpty(authFilter)),
+            compoundExpression([
+              set(ref('filter'), authFilter),
+              iff(
+                not(isNullOrEmpty(ref('args.filter'))),
+                set(ref('filter'), obj({ and: list([ref('filter'), ref('args.filter')]) }))
+              ),
+            ]),
+            iff(not(isNullOrEmpty(ref('args.filter'))), set(ref('filter'), ref('args.filter')))
+          ),
+      // Continue with filter expression transformation...
       iff(
         not(isNullOrEmpty(ref('filter'))),
         compoundExpression([

Temporary Workaround Patch (Compiled JS)

Note: This workaround disables authFilter entirely. Use only until the proper fix is released.

Why Not Use patch-package?

The file that needs patching is a nested dependency, which patch-package doesn't handle well:

node_modules/
  @aws-amplify/graphql-api-construct/
    node_modules/
      @aws-amplify/graphql-relational-transformer/  ← Nested dependency!
        lib/resolver/ddb-references-generator.js

Problems with patch-package for nested dependencies:

1. patch-package expects packages to be in the root node_modules
2. Nested dependencies (packages inside other packages' node_modules) are not supported
3. The patch path would be too complex for patch-package to handle reliably
4. Different package managers (npm, yarn, pnpm) handle nested deps differently

Solution: Custom postinstall script (see workaround below)

Workaround Implementation

File to Patch: node_modules/@aws-amplify/graphql-api-construct/node_modules/@aws-amplify/graphql-relational-transformer/lib/resolver/ddb-references-generator.js

--- a/lib/resolver/ddb-references-generator.js
+++ b/lib/resolver/ddb-references-generator.js
@@ -60,10 +60,9 @@
                 (0, graphql_mapping_template_1.set)((0, graphql_mapping_template_1.ref)('query'), this.makeQueryExpression(references)),
             ];
+            // WORKAROUND: Skip authFilter to avoid DynamoDB error when owner is in primary key
-            setup.push(graphql_transformer_common_1.setArgs, (0, graphql_mapping_template_1.ifElse)((0, graphql_mapping_template_1.not)((0, graphql_mapping_template_1.isNullOrEmpty)(authFilter)), (0, graphql_mapping_template_1.compoundExpression)([
-                (0, graphql_mapping_template_1.set)((0, graphql_mapping_template_1.ref)('filter'), authFilter),
-                (0, graphql_mapping_template_1.iff)((0, graphql_mapping_template_1.not)((0, graphql_mapping_template_1.isNullOrEmpty)((0, graphql_mapping_template_1.ref)('args.filter'))), (0, graphql_mapping_template_1.set)((0, graphql_mapping_template_1.ref)('filter'), (0, graphql_mapping_template_1.obj)({ and: (0, graphql_mapping_template_1.list)([(0, graphql_mapping_template_1.ref)('filter'), (0, graphql_mapping_template_1.ref)('args.filter')]) }))),
-            ]), (0, graphql_mapping_template_1.iff)((0, graphql_mapping_template_1.not)((0, graphql_mapping_template_1.isNullOrEmpty)((0, graphql_mapping_template_1.ref)('args.filter'))), (0, graphql_mapping_template_1.set)((0, graphql_mapping_template_1.ref)('filter'), (0, graphql_mapping_template_1.ref)('args.filter')))), (0, graphql_mapping_template_1.iff)((0, graphql_mapping_template_1.not)((0, graphql_mapping_template_1.isNullOrEmpty)((0, graphql_mapping_template_1.ref)('filter'))), (0, graphql_mapping_template_1.compoundExpression)([
+            setup.push(graphql_transformer_common_1.setArgs, (0, graphql_mapping_template_1.iff)((0, graphql_mapping_template_1.not)((0, graphql_mapping_template_1.isNullOrEmpty)((0, graphql_mapping_template_1.ref)('args.filter'))), (0, graphql_mapping_template_1.set)((0, graphql_mapping_template_1.ref)('filter'), (0, graphql_mapping_template_1.ref)('args.filter'))), (0, graphql_mapping_template_1.iff)((0, graphql_mapping_template_1.not)((0, graphql_mapping_template_1.isNullOrEmpty)((0, graphql_mapping_template_1.ref)('filter'))), (0, graphql_mapping_template_1.compoundExpression)([
                 (0, graphql_mapping_template_1.set)((0, graphql_mapping_template_1.ref)('filterExpression'), (0, graphql_mapping_template_1.methodCall)((0, graphql_mapping_template_1.ref)('util.parseJson'), (0, graphql_mapping_template_1.methodCall)((0, graphql_mapping_template_1.ref)('util.transform.toDynamoDBFilterExpression'), (0, graphql_mapping_template_1.ref)('filter')))),

Impact

This bug affects any Amplify Gen 2 application using:

• Composite primary keys with owner field
• hasMany relationships with eager loading
• Owner-based authorization with allow.ownerDefinedIn("owner")

This is a common pattern for multi-tenant applications.

Workaround

Until this is fixed, users can apply the patch with this postinstall script.

Note: We use a custom postinstall script instead of patch-package because the file to patch is a nested dependency (node_modules/@aws-amplify/graphql-api-construct/node_modules/@aws-amplify/graphql-relational-transformer), which patch-package doesn't handle reliably. See "Why Not Use patch-package?" section above for details.

Create this file: scripts/apply-amplify-patch.cjs

// scripts/apply-amplify-patch.cjs
/**
 * Custom patch script for nested dependency.
 * Cannot use patch-package because @aws-amplify/graphql-relational-transformer
 * is nested inside @aws-amplify/graphql-api-construct/node_modules.
 */
const fs = require('fs');
const path = require('path');

const filePath = path.join(
  __dirname,
  '../node_modules/@aws-amplify/graphql-api-construct/node_modules/@aws-amplify/graphql-relational-transformer/lib/resolver/ddb-references-generator.js'
);

if (!fs.existsSync(filePath)) {
  console.log('⚠️  Amplify resolver file not found, skipping patch');
  process.exit(0);
}

const content = fs.readFileSync(filePath, 'utf8');

if (content.includes('// Don\'t put authFilter in filter expression')) {
  console.log('✅ Amplify resolver already patched');
  process.exit(0);
}

const oldCode = `            ];
            setup.push(graphql_transformer_common_1.setArgs, (0, graphql_mapping_template_1.ifElse)((0, graphql_mapping_template_1.not)((0, graphql_mapping_template_1.isNullOrEmpty)(authFilter)), (0, graphql_mapping_template_1.compoundExpression)([
                (0, graphql_mapping_template_1.set)((0, graphql_mapping_template_1.ref)('filter'), authFilter),
                (0, graphql_mapping_template_1.iff)((0, graphql_mapping_template_1.not)((0, graphql_mapping_template_1.isNullOrEmpty)((0, graphql_mapping_template_1.ref)('args.filter'))), (0, graphql_mapping_template_1.set)((0, graphql_mapping_template_1.ref)('filter'), (0, graphql_mapping_template_1.obj)({ and: (0, graphql_mapping_template_1.list)([(0, graphql_mapping_template_1.ref)('filter'), (0, graphql_mapping_template_1.ref)('args.filter')]) }))),
            ]), (0, graphql_mapping_template_1.iff)((0, graphql_mapping_template_1.not)((0, graphql_mapping_template_1.isNullOrEmpty)((0, graphql_mapping_template_1.ref)('args.filter'))), (0, graphql_mapping_template_1.set)((0, graphql_mapping_template_1.ref)('filter'), (0, graphql_mapping_template_1.ref)('args.filter')))), (0, graphql_mapping_template_1.iff)((0, graphql_mapping_template_1.not)((0, graphql_mapping_template_1.isNullOrEmpty)((0, graphql_mapping_template_1.ref)('filter'))), (0, graphql_mapping_template_1.compoundExpression)([`;

const newCode = `            ];
            // Don't put authFilter in filter expression - it contains primary key fields like 'owner'
            // which should be in the key condition expression, not filter expression
            setup.push(graphql_transformer_common_1.setArgs, (0, graphql_mapping_template_1.iff)((0, graphql_mapping_template_1.not)((0, graphql_mapping_template_1.isNullOrEmpty)((0, graphql_mapping_template_1.ref)('args.filter'))), (0, graphql_mapping_template_1.set)((0, graphql_mapping_template_1.ref)('filter'), (0, graphql_mapping_template_1.ref)('args.filter'))), (0, graphql_mapping_template_1.iff)((0, graphql_mapping_template_1.not)((0, graphql_mapping_template_1.isNullOrEmpty)((0, graphql_mapping_template_1.ref)('filter'))), (0, graphql_mapping_template_1.compoundExpression)([`;

if (!content.includes(oldCode)) {
  console.log('⚠️  Code pattern not found, file may have changed');
  process.exit(0);
}

const patchedContent = content.replace(oldCode, newCode);
fs.writeFileSync(filePath, patchedContent, 'utf8');

console.log('✅ Applied Amplify resolver patch successfully');

Add to package.json:

{
  "scripts": {
    "postinstall": "node scripts/apply-amplify-patch.cjs"
  }
}

Additional Context

When owner is part of the primary key:

1. The query already uses owner in the key condition (partition key)
2. Amplify's authorization layer validates the owner matches the authenticated user
3. DynamoDB will only return items matching the key condition anyway
4. Therefore, adding owner to the filter expression is both redundant and causes a DynamoDB error

The proper fix should:

• Detect which fields from authFilter are already in the key condition (via references)
• Only include non-key fields from authFilter in the filter expression
• This maintains full authorization while avoiding the DynamoDB constraint

Security Implications

The workaround (disabling authFilter) is safe when:

• ✅ owner is part of the primary key (fairly common case)
• ✅ Authorization is defined via allow.ownerDefinedIn("owner")
• ✅ The key condition already enforces owner-based access

The workaround may have issues when:

• ⚠️ Authorization uses additional fields beyond owner
• ⚠️ owner is NOT part of the primary key but still needs filtering
• ⚠️ Complex authorization rules with multiple conditions

For production use, the proper fix (filtering out only key fields from authFilter) should be implemented.

Before submitting, please confirm:

• I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
• I have removed any sensitive information from my code snippets and submission.

[pahud]

Hi @cheruvian,

Confirmed - this is a valid bug in makeHasManyGetItemsConnectionWithKeyResolver.

When owner is in the composite primary key, it's correctly added to the key condition but incorrectly also added to the filter expression via authFilter. DynamoDB prohibits primary key attributes in filter expressions.

Your Approach 2 (checking at code generation time) looks like the right fix:

• Check if references includes owner
• Skip authFilter when there's overlap
• Authorization is still enforced by the key condition

PROBLEM VISUALIZATION
================================================================================

    references array          authFilter object
    ┌──────────────┐         ┌─────────────────────┐
    │ "owner"      │         │ owner: {            │
    │ "parentId"   │         │   eq: "user123"     │
    └──────────────┘         │ }                   │
          │                  └─────────────────────┘
          │                           │
          ▼                           ▼
    Key Condition              Filter Expression
    Expression                 (CONFLICT!)
          │                           │
          └───────────┬───────────────┘
                      │
                      ▼
              Both contain "owner"
              DynamoDB rejects this!

amplify-category-api/packages/amplify-graphql-relational-transformer/src/resolver/ddb-references-generator.ts

Lines 117 to 123 in 857b383

	// add setup filter to query
	setup.push(
	setArgs,
	ifElse(
	not(isNullOrEmpty(authFilter)),
	compoundExpression([
	set(ref('filter'), authFilter),

Making this a p2 bug.

[cheruvian]

A fix for this issue has been submitted in PR #3383.

Root cause: When owner or other auth fields are part of the composite primary key, they're already used in the key condition expression. DynamoDB prohibits using primary key attributes in filter expressions, causing the error: "Filter Expression can only contain non-primary key attributes"

Fix: Filter out any authFilter fields that overlap with the key condition fields (references) before adding them to the filter expression in relational resolvers.