How to test the security controls in this repo:
control-library/— IaC-agnostic control definitions +catalog.yamlinfra_utils/policy_loader.py— loads/parameterizes controls- Item 1 (SCP): CMK-for-Memory SCP applied via
terraform/org-guardrails/ - Item 4 (resource policy): Memory in-account-only resource policy (
enable_resource_policies) - Items 5+6 (Guardrails + interceptor): egress Lambda interceptor + Bedrock Guardrail
on the Gateway (
enable_egress_filter) - Item 3 (Cedar): AgentCore policy engine with an explicit read permit on the
Gateway; Cedar's implicit default-deny covers everything else
(
enable_cedar,cedar_modeLOG_ONLY/ENFORCE) - Item 2 (VPCE + IAM): fine-grained AgentCore VPC endpoint policy + tightened execution
role (
enable_networking,org_id) - Item 7 (observability): SNS + EventBridge alerting on sensitive AgentCore API calls
(
enable_traceability)
There are two layers of testing:
- Part A — Local (no AWS account needed): loader, validator, unit tests,
cdk synth,terraform validate. This is what CI runs and is enough to review a PR. - Part B — Live (needs an AWS account): deploy and exercise the controls for real.
cd /path/to/agentcore-accelerator
# Python venv (repo targets 3.13; 3.12 also works locally)
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
# Node + CDK + Terraform
node --version # v18+
cdk --version # 2.2xx
terraform version # 1.3+source .venv/bin/activate
python - <<'PY'
from infra_utils.policy_loader import load_control, load_catalog
print("controls:", [c["id"] for c in load_catalog()["controls"]])
# SCP renders with its default param
print(load_control("scp.memory.enforce-cmk")["Statement"][0]["Condition"])
# Resource policy renders with injected params
print(load_control("resource-policy.memory.in-account-only", {
"account_id": "111122223333",
"memory_arn": "arn:aws:bedrock-agentcore:us-east-1:111122223333:memory/mem-abc",
"org_id": "o-example",
})["Statement"][1]["Condition"])
PYIf a heredoc misbehaves in your shell, drop the snippet into a temp
.pyfile and run it.
make validate-controls
# or: python scripts/validate_control_library.pyChecks every control file exists, is valid JSON, has matching <<sentinel>>/param
declarations, and renders with no unresolved tokens. Installs of checkov add a deep policy
scan; without it you get a single warning (non-fatal).
make test-controls
# or: python -m pytest tests/ -q → 12 passedcd terraform/org-guardrails
terraform fmt -check -recursive .
terraform init -backend=false
terraform validate # → Success! The configuration is valid.
# Confirm the SCP renders from the shared control-library file (no leftover <<sentinel>>):
printf 'local.cmk_scp_rendered\n' | terraform console
cd ../..Expect the printed policy to contain "bedrock-agentcore:CreateMemory" and the resolved
arn:aws:kms:*:*:key/* pattern (or your override), and no <<kms_key_arn_pattern>>.
The 8 gateway controls are rendered individually from the control-library
(local.gateway_scp_rendered) and then merged into ONE consolidated SCP
(local.gateway_scp_consolidated) to fit the 5-SCPs-per-target Organizations quota.
cd terraform/org-guardrails
terraform init -backend=false
terraform validate
# All 8 gateway controls are rendered from the library:
printf 'keys(local.gateway_scp_rendered)\n' | terraform console # → the 8 control names
# Sentinels resolved in the per-control renders:
printf 'local.gateway_scp_rendered["enforce-approved-idp"]\n' | terraform console | grep -c 'https://' # → 1 (DiscoveryUrl pattern injected)
printf 'local.gateway_scp_rendered["require-cmk"]\n' | terraform console | grep -c 'kms_key_arn_pattern' # → 0 (sentinel resolved; note console wraps output in <<EOT)
# The consolidated document contains every control's statements, no unresolved sentinels,
# and stays under the 5,120-character SCP quota:
printf 'local.gateway_scp_statements[*].Sid\n' | terraform console # → 10 unique Sids (require-cmk and require-policy-engine carry 2 each)
printf 'local.gateway_scp_consolidated\n' | terraform console | grep -c '<<' # → 0 (no unresolved <<sentinel>> tokens)
printf 'length(local.gateway_scp_consolidated)\n' | terraform console # → < 5120
cd ../..These SCPs use control-plane condition keys only: they constrain how a gateway may be configured, not who may invoke one.
GetWorkloadAccessTokenForUserId accepts the user identifier as an unverified string, so the
shipped default must exempt nobody. The exemption param defaults to a role ARN that cannot
exist, which makes the control a blanket deny until an operator supplies a real pattern.
cd terraform/org-guardrails
terraform init -backend=false
terraform validate
VARS='-var target_ids=["ou-example-11111111"]'
# Sentinel resolved, and the statement denies the right action:
printf 'length(regexall("<<", local.identity_userid_scp_rendered))\n' | terraform console $VARS # → 0
printf 'jsondecode(local.identity_userid_scp_rendered).Statement[0]\n' | terraform console $VARS # → Deny on GetWorkloadAccessTokenForUserId
# The default exemption names a role that does not exist, so nothing is exempt:
printf 'jsondecode(local.identity_userid_scp_rendered).Statement[0].Condition.ArnNotLike["aws:PrincipalArn"]\n' \
| terraform console $VARS # → arn:aws:iam::*:role/__no_principal_may_mint_tokens_by_userid__
# Sid must not collide with the consolidated gateway SCP's statements:
printf 'contains(local.gateway_scp_statements[*].Sid, "DenyWorkloadTokenForUserId")\n' \
| terraform console $VARS # → false
# Quota: 1 memory + 1 consolidated gateway + 1 identity = 3 of the 4 usable slots per target
# (5-per-target limit minus FullAWSAccess). This mirrors the attachments_per_target output.
printf '(var.enable_scp_memory_enforce_cmk ? 1 : 0) + (var.enable_gateway_scps ? 1 : 0) + (var.enable_scp_identity_deny_token_for_userid ? 1 : 0)\n' \
| terraform console $VARS # → 3
cd ../..The matching iam.identity-credential-provider-scoped reference policy is covered by the
loader tests in A3: it asserts the workload-identity and OAuth2-provider ARNs carry no
wildcard, since AgentCore does not bind a workload identity to the providers it may read and
IAM is the only thing scoping that access.
The Memory resource policy is attached with the native AWS::BedrockAgentCore::ResourcePolicy.
source .venv/bin/activate
export CDK_DEFAULT_ACCOUNT=111122223333 CDK_DEFAULT_REGION=us-east-1
# Flag OFF → no resource policy
rm -rf cdk.out
cdk synth agentcore-workshop-dev-memory 2>/dev/null \
| grep -c 'AWS::BedrockAgentCore::ResourcePolicy' # → 0
# Flag ON → one native ResourcePolicy with the rendered in-account-only policy
rm -rf cdk.out
cdk synth agentcore-workshop-dev-memory \
-c enable_resource_policies=true -c org_id=o-example123 2>/dev/null \
| grep -oE 'AWS::BedrockAgentCore::ResourcePolicy|o-example123|PrincipalOrgID' | sort | uniq -c
# → 1 each (resource policy present, org_id + PrincipalOrgID deny guard rendered)
# Flag ON but missing org_id → clean failure
rm -rf cdk.out
cdk synth agentcore-workshop-dev-memory -c enable_resource_policies=true 2>&1 \
| grep 'requires org_id'source .venv/bin/activate
export CDK_DEFAULT_ACCOUNT=111122223333 CDK_DEFAULT_REGION=us-east-1
# Flag ON → Guardrail + interceptor Lambda + interceptor config
rm -rf cdk.out
cdk synth agentcore-workshop-dev-gateway -c enable_egress_filter=true 2>/dev/null \
| grep -cE 'AWS::Bedrock::Guardrail|egress-interceptor|InterceptorConfigurations|ApplyGuardrail'
# → 4 (one match per pattern)
# Flag OFF → none of the above
rm -rf cdk.out
cdk synth agentcore-workshop-dev-gateway 2>/dev/null \
| grep -c 'AWS::Bedrock::Guardrail' # → 0source .venv/bin/activate
export CDK_DEFAULT_ACCOUNT=111122223333 CDK_DEFAULT_REGION=us-east-1
# Flag ON → policy engine + LOG_ONLY policy-engine config on the gateway. Cedar is
# implicit default-deny, so no blanket forbid ships (a matching forbid would override
# every permit — ENFORCE would then deny all tool calls, including the permitted reads).
rm -rf cdk.out
cdk synth agentcore-workshop-dev-gateway -c enable_cedar=true 2>/dev/null \
| grep -cE 'AWS::BedrockAgentCore::PolicyEngine|PolicyEngineConfiguration'
# → 2 (engine and the gateway's policy-engine config)
# Exactly one Cedar policy resource — the read permit — and no forbid statement:
rm -rf cdk.out
cdk synth agentcore-workshop-dev-gateway -c enable_cedar=true 2>/dev/null > /tmp/cedar-synth.yaml
grep -c 'Type: AWS::BedrockAgentCore::Policy$' /tmp/cedar-synth.yaml # → 1 (the read permit)
grep -c 'forbid(principal' /tmp/cedar-synth.yaml # → 0 (implicit default-deny)
rm -f /tmp/cedar-synth.yaml
# Try enforce mode
rm -rf cdk.out
cdk synth agentcore-workshop-dev-gateway -c enable_cedar=true -c cedar_mode=ENFORCE 2>/dev/null \
| grep -c 'ENFORCE' # → 1
# Flag OFF → no policy engine config
rm -rf cdk.out
cdk synth agentcore-workshop-dev-gateway 2>/dev/null | grep -c 'PolicyEngineConfiguration' # → 0The endpoint policy allows the AgentCore data-plane actions (invoke + OAuth PRM discovery)
with Principal: "*" and no IAM-principal conditions — OAuth/JWT callers carry no IAM
principal, so any principal-keyed condition would block them (per AWS docs). The org
restriction is a Deny scoped by a Null guard so it only fires for SigV4 callers outside
the org, never for principal-less OAuth traffic.
The networking VPC needs Availability Zone context to synth offline. Provide it once with a
temporary cdk.context.json (git-ignored), then synth:
source .venv/bin/activate
cat > cdk.context.json <<'JSON'
{ "availability-zones:account=111122223333:region=us-east-1": ["us-east-1a","us-east-1b"] }
JSON
rm -rf cdk.out
CDK_DEFAULT_ACCOUNT=111122223333 CDK_DEFAULT_REGION=us-east-1 \
cdk synth agentcore-workshop-dev-networking -c enable_networking=true -c org_id=o-example123 2>/dev/null \
> /tmp/vpce-synth.yaml
grep -c 'bedrock-agentcore.gateway' /tmp/vpce-synth.yaml # → 1 (the AgentCore gateway endpoint service)
grep -c 'PrincipalOrgID' /tmp/vpce-synth.yaml # → 2 (org condition + Null guard, both inside the SigV4-only Deny)
grep -c 'GetRuntimeProtectedResourceMetadata' /tmp/vpce-synth.yaml # → 1 (OAuth PRM discovery allowed with Principal "*")
grep -c '"Null":' /tmp/vpce-synth.yaml # → 1 (Null guard keeps the Deny off principal-less OAuth traffic)
rm -f cdk.context.json /tmp/vpce-synth.yaml && rm -rf cdk.outAlso confirm the tightened execution role scopes SSM to the project path (no "*"):
grep -A2 '"ssm:GetParameter"' -r infra_utils/agentcore_role.py
# resources = arn:aws:ssm:*:*:parameter/{project_name}/*source .venv/bin/activate
export CDK_DEFAULT_ACCOUNT=111122223333 CDK_DEFAULT_REGION=us-east-1
# Flag ON → SNS topic + EventBridge rule on sensitive AgentCore events
rm -rf cdk.out
cdk synth agentcore-workshop-dev-observability -c enable_traceability=true 2>/dev/null \
| grep -cE 'AWS::SNS::Topic|AWS::Events::Rule' # → 3 (topic, topic policy, rule)
# Flag OFF → none
rm -rf cdk.out
cdk synth agentcore-workshop-dev-observability 2>/dev/null | grep -c 'AWS::Events::Rule' # → 0source .venv/bin/activate
# Lint the new/changed security-controls files (all clean):
ruff check infra_utils/policy_loader.py scripts/validate_control_library.py \
stacks/gateway_stack.py stacks/memory_stack.py stacks/networking_stack.py \
stacks/observability_stack.py infra_utils/agentcore_role.py tests/
# New security-controls files are also ruff-format clean:
ruff format --check infra_utils/policy_loader.py scripts/validate_control_library.py \
tests/ tools/egress_interceptor/handler.pyNote: repo-wide
ruff check .reports pre-existing findings in older files; the lint backlog is tracked separately. CI lints the files changed in each merge request.
High-level, since these touch real resources. Use a non-production account. The SCP test additionally requires the Organizations management account (or delegated admin).
source .venv/bin/activate
export ORG_ID=o-yourorgid # required when resource policies are on
# Everything security-focused (networking + security + resource policies + egress filter):
./scripts/deploy.sh deploy --profile security-focused
# Or a single stack:
NON_INTERACTIVE=1 cdk deploy agentcore-workshop-dev-gateway -c enable_egress_filter=trueMEMORY_ARN=$(aws ssm get-parameter --name /agentcore-workshop/dev/memory/memory-arn \
--query Parameter.Value --output text)
aws bedrock-agentcore-control get-resource-policy --resource-arn "$MEMORY_ARN"
# Expect the in-account-only Allow + the aws:PrincipalOrgID Deny guard.aws bedrock list-guardrails | grep egress-guardrail
aws lambda get-function --function-name agentcore-workshop-dev-egress-interceptor \
--query 'Configuration.Environment.Variables'Then invoke the gateway tool through the agent with:
- a prompt-injection style payload → expect the REQUEST interceptor to block (CloudWatch logs
of the interceptor show
blocked (REQUEST)); - content containing an email / SSN → expect PII to be anonymized/blocked in the tool input/output.
Interceptor logs: /aws/lambda/agentcore-workshop-dev-egress-interceptor.
# The policy engine is attached to the gateway; deploy with enable_cedar=true (or the
# security-focused profile). Start in LOG_ONLY so nothing is blocked while you validate.
NON_INTERACTIVE=1 cdk deploy agentcore-workshop-dev-gateway \
-c enable_cedar=true -c cedar_mode=LOG_ONLYInvoke a permitted (read) tool and a non-permitted (write) tool through the agent, then check
the policy decision logs. Only after confirming the expected allow/deny decisions, redeploy
with -c cedar_mode=ENFORCE to actively block.
cd terraform/org-guardrails
terraform apply -var 'target_ids=["ou-abcd-1234wxyz"]'Then, from a member account under that OU, attempt CreateMemory without a KMS key —
expect an explicit AccessDenied from the SCP. Creating with a CMK should succeed.
# Endpoint policy is attached to the AgentCore gateway interface endpoint:
aws ec2 describe-vpc-endpoints \
--filters Name=service-name,Values="com.amazonaws.us-east-1.bedrock-agentcore.gateway" \
--query 'VpcEndpoints[0].PolicyDocument'
# Execution role SSM scope (should be the project path, not "*"):
aws iam get-role-policy --role-name agentcore-workshop-dev-orchestrator-role \
--policy-name <inline-policy-name>aws sns list-topics | grep agentcore-security-alerts
aws events list-rules --name-prefix agentcore-workshop-dev-agentcore-sensitive
# Subscribe an email/SNS endpoint, then trigger a sensitive event (e.g. PutResourcePolicy or
# DeleteGateway in a test resource) and confirm the alert fires. Requires CloudTrail
# management events (enabled by the security stack / enable_security)../scripts/deploy.sh destroy --profile security-focused
cd terraform/org-guardrails && terraform destroy && cd ../..Every pattern shares agent-code/ but builds its own Dockerfile, so a green deploy of one
proves nothing about the others. Run this matrix before a release: it has caught defects that
a single orchestrator invoke cannot reach (a stale container image, a missing forwarded header,
and two missing runtime-role permissions — none of which fail at synth or deploy time).
One redeploy of the orchestrator runtime per pattern, roughly 5–8 minutes each (CodeBuild):
export AWS_PROFILE=<your-profile> AWS_REGION=us-east-1
for p in orchestrator strands-agent langgraph-agent claude-sdk-agent claude-sdk-multi-agent; do
AGENT_PATTERN=$p NON_INTERACTIVE=1 ./scripts/deploy.sh deploy \
--stack agentcore-workshop-dev-runtime-orchestrator
python scripts/invoke.py "Reply with exactly: $p LIVE"
done
# The agui-* patterns speak AG-UI, not plain HTTP — use --agui
for p in agui-strands-agent agui-langgraph-agent; do
AGENT_PATTERN=$p NON_INTERACTIVE=1 ./scripts/deploy.sh deploy \
--stack agentcore-workshop-dev-runtime-orchestrator
python scripts/invoke.py --agui "Reply with exactly: $p LIVE"
doneThen check the tools actually loaded, which exercises the gateway MCP client and the
AgentCore Identity token vault (--agui for the AG-UI patterns):
python scripts/invoke.py "List the names of the tools you have available. Names only."
# Expect the gateway target plus execute_python_securely, e.g.
# sample-tool___text_analysis_tool, execute_python_securelyTwo things worth asserting beyond "it answered":
# The runtime really is on the pattern you asked for (protocol + fresh image)
ARN=$(aws ssm get-parameter --name /agentcore-workshop/dev/runtimes/orchestrator/arn \
--query Parameter.Value --output text)
aws bedrock-agentcore-control get-agent-runtime --agent-runtime-id "${ARN##*/}" \
--query '[agentRuntimeVersion,protocolConfiguration,agentRuntimeArtifact]'
# agui-* patterns → serverProtocol AGUI; all others → HTTP
# Each pattern pushed its own image tag (identical tags mean no rebuild happened)
aws ecr describe-images --repository-name agentcore-workshop-dev-orchestrator \
--query 'sort_by(imageDetails,&imagePushedAt)[-5:].{tags:imageTags,pushed:imagePushedAt}'Results as of the last full run (account-agnostic; re-run per release):
| Pattern | Protocol | Invoke | Gateway tools | Notes |
|---|---|---|---|---|
orchestrator |
HTTP | ✓ | ✓ | default |
strands-agent |
HTTP | ✓ | n/a | gateway/code-interpreter tools not wired in this pattern yet |
langgraph-agent |
HTTP | ✓ | ✓ | needs the memory data-plane IAM actions (ListEvents) |
claude-sdk-agent |
HTTP | ✓ | ✓ | replies include claude_session_id |
claude-sdk-multi-agent |
HTTP | ✓ | ✓ | delegates to the code-analyst subagent |
agui-strands-agent |
AGUI | ✓ | ✓ | typed SSE events; verify with --agui |
agui-langgraph-agent |
AGUI | ✓ | ✓ | slower first response (graph build per request) |
After the matrix run, close the loop on observability — this is what makes module 9's "end-to-end traces" claim true, and it reuses a span from the invokes above:
python scripts/check_observability.py --spans
# PASS: trace <id> searchable (5 spans, service agentcore_workshop_dev_orchestrator.DEFAULT)Opt-in (not in MODULE_VERIFY[9]) because span delivery lags invocation by a minute or
two. Two facts to keep you from "fixing" working telemetry (verified live, ADOT 0.16.0):
spans are searchable via Logs Insights on aws/spans — the path the Transaction Search
console uses — while the classic X-Ray APIs (batch-get-traces, get-trace-summaries)
only serve the indexed sample (Default rule: 1%), so an empty result there is expected,
not a delivery failure. And aws-opentelemetry-distro>=0.18 merely moves spans to
per-agent log groups; it is not needed for delivery or search.
If a pattern fails, read the container logs before changing anything — every failure so far named its own cause there:
aws logs filter-log-events \
--log-group-name "/aws/bedrock-agentcore/runtimes/<runtime-id>-DEFAULT" \
--start-time $(( ($(date +%s) - 900) * 1000 )) \
--query 'events[].message' --output text | grep -iE 'error|denied|traceback'- Interceptor event shape. The Lambda scans all string leaves in
mcp.gatewayRequest/mcp.gatewayResponse. The output contract (interceptorOutputVersion+transformedGateway*) is per AWS docs; confirm the nested request/response shape against live Gateway traces for your target type. - Guardrail version. The interceptor uses the guardrail
DRAFTversion by default. For production, publish a numbered version and pinGUARDRAIL_VERSION. - Cedar semantics. The Cedar policies (
control-library/cedar/) synthesize as text intoAWS::BedrockAgentCore::Policy; CloudFormation does not parse Cedar at synth time. Validate allow/deny behaviour on a live gateway inLOG_ONLYmode (inspect decision logs) before switchingcedar_mode=ENFORCE. Action names must match<TargetName>___<tool_name>. - VPCE service name. The AgentCore Gateway endpoint uses
com.amazonaws.<region>.bedrock-agentcore.gateway(confirmed). If you also need a Runtime data-plane endpoint, confirm its exact service name in your region before adding it. - Traceability depends on CloudTrail. The EventBridge rule only fires if CloudTrail
management events are being recorded (the
securitystack /enable_securityprovides a trail). Subscribe an endpoint to the SNS topic to actually receive alerts. - Resolved: the Memory resource policy now uses the native
AWS::BedrockAgentCore::ResourcePolicyL1 (no custom resource / SDK-name guess).