Skip to content

Commit 7fd5bcf

Browse files
Raunak GuptaRaunak Gupta
Raunak Gupta
authored and
Raunak Gupta
committed
docs: add private-subnet RDS networking guidance and API resource policy
- Add docs/private-rds-networking.md: NAT gateway setup, the private API Gateway (VPCe) limitation with sp_invoke_external_rest_endpoint, test matrix, and DNS caching note. - Add docs/api-resource-policy.md: CLI steps to restrict the regional API to the NAT egress IP. - Link both from README (layout tree + new Networking section).
1 parent 3a251e8 commit 7fd5bcf

3 files changed

Lines changed: 241 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,8 @@ wrapper: the `[SqlFunction]`/`FillRow` shell becomes a Lambda handler.
4646
├── sql/ # Ordered T-SQL scripts (setup, failure, 2025 config, tests, cleanup)
4747
├── sample-data/ # CSV test files with multi-line fields
4848
└── docs/ # Additional documentation
49+
├── private-rds-networking.md # NAT gateway setup + VPCe limitation findings
50+
└── api-resource-policy.md # CLI steps to add NAT-IP resource policy
4951
```
5052

5153
## Prerequisites
@@ -380,6 +382,25 @@ EXEC dbo.ParseCSV_Lambda @csv, N',', 1;
380382
- **Response path** is `$.result.rows` with proxy integration; for non-proxy it
381383
is inside an escaped `$.result.body` string. Use `PRINT @raw` to confirm.
382384

385+
## Networking (private-subnet RDS)
386+
387+
If your RDS instance is in a **private subnet** (no public IP), it has no
388+
outbound internet path by default. `sp_invoke_external_rest_endpoint` resolves
389+
the API hostname via public DNS and calls it over the internet, so you must add
390+
a **NAT gateway** and a `0.0.0.0/0` route on the RDS subnets. RDS stays private
391+
(egress-only). Without this, calls fail with `HRESULT: 0x80072ee7`
392+
(`NAME_NOT_RESOLVED`).
393+
394+
A **private** API Gateway endpoint (interface VPC endpoint) does **not** work
395+
with this RDS feature — the managed engine does not resolve to the VPCe and its
396+
requests do not carry `aws:sourceVpce`, so a private API returns 403. Use a
397+
regional API and restrict it by the NAT egress IP instead. Full findings and the
398+
NAT setup: [Networking for private-subnet RDS](docs/private-rds-networking.md).
399+
400+
To lock the regional API down to your NAT Elastic IP (recommended for private
401+
RDS, and usable for public RDS too), add an API Gateway resource policy:
402+
[Add a resource policy to restrict the API](docs/api-resource-policy.md).
403+
383404
## Performance and cost
384405

385406
Each call is a network round-trip, so this pattern suits batch/file-level

‎docs/api-resource-policy.md‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
# Add a resource policy to restrict the regional API to the NAT egress IP
2+
3+
When the RDS instance is in a private subnet and egresses through a NAT gateway,
4+
add a resource policy to the API so that only your NAT Elastic IP can invoke it.
5+
This is defence-in-depth on top of the API key.
6+
7+
## When to apply
8+
9+
- After Step 3.6 (grant Lambda permission) and before or during Step 3.7
10+
(deploy). Resource policy changes only take effect after a `create-deployment`.
11+
- Or at any time — apply the policy and then redeploy the stage.
12+
13+
## Steps (AWS CLI)
14+
15+
The commands below use the same shell variables as the main walkthrough
16+
(`$API_ID`, `$REGION`, `$ACCOUNT_ID`). Run them in the same terminal session,
17+
or re-export those variables first.
18+
19+
### 1. Capture the NAT gateway's Elastic IP
20+
21+
```bash
22+
NAT_IP=$(aws ec2 describe-nat-gateways \
23+
--nat-gateway-ids <YOUR_NAT_GATEWAY_ID> \
24+
--query 'NatGateways[0].NatGatewayAddresses[0].PublicIp' \
25+
--output text)
26+
27+
echo "NAT egress IP: $NAT_IP"
28+
```
29+
30+
Replace `<YOUR_NAT_GATEWAY_ID>` with the NAT gateway ID in the RDS VPC (e.g.,
31+
`nat-0cf203810f0605bdf`).
32+
33+
### 2. Build and attach the resource policy
34+
35+
```bash
36+
POLICY=$(cat <<EOF
37+
{
38+
"Version": "2012-10-17",
39+
"Statement": [
40+
{
41+
"Effect": "Deny",
42+
"Principal": "*",
43+
"Action": "execute-api:Invoke",
44+
"Resource": "arn:aws:execute-api:${REGION}:${ACCOUNT_ID}:${API_ID}/*",
45+
"Condition": {
46+
"NotIpAddress": { "aws:SourceIp": "${NAT_IP}/32" }
47+
}
48+
},
49+
{
50+
"Effect": "Allow",
51+
"Principal": "*",
52+
"Action": "execute-api:Invoke",
53+
"Resource": "arn:aws:execute-api:${REGION}:${ACCOUNT_ID}:${API_ID}/*"
54+
}
55+
]
56+
}
57+
EOF
58+
)
59+
60+
# Escape inner double quotes for the patch value
61+
ESCAPED_POLICY=$(printf '%s' "$POLICY" | sed 's/"/\\"/g')
62+
63+
aws apigateway update-rest-api \
64+
--rest-api-id "$API_ID" \
65+
--patch-operations "op=replace,path=/policy,value=\"$ESCAPED_POLICY\""
66+
```
67+
68+
### 3. Redeploy the stage
69+
70+
```bash
71+
aws apigateway create-deployment \
72+
--rest-api-id "$API_ID" \
73+
--stage-name prod
74+
```
75+
76+
Resource policy changes are **not active** until you redeploy.
77+
78+
## How the policy works
79+
80+
The two statements evaluate with "explicit Deny wins":
81+
82+
| Statement | Effect | Condition |
83+
| --- | --- | --- |
84+
| 1 | Deny | Source IP is **not** the NAT EIP |
85+
| 2 | Allow | Everyone |
86+
87+
Net result: only requests arriving from the NAT gateway's Elastic IP pass. All
88+
other source IPs are denied with HTTP 403 before the request reaches the Lambda.
89+
Combined with `--api-key-required` on the method, the endpoint is restricted to
90+
your VPC's egress IP **and** a valid API key.
91+
92+
## Works for public RDS too
93+
94+
The policy restricts by source IP, not by whether the RDS instance is private.
95+
If you have a publicly accessible RDS instance, the same pattern applies — just
96+
use the public IP that RDS egresses from (check with `api.ipify.org` as shown in
97+
[`docs/private-rds-networking.md`](private-rds-networking.md)).
98+
99+
## Operational notes
100+
101+
- **NAT EIP dependency.** The policy pins a specific IP. If the NAT gateway or
102+
its Elastic IP is deleted and recreated, the egress IP changes. Update the
103+
policy's `aws:SourceIp` value and redeploy, or RDS receives HTTP 403.
104+
- **Multiple NAT gateways.** If you deploy one NAT per AZ for high availability,
105+
add both EIPs to the condition using an array:
106+
```json
107+
"aws:SourceIp": ["<NAT_IP_AZ1>/32", "<NAT_IP_AZ2>/32"]
108+
```
109+
- **Viewing the policy.** Check the currently deployed policy at any time:
110+
```bash
111+
aws apigateway get-rest-api --rest-api-id "$API_ID" --query 'policy'
112+
```
113+
The returned value is double-escaped JSON. Pipe through
114+
`sed 's/\\"/"/g; s/\\\\//g'` for readability.

‎docs/private-rds-networking.md‎

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
# Networking for private-subnet RDS with `sp_invoke_external_rest_endpoint`
2+
3+
When your Amazon RDS for SQL Server 2025 instance is in a **private subnet**
4+
(no public IP, `PubliclyAccessible = false`), `sp_invoke_external_rest_endpoint`
5+
requires outbound internet access to reach the regional API Gateway endpoint.
6+
This page covers the required networking setup and documents a known limitation
7+
with private API Gateway endpoints.
8+
9+
## What you need (NAT gateway)
10+
11+
`sp_invoke_external_rest_endpoint` resolves the API hostname via **public DNS**
12+
and makes an outbound HTTPS call over the internet. A private-subnet RDS instance
13+
has no internet path by default, so you must provide one:
14+
15+
1. **Create a public subnet** in the same VPC with a route to an internet gateway.
16+
2. **Create a NAT gateway** in that public subnet (requires an Elastic IP).
17+
3. **Add a default route** (`0.0.0.0/0 → NAT gateway`) to the route table used
18+
by the RDS subnets.
19+
20+
After this, outbound traffic from the RDS subnets exits via the NAT gateway's
21+
Elastic IP. RDS stays private (no inbound path from the internet).
22+
23+
### Verify the egress IP
24+
25+
From SQL Server, confirm the NAT path works and note the egress IP:
26+
27+
```sql
28+
DECLARE @ret INT, @response NVARCHAR(MAX);
29+
EXEC @ret = sp_invoke_external_rest_endpoint
30+
@url = N'https://api.ipify.org?format=json',
31+
@method = 'GET',
32+
@timeout = 30,
33+
@response = @response OUTPUT;
34+
SELECT @ret AS ReturnCode, @response AS Response;
35+
-- The "ip" field in the result is the public IP API Gateway sees.
36+
```
37+
38+
The returned IP is what you allowlist in the API Gateway resource policy (see
39+
[`docs/api-resource-policy.md`](api-resource-policy.md)).
40+
41+
### Without the NAT gateway
42+
43+
If there is no outbound route, `sp_invoke_external_rest_endpoint` fails with:
44+
45+
```
46+
Msg 31608, Level 16, State 24 ...
47+
An error occurred, failed to communicate with the external rest endpoint.
48+
HRESULT: 0x80072ee7.
49+
```
50+
51+
`0x80072ee7` is WinHTTP `ERROR_WINHTTP_NAME_NOT_RESOLVED` — DNS resolution
52+
failed because the engine had no path to a public resolver or endpoint.
53+
54+
## Why a private API Gateway endpoint does NOT work
55+
56+
You might expect to use a **private** API Gateway endpoint (endpoint type
57+
`PRIVATE`) with an interface VPC endpoint (`com.amazonaws.<region>.execute-api`)
58+
to keep all traffic inside the VPC. This **does not work** with
59+
`sp_invoke_external_rest_endpoint` on Amazon RDS.
60+
61+
### What was tested
62+
63+
| Test | Result |
64+
| --- | --- |
65+
| Private API + VPCe private DNS enabled, standard hostname from RDS | `0x80072ee7` NAME_NOT_RESOLVED |
66+
| Private API + VPCe hostname with `Host` / `x-apigw-api-id` header from RDS | HTTP 403 ForbiddenException (`aws:sourceVpce` not populated) |
67+
| Same private API + VPCe, called from Linux EC2 in the same subnet | HTTP 200 (success) |
68+
| Regional API via NAT from RDS | HTTP 200 (success) |
69+
| Public endpoint (`api.ipify.org`) via NAT from RDS | HTTP 200 (success) |
70+
71+
### Root cause
72+
73+
The managed RDS engine's `sp_invoke_external_rest_endpoint` implementation:
74+
75+
1. **Does not resolve the API hostname to the VPC endpoint's private ENI IPs**,
76+
even when VPCe private DNS is enabled and functioning for other workloads
77+
(EC2) in the same subnet.
78+
2. **Does not populate `aws:sourceVpce`** in the request context when it does
79+
reach API Gateway (e.g., via the VPCe-specific hostname), so a private API's
80+
resource policy condition `StringNotEquals: aws:sourceVpce` always denies.
81+
82+
An EC2 instance in the same VPC and subnet, using the same security group,
83+
successfully resolves to the VPCe and receives HTTP 200 — proving the VPCe
84+
setup is correct and the limitation is specific to the managed RDS engine.
85+
86+
### Recommendation
87+
88+
Use a **regional** (public) API Gateway endpoint and restrict access with:
89+
90+
- A **resource policy** that allows only the NAT gateway's Elastic IP
91+
(`aws:SourceIp`). See [`docs/api-resource-policy.md`](api-resource-policy.md).
92+
- The **API key** (already required by the method and stored in the
93+
`DATABASE SCOPED CREDENTIAL`).
94+
95+
This gives you IP-level network restriction plus application-level
96+
authentication, while avoiding the VPCe path that RDS cannot use.
97+
98+
## DNS caching note
99+
100+
If the API hostname was previously associated with a private hosted zone (via
101+
VPCe private DNS), the RDS engine may cache stale/negative DNS answers for up to
102+
~15 minutes after the override is removed. During this window,
103+
`sp_invoke_external_rest_endpoint` returns `0x80072ee7`. The fix is time — let
104+
the TTL expire. Rebooting the RDS instance can also clear the engine's DNS cache,
105+
but only after the VPC resolver itself has propagated the correct (public)
106+
answer.

0 commit comments

Comments
 (0)