Private App Access To S3

1. Situation

Return to the application architecture from the first AWS study. The EC2 application instances run in private subnets.

In this design, private means the instances do not have public IP addresses and their subnet route tables do not provide a direct route to an internet gateway. It does not mean the instances must be cut off from every AWS service.

The application now needs to store and retrieve objects from Amazon S3:

• user-uploaded images
• generated reports
• invoices and exports
• application assets

At first, the requirement sounds contradictory:

keep the application private
but let it call a service outside its VPC

S3 is a regional AWS service, but your bucket is not placed inside one of your VPC subnets. That does not mean S3 is outside AWS. It means the application still needs a valid route from its VPC to the regional S3 service.

flowchart LR
  subgraph VPC["Application VPC"]
    App["EC2 application<br/>private subnet"]
  end

  App -. How does this request travel? .-> S3["Amazon S3"]

The application must satisfy two additional requirements:

1. Do not add a public IP address merely to reach S3.
2. Do not pay for a general internet egress path when the application only needs a supported AWS service.

The design question is:

how can a private application reach S3 without using the public internet path or a NAT gateway?

2. Naive Design

Naive option 1: make the application public

Someone gives each EC2 instance a public IP address and places the instances in public subnets. The application can now reach public AWS service endpoints, but the instances also have a direct internet-routing path that the design does not require. Security groups can still block inbound traffic, but the network exposure is broader than necessary.

The application did not need to accept direct internet traffic. It only needed to initiate S3 API calls. Making the servers public solves a much larger problem than the one the architecture actually has.

Naive option 2: send S3 traffic through NAT

A more reasonable design keeps EC2 private and adds a NAT gateway. That can reach S3, but compare the general NAT path with the destination-specific gateway endpoint path:

Read the diagram from left to right. In the upper path, the default route carries S3 traffic through NAT and the internet gateway. In the lower path, the route table recognizes the regional S3 prefix list and selects the gateway endpoint instead. Other public destinations can still follow the NAT route.

The NAT path can work. NAT gateways allow private workloads to initiate outbound connections to many destinations.

The problem is fit. NAT provides broad outbound connectivity and adds hourly and data-processing cost, while this application only needs S3. It is like building a general-purpose highway exit when a controlled service road already reaches the one destination you need.

An exam answer can be functional and still not be the best answer. Look for the option that satisfies security, performance, and cost requirements with the narrowest appropriate service.

Naive option 3: copy access keys onto the server

Hardcoded access keys answer the identity question, not the networking question. Credentials may prove who the caller is, but they do not create a route to S3.

They also introduce secret storage, rotation, leakage, and audit risk. An EC2 IAM role provides temporary credentials without placing long-lived keys in application files or environment variables.

3. What Breaks

Private S3 access becomes much easier when you stop treating it as one problem.

It is three separate questions:

1. Network path: how does the request reach S3?
2. Identity: who is making the S3 API call?
3. Authorization: is that identity allowed to perform this action on this bucket?

Failure 1: the route is missing

The application has a correct IAM role, but its subnet has no path to S3. The SDK waits and eventually returns a connection or timeout error. Permission cannot repair a missing route.

Failure 2: the identity is missing or too broad

The network path works, but the application has no role permission for s3:GetObject. S3 returns AccessDenied.

Giving the role s3:* on * may remove the error, but it creates a much larger security problem. The application should receive only the actions and object paths it needs.

Failure 3: another policy rejects the request

The role allows the action, but another policy rejects it. That policy might be an endpoint policy, bucket policy, AWS Organizations policy, permissions boundary, or KMS key policy.

An explicit deny wins over an allow. This means a request can pass one checkpoint and still fail at the next one.

This is why “the endpoint exists” and “the app can access the bucket” are not equivalent statements.

Imagine an employee travelling from a private office to a warehouse. The road sign sends the employee onto the approved warehouse road. At the road checkpoint, the employee's badge and permitted destination are checked. At the warehouse door, the warehouse applies its own entry rules.

The badge cannot create the road, and the road does not guarantee entry. A successful trip needs both the route and the required permissions.

4. AWS Architecture

Build the solution one responsibility at a time.

Step 1: add an S3 gateway VPC endpoint

A gateway VPC endpoint gives a VPC a route to supported services without requiring an internet gateway or NAT device. For SAA-C03, the two gateway-endpoint services to remember are S3 and DynamoDB.

Create the S3 gateway endpoint in the same AWS Region as the VPC and the S3 buckets it must reach through that endpoint.

The endpoint is not an EC2 instance, NAT appliance, or elastic network interface inside a subnet. AWS manages it as a route-table target. For this endpoint type, think route-table entry, not network interface with a security group.

Step 2: associate the private route tables

When creating the endpoint, select the route tables used by the private subnets that need S3 access. AWS adds an endpoint route similar to this:

The prefix list represents the current regional S3 address ranges. Instead of asking you to maintain every S3 IP range yourself, AWS maintains this destination list for you.

The endpoint route is more specific than a default 0.0.0.0/0 route. When the destination matches the regional S3 prefix list, the route table selects the gateway endpoint.

flowchart LR
  subgraph VPC["Application VPC"]
    EC2["EC2 application<br/>private subnet"] --> Route["Private route table<br/>S3 prefix list → endpoint"]
    Route --> Endpoint["S3 gateway endpoint"]
  end

  Endpoint --> S3["Amazon S3<br/>same Region"]

No public IP is required on the instance. No NAT gateway is required for traffic that matches the S3 endpoint route.

Step 3: give the application an IAM role

Attach an IAM role to the EC2 instance through an instance profile. The role defines the workload's AWS identity; the instance profile is the mechanism that attaches that role to EC2.

The AWS SDK can then obtain temporary credentials automatically and use them to sign S3 API requests.

The role should name the required actions and resources. For example:

allow s3:ListBucket on arn:aws:s3:::private-app-data
allow s3:GetObject and s3:PutObject on arn:aws:s3:::private-app-data/app/*

Notice that listing a bucket and reading an object use different actions and resource ARN shapes. Grant ListBucket only when the application actually needs to list keys.

Step 4: narrow the path and destination

The road now exists, and the application has an identity. The remaining job is to restrict what that identity may do along this path and what the bucket will accept.

An endpoint policy controls which principals, actions, and S3 resources may be accessed through that endpoint. The default endpoint policy allows broad S3 access through the endpoint, so security-sensitive architectures should consider a narrower policy.

A bucket policy controls access from the bucket's point of view. It can restrict requests to an expected role, account, VPC, or endpoint. A condition using aws:SourceVpce can require requests to arrive through a particular VPC endpoint.

These policies are not duplicates. The endpoint policy guards use of the path; the bucket policy guards the resource at the destination. A request must satisfy every applicable policy layer.

flowchart LR
  Role["IAM role<br/>Who may call?"] -. temporary credentials .-> EC2["Private EC2"]

  subgraph VPC["Application VPC"]
    EC2 --> Route["Route table<br/>Where does S3 traffic go?"]
    Route --> Endpoint["Gateway endpoint<br/>Endpoint policy"]
  end

  Endpoint --> S3["S3 bucket<br/>Bucket policy"]

The complete architecture is intentionally boring: no public IP, no proxy server, and no custom credential distribution. Each AWS control answers one clear question.

5. Request Or Data Flow

The application still uses the normal AWS SDK and regional S3 service name. It does not send requests to a custom gateway-endpoint URL. The destination in the code stays the same; the VPC route table changes how the network traffic gets there.

flowchart TD
  App["1. Application calls GetObject"] --> Credentials["2. SDK obtains temporary role credentials"]
  Credentials --> Signed["3. SDK signs the S3 API request"]
  Signed --> Route["4. Route table matches the S3 prefix list"]
  Route --> Endpoint["5. Traffic follows the gateway endpoint"]
  Endpoint --> Auth["6. Endpoint and S3 authorization are evaluated"]
  Auth -->|"Allowed"| Object["7. S3 returns the object"]
  Auth -->|"Denied"| Denied["AccessDenied"]

Walk through the flow carefully:

1. The application asks the SDK to read s3://private-app-data/app/report.pdf.
2. The SDK obtains temporary credentials for the EC2 instance role.
3. The SDK signs an HTTPS request to the regional S3 API.
4. DNS resolves the S3 service normally. The destination matches the AWS-managed S3 prefix list in the private route table.
5. The route sends traffic to the gateway endpoint instead of NAT or an internet gateway.
6. The endpoint policy must permit the request to pass. S3 then evaluates the caller and applicable policies. An applicable allow must exist, and no applicable explicit deny may block it.
7. If the object uses an AWS KMS key, the caller also needs the required KMS permissions.
8. S3 returns the object over the endpoint path.

6. Security Controls

Secure each layer according to the question it answers.

Use temporary credentials

Use an IAM role rather than long-lived access keys. The SDK refreshes temporary role credentials automatically, which removes manual key distribution and rotation from the application.

Scope permissions to the required actions and resources. Separate bucket-level actions such as s3:ListBucket from object-level actions such as s3:GetObject and s3:PutObject.

Keep the bucket private

Enable S3 Block Public Access unless the bucket has a deliberate public use case. Private application access does not require a public bucket.

A bucket policy can restrict sensitive operations to a specific endpoint with aws:SourceVpce.

Test this carefully. A deny that requires the endpoint can also block ordinary AWS Management Console access because console requests do not use that endpoint. In the warehouse analogy, locking every door except the private delivery entrance may also lock out an administrator who normally enters through the front.

For requests that traverse an S3 VPC endpoint, do not use aws:SourceIp as if S3 sees the old public source address. Use an appropriate condition such as aws:SourceVpce, aws:SourceVpc, or aws:VpcSourceIp based on the intended boundary.

Keep network guards aligned

The application's security group must permit outbound HTTPS to S3. Network ACLs must permit the corresponding traffic. Security group rules can reference the S3 prefix list where the design needs a narrower egress rule.

The security group belongs to the workload, not the gateway endpoint. A gateway endpoint does not create an endpoint network interface with its own security group.

7. Resilience Controls

Gateway endpoints are AWS-managed, horizontally scaled VPC components. You do not deploy endpoint instances, choose an instance size, or place one endpoint appliance in each Availability Zone.

The route-table association is the important availability detail. Each application subnet uses a route table, and that route table must be associated with the endpoint.

For example, suppose the application runs in private subnets across three Availability Zones. If only two of their route tables are associated, instances in the third subnet do not receive the same S3 endpoint route.

flowchart LR
  Endpoint["S3 gateway endpoint"] --> RTA["Private route table A"]
  Endpoint --> RTB["Private route table B"]
  Endpoint --> RTC["Private route table C"]
  RTA --> AZA["App subnet<br/>AZ A"]
  RTB --> AZB["App subnet<br/>AZ B"]
  RTC --> AZC["App subnet<br/>AZ C"]

The endpoint solves connectivity to S3. It does not solve every resilience concern:

• It does not replace S3 versioning or replication.
• It does not recover an accidentally deleted object.
• It does not give the application access to unrelated internet destinations.
• It does not fix an incorrect IAM or KMS policy.

Keep NAT or another appropriate egress design when private workloads also need public package repositories, operating-system updates, or third-party APIs.

The gateway endpoint removes matching S3 traffic from the NAT path. It does not make the application's other outbound dependencies disappear.

8. Performance Controls

A gateway endpoint removes an unnecessary NAT hop and avoids sending S3 traffic through a self-managed proxy tier. The application can call S3 directly over AWS-managed networking.

The endpoint does not change S3 into a mounted low-latency disk. S3 remains object storage accessed through APIs. Application performance still depends on object size, request pattern, concurrency, retries, and regional placement.

Use S3 mechanics that match the workload:

• multipart upload for large objects
• parallel or byte-range reads when appropriate
• retry behavior with backoff for transient failures
• CloudFront for public, globally distributed downloads

When a request fails, first ask whether S3 was unreachable or whether S3 received the request and rejected it. Then troubleshoot in layers instead of randomly changing policies:

1. DNS: does the application resolve the intended regional S3 service?
2. Route: is the correct private route table associated with the endpoint?
3. Endpoint policy: may this request pass through the endpoint?
4. IAM and bucket policy: is the S3 action allowed with no explicit deny?
5. KMS: may the role use the encryption key?

Timeouts often suggest a network or DNS problem. AccessDenied usually means the request reached an AWS authorization checkpoint but was rejected by IAM, the endpoint policy, the bucket policy, organization controls, or KMS.

Using the earlier analogy, a timeout means the employee may not have reached the warehouse. AccessDenied means the employee reached a checkpoint but could not pass it.

9. Cost Controls

There is no additional hourly or data-processing charge for an S3 gateway endpoint. S3 storage, request, retrieval, and applicable data-transfer charges still remain.

This creates a common optimization pattern:

S3 and DynamoDB traffic -> gateway endpoints
supported AWS service APIs -> interface endpoints when required
general public internet traffic -> NAT or another egress design

NAT gateways have their own hourly and data-processing costs. If high-volume S3 traffic follows the NAT path, the application pays for a broad egress service it does not need for those requests.

Do not remove NAT merely to make the diagram cheaper. First inventory every outbound dependency. A server that reaches S3 through an endpoint may still need package repositories, external APIs, or AWS services that require other endpoint types.

Interface endpoints for S3 use private IP addresses in your subnets and are billed. They become relevant when requirements such as on-premises access, cross-Region private connectivity, or specific DNS and network behavior rule out a gateway endpoint.

For an exam question about ordinary same-Region S3 access from workloads inside one VPC, the gateway endpoint is usually the simpler and lower-cost starting point.

10. Exam Variants

Read endpoint questions by identifying the source, destination, and type of connectivity.

Follow each clue to one architectural decision

Suppose an exam question says:

EC2 instances in private subnets must upload reports to S3. The solution must avoid NAT gateway cost and must not store AWS credentials on the instances.

There are two independent clues:

avoid NAT for S3 -> S3 gateway endpoint
avoid stored credentials -> EC2 IAM role

Choosing only the endpoint leaves the identity requirement unanswered. Choosing only the role leaves the routing requirement unanswered.

11. Common Traps

Final Mental Model: One-Minute Review

Final Architecture Map

Read this diagram from left to right for the network path, then use the bottom row to remember that the endpoint does not grant access by itself.

flowchart LR
  App["Private application"] -->|"S3 request"| Endpoint["S3 gateway endpoint"]
  Endpoint --> S3["Private S3 bucket"]
  Route["Route table<br/>S3 prefix to endpoint"] -.->|"Selects this path"| Endpoint
  Role["IAM role<br/>caller permission"] -.->|"Authorizes the application"| App
  EndpointPolicy["Endpoint policy<br/>path restriction"] -.->|"Filters endpoint use"| Endpoint
  BucketPolicy["Bucket policy<br/>resource restriction"] -.->|"Filters bucket access"| S3

The endpoint creates the route; it does not grant S3 access by itself. A successful request needs both a valid network path and authorization through the relevant identity, endpoint, and bucket policies.

12. Related Topics

Use VPC Networking Model when route tables, private subnets, and internet gateways are still unfamiliar.

Use Amazon S3 for object storage, bucket security, encryption, versioning, and lifecycle behavior.

Use NAT Gateway vs VPC Endpoints when deciding between general outbound internet access and private AWS service access.

Use Gateway vs Interface VPC Endpoints for deeper comparison of route-table endpoints and private-IP endpoints.

Use Identity Policies vs Resource Policies when IAM role and bucket policy evaluation is unclear.

Official AWS references:

• Gateway endpoints
• Gateway endpoints for Amazon S3
• AWS PrivateLink for Amazon S3
• Use an IAM role for applications on EC2