Private App Access To S3

1. Situation

Return to the application architecture from the first AWS study. The EC2 application instances run in private subnets.

In this design, private means the instances do not have public IP addresses and their subnets do not provide a direct path to an internet gateway, the VPC component that connects internet-bound routes to the internet. Each subnet uses a route table: a set of rules that tells resources in the subnet where to send network traffic. A private subnet can still have controlled routes to other networks and supported AWS services.

The application now needs to store and retrieve objects from Amazon S3:

• user-uploaded images
• generated reports
• invoices and exports
• application assets

At first, the requirement sounds contradictory:

keep the application private
but let it call a service outside its VPC

S3 is a regional AWS service, but your bucket is not placed inside one of your VPC subnets. That does not mean S3 is outside AWS. It means the application still needs a valid route from its VPC to the regional S3 service.

Tap to exploreOpen diagram

The application must satisfy two additional requirements:

1. Do not add a public IP address merely to reach S3.
2. Do not pay for a general internet egress path when the application only needs a supported AWS service.

The design question is:

how can a private application reach S3 without using the public internet path or a NAT gateway?

Tap to exploreOpen diagram

2. Naive Design

Naive option 1: make the application public

Someone gives each EC2 instance a public IP address and places the instances in public subnets. The application can now reach public AWS service endpoints, but the instances also have a direct internet-routing path that the design does not require. Security groups, the virtual firewall rules attached to the instances, can still block inbound traffic, but the network exposure is broader than necessary.

The application did not need to accept direct internet traffic. It only needed to initiate S3 API calls. Making the servers public solves a much larger problem than the one the architecture actually has.

Naive option 2: send S3 traffic through NAT

A more reasonable design keeps EC2 private and adds a NAT gateway. That can reach S3, but compare the general NAT path with the destination-specific gateway endpoint path:

General outbound path: works, but uses NAT

Tap to exploreOpen diagram

Without an S3-specific route, the request follows the subnet's general outbound path. This works, but S3 traffic passes through the NAT gateway and incurs its processing cost.

Destination-specific path: use the S3 endpoint

Tap to exploreOpen diagram

With the endpoint route, matching S3 traffic takes the shorter, destination-specific path. NAT can remain available for unrelated public destinations. The routing rules behind this choice are explained in the architecture section.

The NAT path can work. NAT gateways allow private workloads to initiate outbound connections to many destinations.

The problem is fit. NAT provides broad outbound connectivity and adds hourly and data-processing cost, while this application only needs S3. It is like building a general-purpose highway exit when a controlled service road already reaches the one destination you need.

An exam answer can be functional and still not be the best answer. Look for the option that satisfies security, performance, and cost requirements with the narrowest appropriate service.

Naive option 3: copy access keys onto the server

Hardcoded access keys answer the identity question, not the networking question. Credentials may prove who the caller is, but they do not create a route to S3.

They also introduce secret storage, rotation, leakage, and audit risk. An EC2 IAM role gives the instance an AWS identity and temporary credentials without placing long-lived keys in application files or environment variables.

3. What Breaks

Private S3 access becomes much easier when you stop treating it as one problem.

It is three separate questions:

Tap to exploreOpen diagram

Failure 1: the route is missing

The application has a correct IAM role, but its subnet has no path to S3. The SDK waits and eventually returns a connection or timeout error. Permission cannot repair a missing route.

Failure 2: the identity is missing or too broad

The network path works, but the application has no role permission for s3:GetObject. S3 returns AccessDenied.

Giving the role s3:* on * may remove the error, but it creates a much larger security problem. The application should receive only the actions and object paths it needs.

Failure 3: another policy rejects the request

The role allows the action, but another policy rejects it. That policy might be an endpoint policy, bucket policy, AWS Organizations policy, permissions boundary, or AWS Key Management Service (AWS KMS) key policy. You do not need to master every policy type yet; the important point is that several independent authorization layers may evaluate the same request.

An explicit deny wins over an allow. This means a request can pass one checkpoint and still fail at the next one.

This is why “the endpoint exists” and “the app can access the bucket” are not equivalent statements.

Imagine an employee travelling from a private office to a warehouse. The road sign sends the employee onto the approved warehouse road. At the road checkpoint, the employee's badge and permitted destination are checked. At the warehouse door, the warehouse applies its own entry rules.

The badge cannot create the road, and the road does not guarantee entry. A successful trip needs both the route and the required permissions.

4. AWS Architecture

Build the solution one responsibility at a time.

Start with the routing mental model. A route table is a set of rules associated with a subnet. When a resource in that subnet sends network traffic, the route table uses the destination IP address to choose where the traffic should go next.

Each route has two important parts:

• Destination: the IP address range or AWS-managed group of IP ranges that the rule applies to.
• Target: the next network component that should receive matching traffic, such as a NAT gateway or gateway endpoint.

Tap to exploreOpen diagram

This is the only question the small diagram answers: given a destination, which target should receive the traffic next?

The route table answers where should this traffic go? It does not grant permission to call S3; IAM and resource policies answer that separate authorization question.

Step 1: add an S3 gateway VPC endpoint

A gateway VPC endpoint gives a VPC a route to supported services without requiring an internet gateway or NAT device. For SAA-C03, the two gateway-endpoint services to remember are S3 and DynamoDB.

Create the S3 gateway endpoint in the same AWS Region as the VPC and the S3 buckets it must reach through that endpoint.

The endpoint is not an EC2 instance, NAT appliance, or elastic network interface—a virtual network card with a private IP—inside a subnet. AWS manages it as a route-table target. For this endpoint type, think route-table entry, not network interface with a security group.

Step 2: associate the private route tables

When creating the endpoint, select the route tables used by the private subnets that need S3 access. AWS adds a route whose destination is the AWS-managed S3 prefix list and whose target is the gateway endpoint.

Assume the private subnet also has a NAT gateway for unrelated internet access. The two relevant outbound routes then look like this:

An AWS-managed prefix list is a group of IP address ranges maintained by AWS. The S3 prefix list represents the current IP ranges used by S3 in that Region, so you do not have to track changes to those ranges yourself. The first route therefore says: “When the destination is S3, send the traffic to this S3 gateway endpoint.”

0.0.0.0/0 represents all IPv4 destinations. It is the broadest possible destination and acts as the anything else route. Here, its target is a NAT gateway, which gives instances with private IP addresses an outbound path to public destinations. A real route table also contains a route for traffic inside the VPC; this simplified table shows only the two routes relevant to the comparison.

Read the architecture from left to right:

• Private EC2: the application instance is in a private subnet and has no public IP address.
• Private route table: the route table associated with the private subnet chooses the next path based on the request's destination. “Private route table” is a useful description, not a separate AWS resource type.
• S3 gateway endpoint: a VPC component that provides the private route from the VPC to S3.
• Amazon S3: the regional AWS storage service containing the requested object.

Tap to exploreOpen diagram

What happens when the EC2 instance accesses S3?

First, isolate the network decision. The later request-flow section adds credentials and policy checks.

Tap to exploreOpen diagram

1. The EC2 instance sends an HTTPS request to the regional S3 service.
2. The route table checks which routes match the request's destination IP address.
3. The destination belongs to an IP range in the AWS-managed S3 prefix list.
4. The route table selects the S3 route and sends the traffic to the gateway endpoint.
5. The endpoint carries the traffic to S3 without using the NAT gateway or traversing the public internet.

The S3 prefix-list route wins because route tables use longest-prefix matching, also called route specificity. In simple terms, the most specific matching destination wins. 0.0.0.0/0 matches every IPv4 address, including S3 addresses, but it is only a general fallback. The S3 prefix list contains narrower IP ranges that specifically match S3, so the route to the gateway endpoint is selected. The order of the rows in the table does not determine the result.

No public IP is required on the EC2 instance, and S3 traffic does not require the NAT gateway.

What happens when the EC2 instance accesses Google?

Google's destination IP address is not part of the AWS-managed S3 prefix list, so the S3 route does not match. The remaining matching route is the general 0.0.0.0/0 route, whose target is the NAT gateway.

Tap to exploreOpen diagram

The gateway endpoint changes the path only for matching S3 traffic. It does not provide general internet access, so the subnet still needs NAT or another egress design when the application must reach public websites or third-party APIs. In this example, the NAT gateway sits in a public subnet and uses the VPC's internet gateway for the final internet path.

Step 3: give the application an IAM role

Attach an IAM role to the EC2 instance through an instance profile. The role defines the workload's AWS identity; the instance profile is the mechanism that attaches that role to EC2.

The AWS SDK can then obtain temporary credentials automatically and use them to sign S3 API requests.

The role must name both the allowed API actions and the AWS resources those actions may affect. IAM policies identify resources with an Amazon Resource Name (ARN), a unique AWS resource identifier. In arn:aws:s3:::private-app-data, arn marks the value as an ARN, aws identifies the standard AWS partition, s3 identifies the service, and private-app-data is the bucket name. S3 bucket ARNs omit the Region and account fields, which is why three colons appear before the bucket name.

S3 uses different ARN shapes for a bucket and for the objects stored inside it:

An S3 object key is the object's complete name inside its bucket. For example, in s3://private-app-data/app/receipts/order-123.pdf, the bucket is private-app-data and the key is app/receipts/order-123.pdf. The slashes make the key look like a file path, although S3 stores objects in a flat namespace and treats app/receipts/ as a prefix.

With that vocabulary, the role's permissions can be read as:

allow s3:ListBucket on arn:aws:s3:::private-app-data
allow s3:GetObject and s3:PutObject on arn:aws:s3:::private-app-data/app/*

The two resource types are intentional:

• s3:GetObject and s3:PutObject operate on individual objects, so their resource is an object ARN. If the application already knows the key app/receipts/order-123.pdf, it can request that exact object without first listing the bucket.
• s3:ListBucket operates on the bucket itself. The application needs it only when it asks S3 which object keys exist—for example, “return the objects whose keys begin with app/receipts/.” That request uses an operation such as ListObjectsV2, which returns object names rather than object contents.

If the application should list only its own area, a real IAM policy can combine s3:ListBucket on the bucket ARN with an s3:prefix condition limited to app/*. This preserves the ability to discover the application's objects without allowing it to enumerate unrelated keys elsewhere in the bucket.

Tap to exploreOpen diagram

Step 4: narrow the path and destination

The road now exists, and the application has an identity. The remaining job is to restrict what that identity may do along this path and what the bucket will accept.

An endpoint policy controls which principals (AWS identities), actions, and S3 resources may be accessed through that endpoint. The default endpoint policy allows broad S3 access through the endpoint, so security-sensitive architectures should consider a narrower policy.

A bucket policy controls access from the bucket's point of view. It can restrict requests to an expected role, account, VPC, or endpoint. The built-in aws:SourceVpce condition key can require requests to arrive through a particular VPC endpoint.

These policies are not duplicates. The endpoint policy guards use of the path; the bucket policy guards the resource at the destination. A request must satisfy every applicable policy layer.

Tap to exploreOpen diagram

The complete architecture is intentionally boring: no public IP, no proxy server, and no custom credential distribution. Each AWS control answers one clear question.

5. Request Or Data Flow

The application still uses the normal AWS SDK and regional S3 service name. It does not send requests to a custom gateway-endpoint URL. DNS resolves that service name to S3 service IP addresses, and the route table evaluates the resulting destination IP. The route table does not read the bucket name or URL. The destination in the code stays the same; the VPC route table changes how the network traffic gets there.

Tap to exploreOpen diagram

Walk through the flow carefully:

1. The application asks the SDK to read s3://private-app-data/app/report.pdf.
2. The SDK obtains temporary credentials for the EC2 instance role.
3. The SDK signs an HTTPS request to the regional S3 API.
4. DNS resolves the S3 service normally. The destination matches the AWS-managed S3 prefix list in the private route table.
5. The route sends traffic to the gateway endpoint instead of NAT or an internet gateway.
6. The endpoint policy must permit the request to pass. S3 then evaluates the caller and applicable policies. An applicable allow must exist, and no applicable explicit deny may block it.
7. If the object uses an AWS KMS key, the caller also needs the required KMS permissions.
8. S3 returns the object over the endpoint path.

6. Security Controls

Secure each layer according to the question it answers.

Use temporary credentials

Use an IAM role rather than long-lived access keys. The SDK refreshes temporary role credentials automatically, which removes manual key distribution and rotation from the application.

Scope permissions to the required actions and resources. Separate bucket-level actions such as s3:ListBucket from object-level actions such as s3:GetObject and s3:PutObject.

Keep the bucket private

Enable S3 Block Public Access, the S3 setting that prevents public-access configurations from exposing the bucket, unless the bucket has a deliberate public use case. Private application access does not require a public bucket.

A bucket policy can restrict sensitive operations to a specific endpoint with aws:SourceVpce.

Test this carefully. A deny that requires the endpoint can also block ordinary AWS Management Console access because console requests do not use that endpoint. In the warehouse analogy, locking every door except the private delivery entrance may also lock out an administrator who normally enters through the front.

For requests that traverse an S3 VPC endpoint, do not use aws:SourceIp as if S3 sees the old public source address. Use an appropriate condition such as aws:SourceVpce, aws:SourceVpc, or aws:VpcSourceIp based on the intended boundary.

Keep network guards aligned

The application's security group must permit outbound HTTPS to S3. The subnet-level network access control lists (network ACLs) must permit the corresponding traffic. Security group rules can reference the S3 prefix list where the design needs a narrower egress rule.

The security group belongs to the workload, not the gateway endpoint. A gateway endpoint does not create an endpoint network interface with its own security group.

7. Resilience Controls

You do not run or size an S3 gateway endpoint yourself. AWS manages it and scales its capacity horizontally, which means AWS can add capacity behind the service as traffic grows. The endpoint is not an appliance sitting in one Availability Zone, so you do not create an endpoint instance for every subnet or AZ.

Which subnets receive the S3 endpoint route? Each subnet uses its associated route table to decide where traffic goes. The S3 route appears only in the route tables selected for the gateway endpoint, so every application subnet that needs this private path must use one of those route tables.

For example, suppose the application runs in private subnets across three Availability Zones (AZs), isolated locations within an AWS Region. The diagram shows the misconfiguration to avoid: route tables A and B were selected for the endpoint, but route table C was missed.

Tap to exploreOpen diagram

Instances in AZ A and AZ B receive the S3 route and can use the gateway endpoint. Instances in AZ C do not receive that route. Their S3 traffic therefore uses another matching route—such as 0.0.0.0/0 → NAT gateway—if one exists. If no other route can reach S3, the request fails. This is why every application subnet that needs the endpoint must use a route table selected for it.

The endpoint gives those subnets a network path to S3, but it does not solve every reliability problem:

• It does not replace S3 versioning or replication.
• It does not recover an accidentally deleted object.
• It does not give the application access to unrelated internet destinations.
• It does not fix an incorrect IAM or KMS policy.

If the private instances also need public package repositories, operating-system updates, or third-party APIs, they still need a NAT gateway or another outbound-internet path. This outbound path is often called egress.

In short, the gateway endpoint changes the path for S3 traffic only. It does not change how the application reaches any other destination.

8. Performance Controls

Without the endpoint, private instances may send S3 traffic through a NAT gateway or through proxy servers maintained by your team. The gateway endpoint removes those unnecessary middle steps. The application still calls the normal S3 API, but the traffic follows the AWS-managed endpoint path.

The endpoint changes the network path; it does not change how S3 behaves. S3 is still object storage accessed through API calls, not a local disk or mounted file system. Performance still depends on factors such as object size, how many requests run at once, retry behavior, and whether the application and bucket are in the same Region.

Then choose S3 techniques that match the workload:

• Use multipart upload to split a large object into smaller parts that can be uploaded independently.
• Use parallel or byte-range reads when the application can fetch different parts of a large object at the same time.
• Retry temporary failures with backoff, meaning the application waits progressively longer between attempts instead of retrying continuously.
• Use CloudFront when public users in many locations need fast downloads; that is a content-delivery problem, not a VPC-routing problem.

When a request fails, start with one question: Did the request fail before reaching S3, or did AWS receive it and deny it? Check the path in order instead of changing several policies at once:

Tap to exploreOpen diagram

The error often tells you which half to investigate. A timeout usually points toward DNS or networking. AccessDenied usually means the request reached AWS, but a permission layer rejected it. That denial can come from IAM, the endpoint policy, the bucket policy, an AWS Organizations service control policy (SCP)—an organization-level permission guardrail—or KMS.

Using the warehouse analogy: a timeout means the employee may never have reached the warehouse. AccessDenied means the employee reached a checkpoint, showed a badge, and was not allowed through.

9. Cost Controls

AWS does not add an hourly or data-processing charge for the S3 gateway endpoint itself. You still pay the normal S3 charges that apply to the workload, such as storage, API requests, retrieval, and any data-transfer charges that apply.

A useful way to choose the network path is:

Tap to exploreOpen diagram

A NAT gateway provides general outbound connectivity and has hourly and data-processing costs. Sending a large amount of S3 traffic through NAT means paying a general-purpose internet-egress service for traffic that could use the S3-specific gateway endpoint instead.

Do not remove NAT just because S3 no longer needs it. First list every external destination the application uses. The same server may still need package repositories, third-party APIs, or AWS services that require interface endpoints.

An S3 interface endpoint works differently: it creates private IP addresses in your subnets and has endpoint charges. It is useful when a connected network must reach S3 privately—for example, an on-premises data center connected through VPN or Direct Connect—or when DNS, cross-Region, or network requirements do not fit a gateway endpoint.

For a typical exam question where workloads inside one VPC need private access to S3 in the same Region, start by considering an S3 gateway endpoint. It is usually the simpler and lower-cost fit.

10. Exam Variants

For endpoint questions, ask three things before looking at the answers:

1. Where does the request start? Inside a VPC, on premises, or somewhere else?
2. Where is it going? S3, DynamoDB, another AWS service, or the public internet?
3. What kind of path is required? Same-Region private access, hybrid access between on-premises and AWS, cross-Region access, or general internet access?

Follow each clue to one architectural decision

Suppose an exam question says:

EC2 instances in private subnets must upload reports to S3. The solution must avoid NAT gateway cost and must not store AWS credentials on the instances.

The question contains two separate requirements, so the answer needs two separate controls:

avoid NAT for S3 -> S3 gateway endpoint
avoid stored credentials -> EC2 IAM role

The gateway endpoint solves the network-path requirement. The IAM role solves the credential and permission requirement. Choosing only one leaves half of the problem unsolved.

11. Common Traps

Final Mental Model: One-Minute Review

Final Architecture Map

Tap to exploreOpen diagram

Start with the solid arrows: the EC2 instance sends S3 traffic to its route table, the route table selects the gateway endpoint, and the endpoint carries the traffic to S3. The dotted arrows are not extra network hops. They show the IAM and endpoint policies that must allow the request.

The final rule is simple: the endpoint provides the road, but it does not provide permission. A request succeeds only when the network path works and every applicable permission layer allows the S3 action.

12. Related Topics

Read VPC Networking Model if route tables, private subnets, or internet gateways still feel unclear.

Read Amazon S3 for a deeper explanation of object storage, bucket security, encryption, versioning, and lifecycle behavior.

Read NAT Gateway vs VPC Endpoints when deciding between general outbound internet access and a private path to an AWS service.

Read Gateway vs Interface VPC Endpoints for a deeper comparison of route-table-based gateway endpoints and private-IP-based interface endpoints.

Read Identity Policies vs Resource Policies if the difference between an IAM role policy and an S3 bucket policy is still unclear.

Official AWS references:

• Gateway endpoints
• Gateway endpoints for Amazon S3
• AWS PrivateLink for Amazon S3
• Use an IAM role for applications on EC2
• Identify AWS resources with ARNs
• Naming Amazon S3 objects and keys