1. Situation
In this scenario, a partner is another company—such as a shipping carrier, warehouse operator, or supplier—that exchanges data with the logistics company. Every night, Partner A's system exports a file such as shipments-2026-09-04.csv. One file may contain many shipment records, including order IDs, tracking numbers, delivery statuses, timestamps, and customer addresses.
This file-based integration is common when two companies have already agreed on a file format and delivery schedule, especially when one side uses older software that exchanges whole files on a schedule rather than handling individual API requests. Replacing it with an API would require coordinated changes on both sides, so the logistics company must preserve the partner's existing Secure File Transfer Protocol (SFTP) workflow.
An SFTP client is software running on the partner's side. It connects to the receiving company's SFTP server over an encrypted Secure Shell (SSH) connection, authenticates with a password or cryptographic SSH key, and uploads the file to a remote directory. You can think of SFTP as working with a secure remote folder. A successful upload confirms that the server received the bytes; SFTP does not understand whether the shipment records inside those bytes are correct.
In this AWS design, AWS Transfer Family plays the role of the managed SFTP server, and Amazon S3, AWS object storage, stores the uploaded file behind it. The partner keeps using its existing SFTP client and does not need AWS credentials or new S3 API code.
After arrival, the file is not sent directly into reports or operational systems. A validation program—the processor—checks details such as the expected partner and date, filename, schema, row count, integrity, and security policy. Validated files become trusted input for analytics, such as shipment-performance reports, and reconciliation—comparing the partner's shipment records with the company's internal orders to find missing, duplicated, or inconsistent data. Files that fail the trust checks go to quarantine for investigation.
partner system creates a shipment file
-> SFTP client uploads it
-> AWS Transfer Family accepts the SFTP session
-> S3 stores it in an untrusted incoming area
-> Amazon Simple Queue Service (SQS) holds a small work item until a processor is ready
-> a processor validates the file
-> trusted data goes to analytics and reconciliation; rejected data goes to quarantine
The business requirements are:
- partners keep their existing SFTP clients;
- one partner cannot see another partner's files;
- files are encrypted and never public;
- downstream systems do not trust a file merely because it arrived;
- processing survives bursts and temporary failures;
- repeated events do not process the same object twice; and
- operators can tell which expected files arrived, failed, or remain delayed.
The central lesson is:
received bytes != trusted business data
This architecture is a trust transition, not simply a file-copy diagram.
2. Naive Design
Naive option 1: operate an SFTP server on an EC2 virtual machine
This can work, but the team owns host patching, SFTP configuration, high availability, user lifecycle, host keys, logging, scaling, and storage integration.
The partner requires SFTP—not necessarily a server the company operates itself.
Naive option 2: give partners IAM users and broad S3 access
This forces partners to change tooling and creates long-lived AWS credentials. An AWS Identity and Access Management (IAM) user is an AWS identity with permissions and potentially long-lived access keys. A broad IAM policy can allow one partner to list, read, overwrite, or delete another partner's objects.
Naive option 3: upload directly into the trusted analytics location
Partner A -> trusted/partner-a/shipments-2026-09-04.csv -> analytics starts
The SFTP transfer completed, but the file may have the wrong name, wrong partner, wrong schema, unexpected row count, corrupted business contents, or prohibited sensitive data. Transfer success is not validation success.
Naive option 4: invoke processing directly without a failure plan
An S3 event invokes one Lambda function for every arrival. Traffic bursts create high concurrency, large files exceed the processing model, duplicate notifications repeat side effects, and failed files have no durable work queue or replay process.
3. What Breaks
These are not five failures in the finished architecture. Each story starts with a plausible but incomplete version of the same pipeline: a partner uploads a file, Transfer Family stores it in S3, and software processes it for downstream use. In each version, one control is missing or misconfigured. The failure shows why the completed design in the next section adds that control.
Failure 1: the SFTP folder looks private, but the storage permissions are broad
Starting design: Transfer Family authenticates Partner A and presents incoming/partner-a/ as the partner's SFTP home directory. This logical home directory is a folder-like view created by configuration. However, the IAM role used for Partner A still permits access to a wider path such as incoming/*, which includes every partner's files.
What breaks: The screen looks isolated, but S3 does not have an independent policy rule that rejects access outside Partner A's area. A mapping mistake or overly broad request could therefore expose Partner B's objects. The fix uses both layers: directory mapping controls what the SFTP client displays, while the IAM role and S3 bucket policy allow only Partner A's S3 prefix—the path-like beginning of an object name, such as incoming/partner-a/.
Failure 2: S3 announces the same upload more than once
Starting design: Partner A uploads shipments-2026-09-04.csv. S3 stores the file and sends an object-created notification—a small message saying which S3 object arrived—to the processor. The processor copies the validated records into the trusted area and records the job as complete.
What breaks: S3 may deliver that notification again, or notifications for several uploads may arrive in a different order. This is its at-least-once, unordered delivery model. If the processor treats every message as new work, it can publish the same shipment records twice. The fix is idempotent processing: record a stable identity such as bucket, object key, object version, and processor version, then make repeated notifications for that identity produce only one completed business result. Adding SQS preserves and retries work, but the processor still owns this duplicate protection.
Failure 3: the upload succeeds, but the processor cannot decrypt the file
Starting design: The private S3 bucket encrypts incoming objects with a customer-managed AWS Key Management Service (KMS) key, whose permissions the company controls. Transfer Family writes the object using one IAM role. Later, a different processor role reads the object to validate it.
What breaks: Transfer Family has permission to write and encrypt, so the partner sees a successful upload. The processor has S3 read permission but lacks permission to use the KMS key for decryption, so processing later fails with an access-denied error. One successful actor does not automatically authorize the next actor. The fix gives every required writer, reader, scanner, and recovery process the appropriate S3 permissions and KMS permissions through both IAM and the key policy or grant.
Failure 4: a large file exceeds the Lambda processing design
Starting design: Every new S3 object invokes a Lambda function, and that function downloads, decompresses, scans, and validates the entire file. This works for the small CSV files used during testing.
What breaks: A partner later uploads a multi-gigabyte archive or a file whose validation needs more execution time, memory, or temporary storage than the Lambda design provides. The function repeatedly fails even though the file itself may be valid. A burst of files can also create more concurrent processing than a scanner or database can handle. The fix places SQS between arrival and processing, then chooses compute from the actual workload: Lambda for short, bounded work, or services such as ECS on Fargate, AWS Batch, or Glue for heavier processing.
Failure 5: sensitive data reaches analytics before Macie reports it
Starting design: The company places every successful SFTP upload directly in the trusted analytics area and starts downstream jobs immediately. It assumes Amazon Macie will inspect the file before anything unsafe happens.
What breaks: Macie later discovers personally identifiable information that the file was not supposed to contain, but analytics users may already have read it. Macie performs sensitive-data discovery; it is not an inline approval gate or antivirus engine. The fix lands every upload in an untrusted incoming area first, runs the required schema, business, malware, and classification checks, and promotes the file into the trusted area only after those checks pass. Rejected files go to quarantine instead.
4. AWS Architecture
Build the ingest path one trust boundary at a time.
Step 1: preserve SFTP with a managed Transfer Family endpoint
Partner A already has software that uploads its nightly CSV through SFTP. The logistics company wants that software to keep working, wants the file stored in S3, and does not want to operate SFTP servers. AWS Transfer Family bridges those requirements: Partner A uses SFTP as before, Transfer Family accepts the connection, and S3 stores incoming/partner-a/shipments-2026-09-04.csv.
The simple flow is Partner A → SFTP → Transfer Family → private S3. Direct S3 access would require Partner A to adopt AWS-specific APIs and credentials. A self-managed SFTP server on Amazon EC2 would preserve SFTP, but the company would own patching, availability, keys, scaling, and logs. Transfer Family preserves the familiar protocol while AWS operates the SFTP endpoint.
The endpoint can be publicly accessible when partners connect over the internet. A VPC-hosted endpoint places it in an Amazon Virtual Private Cloud (VPC)—the company's isolated AWS network—and can be internet-facing with selected network controls or internal for VPC and privately connected networks. The choice changes how partners reach the server; it does not change the basic Transfer Family-to-S3 flow.
Transfer Family also needs to know who is connecting. An identity provider is the system that verifies Partner A's password or SSH key. Transfer Family can store users itself or connect to supported external identity sources. After authentication, each user is mapped to:
- an IAM role that controls which S3 operations and object keys the user may access; and
- a home or logical directory that controls the folder-like view shown in the SFTP client.
For example, Partner A may log in and see /shipments-2026-09-04.csv, while Transfer Family maps that view to s3://partner-ingest/incoming/partner-a/shipments-2026-09-04.csv. The mapping controls what Partner A sees. The IAM role and S3 bucket policy control what Partner A can actually list, read, or write. Hiding Partner B's location is helpful navigation, but it is not authorization.
Step 2: isolate partners in an encrypted incoming zone
Map each partner to a dedicated incoming prefix:
s3://partner-ingest/incoming/partner-a/shipments-2026-09-04.csv
s3://partner-ingest/incoming/partner-b/shipments-2026-09-04.csv
Keep S3 Block Public Access enabled so public bucket or object permissions cannot accidentally expose the files. Use least-privilege identity and bucket policies so a partner can perform only the required operations in its mapped prefix.
S3 stores objects by object key rather than in a traditional folder tree. For this file, the key is incoming/partner-a/shipments-2026-09-04.csv, and the prefix is its path-like beginning: incoming/partner-a/. Consoles and SFTP clients make that prefix look like a folder, but the folder appearance provides no security. Isolation comes from policies that allow Partner A's role to access only keys beginning with its prefix. Use separate buckets or accounts when legal or containment requirements need a stronger boundary than prefix policies.
S3 encrypts stored objects by default. Choose server-side encryption with AWS KMS keys (SSE-KMS) when the business needs customer-controlled key permissions, separation of duties, detailed key-use auditing, or specific cross-account access. In that design, S3 encrypts Partner A's CSV using a KMS key, and every later actor that must read the file also needs permission to use that key.
Step 3: turn an object arrival into buffered work
After S3 stores Partner A's CSV, the processor needs to learn that new work exists. Configure an S3 ObjectCreated event notification for the incoming/ prefix and send that small notification to Amazon SQS.
The prefix filter matters. If the processor writes to processed/, that output must not match the same incoming notification and create a processing loop.
SQS is the durable work-ticket tray between S3 and the processor. The CSV bytes stay in S3. The queue message contains a reference closer to bucket = partner-ingest and key = incoming/partner-a/shipments-2026-09-04.csv, plus safe event metadata. When a processor is ready, it reads the ticket and uses its own S3 and KMS permissions to fetch the file. This lets SQS absorb bursts and preserve work while processors are busy or unavailable without trying to carry a multi-megabyte or multi-gigabyte file inside the message.
Because S3 notifications can be duplicated or arrive out of order, the queue and processor must not assume one perfectly ordered event per object.
Step 4: make file processing idempotent
Suppose S3 announces shipments-2026-09-04.csv twice. The second notification should not create a second trusted output or load the same shipment records again. This is the same pattern used in the preceding event-driven scenario: at-least-once delivery requires an idempotent consumer—repeating the same work request must not repeat its business effect.
The processor therefore creates a stable identity for the exact work: bucket, object key, object version, and processor version. With S3 Versioning, an overwritten file at the same key receives a new version ID, so it is legitimate new work. A newer processor version can also intentionally revalidate the old source after a rule is corrected. A DynamoDB item or another durable record can atomically claim this identity and record one completed result. The claim must be one conditional write, not a separate "check, then insert": two workers could otherwise both see no existing record and both continue. The conditional write lets one worker create the claim; the other sees that it lost and stops before producing a second business result.
identity = bucket + object key + version ID + processor version
example = partner-ingest + incoming/partner-a/shipments-2026-09-04.csv + v123 + validator-v4
Then validate what the business actually cares about:
- expected partner and path;
- filename and allowed extension;
- object size and required metadata;
- checksum or integrity evidence supplied by the workflow;
- schema, columns, and data types;
- row counts and business rules; and
- encryption and classification requirements.
An S3 object-created event means the object write completed from S3's perspective. It does not prove that the partner sent the complete business dataset or correct records.
Choose compute from what the file processor must actually do. A Lambda function may be a good fit for checking the filename, metadata, and schema of a small CSV. Downloading, decompressing, and scanning a multi-gigabyte archive may need more time, memory, temporary storage, or specialized software. In that case, use containers through Amazon ECS on AWS Fargate, managed jobs through AWS Batch, or data transformation through AWS Glue. Lambda can still coordinate the job without processing every byte itself.
Step 5: promote or quarantine explicitly
Now give the file an explicit trust state. These states are implemented with S3 prefixes or separate buckets, but first remember what each state means:
These outcomes mean different things:
- Incoming, such as
incoming/partner-a/shipments-2026-09-04.csv, means, "We received Partner A's bytes, but we do not trust the contents yet." - Processed, such as
processed/partner-a/shipments-2026-09-04.csv, means, "The defined checks passed, so downstream systems may use the approved output." - Quarantine, such as
quarantine/partner-a/shipments-2026-09-04.csv, means, "We successfully inspected the file and found a data or policy problem, such as the wrong schema." - Failed processing means, "The system could not finish the decision because the processor or a dependency failed." The source file remains untrusted while its work ticket retries or moves to the DLQ.
Do not delete the only raw copy until validation, reconciliation, and the retention policy allow it. In S3, a logical "move" is commonly implemented as a copy to the new key followed by deletion of the old key, so make the operation retry-safe.
Downstream analytics should consume the processed zone, not watch the raw incoming area directly.
Step 6: add sensitive-data discovery without confusing its job
Two security questions sound similar but need different controls. Amazon Macie asks, "Does this S3 data appear to contain sensitive information such as personally identifiable information (PII) or financial data?" A dedicated malware scanner asks, "Is this file malicious?" Use Macie for sensitive-data discovery and S3 data-security risk, not as antivirus.
Macie analyzes supported S3 objects and produces findings when it discovers categories such as personally identifiable or financial data. A finding gives the security team evidence to review; it does not by itself decide that the file is trusted.
If sensitive data must block promotion, design an explicit classification workflow whose timing, supported formats, KMS access, and failure behavior match that requirement. Do not assume automated discovery is a synchronous upload gate.
Step 7: make freshness and audit visible
Monitor the business promise as well as the AWS machinery. For example: Did Partner A's expected nightly CSV arrive by the deadline? If it arrived, is it still waiting in incoming/, stuck as an old SQS message, quarantined, or already promoted? Useful signals include:
- partner authentication and transfer failures;
- expected file arrival by deadline;
- SQS queue depth and age;
- processor errors, duration, and concurrency;
- messages in the ingest DLQ;
- files in quarantine or failed zones;
- KMS and access-denied errors; and
- time from upload to trusted promotion.
AWS CloudTrail records relevant AWS API and configuration activity. Transfer Family and processing logs provide operational evidence. S3 Versioning keeps earlier object versions, while S3 Object Lock can prevent versions from being deleted or overwritten during a required retention period. Use these or dedicated archival controls when the business must preserve immutable—unchangeable—evidence.
Completed architecture
5. Request Or Data Flow
Learn three lifecycles: transfer, validation, and failure recovery.
Lifecycle 1: partner upload
- Partner A's SFTP client connects to the Transfer Family endpoint and uploads
shipments-2026-09-04.csv. - Transfer Family asks the configured identity provider to authenticate Partner A.
- The user mapping supplies Partner A's logical directory and its scoped S3 access role.
- Transfer Family writes the object to
incoming/partner-a/shipments-2026-09-04.csv. - S3 stores the bytes privately and encrypts them according to the bucket configuration.
- After object creation completes, S3 sends the ingest queue a notification that references the object.
Lifecycle 2: successful validation
- The processor receives the SQS work ticket and atomically claims this object-and-processor version so a duplicate message cannot repeat the result.
- It follows the bucket and key in the message, then reads Partner A's CSV using its own IAM and KMS permissions.
- It checks integrity evidence, filename, schema, row data, and required security rules.
- It writes normalized or approved output into the processed zone; downstream systems never need to read the untrusted incoming copy.
- It records successful completion and deletes the SQS message.
- Downstream systems discover only the trusted output.
Lifecycle 3: operational failure versus quarantine
If Partner A's CSV has the wrong schema, the processor completed its inspection and found a problem with the data. Record the reason and place the file in quarantine. That is a completed trust decision.
If the processor times out or cannot reach a dependency, it never finished inspecting the file. Do not call the data invalid. Let SQS retry the work ticket. After the configured receive attempts, move that failed message to the dead-letter queue (DLQ) and alert an owner.
This distinction prevents temporary infrastructure failures from being mistaken for bad partner data.
6. Security Controls
Isolate partner identities
Use a separate user identity and scoped storage role or session policy—temporary permission restrictions applied to that signed-in session—for each partner pattern. Limit bucket listing and object actions to the intended prefixes. Logical directories should complement, not replace, IAM and bucket policy enforcement.
Keep the bucket private
Enable S3 Block Public Access. Partners enter through the approved Transfer Family endpoint or another explicitly designed path—not a public bucket URL.
Coordinate KMS permissions
Follow Partner A's encrypted CSV through the system. The Transfer Family user role writes it, the processor later reads and decrypts it, a scanner may inspect it, and a recovery process may restore it. A successful upload proves only that the writer had the required permissions; it does not prove that every later reader can decrypt the object. With SSE-KMS, consider every actor:
- the Transfer Family user role that writes incoming objects;
- the processing role that decrypts incoming data and encrypts output;
- Macie or scanning services that require access to in-scope objects; and
- administrators or recovery workflows that need controlled access.
Reading the object and decrypting it are related but separate authorization checks. The processor needs S3 permission to read the object. It also needs IAM permission to request kms:Decrypt, while the KMS key policy or a KMS grant—a separate permission issued for a key—must allow that use, either directly or through IAM delegation. S3 read permission alone is not enough.
Separate transfer encryption from storage encryption
SFTP encrypts data in transit during the partner session. S3 server-side encryption protects stored objects. They cover different parts of the lifecycle.
Limit sensitive events and logs
Do not place file contents or unnecessary personal data in SQS messages or logs. Store object references, processing identifiers, and safe diagnostic metadata.
7. Resilience Controls
Transfer Family keeps the SFTP endpoint available without a self-managed server fleet, and S3 durably stores Partner A's CSV. Neither service guarantees that the validator will be available or complete its work, so the ingestion workflow still needs application-level recovery.
Use SQS to retain arrival work during processor outages and to control concurrency against scanners, databases, or partner-specific dependencies.
Make processing idempotent because both the S3 notification and SQS message delivery can repeat. Two deliveries for version v123 of Partner A's CSV should produce one result. If Partner A overwrites the same key and S3 creates version v124, the identity must treat that as new work rather than confusing it with the completed v123 record.
Retain raw input until the business retention and replay window closes. A corrected processor should be able to replay a known object without asking the partner to resend it unnecessarily.
Alarm on oldest-message age, DLQ depth, quarantine growth, expected-file freshness, and promotion latency. A queue with no visible errors can still represent a missed daily feed.
Use separate buckets or accounts when a mistake or compromise must be contained more strongly than prefix-based permissions allow, or when legal or organizational isolation requires it.
8. Performance Controls
Transfer speed depends on partner bandwidth, latency, protocol implementation, file size, and endpoint connectivity. A managed endpoint cannot make the partner's network faster.
Many small files create more transfer sessions, S3 requests, notifications, queue messages, and processor invocations than a smaller number of appropriately sized files.
Set processor concurrency from sustainable downstream capacity. SQS can absorb a burst, but excessive Lambda concurrency can overwhelm a malware scanner, database, or external validation API.
Start with the work, then choose the service. A short schema check on Partner A's normal CSV has very different runtime needs from decompressing and scanning a multi-gigabyte archive:
| Work shape | Possible processing direction |
|---|---|
| Check a small CSV's metadata and schema in a short, bounded invocation | Lambda may fit |
| Decompress or scan a large archive with longer CPU, memory, or storage needs | ECS/Fargate or AWS Batch may fit |
| Transform approved CSV data into analytics formats at scale | Glue or another data-processing service may fit |
| Coordinate validation, scanning, approval, and promotion as several steps | Step Functions can coordinate the workflow |
Organize processed object keys by values such as date or partner so analytics jobs can skip unrelated files. Analytics systems call this partition pruning. A job should not scan the entire bucket to find one day's approved files.
9. Cost Controls
Important cost drivers include:
- Transfer Family endpoint uptime and transferred data;
- S3 storage, requests, versioning, and lifecycle;
- KMS API usage;
- SQS requests and Lambda or container processing;
- malware or data-validation tooling;
- Macie sensitive data analysis;
- logs, metrics, and retained DLQ or quarantine objects.
Endpoint cost matters even when no partner is transferring a file. Compare the managed service with the full cost of operating and securing a highly available SFTP fleet, not only an EC2 hourly price.
Use lifecycle policies separately for incoming, processed, quarantine, failed, and audit data. Their retention requirements are not automatically the same.
Filter events to avoid recursive processing loops and unnecessary invocations. SQS batching lets one processor invocation receive several work messages; use it where that improves efficiency without making one failed file difficult to isolate from the rest of the batch.
Scope Macie discovery to the security requirement. Scanning every object continuously may cost more than risk-based automated discovery and targeted jobs.
10. Exam Variants
| If the question says... | Think... | Why |
|---|---|---|
| "Partners must continue using SFTP while files land in S3" | AWS Transfer Family | It provides the managed protocol endpoint backed by S3 |
| "Move a large on-premises dataset through a managed transfer task" | AWS DataSync | DataSync is a managed bulk-data movement service, not a partner SFTP endpoint |
| "Provide on-premises Network File System (NFS) or Server Message Block (SMB) access backed by AWS storage" | AWS Storage Gateway | Storage Gateway connects familiar on-premises storage protocols to AWS storage |
| "Prevent one partner from viewing another's uploads" | Separate identity mapping + scoped IAM/bucket access | Directory appearance alone is not authorization |
| "Process arrivals reliably during bursts" | S3 notification → SQS → processor | SQS buffers and supports retry isolation |
| "The same file event may arrive twice" | Idempotent processing | S3 notifications are at-least-once |
| "Encrypt with a customer-controlled key" | SSE-KMS + IAM and key policy permissions | Every required actor needs authorized key use |
| "Discover personally identifiable information (PII) or financial data in S3" | Amazon Macie | Macie performs sensitive data discovery |
| "Scan uploaded files for malware" | Dedicated malware-scanning workflow | Macie is not an antivirus engine |
| "Do not expose unvalidated files to analytics" | Incoming/quarantine/processed trust zones | Promotion happens only after defined validation passes |
11. Common Traps
| Trap | Better reasoning |
|---|---|
| "SFTP means we must run an EC2 file server." | Transfer Family supplies a managed SFTP endpoint backed by S3 or Amazon Elastic File System (EFS), AWS shared file storage. |
| "A logical home directory secures the bucket by itself." | IAM, session, and bucket policies enforce access; directory mapping shapes what the user sees. |
| "A successful upload is trusted data." | Transfer success proves arrival, not schema, business completeness, classification, or malware status. |
| "S3 sends exactly one event in upload order." | Design for duplicate and out-of-order notifications. |
| "A queue makes the processor idempotent." | SQS retains and retries work; the processor must prevent duplicate business effects. |
| "KMS encryption only needs an IAM allow." | The key policy and relevant service/principal permissions must also support the operation. |
| "Macie scans uploads for malware." | Macie discovers sensitive data and S3 security risk; malware scanning is a separate control. |
| "Quarantine and DLQ mean the same thing." | Quarantine is a data-trust decision; a DLQ is operational failure isolation. |
| "Write processed output under the same triggering prefix." | Use event filters and separate output paths to prevent loops. |
| "Lambda is always the processor for an S3 event." | Choose compute from file size, duration, libraries, and downstream limits. |
Final Mental Model: One-Minute Review
| Term | Exact job | Remember it as |
|---|---|---|
| Transfer Family | Accepts the partner's familiar file-transfer protocol without an EC2 SFTP fleet | The managed loading dock |
| User mapping and IAM role | Authenticate the partner and scope its S3 operations | The driver's ID and delivery permit |
| S3 incoming zone | Durably stores received but untrusted files | The inspection bay |
| SSE-KMS | Encrypts stored objects with customer-controlled key permissions when required | The locked storage container |
| S3 event notification | Announces that an object was created; delivery can repeat and be out of order | The arrival bell |
| SQS ingest queue | Buffers validation work and isolates processor outages | The inspection-ticket tray |
| Idempotent processor | Validates one object identity safely even when work repeats | The receiving inspector |
| Processed and quarantine zones | Separate trusted outputs from rejected inputs | The approved warehouse and isolation cage |
| Macie | Discovers sensitive data and S3 policy risk | The sensitive-data classifier, not antivirus |
For Partner A's CSV, the trust path is authenticate → land → buffer → validate → promote or quarantine. Transfer Family proves who delivered the bytes, S3 stores them, SQS preserves a work ticket, the validator decides trust, and Macie supplies sensitive-data visibility rather than malware protection.
12. Related Topics
Use AWS Transfer Family for protocols, endpoints, identity providers, logical directories, and storage backends. Review Amazon S3 for object storage and bucket controls, AWS Key Management Service for key-policy boundaries, Amazon SQS for buffered processing, and Amazon Macie for sensitive data discovery.
Continue to Analytics Data Lake On S3 after the ingest system can reliably produce trusted, organized data.
Official AWS references: