Contents
Course · 13
Which AWS service should hold your data, and which should carry your messages?
This page covers the AWS services that store data (RDS, Aurora, DynamoDB, ElastiCache, MemoryDB), move messages (SQS, SNS, EventBridge, Kinesis, MSK), glue serverless apps together (Step Functions, API Gateway) and watch them (CloudWatch, X-Ray). Every limit was checked on docs.aws.amazon.com on 2026-10-25. Several changed recently, so older notes may be wrong.
Pick the store by access pattern (SQL joins → RDS or Aurora; key lookups at any scale → DynamoDB), and pick the pipe by delivery style (one worker → SQS; many copies → SNS or EventBridge; ordered replayable stream → Kinesis or MSK).
- RDS Multi-AZ is for surviving failure (standby you cannot read); read replicas are for scaling reads (async, readable). Different jobs.
- Aurora keeps 6 copies of your data across 3 AZs and allows up to 15 replicas.
- DynamoDB: item max 400 KB; 1 RCU = one strong read of up to 4 KB per second; 1 WCU = one write of up to 1 KB per second; one partition tops out at 3,000 RCU / 1,000 WCU.
- New limits in 2025–2026: SQS messages up to 1 MiB, EventBridge events up to 1 MB, Kinesis records up to 10 MiB.
- API Gateway REST integration timeout is 29 s by default, and can now be raised for Regional and private APIs.
—Words you need
Databases
01RDS: managed SQL databases
RDS runs a normal relational database for you: AWS handles setup, backups, patching and failover. The RDS guide lists six engines: IBM Db2, MariaDB, Microsoft SQL Server, MySQL, Oracle and PostgreSQL. Aurora has its own guide (next section).
The classic interview trap: Multi-AZ and read replicas sound alike but solve different problems.
| Question | Multi-AZ (DB instance) | Read replica |
|---|---|---|
| Purpose | High availability: survive an AZ or instance failure | Scale reads; also a reporting copy |
| Copy method | Synchronous | Asynchronous (may lag) |
| Can you read from it? | No | Yes |
| Failover | Automatic; typically 60–120 s | Manual promotion to a standalone DB |
| How many | One standby (a Multi-AZ DB cluster has two readable standbys) | Up to 15 per source (quota) |
Backups: automated backups keep daily snapshots plus transaction logs, so you can restore to any second inside the retention window. That is PITR.
Say it plainly in interviews: Multi-AZ for availability, replicas for read scale, backups for "oops, I deleted a table".
Source: What is RDS · RDS quotas
02Aurora: storage built for the cloud
Aurora is AWS's MySQL- and PostgreSQL-compatible database. The big idea: compute and storage are split. The writer and all readers share one cluster volume. So a replica does not need its own full copy of the data.
| Fact (checked 2026-10-25) | Value |
|---|---|
| Storage copies | 6 copies across 3 AZs, written synchronously |
| Copies it can lose | Up to 2 without losing writes; up to 3 without losing reads (Aurora FAQ) |
| Replicas | Up to 15 Aurora Replicas; lag "usually much less than 100 milliseconds" |
| Max storage | 256 TiB on newer engine versions; 128 TiB on older ones |
| Serverless v2 capacity | Measured in ACUs (about 2 GiB memory each); 0 to 256 ACU (MySQL), up to 384 ACU (newer PostgreSQL); steps of 0.5 ACU; 0 ACU means auto-pause |
Need SQL with many fast replicas or spiky load? Aurora (Serverless v2 for spikes). Need the cheapest small SQL box? Plain RDS.
Source: Aurora User Guide pages linked above
03DynamoDB: keys, items and indexes
DynamoDB is a key-value and document database with no servers to run. A table holds items (rows). Each item has a primary key:
- Partition key only: DynamoDB hashes it to choose the partition (storage slice) where the item lives. Lookup is by exact key.
- Partition key + sort key: items with the same partition key sit together, ordered by the sort key. You can ask for ranges, like "all orders of user 42 in March".
| Limit (checked 2026-10-25) | Value |
|---|---|
| Max item size | 400 KB |
| Partition key value | max 2,048 bytes |
| Sort key value | max 1,024 bytes |
| Global secondary indexes per table | 20 (default quota) |
| Local secondary indexes per table | 5 |
| Item collection size with an LSI | 10 GB per partition key |
GSI vs LSI
| Question | Global secondary index (GSI) | Local secondary index (LSI) |
|---|---|---|
| Keys | Any new partition key and sort key | Same partition key, different sort key |
| When created | Any time | Only when the table is created |
| Strongly consistent reads? | No, eventually consistent only | Yes, can be |
| Size limit | None per key | 10 GB per partition key value |
| Capacity | Its own | Shares the table's |
List your queries before you pick keys. A query you did not plan for usually needs a new GSI or a full scan.
Source: DynamoDB constraints
04DynamoDB: capacity, consistency, hot partitions
| Unit | Buys you, per second |
|---|---|
| 1 RCU | 1 strongly consistent read of an item up to 4 KB, or 2 eventually consistent reads |
| Transactional read | 2 RCUs per 4 KB |
| 1 WCU | 1 write of an item up to 1 KB |
| Transactional write | 2 WCUs per 1 KB |
Worked example
Items are 6 KB. You need 100 reads per second and 50 writes per second of 2.5 KB items.
// reads: round item size UP to the next 4 KB
6 KB / 4 KB = 1.5 → 2 RCU per strong read
100 strong reads/s × 2 = 200 RCU
100 eventual reads/s × 2 / 2 = 100 RCU
100 transactional reads/s × 2 × 2 = 400 RCU
// writes: round UP to the next 1 KB
2.5 KB → 3 WCU per write
50 writes/s × 3 = 150 WCU
50 transactional writes/s × 3 × 2 = 300 WCU
Read capacity for the same 100 reads/s of 6 KB items
RCU needed by read type. Hover for the formula.
Show the full table
| Read type | RCU per read | RCU for 100/s |
|---|---|---|
| Eventually consistent | 1 | 100 |
| Strongly consistent | 2 | 200 |
| Transactional | 4 | 400 |
On-demand vs provisioned
| Mode | How you pay | Scaling | Pick when |
|---|---|---|---|
| On-demand (default) | Per request | Instantly handles up to double the previous peak | New or spiky traffic |
| Provisioned | Per RCU/WCU per hour (with optional auto scaling) | You set or auto-scale capacity | Steady, predictable traffic |
Consistency: reads are eventually consistent by default (may miss a write from the last moment). Ask for strongly consistent to always see the latest write; it costs double and is not possible on GSIs or Streams.
Hot partitions: each partition can serve at most 3,000 RCU and 1,000 WCU per second. If one key (say a celebrity's user ID) gets most traffic, that partition throttles even when the table has spare capacity. Adaptive capacity helps: DynamoDB raises throughput for busy partitions and can move hot items to their own partitions. But one single item still cannot pass the per-partition limit.
Round up item sizes, halve for eventual reads, double for transactions, and pick a key with many distinct values.
Source: capacity unit rules · read consistency
05DynamoDB: single-table design, Streams, TTL, transactions, DAX
| Feature | What it does | Key fact (checked 2026-10-25) |
|---|---|---|
| Single-table design | Store many entity types (user, order, address) in one table; the sort key says which type | One query can fetch related items together, at lower cost. Downsides: steep learning curve, shared backups and encryption |
| DynamoDB Streams | An ordered log of item changes | Each record appears exactly once, in order per item; kept for 24 hours |
| TTL | Auto-deletes items after an expiry timestamp | Usually within a few days of expiry; does not use write capacity |
| Transactions | All-or-nothing writes or reads across items and tables | Up to 100 unique items and 4 MB per transaction; costs double capacity |
| DAX | An in-memory cache in front of DynamoDB | Microsecond reads; same API, so few code changes |
PK SK data
USER#42 PROFILE name=Asha, plan=pro
USER#42 ORDER#2026-10-01 total=499
USER#42 ORDER#2026-10-20 total=1299
USER#42 ADDRESS#home city=Pune
// Query PK = USER#42 AND begins_with(SK, "ORDER#") → all orders, sorted by date
Streams + Lambda turn every write into an event; TTL cleans up sessions for free. Never rely on TTL for exact-second expiry.
Source: DynamoDB Developer Guide pages linked above
Caches
06ElastiCache and MemoryDB
ElastiCache runs in-memory caches for you. It supports three engines: Valkey (the open-source fork of Redis), Redis OSS, and Memcached. MemoryDB is different: it is a durable, Redis-compatible database, not just a cache.
| Question | ElastiCache Valkey / Redis OSS | ElastiCache Memcached | MemoryDB |
|---|---|---|---|
| Data types | Rich: lists, sets, sorted sets, hashes, streams | Simple strings | Same as Valkey / Redis |
| Replication / high availability | Yes | No | Yes, Multi-AZ |
| Multi-threaded (per AWS comparison) | See page 05 for engine internals | Yes | Redis-compatible |
| Durability | Cache: data can be lost (Valkey 9.0 adds an optional Multi-AZ log) | None | Every write stored in a Multi-AZ transaction log |
| Latency | Sub-millisecond reads | Sub-millisecond reads | Microsecond reads, single-digit ms writes |
| Use as | Cache, sessions, leaderboards, rate limits | Plain, large, simple cache | Primary database for fast data |
Cache that can be rebuilt from the database → ElastiCache. Data that must survive and still be in memory → MemoryDB.
Source: ElastiCache: choosing an engine · MemoryDB · Course page 05: Redis
Messaging
07SQS and SNS: queues and fan-out
SQS is a queue: a producer puts a message in, one consumer takes it out and deletes it when done. It lets a slow worker absorb a fast producer.
| Question | Standard queue | FIFO queue |
|---|---|---|
| Throughput | "Nearly unlimited" | 300 API calls/s per action per partition; 3,000 msg/s with batches of 10; high-throughput mode up to 70,000 calls/s in the biggest Regions |
| Delivery | At least once (duplicates possible) | Exactly-once processing (5-minute dedup window) |
| Order | Best effort | Strict, per message group ID |
| Setting (checked 2026-10-25) | Value | Why it matters |
|---|---|---|
| Max message size | 1 MiB (was 256 KiB) | Bigger: put it in S3 and send a pointer |
| Retention | Default 4 days, 60 s to 14 days | Unread messages vanish after this |
| Visibility timeout | Default 30 s, max 12 h | Must be longer than your processing time |
| Long polling wait | Up to 20 s | Fewer empty responses, lower cost |
| Delay | 0 to 15 min | Hold messages back before delivery |
| Batch | Up to 10 messages per call | 10× fewer API calls |
The visibility timeout is the key idea. When a consumer receives a message, SQS hides it. If the consumer deletes it in time, it is gone. If the consumer crashes, the timer runs out and the message reappears for someone else. After maxReceiveCount failed tries, SQS moves it to a dead-letter queue (DLQ) so one bad message does not loop forever.
SNS fan-out
SNS is publish/subscribe: one message to a topic is copied to every subscriber. Subscribers can be SQS queues, Lambda, HTTP(S), email, SMS, mobile push or Firehose. The classic pattern: SNS in front of several SQS queues, so each team gets its own durable copy.
maxReceiveCount is "the number of times a consumer can receive a message… before it is moved to a dead-letter queue". Sources: SQS message quotas, standard queues, FIFO exactly-once, DLQs, What is SNS.Standard SQS can deliver twice, so make consumers idempotent (for example, store processed message IDs).
Source: SQS queue quotas · SNS FIFO topics · SNS message filtering
08EventBridge, Kinesis and MSK
EventBridge is an event router. Services put events on an event bus; rules match events by content and send them to targets (up to 5 targets per rule on the classic bus). EventBridge Scheduler runs one-off or cron-style jobs that call a target at a time you set.
Kinesis Data Streams is an ordered log split into shards. Records with the same partition key go to the same shard, in order. Many readers can read the same data, and they can re-read it until it expires. Amazon MSK is managed Apache Kafka (see page 09): provisioned clusters with Standard or Express brokers, or MSK Serverless.
| Kinesis limit (checked 2026-10-25) | Value |
|---|---|
| Write per shard (provisioned) | 1 MB/s or 1,000 records/s |
| Read per shard | 2 MB/s, up to 5 GetRecords calls/s (shared by all classic readers) |
| Enhanced fan-out | 2 MB/s per shard per registered consumer; up to 20 consumers (50 in On-demand Advantage) |
| Retention | 24 h default, up to 365 days |
| Max record size | 10 MiB (was 1 MB); large records use burst capacity |
| On-demand mode | Starts at 4 MB/s write; scales up to 10 GB/s in the largest Regions |
SQS vs SNS vs EventBridge vs Kinesis vs MSK
| Question | SQS | SNS | EventBridge | Kinesis Data Streams | MSK (Kafka) |
|---|---|---|---|---|---|
| Model | Queue: one consumer per message | Pub/sub push to subscribers | Event bus with content rules | Ordered log in shards | Ordered log in partitions |
| Message kept after read? | No, deleted | No, pushed then gone | No | Yes, until retention ends | Yes, until retention ends |
| Replay? | No | No | From an archive (not checked here) | Yes | Yes |
| Ordering | FIFO queues only | FIFO topics only | Not checked here | Per shard | Per partition |
| Max message | 1 MiB | Not checked here | 1 MB per request | 10 MiB | Kafka config (not checked here) |
| Scaling unit | None to manage | None to manage | None to manage | Shards, or on-demand | Brokers, or Serverless |
| Best for | Work queues, buffering, retries | Fan-out, notifications | Routing AWS and SaaS events, schedules | Clickstreams, logs, real-time analytics | Teams already on Kafka, its ecosystem |
Need replay or several independent readers of the same ordered data? A stream (Kinesis or MSK). Otherwise a queue or bus is simpler.
Source: pages linked above · Course page 09: Kafka
Serverless and operations
09Step Functions and API Gateway
Step Functions runs a workflow (a state machine): call service A, wait, branch, retry B on failure, run steps in parallel. You draw the flow; it keeps the state and the history.
| Question | Standard workflow | Express workflow |
|---|---|---|
| Max duration | Up to 1 year | Up to 5 minutes |
| Execution guarantee | Exactly once | At least once (async), at most once (sync) |
| Good for | Long business processes, human approval steps | High-volume short event processing |
API Gateway puts a managed HTTP front door in front of Lambda, HTTP backends or AWS services. It handles auth, throttling and stages.
| Question | REST API | HTTP API |
|---|---|---|
| Price and features | More features, higher price | Fewer features, lower price |
| Only here | API keys, usage plans, per-client throttling, caching, request validation, AWS WAF, private endpoints, edge-optimized endpoints | JWT authorizers built in, automatic deployments |
| Integration timeout | 50 ms – 29 s; can be raised for Regional and private APIs (may lower your throttle quota) | Max 30 s |
| Payload | 10 MB | 10 MB |
Anything longer than ~29 s should not sit behind a synchronous API call: return 202, do the work async, let the client poll or get a callback.
Source: pages linked above
10Serverless patterns
| Path | Who waits | Limits to respect | Failure handling |
|---|---|---|---|
| API Gateway → Lambda → DynamoDB | The user | 29–30 s API timeout; Lambda 6 MB sync payload; 1,000 default concurrency vs 10,000 rps API throttle | Return an error; client retries with an idempotency key |
| S3 event → Lambda → SQS | Nobody | Lambda 15 min; SQS 1 MiB message | Async retries, then a DLQ or failure destination |
Match the limits end to end: API throttle, Lambda concurrency and DynamoDB capacity must agree, or the weakest one throttles.
11CloudWatch, X-Ray, and cost awareness
| Tool | What it gives you | Key fact (checked 2026-10-25) |
|---|---|---|
| CloudWatch metrics | Numbers over time (CPU, latency, queue depth) | Standard resolution 1 minute; high resolution 1 second |
| CloudWatch alarms | Watch a metric, act when it crosses a line | States: OK, ALARM, INSUFFICIENT_DATA |
| CloudWatch Logs | Log storage and search | Kept forever by default; set a retention per log group |
| X-Ray | Distributed tracing: one request across many services, shown as a service map | X-Ray SDKs and daemon in maintenance mode since 25 Feb 2026; AWS recommends OpenTelemetry |
Cost tips (only verified claims)
| Cost trap | Fact | Fix |
|---|---|---|
| NAT gateway data | us-east-1: $0.045 per hour plus $0.045 per GB processed (pricing page) | Gateway endpoints for S3 and DynamoDB (free); cache big downloads |
| Cross-AZ traffic | About $0.01 per GB in each direction (AWS architecture blog; the pricing table did not load for this check) | Keep chatty services and their caches in the same AZ where safe |
| Logs kept forever | Default retention is indefinite | Set retention (for example 30 days) on every log group |
| Provisioned concurrency | Billed even when idle | Use only where cold starts hurt users |
| DynamoDB on-demand | Price cut 50% on 1 Nov 2024 | Re-check old "provisioned is always cheaper" advice |
Alarm on what users feel (errors, p99 latency, queue age), not only CPU. And set log retention on day one.
Source: pages linked above · Course page 14: observability
Practice
—When to use what
| Situation | Pick | Why |
|---|---|---|
| Joins, reports, strict schema, moderate scale | RDS PostgreSQL or MySQL | SQL and transactions; simplest |
| SQL with many readers or fast failover | Aurora | Up to 15 replicas on shared storage |
| SQL with unpredictable spikes | Aurora Serverless v2 | Scales in 0.5 ACU steps; can pause at 0 |
| Key lookups at huge scale, known queries | DynamoDB | Key lookups scale by adding partitions; no servers |
| Same item read thousands of times a second | DAX or ElastiCache | Takes load off one hot partition |
| Session store, leaderboard, rate limiter | ElastiCache (Valkey/Redis OSS) | Rich data types, sub-ms |
| In-memory but must not lose data | MemoryDB | Multi-AZ transaction log |
| Background jobs, smooth out spikes | SQS standard + DLQ | Buffers, retries, near-unlimited throughput |
| Strict order per customer | SQS FIFO with message group = customer ID | Order per group, exactly-once processing |
| One event, many independent consumers | SNS → SQS fan-out, or EventBridge | Each consumer gets its own copy |
| Route events by content from AWS or SaaS | EventBridge | Rules match on event fields |
| Run a job every night at 2 a.m. | EventBridge Scheduler | Managed cron with targets |
| Clickstream, replay, several readers | Kinesis Data Streams | Ordered per shard, kept up to 365 days |
| Team already runs Kafka | MSK | Same APIs and tools |
| Multi-step process with retries and waits | Step Functions Standard | Durable state up to a year |
| Public API with keys, quotas, WAF | API Gateway REST API | Usage plans and WAF only here |
| Simple, cheap Lambda-backed API | API Gateway HTTP API | Lower price, JWT auth built in |
—Common traps
Wrong because: the standby cannot serve reads.
Fix: add read replicas (or Aurora readers).
If missed: async lag shows stale data right after a save.
Fix: read-your-writes from the primary.
If missed: keys like status create hot partitions.
Fix: high-cardinality keys, or write sharding.
If missed: a scan reads (and bills) the whole table.
Fix: design keys or a GSI for each query.
If missed: not supported; GSIs are eventually consistent.
Fix: read the base table, or use an LSI planned at creation.
If missed: a second worker grabs the same message.
Fix: timeout above max processing time; extend it while working.
If missed: one poison message retries forever.
Fix: DLQ with maxReceiveCount and an alarm on its depth.
If missed: items linger up to a few days.
Fix: filter expired items in reads; use Scheduler for exact times.
If missed: quoting 256 KB SQS or 1 MB Kinesis records.
Fix: now 1 MiB and 10 MiB; check the quotas page.
If missed: 29–30 s timeout returns 504 to users.
Fix: accept, queue, return 202 and a job ID.
—Interview questions
Show answer
Show answer
Show answer
Show answer
begins_with(SK, "ORDER#2026-10"). Together they form the primary key and must be unique. Choose a partition key with many distinct values and even traffic (user ID, not country). Limits: partition key value 2,048 bytes, sort key 1,024 bytes, whole item 400 KB.Show answer
Show answer
Show answer
Show answer
Show answer
USER#42 / PROFILE, USER#42 / ORDER#…. Because an item collection lives together, one Query fetches a user and their orders, cheaper and faster than several calls (AWS's example: one query for two small items costs 0.5 RCU vs 1 RCU for two). Avoid it when entities need different backups, encryption keys or table classes, when access patterns are unknown or ad hoc, when the team uses GraphQL resolvers per entity, or when a stream consumer should see only one entity type.Show answer
Show answer
ChangeMessageVisibility for long jobs. And make handlers idempotent, because standard queues are at-least-once anyway.Show answer
Show answer
Show answer
GetRecords calls per second per shard. Four apps each reading 5 MB/s need 20 MB/s total, far over 6 shards × 2 MB/s = 12 MB/s. So use enhanced fan-out, which gives each registered consumer its own 2 MB/s per shard. Or use on-demand mode and let Kinesis scale. Check the partition key spreads evenly, or one shard runs hot.Show answer
Show answer
Show answer
uploads/ in S3. The S3 event triggers a Lambda that makes thumbnails and writes them to thumbs/, then writes metadata to DynamoDB. For load spikes put SQS between S3 and the worker Lambda, so bursts queue instead of throttling, with a DLQ for corrupt images. Make the worker idempotent (same input key → same output key), since events can repeat. Serve images through CloudFront. Watch: Lambda 15-minute cap and memory up to 10,240 MB for big images; never write output into the same prefix that triggers the function, or it loops.Show answer
Show answer
attribute_not_exists). If the put fails, the message was handled; skip it. Do the side effect and mark it done in one transaction where possible, or make the side effect itself keyed (payment provider idempotency key). Add a TTL to the processed table so it cleans itself after the retry window. This makes at-least-once queues (SQS standard, Express workflows) safe.Show answer
—Cheat sheet
| Fact | Value | Source |
|---|---|---|
| RDS Multi-AZ failover | typically 60–120 s | Failover |
| RDS read replicas | up to 15 per source | RDS quotas |
| RDS backup retention | 0–35 days | Retention |
| Aurora storage | 6 copies, 3 AZs | Aurora HA |
| Aurora replicas | up to 15, lag < 100 ms usually | Replication |
| Aurora Serverless v2 | 0–256 ACU (PostgreSQL up to 384) | Serverless v2 |
| DynamoDB item | max 400 KB | Constraints |
| RCU / WCU | 4 KB strong read / 1 KB write | Constraints |
| Partition ceiling | 3,000 RCU, 1,000 WCU | Key design |
| Indexes | 20 GSI (default), 5 LSI | Quotas |
| Transactions | 100 items, 4 MB | Constraints |
| Streams retention | 24 h | Streams |
| SQS message | 1 MiB | SQS quotas |
| SQS retention | default 4 days, max 14 days | SQS quotas |
| SQS visibility timeout | default 30 s, max 12 h | SQS quotas |
| SQS long poll | max 20 s | Queue quotas |
| SQS FIFO | 300 calls/s, 3,000 msg/s batched | SQS quotas |
| EventBridge event | 1 MB per request; 5 targets per rule | PutEvents |
| Kinesis shard write | 1 MB/s or 1,000 records/s | Kinesis quotas |
| Kinesis shard read | 2 MB/s, 5 GetRecords/s | Kinesis quotas |
| Kinesis retention | 24 h to 365 days | Retention |
| Step Functions | Standard 1 year, Express 5 min | Workflow types |
| API Gateway REST timeout | 29 s default max (raisable, Regional/private) | REST quotas |
| API Gateway throttle | 10,000 rps, burst 5,000 | Account quotas |
| CloudWatch resolution | 1 min standard, 1 s high | Concepts |
| NAT gateway (us-east-1) | $0.045/h + $0.045/GB | VPC pricing |
—Sources
| What | Where |
|---|---|
| RDS | What is RDS, Multi-AZ instance, Multi-AZ cluster, failover, read replicas, quotas, backup retention, PITR |
| Aurora | high availability, FAQ, replication, Serverless v2, quotas |
| DynamoDB | constraints, quotas, LSI, read consistency, key design, adaptive capacity, on-demand, data modeling, Streams, TTL, DAX, Nov 2024 price cut |
| ElastiCache, MemoryDB | engine comparison, MemoryDB, MemoryDB consistency |
| SQS, SNS | message quotas, queue quotas, standard, FIFO, DLQ, SNS, SNS FIFO, SNS filtering |
| EventBridge, Kinesis, MSK | PutEvents, EventBridge quotas, Kinesis quotas, Kinesis retention, enhanced fan-out, MSK |
| Step Functions, API Gateway | workflow types, REST quotas, HTTP API quotas, account quotas, REST vs HTTP |
| Lambda limits referenced | Lambda quotas |
| CloudWatch, X-Ray | concepts, alarms, log groups, X-Ray concepts, X-Ray SDK timeline |
| Cost | VPC pricing, data transfer costs (AWS blog), gateway endpoints |