AWS Analytics & Big Data
Redshift Serverless
Run analytics without managing data warehouse capacity or clusters
Amazon Redshift Serverless automatically provisions and scales data warehouse capacity based on workload demand, eliminating the need to choose node types, cluster sizes, or manage manual scaling. You pay only for the compute used during query execution (measured in Redshift Processing Units per second), making it ideal for intermittent analytics workloads. It is backed by the same RA3 architecture as provisioned Redshift and is fully compatible with existing SQL, drivers, and integrations.
Redshift Serverless Architecture - Namespaces, Workgroups, and RPUs
Redshift Serverless introduces two new concepts that replace the traditional cluster:
| Concept | Contains | Analogy |
|---|---|---|
| Namespace | Database objects, schemas, tables, users, audit logs, snapshots | The data layer - like an RDS instance |
| Workgroup | Compute configuration (base RPUs, VPC, security groups, endpoints) | The compute layer - like an RDS instance class |
RPU (Redshift Processing Unit) is the compute unit. You set a base RPU capacity (8-512 RPUs, in multiples of 8) and Redshift Serverless auto-scales above the base when queries demand more capacity. You are billed only for seconds of RPU usage when queries are running.
| RPU Setting | vCPU Equivalent | Best For |
|---|---|---|
| 8 RPU (base) | ~16 vCPU | Light workloads, dev/test, BI tools |
| 32 RPU | ~64 vCPU | Moderate analytics, concurrent dashboards |
| 128 RPU | ~256 vCPU | Heavy ETL, large datasets, high concurrency |
| 512 RPU | ~1024 vCPU | Maximum performance for peak workloads |
Setting a higher base RPU means faster query start time because Redshift can immediately use that capacity. Lower base RPU saves money at idle but causes latency when queries start if scaling is needed. For BI dashboards, set base RPU high enough to avoid cold-start lag.
Serverless vs Provisioned Redshift - When to Use Each
| Dimension | Redshift Serverless | Provisioned Redshift |
|---|---|---|
| Pricing model | Per RPU-second - pay only during queries | Per node-hour - pay 24/7 |
| Idle cost | Zero - no billing when no queries run | Full node cost even at idle |
| Capacity planning | None - auto-scales | You choose node type and count |
| Max capacity | 512 RPU baseline + autoscaling | Up to 128 RA3 nodes |
| Concurrency scaling | Automatic | Manual or concurrency scaling add-on |
| Reserved pricing | Not available | 1 or 3 year Reserved Instances (30-60% savings) |
| Snapshot/restore | Supported | Supported |
| Maintenance windows | Managed by AWS | You control timing |
Rule of thumb: if queries run less than 8-10 hours per day, Serverless is cheaper. If queries run continuously (BI tools polling every minute, scheduled ETL jobs all day), provisioned with Reserved Instances is cheaper. Many organizations run Serverless for dev/staging and provisioned for production.
Data Sharing - Serverless as Consumer of Provisioned Data
Redshift Data Sharing lets provisioned and serverless Redshift clusters share live data without copying. A common pattern: provisioned Redshift does heavy ETL and publishes curated datasets; Serverless namespaces across business units consume them for ad-hoc analysis.
| Scenario | Producer | Consumer |
|---|---|---|
| Centralized data mart | Provisioned RA3 (ETL team) | Serverless (business analysts) |
| Cross-account analytics | Provisioned in data platform account | Serverless in each product team account |
| Dev/test isolation | Provisioned production cluster | Serverless dev namespace (read-only) |
Data sharing is zero-copy - the consumer queries the producer's data directly using Redshift's storage layer (Redshift Managed Storage on S3). There is no ETL job to keep in sync. Updates in the producer are immediately visible to consumers.
Integrations - Redshift Spectrum, Glue, Firehose, and QuickSight
Redshift Serverless integrates with the broader AWS analytics ecosystem:
| Integration | How It Works | Use Case |
|---|---|---|
| Redshift Spectrum | Query S3 data lake directly from Serverless using external tables | Join warehouse tables with S3 raw data |
| Glue ETL | Glue writes directly to Redshift via JDBC or the Redshift connector | Managed ETL pipelines |
| Kinesis Firehose | Firehose COPY command loads into Serverless tables | Real-time event ingestion |
| QuickSight | Direct connector - QuickSight queries Serverless endpoint | Interactive BI dashboards |
| dbt | dbt works via JDBC - same as provisioned Redshift | SQL-based data transformation |
Pricing - RPUs, Storage, and Cost Controls
Redshift Serverless billing has two components:
| Component | Price | Notes |
|---|---|---|
| Compute (RPU-hours) | $0.36/RPU-hour | Billed per second during active query processing |
| Redshift Managed Storage | $0.024/GB-month | Same as provisioned RA3 storage |
| Redshift Spectrum queries | $5/TB scanned | Only when querying external S3 tables |
| Cross-AZ data transfer | Standard rates | Keep workgroups in same AZ as data sources |
To control costs, set a usage limit (max RPU-hours per day/week/month). When the limit is reached, Redshift can alert, disable queries, or block new connections - your choice.
Redshift Serverless cold start: if no queries have run recently, the first query after an idle period takes 5-30 seconds to return. This is noticeable for BI tools that show dashboards immediately on page load. Set a base RPU of at least 8 and use QuickSight SPICE to cache results and avoid cold-start latency in user-facing dashboards.
Interview Focus Points
- 1What is the difference between a Redshift Serverless namespace and a workgroup?
- 2Explain RPU pricing - how does it compare to provisioned Redshift node pricing for different usage patterns?
- 3When would you choose Redshift Serverless over provisioned Redshift, and when would you choose provisioned?
- 4What is Redshift Data Sharing and how does it let serverless and provisioned clusters share data without copying?
- 5How would you prevent runaway costs in Redshift Serverless?
- 6Explain Redshift Serverless cold start - what causes it and how do you mitigate it for user-facing dashboards?
- 7How does Redshift Spectrum work with Serverless, and when would you query S3 via Spectrum vs loading data into Redshift?
- 8Compare Redshift Serverless to Athena for ad-hoc analytics - when is each the better choice?