Immutable Infrastructure
Replace-not-patch philosophy, golden AMIs, Packer
Immutable infrastructure means servers and VMs are never modified in place after deployment — instead, a new image is built, tested, and used to replace the running instance. This eliminates configuration drift, simplifies rollback (swap the launch template AMI), and produces predictable, auditable deployments. Packer (HashiCorp) automates the creation of golden AMIs (AWS), VHDs (Azure), and GCE images with all runtime dependencies baked in.
Key Points
- Golden AMIs include the OS baseline, agents (CloudWatch, SSM, security), runtime (JDK, Node.js), and application binaries — no runtime `apt-get install` or configuration management drift.
- Packer builders support AWS, Azure, GCP, VMware, and Docker in a single JSON/HCL template — one source of truth for cross-environment images.
- EC2 Auto Scaling Groups + Launch Templates: updating the AMI ID in the Launch Template and triggering instance refresh replaces all running instances with zero downtime (configurable min healthy %, pause between replacements).
- Immutable infrastructure dramatically simplifies debugging: if a golden AMI works in staging but not production, the delta is environment configuration (secrets, DNS, IAM), not server state.
- Configuration drift in mutable servers causes "works on my machine" production incidents — immutable infrastructure eliminates this class of failure by design.
- CIS-hardened AMIs (via CIS Benchmarks) bake security controls (SSH config, auditd, kernel parameters) into the image — ensuring every instance starts security-compliant.
- Container images are inherently immutable — each image tag is an immutable SHA256 digest; Kubernetes never modifies a running container, it replaces the pod.
- Blue-green deployment with immutable AMIs: maintain two identical Auto Scaling Groups; shift load balancer traffic from blue (old AMI) to green (new AMI) atomically.
Real-World Example
Netflix pioneered immutable AMIs (Baked AMIs) via their Aminator tool, baking all Java service dependencies and configuration at image creation — enabling them to deploy thousands of times per day with predictable rollback via ASG AMI swap.