Keeping EC2 instances patched and up-to-date is a crucial security practice. However, applying patches to instances within an Auto Scaling Group (ASG) presents unique challenges. Directly patching a running instance can interfere with the ASG’s health checks, potentially causing the ASG to terminate the instance and replace it with an unpatched one. Therefore, automated and controlled approaches are necessary.

The Challenge with Patching ASGs

Auto Scaling Groups automatically adjust the number of instances based on demand or schedules, ensuring application availability and scalability. Directly applying patches (like using AWS-RunPatchBaseline via Run Command) to instances managed by an ASG is often unsuitable for production environments because:

  • Instances might be terminated during patching if they become unhealthy or if scaling actions occur.
  • Patching instances individually leads to configuration drift, where new instances launched by the ASG won’t have the latest patches.

Automated Solutions for Patching ASGs

AWS provides several services and methods to automate the patching process for ASGs safely and effectively:

  1. AWS Systems Manager Automation (Patching Existing Instances Safely):
    • Concept: Use Systems Manager Automation runbooks to coordinate the patching process.
    • Example Runbook (AWS-PatchAsgInstance): This AWS-provided runbook automates patching individual instances within an ASG sequentially. It typically involves:
      • Putting an instance into a Standby state (removing it from load balancing).
      • Applying patches using AWS-RunPatchBaseline.
      • Bringing the instance back into service.
      • Repeating for other instances, often with concurrency controls.
    • Scheduling: This can be scheduled using Systems Manager Maintenance Windows.
    • Benefit: Patches existing instances with minimal disruption if configured correctly.
  2. EC2 Image Builder / Systems Manager Automation (Golden AMI Approach):
    • Concept: Create a new, fully patched Amazon Machine Image (AMI) and update the ASG’s launch configuration or launch template to use this new AMI. Then, gradually replace old instances with new ones based on the patched AMI.
    • Workflow:
      • Use EC2 Image Builder or a Systems Manager Automation document (like AWS-UpdateLinuxAmi or AWS-UpdateWindowsAmi) to:
        • Launch a temporary instance from the current AMI.
        • Apply patches based on a defined patch baseline.
        • Create a new, patched AMI from the temporary instance.
      • Update the ASG’s Launch Template or Launch Configuration to reference the new AMI ID.
      • Use ASG Instance Refresh to perform a rolling replacement of instances, ensuring the group maintains minimum capacity.
    • Benefit: Ensures all new instances launched are fully patched from the start; promotes immutable infrastructure principles.
  3. ASG Instance Refresh with Updated Launch Template:
    • Concept: Manually or programmatically create a new version of your Launch Template with an updated (patched) AMI ID. Then, trigger an Instance Refresh on the Auto Scaling group.
    • Process: The ASG gradually replaces existing instances with new ones launched from the updated Launch Template version, respecting availability requirements.
    • Benefit: A straightforward way to roll out updates based on pre-patched AMIs.

Choosing the Right Approach

  • The Golden AMI approach (using EC2 Image Builder or specific SSM Automation runbooks) is generally considered a best practice for promoting immutable infrastructure and ensuring consistency.
  • Using AWS-PatchAsgInstance via Maintenance Windows can be effective for regularly patching existing instances without recreating AMIs, but requires careful configuration to manage instance states (Standby).

Automating the patching process for Auto Scaling Groups using AWS services like Systems Manager and EC2 Image Builder enhances security posture, ensures consistency, improves operational efficiency, and allows teams to focus on value-added activities rather than manual patching.