Introduction
By leveraging AWS ECS Capacity Providers, you can automate ECS cluster scaling based on task launch requests.
This article shares a real-world example of how we fully automated the scaling of batch processing running on "ECS on EC2" using this mechanism.
Background
We had a system running containers on AWS ECS on EC2. As an additional development requirement, we needed to "run large-scale batch processing for just a few days each month." This prompted us to reconsider our scaling design.
Existing Configuration and Challenges
Traditionally, we used the "CPU utilization" of EC2 Auto Scaling Group (ASG) as the scaling trigger.
Initially, we planned to follow this same scaling approach, but as we examined it more closely, several issues emerged:
- EC2 instances don't auto-scale at batch start; manual EC2 scale-out is required beforehand
- After batch completion, EC2 instances don't auto-scale down; unnecessary instances remain
- Scaling out is delayed by several minutes while waiting for CPU utilization to rise
When task launch requests spiked suddenly, the EC2 instances wouldn't increase until CPU utilization rose, causing task launch failures.
"We want EC2 to automatically increase and decrease in response to task increases"
Capacity Providers seemed like the mechanism that could fulfill this ideal.
What are Capacity Providers?
A mechanism that automatically adjusts the number of EC2 instances based on ECS task launch requests.
The key points are:
- ECS monitors "task launch requests"
- If ECS detects insufficient capacity for task launch requests, it automatically scales out EC2
- When tasks decrease and required capacity also decreases, it automatically scales in EC2
This eliminates the delay of "waiting for CPU utilization to rise" and enables real-time synchronization between ECS demand and EC2 supply.
For more detailed information, the AWS official blog is a helpful reference.
Configuration in Your Environment
Configuration is available from the "Infrastructure" tab in the ECS console.
The key setting is "Target Capacity (%)" which specifies the target resource utilization rate of EC2. For example, if you set it to 90%, scaling is controlled to maintain "90% EC2 resource utilization." For our batch processing, we wanted to use resources without waste, so we set it to 100%.
Warning: Setting Target Capacity to 100% will cause EC2 to run at full capacity.
Further Cost Savings with Binpack Task Placement Strategy
In addition to automatic scaling via Capacity Providers, we changed the task placement strategy to binpack. Binpack is a strategy that consolidates tasks based on the minimum available CPU or memory resources.
Info: By using the binpack strategy, you can efficiently consolidate tasks with fewer EC2 instances.
As a result, we achieved the following:
- Maximum utilization of EC2 resources
- Stable automatic scale-in after batch completion
- Cost reduction compared to the previous approach
Personal Best Practices Summary
| Configuration Item | Recommended Value |
|---|---|
| Target Capacity | 100% |
| Task Placement Strategy | binpack |
| ASG Minimum Instances | 0 |
Conclusion
- By implementing Capacity Providers, you can synchronize ECS task counts with EC2 scaling
- Automatic scaling for batch processing, which was difficult with CPU-based ASG scaling, is now achievable
- Combined with binpack strategy, further cost optimization is possible
Info: Even with ECS on EC2 architecture, implementing Capacity Providers enables operational automation comparable to Fargate.
Additional Notes
Warning: Properly size ECS task CPU/memory. Preventing resource insufficiency and over-allocation improves scaling accuracy. Spot instances and mixed instance types are also possible, which can lead to better cost optimization and improved redundancy.
