Getting the Most out of Amazon EC2 Autoscale Groups with F5 BIG-IP

This article was originally posted on F5 DevCentral.

An Introduction to Amazon EC2 Auto Scaling

One of the core tenets of cloud computing is elasticity, or the ability to scale cloud resources as needed to meet organizational or user needs. Amazon EC2 Auto Scaling is a feature within AWS focused on facilitating the provisioning and deprovisioning of elastic cloud computing resources to meet user demand. This article will cover how F5 BIG-IP can embrace elasticity and leverage the various features of Auto Scaling to manage steps of the F5 BIG-IP lifecycle, such as provisioning, licensing, configuration management, and upgrades.

The Architecture

In a typical F5 BIG-IP architect leveraging Amazon EC2 Auto Scaling, there are two key components. The first key component is the Auto Scaling group itself that contains one to many F5 BIG-IP VE instances as part of a managed group. As the name suggests, the power of an Auto Scaling group is its ability to introduce and remove F5 BIG-IP VE instances in the group based on a specified number of required instances or configured monitoring threshold reflecting user demand.

The second key component is the AWS NLB or Network Load Balancer. While limited in its features compared to an F5 BIG-IP, the NLB plays a critical role in distributing traffic evenly across the F5 BIG-IP VEs within the Auto Scaling group. The NLB also tracks F5 BIG-IP instances added and removed from the Auto Scaling group and load balances only to active instances within the group.

Brought together, an Auto Scaling group containing F5 BIG-IPs and NLB brings the benefits of elasticity and scaling of F5 BIG-IP functionality by provisioning and deprovisioning BIG-IP instances to match organizational capacity needs. Additionally, this architecture has the potential benefit of simplifying maintenance of F5 BIG-IPs by enabling engineers to remove instances from service to perform tasks such as upgrades.

While this architecture brings many new benefits to scaling and maintenance, it also introduces new unique challenges:

• How do you ensure a consistent configuration is applied when a new F5 BIG-IP is provisioned?
• If you are using BYOL licensing with BIG-IQ, how do you ensure licenses are being applied during provisioning and revoked during termination?
• How do you ensure the F5 BIG-IP you just provisioned is tested and fully functioning before going into service?

These are all challenges where Lifecycle Hooks become our secret weapon.

Meet Lifecycle Hooks

A powerful feature contained within Amazon EC2 Auto Scaling is Lifecycle Hooks. Lifecycle Hooks are event-driven triggers executed as instances are added or removed from an Auto Scaling group. Lifecycle Hooks enable the ability to put EC2 instances in a wait state at various steps of the instance’s lifecycle and execute external actions such as an AWS Lambda function.

The power of Lifecycle Hooks in the context of F5 BIG-IP is it enables the ability to execute external AWS Lambda-driven code to perform management tasks such as applying a list of AS3 declarations from an S3 bucket at the time of provisioning or revoking a BYOL license at the time of termination. This feature simplifies the management of F5 BIG-IP Auto Scaling groups by ensuring any newly provisioned F5 BIG-IPs have the needed configurations to match other instances in the group. Additionally, this feature provides the benefit of immediately revoking F5 BIG-IP BYOL licenses when an instance is being deprovisioned, ensuring an organization is maximizing its F5 spend.

In addition to F5 BIG-IP lifecycle management tasks, Lifecycle Hooks can perform initial testing of F5 BIG-IP instances before being placed inside the Auto Scaling group to accept new user traffic. These tests can include use cases such as ensuring a group of VIPs is correctly processing traffic or a WAF policy is blocking a known attack. If the F5 BIG-IP fails the test, the Lifecycle Hook can terminate the failing instance and spin up a new one until the test criteria are met. This workflow automatically reduces the risk of a failing F5 BIG-IP instance receiving user traffic and ensuring a standard level of quality control for F5 BIG-IP instances entering the Auto Scaling group.

The power of Lifecycle Hooks combined with the flexibility AWS Lambda provides for an almost near endless number of possibilities as part of the F5 BIG-IP provisioning and deprovisioning lifecycle. Lifecycle hooks empower cloud engineers and F5 administrators to programmatically trigger event-driven code to perform repetitive management and testing tasks common for scaling F5 BIG-IP deployments. An example of an AWS Lambda function used to perform the lifecycle tasks mentioned above can be found here.

Rolling Updates with Instance Refresh

In addition to Lifecycle Hooks, Amazon EC2 Auto Scaling provides a feature to simplify the ability to upgrade F5 BIG-IPs called Instance Refresh. Instance Refresh enables the ability to incrementally replace one version of an F5 BIG-IP’s machine image with another version of the image in the form of a rolling deployment.

Integrated with Lifecycle Hooks, Instance Refresh can upgrade, configure, and replace instances running an older version of BIG-IP with a new version. Additionally, Instance Refresh by default integrates with NLB to gradually drain connections of the old F5 BIG-IP images before removal, making the service impact of performing an upgrade little to nonexistent. The combined benefit of Instance Refresh with Lifecycle Hooks is an automated upgrade process with the potential of minimal user impact.

Speedier Scaling with Warm Pools

The final helpful feature of Amazon EC2 Auto Scaling when managing F5 BIG-IPs is warm pools. A warm pool is a newly added feature within Auto Scaling that allows for the creation of pre-initialized F5 BIG-IPs that live in a stopped state until the additional capacity is needed.

The benefit of using a warm pool in an F5 BIG-IP Auto Scale group is that it enables the ability to perform many of the time-consuming tasks performed when initializing a virtual appliance ahead of when the BIG-IP is needed. These tasks include licensing, module provisioning, and other onboarding tasks. When tested, the use of a warm pool on average nearly halved the amount of time to add a new BIG-IP into service. The cost of maintaining a warm pool is also relatively small compared to paying for an overprovisioned Auto Scaling group, as AWS does not charge for compute for stopped instances and only charges for storage. When examining the architectural considerations for using Auto Scaling F5 BIG-IP instance, warm pools enable the ability to quickly add new F5 BIG-IP instances in scenarios where user demand may be unpredictable.

Conclusion and Next Steps

In this article, we have covered the many ways Amazon EC2 Auto Scaling can be leveraged to improve the management of F5 BIG-IP instances inside AWS. For further information and a full Terraform example of how the concepts detailed in this article can be implemented, check out this GitHub repository: https://github.com/tylerhatton/f5-warm-pool-demo