IaC: Reigning the Deployment Pipeline

Recently for a customer whose site experienced explosive growth over a small period of time , ie 10x times the traffic, producing quality consistently and aggressively developing new features was a challenge. This means that it’s imperative to quickly ramp up or down on development in response to the needs and create production like environment on the go.

Also challenging was the fact that the dev teams could no longer reliably test without mirroring the entire production stack on local and staging servers,

To capture , these succinctly , the dev teams face such choices during the initial development, staging, and deployment phases. For example, a project begins small and grows to large footprint with many instances , proxies, load balancers, dev, staging and prod environments needs to be provisioned quickly and multi fold.

A new practice in modern infrastructure complementing devops paradigm is taking shape in the form of Infrastructure as a Code or IaC.

In simple terms, IaC is a framework that take coding techniques and extends them to the infrastructure directly, blurring the line between an application and the environment. In a sense, this is the same thing DevOps is doing with the developers and Operations , integrating them into a single entity.

First Steps for deployment pipeline :

1. Find what projects to start with whom, when.
2. Identify as-is, to-be, gaps, the roadmap for the project form a dev team and go.
3. Integrate Ops members in the Dev team to gain outcomes to manage the DevOps team
4. Try to work differently ie establishing the cadence with the DevOps team

What should be the Flow:

1. Dev teams to use production-like dev/test environment with source control
2. Dev teams to run the entire dev/test/prod cycle daily with automated build deployment
3. Define a cadence for daily commit
4. Identify the pull request, put approval process in place
5. build , test and deploy the code in production on-demand

Creating the foundation of deployment pipeline

1. Use production-like environments at every stage of the value stream.
2. Automate , automate and automate
3. Use configuration information stored in version control , no manual work should be required from Operations
4. Recreate the entire production environment based on what’s in source control scripts every time.

The ‘How’ of deployment pipeline : The best approach is a combination of a Dev infrastructure tool , a cloud infrastructure tool and a CM tool.

1. Build a VM image of production stack. It could be running a Ansible Script(hardware provisioning tool) or booting a Amazon file in EC2
2. Post creating / customizing an image for deployment from ‘bare metal[ , boot the target environment with WinPE , ISO Or PXE
3. Use “infrastructure as code” CM tools (e.g. Puppet, Chef, Ansible, etc.) for various versions.
4. Recent practice is to assemble an environment from a set of virtual images or containers (e.g., Docker)
5. Public Cloud ie AWS (Cloud Formation), MS Azure (ARM) provides native tools for creating IaC. Use these native tools or general tools like Terraform

Conclusion : IaC make pathway for a dramatic reduction in developer on boarding time and more reliable testing and Dev environment. It phase out manual configuration and handle everything via tools. This mean that a Prod change for instance, would occur on the Dev local machine , be committed to the project codebase and code reviewed/tested against staging, and finally, be deployed to production via tool.

On a side Note : Containers — notably Docker is gaining ground, the Docker may makes redundant the need for CM tools. The CM problem is taken care by a simple Dockerfile and can help creating / deploying on distributed application or set of services spanning multiple machines quickly and efficiently.