How to enable DevOps for High Performance
DevOps accelerates Dev and Ops flow and shorten but enhances the feedback loops with in the Dev cycle. This, along with Agile incremental and small-loop approaches has led to ‘continuous’ activities or processes and what is called ‘functions’ throughout the software delivery life cycle or management. The following could harness the key function or practices that helps high performance delivery.
Fail Fast : This is to adapt quickly and produce better results. As with The feedback loops is essential for what has not worked as intended . The concept of Minimum Viable Product (MVP) helps teams to create increment product for customer feedback or market trails and possible introduce to wider market or withdraw a product.
Limited Blast Radius: This approaches is a few example practices for failing early and fast so we can adapt quickly during Dev process. The goals is to ensure that any unforeseen problems are isolated asap in deployment process so that the customers are not impacted and to simplify problem determination and ensuring correction.
Fail Early : This is to reduce time and cost to fix. DevOps believe that it is cheaper to fix a defect the earlier it is found in the software delivery cycle (fail fast, fail early — also Shift Left). This means the quicker they can discover ways to improve upon systems and products
More on Shift Left with Containers in this post
Chaos Engineering : This allows experimentation to build confidence with in the teams . It is described as the discipline of experimenting in”production system” to build robustness in the system’s capability to withstand high load and unexpected conditions. Good example is Ecom sites during Black Friday and CyberMonday wherein multitude of load could have catastrophic effect.
SRE DevOps: The above is closely related to SRE (Site Reliability Engineering (Google)). It provides deep, code-level insight about every error and slowdown and helps teams prioritise them to eliminate Sev1 issues in Prod. It leads teams to define prescriptive ways for measuring availability, uptime, outages, failures, etc.SRE embodies
- Stack and bytecode
- JVM State: Threads, Env variables
- log statements (including DEBUG and TRACE in Prod)
- Event analytics (Frequency, failure rate, deployment, Apps)
- Source Code Management
Andon Cord : This is used to stop and improve immediately. The concept came out of the lean principle in Japan and means flagging a defect in a manufacturing pipeline. It is frequently used as an analogy for good practices in DevOps. Since anyone can pull the cord to alert to a issue the first thing that happens is that it is an opportunity to make an improvement.
Change Failure: This is combined with Limited Blast Radius to mitigate the impact. A example is a Canary test or deployment, the change is made available to only a small set of users initially until it is seen to be working as designed. Then it is rolled out to the rest of the user population. Again example from an ECom site makes perfect sense for a limited set of users ie in some countries or regions where the impact could be low for a change.
One of the development practice for limiting the radius is in the following post
