There’s something to be said for a small, scrappy dev team: you can change directions quickly, and try new tools with a minimum of fuss. Your DevOps practices develop organically because they work well for you. But when your organization starts to grow? The DevOps process that worked for your small team may not work for a larger team or multiple teams.
However, there are many large organizations that do DevOps well, scaling up while maintaining efficiency and security. How do they manage it? More importantly, how can you manage it?
GitHub is here to save the day with its Engineering System Success Playbook (ESSP), a practical guide to improving the processes of engineering organizations, no matter the size of your team.
It can be tempting to go for the quick wins when you’re changing a system, but quick wins can cause problems down the road. Boosting speed in one part of the development process can lead to security problems later, for example.
The ESSP is a strategic framework designed to help engineering organizations achieve steady, sustainable, measurable improvements through systems thinking — seeing a problem as part of an interconnected system.
GitHub created the playbook in response to a common issue: many organizations that had adopted GitHub Copilot were asking for a step-by-step process that would help them build a successful DevOps process and measure its success. Many teams didn’t have a clear example of what scaling DevOps might look like, particularly when they introduce new tools, like AI coding assistants.
The ESSP is a blueprint for scaling success, with steps, suggestions, and metrics to track.
GitHub’s ESSP offers a three-step approach for engineering excellence at scale:
The purpose of this step is to assess your current process and find any barriers that are keeping your team from improving its DevOps process. An honest assessment of your current state and knowing what your goals are can help you find the gaps and obstacles that need to be overcome — it also gives you a baseline against which to measure your progress. By mapping your progress, surveying your team and using dashboards, you can start gathering data and metrics that will help you build out that baseline.
Once you know where you are, it’s time to prioritize the tasks that will get you to your goal. The focus of this step is to pinpoint which changes are aligned with your goals. These changes may be technology changes, cultural, social, or process-related. No matter which changes are necessary, it’s important to understand the risk and cost associated with each, as well as the benefits. It’s also important to communicate with your team, and get buy-in from all your stakeholders. Big changes means everyone should be on the same page.
You can’t just make a huge change and leave it at that. When you’re scaling, you need to keep an eye on your process and be willing to adjust it when something doesn’t work. By using dashboards to track performance and iterating as needed, your team can make progress and avoid going back to the old way of doing things.
The ESSP framework is based on four interconnected areas, or zones:
You might be tempted to think of the zones as four equal quarters (like four pieces of pie) but it’s more accurate to think of them as a table: Business outcomes is the top of the table, supported by Developer happiness, Quality, and Velocity. Improving the metrics of any of the table’s legs will improve your business outcomes.
Within each zone, the ESSP suggests three metrics you can monitor in order to measure and improve your team’s engineering performance. It’s important to note that these are downstream measurements and can take a while to change, so the ESSP also recommends identifying some more immediate metrics (leading metrics) to give you something to work with in the meantime.
Let’s take a closer look at each zone and the metrics for each:
Happy developers are productive developers; when developers are able to do their best work and are satisfied with their jobs, this benefits the whole organization. GitHub recommends tracking the following metrics to gauge developer happiness:
Tracking the happiness of your developers can seem like a squishy metric, but there are several ways to do it. GitHub recommends surveys to collect feedback directly from team members. You might also choose some companion metrics that quantify developer experience and productivity. These might include Time to PR, Commits, and changes in manual coding time by comparing before vs. after Copilot adoption and Copilot vs. non-Copilot users.
Code should be secure, reliable, and easy to maintain. These metrics are recommended to track code quality:
Other metrics that measure quality include error reduction, change failure rate, MTTR, and enhanced code review outcomes. It may help to track these as well in order to get a fuller picture of the changes in your code quality over time.
High quality code should not mean delays. Code should be shipped regularly, and at a pace that helps your organization meet business needs.
Because these are downstream metrics, it may be helpful to track other metrics that give a short term picture of your team’s speed. Additional companion metrics may include Throughput, Time to PR, and Cycle Time.
The code that’s shipped should be helping your business innovate and meet its business goals. GitHub recommends the following metrics in this zone:
To track impact on your business goals, you might also want to track companion metrics like time savings, efficiency, and risk reduction.
Identifying metrics is one thing, but tracking them can be a challenge, especially if you’re working with multiple teams and several tools. Collecting and tracking metrics means pulling data from every part of the DevOps process and comparing it. ESSP works best if you can see every recommended metric on one screen.
Opsera Unified Insights let you do that in a single dashboard. Opsera connects all your DevOps tools across the entire software delivery process, giving you a complete view of your software delivery process, allowing you to analyze key metrics such as lead time, velocity, Copilot KPIs, commit statistics, PR requests, deployment frequency, MTTR, and change failure rate.
Unified Insights allows your team to make baseline comparisons with data from up to a year in the past, letting you easily map improvements and see where adjustments need to be made to your process as you scale.
It’s exciting when your organization is growing. Opsera’s dashboards give you access to tailored insights for every stage of the software delivery lifecycle, so you have a central view of all your releases as your teams grow and your DevOps process scales. Unified Insights integrates with more than 150 tools like Jira, Jenkins, Selenium, and Docker, so that you can connect your entire DevOps ecosystem as it grows and changes.
Ready to see it in action? Contact us for a demo today.
