Metrics for Everyone

This year we heard from a lot of folks that have started down their network automation journey. They are seeing initial success, but are finding it difficult to get buy-in from other teams, management, or both. Although frustrating, this experience is not unique to network automation.

DevOps teams have largely experienced the same issue, and looking back to the beginning of DevOps there was a framework put together by John Willis and Damon Edwards in 2010 known as CAMS. This framework set out to define the key pillars of DevOps:

• Culture
• Automation
• Measurement
• Sharing

Using this framework we can break down the buy-in problem (and finally get to the point of the blog, which is metrics):

You or your team have built some piece of automation, and are rightfully excited about it. The culture of your organization is however resistant to change. There are a variety of reasons for this resistance: long history of doing things manually, the expense of building and learning new things, poor results from similar efforts in the past.

Know your audience

Measuring the impact of your automation provides tangible data to combat resistance. It is important however to have the right data for your audience. There are layers to any organization, and each layer has drivers that are relevant to their goals.

For the purpose of this blog we will define three layers of the organization and show examples of metrics that may allay the concerns of that audience – know your audience.

Individual Contributors (Engineers)

This may be network engineers, operations, architects, etc. These are the people most directly related to doing the work. Their primary concerns tend to be:

• Level of effort to learn something new
• Time available to do something different
• Reliability and performance of solutions

The metrics we want to show here revolve around the state of the infrastructure and the health of the overall solution. What systems are being affected? What failures are we seeing? How is the performance of the automation?

Direct Management

This level of the organization sits in the middle, balancing the drivers and concerns of the others

• Capacity and expenditure of the teams resources
• Well being of the team
• Ensuring business objectives are articulated bi-bidirectionally and achieved

At this middle level we want to show the impact of automation on the team, the adoption of automation by other teams, and the impact on the business.

Senior Leadership

The focus at the top of the organization is big picture – the overarching goals of the company

• Overall strategy
• Financial implications
• Long term growth and stability