Productivity Metrics

Question:

What do you think are key productivity metrics for an infrastructure operations group? What according to you are key productivity metrics for running an infrastructure operations group.

Answer:

That is a tough question. I would start with definition of productivity since it is not a generic metric like uptime measurement…

Productivity is a simple measurement of input vs. output. There are several mathematical models but I would recommend staying simple.

The inputs are usual suspects; they are the resources you have: time, people, and money… You may turn each input into another but I would recommend staying with three.

In productivity metrics, my approach is to compare the delta in output for a fixed input. That is why it is slightly different than regular metrics such as plain uptime, or MTTRs.

I like the COBIT classification for the metrics:

Quality Principles: Cost, quality and delivery fulfillment.

Fiduciary Principles: Effectiveness and efficiency of operations, reliability of information, regulatory compliance.

Security requirements: Confidentiality, Integrity, and availability

But it is easy to classify in different ways, the idea is to measure productivity metrics instead of raw metrics (build a baseline for an input and start comparing a baseline of metrics and get an idea on the productivity for certain input)

For the key metrics representation I would go with %#$” (percentage, number, dollar and time,)

The outputs at infrastructure operations to build comparative productivity metrics can be (but not limited to):

Per Role Outputs:

# Last year Level 1 Engineer was closing 8 priority-1 tickets a day this year 20

# Last quarter Level III engineers were completing 2 projects/month, this quarter 1

% Percentage of positive feedbacks per role

Time Based Outputs: Our “Mean Engineering Fix Hours” time was 2 hours now it is 90 minutes..

Time: MTTR/MTBFs baselines

Time: Unplanned downtime baselines

Time: Cycle time provisioning a new infrastructure component was 1 week now it is 3 days

Money based:

$ Per ticket cost was $100 now it is $20

% Percentage of infrastructure costs charged back to business was 50% now 80%

$ Cost of running my team was $x now $y

$ Unplanned downtime impact in $ terms was $x now $y

Quality Based

% Percentage of planned/on time completed change requests – over time/cost

% Percentage of systems compliant with policy requirements – over time/cost

% Percentage of systems with the required OS/patch levels – over time/cost

e.g. Last month our team had 3000 hours 90% of changes were within planned range.

It is easy to deploy custom metrics based on your environment as long as you stay with the productivity focus. You can also build your metrics from the frameworks you are following (PCI, FFIEC, COBIT etc)

Also in reporting you need to explain surges, drops and trend changes that effect productivity metrics.

Yes, it is not an exact science but as it is said “you cannot improve what you cannot measure” . I also recommend Andrew Jaquith’s “Security Metrics” book even if it is security focused.

regards,

- yinal ozkan