My other blog (in Hungarian) merhetetlen.blogspot.com

Thursday 25 August 2011

Quality measurement part 1

My Colonel asked my Regimental Commander who asked my Master Chief to find out how to measure the Quality of a Software. I know it is not unique, and there are hundred of pages and calculations out there in the Universe, but my mind collapsed when it tried to find out how.

I went through all phases of protesting: laughing, crying, bargaining...
We had to create a machine which eats numbers and splits a number, a beautiful, magical number which is comparable, descriptive, short and tells everything about quality.

In one of my stations I tried to make fun of it, to show its incongruity. I created metrics like:

  • # of [public methods | lines of code | methods ] / years of experience of the team
It is definitely about quality, shows the years of experience index of each method... (of course with the presupposition experience is related to the quality)

But at the end we have to find out something else what we can show to the Colonel. The problem is that we can measure billions of things from the architecture to the number of open tickets, or the number of exceptions in the logs or Customer Support calls, or what we can imagine. But we would not reach the Quality, only its part, its few aspects. I could imagine a code which has low complexity, well-commented, but poorly designed, or has hundreds open tickets in its issue tracking system, or its UI has bad user experience, and so on...
What I said to my superiors, we cannot produce a number what our Colonel wants, because it is impossible. We have to identify the measurable factors of quality, gather the numbers, create a number from these numbers, and define where is this number comes from, what it means (definitely not the Quality).

We've found these main factors:
  • Results of static and dynamic code analysis
  • Bugs and feature requests in the tracking system
(hopefully the open tickets could cover the User experience area and the fit of purpose, and anything else what we cannot get from a tool)

We must tell from the very beginning that all project has its own number, and we cannot compare or confront these numbers between projects. And this number itself still does not mean anything only its change in time.

And when we reached this point, I found it interesting: it is feasible, it shows something, it is understandable (mostly, and abstract enough) for the business, and acceptable for the engineering.

Great.

What do you think?

Go to the details in the 2nd part..

No comments:

Post a Comment