This post will be about monitoring project quality on various levels. This post is an introduction to the subject of continuous quality monitoring. I next will give some technical details how to monitor this quality. In next posts I’ll write about my idea of development process and the place of quality management in it, so if you’re interested check out this blog in a few days. Please check posts given in references at the and of article. They discuss important aspects of quality management that will give you yet another perspective. I would like to focus more on technical details of monitoring.
We’ve seen some projects that had high quality, some with low quality. Most projects start with high quality expectations and promises. Some have standards setup by architect or applied as part of corporate policy. Most project have some sort of automated tests. Sadly many projects lack proper monitoring and continuous quality monitoring. I guess most of us experiences cutting down the effort spend on unit or functional testing. Architecture and quality metrics are measured even less often.
Let’s set common definition of quality from the perspective of quality assurance:
- internal quality – these is measured on code level. Here we measure quality metrics like distance from main sequence, number of cyclic relations, afferent coupling, efferent coupling, instability etc. We can also check if code is made according to some standards chosen for this project. Business people does not see this quality, but never the less in the end is affected by it.
- external quality – here we measure if software does what it is meant to do and has proper functional and non-functional characteristics. Business people see this type of quality and this gets more attention. This type of quality is about functional testing, running various security checks, performance, load and stress testing the application. We can measure some architecture level quality metrics like security, reliability, performance.
For majority of projects we cannot completely separate the two because they have the same source. If code has poor quality then probably it will not be scalable, secure or reliable. It will have a lot of bugs and will be difficult to maintain. And most projects start with high expectations and good will. So what goes wrong? It is hard, if possible, to give definitive answer but from my experience I can pinpoint a few reasons:
- Lack of common quality vision and commitment to this vision – management, business and engineers must have a common understanding of quality, consequences of low quality, and level of quality they want to achieve. Showing examples of failed projects here will help to get a common understanding. On the other hand engineers have to remember that software is not an art work made for the sake of making software.
- Lack of continuous inspection – continuous integration with continuous inspection is a must in todays projects. We simply don’t have time and money to discover failures later in the process of making software.
- Lack of proper quality assurance – this is an interesting point. A common example is writing tests to achieve x% coverage, which is simply nothing more than throwing away money. Show me business that wants to throw away money 😉 Another example, mentioned by Petri are incorrectly introduced code reviews.
- Lack of proper architecture and monitoring of implementation of these architecture – requirements will change, especially if the product is successful, so it must be prepared to be extended and modified without harm to its quality.
What can we do ? We can focus on our quality goal and measure it. I won’t discuss how we can setup quality goal, as I believe this depends on organization, its standards and culture. I want to focus on technical aspects of measurements.
References to articles:
- Automation for the people – excellent series about quality and automation.
- Quality Manament of Software Projects, Part 1: The Conflict – first of three posts by Petri Kullanen
- Object oriented quality metrics – description of basics code metrics (PDF)