PM Articles > Alan S. Koch > How Much Quality Can We Afford?

How Much Quality Can We Afford?

Sure, it might be nice to build a higher quality product. But how much quality can we really afford? Well, let's break out our Cost of Quality Calculator and try out the numbers.

What is Our Cost of Quality Today?

First the basics: Let's say we're going to write a system of 25,000 Lines of Code, and our data tells us that we inject about 50 defects per 1000 lines of code. So along with those 25 KLOC, we will also write 1250 defects.

  Defect Injection Defect Removal Defects in Product Cost of Quality
Activity Size Rate # Yield # # /KLOC Rate Hours
Design & Code 25 KLOC 50 1250     1250 50.00    
Totals 1250 0 1250 50.00 0

For a system of that size, we usually end up with a 20-page requirements document and a 40-page high-level design specification. And since we usually inject 5 defects per page in our documents, that gives us 100 defects in the requirements and 200 defects in the design bringing our total to 1550 defects.

  Defect Injection Defect Removal Defects in Product Cost of Quality
Activity Size Rate # Yield # # /KLOC Rate Hours
Write Requirements 20 Pgs 5 100 100 4.00
High Level Design 40 Pgs 5 200 300 12.00
Design & Code 25 KLOC 50 1250 1550 62.00
Totals 1550 0 1550 62.00 0

Of course, we always review the requirements. We review about 10 pages of requirements per hour, finding 40% of the defects in the document. We incur 2 hours of Cost of Quality to find and fix 40 defects.

  Defect Injection Defect Removal Defects in Product Cost of Quality
Activity Size Rate # Yield # # /KLOC Rate Hours
Write Requirements 20 Pgs 5 100 100 4.00
Review Requirements 40% 40 60 2.40 10 2
High Level Design 40 Pgs 5 200 260 10.40
Design & Code 25 KLOC 50 1250 1510 60.40
Totals 1550 40 1510 60.40 2

And of course, we test. Our data tells us that we find 50% of the existing defects in Unit test and that we can do it at the rate of 5 defects found and fixed per hour. We find 40% of the existing defects in Integration (at a rate of 2 hours per defect found & fixed, or 0.5 defects per hour), and 30% in System test (at a rate of 10 hours per defect found and fixed, or 0.1 defects per hour). Unfortunately, we also inject about 2.5 new defects for every 100 we fix.

  Defect Injection Defect Removal Defects in Product Cost of Quality
Activity Size Rate # Yield # # /KLOC Rate Hours
Write Requirements 20 Pgs 5 100 100 4.00
Review Requirements 40% 40 60 2.40 10 2
High Level Design 40 Pgs 5 200 260 10.40
Design & Code 25 KLOC 50 1250 1510 60.40
Unit Test 2.5% 19 50% 755 774 30.96 5 151
Integration Test 2.5% 8 40% 310 472 18.88 0.5 619
System Test 2.5% 4 30% 142 334 13.36 0.1 1416
Totals 1580 1246 334 13.36 2188

So this spreadsheet represents our normal 25KLOC software project. Over a person-year of effort spent in Cost of Quality work, and hundreds of defects delivered. More than 13 defects per 1000 lines of code is pretty bad.

How much will it cost us to improve it?

What if we were to do a more formal inspection of the requirements? Let's say we could only inspect about 2 pages per hour, but would find 65% of the existing defects.

  Defect Injection Defect Removal Defects in Product Cost of Quality
Activity Size Rate # Yield # # /KLOC Rate Hours
Write Requirements 20 Pgs 5 100 100 4.00
Inspect Requirements 65% 65 35 1.40 2 10
High Level Design 40 Pgs 5 200 235 9.40
Design & Code 25 KLOC 50 1250 1485 59.40
Unit Test 2.5% 19 50% 743 761 30.44 5 149
Integration Test 2.5% 8 40% 304 464 18.57 0.5 609
System Test 2.5% 3 30% 139 328 13.14 0.1 1393
Totals 1580 1251 328 13.14 2160

We found more defects, but look! Our cost of Quality went down! Why did that happen? Because those defects we removed early in the inspection didn't get found later in testing when they are more expensive. Cool! Let's try inspecting our design document, too!

  Defect Injection Defect Removal Defects in Product Cost of Quality
Activity Size Rate # Yield # # /KLOC Rate Hours
Write Requirements 20 Pgs 5 100 100 4.00
Inspect Requirements 65% 65 35 1.40 2 10
High Level Design 40 Pgs 5 200 235 9.40
HL Design Inspection 65% 153 82 3.29 2 20
Design & Code 25 KLOC 50 1250 1332 53.29
Unit Test 2.5% 17 50% 666 683 27.31 5 133
Integration Test 2.5% 7 40% 273 416 16.66 .05 546
System Test 2.5% 3 30% 125 295 11.79 0.1 1249
Totals 1577 1282 295 11.79 1959

Yes! Same again! Fewer defects and less time spent in Cost of Quality! Our coding is our biggest source of defects. What if we inspect the code as well? Let's plan for the same 65% yield and say we can inspect code at 100 LOC per hour. (That's gonna be a lot of hours, but here goes!)

  Defect Injection Defect Removal Defects in Product Cost of Quality
Activity Size Rate # Yield # # /KLOC Rate Hours
Write Requirements 20 Pgs 5 100 100 4.00
Inspect Requirements 65% 65 35 1.40 2 10
High Level Design 40 Pgs 5 200 235 9.40
HL Design Inspection 65% 153 82 3.29 2 20
Design & Code 25 KLOC 50 1250 1332 53.29
Design & Code Inspections 65% 866 466 18.65 100 250
Unit Test 2.5% 6 50% 233 239 9.56 5 47
Integration Test 2.5% 2 40% 96 146 5.83 0.5 191
System Test 2.5% 1 30% 44 103 4.13 0.1 437
Totals 1559 1456 103 4.13 955

Jackpot! We improved our defect level by almost a factor of three! And we cut our Cost of Quality by more than half! We have achieved a respectable defect level of less than 5 defects per 1000 lines of code while investing less than half of the quality effort we started with.

So, What Do The Numbers Mean?

Of course all of this is theoretical and based on some basic averages that many organizations have reported. This exercise will be most interesting when you plug in your own organization's numbers.

What? You don't have these numbers? Better start measuring your Cost of Quality today!

P.S. If you want to use my spreadsheet instead of building your own, just download it here or from my website at www.ASKProcess.com/resources/CostOfQuality.xls.



Related Resources
Our Requirements Walkthrough Checklist can help you organize a productive inspection of requirements. Make sure everyone knows what they should look for when reviewing requirements.



Comments
Not all comments are posted. Posted comments are subject to editing for clarity and length.

Nicely Done! I fully agree with the ROI of conducting Requirements and Design Formal Inspections (or whatever kind of inspection works best in your enviroment). I've had some luck with modeling this kind of early defect removal performance and generally see a gain of 300% for these early defect removal efforts. In other words, roughly, for every solid defect you remove early (Requiements or Design) you'll reduce the count of solid defects you find later (in formal test) by 3 defects. For more on this visit my website. http://peerreviewprocessroi.com/


Post a comment




(Not displayed with comment.)









©Copyright 2000-2017 Emprend, Inc. All Rights Reserved.
About us   Site Map   View current sponsorship opportunities (PDF)
Contact us for more information or e-mail info@projectconnections.com
Terms of Service and Privacy Policy

Stay Connected
Get our latest content delivered to your inbox, every other week. New case studies, articles, templates, online courses, and more. Check out our Newsletter Archive for past issues. Sign Up Now

Got a Question?
Drop us an email or call us toll free:
888-722-5235
7am-5pm Pacific
Monday - Friday
We'd love to talk to you.

Learn more about ProjectConnections and who writes our content. Want to learn more? Compare our membership levels.