As a tester, are you ever asked by a stakeholder to describe what tests you have and how they cover the application? Here are some rules-of-thumb to think about before you get the questions.
Find out what matters: Start by asking questions of the stakeholders. Listen, listen, listen to their answers.
Report on coverage in terms of what matters: The CIO may not care about unit-level coverage metrics (lines of code, classes, etc...) She probably will care about whether key business processes supported by the system have been tested. Your report to this audience should probably orient coverage towards business processes.
There may be different view of what matters: I mentioned things the CIO may care about, but the network security organization will probably care greatly about access control and other types of security testing. Can your coverage reporting mechanism accommodate different views of coverage?
What matters changes: The things that are important at the outset of a development effort may not stay important. Things that are unknown at the outset may wind up being very important. How easy is it to modify information about what tests cover?
Talk about what matters at the highest (useful) level: If we're doing things correctly, we're probably tracking coverage information at very fine-grained levels. There is a temptation to show all the details up front. Resist this....not because you don't want to share the details, but because details can confuse the bigger picture. A picture like this should not start discussions about coverage.
This is a better choice:
With that said, you want to be able to drill down easily into greater detail.
Future posts will dive into the nitty-gritty of following these rules-of-thumb.
