Sorry for hijacking your thread, Hexorg. To be honest I haven't looked at your app, but I'm glad to see that you're learning by doing -- that's the only effective way to learn.
DJSPIN80 wrote:
Honestly, I don't understand why Unit Testing was never really part of my workflow. Being able to test code coverage is a wonderful thing. Especially if you work in a statically typed language like C# or Java (being used for Android) or eve objective-c is a must. All you have to do is create mocks (or fakes) that mimic those objects that you need to test and test it. By verifying that your code does exactly what it's doing plus being able to test for edge cases like null values. It's not imperative yuo reach 100% code coverage but if you're testing with code coverage in mind, then you can verify that your code behaves as it should.
Reading this scared me so much I logged in to post a response! Code coverage is
not a good test suite quality metric. In fact, it's an outright dangerous one.
The statement that "if you're testing with code coverage in mind, then you can verify that your code behaves as it should" cannot be more wrong, and I'm really frightened to see a skilled, experienced software engineer be lulled into such a false sense of security.
Many bugs do indeed lie in simple corner cases, and complete code coverage will find a lot of bugs. But really, this does
not verify your code behaves as it should. Take the following (extremely simple) example:
Code:
void bar_1(){
lock(&something);
}
void bar_2(){
lock(&something);
}
void foo(){
if(a) bar_1();
if(b) bar_2();
}
See the problem? You can have a test suite which has 100% code coverage and yet will still never encounter the deadlock. I think code coverage is such a bad metric not because it potentially leaves so many bugs uncovered, but because it leaves some bugs uncovered and draws programmers into incorrect conclusions about their code's correctness.
A better metric would be feasible path coverage, i.e. the percentage of feasible paths through a program's CFG which are covered by the test suite. Of course, this has its own problems. For one, it's extremely difficult to quantify the number of feasible paths, and determining whether a single path is feasible can be an intractable problem (probably the biggest reason it isn't used as a metric). For two, the number of feasible paths generally grows exponentially with the size of the program, making it infeasible to generate, let alone manually develop, a large enough set of tests to obtain any significant path coverage. And finally, it's also an unsound metric; 100% feasible path coverage does not imply the absence of any bugs, or even the absence of simple bugs like OOB errors:
Code:
#define S_MAX 256
#define SMAX 128
void foo(int s, int data){
char buf[SMAX];
//...
if(s < S_MAX){
buf[s] = data;
}
//...
}
This is a perfectly plausible bug which could be missed even by test suites with 100% feasible path coverage. Even worse, it's potentially exploitable.
Anyway, I got off on a tangent. My point was simple. Test suites are a good idea. Test suites with 100% code coverage are an even better idea. Test suites with 100% feasible path coverage are a pipe dream, but an even better idea still. Yet
none of them provide
verification of your software.
Back to occasional lurking... sorry again for hijacking your thread, OP.
Login
Register
FAQ
Search
You and your tests! You'd be happy to note that I wrote an error logging/email library and put a couple tests in it! However you'll be sad to note that we aren't deploying the tests with the library 
© 2010 Future US, Inc. All rights reserved.