One problem with measuring software success in my organization is that too often the focus is on meeting project timelines and budget as opposed to measuring whether or not the business need was actually met. I see IT project managers being rewarded for delivering on time and on budget, even when the product that's delivered ultimately ends up being a flop. IT system implementations can fail for a variety of reasons, including a changing business environment, poor user interface, performance problems, etc, etc. But good development teams stay on top of what is happening in the business, whether they are solving the right problem and whether the software will ultimately be effective.
This probably isn't the kind of measurement that you are looking for, but I'd love to see more development efforts graded on whether or not a system is still being used 18 months down the road.
JIB
Submitted by Chris Arguin on Wednesday March 16th, 2011 12:56 am
One thing I've started to use is a measure of the fix rejection rate... the ratio of times the developer marked a bug as fixed and SQA rejected the fix.
That's extractable from most bug tracking software. Practically speaking, it points to the developer either not communicating with the submitter as to what the problem was, not completely fixing the problem, or inadequate testing.
Like any other statistic, you need to pay attention to the details... sometimes SQA rejects a bug incorrectly. But the average reject rate on my team is 7%, with a 1 sigma of 3%... and yet I had developers with a rate as high as 30%. That fed directly into this year's goals.
Chris Arguin
A different angle on this - measure utility
One problem with measuring software success in my organization is that too often the focus is on meeting project timelines and budget as opposed to measuring whether or not the business need was actually met. I see IT project managers being rewarded for delivering on time and on budget, even when the product that's delivered ultimately ends up being a flop. IT system implementations can fail for a variety of reasons, including a changing business environment, poor user interface, performance problems, etc, etc. But good development teams stay on top of what is happening in the business, whether they are solving the right problem and whether the software will ultimately be effective.
This probably isn't the kind of measurement that you are looking for, but I'd love to see more development efforts graded on whether or not a system is still being used 18 months down the road.
JIB
Fix rejection rate
One thing I've started to use is a measure of the fix rejection rate... the ratio of times the developer marked a bug as fixed and SQA rejected the fix.
That's extractable from most bug tracking software. Practically speaking, it points to the developer either not communicating with the submitter as to what the problem was, not completely fixing the problem, or inadequate testing.
Like any other statistic, you need to pay attention to the details... sometimes SQA rejects a bug incorrectly. But the average reject rate on my team is 7%, with a 1 sigma of 3%... and yet I had developers with a rate as high as 30%. That fed directly into this year's goals.
Chris Arguin