The Windows Vista team knows something about finding bugs after all:
“Donnelly [who manages part of Microsoft’s Vista test operation] tries to do the opposite of what an IT manager would recommend. He changes all the default settings, for instance. And instead of testing a clean installation on a new machine, he’ll try to upgrade an older model. ‘You find bugs,’ he said, ‘You absolutely find bugs that way.’”
It’s a start.
The security guy in me has a hard time choking down the Unit Testing doctrine. Programmers don’t find deep bugs in their own code. Machines don’t find deep bugs when running in a contrived development/testing environment. These approaches find surface bugs, and that’s valuable, but they shouldn’t help you sleep at night.
People doing dumb and/or malicious shit finds deep bugs. You can’t script dumb and malicious.
1 comments:
<em>This comment was imported.</em>
Author: <a href="http://www.holygoat.co.uk">Rich</a><br />
Posted: 2006-11-23 04:24:29<br />
… sure, but the motivation for always testing in a controlled environment is to ensure reproducability. You can't say you've fixed a bug unless you can reproduce the initial state and demonstrate that the debug does not occur.
I'm no better, of course — I tend to prefer real-world testing — but that's the justification.
Post a Comment