A Chicago Tribune article recounts how a software bug in an infusion pump lead to brain-death for a patient in 2009 (“Medical Industry Taking Hard Look at Software Faults,” Christine Mai-Duc, Chicago Tribune, August 31, 2011, p. 19)
It reports that the US Food and Drug Administration (FDA), which regulates and monitors the development, testing, and marketing of medical devices, has been criticized for not requiring more testing or allowing release of untested products. The article notes that US Federal Aviation Administration (FAA) regulations for aviation software are more rigorous, and quotes U Mass Professor Kevin Fu as stating that the FDA “was caught off-guard as to how significant software would be.” I’ve worked on several FDA-regulated software projects over the last fifteen years, and I certainly would not characterize the FDA as failing to understand and act on the hazards of software defects. FDA guidance for software development is quite clear about the need for comprehensive testing.
Both the FAA and the FDA specify general requirements for software process and artifacts that may be met in many ways. The FDA calls for a documented quality management system that includes certain basic software development activities, including software testing. Producers must be able to prove that the documented process meets all the general requirements and is routinely and correctly followed. Click here to view the details of the FDA guidance. The software process and artifacts of FAA DO-178b are very similar.
However, DO-178b is prescriptive about one narrow aspect of testing: it requires that tests reach all combinations of software conditions defined in a program. Interestingly, the FDA regulations call for thorough in-use (clinical) testing (“validation”), which has no direct analog in the FAA regulation.
The Tribune article concludes by noting that “Fu, Pfleeger, and others believe implementing testing standards and engineering practices prevalent in other software-reliant industries, such as avionics, could diminish some of the risks.”
Although requiring the FAA’s higher code coverage could diminish some risks, I doubt that it alone would make an appreciable difference in the number of serious software defects in medical devices. The Tribune story states that the pump failure was a result of a “memory buffer overflow” that occurred when “all three of the pump’s channels failed.” Compliance with DO-178b would not necessarily have revealed that bug. Also, DO-178b predates object-oriented programming and the condition test coverage it calls for does not provide any better chance of revealing the kind of bugs that are specific to object-oriented programming languages like C++ or Java. (A revision to address these and other issues, DO-178c, is near release.) The article confuses post-release product evaluation under FDA’s 510k regulation with pre-release software testing. The 510k process calls for evaluating a new product with comparable products, after it has completed full development testing. Similarly, the FAA certification process requires that flight software has been developed in compliance with DO-178b prior to certification of an entire aircraft.
Although it is certainly true that quality cannot be tested-in to software systems, very high-reliability software in critical systems can be achieved. Software testing is essential for this achievement. None of the tools and methods needed to do this are secret, proprietary, extraordinarily expensive, or impractical. Nor are they the exclusive province of avionics developers or the result of regulatory extent/compliance. In all cases that I know of, they result from applying well-known lessons of software engineering consistently and effectively.
Is it predestined that only a lucky few are competent? That the serious risk posed from mediocre or dangerous producers is no more avoidable than bad weather and earthquakes? In a word, no.
What does it take to be competent? My list of software development practices for very high reliability follows. I cannot think of any merely annoying or catastrophic software bug inflicted on society that would not have been found and removed before it escaped, had the producer done all of the following.
Achieving these goals will result in very reliable software systems. Nothing in this list is new, exotic, proprietary, or prohibitively expensive. None of it is specific to any programming language or platform. By the way, none of this is required or even suggested by Agile development practices.
Here’s a very simple self-assessment for software producers. If you’re doing at least 80% or better of the list, consider yourself competent. If you’re doing at least half, mediocre. Less than half, dangerous. If you’re close to a cutoff, assume the lower side, unless you can make strong case to the contrary. If you work for a dangerous firm that produces safety-critical products, please let us know, so we can stop using them.