Extending The Reach Of Statistical Software Testing

Extending the Reach of Statistical Software Testing

Statistical software testing is a promising technique for reducing the software testing burden by applying test cases to the software under test according to a model of the expected usage of the system in operation, called an operational profile. However, current statistical testing techniques have not been put into general practice, as they do not scale well as the complexity of the systems under test increases. In particular, as system complexity increases, the matrices required to generate test cases and perform model analysis can grow dramatically, even exponentially, overwhelming the test generation and analysis applications. In this work, a new technique for representing the operational profile is proposed to mitigate the complexity issue for statistical test case generation. The use of a state-based requirements specification model as the basis of the operational profile leverages parallelism to reduce the visible size of the model. The specification model can then be extended to include conditional probabilities of input data occurrence. * This work also proposes a statistical testing framework using this specification-based operational profile to generate and execute test cases. Such a framework allows us to evaluate the effectiveness and efficiency of this statistical testing technique.

Dissertation Abstracts International

Statistics, Testing, and Defense Acquisition

For every weapons system being developed, the U.S. Department of Defense (DOD) must make a critical decision: Should the system go forward to full-scale production? The answer to that question may involve not only tens of billions of dollars but also the nation's security and military capabilities. In the milestone process used by DOD to answer the basic acquisition question, one component near the end of the process is operational testing, to determine if a system meets the requirements for effectiveness and suitability in realistic battlefield settings. Problems discovered at this stage can cause significant production delays and can necessitate costly system redesign. This book examines the milestone process, as well as the DOD's entire approach to testing and evaluating defense systems. It brings to the topic of defense acquisition the application of scientific statistical principles and practices.