Quality of service (QoS)-intensive software must satisfy stringent requirements for persistent software attributes (PSAs), such as assured latency/jitter/throughput values, scalability, dependability, and security. Examples of QoS-intensive software include high-performance scientific computing systems ( e.g., high energy physics experiments and computational fluid dynamics), distributed real-time and embedded systems ( e.g., control systems for the electrical power grid and commercial air traffic), and the accompanying systems software ( e.g., operating systems, middleware, and language processing tools). QoS-intensive software is increasingly affected by trends toward distributed and evolution-oriented development processes and demand for user-specific customization.
Today's global information technology economy and n-tier architectures often involve developers distributed across geographical locations, time zones, and business organizations. One goal of distributed development is to ensure rapid change by having developers work simultaneously, with minimal direct inter-developer coordination. Distributed development processes can increase churn rates in the software base, which can lead to rapid, unpredictable, and potentially uncontrolled changes in persistent software attributes (PSAs), including reliability, scalability, efficiency, adaptability, maintainability, and portability. The same situation occurs in evolution-oriented processes [], where many small increments are routinely added to the base system. What is needed are quality assurance (QA) techniques that can coordinate remote developers and cope with frequent software changes by detecting, diagnosing, and helping fix faulty and/or inefficient changes as quickly and automatically as possible.
In addition, QoS-intensive software needs to be fine-tuned to specific (and often changing) user platforms/contexts to meet its performance requirements. This is commonly done by (re)adjusting a large set of (10's-100's) configuration options that control parameters such as different workloads; operating system, middleware and application feature sets; compiler flags; and/or run-time optimization settings. Moreover, the configuration options that maximize QoS for a particular set of hardware, OS and compiler platforms may not produce optimal QoS for a different platform combination.
While this ``explosion'' in the software configuration space promotes adaptability and portability, it places enormous demands on the developers who must must ensure that their decisions and modifications work across this large (and often changing) configuration space. In fact, experience has shown that the reliability, portability, and efficiency PSAs of QoS-intensive software cannot be assured without extensive QA on a range of requirements and operating environments [].
In practice, however, budgets for development and in-house QA are limited. Therefore, developers often can only assess PSAs on a few software configurations and then extrapolate to the entire configuration space. This allows many sources of PSA degradation to escape detection until systems are fielded. Moreover, those few in-house tested software configurations are often selected in an ad hoc manner so PSAs are not evaluated systematically. What is needed are QA techniques that can cope with exploding software configuration and validation spaces.