next up previous
Next: Related work. Up: Introduction Previous: Emerging trends and

Addressing PSA challenges with distributed continuous quality assurance and model-driven techniques.

Two promising techniques for addressing the QA challenges described above include distributed continuous quality assurance (DCQA) and model-driven techniques. DCQA techniques are designed to improve PSAs iteratively, opportunistically, efficiently, and continuously in multiple, geographically distributed locations []. In particular, DCQA techniques have been shown to help assure the portability and reliability PSAs of QoS-intensive software by validating functional correctness and QoS satisfaction across a wide range of hardware, OS, network, and compiler platforms. For example, the Skoll DCQA environment (www.cs.umd.edu/projects/skoll) provides a framework for executing a variety of QA tasks continuously across a grid of computers distributed around the world and analyzing the results to evaluate PSAs. Likewise, the Dart DCQA environment (public.kitware.com/Dart) supports a continuous build/test process that starts whenever repository check-ins occur.

Model-driven techniques help to minimize the development, validation, and evolution effort associated with DCQA activities by capturing the customizability of QoS-intensive software within higher-level models that help to automate the analysis and synthesis of various artifacts, such as component interfaces, implementations, and glue code; configuration files; and deployment scripts []. In particular, model-driven techniques help assure the correctness, maintenance, and porting PSAs of QoS-intensive software by automatically generating artifacts required for QA activities, such as regression testing and performance benchmarking. For example, the Options Configuration Modeling language (OCML) [] allows developers to model middleware configuration options as high-level models. Likewise, the Benchmarking Generation Modeling Language (BGML) [] allows developers to automatically generate sophisticated benchmarking experiments.

This article describes how model-driven DCQA processes and tools can work separately and together to help monitor, safeguard, enforce, and reassert desirable PSAs after changes occur in QoS-intensive software. In particular, we describe two DCQA processes implemented in the model-driven Skoll environment that help to ensure PSAs related to both correctness and performance across large configuration spaces. To validate our approach, we present results from applying our model-driven DCQA processes and tools on ACE+TAO (www.dre.vanderbilt.edu/Download.html), which are widely-used QoS-enabled middleware consisting of 2,000,000 lines of continuously evolving C++ frameworks, functional regression tests, and performance benchmarks contained in 4,500 files that average over 300 CVS commits per week. Our results show that (1) DCQA techniques improve the ability of developers to detect and pinpoint software portability and availability problems, (2) integrating DCQA and model-driven techniques can significantly enhance the process of identifying key subsets of options that affect PSAs, and (3) monitoring only these selected options helps developers understand effects of system changes based on these PSAs with an acceptable level of effort.



next up previous
Next: Related work. Up: Introduction Previous: Emerging trends and



Douglas C. Schmidt
Fri Sep 3 13:41:43 CDT 2004