This article motivated the need for -- and design of -- model-driven distributed continuous quality assurance (DCQA) processes and showed several examples of how these processes can be used by developers of QoS-intensive software to help assess and assure persistent software attributes (PSAs). We rapidly implemented two such DCQA processes using model-driven tools, executed them in the Skoll environment, and demonstrated their effectiveness via several feasibility studies involving the widely used ACE+TAO middleware.
These studies, presented in Section 
showed how DCQA
processes helped ensure key PSAs, such as functional correctness,
maintainability, portability, and efficiency by: (1) monitoring
( e.g., the clean compilation scenarios described in
Section ), (2) safeguarding ( e.g., via
feedback to developers on failing configurations and its isolation as
described in Section ), (3) summarizing (
e.g., via identification of main effects configurations that highly
influence QoS as described in Section ), and (4)
reasserting ( e.g., via rerunning tests to validate main effects
options) these attributes on a range of platforms.
They also showed how Skoll-based DCQA processes reduced the level of
effort -- both in time and resources -- required to assure the PSAs
mentioned above in rapidly changing environments characterized by a
multitude of configuration options and diverse OS/compiler platforms.
Despite our initial progress, much work remains to be done. For example, currently the overhead of specifying DCQA processes is more than we would like. We also do not yet support QA tasks that require human evaluation, such as evaluating usability or code maintainability. Moreover, our experiments thus far show that Skoll works best if we have a large number of client machines to run the experiments, so recruiting users to donate their computing resources -- and assuring the security of these resources -- is becoming increasingly important.
It's also important to note that the work presented here is only an initial foray into a broader R&D agenda on DCQA processes for Remote Analysis and Measurement of Software Systems (RAMSS) (see measure.cc.gt.atl.ga.us/ramss). To date, only a handful of research efforts [,,,] have studied such processes systematically, so there are many unresolved challenges and risks, such as how best to structure DCQA processes, what types of QA tasks can be distributed effectively, and how the costs/benefits of DCQA processes compare to conventional in-house QA processes. To address these issues, we are working with other researchers in the RAMSS community to develop tools, services, and algorithms needed to create, prototype, and evaluate various types of DCQA processes focused on functional testing, QoS evaluation, and usage profiling of highly configurable software program families.
= =0.8 .5 - =.9