# Computer Systems Performance Analysis: Design of Experiments

A tutorial by Prof. Raj Jain given at XXIX Brazilian Symposium on Computer Networks and Distributed Systems, May 30-June 3, 2011, Campo Grande, Mato Grosso do Sul, Brazil.

Performance often depends upon more than one factor such as the system and the workload. Proper analysis requires that the effects of each factor be isolated from those of others so that meaningful statements can be made about different levels of the factor, for instance, different systems. Such analysis is the main topic of this tutorial. The techniques presented in this part will enable you to do the following:

• Design a proper set of experiments for measurement or simulation.
• Develop a model that best describes the data obtained.
• Estimate the contribution of each alternative (for example, each processor and each workload) to the performance.
• Isolate the measurement errors.
• Estimate confidence intervals for model parameters.
• Check if the alternatives are significantly different.
• Check if the model is adequate.
The goal of a proper experimental design is to obtain the maximum information with the minimum number of experiments. This saves considerable labor that would have been spent gathering data. The tutorial is based on Part IV of my book "The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling." The techniques of the course can be used to analyze and compare any type of systems including algorithms, protocols, network, or database systems.

The tutorial covers the following topics:

• Computer Systems Performance Analysis: Design of Experiments
• Performance Analysis
• Experimental Design
• Text Book
• Overview
• Module 1: Introduction to Design of Experiments
• Overview
• Terminology
• Common Mistakes in Experimentation
• Types of Experimental Designs
• Example
• A Sample Fractional Factorial Design
• Summary I
• Module 2: 2k Factorial Designs
• Overview
• 2k Factorial Designs
• 22 Factorial Designs
• Model
• Sign Table Method
• Allocation of Variation
• Case Study 17.1: Interconnection Nets
• 22 Design for Interconnection Networks
• Interconnection Networks Results
• General 2k Factorial Designs
• 2k Design Example
• Analysis of 2k Design
• Summary
• Module 3: 2kr Factorial Designs
• Overview
• 2kr Factorial Designs
• Computation of Effects
• Experimental Errors: Example
• Allocation of Variation
• Confidence Intervals For Effects
• Assumptions
• Visual Tests
• Multiplicative Models
• Analysis Using Multiplicative Model
• Variation Explained by the Two Models
• Visual Tests
• Interpretation of Results
• Summary
• Module 4: 2k-p Fractional Factorial Designs
• Overview
• 2k-p Fractional Factorial Designs
• Example: 27-4 Design
• Fractional Design Features
• Analysis of Fractional Factorial Designs
• Sign Table for a 2k-p Design
• Example: 27-4 Design
• Example: 24-1 Design
• Confounding
• Other Fractional Factorial Designs
• Summary
• Other Designs

Presentation slides in Adobe Acrobat Format: 1 slide/page

View Slides+Audio ( Part 1, Part 2 )

Right click and save to download Slides+Audio ( Part 1, Part 2 ) podcast.
Frequently asked questions about remote audio/video viewing

Back to other tutorials by Raj Jain
Complete List of Audio/Video Lectures by Raj Jain