CS101: Documentation, Testing, and Debugging

Copyright © 1996-97 Kenneth J. Goldman

Documentation is textual information within a program that helps convey the meaning of the program to human readers who need to understand, modify, or debug the program.

Forms of Documentation

• Inherent documentation -- "self-documenting" code is the best form of documentation. Carefully chosen names for variables, procedures, etc. are extremely important in conveying their purpose. Code should be presented in a logical, easy-to-follow manner, conforming to standard conventions of order and indentation.
• Inline documentation -- occasional comments within the code, usually to the right, that explain the purpose of that line of code. Sometimes, a variable or parameter needs more explanation, or the purpose of a procedure is not immediately obvious from the name. Assumptions must be included. (In Java, everything from a // until the end of a line is considered a comment, and the compiler ignores it.)
• Header comments -- a paragraph at the top of a file that explains the purpose of its contents. This is useful for someone unfamiliar with the program who needs an overview.
• External documentation -- information for programmers wishing to incorporate your code into a larger project. The functionality of each piece is explained, but the internal implementation details are not.
• User manual -- information for users of the application on how to use the features of the program.

In CS101, we will concentrate on the first 3 forms of documentation. You will see examples of external documentation in CS101, and may begin writing some of your own external documentation in CS102. All of your code in CS101 should conform to the CS101 Style Guide.

Principles of Testing

1. Successful testing is not a proof of correctness (unless you try every input exhaustively), but it can provide useful reassurance.
2. Design test cases. Testing should not be "random."
   • When the number of possible inputs is small, try them all.
   • Test all "branches" of the code. For example, be sure to include test cases that go through the if part as well as the else part of a conditional statement.
   • Test boundary cases, like 0, 1, -1, etc.
   • Look for ways to "break" the program, and try those to be sure they don't.
3. Know what the expected results of your tests should be. Otherwise, you might not recognize an incorrect result.
4. Always be aggressive about testing. If you test gingerly, you won't uncover as many errors. Remember, the objective in testing is to find errors.
5. Save your test program (and the expected results) so you can run it again later. Whenever modifying a program to fix one problem, there is a risk that you might have caused a different problem, so it's best to repeat all the tests.

• Compilation Errors
  • Syntax errors -- The compiler cannot understand your program because it does not follow the Java syntax (the rules for a valid program). Common syntax errors include

    missing or misplaced ; or },

    missing return type for a procedure,

    missing or duplicate variable declaration.

  • Type errors -- these include

    type mismatch on assignment,

    type mismatch between actual and formal parameters (the Java compiler might say "no such method ..." instead).

• Run-time Errors
  • Output errors -- the program runs but produces an incorrect result. This indicates an error in the meaning of the program (logic error).
  • Exceptions -- the program terminates abnormally. Examples include

    division by zero,

    null pointer (we'll see what this means later),

    out of memory.

    These also indicate an error in the semantics (logic error).
  • Non-termination -- the program does not terminate as expected, but continues running "forever." (We'll see how this could happen later in the course.)

Note: Errors may cause the compiler to go "off course." Try correcting the first few errors and running the compiler again.

Correcting Run-time Errors (Debugging)

1. Avoid it. Taking the time to write a correct and well-documented program from the outset will pay back many times over. Finding and correcting bugs takes a lot of time, and it can be frustrating, especially if documentation is insufficient. However, nobody writes perfect software all the time.
2. Debug on a small scale. Thoroughly test each component of an application (each method of each object) piece by piece, correcting errors as you go. This is much easier than finding errors in a large program.
3. Understand the symptom. Before changing a program, understand the error and where in the program it occurred. Look at what the program tried to do and where it went wrong.
4. Reason about each error, trying to understand how it could have arisen. This is an important problem-solving task. Form several hypotheses about how the error might have arisen and explore them.
5. Find the source of the error. Don't waste time making "random" changes to your program. After reasoning about the error, you can start searching back through the execution of your program to see where things went wrong. A debugger can help.
   1. Examine the call stack and the values of parameters and local variables. See if it matches your expectation of what the program "should" do.
   2. Trace through an execution of your program by hand and with a debugger, stopping at certain points in the execution and/or printing out values along the way.
6. Correct the error and test thoroughly.

What you can do	When or why it's useful	How (in Symantec Cafe)
Start (or stop) debugging	Locate the source of an error in your program	Debug Menu (Start/Stop Debugging) F4
Step through execution	Watch each step of the program as it executes to find out what went wrong
Step over	Executes a line/procedure without showing you the internal steps	F8
Step into	Goes into the procedure call to show you the internal steps	F10
See the value of a variable in the code at the current point in the execution	Check the state of the program against what you expect	Position the mouse cursor over the variable and the value will pop up
Examine the call stack	See the history of how you got where you are	Look at the call window
Examine the values of parameters and local variables for procedures on the call stack	Isolate the error to a particular procedure	Right click on the relevant line in the call window and select "show data"
Set breakpoints	Mark places in the code to stop the execution for study	Right mouse click on the line of code and select "set/clear breakpoint", or click on the line of code and press to set the breakpoint, to clear a breakpoint which is already set.
Go until breakpoint	Execute the program, pausing at the first breakpoint or exception encountered, or running until the program terminates normally.