Now that we know that the continuous compiler performs well compared to both traditional compilation and traditional interpretation, at least when using the replace-preemptive scheme, we take a closer look at the performance of the continuous compiler using replace-at-call to see if the added complexity of replace-preemptive is actually necessary.
Since we are now comparing the performance of the continuous compiler with itself, we are switching back to a dual-processor system. The times reported in this section, and in the rest of the chapter, are all based on a dual-processor system.
Table 6.9 compares the performance of ghostview using replace-at-call and replace-preemptive for all seven of the compilation strategies described in Section 3.2.
Table 6.9: Comparison of Replacement Strategies with
Ghostview
Although the optimal compilation strategy differs for the two replacement schemes--largest-first is the best with replace-at-call while longest-overall is the best with replace-preemptive--the tremendous performance improvement of replace-preemptive over replace-at-call holds for all of the strategies.
When we look at the distribution of time over the functions in ghostview, we find that the vast majority of the time is spent in the function ``main.'' In fact, when running ghostview with replace-at-call and most-frequently-executed-so-far, 369.100 seconds are spent in main. This is over 97% of the total execution time. And all of it is accumulated during interpretation.
The problem is that with replace-at-call, the continuous compiler will never get to run the native-code version of the function main, even if it gets compiled first. This is because with replace-at-call, once interpretation of a function has begun, that instantiation of the function will be interpreted until it exits.
With replace-preemptive on the other hand, execution of the function main can switch from interpretation to native execution as soon as the file is compiled. With the most-frequently-executed-so-far strategy, this file completes compilation at 16.9 seconds into program execution. Because of this, only 8.978 seconds are spent on interpretation of main. From then on, the native-code version of main is executed, accumulating another 36.013 seconds of execution time. So, with replace-preemptive, only 44.991 seconds are spent in the function main. This is still over 83% of the total program execution time, but because of the switch to native code, it is a vast improvement over the performance with replace-at-call.
In actuality, the situation with ghostview is not as bad as the numbers indicate. Due to a quirk in the way eer, our tool for collecting the behavior traces (see Section 4.4), works, the time spent in dynamically linked functions is charged to the calling statically-linked function. So the function main in ghostview is getting charged with a lot of time that was actually spent in various X-Window library routines, including XtAppMainLoop. However, the general principle remains true. With the replace-at-call strategy, the first few functions entered, including the main function, are going to be interpreted throughout the program's lifetime. If one of those functions happens to be large, or an event-loop, or any type of loop in which a lot of time is spent, then the program will exhibit the same performance degradation as exhibited by ghostview when using replace-at-call.
The behavior of pico with regard to replacement strategy appears to be just the opposite as that of ghostview. Ghostview shows a vast improvement when using replace-preemptive instead of replace-at-call. Pico shows no improvement at all.
Table 6.10: Pico (Replace-at-Call)
Table 6.10 shows the performance of pico using replace-at-call for all of the compilation strategies, broken down to show where the time is spent. The column ``Time Comp.'' gives the time spent by the Compilation module compiling the program. A mark in the ``Completed Comp.'' column means that the Compiler module completed compiling the entire program during execution. (This was true for every run of pico.) The ``Time Spent in Interp. Code,'' ``Comp. Code,'' and ``Lib. Code'' columns list the total time accumulated interpreting source code, executing compiled native code, and executing precompiled library code, respectively. The ``Total Time'' column gives the total execution time of the program.
Table 6.11: Pico (Replace-Preemptive)
Table 6.11 shows the performance of pico using the replace-preemptive strategy. A close examination of Table 6.10 and Table 6.11 will reveal that they are practically identical.
The explanation for this invariance can be found by comparing the amount of time spent in precompiled library code to the time spent in both interpreted and compiled source code. Over 98% of the total execution time is spent in library code. Upon examining where this time was spent, it turns out that 63.294 seconds were spent reading input from the keyboard.
This phenomenon is not restricted to pico. Any program which exhibits this behavior will see similar performance benefits. The higher the percentage of time that a program spends in library functions, or waiting for I/O, the better its performance will be using the continuous compiler. We examine this subject again in Section 6.6.
Table 6.12 and Table 6.13 show the performance of the render program for the replace-at-call and replace-preemptive methods, respectively.
Table 6.12: Render (Replace-at-Call)
Table 6.13: Render (Replace-Preemptive)
The render program could be considered the opposite of pico since it spends almost no time at all in precompiled library code, only 0.018 seconds. This is far less than 1% of the total execution time for even the best case performance.
The replace-preemptive strategy again performs better than replace-at-call, although the improvement is not nearly as drastic as with ghostview.
The values of 194.891 seconds for the random compilation strategy and 213.036 seconds for the most-frequently-executed-overall strategy with replace-at-call is something of an anomaly; they are the only cases for which the performance of the continuous compiler is worse than that of the traditional model.
One other thing to note in the scores for render is that the compilation strategy which gives the best performance differs for the two replacement strategies. This is looked at in more detail in Section 6.5.