In this section we examine the effect of the compilation selection strategy on the performance of the continuous compiler. Table 6.14 shows the performance of ghostview for each of the compilation strategies using replace-preemptive.
Table 6.14: Ghostview (Replace-Preemptive)
The top two winners are longest-so-far and longest-overall, with a slight advantage going to longest-overall. This is not too surprising since these two methods select which files to compile based on usage of those files. However, it is interesting to note that, while the most-frequently-executed-so-far strategy performed well, the most-frequently-executed-overall strategy gave fairly poor performance.
The function which was called the most is the user function ``readline'' with 7018 calls. This is much more that the next highest total of 105 calls. However, a total of only about 1.22 seconds are spent in the readline function. So, even though the function is called many times, compiling it first does not help because only a short amount of time is spent in it. The poor performance of most-frequently-executed-overall is due to the compilation of the function main being put off until after the relatively useless compilation of the frequently called functions. On the other hand, the reason that most-frequently-executed-so-far does so much better is that when the Compiler module first picks a file to compile, main is one of the only functions that has been called, so it gets compiled earlier.
It is also important to note that for the four fastest cases--most-frequently-executed-so-far, longest-so-far, largest-first, and longest-overall--the Compiler module did not have enough time to complete compilation of the entire program. The compilation and link time of ghostview using a traditional compiler is 74.9 seconds (see Table 6.2). So, in these four cases, the continuous compiler completed execution of the program in less time than just compiling the program would take with the traditional model.
Table 6.15 shows the performance of ghostview for each of the compilation strategies using replace-at-call.
Table 6.15: Ghostview (Replace-at-Call)
Because of the overwhelming effect of the function main as described in Section 6.4, there is not much variance in the performance of the compilation strategies. However, the largest-first strategy does have a slight advantage. This same advantage can be seen in the performance of render with the replace-at-call method (see Table 6.12).
At first, the good performance of the largest-first strategy might seem rather counterintuitive since it minimizes the throughput of the Compiler module. Yet it sometimes gives the best performance for the continuous compiler as a whole when using replace-at-call.
The key is again the use of replace-at-call. The four compilation methods that are based on statistics collected while the program is running--longest-so-far, most-frequently-executed-so-far, longest-overall, and most-frequently-executed-overall--all give preference to those functions in which a lot of time has been spent. However, it is often likely that these functions are the very ones that we are stuck interpreting for the entire lifetime of the program because of the replace-at-call method. So, by compiling the files containing those functions first, we are actually wasting time since we can never use the compiled versions during the current program's execution.
So why does the largest-first strategy, which minimized the throughput of the Compiler module, perform better than the shortest-first strategy, which maximizes the throughput of the Compiler module, for ghostview and render when using replace-at-call? We will find when we look at the results for some generated programs in the next two sections that this is not true of all programs. Which compilation strategy gives the best performance seems to be related to the structure of the particular program being executed in a nontrivial manner.
In most cases, especially when using replace-preemptive, the longest-overall strategy does the best. However, this strategy unfortunately requires knowledge about the program that might not be available. It might be possible, though, to apply data saved from previous runs of the program to try to approximate longest-overall for the current run. As an alternative, longest-so-far, which could also be considered an approximation to longest-overall, seems to perform almost as well.