Warning: You need Java 8 installed for this lab to work. The Urbauer computers don't seem to have Java 8 so please do this with somebody who has Java 8.
This is a new lab, which has the following implications:
- There may be some mistakes in the lab write-ups.
- The TAs may not be as helpful as usual, because they have not done this lab.
- I will allow the time needed for us to complete the lab, extending deadlines as needed.
Thanks for your patience!
A more global and famous example of such a facility is Hadoop. From its beginnings, this service aimed to provide service that could scale from a single server to thousands of nodes. The service anticipates failure, in which case it can restart a computation on another node.
The most common programming paradigm for services such as Hadoop is the MapReduce paradigm:
As an example, consider summing a million numbers:In this portion of the course, you will implement a form of such an infrastructure. The work will be limited to a single machine, but tasks can be mapped across multiple threads. The pedagogical goals of this lab are related to the following questions:
- The million numbers are distributed among n nodes, so that each node receives approximately 1,000,000/n numbers to be summed.
- When a node has finished its work, it has a partial sum that must be added to all of the other nodes' sums to obtain the final answer. The reduce step therefore consists of obtaining and summing the nodes' partial sums. This can be done naively or cleverly.
This will allow us to author some visualization and GUI control components in subsequent assignments.
Part of our design will be to articulate the PCS messages.
I have workstations that have many cores, each capable of operating independently. Some of my cores are hyperthreaded, allowing even more concurrent execution.This user will have more to say about what he or she wants in the upcoming installments of this lab. For now, we must implement only what has been described in the story. From that story, some unit tests will be written (my job, perhaps yours as well) and you will develop the code to meet those tests. Tempted though you may be, you will not at this point introduce any superfluous features, from the point of view of the user story. This parsimonious development style is also a tenet of the agile software development process.
Abstractly, I think of the workstations has having a certain nubmer of nodes, each of which can execute some task independently. I would like to make use of these nodes in running some MapReduce computations. I would like to submit a job, have the job decomposed into independent tasks, and then have those tasks distributed across the nodes of my machine. When the tasks are done, their partial results will be turned into a result for the job by some code I will supply.
You have some work to do inside Node but the instantiation and .start()ing of Nodes is done in HaWUp's constructor.
Each Task has a waitForResult() method that must block until the Task has finished. There are comments in the source file to direct your activity.
As described in class, using Java's built-in wait() statement can make code look messy because of the checked exception you must expect.
As in class, you have Wrappers.wait(o) and Wrappers.notify(o) which handle the exception for you, leaving the rest of your code looking nicer. There is also Wrappers.sleep(ms), and all of these are found in the hawup.utils package.
The unit tests and other tests are found in the hawup.testing package.
Different computers can operate at different speeds. To show the benefit of hawup, it is helpful to have an example that takes a known amount of time on one Node.
You will next experiment to determine the appropriate repeat factor so that a computation takes between 45 and 60 seconds.
It should fail, unless you are using a computer from the 1950's. But if you were doing that, your computer would be the size of the Urbauer building, and it probably does not have a web browser, so you wouldn't be reading this. Enough said.Your task here is to modify the repeatFactor value until the test takes between 45 and 60 seconds. The unit test will check for this.
Once you have determined the proper value, set the repeatFactor static variable to that value in the SumLoToHi class.
Using what you learned in class:
- You must Wrappers.wait(this) in the getTime() method until this.end != null.
- You must also call Wrappers.notifyAll(this) just after any point in the code where this.end could become non-null.
When you done with this studio, you must be cleared by the TA to receive credit.
- Commit all your work to your repository!If you do not commit, the TAs cannot grade your work and you will receive a 0 for this assignment!
- Fill in the form below with the relevant information
- Have a TA check your work
- The TA should check your work and then fill in his or her name
- Click OK while the TA watches
- If you request propagation, it does not happen immediately, but should be posted in the next day or so