Assigned September 25th, to be completed by October 3rd.

You are expected to do your own work on all homework assignments. You may (and are encouraged to) engage in general discussions with your classmates regarding the assignments, but specific details of a solution, including the solution itself, must always be your own work. (please review the section on Cheating in the admin page.)

What to turn in?

You should turn in (using Laulima) an electronic tar gzipped archive named ics332_hw4_UHUSERNAME.tar.gz. The archive MUST contain a single top-level directory called ics332_hw4_UHUSERNAME, where “UHUSERNAME” is your UH user name (e.g., for me, it would be ics332_hw4_esb). In that directory you should have all the files named exactly as specified in the questions below.

Expected contents of the ics332_hw4_UHUSERNAME directory (which contains two subdirectories paths and fswatcher):

Environment

Exercise #1 can be done on any environment, while Exercise #2 requires a Linux platform. As usual, we’ll test everything on Linux.

Exercise #1: Multi-threading for Speed [50 pts]

Overview

You are hired by a company and tasked with setting up a driving directions app in Java. Like Google Map, the idea is to pre-compute a bunch of all-pair-shortest-paths so that user queries can be answered quickly. The concern is that the compute time will be problematic because, as you know from ICS311, the Floyd-Warshall algorithm has complexity O(n3) for a graph with n vertices.

Apparently, you’re the only one who knows how to use threads in Java (side note: in real life this should not happen). The goal is to multi-thread the application to see how much faster it can run.

This exercise corresponds to the simplest possible “more speed with threads” scenario.

Question #1 [3 pts]

You are given two Java source files that form a package called paths:

Download these files to a directory called paths, and you can compile and run from the command-line as follows (or use whatever IDE, etc.):

% ls -R
paths

./paths:
ComputePaths.java  FloydWarshall.java

% javac paths/*.java

% java paths.ComputePaths
Usage: java ComputePaths <graph size> <# threads>

The program requires two command-line arguments:

Running the program for graphs with 250 vertices (and passing 1 for the number of threads, which is ignored anyway) produces this output on my laptop:

% java paths/ComputePaths 250 1
Ran Floyd-Warshall on Graph #   0: 58187
Ran Floyd-Warshall on Graph #   1: 55577
Ran Floyd-Warshall on Graph #   2: 54699
[...]
Ran Floyd-Warshall on Graph #2518: 105764
Ran Floyd-Warshall on Graph #2519: 105785
All graphs computed in 59.627 seconds

By trial-and-error (i.e., doing a human-driven binary search), determine the graph size that leads this code to run in between 25 and 30 seconds on your machine. Note that repeated runs will show some variation.

In your report (README.txt) simply state what value you found for the graph size. Use this value for all subsequent questions.

Warning: You should run your code on a “quiescent” system, i.e., without running any other applications at the same time.

Question #2 [35 pts]

Enhance ComputePaths.java so that it uses the specified number of threads. The idea is to split up the work among threads. For instance, when using 2 threads each thread should compute 2520/2 = 2260 graphs; using 3 threads, each thread should compute 2520/3 = 840 graphs; etc. Note that I picked 2520 because it is divisible by 1, 2, 3, …, 10.

For instance, on my laptop, repeating the run from the previous question but specifying 2 threads, the output from the enhanced program looks like:

% java paths/ComputePaths 250 2
Ran Floyd-Warshall on Graph #1260: 79359
Ran Floyd-Warshall on Graph #   0: 58187
Ran Floyd-Warshall on Graph #   1: 55577
[...]
Ran Floyd-Warshall on Graph #2518: 105764
Ran Floyd-Warshall on Graph #2519: 105785
All graphs computed in 29.967 seconds

You should observe that using 2 threads your program goes faster. If not, you most likely have a bug. Make sure your program uses the number of threads that’s specified.

Hints:

Question #3 [12 pts]

Run your code for 1, 2, 3, …, 10 threads. For each number of threads, run your code 5 times and compute the average execution time. In your report (README.txt) list the average execution time for each number of threads. Then answer the following quick questions:

Exercise #2: Multi-threading for Interactivity [50 pts]

Overview

You get hired by a company that does a lot of Java development in Linux environments. Many applications being developed there need a “file system watcher” feature: something that will give them a callback whenever a file is created in some directory. You get tasked with implementing a FSWatcher class that provides that feature.

Disclaimer: As you might expect, Java already has something like this. It’s in the java.nio package. It’s really difficult to come up with an original systems idea, and when one does, one doesn’t “waste” it on a programming assignment for a course but instead make money :) So, we are pretending java.nio (which is a wonderful package) does not provide this feature.

Question #0 [0 pts]

Of course you don’t want to implement all that file system watching stuff in Java directly because it’s a lot of work. Instead, you happen to know that there is a Linux package called inotify-tools that provides a program called inotifywait that does what you need!

Install the inotify-tools package as follows:

% sudo apt install inotify-tools

Now, test inotifywait with the following command:

% inotifywait -e create -m /tmp/
Setting up watches.
Watches established.

The above command will not return (i.e., you won’t get the prompt back), but it will print some message each time a new file is created in directory /tmp/. Let’s test this. In another terminal/shell just create a file in /tmp/, for instance by doing touch /tmp/whatever. You should now see one output line from the inotifywait command:

% inotifywait -e create -m /tmp/
Setting up watches.
Watches established.
/tmp/ CREATE whatever

This line of output from inotifywait says that in /tmp/ a file names whatever was created!

Make sure you have all the above working on your system before you proceed to the next question.

Question #1 [25 pts]

You are provided a use-case (which you should not modify) for this question: FSWatcherUseCaseQ1.java

This class contains a main() method, and is in package fswatcher. It expects another class in that package, FSWatcherQ1, which you are to implement (in source file FSWatcherQ1.java).

The FSWatcherQ1 class should just have single method called watch() with the following signature:

public void watch(String dirname, Consumer<String> method);

The watch() should:

Once implemented, you can run FSWatcherUseCaseQ1:

% java fswatcher/FSWatcherUseCaseQ1

This is an interactive application, which is pretty self explanatory. Use CTRL+C to terminate the application.

Doing the above likely involves things you’ve never done in Java, so below are two brief “tutorials”.

Tutorial #1: Calling a lambda expression

Your method needs to call the lambda expression passed to it (the method parameter). Without getting into any details, assuming the string you want to pass to method is stored in variable line, you would invoke method as:

method.accept(line);

That’s it. So, whenever you get a line produced by inotifywait, just do the above. See the large Java documentation about lambdas for all wonderful details.

Tutorial #2: Creating an external process from the JVM

It turns out that Java can create a process in a fork-exec fashion. This is done with something called ProcessBuilder. You can of course find tons of tutorial information on-line for this. Instead of duplicating this here, I am simply showing a small example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

String[] tokens = ("ls -la /tmp/").split("[ \t\n]+");
Process p = null;
ProcessBuilder pb = new ProcessBuilder(tokens);

try {
    p = pb.start();
} catch (IOException e) {
    System.err.println(e.getMessage());
    System.exit(1);
}

InputStream  sstdout = p.getInputStream();
BufferedReader stdout = new BufferedReader(new InputStreamReader(sstdout));

String output_line;
while (true) {
    output_line = stdout.readLine();
    if (output_line == null) {
        break;
    } else {
        System.out.println("Got some output from the ls command: " + output_line);
    }
}

At line 1, we create an array of strings based on a command-line string. In this example we want to run ls -la /tmp/ (which lists files in /tmp/), and we simply build array [“ls”, “-la”, “/tmp/”].

At line 3 we create a ProcessBuilder object for the command-line, which we start at line 6 (catching an exception if there is an error). This produced a Process object.

At line 12 we get an input stream based on the external process’ output (i.e., the output of the process is input to us).

The loop at line 15 just gets output from the ls process and prints then out.

So this is just like fork-exec, but in Java!

Question #2 [25 pts]

The implementation in the previous question is really, really weak because we can only watch a single directory. That is, we don’t get control back at all. The objective now is to enhance your implementation, in a new class called FSWatcherQ2 to avoid this problem.

The way to do this is to use a thread to do the “getting lines from inotifywait” loop. In other terms, your watch() method does:

You are provided with a use-case: FSWatcherUseCaseQ2.java

The use-case is pretty self-explanatory. Use CTRL+C to terminate the application.

Question #3 [10 pts] [EXTRA CREDIT]

Enhance your code, in a new class called FSWatcherQ3 so that it implements a new method stopWatch() with the following signature:

    public void stopWatch(String dirname);

When invoked, this method will cause your file system watcher to stop watching a particular directory (or do nothing if that directory wasn’t watched).

You are provided with a use-case: FSWatcherUseCaseQ3.java

The use-case is pretty self-explanatory. It now has a “Quit” option.

Hint: