Operating Systems Project 2


The goals of this project are:
  1. to learn about threads and pthreads
  2. to implement a simple multi-threaded program
  3. to practice using C

This is an individual or group project, at your choice and as described in the course administration page. You may use any standard libraries -- check with the instructor if you think you might want to use non-standard library functions.

The finished project must be sent by email to the instructor. If you work as a group, make sure to send only ONE copy of the project to the instructor, and to cc every member of the group.

Please send me your source file in an attachment. The beginning of your code should be a comment with the equivalent of a README file, reporting whether your code works or not, the operating system you used to test your code, and anything else I need to know.

If you are unclear about one or more of these requirements, please contact the instructor or ask your fellow students on the mailing list.

Please send in your project on time -- late submissions will not be accepted, and I prefer to have partially-working projects rather than no project at all.

Initial Program with Race Conditions and Busy Waiting

Carefully study the program p2.c. It is a multithreaded program with a well-understood race condition: incrementing an integer from multiple threads leads to inconsistent results.

While studying the program, also run it to make sure you understand what it does and how it works, and especially, where the race conditions are. As you do this, feel free to modify the program to enhance your understanding -- but the program you turn in must be modified from the original program.

After you feel you understand this program, you should be in a good position to understand the rest of this assignment.

The program has one particularly clear race condition: if multiple threads increment state->counter at the same time, occasionally the increments will be lost. This can be seen by running with sufficiently large number of threads and loops, e.g. with the default parameters. Conversely, the sequential code is correct -- running with a single thread will never return an incorrect result.

There is a second race condition, very similar to the first but harder to reproduce. This is where, at the end of the main loop in thread, state.threads is decremented. This is harder to reproduce because each thread only decrements this once in its lifetime, rather than in a loop. Nonetheless, your instructor has been able to see this problem with as few as 100 threads, and to see it almost reliably with 1000 threads. If one or more of these decrements is lost, the main program will never terminate (in which case, Control-C will terminate the program if you are running it in a terminal. If you are running in an IDE, the IDE will have a way of stopping the program).

Make sure you practice this yourself, to establish for yourself a number of threads that on your computing platform will reliably cause your program to not terminate.

Note that, whether or not the decrement is lost (i.e., whether or not the program terminates), the thread number printed may be duplicated, sometimes more than once, and in any case the values may not be printed in order -- the value given to printf may or may not match the value decremented, and threads may be suspended between the call to printf and the decrement operation, or even in the middle of the call to printf.

Finally, after studying the program carefully, you will note that there are two busy-waiting loops (also known as spinlocks), one near the beginning of thread, and one near the end of main. These loops are used for synchronization: the loop in thread is used to start all the threads at about the same time, and the loop in main is used to wait for all of the threads to finish.

The goal of this project is to get you to practice using C thread primitives to both resolve race conditions, and to avoid busy-waiting.

Threads

Every process has at least one thread, which we generally call the main thread. When the main thread exits, the program may or may not terminate -- some runtime systems wait for all threads to end before terminating the program, others terminate the program as soon as the main thread exists, even if other threads are still running.

Any thread may call pthread_create with as third parameter the name of a C function, and that function will then execute in a new thread.

The type of the parameter to the thread function is a void pointer (a pointer to an unspecified type), and the return type is also a void pointer. Thread always only returns NULL, which is not useful. The parameter to thread is a pointer to a global variable that includes a counter, a number of loops, the number of threads currently running, and a start variable.

Mutexes

Each thread has its own stack, meaning each function call within the thread has its own parameters and local variables. However, any global variable (including local variables with the keyword static) are shared among all the threads. Race conditions are present when multiple CPU operations are needed to consistently access or modify one or more global variable.

Another way of describing race conditions is to consider the collection of operations on a global variable that must be consistent. Such a collection of operations is called a transaction. In the program for this project, incrementing state->counter is a transaction. In your bank, subtracting money from one account and adding the same amount to another account is a transaction. In both cases, we will only get consistency if at most one thread at a time gets access to the global variables that are part of the transaction. In the case of a bank, the two accounts are the global variables of the transaction, and no other threads should be able to access either of the two accounts while the transaction is in process. Likewise, in our program, the counter is the global variable, and no other threads should be able to access the counter while one thread is reading the value, modifying it, and storing back the result.

A transaction is described in terms of the global data that must be accessed atomically. A different description is that of the critical section, which is a section of code that, if executed concurrently by different threads, will lead to inconsistencies. It is clear that the sequence of operations implementing a transaction are in a critical section. The concept of critical section is a little less clear when the data becomes more complicated -- for example, if a concurrent program maintains several queues, all the code accessing a given queue must exclude other threads accessing the same queue, but needn't exclude threads accessing different queues. Even more interestingly, as long as the queue is not empty, different threads can be adding data to a queue at the same time as other threads are removing data from the same queue, but we ignore this detail (concurrent adding and removing from the same queue) in what follows.

A mutual exclusion variable, or mutex, is a special kind of boolean that the pthread code knows how to modify atomically. A mutex is in one of two states: locked or unlocked. Mutexes are created in the unlocked state, and can be locked by calling pthread_mutex_lock. Once a mutex is locked, subsequent calls to pthread_mutex_lock will block the calling thread, until that thread calls pthread_mutex_unlock.

Note that complicated things can happen if a lock acquired by thread t1 is unlocked by a different thread t2, or if the same thread that has already locked a mutex again calls pthread_mutex_lock on the same mutex. Your life will be simpler if you are careful to avoid these two cases, which are rarely useful in practice.

Mutexes are designed to support atomic transaction processing. The programmer must create a mutex for each unit of data (for example, a mutex for each queue) that must be accessed consistently. Before the start of the code that accesses the unit of data, the code locks the mutex corresponding to that data. The mutex is unlocked once the transaction is complete.

If any and all code that accesses a unit of data acquires the corresponding mutex before accessing the data, and releases it afterwards, then we are guaranteed that at any given time, at most one thread is accessing that data (by the way, the purpose of the synchronized keyword in Java is mutual exclusion).

Unfortunately, C offers no support to either guarantee that any and all code that accesses a unit of data will acquire the corresponding mutex before accessing the data, nor to guarantee that the mutex is eventually released. However, the program in this project is simple enough that these should not be concerns.

Be sure to initialize each of your mutex variables before use, either by calling pthread_mutex_init, or by setting it to PTHREAD_MUTEX_INITIALIZER. The default attributes should be fine for this project.

Condition Variables

Mutexes are sufficient to guarantee exclusive access to shared global variables. However, they are not designed to coordinate behavior among different threads. For example, in this project, all threads loop until the start global variable is set. This is wasteful of energy and CPU time.

Condition variables are designed to support coordination among threads. A thread that holds a mutex can call pthread_cond_wait on that mutex and a specific condition variable. This call releases the mutex (which allows other threads to then acquire that mutex) and blocks the calling thread until another thread calls pthread_cond_signal or pthread_cond_broadcast on the given condition variable.

By using condition variables, a thread can wait to receive a signal while using neither energy nor CPU time.

Be sure to initialize each of your condition variables before use, either by calling pthread_cond_init, or by setting it to PTHREAD_COND_INITIALIZER. The default attributes should be fine for this project.

Deliverables

You must modify p2.c to accomplish the following, preferably in the given order:

  1. add a mutex to make sure the overall count is always correct. The critical section MUST be only the increment operation, and will lose points if it includes anything else.
    Because of all the locking and unlocking, be prepared that at this point your code is likely to slow down significantly (for me, about a factor of 20). Locking and unlocking is more expensive than just incrementing a variable, especially when you have many threads contending for the same lock and being suspended and restarted. On the other hand, if your code completely freezes, most likely you have failed to unlock your mutex.
  2. add a second mutex to make sure the thread count is always correct and the program always terminates. I leave it to your judgement to decide what is included in this critical section, but be aware that bad decisions may lose points.
  3. add a condition variable so that the main thread doesn't have to busy-wait until the threads are all done, but can instead just call pthread_cond_wait (either once, or in a loop). I again leave it to your judgement to decide what mutex this signal is associated with, and again bad decisions may lose points.
  4. figure out how to replace the busy-wait at the beginning of each thread with the carefully considered use of a mutex, a signal, or both. You are allowed to use at most one mutex and/or one signal for this part, no matter how many threads there are, and you cannot use the same signal or same mutex as in any of the other parts of this project.

Having done as much as you can of this, please turn in your modified p2.c.

Each of the four sections are worth 25% of the grade. Note that it is not enough for your code to work correctly some of the time -- it must work reliably.

This project is simple enough that it requires a relatively small amount of code, so make sure your code is as clear as possible, using comments where necessary. Points may be lost if the code is sufficiently obscure or is missing the initial comment that tells me how to run it. Any substantial changes to the original code (other than those specified here) will also lose points.

Each of the mutexes and condition variables must be global. As such, they can each be declared as their own global variables, or as part of the state variable. I have no preference which of these you choose, but I do ask that you be consistent -- either have them all as global variables, or include them all in the state variable.