Today's Outline

• linux kernel time
• concurrency in the kernel
• atomic operations
• spinlocks, preemption, and interrupts
• semaphores

Timer Code Synchronization

• the system time, variable xtime, has many readers and few writers
• a seq_lock combines a lock and a counter:
  • lock and increment the counter before writing
  • increment the counter and unlock after writing
  • check counter before and after reading: if they are the same and even, the value read is OK
  • if counters are odd or have changed during reading, read again
• synchronization is needed because reading multiple shared variables (e.g. the seconds and nanoseconds of xtime) is only reasonable if none of them has changed

Example system clock race condition

• this is what might happen if access to the clock were not synchronized:
• clock interrupt handler:

    xtime.tv_nsec += 1000000000 / HZ;      /* increment by one tick */
    while (xtime.tv_nsec >= 1000000000) {  /* roll over any ns > 10^9 */
      xtime.tv_sec++;
      xtime.tv_nsec -= 1000000000;
    }
  

• gettimeofday system call:

    return_value->tv_sec = xtime.tv_sec;
    /* timer interrupt here will lead to incorrect return value.
       for example, successive reads may return such times as
       10.999s, 10.001s, 11.001s, .... */
    return_value->tv_usec = xtime.tv_nsec / 1000;
  

• similar things (e.g. the clock incrementing from 10.999s to 11.999s to 11.001s) can happen if gettimeofday reads the values in the opposite order
• since time should never go back, it is important to use the seq_lock to repeat the reading until the time can be read in a stable manner
• see Love, p. 153, which also compensates for the case where jiffies has been updated but wall_jiffies has not yet been updated

Sources of Concurrency in the Kernel

• interrupts may cause code to run interleaved (unless interrupts are disabled)
• kernel preemption may cause different tasks to run interleaved while in the kernel (preemption is originally caused by an interrupt)
• sleeping allows other tasks to run while this task is suspended
• synchronization with user space (blocking) does the same
• SMP provides true concurrency
• Love, p. 124 (p. 111 in the 1st edition)

integer atomic operations

• special type atomic_t, integer with at least 24 bits
• atomic_read to convert it to an integer
• change value atomically:
  • atomic_set to set the value
  • atomic_add to add a value
  • atomic_inc to add 1, atomic_dec to subtract 1
• change and test the value atomically:
  • atomic_sub_and_test returns true if the result is zero
  • atomic_add_negative returns true if the result is negative
  • atomic_dec_and_test returns true if the result is zero
  • atomic_inc_and_test returns true if the result is zero

bit atomic operations

• work on any bit of any pointer (e.g. bit 300 of a char * is an acceptable specification)
• change value atomically:
  • set_bit(int bit, void * pointer) to set the bit to 1
  • clear_bit(int bit, void * pointer) to set the bit to 0
  • change_bit(int bit, void * pointer) to complement the bit
• test or test and set:
  • test_and_set_bit (int bit, void * pointer) returns the old value
  • test_and_clear_bit (int bit, void * pointer) returns the old value
  • test_and_change_bit (int bit, void * pointer) returns the old value
  • test_bit (int bit, void * pointer) returns the value
• non-atomic versions also exist -- prefix two underscores to the function (or macro) name

spinlocks, preemption, and interrupts

• spin lock could be implemented via a test_and_set or atomic_sub_and_test
• spin_lock and spin_unlock
• if:
  • an interrupt handler tries to acquire a spin lock that is locked,
  • it will spin forever (except possibly on an SMP)
  • so if the lock may be used by an interrupt handler, need to suspend interrupts before acquiring the lock:
  • spin_lock_irqsave and spin_unlock_irqrestore
  • can be called with interrupts already disabled
  • so spin_lock_irqsave and spin_unlock_irqrestore are what should be used in most cases

semaphores

• semaphore down() operation decrements it, blocking if zero, and up() increments the variable or wakes up one waiting thread
• since may block, cannot be used in interrupt or soft IRQ or tasklet code
• a binary semaphore is called a mutex in Linux: DECLARE_MUTEX for a static mutex or init_MUTEX for a dynamic mutex
• a counting semaphores is initialized with a value, allows that many simultaneous locks: DECLARE_SEMAPHORE_GENERIC for a static semaphore or sema_init for a dynamic semaphore
• for both, use down_interruptible to acquire
• for both, use up to release
• down will suspend the process and not allow it to be interrupted, i.e. not awaken it when there is a signal, so should be used sparingly