Operating Systems Project 2

The goals of this project are:
  1. to install a Linux you can use in future projects
  2. to customize the kernel
  3. to verify that you can print from the kernel (and therefore have at least rudimentary debugging capabilities)
  4. to see what is involved in adding a new system call to Linux

This project should be done by each person in the class, whether individually, or collaborating with others in the class. At the end of the project, you should have your own installation of Linux that can be used in future projects.

You may use any libraries that are part of the standard Linux distribution.

The project should be sent to the instructor by email. Please send me the following:

  1. a brief statement that you have successfully installed linux on your target system, and that you are able to execute programs and recompile and reinstall the operating system. If you are not able to do so, a brief description of what you have been able to achieve.
  2. a brief description of your installation, including whether on a simulator (and if so, which one) or on what hardware, and if on actual hardware, whether you have a dedicated machine or a machine that multi-boots linux and other operating systems. Also (whether on real hardware or on a simulator), the size of your disk and the amount of memory available to the OS.
  3. a copy of the parts of the file fs/exec.c modified as explained in Section 2. Please only send me the differences between the original fs/exec.c, and your modified version, starting two lines before the first change and ending two lines after your last change -- diff -C 2 should do the trick. This may be sent inline or as an attachment.
  4. a copy of the modified parts of the file under kernel/ that you changed in Section 3.

Please send in your project on time -- late submissions will not be accepted, and I prefer to have partially-working projects rather than no project at all.

1. Installation

I expect that most of the effort in this project will go towards installing and configuring Linux.

You must select a target platform, whether a simulator or actual hardware. Simulators include qemu/KVM, virtualbox, Bochs, Plex86 and Vmware. Bochs may be the slowest, but will run on non-x86 platforms.

Install any distribution of linux that includes a complete build environment (gcc, make). If you are uncertain which to install, you may try ubuntu, fedora, debian, or Open Suse. Whichever distribution you install, if you have a choice, I encourage you to install the smallest (minimal) available version of the distribution. You may choose any other linux distribution, but be aware that the instructor may have limited experience with it. Please start early -- an install may take the better part of a day, or longer if you have slow internet (hint: UH has fast internet) or have to do it more than once.

After installing linux, this would be a good time to browse and explore the /proc directory on your system.

In your newly-installed linux, create a new directory (for example, src). Then, still in your new linux, use a browser to go to kernel.org and download the tarball for version 4.20.5, which is the latest stable kernel. Again, you may download a different version, but you'll be on your own as far as getting help from the instructor or your classmates.

Save the tarball in your src directory, and run:

tar xvJf linux-4.20.5.tar.xz
The tar archive will be unpacked in a sub-directory called linux-4.20.5.

Detailed instructions for configuring and compiling linux are at the kernel.org README. You have to read carefully and use your judgement to decide what parts apply to you, and which don't. You might also enjoy reading How to Compile the Linux Kernel, from wikihow. Both documents have parts that are out-of-date, so use them with caution.

If, at any point, you run a command that fails, read the error messages carefully and try to figure out what to do. On my minimal installation of ubuntu, I had to install "gcc", "make", "bison", "flex", "libncurses-dev", "qt5-default" (or "qt5"), and "pkg-config" just to get the configuration going. "libssl-dev" and "libelf-dev" were also needed, based on error messages and warnings that I saw (make sure you can compile the kernel with as few warnings as possible).

Sample invocation:

sudo apt install gcc make bison flex qt5-default pkg-config libssl-dev libelf-dev libncurses-dev
On a fedora system, I would use yum instead of apt.

Once you have explored the configuration options with "make xconfig" or "make menuconfig", I suggest that you run "make mrproper; make localyesconfig" (make mrproper deletes any existing .config file). I suggest localconfig or localyesconfig because every other configuration I tried, including "make tinyconfig", failed to boot in my qemu setup. Once you have run make localconfig, you can re-run "make xconfig" to tweak the setup.

Note that on my system, "make mrproper; make menuconfig" and taking all the default options, took over 4 hours to compile (see also here), and the result filled up 19 GB of disk space on the guest. The installed modules took an additional 5.1GB, meaning you might need to have a total of 25GB of free disk space before you build.

You may also want to consult the wikihow guide to configuring the Linux kernel

Once you are happy with your configuration, typing

make modules
should build the kernel.

If your system uses grub to boot, now would be a good time to edit /etc/default/grub (use "sudo your-editor /etc/default/grub") to set GRUB_TIMEOUT_STYLE=menu and GRUB_TIMEOUT=5 (5 seconds, or any other value greater than 0). When you make install, this file will be used to generate the grub menu. The menu might be very useful to boot an older kernel if the kernel you compiled is broken.

If you ever change anything in the /boot directory (where your kernel and initial ramdisk reside), and if your system uses grub, running update-grub after the change is a very good idea.

Also run uname -a to verify the kernel version number of your installation. It will likely be different from the version you are compiling, 4.20.5. This gives you an easy way to tell which kernel you are running. I also usually modify the local version setting (under "General setup") to add a string which helps me identify the kernel.

The next step is

sudo make modules_install
sudo make headers_install INSTALL_HDR_PATH=/usr/local
sudo make install
(if you have built the tinyconfig, you have no modules -- do not make modules_install, and you will get an error, which you may ignore, when you make install, about the missing modules directory).

You may have questions while doing this. The documentation, the web, the course mailing list, and the instructor often have some of the answers.

2. Testing your kernel hacking abilities

I want you to print the name of every command being executed. To do this, you have to modify file fs/exec.c appropriately, then send me your changes (not the entire exec.c file, please) and the 2 lines before and after your changes (my own solution adds a single line, so I would send 5 lines total). You may find the command diff -C 2 exec.c old_exec.c useful for this.

For example, if a process executes /bin/ls, your code should print to the console "executing /bin/ls". This mainly confirms that you are executing the kernel you think you are executing, yet not get in the way of running Linux.

If you use printk to print, your output is almost certain to be under /var/log -- for me it was in /var/log/syslog. This command showed that my code was working correctly:

tail -f /var/log/syslog
printk can be used in the same way as printf, but unlike printf, may be used in the kernel.

After making these changes, recompile and make sure your kernel is printing each command that you execute.

Finally, once you have verified that everything works, send me the changes to exec.c, then change them back so you have a "normal" operating system that you can use in future projects.

3. Adding a system call

Follow the instructions here (beginning with "Generic System Call Implementation", and ending before "Compatibility System Calls") to create a new system call,
unsigned int my_getpriority (void);
(note: at least one student reported that long my_getpriority (void); works better.

The priority is part of the struct task_struct structure, defined in include/linux/sched.h (in linux, the term "task" is used to refer to both processes and threads). For an example of how to access the task structure of the current process, see the do_exit function in kernel/exit.c (many other system calls have similar code).

The linux task structure has 4 different priority fields. Please pick one that changes when you run with nice.

I suggest you add your code to one of the existing source files under kernel/. Alternatively, you may add a new file, but then you have to figure out the makefile system.

Once you've done this correctly and installed your modified kernel and rebooted, you should be able to compile and run the my_getpriority.c program.

4. Optional

If you have the time and interest, you may do parts this project for Minix as well. For parts 1 and 2 you may follow these instructions, and for part 3, these instructions, and for part 3,

Likewise, you may do this for OpenBSD. As for seL4, one of the goals of the OpenBSD project is a more secure operating system. Note that you will have to figure things out yourself, I don't have detailed instructions for OpenBSD.

This part is optional, and will not be graded, but will be reviewed if you submit it.

If you would prefer to do get credit for installing and modifying a Minix or BSD OS rather than Linux, please talk with the instructor (in person, f2f) as far in advance as possible.