Why kernel programming

To do so, you must have a proper kernel workspace environment set up, which we have successfully done in this chapter. Now that our environment is ready, let's move on and explore the brave world of Linux kernel development! The next two chapters will teach you how to download, extract, configure, and build a Linux kernel from source. To help you delve deeper into the subject with useful materials, we provide a rather detailed list of online references and links and at times, even books in a Further reading document in this book's GitHub repository.

Kaiwan has worked on many aspects of the Linux system programming stack, including Bash scripting, system programming in C, kernel internals, device drivers, and embedded Linux work. His contributions include drivers to the mainline Linux OS and many smaller projects hosted on GitHub.

His Linux passion feeds well into his passion for teaching these topics to engineers, which he has done for well over two decades now. It doesn't hurt that he is a recreational ultrarunner too. Over 30 recipes to develop custom drivers for your embedded Linux applications. About this book Linux Kernel Programming is a comprehensive introduction for those new to Linux kernel and module development.

Publication date: March Publisher Packt. Pages ISBN Download code from GitHub. This chapter will take us through the following topics, which will help us set up our environment: Running Linux as a guest VM Setting up the software — distribution and packages A few additional useful projects.

Technical requirements You will need a modern desktop PC or laptop. Ubuntu Desktop specifies the following as "recommended system requirements" for the installation and usage of the distribution: A 2 GHz dual core processor or better. Internet access is definitely helpful and required at times. Unless noted otherwise, the code output we show in the book is the output as seen on an x Ubuntu Y ou should realize that due to usually minor distribution — and even within the same distributions but differing versions — differences, the output shown here may not perfectly match what you see on your Linux system.

Running Linux as a guest VM As discussed previously, a practical and convenient alternative to using a native Linux system is to install and use the Linux distribution as a guest OS on a VM. Both of these are freely available. It's just that the code for this book has been tested on VirtualBox 6. Installing a bit Linux guest Here, I won't delve into the minutiae of installing Linux as a guest on Oracle VirtualBox, the reason being that this installation is not directly related to Linux kernel development.

While you install the Linux VM, keep the following things in mind. Let's see how to do this: Our first step is to ensure that our CPU supports virtualization: There are two broad ways to check this while on a Windows host : One, run the Task Manager app and switch to the Performance tab.

Below the CPU graph, you will see, among several other things, Virtualization , with Enabled or Disabled following it. A second way to check on Windows systems is to open a Command window cmd. In Command Prompt, type systeminfo and press Enter. Among the output seen will be the Virtualization Enabled in firmware line. It will be followed by either Yes or No.

Figure 1. Let's see how to do this on an Ubuntu guest session: First, update your Ubuntu guest OS's software packages. You can do so using the following command: sudo apt update sudo apt upgrade Copy. As of the time of writing, Fedora 29 has an issue with the installation of the vboxsf kernel module required for the Shared Folders feature.

If this refuses to work, you can simply transfer files between your host and guest VM over SSH using scp 1 ; to do so, install and start up the SSH daemon with the following commands: sudo yum install openssh-server sudo systemctl start sshd. Sometimes, especially when the overhead of the X Window System or Wayland GUI is too high, it's preferable to simply work in console mode. You can do so by appending 3 the run level to the kernel command line via the bootloader.

However, working in console mode within VirtualBox may not be that pleasant an experience for one, the clipboard is unavailable, and the screen size and fonts are less than desirable. Thus, simply doing a remote login via ssh , putty , or equivalent into the VM from the host system can be a great way to work. I encourage you to do the same: Figure 1. As the details of setting up the emulated Raspberry Pi via QEMU go beyond the scope of this book, we will not be covering it.

In fact, for professional development and product work, the Raspberry Pi is really not the best choice, for several reasons Having said that, as a learning and basic prototyping environment it's hard to beat, with the strong community and tech hobbyist support it enjoys. Setting up the software — distribution and packages It is recommended to use one of the following or later stable version Linux distributions.

The following screenshot shows you the recommended version and the user interface: Figure 1. Ubuntu It is interesting to know that the 5. This is good news: this book's content remains current and valid for years to come! It's important to realize , for maximized security with the latest defenses and fixes , that you must run the most recent Long Term Support LTS kernel possible for your project or product.

Then, follow along: Log in to your Linux guest VM and first run the following commands within a Terminal window on a shell : sudo apt update sudo apt install gcc make perl Copy.

Installing required software packages To install the packages, take the following steps: Within the Ubuntu VM, first do the following: sudo apt update Copy. This book, at times, mentions that running a program on another CPU architecture — typically ARM — might be a useful exercise. If you want to try interesting! To install the required QEMU packages, do the following: For installation on Ubuntu, use the following: sudo apt install qemu-system-arm Copy.

To get the version number on Fedora, just type the preceding command and after typing the required package name here, qemu-system-arm- , press the Tab key twice. It will auto-complete, providing a list of choices. Choose the latest version and press Enter. Installing a cross compiler If you intend to write a C program that is compiled on a certain host system but must execute on another target system, then you need to compile it with what's known as a cross compiler or cross toolchain.

It could even be an x guest VM, no issues, but run our code on an ARM target: On Ubuntu, you can install the cross toolchain with the following: sudo apt install crossbuild-essential-armhf Copy. For Fedora Linux, the same tip as earlier applies — use the Tab key to help auto-complete the command. Important installation notes We will now mention a few remaining points, most of them pertaining to software installation or other issues when working on particular distributions: On CentOS 8, you can install Python with the following command: sudo dnf install python3 Copy.

Additional useful projects This section brings you details of some additional miscellaneous projects that you might find very useful indeed. Let's get started with the well-known and important Linux man pages project. The tldr variant While we're discussing man pages, a common annoyance is that the man page on a command is, at times, too large. All Ubuntu repos have the tldr package. Install it with sudo apt install tldr. Locating and using the Linux kernel documentation The community has developed and evolved the Linux kernel documentation into a good state over many years of effort.

Don't worry if the preceding details are not crystal clear yet. I suggest you first read Chapter 2 , Building the 5. Static analysis tools for the Linux kernel Static analyzers are tools that, by examining the source code, attempt to identify potential errors within it. You can install it on Ubuntu with sudo apt install clang clang-tools. I installed one on my Ubuntu The description on its GitHub page sums it up: It outputs a simple visualization of the complete memory map of a given process in a vertically-tiled format ordered by descending virtual address.

Do note, though, that I am still working on this utility; it's currently under development Feedback and contributions are most appreciated! To use BPF, you will need a system with the following: Linux kernel 4. Why not on our guest Linux VM? You can, when running a distro kernel such as an Ubuntu- or Fedora-supplied kernel. It provides functions like, memory mapping, shared virtual memory, swapping etc. Virtual memory is provided for the user applications by swapping in and out the applications with the RAM and swap space in Disk drive, so that the running application feel that it is having lot more memory available in RAM than it actually dose.

Linux supports various kinds of file systems. The responsibility of the subsystem is to manage the files. All the kernel modules interact with this subsystem. It also interacts with device drivers forming a virtual file called device file.

Both the kernel and the hardware can read into or read from the device file. Thus it forms an interface between kernel and actual hardware. The Inter-Process Communication IPC subsystem provides a means, and controls the communication between various processes. The network interface subsystem provides network access to the linux machine. This subsystem directly interacts with network device driver.

A device driver is a specific code written for accessing and controlling a particular kind of device. It forms a device file through which the user applications can access the device just like accessing a file. So the application developers can do their coding without need to know the hardware details. The device driver code alone forms the largest part of the kernel. The device driver subsystem is again divided into character driver, block driver, network driver etc. Where can we find important piece of codes for these subsystems?

Shared memory IPC ————————————————————————————-. Signal IPC —————————————————————————————————. Pipe IPC ——————————————————————————————————.

IPC manager ————————————————————————————————. Message queue manager ——————————————————————————-. Network device manager ————————————————————————. This way, the -next kernel gives a summary outlook onto what will be expected to go into the mainline kernel at the next merge period. Adventurous testers are very welcome to runtime-test the -next kernel. Users are encouraged to report all bugs that they find in this tool.

For details on how to use the kernel bugzilla, please see:. One of the best ways to put into practice your hacking skills is by fixing bugs reported by other people. If you want to be advised of the future bug reports, you can subscribe to the bugme-new mailing list only new bug reports are mailed here or to the bugme-janitor mailing list every change in the bugzilla is mailed here.

As some of the above documents describe, the majority of the core kernel developers participate on the Linux Kernel Mailing list. Details on how to subscribe and unsubscribe from the list can be found at:. There are archives of the mailing list on the web in many different places. Use a search engine to find these archives. For example:. It is highly recommended that you search the archives about the topic you want to bring up, before you post it to the list.

A lot of things already discussed in detail are only recorded at the mailing list archives. Most of the individual kernel subsystems also have their own separate mailing list where they do their development efforts. Please remember to follow good behavioral habits when using the lists. Though a bit cheesy, the following URL has some simple guidelines for interacting with the list or any list :.

If multiple people respond to your mail, the CC: list of recipients may get pretty large. Make sure you use a mail program that does not mangle spaces and tab characters. A good first test is to send the mail to yourself and try to apply your own patch by yourself.

The goal of the kernel community is to provide the best possible kernel there is. When you submit a patch for acceptance, it will be reviewed on its technical merits and those alone. So, what should you be expecting? Remember, this is part of getting your patch into the kernel. You have to be able to take criticism and comments about your patches, evaluate them at a technical level and either rework your patches or provide clear and concise reasoning as to why those changes should not be made.

If there are no responses to your posting, wait a few days and try again, sometimes things get lost in the huge volume. In a community that is looking for the best technical solution possible, there will always be differing opinions on how beneficial a patch is.

You have to be cooperative, and willing to adapt your idea to fit within the kernel. Or at least be willing to prove your idea is worth it. Remember, being wrong is acceptable as long as you are willing to work toward a solution that is right. It is normal that the answers to your first patch might simply be a list of a dozen things you should correct.

This does not imply that your patch will not be accepted, and it is not meant against you personally. Simply correct all issues raised against your patch and resend it.

The kernel community works differently than most traditional corporate development environments. Here are a list of things that you can try to do to avoid problems:. Another way the kernel community is different than most traditional software engineering work environments is the faceless nature of interaction. One benefit of using email and irc as the primary forms of communication is the lack of discrimination based on gender or race. The Linux kernel work environment is accepting of women and minorities because all you are is an email address.

A man may be named Andrea and a woman may be named Pat. Most women who have worked in the Linux kernel and have expressed an opinion have had positive experiences. The language barrier can cause problems for some people who are not comfortable with English. A good grasp of the language can be needed in order to get ideas across properly on mailing lists, so it is recommended that you check your emails to make sure they make sense in English before sending them.

The Linux kernel community does not gladly accept large chunks of code dropped on it all at once. The changes need to be properly introduced, discussed, and broken up into tiny, individual portions. This is almost the exact opposite of what companies are used to doing. Your proposal should also be introduced very early in the development process, so that you can receive feedback on what you are doing.

It also lets the community feel that you are working with them, and not simply using them as a dumping ground for your feature. A 5 line patch can be applied by a maintainer with barely a second glance.

However, a line patch may take hours to review for correctness the time it takes is exponentially proportional to the size of the patch, or something.