Security Hardened Kernels for Linux Servers An MS Thesis by Sowgandh Sunil Gadi A thesis submitted in partial fulfillment of the requirements for the degree of Master of Science in Computer Engineering Thesis Advisor: Prabhaker Mateti

• PDF of the thesis
• Slides of the thesis defense talk [ pdf 3 MB]
• Patch file for Linux kernel 2.4.2.3

Abstract

The typical kernels installed by well known Linux distributions are rarely secure. In this thesis, we consider the problem of developing security hardened Linux kernels intended for server machines.

There are many kernel patches aimed at improving kernel security. Unfortunately, none of these have technical explanations of the prevention techniques. This thesis fills this gap and offers detailed explanations of root causes for the exploits and various intrusion techniques used by an attacker, using exploit programs posted in security forums. We explain prevention techniques of known open source kernel patches and the side effects of these patches.

We critically review five independent kernel source code patches, known as OWL, Segmented-PAX, KNOX, RSX, and Paged-PAX, which aim to prevent buffer overflow attacks. We show that two of these patches are ineffective though their ideas are workable. We also discuss the performance impact of these kernel patches.

Our secure kernels prevent many other types of exploits, including {\tt chroot} breaking, temporary file race condition, file descriptor leakage, LKM based rootkits, and {\tt /dev/kmem} rootkits.

Using kernel threads we have designed and implemented a new {\em kernel logger}, and a new {\em kernel integrity checker}. Kernel logger provides secure logging in addition to {\tt syslogd}. Kernel integrity checker can detect on-the-fly kernel modifications as well as yet-to-be discovered attacks. We added {\em trusted path mapping} to the kernel for preventing the execution of binaries from arbitrary path names. We also designed a new feature that enables treating a file system as read-only that works more robustly and more securely than merely mounting it as read-only. We believe that systems whose primary function is to be servers must deploy specially built kernels not only for performance reasons but even more importantly for security reasons. Often, unneeded services are exploited by the intruders. The construction process of our secure kernels provides pruning control at the level of system calls, capabilities, memory devices, network interface configuration, routing table configuration, and ext file system attributes. Many of these items are eliminated at compile-time whereas the remainder is frozen at run-time soon after the initial boot. As a feasibility study, we describe in detail the construction of secure kernels for Anonymous FTP server, Web server, Mail server, and a File server.