College of Engineering & CS
Wright State University
Dayton, Ohio 45435-0001 |
Hardening an
Operating System
|
| |
|
|
| |
| Abstract: We distinguish hardening a system
from proper configuration and fortification. In this
lecture, we describe areas of tightening security during the
design and construction of systems rather than after their
deployment. |
| |
| This work is supported in part by NSF DUE-9951380. |
|
- Educational Objectives
- Hardening an Operating System
- Careful Recompilation
- Patching a Kernel
- Lab Experiment
- Acknowledgements
- References
- Understand hardening in contrast to proper configuration and fortification
of a system.
- Become familiar with what makes and OS trusted and/or secure.
Hardening an Operating System
We distinguish, even though it is not common yet, hardening a system
from proper configuration and fortification. In this lecture, we describe
areas of tightening security during the design and construction of systems
rather than after their deployment.
Careful Recompilation
Stack smashing (buffer overflow) attacks are among the most common.
By and large, these are programming errors that can be caught by analytical
techniques. Newer compilers are mechanizing these techniques. For
example, such vulnerabilities are carefully checked by the Immunix's "StackGuard"
enhancements to the GNU egcs compiler. An entire RedHat distribution has
been recompiled with this compiler resulting in the distribution of Immunix.
Patching a Kernel
Patches are often released in binary form, but they they can also be at the
source code level. These replace section(s) of code in the kernel.
Often a patch is in response to a newly discovered security hole. There
are proactive modifications also in the open source OS. For example, LIDS
(Linux Intrsion Detection System) is a series of kernel patches that
enable loadable module and mount point locking [www.soaring-bird.com.cn/oss_proj/lids/].
Secure Operating Systems
There are "secure operating systems" and "trusted operating
systems." Alas, no one can offer technically rigorous definitions for
these terms. For example, we found the following quote, "Windows NT
is a secure operating system but only if it's configured correctly."
Nevertheless, operating systems that lay claim to either being secure or trusted
are better designed and engineered from their inception with a concern for
security. Both NT, several of Unix derivatives claim to be secure
and trusted.
In a later lecture we will discuss the so-called Orange Book, and its
classification of levels of security.
Trusted Operating Systems
The following is from a white paper by a company that sells a trusted
operating system [www.argus-systems.com/]
It offers concrete enough details that we can ignore the specific slant it
obviously has towards a single product.
Least Privilege
On any computer system, certain system programs or utilities must be granted
the ability to bypass the security constraints normally imposed by the system.
For example, in order to create a backup of all files on the system disks, an
administrator must be able to run a backup program that is able to read all
files on the disk, even though the administrator would not normally be allowed
such access. Other powerful programs must also be carefully controlled, such as
the programs to shut down the system, create new users, and repair damaged file
systems. On a standard Unix system, the operating system has been designed so
that one user ID, called root or superuser, can bypass all
security restrictions and limitations. Windows NT systems exhibit similar
vulnerabilities with the "System" and "Administrator"
accounts.
The inability of standard Unix to grant only limited rights to a program is
not the system's only weakness. In Unix, when one program starts another the
newly created program runs with the user ID of the first program. This means
that a malicious user who can exploit a bug in a root program may be able to
start up an interactive root session. If a user is running as root, every
program he runs will have unlimited privileges on the system.
On a trusted system the superuser privilege is broken down into many smaller
privileges. Thus the backup program may be able to read any file, but it cannot
shut down the system, modify files, or send random network packets. The use of
many limited capabilities instead of a single all-powerful mechanism is called
the least privilege principle. On a trusted operating system, programs,
not users, are given privileges. The backup program will have only the
privileges it needs to do its job, and any program that it starts will not
inherit the backup program's privileges. A specific user may have the right to
run certain programs, but the user's session is in no way privileged. The
programs with privileges are few, have limited power, and are carefully
controlled.
The use of least privilege eliminates most of the security problems commonly
reported with standard operating systems. Even if a specific superuser bug may
go unfixed, on a trusted operating system exploitation of the bug will not allow
a malicious user to bypass all system security.
Labeling and Partitioning
The traditional Unix model allows all users to share all system resources,
with access to file system objects limited by the owner of the object. This
model, which is sufficient for systems requiring low levels of security, is
unacceptable for sites that require enhanced levels of security or that cannot
allow system security enforcement to be dependent on the user's voluntary
adherence to site security policy.
In order to create a mechanism allowing users and processes to operate in
isolation, it is necessary to add new security attributes to file system
objects, processes, network interfaces, host addresses, and other system
resources. These new attributes are called sensitivity labels (SLs).
The sensitivity labels can be used to restrict a process or user to either
read-only access to an object or no access at all. Since SLs can not be modified
by users or unprivileged processes, a system's security policy can be enforced
without relying on user compliance. Access control using system-enforced labels
is called mandatory access control (MAC).
Labeling and MAC can be applied at the network interface level. Incoming
packets from a network interface are assigned SLs, which may be based on the
source IP address or the network interface. Outgoing packets will have the label
of the process or daemon that created them. An incoming or outgoing packet will
be dropped if the SL of the packet is not valid for both the interface and the
remote host. In addition, the SL can be inserted into the packet header so that
trusted operating systems can share security information over the network.
Processes, files, and other resources that have the same label are said to be
in the same compartment or partition. Programs, data, and network
interfaces can be split into separate, isolated partitions with restricted
access between them. Packets associated with one compartment cannot be read or
spoofed by applications or network services in another.
Partitioning is a critical security feature when a system will be handling
different classes of users and jobs, such as users from different departments or
network daemons interacting with different interfaces or hosts. The ability to
partition a network server is a key component of PitBull .comPack in providing
the level of assurance needed to support critical network and transaction
servers.
For example, network services, such as HTTP daemons and associated CGI
scripts, can be given read-only access to some files (for example, critical web
pages and CGI directories) and can be completely isolated from other resources
(such as internal network interfaces, system files, and other network daemons).
A network server configured in this way can protect its web pages and CGI
scripts from ever being modified from an external connection, even if damaging
bugs exist in the commercial software that allow incoming users to execute any
random sequence of machine instructions. Access by the network services to other
network interfaces and hosts can be easily limited as well.
Login Enhancements
Trusted operating systems employ enhanced account security features,
including:
- Machine-generated passwords: The computer can provide a list of
randomly generated pronounceable passwords, from which the user must select
one. This feature prevents users from using common words or names as
passwords.
- Long passwords: The maximum length for passwords has been increased
to 31 characters.
- Password encryption algorithms: Trusted operating systems typically
provide a mechanism for replacing the default password encryption algorithms
with alternative algorithms, usually through replacement of dynamic
libraries.
- Login hooks: The identification-and-authorization login process can
be extended to include additional, system-specific authorization checks. An
example of this is PitBull .comPack's Four-Eyes program, which can
force "special privilege" accounts to log in with an additional
account name and password for extra security. Other advanced measures, such
as biometric identification or access tokens, can be accommodated using this
feature.
Authorizations and Privileges
Authorization is an attribute of a user account under a trusted
operating system that enables the user to execute a subset of operating system
applications and utilities. Privileges are attributes assigned to
applications that give them different grades of access to processes or
resources. Together, these two sets of attributes enable programs to behave
differently toward different users. Applications can be extended to define and
check new authorizations, providing a standardized protocol by which
applications and the operating platform can communicate to grant permissions to
users. Authorizations can be used to divide administrative duties into separate
roles, which can then be assigned to different users.
Auditing
Trusted operating systems employ an enhanced auditing mechanism, which
allows applications to trace a wider variety of security-related events than
standard operating systems. The information on these transactions is appended to
an audit trail in an isolated partition, which is protected by both
discretionary and mandatory access control mechanisms. This approach prevents
intruders from covering their tracks and eliminating traces of penetration
attempts. By protecting the audit record, sufficient evidence is maintained to
support litigation and law enforcement actions.
Most IS and corporate managers, already hard-pressed to maintain daily
systems operations, face significant barriers to incorporating new technologies
and adequate systems security. Managers seeking to upgrade security on their
systems are thus often forced to rely on vendor claims of security performance.
As new software emerges and inevitable upgrades to existing software pour in, IS
professionals typically assume that the vendors have a vested interest in the
security of their products. Given the potential implications of security system
failure, it is critical that managers concentrate on security solutions that
have undergone independent evaluation, testing, and certification.
Trusted operating systems undergo evaluation of their overall design,
verification of the integrity and reliability of their source code, and
systematic, independent penetration evaluation. One of the most highly respected
evaluation tools is the Information Technology Security Evaluation Criteria (ITSEC),
an internationally recognized set of standards for the evaluation, testing, and
certification of IT security products. ITSEC certification, performed by an
independent body, provides the end user with confidence that the claims made
about the security functionality of a product are valid. In addition, this
certification indicates that these claims have been tested against a
predetermined level of assurance and that the vendor has a high level of
expertise and a strong commitment to security.
none
- Bastille-Linux Scripts to Secure Linux, http://www.sans.org/newlook/projects/bastille_linux.htm
Reference
- http://immunix.org Carefully
re-compiled RedHat distribution. Worth visiting. Reference
- Seán Boran, Hardening Solaris, October 25, 1999, http://securityportal.com/cover/
coverstory19991025.html Reference.
- Michael Espinala, "The Hardening of Microsoft Windows NT Operating
System Version 4.0", March 1998. Proper configuration,
actually. Reference.