The objectives of this lab experiment are to make you :
An OS controls all the hardware of a computer, and every program currently running. All computer systems, by definition, include an OS. Several modern day gadgets, such as cell phones, MP3 players, wrist watches, PDAs, video game consoles, and TiVO, are specialized computer systems. Many appliances, such as microwave ovens, dishwashers and TVs, contain embedded computer systems.
Currently popular OS for personal computers include Windows, Linux, MacOS, and Solaris. Our discussion in this article is so modern that software known as CP/M (1960s), MS-DOS (1960s) will not qualify as OS.
A computer system can be divided into three pieces:
A typical PC fits this classification very well; but, there are many classes of computers that do not. Some computer systems may not have a key board or a hard disk.
A program is a sequence of CPU instructions (also called machine code) structured in a very specific way. Software is a generic synonym for programs. Applications are a subclass of software.
The CPU can execute its instructions only when they are in RAM. We have so many programs that it is infeasible to leave them all sitting in RAM. The OS stores them on HDD and loads them into RAM as needed.
Fetch, decode, and execute cycle of a CPU.
CPU can generate traps and faults.
Devices raise interrupts.
The moment a computer system is powered on, the firmware is in control of every piece of the hardware. On a PC, this firmware is called the BIOS (basic input output subsystem).
Boot sequence. POST. Locating bootable media. Loading the MBR. Loading the OS boot loader.
From the book by Silbershatz and others: "What are the three main purposes of an operating system? Answer: (i) To provide an environment for a computer user to execute programs on computer hardware in a convenient and efficient manner. (ii) To allocate the separate resources of the computer as needed to solve the problem given. The allocation process should be as fair and efficient as possible. (iii) As a control program it serves two major functions: (a) supervision of the execution of user programs to prevent errors and improper use of the computer, and (b) management of the operation and control of I/O devices.
There are many definitions of OS based on its typical functionality.
A very useful definition of an OS is based on its components. If we were to spread out all the code that constitutes an OS on a piece of cloth, we can then cut it up into the following components:
OS=VM + FileM + ProcM+Net+IO+UserM +(Boot)+[SysPrg]
Strictly speaking, an OS does not include GUI, nor CLI. In the world of Linux, the GUI is provided by X11, Gnome and KDE (http://en.wikipedia.org/wiki/Kde) software subsystems, and CLI by bash, csh, zsh or other such shells. These are not strictly part of the Linux OS. In the Windows world, the GUI is provided by (plain) Explorer, and the CLI by cmd or PowerShell. These are not strictly part of the Windows OS.
Programs like Word, Internet Explorer, Konqueror are applications. Visual Studio, g++, Active Python and such are (sub-) categorized as development tools.
The above components can also be combined and then re-divided into a kernel + initrd + drivers + interrupt handlers.
The code of SystemCalls is not disjoint from the code of previously mentioned components. It is made of different pieces from these components. Linux and Windows XP have around 350 system calls.
Reboot to do warm boot Warm skips first few steps.
Ubuntu and other distributions use UUID (Universally Unique Identifier) to indicate the root device. From the Wiki: In its canonical form, a UUID consists of 32 hexadecimal digits, displayed in 5 groups separated by hyphens, for a total of 36 characters. For example: 550e8400-e29b-41d4-a716-446655440000 .
See Required Reading below.
There are way too many (about a thousand?) distributions. No one has a complete list, but the DistroWatch site is quite extensive. The "major" distributions are (in no particular order): Debian (Ubuntu), RedHat, SuSE, Gentoo, and SlackWare. Most of the minor distributions are specializations of these.
We are using an ISO customized for OSIS Lab usage. It is about 3.5 GB in size. Its lineage is: osislablinuxlive.iso ⇒ Linux Mint ⇒ Kubuntu Linux ⇒ Debian.
Windows 3.1, 95, 98 are from one code lineage. The NT, 2000, XP, Vista and Windows 7 are from a different lineage.
There are two ways of installing an OS onto a USB mass storage drive (USBD). This assumes that your laptop/desktop is recent enough to permit booting from a USBD.
Insert the USBD and start the OS installation. During the install, select the USBD as the target drive. Modern OS installation procedures will discover the USBD as if it is a normal HDD, and let you partition, format, and mount the partitions. This has the advantage of higher speed, but requires a larger USBD.
Boot into a host OS. It can be Linux or Windows. Transfer the content of an OS LiveCD onto the USBD. Then, install an OS boot loader on the USBD. Most live CDs have their larger files compressed, and before they can be used must be uncompressed, either before hand or on the fly as needed. So, this does not have the same speed as an installation done as in the above. But, the required size capacity of the USBD is no more than that of the live CD. Knoppix Live CD compresses about 2 GB worth of programs onto a 700+ MB CD. There are live CDs ranging from 50 MB to 760 MB. There are live CDs of many OS: Linux, FreeDOS, FreeBSD, OpenBSD, Solaris, and even Windows XP. There are now a few live DVDs.
In this lab, we will be following method 2, using Linux as the host OS. You will be installing a Linux distribution on your own USBD, which should be at least 1 GB in size. Be aware that all its content will be destroyed during this lab. (Note that some (old) USBD are known to have hardware issues when used as bootable devices.)
In this lab, you will be invoking some commands that are
privileged because they have a destructive possibility. You are
expected to look up the man pages of sudo, fdisk, mkfs.ext2,
mkfs.vfat, and grub. Be extremely
careful in what arguments you supply to these commands; double-check
before you press return.
Unless a PC has disabled booting from a USBD, this project lets you boot from it and become the privileged user named root.
[Partitioning the USBD] All mass storage devices can be viewed as a long sequence of sectors/blocks. These devices can read/write in units of blocks only; they cannot read/write single bytes. A consecutive section of blocks is a partition. A large hard disk can be divided into several partitions. A small disk will be made into single partition. There are primary, extended and logical partitions. For simplicitly, we divide the USBD into two primaries.
We make a VFAT volume on the first partition of the USBD so that the USBD can be readily used in Windows also.
Linux and hence the partitioning program "think" that the USBD is a HDD, so the use of terms like cylinders etc. should be discounted.
The partions on a 4 GB USBD should look like the following. Your numbers may not match exactly with these. A block is 1 kB.
Disk /dev/sdc: 4009 MB, 4009754624 bytes 126 heads, 22 sectors/track, 2825 cylinders Units = cylinders of 2772 * 512 = 1419264 bytes Disk identifier: 0x000a8f3b Device Boot Start End Blocks Id System /dev/sdc1 1 283 392227 c W95 FAT32 (LBA) /dev/sdc2 284 2825 3523212 83 Linux
[Making File Volumes on USBD] Before any thing can be stored on a partition, it needs to be initialized. This is known as making a file volume. Formatting is a lower level activity, but in the Windows world this often means making a file volume.
[Mounting ] To access the files on a volume, it must be "mounted." Note that files are persistent on a USBD or HDD, but are not "on-line" until mounted.
[Copy files of LinuxOS] Typical commands that you are familiar
with such as ls, mkdir, bash, and the KDE etc. are all
packed into one file named filesystem.squashfs. We
can crudely describe this as a zip archive. More precisely, it
is the compressed file-system image containing many files. The Linux
kernel consists of two files: vmlinuz and the initial
compressed-ram-disk file initrd.gz.
This section describes a crucial step, enabling an OS boot loader, in our lab procedure. We are using GRUB as our OS loader.
sudo cp -rapv /boot/grub /tmp/U
The boot menu that GRUB will display is dictated by the file given
by /tmp/U/grub/menu.lst. Each "stanza" for this
boot loader menu begins with the title-line. The
kernel line specifies the file to load as the kernel,
and the parameters that GRUB should pass-on to the kernel. The
initrd line specifies the initial ram disk file.
This is expanded by the kernel during the first stage of the kernel
boot.
Save the following as a plain text file named
/tmp/menu.lst.
default 1 color green/black yellow/blue title Booting via Grub from OSIS Lab Linux Live USBD title OSIS Lab Linux Live USBD in Probed Graphical Mode kernel /casper/vmlinuz file=/cdrom/preseed/custom.seed boot=casper splash noprompt initrd /casper/initrd.gz title OSIS Lab Linux Live USBD in Safe Graphical Mode kernel /casper/vmlinuz file=/cdrom/preseed/custom.seed boot=casper xforcevesa splash initrd /casper/initrd.gz title OSIS Lab Linux Live USBD in Text Only Mode kernel /casper/vmlinuz file=/cdrom/preseed/custom.seed boot=casper textonly quiet initrd /casper/initrd.gz title Check the CD/DVD for defects kernel /casper/vmlinuz boot=casper integrity-check splash initrd /casper/initrd.gz title Non-Stop Test Memory until ESC key is pressed kernel /boot/memtest title Boot the First Hard Disk root (hd0) chainloader +1
menu.lst file into the
/tmp/U/grub directory:
sudo cp /tmp/menu.lst /tmp/U/grub/
Please become familiar with what you should see in installing GRUB on the USBD before attempting this lab. On the first sign of unexpected output, stop and call one of the lab helpers to trouble shoot.
GRUB presents itself below as an interactive shell with
grub> prompt. The rest of what is on the prompt
is typed by me/you. Output of GRUB is indented from the left
margin by one space. We inform GRUB that the root of our
OS installation is the second hard disk (hd1; GRUB counts starting
from 0), partition 2 (the 1 after comma).
The find command is used solely to make sure that we
are about to install the boot loader on the USBD and not the main
hard disk; the root (hd1,1) you chose
must match with the output of find. Invoke
Grub as:
sudo grub
The prompt by this program is "grub>". Note that there is a space after the first token in the commands to grub.
grub. Your session may have a few
differences. At the end, the "... succeeded" ought to appear; if not,
some thing went wrong.
Probing devices to guess BIOS drives. This may take a long time. grub> root (hd1,1) grub> find /grub/menu.lst (hd1,1) grub> setup (hd1) Checking if "/boot/grub/stage1" exists... no Checking if "/grub/stage1" exists... yes Checking if "/grub/stage2" exists... yes Checking if "/grub/e2fs_stage1_5" exists... yes Running "embed /grub/e2fs_stage1_5 (hd1)"... failed (this is not fatal) Running "embed /grub/e2fs_stage1_5 (hd1,1)"... failed (this is not fatal) Running "install /grub/stage1 (hd1) /grub/stage2 p /grub/menu.lst "... succeeded Done. grub> quit
Note that in the dialog above we specify (hd1,1) as
the root because we booted from the hard disk (which is therefore
hd0). When you chose the USBD as the boot device,
it becomes hd0 and the hard disk that normally would be
labeled hd0 now becomes hdN for some N > 0.
The OS-on-USBD you produce at the end of this Lab is will work like any other hard disk installed Linux. There are several limitations of course: (i) Due to the smaller size of the USBD compared to typical HDD, the range of installed software is limited. (ii) The speed suffers: (ii-a) USBD are several times slower than HDD, and hence. (ii-b) Also, there may not be a swap partition. (ii-c) On the USBD, we use a compressed read-only file system that is uncompressed piece meal as needed. (iii) Because it is read-only, new or updated software cannot be installed.
Once you boot from the USBD, there may be couple of dialogs about keyboard, time zone, etc. depending on the ISO image we chose.
Some motherboards of desktops/laptops have trouble booting (hangs
or reboots) unless acpi is turned off, etc. So,
experiment with the kernel parameters by interacting with GRUB.
As you shutdown, there may be a dialog about other virtual terminals being active. Just ignore this.
Our procedure in creating this USBD mimics a LiveCD. So, there may be a dialog about removing "the CD" and then pressing Enter. Just press Enter. This can be eliminated by appending the token "noprompt" after the token "splash" to the grub menu stanzas.
Our USBD, the way we constructed it, does not save any customization of appearances, etc. that you may make. So, on the next boot from this USBD these are all lost.
After booting from this USBD, you can mount the first (vfat) partition, and save your work there.
All work is expected, but not required, to be carried out in the Operating Systems and Internet Security (OSIS) Lab, 429 Russ. But, you are welcome to work wherever. Note that use of both Linux and Windows and other software, that may not always be installed in other facilities, may be needed.
The USBD size expected depends on the ISO image we are using. It is best to have at least a 4 GB USBD. It can be much bigger (untested in the TB range).
During this Lab, accept no pop up dialogs that offer to auto-mount the USB drive.
Do all the following in a Linux konsole window.
"Record" below means copying-and-pasting appropriate lines (e.g., (i)
a command and (ii) its output) that appear in this window into a plain
text file named myLabJournal.txt.
[Partitioning the USBD] Insert your USBD. In the OSIS lab, its
device name will be /dev/sdb. If a popup dialog asks you
about mounting, do not click on OK; click on Do-Nothing. When doing
this experiment elsewhere the name of the USBD may be
different. If you use the wrong name, you will be wiping out
what was on the (wrongly) named device. Invoke:
sudo fdisk
/dev/sdb
Delete all existing partitions on your drive. Press d
and type the number of the partition to be deleted. Do this for all
partitions.
We are going to make two primary partitions. Make the first partition taking up all the space on the USB leaving 3500+ MB for Linux.
You type n for new, p for primary, 1 for
partition number, then we specify the needed size.
The size specification can be confusing mostly because we see "cylinders" and this is a USBD. Accept the default start cylinder of 1. Specify the size using the syntax of "+sizeM", that is, the plus-sign, followed by the needed size as a number, then a big M, all without any spaces in them. E.g., "+390M" (without the double quotes).
Make the second partition to be a 3500+ MB primary partition for
Linux. Type n for new, p for primary,
2 for partition number, and accept the defaults.
Make sure that this partition is 3500+ MB in size. In the partition
example above, this second partition came out with 3523212 blocks. If
not, redo by going back to deleting all the partitions, and choosing a
smaller first partition.
Set the type of the first partition to vfat. Type t, 1,
c (or 6 etc).
Record a listing of these partitions. Type p for
print details of the partitions.
Write to the USBD the internal data structures that the above
steps have created: w for write.
[Making File Volumes on USBD] This will take a few seconds.
vfat file volume on the Windows
partition.
sudo /sbin/mkfs.vfat /dev/sdb1mkfs.vfat.
Create an ext2 file volume on the Linux partition:
sudo /sbin/mkfs.ext2 -L casper-rw
/dev/sdb2
Record the output of mkfs.ext2.
Mount OS image:
mkdir -p /tmp/P sudo mount -o loop,ro /var/ISO/osislablinuxlive.iso /tmp/PThis is the source image of the Linux OS image we are installing.
Mount your USBD:
mkdir -p /tmp/U
sudo mount /dev/sdb2 /tmp/U
df -Tdf.
time sudo cp -frapv /tmp/P/casper /tmp/U time sudo cp -frapv /tmp/P/preseed /tmp/U time sudo cp -frapv /tmp/P/.disk /tmp/U
The first copy above is of a 3500 MB directory named casper. This
may take 5 to 20 minutes depending on the speed of your USBD.
Record the time outputs.
[Install GRUB] Follow the GRUB installation section above.
[Verify that all needed files are present] Record the output of
ls -lR /tmp/U
[Done] Unmount your USBD.
sudo umount /dev/sdb2
USBD is ready to be used as a bootable device. Leave the USBD
in the PC. Reboot the machine. At the POST, press F12 for
multi boot device choice. Choose USB Device. Voila!
Symptom: Takes a long while, and finally you see an "initramfs" prompt. Cause: The ext2 volume made must have a label of casper-rw, but does not. Fix: Redo the lab starting from Section 5 step 2.
Symptom: Does not recognize USBD as a bootable device. Cause: Grub enabling on the USB might have problems. Fix: It is too hard to verify. Instead, just redo the Grub Installation above.
Symptoms: Varied. Overall, cannot boot into Linux Live. Cause: Not all USBs are "good". Many develop bad sectors (i.e., when we write data D in them and read it back some time later we get some thing different from D) over time. Not a fix, but Linux command "badblocks" can search a device for bad blocks.
Symptom: Windows sees only the first partion. Cause: By Windows policy, I guess. Fix: If you wish to get rid of the two partitions on the USBD, and bring it back to factory default condition, use fdisk to delete the two partitions, make one FAT32 partition occupying the entire device, and write the new partion table back to the device. Invoke mkfs.vfat on that partition.
turnin. Turn in the files
ReadMe.txt myLabJournal.txt answers.txt.
ReadMe.txt how you did it. Prepare as closely as
possible the equivalent of myLabJournal.txt and turn
them in.