This lab will install a Linux distribution that can bee booted and run entirely from a USB drive. The Linux system that is so-installed is a full-scale system, suitable for all the CS/CEG courses of WSU. This lab is done early in the term so that all the students have "their own" Linux that they can run on home PCs without disturbing Windows or other software. We will be using the term process a few times. You will understand processes better in a couple of weeks.
This term our installation uses the Knoppix 6.7.1 Live DVD image., and Android 2.2 Froyo.
The objectives of this lab experiment are to make you :
Unless a PC BIOS has disabled booting from a USBD, this lab lets you boot from the USB drive and become the privileged user named root.
An OS controls all the hardware of a computer, and every program currently running. All computer systems, by definition, include an OS. Several modern day gadgets, such as cell phones, MP3 players, wrist watches, PDAs, video game consoles, and TiVO, are specialized computer systems. Many appliances, such as microwave ovens, dishwashers and TVs, contain embedded computer systems.
Currently popular OS for personal computers include Windows, Linux, MacOS, and Solaris. Our discussion in this article is so modern that software known as CP/M (1960s), MS-DOS (1960s) will not qualify as OS.
A computer system can be divided into three pieces:
A typical PC fits this classification very well; but, there are many classes of computers that do not. E.g., some computer systems may not have a key board or a hard disk.
A program is a sequence of CPU instructions (also called machine code) structured in a very specific way. Software is a generic synonym for programs. Applications are a subclass of software.
The CPU can execute its instructions only when they are in RAM. We have so many programs that it is infeasible to leave them all sitting in RAM. The OS stores them on a HDD and loads them into RAM as needed.
We will explain in the lectures the "Fetch, Decode, and Execute cycle" of a CPU.
CPU can generate traps and faults.
Devices raise interrupts. Interrupt handling is done by the OS.
The moment a computer system is powered on, the firmware is in control of every piece of the hardware. On a PC, this firmware is called the BIOS (basic input output subsystem).
A computer system must not be shut down "ungracefully". All the running processes need to be informed of the impending shut down, and wait politely for them to terminate. This is known as graceful termination of the OS. Most systems can now detect a power failure, and initiate a shutdown. However, un-responding processes not withstanding, a shutdown cannot be postponed indefinitely, and the OS forcibly terminates.
There are many definitions of OS based on its typical functionality. From the book by Silbershatz and others: "What are the three main purposes of an operating system? Answer: (i) To provide an environment for a computer user to execute programs on computer hardware in a convenient and efficient manner. (ii) To allocate the separate resources of the computer as needed to solve the problem given. The allocation process should be as fair and efficient as possible. (iii) As a control program it serves two major functions: (a) supervision of the execution of user programs to prevent errors and improper use of the computer, and (b) management of the operation and control of I/O devices."
A very useful definition of an OS is based on its components. If we were to spread out all the code that constitutes an OS on a piece of cloth, we can then cut it up into the following components:
OS = VM + FileM + ProcM + Net + IO + UserM + (Boot) + [SysPrg]
Strictly speaking, an OS does not include GUI, nor CLI. In the world of Linux, the GUI is provided by X11, Gnome and KDE software subsystems, and CLI by bash, csh, zsh or other such shells. These are not strictly part of the Linux OS. In the Windows world, the GUI is provided by (plain) Explorer, and the CLI by cmd or PowerShell. These are not strictly part of the Windows OS.
Programs like Word, Internet Explorer, Konqueror are applications. Visual Studio, g++, Active Python and such are (sub-) categorized as development tools.
The above components can also be combined and then re-divided into a kernel + initial ram disk components + drivers + interrupt handlers.
The code of system calls is not disjoint from the code of previously mentioned components. It is made of different pieces from these components. Linux and Windows have around 350 system calls each. The details of Linux syscalls can be found in the files:
-rw-r--r-- 1 root root 1403 2010-01-03 06:57 /usr/include/sys/syscall.h -rw-r--r-- 1 root root 22041 2010-01-28 00:56 /usr/include/asm/unistd_64.h
For Windows, see the systems calls list I saved some time ago. For Windows 7 system calls that are related to the syscall.h above, see http://technet.microsoft.com/en-us/library/cc754234.aspx
/bin, /usr/bin, ...: These directories do contain programs, but these are not considered part of the OS. The programs in these directories are considered simply applications.
(From http://technet.microsoft.com/en-us/library/bb457124.aspx) The OS system files are located in the Windows\System32 , Winnt\System32, or Windows\SysWOW64 directory. The startup files are located in the boot or root directory (for example, C:\) of your Windows installation.
Most of the OS stays RAM resident and dormant until (i) a system call is made by a process, (ii) a hardware interrupt causes a handler in the OS to run, or (iii) a software triggered even causes a handler in the OS to run. Consequently, viewing an OS only as a collection of processes is going to be incomplete.
| PID | TTY | STAT | TIME | COMMAND | Functionality |
| 1 | ? | Ss | 0:00 | /sbin/init | The First Process |
| 2 | ? | S< | 0:00 | [kthreadd] | |
| 3 | ? | S< | 0:00 | [migration/0] | Processes shown within brackets are part of the Kernel. |
| 524 | ? | S< s | 0:00 | udevd | Dynamic Device Manager service |
| 1053 | ? | Ss | 0:00 | portmap | RPC port map service |
| 1056 | ? | Ss | 0:00 | dd bs=1 if=/proc/kmsg ... | Kernel message logging |
| 1063 | ? | Sl | 0:00 | rsyslogd | |
| 1088 | ? | Ss | 0:10 | hald | Hardware Abstraction Layer |
| 1357 | ? | S< | 0:00 | [nfsiod] | NFS Serivice IO |
| 1403 | ? | S | 0:00 | hald-addon-cpufreq | CPU frequency |
| 1404 | ? | S | 0:00 | hald-addon-acpi: | ACP listening |
| 1479 | tty4 | Ss+ | 0:00 | /sbin/getty | User login process |
| 1503 | ? | Ss | 0:00 | acpid | |
| 2159 | ? | S | 0:00 | dbus-launch | |
| 2738 | ? | Ss | 0:00 | /usr/sbin/nmbd | Samba Name Service |
| 2748 | ? | Ss | 0:00 | /usr/sbin/smbd | Samba File Service |
For a list of Windows processes, invoke Task Manager and then select Show processes from all users.
We will explain in the lectures: Boot sequence. POST. Locating bootable media. Loading the MBR. Loading the OS boot loader.
1. Powered on. BIOS->POST. ROM and RAM. CPU fetch-decode-execute cycle.Ubuntu and other distributions use UUID (Universally Unique Identifier) to indicate the root device. From the Wiki: In its canonical form, a UUID consists of 32 hexadecimal digits, displayed in 5 groups separated by hyphens, for a total of 36 characters. For example: 550e8400-e29b-41d4-a716-446655440000 .
See Required Reading below.
There are way too many (about a thousand?) distributions. No one (?) has a complete list, but the DistroWatch site is quite extensive. The "major" distributions are (in no particular order): Debian (Ubuntu), RedHat, SuSE, Gentoo, and SlackWare. Most of the other distributions are specializations of these.
We are using a Knoppix DVD ISO. It is about 3.5 GB in size.
Windows 3.1, 95, 98 are from one code lineage. The NT, 2000, XP, Vista and Windows 7 are from a different lineage.
There are two ways of installing an OS onto a USB mass storage drive (USBD). This assumes that your laptop/desktop is recent enough to permit booting from a USBD.
Insert the USBD and start the OS installation. During the install, select the USBD as the target drive. Modern OS installation procedures will discover the USBD as if it is a normal HDD, and let you partition, format, and mount the partitions. This has the advantage of higher speed, but requires a larger USBD.
Boot into a host OS. It can be Linux or Windows. Transfer the content of an OS LiveCD onto the USBD. Then, install an OS boot loader on the USBD. Most live CDs have their larger files compressed, and before they can be used must be uncompressed, either before hand or on the fly as needed. So, this does not have the same speed as an installation done as in the above. But, the required size capacity of the USBD is no more than that of the live CD. Knoppix Live CD compresses about 2 GB worth of programs onto a 700+ MB CD. There are live CDs ranging from 50 MB to 760 MB. There are live CDs of many OS: Linux, FreeDOS, FreeBSD, OpenBSD, Solaris, and even Windows. There are now (2010) over a hundred live DVDs.
In this lab, we will be following method 2, using Linux as the host OS. You will be installing a Linux distribution on your own USBD, which should be at least 1 GB in size. Be aware that all its content will be destroyed during this lab. (Note that some (old) USBD are known to have hardware issues when used as bootable devices.)
In this lab, you will be invoking some commands that are privileged
because they have a destructive possibility. You are expected to
look up the man pages of sudo, fdisk, mkfs.ext2,
mkfs.vfat, and grub. Be extremely
careful in what arguments you supply to these commands; double-,
triple-check before you press return.
This section is not a step-by-step procedure. It explains the behind the scenes reasoning of the steps you will carry out later on. Peculiarly perhaps, but we are not just interested in creating a bootable USBD; we are more interested in understanding the internal structure of what directories and files such a USBD has. There is a lab-oriented step for each of these items in the next section.
[Partitioning the USBD] All mass storage devices can be viewed as a long sequence of sectors/blocks. These devices can read/write in units of blocks only; they cannot read/write single bytes. A consecutive section of blocks is a partition. A large hard disk can be divided into several partitions. A small disk will be made into single partition. There are primary, extended and logical partitions. For simplicity, we divide the USBD into two primaries.
We make a VFAT volume on the first partition of the USBD so that the USBD can be readily used in Windows also.
Linux and hence the partitioning program "think" that the USBD is a HDD, so the use of terms like cylinders etc. should be discounted.
The partitions on a 4 GB USBD should look like the following. Your numbers may not match with these. A block is 1 KB.
Disk /dev/sdb: 3999 MB, 3999268864 bytes 82 heads, 18 sectors/track, 5292 cylinders, total 7811072 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xc3072e18 Device Boot Start End Blocks Id System /dev/sdb1 2048 67583 32768 83 Linux /dev/sdb2 * 67584 7811071 3871744 83 LinuxThe above partition #2 is barely enough; if we try to install in a smaller partition the image we chose this term, we will run out of space.
KNOPPIX. We can
crudely describe this as a zip archive. More precisely, it is
the compressed file-system image containing many files. The Linux
kernel consists of two files: vmlinuz and the initial
compressed-ram-disk file initrd.gz.
The OS-on-USBD you produce at the end of this Lab is will work like any other hard disk installed Linux. There are several limitations of course: (i) Due to the smaller size of the USBD compared to typical HDD, the range of installed software is limited. (ii) The speed suffers: (ii-a) USBD are several times slower than HDD, and hence. (ii-b) Also, there may not be a swap partition. (ii-c) On the USBD, we use a compressed read-only file system that is uncompressed piece meal as needed. (iii) Because it is read-only, new or updated software cannot be installed.
Once you boot from the USBD, there may be couple of dialogs about keyboard, time zone, etc. depending on the ISO image we chose.
Some motherboards of desktops/laptops have trouble booting
(hangs or reboots) unless acpi is turned off, etc.
So, experiment with the kernel parameters by interacting with GRUB.
Our procedure in creating this USBD mimics a LiveCD. So, there may be a dialog about removing "the CD" and then pressing Enter. Just press Enter. This can be eliminated by appending the token "noprompt" after the token "splash" to the grub menu stanzas.
Our USBD, the way we constructed it, does not save any customization of appearances, etc. that you may make. So, on the next boot from this USBD these are all lost.
After booting from this USBD, you can mount the first (vfat) partition, and save your work there.
The various steps are written up as separate subsections below. It is expected that you have understood the previous sections, especially the Overview.
myLabJournal.txt and also
in answers.txt. As usual, add additional
details of your experience in the journal.
All work is expected, but not required, to be carried out in the Operating Systems and Internet Security (OSIS) Lab, 429 Russ. But, you are welcome to work wherever. Note that use of both Linux and Windows and other software, that may not always be installed in other facilities, may be needed.
/dev/sdb. If a popup dialog asks you
about mounting, do not click on OK; click on
Do-Nothing. (When doing this experiment elsewhere the name of the
USBD may be different. If you use the wrong name, you will
be wiping out what was on the (wrongly) named device.)konsole window. Invoke:
sudo fdisk /dev/sdb
d and type the number of the partition
to be deleted. Do this for all partitions. If the USBD was a
new one, there will just be one FAT partition, and the delete
operation will not even ask for the partition number.
See the Troble Shooting notes below regarding the size of the USBD you are using before proceeding.
Type n for new, p for primary,
1 for partition number, then specify the needed
size.
Type t, 1, c (or 6 etc.) to set
the type of the first partition to vfat (or fat).
(The size specification can be confusing mostly because we see "cylinders" and this is a USBD. Accept the default start cylinder. Specify the size using the syntax of "+sizeM", that is, the plus-sign, followed by the needed size as a number, then a big M, all without any spaces in them. E.g., "+80M" (without the double quotes). )
n for new, p for primary,
2 for partition number, and accept the defaults for the next two
choices.p to
print details of the partitions.
Record this listing of partitions. w for
write.
This will take only a few seconds each.
vfat file volume on the Windows partition.
sudo /sbin/mkfs.vfat -n WSUVFAT /dev/sdb1Record the output of
mkfs.vfat.
sudo /sbin/mkfs.ext2 -L WSUKNOPPIX671 /dev/sdb2Record the output of
mkfs.ext2.
mkdir -p /tmp/KnoppixISO sudo mount -o loop,ro /var/ISO/KNOPPIX_V6.7.1*.iso /tmp/KnoppixISOThis is the source image of the Linux OS image we are installing.
mkdir -p /tmp/USBD sudo mount /dev/sdb2 /tmp/USBD df -TRecord the output of
df.time sudo cp -frapv /tmp/KnoppixISO/* /tmp/USBD
The copy above includes a 3900+ MB directory named KNOPPIX
. This may take 15 to 20 minutes depending on the speed of your
USBD. Record the time outputs.
sudo cp -p /boot/memtest86+.bin /tmp/USBD/boot/(Well, ... this is really not part of Linux OS, but useful to have on this USBD.)
(Occasionally this step fails. If you are arriving here in a
second session of this Lab: Do not carry this step out prior to
mounting /tmp/USBD. Do 5.3 Mounting step b
above.)
In this step, we are using legacy GRUB, not GRUB version 2, as the older version is simpler to setup.
sudo cp -rapv ~ceg23300/grub /tmp/USBD/boot
/tmp/USBD/grub/menu.lst. Each "stanza"
for this boot loader menu begins with
the title-line. The kernel line
specifies the file to load as the kernel, and the parameters that
GRUB should pass-on to the kernel. The initrd
line specifies the initial ram disk file. This is expanded
by the kernel during the first stage of the kernel boot.
Save the following as a plain text file
named /tmp/menu.lst.
default 1 color blue/black black/light-gray title Wright State University Linux Knoppix Live USB Drive boot title Android 2.2 Froyo Live kernel /Android/kernel root=/dev/ram0 acpi_sleep=s3_bios,s3_mode SRC=/Android DATA=/Android/Data initrd /Android/initrd.img title Linux Knoppix 64-bit Live USBD in Probed Graphical Mode kernel /boot/isolinux/linux64 noeject noprompt ramdisk_size=100000 lang=en tz=localtime keyboard=us xkeyboard=us initrd /boot/isolinux/minirt.gz boot title Linux Knoppix 32-bit Live USBD in Probed Graphical Mode kernel /boot/isolinux/linux noeject noprompt ramdisk_size=100000 lang=en tz=localtime keyboard=us xkeyboard=us initrd /boot/isolinux/minirt.gz boot title Linux Knoppix Live USBD in FailSafe Mode (even so, might not work) kernel /boot/isolinux/linux ramdisk_size=100000 lang=en tz=localtime vga=normal noapic nolapic xmodule=vesa initrd /boot/isolinux/minirt.gz boot title Non-Stop Memory Test until ESC key is pressed kernel /boot/memtest86+.bin boot title Good Old FreeDOS kernel /boot/isolinux/memdisk initrd /boot/isolinux/balder.img boot title Boot the First Hard Disk root (hd0) chainloader +1
/tmp/menu.lst file.
sudo cp /tmp/menu.lst /tmp/USBD/boot/grub/
GRUB presents itself below as an interactive shell with
grub> prompt. The rest of what is on the
prompt is typed by me/you (shown in italics). Output
of GRUB is indented from the left margin by one space. We
inform GRUB that the root of our OS installation is (hd1,1);
the second hard disk is hd1 (GRUB counts starting from 0),
partition 2 (the 1 after comma).
The find command is used solely to make sure that we
are about to install the boot loader on the USBD and not the main
hard disk; the root (hd1,1) you chose
must match with the output of find. Invoke
Grub as:
sudo grub-legacy
grub. Your session may have a few
differences. At the end, the "... succeeded" ought to
appear; if not, some thing went wrong. Note that there is a
space after the first token in the commands to grub.
Probing devices to guess BIOS drives. This may take a long time. grub> root (hd1,1) grub> find /boot/grub/menu.lst (hd1,1) grub> setup (hd1) Checking if "/boot/grub/stage1" exists... no Checking if "/grub/stage1" exists... yes Checking if "/grub/stage2" exists... yes Checking if "/grub/e2fs_stage1_5" exists... yes Running "embed /grub/e2fs_stage1_5 (hd1)"... failed (this is not fatal) Running "embed /grub/e2fs_stage1_5 (hd1,1)"... failed (this is not fatal) Running "install /grub/stage1 (hd1) /grub/stage2 p /grub/menu.lst "... succeeded Done. grub> quit
Note that in the dialog above we specify (hd1,1) as
the root because we booted from the hard disk (which is therefore
hd0). When you choose the USBD as the boot device,
it becomes hd0 and the hard disk that normally would be
labeled hd0 now becomes hdN for some N > 0.
ls -lR /tmp/USBDsudo umount /dev/sdb2
answers.txt file.
You can skip this step if you are satisfied without earning yet
more bonus points. All the ISO files needed in these Bonus Points steps
have already been downloaded and are available on our local server at /var/ISO. Do
not store your own private copies of any ISO files on OSIS Lab PCs.
If you skip this step, the entry in the Grub menu for Android will do nothing. Obviously.
"Android is an open-source software stack for mobile devices, and a corresponding open-source project led by Google." At its base is Linux. There is a version of Android Froyo that can run on x86 PCs ( www.android-x86.org/). Below are the steps for installing it without disturbing the Knoppix we installed above.
mkdir -p /tmp/AndroidISO sudo mount -o loop,ro /var/ISO/android-x86-2.2-eeepc.iso /tmp/AndroidISO
This is the source image of the Android OS image we are installing.
mkdir -p /tmp/USBD sudo mount /dev/sdb2 /tmp/USBDRecord the output of
df -T
df.Copy the entire image of Android OS:
sudo mkdir /tmp/USBD/Android time sudo cp -frapv /tmp/AndroidISO/* /tmp/USBD/Android
The entire AndroidISO is under 100 MB. So, this will take only a few seconds.
sudo umount /dev/sdb2
/var/ISO/android-x86-4.0-*MeeGo (http://meego.com) is an open source Linux project by Intel, Maemo, Nokia and the Linux Foundation for deployment on mobile device platforms. It can be installed on a typical x86 desktop PC also.
We have already downloaded
versions of MeeGo, and are available to all OSIS Lab machines while
running Linux as /var/ISO/meego-netbook-ia32-*.iso Install it on the
side on your USBD produced above. That is, MeeGo should co-exist
with Knoppix and Android that were already installed. Add a new
entry into the Grub menu.lst that enables booting into
MeeGo.
The details of what you need to do to make this work are left for
you to work out. This is not as easy as setting up the Android
boot! Describe in detail how you did it all in
the myLabJournal.txt and summarize it
in answers.txt file.
Install JoliCloud ( http://www.jolicloud.com/) on your USBD along with the Knoppix above. You will need an 8GB USBD. We already downloaded the image; it is at /var/ISO/Joli-OS-1.2.iso
The details of what you need to do to make this work are left for you to work out.
This list is obviously very incomplete.
So use the older Knoppix 6.4.4 version which is smaller than the 6.7.1 version by about 200MB. In the Mounting section above, replace
sudo mount -o loop,ro /var/ISO/KNOPPIX_V6.7.1*.iso /tmp/KnoppixISOwith
sudo mount -o loop,ro /var/ISO/KNOPPIX_V6.4.4*.iso /tmp/KnoppixISO
ReadMe.txt.
Explain in answers.txt how you did it. Show a record
of your effort in myLabJournal.txt. Turn these in.
turnin. Turn in the files ReadMe.txt
myLabJournal.txt answers.txt.