Saturday, 3 December 2016

Understanding Filesystems

Linux root filesystem
•       The organization of data is done using the root file system
•       The program binary files and libraries separated and located into specific directories
•       Basic programs are installed in /bin and /sbin and basic libraries in /lib
•       All other programs are installed in /usr/bin and /usr/sbin and all other libraries in /usr/lib
•        In the past, on Unix systems, /usr was very often mounted over the network, through NFS
•       In order to allow the system to boot when the network was down, some binaries and libraries are stored in /bin, /sbin and /lib
•       /bin and /sbin contain programs like ls, ifconfig, cp, bash, etc.
•        /lib contains the C library and sometimes a few other basic libraries
•        All other programs and libraries are in /usr

Important directories
•       /bin Basic programs
•       /boot Kernel image (only when the kernel is loaded from a filesystem, not common on non-x86 architectures)
•       /dev Device files
•       /etc System-wide configuration
•       /home Directory for the users home directories
•       /lib Basic libraries
•       /media Mount points for removable media
•       /mnt Mount points for static media
•       /proc Mount point for the proc virtual filesystem
•       /root Home directory of the root user
•       /sbin Basic system programs
•       /sys Mount point of the sysfs virtual  filesystem
•       /tmp Temporary files
•       /usr /usr/bin Non-basic programs
•       /usr/lib Non-basic libraries
•       /usr/sbin Non-basic system programs
•       /var Variable data files. This includes spool directories and files, administrative and logging data, and transient and temporary

Library components
Proc file system
•       The proc virtual filesystem exists since the beginning of Linux
•       It allows
–      The kernel to expose statistics about running processes in the system
–      The user to adjust at runtime various system parameters about process management, memory management, etc.
•       The proc filesystem is used by many standard userspace applications, and they expect it to be mounted in /proc
•       Applications such as ps or top would not work without the proc filesystem
•       Command to mount /proc:
–      mount -t proc nodev /proc
•       Documentation/filesystems/proc.txt in the kernel sources
•       man proc

/proc contents
•       One directory for each running process in the system
–      /proc/<pid>
–      cat /proc/3840/cmdline
–      It contains details about the les opened by the process, the CPU and memory usage, etc.
•       /proc/interrupts, /proc/devices, /proc/iomem, /proc/ioports contain general device-related information
•       /proc/cmdline contains the kernel command line
•       /proc/sys contains many files that can be written to to adjust kernel parameters
–      They are called sysctl. See Documentation/sysctl/ in kernel sources.
–      Example
•       echo 3 > /proc/sys/vm/drop_caches

Sysfs filesystem
•       The sysfs filesystem is a feature integrated in the 2.6 Linux kernel
•       It allows to represent in userspace the vision that the kernel has of the buses, devices and drivers in the system
•       It is useful for various userspace applications that need to list and query the available hardware, for example udev
•       All applications using sysfs expect it to be mounted in the /sys directory
•       Command to mount /sys:
–      mount -t sysfs nodev /sys
–      $ ls /sys/
block bus class dev devices firmware fs kernel

Basic applications
•       In order to work, a Linux system needs at least a few applications
•        An init application, which is the first userspace application started by the kernel after mounting the root filesystem
•       The kernel tries to run /sbin/init, /bin/init, /etc/init and /bin/sh.
•       If none of them are found, the kernel panics and the boot process is stopped.
•       The init application is responsible for starting all other userspace applications and services
•       Usually a shell, to allow a user to interact with the system
•       Basic Unix applications, to copy les, move les, list files (commands like mv, cp, mkdir, cat, etc.)
•       Those basic components have to be integrated into the root filesystem to make it usable

Overall booting process

Building root filesystem
•       Basic root file system structure
–      The top level directories in the root file system each have a specific use and purpose.
–      Common top-level directories:
–      /bin/, /boot, /dev, /etc, /home, /lib, /proc, /root, /tmp, /usr, /var etc
•       To work on the root filesystem, let's move into the directory we created for this purpose:
                $ cd ${PRJROOT}/rootfs
•       We now create the core root filesystem directories required for our system:
•       $ mkdir bin dev etc lib proc sbin tmp usr var
•       For the various application packages
•       We can proceed with the creation of the /usr hierarchy:
•       $ mkdir usr/bin usr/lib usr/sbin
•       The last entries to create are in the /var directory:
•       $ mkdir var/lib var/lock var/log var/run var/tmp
•       With the root filesystem skeleton now ready, let's place the various software components in their appropriate locations.
•       Placa all the libraries including glibc and application libraries in /lib dir
•       Copy the compiled kernel modules in lib dir
•       cp “all_kernel_modules.ko” ${PRJROOT}/rootfs/lib/modules
•       Create the device files in ${PRJROOT}/rootfs/dev dir using mknod command
•       # mknod -m 600 mem c 1 1
•       # mknod -m 666 null c 1 3
•       # mknod -m 666 zero c 1 5
•       # mknod -m 644 random c 1 8

Main system applications
•       Beyond the kernel's functionality and the root filesystem's structure, Linux inherits Unix's very rich command set.
•       We can decide what all commands we need to include into the target box.
•       There are two approaches          
–      Select and cross compile each of the application, ie complete standard application
–      Use busybox

Complete standard application
•       If you would like to selectively include some of the standard applications found in main stream distributions, your best bet is to start with the Linux From Scratch project located at http://www.linuxfromscratch.org/.
•       This project aims at providing explanations and links to packages to help you build your own custom distributions.

Busybox
•       A Linux system needs a basic set of programs to work
–       An init program
–       A shell
•       Various basic utilities for le manipulation and system configuration
•       In normal Linux systems, those programs are provided by different projects
–      coreutils, bash, grep, sed, tar, wget, modutils, etc. are all different projects
–      A lot of different components to integrate
–      Components not designed with embedded systems constraints in mind: they are not very configurable and have a wide range of features
•       Busybox is an alternative solution, extremely common on embedded systems

General purpose toolbox: Busybox
•       Rewrite of many useful Unix command line utilities
–      Integrated into a single project, which makes it easy to work with
–      Designed with embedded systems in mind: highly configurable, no unnecessary features
•       All the utilities are compiled into a single executable, /bin/busybox
–      Symbolic links to /bin/busybox are created for each application integrated into Busybox
•       http://www.busybox.net/
•       Although BusyBox does not support all the options provided by the commands it replaces, the subset it provides is sufficient for most typical uses

Configuring busybox
•       Get the latest stable sources from http://busybox.net
•       Congure BusyBox (creates a .config le):
–      make defconfig
                Good to begin with BusyBox. Congures BusyBox with all options for regular users.
•       make allnoconfig
–      Unselects all options. Good to congure only what you need.
•       make xconfig (graphical, needs the libqt3-mt-dev package) or make menuconfig (text)
–      Same conguration interfaces as the ones used by the Linux kernel (though older versions are used).

Compiling Busybox
•       Using “make menuconfig” command you can choose the commands to compile and the commands options and features that you need.
•       Set the cross-compiler prefix in the conguration interface:
–      BusyBox Settings -> Build Options -> Cross Compiler prefix
–      Example: arm-linux-
•       Set the installation directory in the conguration interface:
–      BusyBox Settings -> Installation Options -> BusyBox installation prefix
•       Add the cross-compiler path to the PATH environment variable:
–      export PATH=/usr/xtools/arm-unknown-linux-uclibcgnueabi/bin:$PATH
•       Compile BusyBox:
–      Make
–      Install it (this creates a Unix directory structure symbolic links to the busybox executable):
•       make install

Using busybox commands
•       To understand how best to use BusyBox, let's first take a look at the components installed on the target's root filesystem by BusyBox's build process. As observed there is  only one executable was installed, /bin/busybox.
•       This is the single binary with support for all the commands configured  This binary is never called directly, however. Instead, symbolic links bearing the original commands' names have been created to /bin/busybox. Such symbolic links have been created in all the directories in which the original commands would be found, including /bin, /sbin, /usr/bin, and /usr/sbin.
•       When you type a command during the system's normal operation, the busybox command is invoked via the symbolic link. In turn, busybox determines the actual command you were invoking using the name being used to run it. /bin/ls, for instance, points to /bin/busybox.
•       When you type ls, the busybox command is called and it determines that you were trying to use the ls command, because ls is the first argument on the command line

Busy box
•       Busy box provides all the required functionality while still remaining a very small-size application
•       BusyBox implements many commands. Here are a few:
•        ar, cat, chgrp, chmod, chown, chroot, cp, cpio, date, dd, df, dmesg, dos2unix, du, echo, env, expr, find, grep, gunzip, gzip, halt, id, ifconfig, init, insmod, kill, killall, ln, ls, lsmod, md5sum, mkdir, mknod, modprobe, more, mount, mv, ping, ps, pwd, reboot, renice, rm, rmdir, rmmod, route, rpm2cpio, sed, stty, swapon, sync, syslogd, tail, tar, telnet, tftp, touch, traceroute, umount, uname, uuencode, vi, wc, which, and whoami.
•       Although BusyBox does not support all the options provided by the commands it replaces, the subset it provides is sufficient for most typical uses.


No comments:

Post a Comment