Saturday 3 December 2016

Understanding Filesystems

Linux root filesystem
       The organization of data is done using the root file system
       The program binary files and libraries separated and located into specific directories
       Basic programs are installed in /bin and /sbin and basic libraries in /lib
       All other programs are installed in /usr/bin and /usr/sbin and all other libraries in /usr/lib
        In the past, on Unix systems, /usr was very often mounted over the network, through NFS
       In order to allow the system to boot when the network was down, some binaries and libraries are stored in /bin, /sbin and /lib
       /bin and /sbin contain programs like ls, ifconfig, cp, bash, etc.
        /lib contains the C library and sometimes a few other basic libraries
        All other programs and libraries are in /usr

Important directories
       /bin Basic programs
       /boot Kernel image (only when the kernel is loaded from a filesystem, not common on non-x86 architectures)
       /dev Device files
       /etc System-wide configuration
       /home Directory for the users home directories
       /lib Basic libraries
       /media Mount points for removable media
       /mnt Mount points for static media
       /proc Mount point for the proc virtual filesystem
       /root Home directory of the root user
       /sbin Basic system programs
       /sys Mount point of the sysfs virtual  filesystem
       /tmp Temporary files
       /usr /usr/bin Non-basic programs
       /usr/lib Non-basic libraries
       /usr/sbin Non-basic system programs
       /var Variable data files. This includes spool directories and files, administrative and logging data, and transient and temporary

Library components
Proc file system
       The proc virtual filesystem exists since the beginning of Linux
       It allows
      The kernel to expose statistics about running processes in the system
      The user to adjust at runtime various system parameters about process management, memory management, etc.
       The proc filesystem is used by many standard userspace applications, and they expect it to be mounted in /proc
       Applications such as ps or top would not work without the proc filesystem
       Command to mount /proc:
      mount -t proc nodev /proc
       Documentation/filesystems/proc.txt in the kernel sources
       man proc

/proc contents
       One directory for each running process in the system
      /proc/<pid>
      cat /proc/3840/cmdline
      It contains details about the les opened by the process, the CPU and memory usage, etc.
       /proc/interrupts, /proc/devices, /proc/iomem, /proc/ioports contain general device-related information
       /proc/cmdline contains the kernel command line
       /proc/sys contains many files that can be written to to adjust kernel parameters
      They are called sysctl. See Documentation/sysctl/ in kernel sources.
      Example
       echo 3 > /proc/sys/vm/drop_caches

Sysfs filesystem
       The sysfs filesystem is a feature integrated in the 2.6 Linux kernel
       It allows to represent in userspace the vision that the kernel has of the buses, devices and drivers in the system
       It is useful for various userspace applications that need to list and query the available hardware, for example udev
       All applications using sysfs expect it to be mounted in the /sys directory
       Command to mount /sys:
      mount -t sysfs nodev /sys
      $ ls /sys/
block bus class dev devices firmware fs kernel

Basic applications
       In order to work, a Linux system needs at least a few applications
        An init application, which is the first userspace application started by the kernel after mounting the root filesystem
       The kernel tries to run /sbin/init, /bin/init, /etc/init and /bin/sh.
       If none of them are found, the kernel panics and the boot process is stopped.
       The init application is responsible for starting all other userspace applications and services
       Usually a shell, to allow a user to interact with the system
       Basic Unix applications, to copy les, move les, list files (commands like mv, cp, mkdir, cat, etc.)
       Those basic components have to be integrated into the root filesystem to make it usable

Overall booting process

Building root filesystem
       Basic root file system structure
      The top level directories in the root file system each have a specific use and purpose.
      Common top-level directories:
      /bin/, /boot, /dev, /etc, /home, /lib, /proc, /root, /tmp, /usr, /var etc
       To work on the root filesystem, let's move into the directory we created for this purpose:
                $ cd ${PRJROOT}/rootfs
       We now create the core root filesystem directories required for our system:
       $ mkdir bin dev etc lib proc sbin tmp usr var
       For the various application packages
       We can proceed with the creation of the /usr hierarchy:
       $ mkdir usr/bin usr/lib usr/sbin
       The last entries to create are in the /var directory:
       $ mkdir var/lib var/lock var/log var/run var/tmp
       With the root filesystem skeleton now ready, let's place the various software components in their appropriate locations.
       Placa all the libraries including glibc and application libraries in /lib dir
       Copy the compiled kernel modules in lib dir
       cp “all_kernel_modules.ko” ${PRJROOT}/rootfs/lib/modules
       Create the device files in ${PRJROOT}/rootfs/dev dir using mknod command
       # mknod -m 600 mem c 1 1
       # mknod -m 666 null c 1 3
       # mknod -m 666 zero c 1 5
       # mknod -m 644 random c 1 8

Main system applications
       Beyond the kernel's functionality and the root filesystem's structure, Linux inherits Unix's very rich command set.
       We can decide what all commands we need to include into the target box.
       There are two approaches          
      Select and cross compile each of the application, ie complete standard application
      Use busybox

Complete standard application
       If you would like to selectively include some of the standard applications found in main stream distributions, your best bet is to start with the Linux From Scratch project located at http://www.linuxfromscratch.org/.
       This project aims at providing explanations and links to packages to help you build your own custom distributions.

Busybox
       A Linux system needs a basic set of programs to work
       An init program
       A shell
       Various basic utilities for le manipulation and system configuration
       In normal Linux systems, those programs are provided by different projects
      coreutils, bash, grep, sed, tar, wget, modutils, etc. are all different projects
      A lot of different components to integrate
      Components not designed with embedded systems constraints in mind: they are not very configurable and have a wide range of features
       Busybox is an alternative solution, extremely common on embedded systems

General purpose toolbox: Busybox
       Rewrite of many useful Unix command line utilities
      Integrated into a single project, which makes it easy to work with
      Designed with embedded systems in mind: highly configurable, no unnecessary features
       All the utilities are compiled into a single executable, /bin/busybox
      Symbolic links to /bin/busybox are created for each application integrated into Busybox
       http://www.busybox.net/
       Although BusyBox does not support all the options provided by the commands it replaces, the subset it provides is sufficient for most typical uses

Configuring busybox
       Get the latest stable sources from http://busybox.net
       Congure BusyBox (creates a .config le):
      make defconfig
                Good to begin with BusyBox. Congures BusyBox with all options for regular users.
       make allnoconfig
      Unselects all options. Good to congure only what you need.
       make xconfig (graphical, needs the libqt3-mt-dev package) or make menuconfig (text)
      Same conguration interfaces as the ones used by the Linux kernel (though older versions are used).

Compiling Busybox
       Using “make menuconfig” command you can choose the commands to compile and the commands options and features that you need.
       Set the cross-compiler prefix in the conguration interface:
      BusyBox Settings -> Build Options -> Cross Compiler prefix
      Example: arm-linux-
       Set the installation directory in the conguration interface:
      BusyBox Settings -> Installation Options -> BusyBox installation prefix
       Add the cross-compiler path to the PATH environment variable:
      export PATH=/usr/xtools/arm-unknown-linux-uclibcgnueabi/bin:$PATH
       Compile BusyBox:
      Make
      Install it (this creates a Unix directory structure symbolic links to the busybox executable):
       make install

Using busybox commands
       To understand how best to use BusyBox, let's first take a look at the components installed on the target's root filesystem by BusyBox's build process. As observed there is  only one executable was installed, /bin/busybox.
       This is the single binary with support for all the commands configured  This binary is never called directly, however. Instead, symbolic links bearing the original commands' names have been created to /bin/busybox. Such symbolic links have been created in all the directories in which the original commands would be found, including /bin, /sbin, /usr/bin, and /usr/sbin.
       When you type a command during the system's normal operation, the busybox command is invoked via the symbolic link. In turn, busybox determines the actual command you were invoking using the name being used to run it. /bin/ls, for instance, points to /bin/busybox.
       When you type ls, the busybox command is called and it determines that you were trying to use the ls command, because ls is the first argument on the command line

Busy box
       Busy box provides all the required functionality while still remaining a very small-size application
       BusyBox implements many commands. Here are a few:
        ar, cat, chgrp, chmod, chown, chroot, cp, cpio, date, dd, df, dmesg, dos2unix, du, echo, env, expr, find, grep, gunzip, gzip, halt, id, ifconfig, init, insmod, kill, killall, ln, ls, lsmod, md5sum, mkdir, mknod, modprobe, more, mount, mv, ping, ps, pwd, reboot, renice, rm, rmdir, rmmod, route, rpm2cpio, sed, stty, swapon, sync, syslogd, tail, tar, telnet, tftp, touch, traceroute, umount, uname, uuencode, vi, wc, which, and whoami.
       Although BusyBox does not support all the options provided by the commands it replaces, the subset it provides is sufficient for most typical uses.


No comments:

Post a Comment