Driver Development and Memory Management

Driver development – memory management

•       Physical memory and virtual memory

•       Virtual memory organisation

•       Physical and virtual memory mapping

•       Accessing physical memory

•       Allocators in kernel memory

•       Kmalloc alocator and APIs

•       Vmalloc allocator and APIs

Virtual memory organization: 1GB/3GB

•       1GB reserved for kernel-space

•       Contains kernel code and core data structures identical in all address spaces

•       Most memory can be a direct mapping of physical memory at a fixed offset

•       Complete 3GB exclusive mapping available for each user-space process

•       Process code and data (program, stack, …)

•       Memory-mapped files

•       Not necessarily mapped to physical memory

•       (demand fault paging used for dynamic mapping to physical memory pages)

•       Differs from one address space to the other

Physical and virtual address

Kernel memory

•       Kernel memory allocators allocate physical pages, and kernel allocated memory cannot be swapped out, so

no fault handling required for kernel memory.

•       Most kernel memory allocation functions also return a kernel virtual address to be used within the kernel space.

•       Kernel memory low-level allocator manages pages. This is the finest granularity (usually 4 kB, architecture dependent).

•       However, the kernel memory management handles smaller

memory allocations through its allocator (see slabs / SLUB

allocator – used by kmalloc).

Allocators in the kernel

Page allocators

•       Appropriate for large allocations

•       A page is usually 4K, but can be made greater in some architectures.

•       Buddy allocator strategy, so only allocations of power of two number of pages are possible: 1 page, 2 pages, 4 pages, 8 pages, 16 pages, etc.

•       The allocated area is virtually contiguous (of course), but also physically contiguous. It is allocated in the identity-mapped part of the kernel memory space.

•       This means that large areas may not be available or hard to retrieve due to physical memory fragmentation.

Page allocator APIs

•       unsigned long get_zeroed_page(int flags);

•       Returns the virtual address of a free page, initialized to zero

•       unsigned long __get_free_page(int flags);

•       Same, but doesn't initialize the contents

•       unsigned long __get_free_pages(int flags, unsigned int order);

•       Returns the starting virtual address of an area of several contiguous pages in physical RAM, with order being log2(<number_of_pages>).Can be computed from the size with the get_order() function.

•       void free_page(unsigned long addr);

•       Frees one page.

•       void free_pages(unsigned long addr, unsigned int order);

•       Frees multiple pages. Need to use the same order as in allocation.

Page allocator flags

•       The most common ones are:

•       GFP_KERNEL

•       Standard kernel memory allocation. The allocation may block in order to find enough available memory. Fine for most needs, except in interrupt handler context.

•       GFP_ATOMIC

•       RAM allocated from code which is not allowed to block (interrupt handlers or critical sections). Never blocks, allows to access emergency pools, but can fail if no free memory is readily available.

•       GFP_DMA

•       Allocates memory in an area of the physical memory usable for  DMA transfers.

•       Others are defined in include/linux/gfp.h

•       (GFP: __get_free_pages).

SLAB allocator

•       The SLAB allocator allows to create caches, which contains a set of objects of the same size

•       The object size can be smaller or greater than the page size

•       The SLAB allocator takes care of growing or reducing the size of the cache as needed, depending on the number of allocated objects. It uses the page allocator to allocate and free pages.

•       SLAB caches are used for data structures that are present in many instances in the kernel: directory entries, file objects, network packet descriptors, process descriptors, etc.

•       See /proc/slabinfo

•       They are rarely used for individual drivers.

•       See include/linux/slab.h for the API

Kmalloc allocator

•       The kmalloc allocator is the general purpose memory allocator in the Linux kernel, for objects from 8 bytes to 128 KB

•       For small sizes, it relies on generic SLAB caches, named kmalloc-XXX in /proc/slabinfo

•       For larger sizes, it relies on the page allocator

•       The allocated area is guaranteed to be physically contiguous

•       The allocated area size is rounded up to the next power of two size

•       It uses the same flags as the page allocator (GFP_KERNEL,

•       GFP_ATOMIC, GFP_DMA, etc.) with the same semantics.

•       Should be used as the primary allocator unless there is a strong reason to use another one.

Kmalloc API

•       #include <linux/slab.h>

•       void *kmalloc(size_t size, int flags);

•       Allocate size bytes, and return a pointer to the area (virtual address)

•       size: number of bytes to allocate

•       flags: same flags as the page allocator

•       void kfree (const void *objp);

•       Free an allocated area

•       void *kzalloc(size_t size, gfp_t flags);

•       Allocates a zero-initialized buffer

Vmalloc allocator

•       The vmalloc allocator can be used to obtain virtually contiguous memory zones, but not physically contiguous.

•       The requested memory size is rounded up to the next page.

•       The allocated area is in the kernel space part of the address space, but outside of the identically-mapped area

•       Allocations of fairly large areas is possible, since physical memory fragmentation is not an issue, but areas cannot be used for DMA, as DMA usually requires physically contiguous buffers.

•       API in <linux/vmalloc.h>

•       void *vmalloc(unsigned long size);

•       Returns a virtual address

•       void vfree(void *addr);

•       Frees the allocated area

•       Using the kmalloc/kfree APIs writing the read/write function of the character device dirver

•       Write a kernel module to check the upper limit of kmalloc API

•       Write a kernel module allocating memory using vmalloc() APIs.