Subversion Repositories HelenOS-doc

  <section>

    <title>Virtual memory management</title>

    <section>

      <title>Introduction</title>

      <para>Virtual memory is a special memory management technique, used by

      kernel to achieve a bunch of mission critical goals. <itemizedlist>

          <listitem>

             Isolate each task from other tasks that are running on the system at the same time.

          </listitem>

          <listitem>

             Allow to allocate more memory, than is actual physical memory size of the machine.

          </listitem>

          <listitem>

             Allowing, in general, to load and execute two programs that are linked on the same address without complicated relocations.

          </listitem>

        </itemizedlist></para>

      <para><!--

                <para>

                        Address spaces. Address space area (B+ tree). Only for uspace. Set of syscalls (shrink/extend etc).

                        Special address space area type - device - prohibits shrink/extend syscalls to call on it.

                        Address space has link to mapping tables (hierarchical - per Address space, hash - global tables).

                </para>

--></para>

    </section>

    <section>

      <title>Address spaces</title>

      <section>

        <title>Address space areas</title>

        <para>Each address space consists of mutually disjunctive continuous

        address space areas. Address space area is precisely defined by its

        base address and the number of frames/pages is contains.</para>

        <para>Address space area , that define behaviour and permissions on

        the particular area. <itemizedlist>

            <listitem>

              <emphasis>AS_AREA_READ</emphasis>

               flag indicates reading permission.

            </listitem>

            <listitem>

              <emphasis>AS_AREA_WRITE</emphasis>

               flag indicates writing permission.

            </listitem>

            <listitem>

              <emphasis>AS_AREA_EXEC</emphasis>

               flag indicates code execution permission. Some architectures do not support execution persmission restriction. In this case this flag has no effect.

            </listitem>

            <listitem>

              <emphasis>AS_AREA_DEVICE</emphasis>

               marks area as mapped to the device memory.

            </listitem>

          </itemizedlist></para>

        <para>Kernel provides possibility tasks create/expand/shrink/share its

        address space via the set of syscalls.</para>

      </section>

      <section>

        <title>Address Space ID (ASID)</title>

        <para>When switching to the different task, kernel also require to

        switch mappings to the different address space. In case TLB cannot

        distinguish address space mappings, all mapping information in TLB

        from the old address space must be flushed, which can create certain

        uncessary overhead during the task switching. To avoid this, some

        architectures have capability to segregate different address spaces on

        hardware level introducing the address space identifier as a part of

        TLB record, telling the virtual address space translation unit to

        which address space this record is applicable.</para>

        <para>HelenOS kernel can take advantage of this hardware supported

        identifier by having an ASID abstraction which is somehow related to

        the corresponding architecture identifier. I.e. on ia64 kernel ASID is

        derived from RID (region identifier) and on the mips32 kernel ASID is

        actually the hardware identifier. As expected, this ASID information

        record is the part of <emphasis>as_t</emphasis> structure.</para>

        <para>Due to the hardware limitations, hardware ASID has limited

        length from 8 bits on ia64 to 24 bits on mips32, which makes it

        impossible to use it as unique address space identifier for all tasks

        running in the system. In such situations special ASID stealing

        algoritm is used, which takes ASID from inactive task and assigns it

        to the active task.<classname></classname></para>

      </section>

    </section>

    <section>

      <title>Virtual address translation</title>

      <section id="pagING">

        <title>Paging</title>

        <section>

          <title>Introduction</title>

          <para>Virtual memory is usually using paged memory model, where

          virtual memory address space is divided into the

          <emphasis>pages</emphasis> (usually having size 4096 bytes) and

          physical memory is divided into the frames (same sized as a page, of

          course). Each page may be mapped to some frame and then, upon memory

          access to the virtual address, CPU performs <emphasis>address

          translation</emphasis> during the instruction execution.

          Non-existing mapping generates page fault exception, calling kernel

          memory access. Information for pages mapping is stored by kernel in

          the <link linkend="page_tables">page tables</link></para>

          <para>The majority of the architectures use multi-level page tables,

          which means need to access physical memory several times before

          getting physical address. This fact would make serios performance

          overhead in virtual memory management. To avoid this <link

          linkend="tlb">Traslation Lookaside Buffer (TLB)</link> is

          used.</para>

          <para>HelenOS kernel has two different approaches to the paging

          implementation: <emphasis>4 level page tables</emphasis> and

          <emphasis>global hash table</emphasis>, which are accessible via

          generic paging abstraction layer. Such different functionality was

          caused by the major architectural differences between supported

          structure of pointers to basic mapping functions

          <emphasis>page_mapping_operations</emphasis>. To achieve different

          functionality of page tables, corresponding layer must implement

          functions, declared in

          <emphasis>page_mapping_operations</emphasis></para>

          <para>Thanks to the abstract paging interface, there was a place

          left for more paging implementations (besides already implemented

          hieararchical page tables and hash table), for example B-Tree based

          page tables.</para>

        </section>

        <section>

          <title>Hierarchical 4-level page tables</title>

          <para>Hierarchical 4-level page tables are the generalization of the

          hardware capabilities of most architectures. Each address space has

          its own page tables.<itemizedlist>

              <listitem>

                 ia32 uses 2-level page tables, with full hardware support.

              </listitem>

              <listitem>

                 amd64 uses 4-level page tables, also coming with full hardware support.

              </listitem>

              <listitem>

                 mips and ppc32 have 2-level tables, software simulated support.

              </listitem>

            </itemizedlist></para>

        </section>

        <section>

          <title>Global hash table</title>

          <para>Implementation of the global hash table was encouraged by the

          ia64 architecture support. One of the major differences between

          global hash table and hierarchical tables is that global hash table

          exists only once in the system and the hierarchical tables are

          maintained per address space. </para>

          <para>Thus, hash table contains information about all address spaces

          mappings in the system, so, the hash of an entry must contain

          information of both address space pointer or id and the virtual

          on the close addresses, so it uses least significant bits for hash;

          also it assumes that the virtual page addresses have roughly the

          same probability of occurring, so the least significant bits of VPN

          compose the hash index.</para>

          <para>- global page hash table: existuje jen jedna v celem systemu

          (vyuziva ji ia64), pozn. ia64 ma zatim vypnuty VHPT. Pouziva se

          genericke hash table s oddelenymi collision chains. ASID support is

        </section>

      </section>

      <section id="tlb">

        <title>Translation Lookaside buffer</title>

        <para>Due to the extensive overhead during the page mapping lookup in

        the page tables, all architectures has fast assotiative cache memory

        built-in CPU. This memory called TLB stores recently used page table

        entries.</para>

        <section id="tlb_shootdown">

          <title>TLB consistency. TLB shootdown algorithm.</title>

          <para>Operating system is responsible for keeping TLB consistent by

          invalidating the contents of TLB, whenever there is some change in

          page tables. Those changes may occur when page or group of pages

          were unmapped, mapping is changed or system switching active address

          space to schedule a new system task. Moreover, this invalidation

          operation must be done an all system CPUs because each CPU has its

          own independent TLB cache. Thus maintaining TLB consistency on SMP

          configuration as not as trivial task as it looks on the first

          glance. Naive solution would assume that is the CPU which wants to

          invalidate TLB will invalidate TLB caches on other CPUs. It is not

          possible on the most of the architectures, because of the simple

          fact - flushing TLB is allowed only on the local CPU and there is no

          possibility to access other CPUs' TLB caches, thus invalidate TLB

          remotely.</para>

          <para>Technique of remote invalidation of TLB entries is called "TLB

          shootdown". HelenOS uses a variation of the algorithm described by

          D. Black et al., "Translation Lookaside Buffer Consistency: A

          Software Approach," Proc. Third Int'l Conf. Architectural Support

          for Programming Languages and Operating Systems, 1989, pp.

          113-122.</para>

          <para>As the situation demands, you will want partitial invalidation

          of TLB caches. In case of simple memory mapping change it is

          necessary to invalidate only one or more adjacent pages. In case if

          the architecture is aware of ASIDs, when kernel needs to dump some

          ASID to use by another task, it invalidates only entries from this

          particular address space. Final option of the TLB invalidation is

          the complete TLB cache invalidation, which is the operation that

          flushes all entries in TLB.</para>

          <para>TLB shootdown is performed in two phases.</para>

          <formalpara>

            <title>Phase 1.</title>

            <para>First, initiator locks a global TLB spinlock, then request

            is being put to the local request cache of every other CPU in the

            system protected by its spinlock. In case the cache is full, all

            requests in the cache are replaced by one request, indicating

            global TLB flush. Then the initiator thread sends an IPI message

            indicating the TLB shootdown request to the rest of the CPUs and

            waits actively until all CPUs confirm TLB invalidating action

            execution by setting up a special flag. After setting this flag

            this thread is blocked on the TLB spinlock, held by the

            initiator.</para>

          </formalpara>

          <formalpara>

            <title>Phase 2.</title>

            <para>All CPUs are waiting on the TLB spinlock to execute TLB

            invalidation action and have indicated their intention to the

            initiator. Initiator continues, cleaning up its TLB and releasing

            the global TLB spinlock. After this all other CPUs gain and

            immidiately release TLB spinlock and perform TLB invalidation

            actions.</para>

          </formalpara>

        </section>

      </section>

    </section>

    <section>

      <title>---</title>

      <para>At the moment HelenOS does not support swapping.</para>

      <para>- pouzivame vypadky stranky k alokaci ramcu on-demand v ramci

      as_area - na architekturach, ktere to podporuji, podporujeme non-exec

      stranky</para>

    </section>

  </section>

      information about number of free and busy frames. Physical memory

        </mediaobject>

      <formalpara>

        <para>Frame allocator provides physical memory allocation for the

        kernel. Because of zonal organization of physical memory, frame

        allocator is always working in context of some zone, thus making

        impossible to allocate a piece of memory, which lays in different

        zone, which cannot happen, because two adjacent zones can be merged

        into one. Frame allocator is also being responsible to update

        information on the number of free/busy frames in zone. Physical memory

        allocation inside one <link linkend="zones_and_frames">memory

        zone</link> is being handled by an instance of <link

        linkend="buddy_allocator">buddy allocator</link> tailored to allocate

        blocks of physical memory frames.</para>

            </listitem>

            <listitem>

            </listitem>

        <para>In the buddy allocator, the memory is broken down into

        power-of-two sized naturally aligned blocks. These blocks are

        organized in an array of lists, in which the list with index i

        contains all unallocated blocks of size

        <mathphrase>2<superscript>i</superscript></mathphrase>. The index i is

        called the order of block. Should there be two adjacent equally sized

        blocks in the list i<mathphrase />(i.e. buddies), the buddy allocator

        would coalesce them and put the resulting block in list <mathphrase>i

        + 1</mathphrase>, provided that the resulting block would be naturally

        aligned. Similarily, when the allocator is asked to allocate a block

        of size <mathphrase>2<superscript>i</superscript></mathphrase>, it

        first tries to satisfy the request from the list with index i. If the

        request cannot be satisfied (i.e. the list i is empty), the buddy

        allocator will try to allocate and split a larger block from the list

        with index i + 1. Both of these algorithms are recursive. The

        recursion ends either when there are no blocks to coalesce in the

        former case or when there are no blocks that can be split in the

        latter case.</para>

        <!--graphic fileref="images/mm1.png" format="EPS" /-->

        <para>This approach greatly reduces external fragmentation of memory

        and helps in allocating bigger continuous blocks of memory aligned to

        their size. On the other hand, the buddy allocator suffers increased

        internal fragmentation of memory and is not suitable for general

        kernel allocations. This purpose is better addressed by the <link

        linkend="slab">slab allocator</link>.</para>

      </section>

        <emphasis>"HelenOS Generic Kernel Reference Manual"</emphasis>.</para>

          <para>The buddy allocator always uses the first frame to represent

          the frame block. This frame contains <varname>buddy_order</varname>

          variable to provide information about the block size it actually

          represents (

          <mathphrase>2<superscript>buddy_order</superscript></mathphrase>

          frames block). Other frames in block have this value set to magic

          <constant>BUDDY_INNER_BLOCK</constant> that is much greater than

          buddy <varname>max_order</varname> value.</para>

          <para>Each <varname>frame_t</varname> also contains pointer member

          to hold frame structure in the linked list inside one order.</para>

        </formalpara>

        <formalpara>

          <title>Allocation algorithm</title>

          <para>Upon <mathphrase>2<superscript>i</superscript></mathphrase>

          frames block allocation request, allocator checks if there are any

          blocks available at the order list <varname>i</varname>. If yes,

          removes block from order list and returns its address. If no,

          recursively allocates

          <mathphrase>2<superscript>i+1</superscript></mathphrase> frame

          block, splits it into two

          <mathphrase>2<superscript>i</superscript></mathphrase> frame blocks.

          Then adds one of the blocks to the <varname>i</varname> order list

          and returns address of another.</para>

        </formalpara>

        <formalpara>

          <title>Deallocation algorithm</title>

          <para>Check if block has so called buddy (another free

          <mathphrase>2<superscript>i</superscript></mathphrase> frame block

          that can be linked with freed block into the

          <mathphrase>2<superscript>i+1</superscript></mathphrase> block).

          Technically, buddy is a odd/even block for even/odd block

          respectively. Plus we can put an extra requirement, that resulting

          block must be aligned to its size. This requirement guarantees

          natural block alignment for the blocks coming out the allocation

          system.</para>

          <para>Using direct pointer arithmetics,

          <varname>frame_t::ref_count</varname> and

          <varname>frame_t::buddy_order</varname> variables, finding buddy is

          done at constant time.</para>

        <para><termdef><glossterm>Slab</glossterm> represents a contiguous

        piece of memory, usually made of several physically contiguous

        pages.</termdef> <termdef><glossterm>Slab cache</glossterm> consists

        of one or more slabs.</termdef></para>

        <para>The majority of memory allocation requests in the kernel are for

        small, frequently used data structures. For this purpose the slab

        allocator is a perfect solution. The basic idea behind the slab

        allocator is to have lists of commonly used objects available packed

        into pages. This avoids the overhead of allocating and destroying

        commonly used types of objects such threads, virtual memory structures

        etc. Also due to the exact allocated size matching, slab allocation

        completely eliminates internal fragmentation issue.</para>

      </section>

      <section>

        <title>Implementation</title>

            <imageobject role="fop">

              <imagedata fileref="images.vector/slab_alloc.svg" format="SVG" />

            </imageobject>

            <listitem>

               empty slabs are deallocated immediately (in Linux they are kept in linked list, in Solaris ???)

            </listitem>