Subversion Repositories HelenOS-doc

<?xml version="1.0" encoding="UTF-8"?>
<chapter id="ipc">
  <?dbhtml filename="ipc.html"?>

  <title>IPC</title>

  <para>Due to the high intertask communication traffic, IPC becomes critical
  subsystem for microkernels, putting high demands on the speed, latency and
  reliability of IPC model and implementation. Although theoretically the use
  of asynchronous messaging system looks promising, it is not often
  implemented because of a problematic implementation of end user
  applications. HelenOS implements a fully asynchronous messaging system but
  with a special layer providing a user application developer a reasonably
  synchronous multithreaded environment sufficient to develop complex
  protocols.</para>

  <section>
    <title>Services provided by kernel</title>

    <para>Every message consists of 4 numeric arguments (32-bit and 64-bit on
    the corresponding platforms), from which the first one is considered a
    method number on message receipt and a return value on answer receipt. The
    received message contains identification of the incoming connection, so
    that it can distinguish the messages between different senders. Internally
    the message contains pointer to the originating task and to the source of
    the communication channel. If the message is forwarded, the originating
    task identifies the recipient of the answer, the source channel identifies
    connection in case of a hangup message.</para>

    <para>Every message must be eventually answered. The system keeps track of
    all messages, so that it can answer them with appropriate error code
    should one of the connection parties fail unexpectedly. To limit buffering
    of messages in the kernel, every process is limited in a number of
    asynchronous messages it may have unanswered simultanously. If the limit
    is reached, the kernel refuses to send any other message, until some of
    the active messages are answered.</para>

    <section>
      <title>Low level IPC</title>

      <para>The whole IPC subsystem consists of one-way communication
      channels. Each task has one associated message queue (answerbox). The
      task can open connections (identified by phone id) to other tasks, send
      and forward messages through these connections and answer received
      messages. Every sent message is identified by a unique number, so that
      the response can be later matched against it. The message is sent over
      the phone to the target answerbox. Server application periodically
      checks the answerbox and pulls messages from several queues associated
      with it. After completing the requested action, server sends a reply
      back to the answerbox of the originating task. </para>

      <para>If a need arises, it is possible to <emphasis>forward</emphasis> a
      recevied message throught any of the open phones to another task. This
      mechanism is used e.g. for opening new connections.</para>
    </section>

    <section>
      <title>Services for user application</title>

      <para>On top of this simple protocol the kernel provides special
      services including opening new connection to other tasks, offering
      callback connections and sending and receiving address space areas.
      </para>
    </section>
  </section>

  <section>
    <title>Userspace view</title>

    <para>The conventional design of the asynchronous api seems to produce
    applications with one event loop and several big switch statements.
    However, by intensive utilization of user-space threads, it was possible
    to create an environment that is not necesarilly restricted to this type
    of event-driven programming and allows for more fluent expression of
    application programs. </para>

    <section>
      <title>Single point of entry</title>

      <para>Each tasks is associated with only one answerbox. If a
      multi-threaded application needs to communicate, it must be not only
      able to send a message, but it should be able to retrieve the answer as
      well. If several threads pull messages from task answerbox, it is a
      matter of fortune, which thread receives which message. If a particular
      thread needs to wait for a message answer, an idle
      <emphasis>manager</emphasis> task is found or a new one is created and
      control is transfered to this manager task. The manager tasks pops
      messages from the answerbox and puts them into appropriate queues of
      running tasks. If a task waiting for a message is not running, the
      control is transferred to it. </para>

      <para>Very similar situation arises when a task decides to send a lot of
      messages and reaches kernel limit of asynchronous messages. In such
      situation 2 remedies are available - the userspace liberary can either
      cache the message locally and resend the message when some answers
      arrive, or it can block the thread and let it go on only after the
      message is finally sent to the kernel layer. With one exception HelenOS
      uses the second approach - when the kernel responds that maximum limit
      of asynchronous messages was reached, control is transferred to manager
      thread. The manager thread then handles incoming replies and when space
      is available, sends the message to kernel and resumes application thread
      execution.</para>

      <para>If a kernel notification is received, the servicing procedure is
      run in the context of the manager thread. Although it wouldn't be
      impossible to allow recursive calling, it could potentially lead to an
      explosion of manager threads. Thus, the kernel notification procedures
      are not allowed to wait for a message result, they can only answer
      messages and send new ones without waiting for their results. If the
      kernel limit for outgoing messages is reached, the data is automatically
      cached within the application. This behaviour is enforced automatically
      and the decision making is hidden from developers view.</para>
    </section>

    <section>
      <title>Synchronization problem</title>

      <para>Unfortunately, in the real world is is never so easy. E.g. if a
      server handles incoming requests and as a part of it's response sends
      asynchronous messages, it can be easily prempted and other thread may
      start intervening. This can happen even if the application utilizes only
      1 kernel thread. Classical synchronization using semaphores is not
      possible, as locking on them would block the thread completely and the
      answer couldn't be ever processed. The IPC framework allows a developer
      to specify, that the thread should not be preempted to any other thread
      (except notification handlers) while still being able to queue messages
      belonging to other threads and regain control when the answer arrives.
      </para>

      <para>This mechanism works transparently in multithreaded environment,
      where classical locking mechanism (futexes) should be used. The IPC
      framework ensures that there will always be enough free threads to
      handle the threads requiring correct synchronization and allow the
      application to run more user-space threads inside the kernel threads
      without the danger of locking all kernel threads in futexes.</para>
    </section>

    <section>
      <title>The interface</title>

      <para></para>
    </section>
  </section>
</chapter>