Subversion Repositories HelenOS-doc

Rev

Rev 137 | Rev 160 | Go to most recent revision | Show entire file | Ignore whitespace | Details | Blame | Last modification | View Log | RSS feed

Rev 137 Rev 157
Line 29... Line 29...
29
    response.</para>
29
    response.</para>
30
 
30
 
31
    <para>Every message must be eventually answered. The system keeps track of
31
    <para>Every message must be eventually answered. The system keeps track of
32
    all messages, so that it can answer them with appropriate error code
32
    all messages, so that it can answer them with appropriate error code
33
    should one of the connection parties fail unexpectedly. To limit buffering
33
    should one of the connection parties fail unexpectedly. To limit buffering
34
    of the messages in the kernel, every process is has a limited account of
34
    of the messages in the kernel, every task has a limit on the amount of
35
    asynchronous messages it can send simultanously. If the limit is reached,
35
    asynchronous messages it can send simultaneously. If the limit is reached,
36
    the kernel refuses to send any other message, until some active message is
36
    the kernel refuses to send any other message, until some active message is
37
    answered.</para>
37
    answered.</para>
38
 
38
 
39
    <para>To facilitate kernel-to-user communication, the IPC subsystem
39
    <para>To facilitate kernel-to-user communication, the IPC subsystem
40
    provides notification messages. The applications can subscribe to a
40
    provides notification messages. The applications can subscribe to a
Line 47... Line 47...
47
    <section>
47
    <section>
48
      <title>Low Level IPC</title>
48
      <title>Low Level IPC</title>
49
 
49
 
50
      <para>The whole IPC subsystem consists of one-way communication
50
      <para>The whole IPC subsystem consists of one-way communication
51
      channels. Each task has one associated message queue (answerbox). The
51
      channels. Each task has one associated message queue (answerbox). The
52
      task can call other tasks and connect it's phones to their answerboxes.,
52
      task can call other tasks and connect its phones to their answerboxes,
53
      send and forward messages through these connections and answer received
53
      send and forward messages through these connections and answer received
54
      messages. Every sent message is identified by a unique number, so that
54
      messages. Every sent message is identified by a unique number, so that
55
      the response can be later matched against it. The message is sent over
55
      the response can be later matched against it. The message is sent over
56
      the phone to the target answerbox. Server application periodically
56
      the phone to the target answerbox. The server application periodically
57
      checks the answerbox and pulls messages from several queues associated
57
      checks the answerbox and pulls messages from several queues associated
58
      with it. After completing the requested action, server sends a reply
58
      with it. After completing the requested action, the server sends a reply
59
      back to the answerbox of the originating task. If a need arises, it is
59
      back to the answerbox of the originating task. If a need arises, it is
60
      possible to <emphasis>forward</emphasis> a recevied message throught any
60
      possible to <emphasis>forward</emphasis> a received message through any
61
      of the open phones to another task. This mechanism is used e.g. for
61
      of the open phones to another task. This mechanism is used e.g. for
62
      opening new connections.</para>
62
      opening new connections to services via the naming service.</para>
63
 
63
 
64
      <para>The answerbox contains four different message queues:</para>
64
      <para>The answerbox contains four different message queues:</para>
65
 
65
 
66
      <itemizedlist>
66
      <itemizedlist>
67
        <listitem>
67
        <listitem>
Line 98... Line 98...
98
          </imageobject>
98
          </imageobject>
99
        </mediaobject>
99
        </mediaobject>
100
      </figure>
100
      </figure>
101
 
101
 
102
      <para>The communication between task A, that is connected to task B
102
      <para>The communication between task A, that is connected to task B
103
      looks as follows: Task A sends a message over it's phone to the target
103
      looks as follows: task A sends a message over its phone to the target
104
      asnwerbox. The message is saved in task B incoming call queue. When task
104
      asnwerbox. The message is saved in task B's incoming call queue. When task
105
      B fetches the message for processing, it is automatically moved into the
105
      B fetches the message for processing, it is automatically moved into the
106
      dispatched call queue. After the server decides to answer the message,
106
      dispatched call queue. After the server decides to answer the message,
107
      it is removed from dispatched queue and the result is moved into the
107
      it is removed from dispatched queue and the result is moved into the
108
      answer queue of task A.</para>
108
      answer queue of task A.</para>
109
 
109
 
110
      <para>The arguments contained in the message are completely arbitrary
110
      <para>The arguments contained in the message are completely arbitrary
111
      and decided by the user. The low level part of kernel IPC fills in
111
      and decided by the user. The low level part of kernel IPC fills in
112
      appropriate error codes if there is an error during communication. It is
112
      appropriate error codes if there is an error during communication. It is
113
      assured that the applications are correctly notified about communication
113
      assured that the applications are correctly notified about communication
114
      state. If a program closes the outgoing connection, the target answerbox
114
      state. If a program closes the outgoing connection, the target answerbox
115
      receives a hangup message. The connection identification is not reused,
115
      receives a hangup message. The connection identification is not reused
116
      until the hangup message is acknowledged and all other pending messages
116
      until the hangup message is acknowledged and all other pending messages
117
      are answered.</para>
117
      are answered.</para>
118
 
118
 
119
      <para>Closing an incoming connection is done by responding to any
119
      <para>Closing an incoming connection is done by responding to any
120
      incoming message with an EHANGUP error code. The connection is then
120
      incoming message with an EHANGUP error code. The connection is then
121
      immediately closed. The client connection identification (phone id) is
121
      immediately closed. The client connection identification (phone id) is
122
      not reused, until the client issues closes it's own side of the
122
      not reused, until the client closes its own side of the
123
      connection ("hangs his phone up").</para>
123
      connection ("hangs his phone up").</para>
124
 
124
 
125
      <para>When a task dies (whether voluntarily or by being killed), cleanup
125
      <para>When a task dies (whether voluntarily or by being killed), cleanup
126
      process is started.</para>
126
      process is started.</para>
127
 
127
 
128
      <orderedlist>
128
      <orderedlist>
129
        <listitem>
129
        <listitem>
130
          <para>Hangs up all outgoing connections and sends hangup messages to
130
          <para>hangs up all outgoing connections and sends hangup messages to
131
          all target answerboxes.</para>
131
          all target answerboxes,</para>
132
        </listitem>
132
        </listitem>
133
 
133
 
134
        <listitem>
134
        <listitem>
135
          <para>Disconnects all incoming connections.</para>
135
          <para>disconnects all incoming connections,</para>
136
        </listitem>
136
        </listitem>
137
 
137
 
138
        <listitem>
138
        <listitem>
139
          <para>Disconnects from all notification channels.</para>
139
          <para>disconnects from all notification channels,</para>
140
        </listitem>
140
        </listitem>
141
 
141
 
142
        <listitem>
142
        <listitem>
143
          <para>Answers all unanswered messages from answerbox queues with
143
          <para>answers all unanswered messages from answerbox queues with
144
          appropriate error code.</para>
144
          appropriate error code and</para>
145
        </listitem>
145
        </listitem>
146
 
146
 
147
        <listitem>
147
        <listitem>
148
          <para>Waits until all outgoing messages are answered and all
148
          <para>waits until all outgoing messages are answered and all
149
          remaining answerbox queues are empty.</para>
149
          remaining answerbox queues are empty.</para>
150
        </listitem>
150
        </listitem>
151
      </orderedlist>
151
      </orderedlist>
152
    </section>
152
    </section>
153
 
153
 
Line 155... Line 155...
155
      <title>System Call IPC Layer</title>
155
      <title>System Call IPC Layer</title>
156
 
156
 
157
      <para>On top of this simple protocol the kernel provides special
157
      <para>On top of this simple protocol the kernel provides special
158
      services closely related to the inter-process communication. A range of
158
      services closely related to the inter-process communication. A range of
159
      method numbers is allocated and protocol is defined for these functions.
159
      method numbers is allocated and protocol is defined for these functions.
160
      The messages are interpreted by the kernel layer and appropriate actions
160
      These messages are interpreted by the kernel layer and appropriate actions
161
      are taken depending on the parameters of message and answer.</para>
161
      are taken depending on the parameters of the message and the answer.</para>
162
 
162
 
163
      <para>The kernel provides the following services:</para>
163
      <para>The kernel provides the following services:</para>
164
 
164
 
165
      <itemizedlist>
165
      <itemizedlist>
166
        <listitem>
166
        <listitem>
167
          <para>Creating new outgoing connection</para>
167
          <para>creating new outgoing connection,</para>
168
        </listitem>
168
        </listitem>
169
 
169
 
170
        <listitem>
170
        <listitem>
171
          <para>Creating a callback connection</para>
171
          <para>creating a callback connection,</para>
172
        </listitem>
172
        </listitem>
173
 
173
 
174
        <listitem>
174
        <listitem>
175
          <para>Sending an address space area</para>
175
          <para>sending an address space area,</para>
176
        </listitem>
176
        </listitem>
177
 
177
 
178
        <listitem>
178
        <listitem>
179
          <para>Asking for an address space area</para>
179
          <para>asking for an address space area.</para>
180
        </listitem>
180
        </listitem>
181
      </itemizedlist>
181
      </itemizedlist>
182
 
182
 
183
      <para>On startup every task is automatically connected to a
183
      <para>On startup, every task is automatically connected to a
184
      <emphasis>name service task</emphasis>, which provides a switchboard
184
      <emphasis>naming service task</emphasis>, which provides a switchboard
185
      functionality. To open a new outgoing connection, the client sends a
185
      functionality. In order to open a new outgoing connection, the client sends a
186
      <constant>CONNECT_ME_TO</constant> message using any of his phones. If
186
      <constant>CONNECT_ME_TO</constant> message using any of his phones. If
187
      the recepient of this message answers with an accepting answer, a new
187
      the recepient of this message answers with an accepting answer, a new
188
      connection is created. In itself, this mechanism would allow only
188
      connection is created. In itself, this mechanism would allow only
189
      duplicating existing connection. However, if the message is forwarded,
189
      duplicating existing connection. However, if the message is forwarded,
190
      the new connection is made to the final recipient.</para>
190
      the new connection is made to the final recipient.</para>
191
 
191
 
192
      <para>On startup every task is automatically connect to the name service
192
      <para>In order for a task to be able to forward a message, it
193
      task, which acts as a switchboard and forwards requests for connection
193
      must have a phone connected to the destination task.
194
      to specific services. To be able to forward a message it must have a
194
      The destination task establishes such connection by sending the <constant>CONNECT_TO_ME</constant>
195
      phone connected to the service tasks. The task creates this connection
195
      message to the forwarding task. A callback connection is opened afterwards.
196
      using a <constant>CONNECT_TO_ME</constant> message which creates a
-
 
197
      callback connection. Every service that wants to receive connections
196
      Every service that wants to receive connections
198
      asks name service task to create a callback connection.</para>
197
      has to ask the naming service to create the callback connection via this mechanism.</para>
199
 
198
 
200
      <para>Tasks can share their address space areas using IPC messages. The
199
      <para>Tasks can share their address space areas using IPC messages. The
201
      2 message types - <constant>AS_AREA_SEND</constant> and <constant>AS_AREA_RECV</constant> are used for sending and
200
      two message types - <constant>AS_AREA_SEND</constant> and <constant>AS_AREA_RECV</constant> are used for sending and
202
      receiving an address area respectively. The shared area can be accessed
201
      receiving an address space area respectively. The shared area can be accessed
203
      as soon as the message is acknowledged.</para>
202
      as soon as the message is acknowledged.</para>
204
    </section>
203
    </section>
205
  </section>
204
  </section>
206
 
205
 
207
  <section>
206
  <section>
208
    <title>Userspace View</title>
207
    <title>Userspace View</title>
209
 
208
 
210
    <para>The conventional design of the asynchronous api seems to produce
209
    <para>The conventional design of the asynchronous API seems to produce
211
    applications with one event loop and several big switch statements.
210
    applications with one event loop and several big switch statements.
212
    However, by intensive utilization of user-space threads, it was possible
211
    However, by intensive utilization of userspace pseudo threads, it was possible
213
    to create an environment that is not necesarilly restricted to this type
212
    to create an environment that is not necessarily restricted to this type
214
    of event-driven programming and allows for more fluent expression of
213
    of event-driven programming and allows for more fluent expression of
215
    application programs.</para>
214
    application programs.</para>
216
 
215
 
217
    <section>
216
    <section>
218
      <title>Single Point of Entry</title>
217
      <title>Single Point of Entry</title>
219
 
218
 
220
      <para>Each tasks is associated with only one answerbox. If a
219
      <para>Each task is associated with only one answerbox. If a
221
      multi-threaded application needs to communicate, it must be not only
220
      multithreaded application needs to communicate, it must be not only
222
      able to send a message, but it should be able to retrieve the answer as
221
      able to send a message, but it should be able to retrieve the answer as
223
      well. If several threads pull messages from task answerbox, it is a
222
      well. If several pseudo threads pull messages from task answerbox, it is a
224
      matter of fortune, which thread receives which message. If a particular
223
      matter of coincidence, which thread receives which message. If a particular
225
      thread needs to wait for a message answer, an idle
224
      thread needs to wait for a message answer, an idle
226
      <emphasis>manager</emphasis> task is found or a new one is created and
225
      <emphasis>manager</emphasis> pseudo thread is found or a new one is created and
227
      control is transfered to this manager task. The manager tasks pops
226
      control is transfered to this manager thread. The manager threads pop
228
      messages from the answerbox and puts them into appropriate queues of
227
      messages from the answerbox and put them into appropriate queues of
229
      running tasks. If a task waiting for a message is not running, the
228
      running threads. If a pseudo thread waiting for a message is not running, the
230
      control is transferred to it.</para>
229
      control is transferred to it.</para>
231
 
230
 
232
      <figure float="1">
231
      <figure float="1">
233
        <title>Single point of entry</title>
232
        <title>Single point of entry</title>
234
        <mediaobject id="ipc2">
233
        <mediaobject id="ipc2">
Line 246... Line 245...
246
        </mediaobject>
245
        </mediaobject>
247
 
246
 
248
      </figure>
247
      </figure>
249
 
248
 
250
      <para>Very similar situation arises when a task decides to send a lot of
249
      <para>Very similar situation arises when a task decides to send a lot of
251
      messages and reaches kernel limit of asynchronous messages. In such
250
      messages and reaches the kernel limit of asynchronous messages. In such
252
      situation 2 remedies are available - the userspace liberary can either
251
      situation, two remedies are available - the userspace library can either
253
      cache the message locally and resend the message when some answers
252
      cache the message locally and resend the message when some answers
254
      arrive, or it can block the thread and let it go on only after the
253
      arrive, or it can block the thread and let it go on only after the
255
      message is finally sent to the kernel layer. With one exception HelenOS
254
      message is finally sent to the kernel layer. With one exception, HelenOS
256
      uses the second approach - when the kernel responds that maximum limit
255
      uses the second approach - when the kernel responds that the maximum limit
257
      of asynchronous messages was reached, control is transferred to manager
256
      of asynchronous messages was reached, the control is transferred to a manager
258
      thread. The manager thread then handles incoming replies and when space
257
      pseudo thread. The manager thread then handles incoming replies and, when space
259
      is available, sends the message to kernel and resumes application thread
258
      is available, sends the message to the kernel and resumes the application thread
260
      execution.</para>
259
      execution.</para>
261
 
260
 
262
      <para>If a kernel notification is received, the servicing procedure is
261
      <para>If a kernel notification is received, the servicing procedure is
263
      run in the context of the manager thread. Although it wouldn't be
262
      run in the context of the manager pseudo thread. Although it wouldn't be
264
      impossible to allow recursive calling, it could potentially lead to an
263
      impossible to allow recursive calling, it could potentially lead to an
265
      explosion of manager threads. Thus, the kernel notification procedures
264
      explosion of manager threads. Thus, the kernel notification procedures
266
      are not allowed to wait for a message result, they can only answer
265
      are not allowed to wait for a message result, they can only answer
267
      messages and send new ones without waiting for their results. If the
266
      messages and send new ones without waiting for their results. If the
268
      kernel limit for outgoing messages is reached, the data is automatically
267
      kernel limit for outgoing messages is reached, the data is automatically
269
      cached within the application. This behaviour is enforced automatically
268
      cached within the application. This behaviour is enforced automatically
270
      and the decision making is hidden from developers view.</para>
269
      and the decision making is hidden from the developer.</para>
271
 
270
 
272
      <figure float="1">
271
      <figure float="1">
273
        <title>Single point of entry solution</title>
272
        <title>Single point of entry solution</title>
274
        <mediaobject id="ipc3">
273
        <mediaobject id="ipc3">
275
          <imageobject role="pdf">
274
          <imageobject role="pdf">
Line 291... Line 290...
291
    <section>
290
    <section>
292
      <title>Ordering Problem</title>
291
      <title>Ordering Problem</title>
293
 
292
 
294
      <para>Unfortunately, the real world is is never so simple. E.g. if a
293
      <para>Unfortunately, the real world is is never so simple. E.g. if a
295
      server handles incoming requests and as a part of its response sends
294
      server handles incoming requests and as a part of its response sends
296
      asynchronous messages, it can be easily prempted and other thread may
295
      asynchronous messages, it can be easily preempted and another thread may
297
      start intervening. This can happen even if the application utilizes only
296
      start intervening. This can happen even if the application utilizes only
298
      1 kernel thread. Classical synchronization using semaphores is not
297
      one userspace thread. Classical synchronization using semaphores is not
299
      possible, as locking on them would block the thread completely so that
298
      possible as locking on them would block the thread completely so that
300
      the answer couldn't be ever processed. The IPC framework allows a
299
      the answer couldn't be ever processed. The IPC framework allows a
301
      developer to specify, that part of the code should not be preempted by
300
      developer to specify, that part of the code should not be preempted by
302
      any other thread (except notification handlers) while still being able
301
      any other pseudo thread (except notification handlers) while still being able
303
      to queue messages belonging to other threads and regain control when the
302
      to queue messages belonging to other pseudo threads and regain control when the
304
      answer arrives.</para>
303
      answer arrives.</para>
305
 
304
 
306
      <para>This mechanism works transparently in multithreaded environment,
305
      <para>This mechanism works transparently in multithreaded environment,
307
      where additional locking mechanism (futexes) should be used. The IPC
306
      where additional locking mechanism (futexes) should be used. The IPC
308
      framework ensures that there will always be enough free kernel threads
307
      framework ensures that there will always be enough free userspace threads
309
      to handle incoming answers and allow the application to run more
308
      to handle incoming answers and allow the application to run more
310
      user-space threads inside the kernel threads without the danger of
309
      pseudo threads inside the usrspace threads without the danger of
311
      locking all kernel threads in futexes.</para>
310
      locking all userspace threads in futexes.</para>
312
    </section>
311
    </section>
313
 
312
 
314
    <section>
313
    <section>
315
      <title>The Interface</title>
314
      <title>The Interface</title>
316
 
315
 
317
      <para>The interface was developed to be as simple to use as possible.
316
      <para>The interface was developed to be as simple to use as possible.
318
      Classical applications simply send messages and occasionally wait for an
317
      Classical applications simply send messages and occasionally wait for an
319
      answer and check results. If the number of sent messages is higher than
318
      answer and check results. If the number of sent messages is higher than
320
      kernel limit, the flow of application is stopped until some answers
319
      the kernel limit, the flow of application is stopped until some answers
321
      arrive. On the other hand server applications are expected to work in a
320
      arrive. On the other hand, server applications are expected to work in a
322
      multithreaded environment.</para>
321
      multithreaded environment.</para>
323
 
322
 
324
      <para>The server interface requires developer to specify a
323
      <para>The server interface requires the developer to specify a
325
      <function>connection_thread</function> function. When new connection is
324
      <function>connection_thread</function> function. When new connection is
326
      detected, a new userspace thread is automatically created and control is
325
      detected, a new pseudo thread is automatically created and control is
327
      transferred to this function. The code then decides whether to accept
326
      transferred to this function. The code then decides whether to accept
328
      the connection and creates a normal event loop. The userspace IPC
327
      the connection and creates a normal event loop. The userspace IPC
329
      library ensures correct switching between several userspace threads
328
      library ensures correct switching between several pseudo threads
330
      within the kernel environment.</para>
329
      within the kernel environment.</para>
331
    </section>
330
    </section>
332
  </section>
331
  </section>
333
</chapter>
332
</chapter>
334
333