Rev 102 | Rev 141 | Go to most recent revision | Show entire file | Ignore whitespace | Details | Blame | Last modification | View Log | RSS feed
Rev 102 | Rev 131 | ||
---|---|---|---|
Line 15... | Line 15... | ||
15 | highly concurrent environment needs safe and efficient ways to handle |
15 | highly concurrent environment needs safe and efficient ways to handle |
16 | mutual exclusion and synchronization of many execution flows.</para> |
16 | mutual exclusion and synchronization of many execution flows.</para> |
17 | </section> |
17 | </section> |
18 | 18 | ||
19 | <section> |
19 | <section> |
20 | <title>Active kernel primitives</title> |
20 | <title>Active Kernel Primitives</title> |
21 | 21 | ||
22 | <section> |
22 | <section> |
23 | <indexterm> |
23 | <indexterm> |
24 | <primary>synchronization</primary> |
24 | <primary>synchronization</primary> |
25 | 25 | ||
Line 70... | Line 70... | ||
70 | critical section and those that implement locking and unlocking of the |
70 | critical section and those that implement locking and unlocking of the |
71 | respective spinlock is not implicit on some processor architectures. As |
71 | respective spinlock is not implicit on some processor architectures. As |
72 | a result, the processor needs to be explicitly told about each |
72 | a result, the processor needs to be explicitly told about each |
73 | occurrence of such a dependency. Therefore, HelenOS adds |
73 | occurrence of such a dependency. Therefore, HelenOS adds |
74 | architecture-specific hooks to all <code>spinlock_lock()</code>, |
74 | architecture-specific hooks to all <code>spinlock_lock()</code>, |
75 | <code>spinlock_trylock()</code> and <code>spinlock_unlock()</code> functions |
75 | <code>spinlock_trylock()</code> and <code>spinlock_unlock()</code> |
76 | to prevent the instructions inside the critical section from permeating |
76 | functions to prevent the instructions inside the critical section from |
77 | out. On some architectures, these hooks can be void because the |
77 | permeating out. On some architectures, these hooks can be void because |
78 | dependencies are implicitly there because of the special properties of |
78 | the dependencies are implicitly there because of the special properties |
79 | locking and unlocking instructions. However, other architectures need to |
79 | of locking and unlocking instructions. However, other architectures need |
80 | instrument these hooks with different memory barriers, depending on what |
80 | to instrument these hooks with different memory barriers, depending on |
81 | operations could permeate out.</para> |
81 | what operations could permeate out.</para> |
82 | 82 | ||
83 | <para>Spinlocks have one significant drawback: when held for longer time |
83 | <para>Spinlocks have one significant drawback: when held for longer time |
84 | periods, they harm both parallelism and concurrency. The processor |
84 | periods, they harm both parallelism and concurrency. The processor |
85 | executing <code>spinlock_lock()</code> does not do any fruitful work and |
85 | executing <code>spinlock_lock()</code> does not do any fruitful work and |
86 | is effectively halted until it can grab the lock and proceed. |
86 | is effectively halted until it can grab the lock and proceed. |
Line 100... | Line 100... | ||
100 | synchronization primitives.</para> |
100 | synchronization primitives.</para> |
101 | </section> |
101 | </section> |
102 | </section> |
102 | </section> |
103 | 103 | ||
104 | <section> |
104 | <section> |
105 | <title>Passive kernel synchronization</title> |
105 | <title>Passive Kernel Synchronization</title> |
106 | 106 | ||
107 | <section> |
107 | <section> |
108 | <indexterm> |
108 | <indexterm> |
109 | <primary>synchronization</primary> |
109 | <primary>synchronization</primary> |
110 | 110 | ||
111 | <secondary>- wait queue</secondary> |
111 | <secondary>- wait queue</secondary> |
112 | </indexterm> |
112 | </indexterm> |
113 | 113 | ||
114 | <title>Wait queues</title> |
114 | <title>Wait Queues</title> |
115 | 115 | ||
116 | <para>A wait queue is the basic passive synchronization primitive on |
116 | <para>A wait queue is the basic passive synchronization primitive on |
117 | which all other passive synchronization primitives are built. Simply |
117 | which all other passive synchronization primitives are built. Simply |
118 | put, it allows a thread to sleep until an event associated with the |
118 | put, it allows a thread to sleep until an event associated with the |
119 | particular wait queue occurs. Multiple threads are notified about |
119 | particular wait queue occurs. Multiple threads are notified about |
Line 122... | Line 122... | ||
122 | wait queue as a missed wakeup and later forwarded to the first thread |
122 | wait queue as a missed wakeup and later forwarded to the first thread |
123 | that decides to wait in the queue. The inner structures of the wait |
123 | that decides to wait in the queue. The inner structures of the wait |
124 | queue are protected by a spinlock.</para> |
124 | queue are protected by a spinlock.</para> |
125 | 125 | ||
126 | <para>The thread that wants to wait for a wait queue event uses the |
126 | <para>The thread that wants to wait for a wait queue event uses the |
127 | <code>waitq_sleep_timeout()</code> function. The algorithm then checks the |
127 | <code>waitq_sleep_timeout()</code> function. The algorithm then checks |
128 | wait queue's counter of missed wakeups and if there are any missed |
128 | the wait queue's counter of missed wakeups and if there are any missed |
129 | wakeups, the call returns immediately. The call also returns immediately |
129 | wakeups, the call returns immediately. The call also returns immediately |
130 | if only a conditional wait was requested. Otherwise the thread is |
130 | if only a conditional wait was requested. Otherwise the thread is |
131 | enqueued in the wait queue's list of sleeping threads and its state is |
131 | enqueued in the wait queue's list of sleeping threads and its state is |
132 | changed to <constant>Sleeping</constant>. It then sleeps until one of |
132 | changed to <constant>Sleeping</constant>. It then sleeps until one of |
133 | the following events happens:</para> |
133 | the following events happens:</para> |
134 | 134 | ||
135 | <orderedlist> |
135 | <orderedlist> |
136 | <listitem> |
136 | <listitem> |
137 | <para>another thread calls <code>waitq_wakeup()</code> and the thread |
137 | <para>another thread calls <code>waitq_wakeup()</code> and the |
138 | is the first thread in the wait queue's list of sleeping |
138 | thread is the first thread in the wait queue's list of sleeping |
139 | threads;</para> |
139 | threads;</para> |
140 | </listitem> |
140 | </listitem> |
141 | 141 | ||
142 | <listitem> |
142 | <listitem> |
143 | <para>another thread calls <code>waitq_interrupt_sleep()</code> on the |
143 | <para>another thread calls <code>waitq_interrupt_sleep()</code> on |
144 | sleeping thread;</para> |
144 | the sleeping thread;</para> |
145 | </listitem> |
145 | </listitem> |
146 | 146 | ||
147 | <listitem> |
147 | <listitem> |
148 | <para>the sleep times out provided that none of the previous |
148 | <para>the sleep times out provided that none of the previous |
149 | occurred within a specified time limit; the limit can be |
149 | occurred within a specified time limit; the limit can be |
Line 151... | Line 151... | ||
151 | </listitem> |
151 | </listitem> |
152 | </orderedlist> |
152 | </orderedlist> |
153 | 153 | ||
154 | <para>All five possibilities (immediate return on success, immediate |
154 | <para>All five possibilities (immediate return on success, immediate |
155 | return on failure, wakeup after sleep, interruption and timeout) are |
155 | return on failure, wakeup after sleep, interruption and timeout) are |
156 | distinguishable by the return value of <code>waitq_sleep_timeout()</code>. |
156 | distinguishable by the return value of |
157 | Being able to interrupt a sleeping thread is essential for externally |
157 | <code>waitq_sleep_timeout()</code>. Being able to interrupt a sleeping |
158 | initiated thread termination. The ability to wait only for a certain |
158 | thread is essential for externally initiated thread termination. The |
159 | amount of time is used, for instance, to passively delay thread |
159 | ability to wait only for a certain amount of time is used, for instance, |
160 | execution by several microseconds or even seconds in |
160 | to passively delay thread execution by several microseconds or even |
161 | <code>thread_sleep()</code> function. Due to the fact that all other |
161 | seconds in <code>thread_sleep()</code> function. Due to the fact that |
162 | passive kernel synchronization primitives are based on wait queues, they |
162 | all other passive kernel synchronization primitives are based on wait |
163 | also have the option of being interrutped and, more importantly, can |
163 | queues, they also have the option of being interrutped and, more |
164 | timeout. All of them also implement the conditional operation. |
164 | importantly, can timeout. All of them also implement the conditional |
165 | Furthemore, this very fundamental interface reaches up to the |
165 | operation. Furthemore, this very fundamental interface reaches up to the |
166 | implementation of futexes - userspace synchronization primitive, which |
166 | implementation of futexes - userspace synchronization primitive, which |
167 | makes it possible for a userspace thread to request a synchronization |
167 | makes it possible for a userspace thread to request a synchronization |
168 | operation with a timeout or a conditional operation.</para> |
168 | operation with a timeout or a conditional operation.</para> |
169 | 169 | ||
170 | <para>From the description above, it should be apparent, that when a |
170 | <para>From the description above, it should be apparent, that when a |
171 | sleeping thread is woken by <code>waitq_wakeup()</code> or when |
171 | sleeping thread is woken by <code>waitq_wakeup()</code> or when |
172 | <code>waitq_sleep_timeout()</code> succeeds immediately, the thread can be |
172 | <code>waitq_sleep_timeout()</code> succeeds immediately, the thread can |
173 | sure that the event has occurred. The thread need not and should not |
173 | be sure that the event has occurred. The thread need not and should not |
174 | verify this fact. This approach is called direct hand-off and is |
174 | verify this fact. This approach is called direct hand-off and is |
175 | characteristic for all passive HelenOS synchronization primitives, with |
175 | characteristic for all passive HelenOS synchronization primitives, with |
176 | the exception as described below.</para> |
176 | the exception as described below.</para> |
177 | </section> |
177 | </section> |
178 | 178 | ||
Line 185... | Line 185... | ||
185 | 185 | ||
186 | <title>Semaphores</title> |
186 | <title>Semaphores</title> |
187 | 187 | ||
188 | <para>The interesting point about wait queues is that the number of |
188 | <para>The interesting point about wait queues is that the number of |
189 | missed wakeups is equal to the number of threads that will not block in |
189 | missed wakeups is equal to the number of threads that will not block in |
190 | <code>watiq_sleep_timeout()</code> and would immediately succeed instead. |
190 | <code>watiq_sleep_timeout()</code> and would immediately succeed |
191 | On the other hand, semaphores are synchronization primitives that will |
191 | instead. On the other hand, semaphores are synchronization primitives |
192 | let predefined amount of threads into their critical section and block |
192 | that will let predefined amount of threads into their critical section |
193 | any other threads above this count. However, these two cases are exactly |
193 | and block any other threads above this count. However, these two cases |
194 | the same. Semaphores in HelenOS are therefore implemented as wait queues |
194 | are exactly the same. Semaphores in HelenOS are therefore implemented as |
195 | with a single semantic change: their wait queue is initialized to have |
195 | wait queues with a single semantic change: their wait queue is |
196 | so many missed wakeups as is the number of threads that the semphore |
196 | initialized to have so many missed wakeups as is the number of threads |
197 | intends to let into its critical section simultaneously.</para> |
197 | that the semphore intends to let into its critical section |
- | 198 | simultaneously.</para> |
|
198 | 199 | ||
199 | <para>In the semaphore language, the wait queue operation |
200 | <para>In the semaphore language, the wait queue operation |
200 | <code>waitq_sleep_timeout()</code> corresponds to semaphore |
201 | <code>waitq_sleep_timeout()</code> corresponds to semaphore |
201 | <code>down</code> operation, represented by the function |
202 | <code>down</code> operation, represented by the function |
202 | <code>semaphore_down_timeout()</code> and by way of similitude the wait |
203 | <code>semaphore_down_timeout()</code> and by way of similitude the wait |
Line 226... | Line 227... | ||
226 | <code>mutex_trylock()</code> and the unlocking operation is called |
227 | <code>mutex_trylock()</code> and the unlocking operation is called |
227 | <code>mutex_unlock()</code>.</para> |
228 | <code>mutex_unlock()</code>.</para> |
228 | </section> |
229 | </section> |
229 | 230 | ||
230 | <section> |
231 | <section> |
231 | <title>Reader/writer locks</title> |
232 | <title>Reader/Writer Locks</title> |
232 | 233 | ||
233 | <indexterm> |
234 | <indexterm> |
234 | <primary>synchronization</primary> |
235 | <primary>synchronization</primary> |
235 | 236 | ||
236 | <secondary>- read/write lock</secondary> |
237 | <secondary>- read/write lock</secondary> |
Line 279... | Line 280... | ||
279 | whether a simpler but equivalently fair solution exists.</para> |
280 | whether a simpler but equivalently fair solution exists.</para> |
280 | 281 | ||
281 | <para>The implementation of rwlocks as it has been already put, makes |
282 | <para>The implementation of rwlocks as it has been already put, makes |
282 | use of one single wait queue for both readers and writers, thus avoiding |
283 | use of one single wait queue for both readers and writers, thus avoiding |
283 | any possibility of starvation. In fact, rwlocks use a mutex rather than |
284 | any possibility of starvation. In fact, rwlocks use a mutex rather than |
284 | a bare wait queue. This mutex is called <emphasis>exclusive</emphasis> and is |
285 | a bare wait queue. This mutex is called <emphasis>exclusive</emphasis> |
285 | used to synchronize writers. The writer's lock operation, |
286 | and is used to synchronize writers. The writer's lock operation, |
286 | <code>rwlock_write_lock_timeout()</code>, simply tries to acquire the |
287 | <code>rwlock_write_lock_timeout()</code>, simply tries to acquire the |
287 | exclusive mutex. If it succeeds, the writer is granted the rwlock. |
288 | exclusive mutex. If it succeeds, the writer is granted the rwlock. |
288 | However, if the operation fails (e.g. times out), the writer must check |
289 | However, if the operation fails (e.g. times out), the writer must check |
289 | for potential readers at the head of the list of sleeping threads |
290 | for potential readers at the head of the list of sleeping threads |
290 | associated with the mutex's wait queue and then proceed according to the |
291 | associated with the mutex's wait queue and then proceed according to the |
291 | procedure outlined above.</para> |
292 | procedure outlined above.</para> |
292 | 293 | ||
293 | <para>The exclusive mutex plays an important role in reader |
294 | <para>The exclusive mutex plays an important role in reader |
294 | synchronization as well. However, a reader doing the reader's lock |
295 | synchronization as well. However, a reader doing the reader's lock |
295 | operation, <code>rwlock_read_lock_timeout()</code>, may bypass this mutex |
296 | operation, <code>rwlock_read_lock_timeout()</code>, may bypass this |
296 | when it detects that:</para> |
297 | mutex when it detects that:</para> |
297 | 298 | ||
298 | <orderedlist> |
299 | <orderedlist> |
299 | <listitem> |
300 | <listitem> |
300 | <para>there are other readers in the critical section and</para> |
301 | <para>there are other readers in the critical section and</para> |
301 | </listitem> |
302 | </listitem> |
Line 313... | Line 314... | ||
313 | conditions are not fulfilled, the reader normally waits until the |
314 | conditions are not fulfilled, the reader normally waits until the |
314 | exclusive mutex is granted to it.</para> |
315 | exclusive mutex is granted to it.</para> |
315 | </section> |
316 | </section> |
316 | 317 | ||
317 | <section> |
318 | <section> |
318 | <title>Condition variables</title> |
319 | <title>Condition Variables</title> |
319 | 320 | ||
320 | <indexterm> |
321 | <indexterm> |
321 | <primary>synchronization</primary> |
322 | <primary>synchronization</primary> |
322 | 323 | ||
323 | <secondary>- condition variable</secondary> |
324 | <secondary>- condition variable</secondary> |
Line 350... | Line 351... | ||
350 | <varname>condition</varname> = <constant>true</constant>; |
351 | <varname>condition</varname> = <constant>true</constant>; |
351 | <function>condvar_signal</function>(<varname>cv</varname>); /* <remark>condvar_broadcast(cv);</remark> */ |
352 | <function>condvar_signal</function>(<varname>cv</varname>); /* <remark>condvar_broadcast(cv);</remark> */ |
352 | <function>mutex_unlock</function>(<varname>mtx</varname>);</programlisting> |
353 | <function>mutex_unlock</function>(<varname>mtx</varname>);</programlisting> |
353 | </example> |
354 | </example> |
354 | 355 | ||
355 | <para>The wait operation, <code>condvar_wait_timeout()</code>, always puts |
356 | <para>The wait operation, <code>condvar_wait_timeout()</code>, always |
356 | the calling thread to sleep. The thread then sleeps until another thread |
357 | puts the calling thread to sleep. The thread then sleeps until another |
357 | invokes <code>condvar_broadcast()</code> on the same condition variable or |
358 | thread invokes <code>condvar_broadcast()</code> on the same condition |
358 | until it is woken up by <code>condvar_signal()</code>. The |
359 | variable or until it is woken up by <code>condvar_signal()</code>. The |
359 | <code>condvar_signal()</code> operation unblocks the first thread blocking |
360 | <code>condvar_signal()</code> operation unblocks the first thread |
360 | on the condition variable while the <code>condvar_broadcast()</code> |
361 | blocking on the condition variable while the |
361 | operation unblocks all threads blocking there. If there are no blocking |
362 | <code>condvar_broadcast()</code> operation unblocks all threads blocking |
362 | threads, these two operations have no efect.</para> |
363 | there. If there are no blocking threads, these two operations have no |
- | 364 | efect.</para> |
|
363 | 365 | ||
364 | <para>Note that the threads must synchronize over a dedicated mutex. To |
366 | <para>Note that the threads must synchronize over a dedicated mutex. To |
365 | prevent race condition between <code>condvar_wait_timeout()</code> and |
367 | prevent race condition between <code>condvar_wait_timeout()</code> and |
366 | <code>condvar_signal()</code> or <code>condvar_broadcast()</code>, the mutex |
368 | <code>condvar_signal()</code> or <code>condvar_broadcast()</code>, the |
367 | is passed to <code>condvar_wait_timeout()</code> which then atomically |
369 | mutex is passed to <code>condvar_wait_timeout()</code> which then |
368 | puts the calling thread asleep and unlocks the mutex. When the thread |
370 | atomically puts the calling thread asleep and unlocks the mutex. When |
369 | eventually wakes up, <code>condvar_wait()</code> regains the mutex and |
371 | the thread eventually wakes up, <code>condvar_wait()</code> regains the |
370 | returns.</para> |
372 | mutex and returns.</para> |
371 | 373 | ||
372 | <para>Also note, that there is no conditional operation for condition |
374 | <para>Also note, that there is no conditional operation for condition |
373 | variables. Such an operation would make no sence since condition |
375 | variables. Such an operation would make no sence since condition |
374 | variables are defined to forget events for which there is no waiting |
376 | variables are defined to forget events for which there is no waiting |
375 | thread and because <code>condvar_wait()</code> must always go to sleep. |
377 | thread and because <code>condvar_wait()</code> must always go to sleep. |
Line 398... | Line 400... | ||
398 | HelenOS by dividing the <code>waitq_sleep_timeout()</code> function into |
400 | HelenOS by dividing the <code>waitq_sleep_timeout()</code> function into |
399 | three pieces:</para> |
401 | three pieces:</para> |
400 | 402 | ||
401 | <orderedlist> |
403 | <orderedlist> |
402 | <listitem> |
404 | <listitem> |
403 | <para><code>waitq_sleep_prepare()</code> prepares the thread to go to |
405 | <para><code>waitq_sleep_prepare()</code> prepares the thread to go |
404 | sleep by, among other things, locking the wait queue;</para> |
406 | to sleep by, among other things, locking the wait queue;</para> |
405 | </listitem> |
407 | </listitem> |
406 | 408 | ||
407 | <listitem> |
409 | <listitem> |
408 | <para><code>waitq_sleep_timeout_unsafe()</code> implements the core |
410 | <para><code>waitq_sleep_timeout_unsafe()</code> implements the core |
409 | blocking logic;</para> |
411 | blocking logic;</para> |
Line 414... | Line 416... | ||
414 | <code>waitq_sleep_timeout_unsafe()</code>; after this call, the wait |
416 | <code>waitq_sleep_timeout_unsafe()</code>; after this call, the wait |
415 | queue spinlock is guaranteed to be unlocked by the caller</para> |
417 | queue spinlock is guaranteed to be unlocked by the caller</para> |
416 | </listitem> |
418 | </listitem> |
417 | </orderedlist> |
419 | </orderedlist> |
418 | 420 | ||
419 | <para>The stock <code>waitq_sleep_timeout()</code> is then a mere wrapper |
421 | <para>The stock <code>waitq_sleep_timeout()</code> is then a mere |
420 | that calls these three functions. It is provided for convenience in |
422 | wrapper that calls these three functions. It is provided for convenience |
421 | cases where the caller doesn't require such a low level control. |
423 | in cases where the caller doesn't require such a low level control. |
422 | However, the implementation of <code>condvar_wait_timeout()</code> does |
424 | However, the implementation of <code>condvar_wait_timeout()</code> does |
423 | need this finer-grained control because it has to interleave calls to |
425 | need this finer-grained control because it has to interleave calls to |
424 | these functions by other actions. It carries its operations out in the |
426 | these functions by other actions. It carries its operations out in the |
425 | following order:</para> |
427 | following order:</para> |
426 | 428 | ||
Line 452... | Line 454... | ||
452 | </orderedlist> |
454 | </orderedlist> |
453 | </section> |
455 | </section> |
454 | </section> |
456 | </section> |
455 | 457 | ||
456 | <section> |
458 | <section> |
457 | <title>Userspace synchronization</title> |
459 | <title>Userspace Synchronization</title> |
458 | 460 | ||
459 | <section> |
461 | <section> |
460 | <title>Futexes</title> |
462 | <title>Futexes</title> |
461 | 463 | ||
462 | <indexterm> |
464 | <indexterm> |
Line 514... | Line 516... | ||
514 | did, then the futex is locked by another thread and the library uses the |
516 | did, then the futex is locked by another thread and the library uses the |
515 | <constant>SYS_FUTEX_SLEEP</constant> syscall to put the thread asleep. |
517 | <constant>SYS_FUTEX_SLEEP</constant> syscall to put the thread asleep. |
516 | If the counter decreased to 0, then there was no contention and the |
518 | If the counter decreased to 0, then there was no contention and the |
517 | thread can enter the critical section protected by the futex. When the |
519 | thread can enter the critical section protected by the futex. When the |
518 | thread later leaves that critical section, it, using library function |
520 | thread later leaves that critical section, it, using library function |
519 | <code>futex_up()</code>, atomically increments the counter. If the counter |
521 | <code>futex_up()</code>, atomically increments the counter. If the |
520 | value increased to one, then there again was no contention and no action |
522 | counter value increased to one, then there again was no contention and |
521 | needs to be done. However, if it increased to zero or even a smaller |
523 | no action needs to be done. However, if it increased to zero or even a |
522 | number, then there are sleeping threads waiting for the futex to become |
524 | smaller number, then there are sleeping threads waiting for the futex to |
523 | available. In that case, the library has to use the |
525 | become available. In that case, the library has to use the |
524 | <constant>SYS_FUTEX_WAKEUP</constant> syscall to wake one sleeping |
526 | <constant>SYS_FUTEX_WAKEUP</constant> syscall to wake one sleeping |
525 | thread.</para> |
527 | thread.</para> |
526 | 528 | ||
527 | <para>So far, futexes are very elegant in that they don't interfere with |
529 | <para>So far, futexes are very elegant in that they don't interfere with |
528 | the kernel when there is no contention for them. Another nice aspect of |
530 | the kernel when there is no contention for them. Another nice aspect of |