Subversion Repositories HelenOS-doc

to fill gaps after functionality that the genuine HelenOS code did

to fill gaps after functionality that the genuine HelenOS code did

not provide itself.

not provide itself.

There is simply too much third party software that is somehow related to

There is simply too much third party software that is somehow related to

HelenOS to be covered all. This chapter attempts to present our experience

HelenOS to be covered all. This chapter attempts to present our experience

\section{Communication tools}

\section{Communication tools}

Although the developers know each other in person, the development, with the

Although the developers know each other in person, the development, with the

exception of kernel camps, has been pretty much independent as far as locality

exception of kernel camps, has been pretty much independent as far as locality

and time goes. In order to work effectively, we have established several communication

and time goes. In order to work effectively, we have established several communication

to control subsriptions and search mailing list archives on-line.

to control subsriptions and search mailing list archives on-line.

\end{description}

\end{description}

\section{Third party components of HelenOS}

\section{Third party components of HelenOS}

HelenOS itself contains third party software. In the first place, amd64 and ia32 architectures

HelenOS itself contains third party software. In the first place, amd64 and ia32 architectures

after the Kernel Camp 2005 when {\MD} had made HelenOS Multiboot specification compliant. Because of

after the Kernel Camp 2005 when {\MD} had made HelenOS Multiboot specification compliant. Because of

Grub, HelenOS can be booted from several types of devices. More importantly, we use

Grub, HelenOS can be booted from several types of devices. More importantly, we use

Grub to load HelenOS userspace modules as well.

Grub to load HelenOS userspace modules as well.

Another third-party piece of the HelenOS operating system is the userspace {\tt malloc()}.

Another third-party piece of the HelenOS operating system is the userspace {\tt malloc()}.

domain {\tt dlmalloc} instead. This allocator could be easily integrated into our uspace tree

domain {\tt dlmalloc} instead. This allocator could be easily integrated into our uspace tree

and has proven itself in other projects as well. Its derivative, {\tt ptmalloc}, has been part of the

and has proven itself in other projects as well. Its derivative, {\tt ptmalloc}, has been part of the

GNU C library for some time. However, the version we are using is not optimized for SMP and multithreading.

GNU C library for some time. However, the version we are using is not optimized for SMP and multithreading.

We plan to eventually replace it with another allocator.

We plan to eventually replace it with another allocator.

library was simplified and ported to HelenOS. Even though filesystem

library was simplified and ported to HelenOS. Even though filesystem

calls were removed from the library, it still heavily depends on {\tt libc}.

calls were removed from the library, it still heavily depends on {\tt libc}.

By porting {\tt libpci} to HelenOS, we demonstrated that applications and libraries

By porting {\tt libpci} to HelenOS, we demonstrated that applications and libraries

are, given enough effort, portable to HelenOS.

are, given enough effort, portable to HelenOS.

\section{Build tools}

\section{Build tools}

Assembler, linker and compiler are by all means the very focal point of attention

Assembler, linker and compiler are by all means the very focal point of attention

of all operating system projects. Quality of these tools influences

of all operating system projects. Quality of these tools influences

operating system performance and, what is more important, stability. HelenOS has

operating system performance and, what is more important, stability. HelenOS has

(i.e. the compiler). There is only little chance that it could be compiled and

(i.e. the compiler). There is only little chance that it could be compiled and

linked using some other tools unless those tools are compatible with the GNU build tools.

linked using some other tools unless those tools are compatible with the GNU build tools.

As our project declares support for five different processor architectures,

As our project declares support for five different processor architectures,

we needed to have five different flavors of the build utilities installed.

we needed to have five different flavors of the build utilities installed.

From the second point of view, we have tested these programs by our operating system.

From the second point of view, we have tested these programs by our operating system.

Because of the scope and uniqueness of this testing and because we did find some issues,

Because of the scope and uniqueness of this testing and because we did find some issues,

we want to dedicate some more space to what we have found.

we want to dedicate some more space to what we have found.

\subsection{Bochs}

\subsection{Bochs}

It is capable of emulating ia32 machine and for some time also amd64.

It is capable of emulating ia32 machine and for some time also amd64.

Bochs is an emulator and thus the slowest from virtual environments capable

Bochs is an emulator and thus the slowest from virtual environments capable

of simulating the same cathegory of hardware. On the other hand, it is extremely

of simulating the same cathegory of hardware. On the other hand, it is extremely

portable, compared to much faster virtualizers and emulators using dynamic translation

portable, compared to much faster virtualizers and emulators using dynamic translation

of instructions. Lately, there have been some plans to develop or port dynamic translation

of instructions. Lately, there have been some plans to develop or port dynamic translation

simulation, but this has also proven not very useful as we often needed to debug

simulation, but this has also proven not very useful as we often needed to debug

problems that existed only in multiprocessor configurations, which {\tt gdb}

problems that existed only in multiprocessor configurations, which {\tt gdb}

does not understand.

does not understand.

\subsection{GXemul}

\subsection{GXemul}

used it only for mips32 emulation in both little-endian and big-endian modes.

used it only for mips32 emulation in both little-endian and big-endian modes.

It seems to be pretty featurefull and evolving but we don't use all its functionality.

It seems to be pretty featurefull and evolving but we don't use all its functionality.

GXemul is very user friendly and has debugging features. It is more realistic

GXemul is very user friendly and has debugging features. It is more realistic

than msim. However, our newly introduced TLS support triggered a bug in the {\tt rdhwr}

than msim. However, our newly introduced TLS support triggered a bug in the {\tt rdhwr}

instruction emulation while msim functioned as expected. Fortunatelly, the author

instruction emulation while msim functioned as expected. Fortunatelly, the author

of GXemul is very cooperative and has fixed the problem for future versions as well as

of GXemul is very cooperative and has fixed the problem for future versions as well as

provided a quick hack for the old version.

provided a quick hack for the old version.

\subsection{msim}

\subsection{msim}

Its simulated environment is not very realistic, but the processor simulation

Its simulated environment is not very realistic, but the processor simulation

is good enough for operating system development. In this regard, the simulator is

is good enough for operating system development. In this regard, the simulator is

comparable to HP's ia64 simulator Ski. Another similar aspect of these two is

comparable to HP's ia64 simulator Ski. Another similar aspect of these two is

relatively strong debugger.

relatively strong debugger.

Curiously, this simulator contained the biggest number of defects and inaccuracies

Curiously, this simulator contained the biggest number of defects and inaccuracies

that we have ever discovered in a simulator.  Fortunately, all of them have been

that we have ever discovered in a simulator.  Fortunately, all of them have been

eventually fixed.

eventually fixed.

\subsection{PearPC}

\subsection{PearPC}

no debugging features, but fortunatelly its sources are available under

no debugging features, but fortunatelly its sources are available under

an open source license. This enabled {\OP} and {\MD} to alter its sources

an open source license. This enabled {\OP} and {\MD} to alter its sources

in a way that this modified version allowed some basic debugging.

in a way that this modified version allowed some basic debugging.

\subsection{QEMU}

\subsection{QEMU}

ia32 and amd64. It can simulate SMP, but contrary to Bochs, it uses dynamic

ia32 and amd64. It can simulate SMP, but contrary to Bochs, it uses dynamic

translation of emulated instructions and performs much better because of

translation of emulated instructions and performs much better because of

that.

that.

This emulator seemed to realistically emulate the {\tt hlt} instruction,

This emulator seemed to realistically emulate the {\tt hlt} instruction,

with {\tt gdb} can be pretty comfortable\footnote{Especially when the kernel is

with {\tt gdb} can be pretty comfortable\footnote{Especially when the kernel is

compiled with {\tt -g3}.} until one needs to debug a SMP kernel running on multiple

compiled with {\tt -g3}.} until one needs to debug a SMP kernel running on multiple

processors.

processors.

\subsection{Simics}

\subsection{Simics}

This proprietary piece of software was available to us under an academic license for free.

This proprietary piece of software was available to us under an academic license for free.

Simics can be set to simulate many different configurations of many different machines.

Simics can be set to simulate many different configurations of many different machines.

It has the most advanced debugging features we have ever seen. To highlight some, its

It has the most advanced debugging features we have ever seen. To highlight some, its

memory access tracing ability has been really helpfull to us. During device driver

memory access tracing ability has been really helpfull to us. During device driver

during application processors start. Another bugs found were related to amd64 and mips32.

during application processors start. Another bugs found were related to amd64 and mips32.

As for amd64, Simics did not report general protection fault when {\tt EFER.NXE} was 0 and a non-executable

As for amd64, Simics did not report general protection fault when {\tt EFER.NXE} was 0 and a non-executable

page was found (\#4214). As for mips32, Simics misemulated {\tt MSUB} and {\tt MSUBU} instructions.

page was found (\#4214). As for mips32, Simics misemulated {\tt MSUB} and {\tt MSUBU} instructions.

\subsection{Ski}

\subsection{Ski}

Ski is just an Itanium processor simulator and as such does not simulate a real machine. In fact, there

Ski is just an Itanium processor simulator and as such does not simulate a real machine. In fact, there

is no firmware and no configuration tables (e.g. memory map) present in Ski! On the other hand, the missing parts can be supplied externally\footnote{This

is no firmware and no configuration tables (e.g. memory map) present in Ski! On the other hand, the missing parts can be supplied externally\footnote{This

is actually how Linux runs in this simulator.}. The simulator provides means of interaction with

is actually how Linux runs in this simulator.}. The simulator provides means of interaction with

host system devices via Simulator SystemCalls (SSC). The simulator itself has graphical interface

host system devices via Simulator SystemCalls (SSC). The simulator itself has graphical interface

with pretty powerful, but not as good as those of Simics, debugging facilities.

with pretty powerful, but not as good as those of Simics, debugging facilities.

that Ski confuses VMA and LMA. This, what we believe to be a bug in Ski, has not shown in Linux since Linux always has

that Ski confuses VMA and LMA. This, what we believe to be a bug in Ski, has not shown in Linux since Linux always has

LMA equal to VMA. People from the Ski mailing list had tried to help us but our repeated problem report didn't

LMA equal to VMA. People from the Ski mailing list had tried to help us but our repeated problem report didn't

make it far enough for the HP to fix or at least clarify the issue. Finally, we adopted a workaround implemented by {\JJ}

make it far enough for the HP to fix or at least clarify the issue. Finally, we adopted a workaround implemented by {\JJ}

that simply swaps LMA and the program entry point in the kernel ELF image.

that simply swaps LMA and the program entry point in the kernel ELF image.

HelenOS development. It virtualizes the ia32 host machine. Since VMware

HelenOS development. It virtualizes the ia32 host machine. Since VMware

version 5.5, we made use of its possibility to run the guest system

version 5.5, we made use of its possibility to run the guest system

(i.e. HelenOS) on multiple processors. VMware has no support for

(i.e. HelenOS) on multiple processors. VMware has no support for

debugging but is very useful for compatibility and regression testing

debugging but is very useful for compatibility and regression testing

because it's closest to the real hardware. VMware, being a virtualizer,

because it's closest to the real hardware. VMware, being a virtualizer,