Loading and executing code in Windows CE and .NET Compact Framework

Creating applications for Windows operating systems dedicated to devices with limited computing power (Windows CE, Windows Embedded CE 7, Windows Phone or Pocket PC) poses many challenges related to availability of resources to developers.
The example of limited, however valuable resource, is memory.

It would seem that the .NET or Java virtual machines should ensure its optimal use, but experience shows that it is not always the case.

While the amount of RAM in personal computers is growing at a captivating pace, and address space on 64-bit operating systems far exceeds the needs of most applications, embedded systems often operate on the border of their abilities, and their developers are forced to invent techniques to optimize their applications. In this article I will deal with the problem of memory used to store code in Windows CE and Compact Framework.

Virtual address space

Systems of Windows Embedded Compact group are 32-bit and, like other 32-bit Windows they support 4GB of address space.
Just like in the desktop version, the upper 2GB are reserved by the operating system.

Usage of the bottom half of the address space significantly differs from e.g. Windows XP, where 2GB of a virtual memory is reproduced and fully accessible to each running process. In Windows CE only a part of this space is directly available to a developer.

In older versions of the system (Windows CE 5) all processes running on the system must fit in shared 2GB memory. In this area there are 32 slots of an application, 32 MB each. In practice, this means that these systems support a maximum of 32 processes, and the private memory of each is limited to 32 MB. In addition, 32MB are available for the Execute-In-Place (XIP) DLLs, which may be added to the system image by the Platform Builder.

Since version 6.0 this restriction has been lifted and the newer, compact Windows supports 32K processes simultaneously.
In practice, the limitation is the system resources. Just like in Windows XP, private address space is reproduced for running applications, but only half of it is fully accessible to developers. It is worth mentioning, that Windows Embedded CE 6 supports up to 512MB of physical memory. The limit was raised to 3GB in version 7.

For more information on the organization of the memory address space in the Windows CE systems go to [1] and [2].

Memory usage in .NET Compact Framework

The .NET runtime environment organizes the address space of a managed process. Private memory used by the .NET applications can be simply divided into 4 areas presented in Figure 1:

Figure 1

Each of these areas belongs to process private memory and is designed for specific applications. Their roles can be summarised in a following way:

• GC Heap – this is an area of memory reserved for storing instances of reference types (objects). The Garbage Collector deals with allocating and cleaning the memory.
• JIT Heap – here, machine code is stored, generated during compilation of the Common Intermediate Language (CIL) by the JIT compiler.
• App Domain Heap – this is an area to which the metadata of data types is loaded. A representation of the classes and methods used is created in the memory.
• Thread Stacks – 64KB stack is allocated by default for each thread. The operating system deals with threads management, and the CLR can commission their creation. The maximum number of threads is not limited by the architecture of Windows CE.

Loading native code

It is simple in unmanaged applications; the machine code is loaded from the file into memory, and then executed by the processor. In WinCE 5, and older, all DLLs are in 32 MB slot of a process, which often results in a problem known as „DLL Crunch”. It consists in „eating up” the process private memory by loaded libraries. It is presented in the Figure 2.

Figure 2

In this case, the program may run out of available memory, even if the system still has free physical memory.

Loading managed code

The runtime environment of .NET Compact Framework uses the Just-In-Time (JIT) compiler to translate byte code (CIL) into machine code. It has a direct impact on the use of process private memory.

Opposite to native libraries .NET assemblies are treated as memory-mapped files and are not a subject to the DLLs loading mechanism to the private address space of the process. This results from the fact that from the point of view of the operating system they are not DLLs but common binary files.

When the CLR loads the managed library to use, it is mapped in the memory area reserved for memory-mapped files. In Windows CE 5, and older this area has a size of 1GB. It is a significant difference compared with 32 MB of applications private space. Types of data collected from a managed DLL are loaded into the App Domain Heap, and a JIT compiler translates methods on demand, and places their native code in its heap. The size of this heap has a direct impact on consumption of private memory. It should be noted that the native code takes approx. 3 to 4 times more memory than the corresponding CIL.

More about JIT compilation in Compact Framework can be found on the blog [3].

Managed executable file (EXE) is treated by Windows CE in a similar way as native counterparts, which means it is always copied to the address space of the application. It indicates that the contained CIL code will occupy valuable private memory. It could be noticed most easily in the system offering 32 MB of address space. If your application is struggling with a shortage of this resource, you should create a very small EXE and move the code to the linked library. For example, if you put an OptimizedApp class into a DLL, your program can start in a following way:

class Program
{
    static void Main(string[] args)
    {
        OptimizedApp.Main(args);
    }
}

JIT

The use of JIT compiler provides .NET code portability, however it also has drawbacks. The most important one is the necessity of translating CIL in runtime. It extends execution time of methods that have not yet been compiled by the JIT. This delay (next to Garbage Collector) causes non-determinism of action, and is quoted as one of the biggest disadvantages of managed technologies (.NET, Java) in real-time applications.

In the „full”.NET Framework, we can deal with this problem by previously compiling CIL to native code using NGen tool. Unfortunately, this technique is unavailable for compact Windows.

What to do when our application in Compact Framework must meet real-time criteria? We can consider rewriting the most critical section of code to the native version. However, we must remember overhead caused by the P/Invoke and data marshalling, as well as the consequences of loading the native DLLs into the private memory.

Due to the strict time criteria, one more method of optimization was added to the application I am working on. We determined the critical call paths, which must comply with time constraints during the execution. While initialising the application, in background we forcibly call methods that are on these paths. In result this means that before the first „real” invocation their machine code is already in the cache (JIT heap).

The convenience here is the way the JIT works, which always compiles entire methods, even if the call is obviously causing an exception.

public void Foo(object obj)
{
    if (obj == null)
    {
        throw new ArgumentNullException("obj");
    }

    // The actual operation
    // ...
}

For the above presented example simply call Foo (null) to compile method code. Using this optimization technique makes the JIT heap size increases by approx. 1MB after initialization of our application. This means that 1MB of the code will not be compiled before the first call of critical sections. As a result, the first execution of real-time operation is loaded with much less overhead caused by the JIT.

External links

[1] Windows CE .NET Advanced Memory Management – http://msdn.microsoft.com/en-us/library/ms836325.aspx
[2] Windows Embedded CE 6.0 Advanced Memory Management – http://msdn.microsoft.com/en-us/library/bb331824.aspx
[3] The Design of the .Net Compact Framework CLR, Part II: JIT Compiler Design Considerations
– http://blogs.msdn.com/b/stevenpr/archive/2005/12/12/502978.aspx
[4] MemMaker for the .NET Compact Framework – http://robtiffany.com/memmaker-for-the-net-compact-framework/