Missing Memory Protection Mechanisms in the RustyHermit Unikernel


As part of his master thesis Leonard Rapp analyzes the security of RustyHermit, a research unikernel developed at RWTH Aachen University. The first findings are presented in this blog post.

The basic idea of unikernels is to compile a library operating system together with an application into a single purpose and single address space image which can be run on a hypervisor. This approach is much more lightweight in terms of code size, startup time and performance than a traditional operating system.
RustyHermit is written in Rust and developed for high performance computing. It is still under development and not meant to be used in production.

Summary and Impact

Traditional operating systems implement various countermeasures to prevent an attacker from using certain exploitation techniques against a vulnerable application, such as stack canaries (K)ASLR and W^X memory segments.
Small proof of concepts were developed to examine RustyHermit for some classic exploit mitigations.

Even if RustyHermit does not contain a vulnerability, it lacks some protection mechanisms well established in traditional operating systems that hinder attackers from exploiting a vulnerable application.
RustyHermit does not implement any address space randomization. The intended compile procedures do not support stack canaries. The stack and heap are mapped as nonexecutable, but the code segment is writable. Thus, an attacker exploiting an application containing a buffer overflow or arbitrary write vulnerability can easily gain full control over the unikernel and arbitrarily manipulate kernel functionalities.

Detailed Analysis

The full analysis along with proof of concept code will be published as part of the master thesis in April 2022. Nevertheless, the core parts are presented in this blog post.

No Support for Stack Canaries

Stack canaries are random values placed on the stack before the return address. The idea is to check the value directly before the function returns in order to detect if the stack was overwritten.
According to the official Rust documentation, the Rust compiler does not support stack smashing protection.
Thus, a Rust application containing a buffer overflow vulnerability in an unsafe code block would not be protected by stack canaries by design.
While this might be a relatively unlikely scenario, a vulnerability in a C application running on top of RustyHermit might be more common.
With the hermit-playground repository, one can build such a pure C application for RustyHermit.
If one uses the proposed docker container to build the application and sets the CFLAGS=--fstack-protector-all environment variable, this leads to an error message indicating that libssp (the C library for stack smashing protection) is not present in the docker container.

/opt/hermit/lib/gcc/x86_64-hermit/6.3.0/../../../../x86_64-hermit/bin/ld: cannot find -lssp_nonshared
/opt/hermit/lib/gcc/x86_64-hermit/6.3.0/../../../../x86_64-hermit/bin/ld: cannot find -lssp
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/test.dir/build.make:84: test] Error 1
make[1]: *** [CMakeFiles/Makefile2:76: CMakeFiles/test.dir/all] Error 2
make: *** [Makefile:84: all] Error 2

It was not possible to compile C applications for RustyHermit with stack smashing protection enabled when following the official instructions.
Stack smashing protection not being present has the effect that an attacker can gain control over the program execution if the application contains a buffer overflow vulnerability by overwriting the stack up to the return address.

This issue was already fixed by the maintainers shortly after reporting it.


To test whether RustyHermit randomizes the memory layout, a simple test program was developed allocating variables in the data, heap and stack segment as well as defining a function which is placed in the code segment. When the code is executed, the addresses of the allocated variables and the function are printed.

fn main() {                          
    println!("===static data===");   
    static MOSTLY :&str = "harmless";
    println!("MOSTLY: {} at {:p}", MOSTLY, MOSTLY as *const str);

    let so_long = String::from("and thanks for all the fish!");
    println!("so_long: {} at {:p}", so_long, &so_long as *const String);

    println!("build_hyperspace_route() at {:p}", build_hyperspace_route as *const ());

    let answer :i32 = 42;
    println!("answer: {} at {:p}", answer, &answer as *const i32);

fn build_hyperspace_route() {

When recompiling and executing the test code multiple times, it always prints the same addresses, indicating no randomization is implemented.
This has the effect that attackers who are able to overwrite the return address (which is easier due to the lack of stack canaries) can simply replace it by a target address of their choice without needing another vulnerability to leak the memory layout.

Writable Code Segment

According to the W^X policy, the stack and heap segment are mapped as nonexecutable. This hinders an attacker from writing exploit code to stack or heap and execute it afterwards by overwriting the return address when exploiting a buffer overflow.
The code segment does not stick to the W^X policy and is writable. The idea of the proof of concept code below is to manipulate the content of the function pointer to func_to_overwrite(). It originally consists of a single ret statement followed by int3 instructions (opcode: 0xccccccc3) and is overwritten with call rbx; ret; int3 (opcode: 0xccc3d3ff). Before executing the manipulated function, the call_target() function’s address is written to ebx.

fn main() {
    println!("Call unmodified func_to_overwrite()");
    println!("Returned from unmodified func_to_overwrite()");
    println!("Call modified func_to_overwrite()");
    unsafe {
        // overwrite original function consisting of a single ret instruction with: call rbx; ret; int3;
        *(func_to_overwrite as *mut i64) = 0xccc3d3ff;
        // move address of target function to rbx
        asm!("mov rbx, {0}", in(reg) call_target as *const ());
        // call modified function
    println!("Returned from modified func_to_overwrite()");

fn func_to_overwrite() {

fn call_target() {
    println!("Called target!");

As one can see in the output, the function placed in the code segment was successfully modified and the modified version executed, it calls the call_target() function.

Call unmodified func_to_overwrite()
Returned from unmodified func_to_overwrite()
Call modified func_to_overwrite()
Called target!
Returned from modified func_to_overwrite()

An attacker exploiting an application containing an arbitrary write vulnerability can use this to overwrite the (kernel) code being executed. This can be used to circumvent security mechanisms like stack canaries and arbitrarily manipulate the unikernel’s core functionality. Possible targets could be the network stack or the hypercall interface.

All findings were also verified in the RustyHermit source code, which is available on GitHub.


Some of the problems were already known to the developers. As RustyHermit is an ongoing research project which is not meant for use in production, it was agreed upon publicly document the security considerations in GitHub issues:

About X41 D-SEC GmbH

X41 is an expert provider for application security services. Having extensive industry experience and expertise in the area of information security, a strong core security team of world class security experts enables X41 to perform premium security services.

Fields of expertise in the area of application security are security centered code reviews, binary reverse engineering and vulnerability discovery. Custom research and IT security consulting and support services are core competencies of X41.