NEWS
Introduction
As part of his master thesis Leonard Rapp analyzes the security of various popular unikernels.
The first two blog post discussing the results of an in-depth analysis of the research unikernel RustyHermit can be found here and here. This blog post will give an overview of a basic analysis of the popular unikernels OSv, nanos, Unikraft, and Mini-OS.
Leonard will also present his work at GPN20 on May 21, 4:30pm. For details see the official schedule.
The basic idea of unikernels is to compile a library operating system together with an application into a singe purpose and single address space image which can be run on a hypervisor. This approach is much more lightweight in terms of code size, startup time and performance than a traditional operating system.
Multiple unikernels claim to have a high grade of security or to be even more secure than Linux.
It was decided to analyze the popular unikernel projects OSv, nanos, Unikraft, and Mini-OS for the implementation of ASLR, W^X, stack canaries and the process of generating and handling random numbers.
Summary and Impact
For various unikernel projects, some of the most basic security mitigations present in e.g. Linux are not implemented at all or fundamentally flawed. Thus, it is significantly easier for an attacker to successfully exploit a vulnerable application in one of those unikernels.
| RustyHermit | OSv | nanos | Unikraft | Mini-OS | |
|---|---|---|---|---|---|
| ASLR | ☓ | ☓ | ✓ | ☓ | ☓ |
| W^X | (☓) | ✓ | ✓ | (☓) | ✓ |
| Stack Canaries | - | (☓) | ✓ | (✓) | ☓ |
| Random Numbers | ✓ | ✓ | (☓) | (✓) | ☓ |
Where ✓ means implemented, (✓) means implemented but weak default, (☓) means only partly implemented, partly vulnerable or not best practice, ☓ means not implemented or completely broken, - means not applicable.
Detailed Analysis
Providing a full analysis of all findings would go beyond the scope of this blog post, thus only the key findings are explained in detail here.
The analysis was conducted using a simple test program developed for this thesis as well as using manual code review. All code necessary to reproduce the thesis’ results will be published on GitHub.
RustyHermit
RustyHermit is a research unikernel developed at RWTH Aachen University and written in Rust. The core part of the thesis is an in-depth analysis of RustyHermit, the key findings were already presented here and here.
OSv
Originally, OSv was developed by “Cloudius Systems”, but is now maintained by volunteers.The analyzed version is commit hash 2f1bed26876e26e57bb9402e7f5ca312424ca5b6.
ASLR and W^X
There is an open issue on GitHub discussing the implementation of ASLR and W^X. Also the test script showed that both are not implemented.
Stack Canaries
The main Makefile of OSv uses the -fno-stack-protector flag, which explicitly disables stack smashing protections / stack canaries. When creating an application, one can enable stack canaries by using e.g.\ the -fstack-protector-all flag. While this advises the compiler to generate the necessary stack checking code, the OS is still in charge of generating a random value and placing it at the correct address so that it can be copied onto the stack.
OSv ships the musl libc with its code, but does not use musl’s stack smashing protection code. Instead, OSv implements its own stack smashing protection in runtime.cc, defining a static canary value.
There is a flaw in the canary value initialization process, leading to the canary value always being 0 when explicitly enabling stack smashing protection. Due to time constraints it was not possible to conduct a root cause analysis yet.
Interestingly, the code being faulty might even make it more secure here, as many string based input function stop reading input when a 0 byte occurs. Thus, it might be harder for an attacker to include a 0 byte in the exploit string than to include a static canary not containing a 0 byte.
nanos
nanos is developed with a commercial background and used in production. The analyzed version is commit hash 713ef9f2edefa62ddd8024639d690da9706875e2.
On their website they claim that “Nanos unikernels have all the same security protections you’d find in a Linux system and more”[1]. And indeed, nanos properly implements ASLR, W^X and stack canaries.
Random Numbers
Nanos uses the ChaCha20 algorithm for random number generation. When running nanos as a unix process, the initial seed is generated using the random() function. When running it on PC platform, nanos uses the cryptographically secure random() or rdseed to seed the PRNG. If rdrand / rdseed fails, the clock is used as a fallback seed.
A comment in nanos’ RNG implementation says:
/* likely not a good fallback - look for another */
Indeed, there are more secure options than seeding with the timestamp, as an attacker might be able to brute-force it. Same is true when running as a riscv-virt service. Here the code comment directly proposes more secure options, but they are not implemented:
// XXX add qemu random device? virtio-rng-device?
When running as a virt service, nanos uses rdtsc. This is problematic, because rdtsc presents the CPU cycles since the last reset, so the value has a low entropy since it is retrieved always exactly at the same point in the code shortly after starting the unikernel.
Besides this rather small issues already known to the developers, X41 uncovered a major flaw in the process nanos passes random numbers to the process. A minimal application was written which prints the canary value using the code below:
#include <stdio.h>
void __attribute__ ((noinline)) print_canary() {
long marker = 0xACDC;
printf("Canary: 0x%lx\n", *((&marker)+0x1));
}
int main(int argc, char* argv[]) {
print_canary();
}
The code below was used to run the application multiple times and create a simple statistic of the used canary values.
from pwn import *
import array
canaries = []
for i in range(0, 50):
p = process(['ops', 'run', 'canary'])
p.recvuntil(b'Canary: ')
canary = p.recvline(timeout=1.0).decode("utf-8")[:-1]
print(canary)
canaries.append(int(canary, 16))
p.close()
sleep(0.2)
print("Statistics:")
count = 0
old_value = sorted(canaries)[0]
for canary in sorted(canaries):
if canary == old_value:
count += 1
else:
print(f"{count} x {old_value}")
count = 1
old_value = canary
print(f"{count} x {old_value}")
This lead to the following statistic clearly showing a flaw in the process of random number handling:
Statistics:
12 x 202891351808
13 x 4996285648031596800
16 x 6003937490507091968
9 x 8029762268323730176
As this behavior was uncovered shortly before the master thesis deadline, it was reported to the nanos developers without conducting a root cause analysis first. The nanos developers reacted highly professional and fixed the issue less than two hours after reporting it. According to their analysis, the behavior was caused by their “process initialization code, which was failing to correctly pass random data via the AT_RANDOM auxiliary entry”.
Unikraft
According to the authors, Unikraft implements multiple security mechanisms present in traditional operating systems.
The analyzed version is commit hash da0f1eeb139db3de0e23801920d4b736bc4f49e8.
ASLR
The unikernel build configuration lacks a possibility to compile it as Position Independent Executable (PIE), which is a prerequisite for ASLR. There is a GitHub pull request adding this feature since June 25th, 2021, but it has changes requested by a reviewer and was orphaned for multiple months. Thus, ASLR is not implemented at the moment, but might be in one of the upcoming releases.
W^X
When trying to execute code on the stack and heap, the test program stops with a page fault es expected, But writing at arbitrary addresses in the code segment and executing the written code succeeds.
An attacker exploiting an application containing an arbitrary write vulnerability can use this to overwrite the (kernel) code being executed. This can be used to circumvent security mechanisms like stack canaries and arbitrarily manipulate the unikernel’s core functionality. Possible targets could be the network stack or the hypercall interface.
Stack Canaries
Unikraft uses a custom library called libuksp for stack smashing protection. Depending on the configuration, it either uses 0xFF0A0D00 as terminator canary (which is the default option), a static canary provided via the configuration file (default: 42) or a randomly generated canary. The quality of the random canary depends on the configuration of libukswrand, which is discussed in the section below.
The effectiveness of the stack canary protection depends on the choice of which type of canary to use. A static default canary is only a very weak protection, as the attacker knows the canary value and could simply embed the known canary in the correct position in the attack string. Even a random canary with a static seed or the timestamp as seed might be not optimal, as the but the attacker might be able to brute-force the correct seed and thus reproduce the sequence of pseudo random numbers. For this, the attacker could crash the unikernel by overwriting the canary with an arbitrary value.
When a new unikernel instance is now spawned as replacement for the crashed one, the attacker roughly knows the timestamp, which makes a brute force attack much more feasible.
Random Numbers
In Unikraft, the random number code can be found in libukswrand. It contains a ChaCha implementation (which is used by default) as well a Multiply-with-Carry PRNG implementation. It depends on the configuration which one is used.
The RNG algorithms are initially seeded with either a static number, the time or a value read from rdrand, depending on the configuration. Seeding with the system time is the default here.
Both, the stack canaries and the RNG provide secure options. However, the default values are not the most secure ones, which is not optimal since many users will use the default option.
Mini-OS
Mini-OS is a minimalist OS for the Xen hypervisor, written in C. It implements some basic functionalities and was originally developed to be used as an OS within Xen for dom0 disaggregation, but is also used as a base for multiple unikernel projects. The analyzed version is commit hash 2535683834cc33729cad8d444caa3c170942fe26.
While W^X is properly implemented, ASLR and stack canaries are not implemented at all. The minos.mk file contains the -fno-stack-protector flag to explicitly disable stack smashing protection. When manually enabling it, the unikernel gets stuck and does not generate any output.
Random Numbers
Mini-OS does not use random numbers itself, as ASLR and stack canaries are not implemented and lwIP uses its own RNG. It does not provide a source of cryptographically secure random numbers to use in applications, but gives the test code shown below as an example how to generate random numbers.
63 #ifndef HAVE_LIBC
64 /* Should be random enough for our uses */
65 int rand(void)
66 {
67 static unsigned int previous;
68 struct timeval tv;
69 gettimeofday(&tv, NULL);
70 previous += tv.tv_sec + tv.tv_usec;
71 previous *= RAND_MIX;
72 return previous;
73 }
74 #endif
As shown in lines 63 and line 74, the rand() implementation is encapsulated in a #ifndef HAVE_LIBC block with the effect that the custom rand() implementation is only used if no libc is present. The basic working principle is that the number of current seconds is added to the number of current milliseconds, multiplied with a constant called RAND_MIX and added to the result from the previous rand() call, which is saved in the static unsigned int previous variable.
The problem with this approach is that the possible amount of random numbers after the first call is the amount of possible seconds plus the amount of possible microseconds and thus only 60+1000 = 1060. Multiplying with RAND_MIX spreads the random numbers over a wider range but does not increase the amount of possible random numbers in the first round. Especially in the first round, the amount of possible random numbers is low enough that an attacker can pre-compute and brute-force them.
It is even more insecure when a libc is present, as rand() is used multiple times in the code without calling srand() first. This leads to the PRNG being seeded with 1 and thus producing highly deterministic sequence of numbers.
When discussing the findings with the main developer, he stated that the missing security features are caused by the minimalist approach of the project and other work has a higher priority. However, patches improving the security of Mini-OS are always welcome.
[1] https://nanovms.com/security
About X41 D-SEC GmbH
X41 is an expert provider for application security services. Having extensive industry experience and expertise in the area of information security, a strong core security team of world class security experts enables X41 to perform premium security services.
Fields of expertise in the area of application security are security centered code reviews, binary reverse engineering and vulnerability discovery. Custom research and IT security consulting and support services are core competencies of X41.