Stack overflow when enabling `LIBPOSIX_PROCESS_SIGNAL`

[razvand]

Describe the bug

Enabling LIBPOSIX_PROCESS_SIGNAL (and disabling LIBUKSIGNAL), with the application making calls to signal-related functions, will cause a stack overflow:

[    1.378739] CRIT: [libukvmem] Guard page 0x10000c3000 of stack VMA 0x10000c3000 - 0x10000d8000 hit!
rt_sigaction(0xb, 0x10000ffdb0, ...) = 0x0
[    1.381522] CRIT: [libukvmem] Guard page 0x10000c3000 of stack VMA 0x10000c3000 - 0x10000d8000 hit!
[    1.383468] CRIT: [libposix_process] Cannot deliver SIGSEGV for pf at 0x10000c3af8

This can be fixed by increasing the stack size, i.e. adding / updating the lines below in the Kraftfile:

    CONFIG_STACK_SIZE_PAGE_ORDER: 5
    CONFIG_AUXSTACK_SIZE_PAGE_ORDER: 5

Steps to reproduce

Run the tokio1-rust1.75 example in the catalog repository; other examples running in bincompat (mostly Rust, Node or Python-based would also fail). Run the commands below (or add them to a script:

# Remove previous clone, if any.
rm -fr catalog
                                                                                                                                                                                              
# Clone repository.                                                                                                                                                                           
git clone https://github.com/unikraft/catalog catalog

# Update base Kraftfile configuration.
sed -i "/CONFIG_LIBPOSIX_PROCESS_MULTITHREADING.*$/a\\
    CONFIG_LIBPOSIX_PROCESS_CLONE: 'y'\n\
    CONFIG_LIBPOSIX_PROCESS_SIGNAL: 'y'\n\
    CONFIG_LIBPOSIX_PROCESS_MAX_PID: '48'" catalog/library/base/Kraftfile
sed -i "/CONFIG_STACK_SIZE_PAGE_ORDER.*$/a\\
    CONFIG_AUXSTACK_SIZE_PAGE_ORDER: 5'" catalog/library/base/Kraftfile

# Update runtime in example configuration to point to updated base.
sed -i "s/^runtime:.*/runtime: local-base:latest/" catalog/examples/tokio1-rust1.75/Kraftfile

# Build and package base.
cd catalog/library/base
rm -fr .config* .unikraft
kraft build --log-level debug --log-type basic --no-cache --no-update --plat qemu --arch x86_64 .
kraft pkg --name "local-base:latest" --plat qemu --arch x86_64

# Run example.
cd ../../examples/tokio1-rust1.75
memory="512Mi"
ports="8080:8080"
kraft run --log-level debug --log-type basic --rm --plat qemu --arch x86_64 --memory "$memory" --port "$ports" .

Note that updating the catalog/library/base/Kraftfile to:

    CONFIG_STACK_SIZE_PAGE_ORDER: 5 # 128 * 4K = 512K

will make things work. You will have to, of course, rebuild and repackage base and rerun the application:

# Build and package base.
cd catalog/library/base
rm -fr .config* .unikraft
kraft build --log-level debug --log-type basic --no-cache --no-update --plat qemu --arch x86_64 .
kraft pkg --name "local-base:latest" --plat qemu --arch x86_64

# Run example.
cd ../../examples/tokio1-rust1.75
memory="512Mi"
ports="8080:8080"
kraft run --log-level debug --log-type basic --rm --plat qemu --arch x86_64 --memory "$memory" --port "$ports" .

Expected behavior

The application should work, giving out an output such as:

en1: Interface is up
en1: Set IPv4 address 10.0.2.15 mask 255.255.255.0 gw 10.0.2.2
Powered by Unikraft Pan (0.19.1~1d3ad9d)
Listening on: http://0.0.0.0:8080
[...]

Which architectures were you using or does this bug affect?

x86_64

Which platforms were you using or does this bug affect?

kvm

Relevant log output

[    1.378739] CRIT: [libukvmem] Guard page 0x10000c3000 of stack VMA 0x10000c3000 - 0x10000d8000 hit!
rt_sigaction(0xb, 0x10000ffdb0, ...) = 0x0
[    1.381522] CRIT: [libukvmem] Guard page 0x10000c3000 of stack VMA 0x10000c3000 - 0x10000d8000 hit!
[    1.383468] CRIT: [libposix_process] Cannot deliver SIGSEGV for pf at 0x10000c3af8

[michpappas]

This is an out-of-stack situation that would require bumping the affected application's stack size. Normally one would do that via CONFIG_APPELFLOADER_STACK_SZMB, but for various reasons that value is not propagated to libposix-process, so when execve() reallocates the stack, it uses CONFIG_STACK_SIZE_PAGE_ORDER. If the latter is smaller than the former, the application will have a smaller stack than what it expects. Clearly this is incorrect, as the application's stack size should be controlled by RLIMIT_STACK, and at the moment rlimit syscalls are stubs. We will address that soon. Until then, affected applications can work around the issue by bumping CONFIG_STACK_SIZE_PAGE_ORDER.

[Harshdhall01]

Hi @razvand @michpappas

This PR has been open since August addressing issue #1693. I implemented
the workaround suggested by @michpappas - bumping CONFIG_STACK_SIZE_PAGE_ORDER
and CONFIG_AUXSTACK_SIZE_PAGE_ORDER to 5 to provide 512KB stack space.

I've tested this with the tokio1-rust1.75 example and confirmed it resolves
the stack overflow errors. The application now runs successfully without
guard page violations.

Would appreciate your review when you have a moment! Happy to make any
adjustments if needed.

[michpappas]

rlimit is not coming on 0.21.0 so it's mostly down to @razvand to decide whether to deploy an interim fix to the catalog for this.

[razvand]

Hi, @Harshdhall01.

What is the PR you mention? This is an issue.

As @michpappas said, this is not planned to make its way to the Unikraft core repository until 0.21.0. The best way is for your to create the corresponding contributions (as PRs) to the catalog and catalog-core repositories.

[Harshdhall01]

Hi @razvand!

I actually already created the PR for the catalog repository back in August:
unikraft/catalog#225

The PR implements the workaround in catalog/library/base as suggested by
@michpappas. It's been open for a few months waiting for review.