Skip to content

move some tools/lkl/Makefile.autoconf logic to kconfig#603

Merged
tavip merged 10 commits intolkl:masterfrom
ddiss:lkl_mv_autoconf_to_kconfig
Jun 16, 2025
Merged

move some tools/lkl/Makefile.autoconf logic to kconfig#603
tavip merged 10 commits intolkl:masterfrom
ddiss:lkl_mv_autoconf_to_kconfig

Conversation

@ddiss
Copy link
Copy Markdown

@ddiss ddiss commented Jun 2, 2025

This is (currently) one extra change atop the #602 s390 fixes.
I think it makes sense to move the Makefile.autoconf logic closer to where it's used, so arch specific kernel default settings can be moved into arch/lkl/Kconfig . Kconfig can call external shell scripts, which should allow us to share some logic between arch and tools while avoiding the cat kernel.config >> ../../.config gymnastics.

The following changes since commit d2df129665a47f8a2ebedcee45fb6e8172ee8170:

  Merge pull request #605 from ddiss/windows_ci (2025-06-09 11:43:50 -0700)

are available in the Git repository at:

  https://github.com/ddiss/linux lkl_mv_autoconf_to_kconfig

for you to fetch changes up to c9f0472ec6a65fe619db0b4b953385cd5f80c873:

  lkl: drop explicit CONFIG_SHMEM=n setting (2025-06-12 19:56:13 +1000)

----------------------------------------------------------------
David Disseldorp (9):
      arch/lkl: fix Kconfig whitespace to match rest of kernel
      lkl: remove obsolete STACK_HASH_ORDER setting
      arch/lkl: set OUTPUT_FORMAT, 64BIT, CPU_BIG_ENDIAN defaults
      arch/lkl: use a common helper to determine OUTPUT_FORMAT
      lkl: set CONFIG_LKL_FUZZING=y in fuzzing_defconfig
      lkl: set host memcpy/set/move defaults in Kconfig
      lkl: move KASAN_SHADOW_OFFSET/_SIZE defaults to Kconfig
      lkl: drop LKL_MEMORY_START and LKL_TASK_UNMAPPED_BASE autoconf
      lkl: drop explicit CONFIG_SHMEM=n setting

Sumanth Korikkar (1):
      mm/shmem: inline shmem_is_huge() for disabled transparent hugepages

 arch/lkl/Kconfig                           | 174 +++++++++++++++--------------
 arch/lkl/configs/fuzzing_defconfig         |   1 +
 arch/lkl/scripts/cc-objdump-file-format.sh |   8 ++
 include/linux/shmem_fs.h                   |   9 ++
 mm/shmem.c                                 |   6 -
 tools/lkl/Makefile.autoconf                |  35 ------
 6 files changed, 107 insertions(+), 126 deletions(-)
 create mode 100755 arch/lkl/scripts/cc-objdump-file-format.sh

@ddiss ddiss marked this pull request as draft June 2, 2025 05:52
@ddiss
Copy link
Copy Markdown
Author

ddiss commented Jun 2, 2025

the clang build is throwing

*** Default configuration is based on 'defconfig'
ld.lld: error: unknown argument '-print-output-format'
arch/lkl/Kconfig:42: syntax error
arch/lkl/Kconfig:41: invalid statement
arch/lkl/Kconfig:42: invalid statement

I need to revisit the logic at:

config OUTPUT_FORMAT
       string "Output format"
       default $(shell,$(LD) -r -print-output-format) if LD_IS_BFD
       default "elf64-x86-64" if LD_IS_LLD && $(success,test "$(CROSS_COMPILE)" = "x86_64-linux-gnu")

@ddiss
Copy link
Copy Markdown
Author

ddiss commented Jun 3, 2025

...

I need to revisit the logic at:

config OUTPUT_FORMAT
       string "Output format"
       default $(shell,$(LD) -r -print-output-format) if LD_IS_BFD
       default "elf64-x86-64" if LD_IS_LLD && $(success,test "$(CROSS_COMPILE)" = "x86_64-linux-gnu")

I wonder whether we could use something like this instead:

$ echo 'void foo(void) {}'| gcc -x c - -c -o out && objdump -p out|awk '/file format/ {print $4}'
elf64-x86-64
$ echo 'void foo(void) {}'| clang -x c - -c -o out && llvm-objdump -p out|awk '/file format/ {print $4}'
elf64-x86-64

i.e. with $(CC) and $(OBJDUMP) it'd be common for GCC and LLVM. The existing behaviour of only allowing elf64-x86-64 with LLVM appears a bit broken. Either way this would come as change separate to the Makefile.autoconf -> Kconfig move.

@ddiss ddiss force-pushed the lkl_mv_autoconf_to_kconfig branch from 72f3d60 to 683a1f0 Compare June 3, 2025 13:53
@ddiss
Copy link
Copy Markdown
Author

ddiss commented Jun 3, 2025

v2:

  • use tabs in Kconfig like the rest of the kernel
  • use a gcc/llvm common helper script to determine OUTPUT_FORMAT

@ddiss
Copy link
Copy Markdown
Author

ddiss commented Jun 3, 2025

This is a scheduled Windows Server 2019 brownout. The Windows Server 2019 image will be removed on 2025-06-30.

It'd be nice to get Windows builds tested for this PR. Does anyone know how to upgrade the workflow to target a more recent (e.g. Server 2025) version?

@ddiss ddiss force-pushed the lkl_mv_autoconf_to_kconfig branch from df87005 to 6804d3a Compare June 4, 2025 05:41
@ddiss
Copy link
Copy Markdown
Author

ddiss commented Jun 4, 2025

Windows 2022 build / tests are passing 🎉 . I'll keep the draft tag for now while I look at what other kernel .config autoconf options can be easily moved into arch/lkl/Kconfig. I also think it makes sense to first merge the s390 fixes via #602 .

@ddiss ddiss force-pushed the lkl_mv_autoconf_to_kconfig branch from 6804d3a to 3bc246b Compare June 5, 2025 10:48
@ddiss
Copy link
Copy Markdown
Author

ddiss commented Jun 5, 2025

update:

  • moved the cc-objdump-file-format.sh helper under the existing arch/lkl/scripts instead of creating a new arch/lkl/tools dir.
  • moved CONFIG_LKL_FUZZING and CONFIG_LKL_HOST_MEM* default logic from Makefile.autoconf into arch/lkl/Kconfig

ddiss added 8 commits June 10, 2025 11:20
Use tabs instead of spaces.

Signed-off-by: David Disseldorp <ddiss@suse.de>
CONFIG_STACK_HASH_ORDER was removed via mainline kernel commit
f998792 ("lib/stackdepot: replace CONFIG_STACK_HASH_ORDER with
automatic sizing") at around the same time as this was added via
downstream LKL commit facd006 ("lkl: add KASAN support").

The automatic sizing uses a minimum of 12, which matches the obsolete
setting.

Signed-off-by: David Disseldorp <ddiss@suse.de>
Kconfig makes it relatively easy to set kernel config parameters based
on compiler settings, etc. So move some of the logic from
tools/lkl/Makefile.autoconf into arch/lkl/Kconfig .
Moving non-tools specific parameters out of Makefile.autoconf should
make it easier to build LKL as a standalone library.

Signed-off-by: David Disseldorp <ddiss@suse.de>
The existing scripts work for GCC but restrict LLVM builds to
elf64-x86-64 only. The new cc-objdump-file-format.sh helper script works
with both gcc/clang and objdump/llvm-objdump for CC and OBJDUMP
respectively.

Signed-off-by: David Disseldorp <ddiss@suse.de>
Makefile.autoconf::do_autoconf_fuzzing sets KCONFIG=fuzzing_defconfig so
appending CONFIG_LKL_FUZZING=y to .config via tools/lkl/kernel.config
shouldn't be necessary.

Signed-off-by: David Disseldorp <ddiss@suse.de>
The LKL_HOST_MEMCPY, LKL_HOST_MEMSET and LKL_HOST_MEMMOVE kernel config
parameters are currently set via tools/lkl/Makefile.autoconf based on
whether OUTPUT_FORMAT starts with "elf64" or "elf32".
Use the same logic in arch/lkl/Kconfig to set the parameter defaults and
remove the corresponding Makefile.autoconf set_kernel_config calls.

Signed-off-by: David Disseldorp <ddiss@suse.de>
OUTPUT_FORMAT is known within Kconfig, so we can use the same
"elf64-x86-64" specific logic when setting the defaults for
CONFIG_KASAN_SHADOW_OFFSET and CONFIG_KASAN_SHADOW_SIZE.

The LKL_HOST_CONFIG_KASAN_SHADOW_OFFSET/_SIZE autoconf variables appear
to be unused, so drop them for now. If necessary we could put them back
in arch/lkl/include/generated/uapi/asm/config.h .

Signed-off-by: David Disseldorp <ddiss@suse.de>
The LKL_MEMORY_START=0x50000000 and LKL_TASK_UNMAPPED_BASE=0x100000
kernel config values set in Makefile.autoconf match the arch/lkl/Kconfig
defaults, so avoid explicitly setting them.

The corresponding LKL_HOST_CONFIG_LKL_MEMORY_START/_TASK_UNMAPPED_BASE
lkl_autoconf.h consts appear to be unused, so also drop them for now.

Signed-off-by: David Disseldorp <ddiss@suse.de>
@ddiss ddiss force-pushed the lkl_mv_autoconf_to_kconfig branch from ebda356 to 62514fe Compare June 10, 2025 14:24
@ddiss
Copy link
Copy Markdown
Author

ddiss commented Jun 10, 2025

update:

  • rebase against master branch and reorder
  • add drop LKL_MEMORY_START and LKL_TASK_UNMAPPED_BASE autoconf change

sumanthkorikkar and others added 2 commits June 12, 2025 19:55
In order to  minimize code size (CONFIG_CC_OPTIMIZE_FOR_SIZE=y),
compiler might choose to make a regular function call (out-of-line) for
shmem_is_huge() instead of inlining it. When transparent hugepages are
disabled (CONFIG_TRANSPARENT_HUGEPAGE=n), it can cause compilation
error.

mm/shmem.c: In function `shmem_getattr':
./include/linux/huge_mm.h:383:27: note: in expansion of macro `BUILD_BUG'
  383 | #define HPAGE_PMD_SIZE ({ BUILD_BUG(); 0; })
      |                           ^~~~~~~~~
mm/shmem.c:1148:33: note: in expansion of macro `HPAGE_PMD_SIZE'
 1148 |                 stat->blksize = HPAGE_PMD_SIZE;

To prevent the possible error, always inline shmem_is_huge() when
transparent hugepages are disabled.

Link: https://lkml.kernel.org/r/20240409155407.2322714-1-sumanthk@linux.ibm.com
Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Acked-by: David Hildenbrand <david@redhat.com>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Ilya Leoshkevich <iii@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
(cherry picked from commit 1f73784)
The previous mainline commit 1f73784 ("mm/shmem: inline
shmem_is_huge() for disabled transparent hugepages") fixes the described
build failure, so leave the CONFIG_SHMEM setting up to the user (it's
enabled by default with MMU=y).

Signed-off-by: David Disseldorp <ddiss@suse.de>
@ddiss
Copy link
Copy Markdown
Author

ddiss commented Jun 12, 2025

update:

  • cherry-pick mainline commit 1f73784 ("mm/shmem: inline shmem_is_huge() for disabled transparent hugepages")
  • drop the explicit CONFIG_SHMEM=n setting from Makefile.autoconf, now that it builds successfully

The changes so here far are all minor and incremental. I'll drop the WIP flag for now.
I think my only real (minor) concern with the series is the multiple cc-objdump-file-format.sh calls, but I don't expect any significant built time increase.

@ddiss ddiss marked this pull request as ready for review June 12, 2025 10:38
@ddiss ddiss changed the title WIP: move some tools/lkl/Makefile.autoconf logic to kconfig move some tools/lkl/Makefile.autoconf logic to kconfig Jun 12, 2025
@ddiss
Copy link
Copy Markdown
Author

ddiss commented Jun 12, 2025

Windows CI failure appears unrelated to the changes here:

  TEST       *        nanosleep 
  TEST       not ok   nanosleep 
sleep 78570700 (ret=0), expected sleep 87654321

Copy link
Copy Markdown
Member

@tavip tavip left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems reasonable and a good improvement IMO, thanks @ddiss for doing all this work! Lets wait a couple of days to get @rodionov a chance to take a look as well.

@rodionov
Copy link
Copy Markdown

rodionov commented Jun 14, 2025

Thank you for looping me in and doing this great clean up, folks! LGTM!

@tavip tavip merged commit b89fbb9 into lkl:master Jun 16, 2025
17 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants