add overlaybd-resize tool for userspace ext4 resize on overlaybd images#408
Merged
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a new userspace CLI (overlaybd-resize) to resize ext4 filesystems inside overlaybd images without requiring a kernel block device, along with unit tests and build system integration for the required Photon/e2fsprogs components.
Changes:
- Adds
src/tools/overlaybd-resize.cppimplementing shrink/expand flows for images with or without an existing upper layer. - Adds
src/test/resize_test.cppwith unit tests covering resize scenarios and data integrity checks. - Integrates resize support into the build via CMake updates (Photon/e2fsprogs fetching and new tool/test targets).
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| src/tools/overlaybd-resize.cpp | New CLI tool implementing ext4 resize on overlaybd images in userspace. |
| src/tools/CMakeLists.txt | Adds build/install target for overlaybd-resize gated by PHOTON_ENABLE_RESIZE. |
| src/test/resize_test.cpp | New gtest-based unit tests for shrink/expand and edge cases. |
| src/test/CMakeLists.txt | Adds resize_test target and registers it with CTest (gated). |
| CMake/Findphoton.cmake | Updates Photon fetch revision and enables resize-related build flags. |
| CMake/Finde2fs.cmake | Updates e2fsprogs fetch revision and exposes resize build directories for object linking. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+39
to
+46
| uint64_t multiplier = 1024ULL * 1024; // default MB | ||
| if (end && (*end == 'G' || *end == 'g')) | ||
| multiplier = 1024ULL * 1024 * 1024; | ||
| else if (end && (*end == 'T' || *end == 't')) | ||
| multiplier = 1024ULL * 1024 * 1024 * 1024; | ||
| else if (end && (*end == 'M' || *end == 'm')) | ||
| multiplier = 1024ULL * 1024; | ||
|
|
Comment on lines
+276
to
+285
| ImageService *imgservice = nullptr; | ||
| photon::fs::IFile *imgfile = nullptr; | ||
| create_overlaybd(service_config_path, config_path, imgservice, imgfile); | ||
| if (!imgfile) { | ||
| fprintf(stderr, "failed to open overlaybd image\n"); | ||
| if (is_expand) { | ||
| update_vsize(existing_data_path, (uint64_t)current_vsize); | ||
| } | ||
| return 1; | ||
| } |
Comment on lines
+354
to
+364
| // Step 3: open image via ImageService | ||
| ImageService *imgservice = nullptr; | ||
| photon::fs::IFile *imgfile = nullptr; | ||
| create_overlaybd(service_config_path, tmp_config, imgservice, imgfile); | ||
| if (!imgfile) { | ||
| fprintf(stderr, "failed to open overlaybd image\n"); | ||
| unlink(tmp_config.c_str()); | ||
| unlink(data_path.c_str()); | ||
| unlink(index_path.c_str()); | ||
| return 1; | ||
| } |
Comment on lines
+107
to
+123
| static std::string parse_upper_data_path(const std::string &config_path) { | ||
| FILE *fp = fopen(config_path.c_str(), "r"); | ||
| if (!fp) return ""; | ||
| char readbuf[65536]; | ||
| rapidjson::FileReadStream is(fp, readbuf, sizeof(readbuf)); | ||
| rapidjson::Document doc; | ||
| doc.ParseStream(is); | ||
| fclose(fp); | ||
|
|
||
| if (doc.HasParseError()) return ""; | ||
| if (!doc.HasMember("upper")) return ""; | ||
| if (!doc["upper"].IsObject()) return ""; | ||
| if (!doc["upper"].HasMember("data")) return ""; | ||
| if (!doc["upper"]["data"].IsString()) return ""; | ||
|
|
||
| return doc["upper"]["data"].GetString(); | ||
| } |
Comment on lines
+34
to
+53
| static const char *TEST_IMG = "/tmp/resize_test.img"; | ||
|
|
||
| static photon::fs::IFile *create_image(uint64_t size_mb) { | ||
| auto file = photon::fs::open_localfile_adaptor( | ||
| TEST_IMG, O_RDWR | O_CREAT | O_TRUNC, 0644, 0); | ||
| if (!file) return nullptr; | ||
| if (file->ftruncate(size_mb * MB) < 0) { | ||
| delete file; | ||
| return nullptr; | ||
| } | ||
| if (photon::fs::make_extfs(file) < 0) { | ||
| delete file; | ||
| return nullptr; | ||
| } | ||
| return file; | ||
| } | ||
|
|
||
| static void cleanup() { | ||
| ::unlink(TEST_IMG); | ||
| } |
Comment on lines
+139
to
+141
| // virtual_size is at offset 48 in HeaderTrailer (after magic0[8] + magic1[16] + size[4] + flags[4] + index_offset[8] + index_size[8]) | ||
| uint64_t *vsize_ptr = (uint64_t *)(buf + 48); | ||
| *vsize_ptr = new_vsize; |
BigVan
reviewed
Jun 18, 2026
|
|
||
| CLI::App app{"overlaybd-resize: resize ext4 filesystem on overlaybd image in userspace"}; | ||
| app.add_option("--config", config_path, | ||
| "overlaybd image config (config.v1.json from rpull)") |
| app.add_option("--size", size_str, | ||
| "target size (e.g. 8G, 500G, 8192M)") | ||
| ->required(); | ||
| app.add_option("--data_dir", data_dir, |
Member
There was a problem hiding this comment.
remove this...
do resize on the default 'upper' in config.v1.json
OsentryO
force-pushed
the
feat/overlaybd-resize
branch
from
eb26289 to
17c40a2
Compare
Comment on lines
+34
to
+35
| if (PHOTON_ENABLE_RESIZE) | ||
| set_source_files_properties( |
Comment on lines
+53
to
+54
| if (PHOTON_ENABLE_RESIZE) | ||
| set_source_files_properties( |
Comment on lines
+67
to
+69
| if (PHOTON_ENABLE_RESIZE) | ||
| install(TARGETS overlaybd-resize DESTINATION /opt/overlaybd/bin) | ||
| endif() |
Comment on lines
+14
to
+15
| set(E2FS_RESIZE_DIR ${e2fsprogs_SOURCE_DIR}/build/resize CACHE STRING "path to e2fsprogs resize build dir") | ||
| set(E2FS_INSTALL_LIB_DIR ${e2fsprogs_SOURCE_DIR}/build/v1.47.0-opt/lib CACHE STRING "path to e2fsprogs install-libs output") |
Comment on lines
+172
to
+176
| if (!parse_upper_paths(config_path, data_path, index_path) || data_path.empty()) { | ||
| fprintf(stderr, "error: config has no upper writable layer. " | ||
| "overlaybd-resize requires an existing upper layer.\n"); | ||
| return 1; | ||
| } |
Comment on lines
+45
to
+49
| auto f = open_localfile_adaptor(data_path.c_str(), O_RDONLY, 0, 0); | ||
| if (!f) { | ||
| fprintf(stderr, "failed to open data file for reading vsize: %s\n", data_path.c_str()); | ||
| return -1; | ||
| } |
Comment on lines
+87
to
+91
| auto f = open_localfile_adaptor(path.c_str(), O_RDWR, 0, 0); | ||
| if (!f) { | ||
| fprintf(stderr, "failed to open file for vsize update: %s\n", path.c_str()); | ||
| return -1; | ||
| } |
Comment on lines
+227
to
+230
| struct stat st; | ||
| imgfile->fstat(&st); | ||
| printf("Image vsize: %lld MB, target: %llu MB\n", | ||
| (long long)st.st_size / 1024 / 1024, target_size / 1024 / 1024); |
OsentryO
force-pushed
the
feat/overlaybd-resize
branch
2 times, most recently
from
0f4b3ff to
73489e0
Compare
| @@ -0,0 +1,102 @@ | |||
| #include <photon/common/alog.h> | |||
Comment on lines
+626
to
+630
| auto rw = (LSMT::IFileRW *)m_file; | ||
| struct stat st; | ||
| m_file->fstat(&st); | ||
| uint64_t current = (uint64_t)st.st_size; | ||
|
|
Comment on lines
+659
to
+665
| if (target_size < current) { | ||
| // Shrink: update vsize after filesystem is shrunk | ||
| if (rw->update_vsize(target_size) != 0) { | ||
| LOG_ERROR("filesystem resized but failed to update vsize. Re-run to retry."); | ||
| return -1; | ||
| } | ||
| } |
Comment on lines
+40
to
+44
| ${E2FS_RESIZE_DIR}/resize2fs.o | ||
| ${E2FS_RESIZE_DIR}/extent.o | ||
| ${E2FS_RESIZE_DIR}/resource_track.o | ||
| ${E2FS_LIBRARIES} | ||
| com_err |
OsentryO
force-pushed
the
feat/overlaybd-resize
branch
from
73489e0 to
6f2d6c2
Compare
Comment on lines
3
to
+5
| set(PHOTON_ENABLE_EXTFS ON) | ||
| set(PHOTON_ENABLE_RESIZE ON) | ||
| add_definitions(-DPHOTON_ENABLE_RESIZE) |
| } | ||
| DEFER({ delete imgservice; }); | ||
|
|
||
| auto imgfile = (ImageFile *)imgservice->create_image_file(config_path.c_str(), ""); |
Comment on lines
+33
to
+39
| # overlaybd-resize: userspace ext4 resize for overlaybd images | ||
| if (NOT ORIGIN_EXT2FS) | ||
| add_executable(overlaybd-resize overlaybd-resize.cpp) | ||
| target_include_directories(overlaybd-resize PUBLIC ${PHOTON_INCLUDE_DIR}) | ||
| target_link_libraries(overlaybd-resize photon_static overlaybd_image_lib) | ||
| set_target_properties(overlaybd-resize PROPERTIES INSTALL_RPATH "/opt/overlaybd/lib") | ||
| endif() |
Comment on lines
+52
to
+54
| if (NOT ORIGIN_EXT2FS) | ||
| install(TARGETS overlaybd-resize DESTINATION /opt/overlaybd/bin) | ||
| endif() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add
overlaybd-resizetool that performs ext4 filesystem resize (shrink/expand) on overlaybd images in userspace, without kernel block device.Changes
src/tools/overlaybd-resize.cpp— CLI tool that resizes ext4 filesystem on existing writable layer + updates vsizesrc/test/resize_test.cpp— 10 unit tests covering shrink/expand/relocation/edge casesCMake/Findphoton.cmake: point to upstream release/0.6 (commit 0178d14) which includesresize_extfsbackport (extfs: add resize_extfs interface for ext4 filesystem resize alibaba/PhotonLibOS#1276)CMake/Finde2fs.cmake: point to merged PR resize2fs: optimize shrink/expand performance and fix sparse_super2 bug data-accelerator/e2fsprogs#10, addE2FS_RESIZE_DIRandE2FS_INSTALL_LIB_DIRvariablessrc/tools/CMakeLists.txt: addoverlaybd-resizebuild target (gated byPHOTON_ENABLE_RESIZE)src/test/CMakeLists.txt: addresize_testbuild targetUsage
Size is specified in GB (e.g.,
8for 8GB,500for 500GB). Only integer values are accepted.Requirements:
upperlayer (data + index files)Design
Operates on existing writable layer (upper required):
Key features:
exit(-1)increate_overlaybd, enabling proper error handling and rollbackDependencies
PHOTON_ENABLE_RESIZE=ON(backported to release/0.6 in PR extfs: add resize_extfs interface for ext4 filesystem resize alibaba/PhotonLibOS#1276)Testing