Skip to content

refactor(tests): improve transaction waiting logic and epoch checks i… #5057

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Frozen wants to merge 2 commits into
base: dev
Choose a base branch
from
+271 −104
Draft

refactor(tests): improve transaction waiting logic and epoch checks i… #5057

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2ca3ca7
refactor(tests): improve transaction waiting logic and epoch checks i…
Frozen Jun 12, 2026
79176ba
refactor(ci): enhance caching and build processes for localnet tests
Frozen Jun 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .github/workflows/ci-pr.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,21 @@ jobs:
go-version: '1.24.2'
cache-dependency-path: go/src/github.com/harmony-one/harmony/go.sum

- &cache-native-libs

@mur-me mur-me Jun 15, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Idea is good, but you aren't changing the way how CI works rn, because bls and mcl are still building every time.

Main idea of cache is to cache some actions one time and skip them on the next run if they are present.

What we are seeing on the 2 runs of the same PR is almost same timing results.

  • one thing - make on the harmony repo automatically build mcl and bls again.

Example diagram of the build stage:

flowchart TD
    A["Checkout harmony/mcl/bls"] --> B["Set up Go"]
    B --> C["actions/cache restores native mcl/bls cache"]

    C --> D{"Exact cache hit?"}

    D -->|yes| E["Cached bin/lib/obj restored"]
    D -->|no| F["No exact cached native outputs"]

    E --> G["Build mcl runs anyway"]
    F --> G

    G --> H["Build bls runs anyway"]
    H --> I["Build harmony runs"]

    I --> J["Job completes"]

    J --> K{"Cache miss and job success?"}
    K -->|yes| L["actions/cache may save native outputs"]
    K -->|no| M["No new native cache saved"]
Loading

name: Cache native mcl/bls libraries
uses: actions/cache@v4

@mur-me mur-me Jun 15, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lest use best practice here - last version + sha1 # v1.1.1 comment

with:
path: |
go/src/github.com/harmony-one/mcl/bin
go/src/github.com/harmony-one/mcl/lib
go/src/github.com/harmony-one/mcl/obj
go/src/github.com/harmony-one/bls/bin
go/src/github.com/harmony-one/bls/lib
go/src/github.com/harmony-one/bls/obj
key: ${{ runner.os }}-native-libs-${{ hashFiles('go/src/github.com/harmony-one/mcl/**', 'go/src/github.com/harmony-one/bls/**') }}
restore-keys: |
${{ runner.os }}-native-libs-

- &build-mcl
name: Build mcl
run: make -j4
Expand All @@ -82,6 +97,7 @@ jobs:
- *checkout-mcl
- *checkout-bls
- *setup-go
- *cache-native-libs
- *build-mcl
- *build-bls
- *build-harmony
Expand All @@ -107,6 +123,7 @@ jobs:
- *checkout-mcl
- *checkout-bls
- *setup-go
- *cache-native-libs
- *build-mcl
- *build-bls
- *build-harmony
Expand All @@ -121,6 +138,10 @@ jobs:
path: go/src/github.com/harmony-one/harmony-test
persist-credentials: false

- &setup-docker-buildx

@mur-me mur-me Jun 15, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need this one?
all github runners has docker by default

name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Run RPC checker
run: |
make go-get
Expand All @@ -140,10 +161,12 @@ jobs:
- *checkout-mcl
- *checkout-bls
- *setup-go
- *cache-native-libs
- *build-mcl
- *build-bls
- *build-harmony
- *checkout-harmony-test
- *setup-docker-buildx

- name: Run pyhmy checker
run: |
Expand Down
14 changes: 5 additions & 9 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,24 +99,21 @@ debug-ext:
# add VERBOSE=true before bash or run `export VERBOSE=true` on the shell level for have max logging
# add LEGACY_SYNC=true before bash or run `export LEGACY_SYNC=true` on the shell level to switch to the legacy sync
./test/debug.sh ./test/configs/local-resharding-with-external.txt 64 64 &
echo sleep 10s before creating the external validator
sleep 10
./test/wait_rpc_ready.sh http://localhost:9500 60 1
bash ./test/build-localnet-validator.sh

debug-multi-bls:
# add VERBOSE=true before bash or run `export VERBOSE=true` on the shell level for have max logging
# add LEGACY_SYNC=true before bash or run `export LEGACY_SYNC=true` on the shell level to switch to the legacy sync
./test/debug.sh ./test/configs/local-multi-bls.txt 64 64 &
echo sleep 10s before creating the external validator
sleep 10
./test/wait_rpc_ready.sh http://localhost:9500 60 1
bash ./test/build-localnet-validator.sh

debug-multi-bls-with-terminal:
# add VERBOSE=true before bash or run `export VERBOSE=true` on the shell level for have max logging
# add LEGACY_SYNC=true before bash or run `export LEGACY_SYNC=true` on the shell level to switch to the legacy sync
screen -L -Logfile ./tmp_log/localnet_terminal.log -dmS localnet bash -c './test/debug.sh ./test/configs/local-multi-bls.txt 64 64; echo "Press any key to exit..."; read -n 1'
echo sleep 10s before creating the external validator
sleep 10
./test/wait_rpc_ready.sh http://localhost:9500 60 1
bash ./test/build-localnet-validator.sh
screen -r localnet

Expand All @@ -129,8 +126,7 @@ debug-multi-bls-multi-ext-node:
# add VERBOSE=true before bash or run `export VERBOSE=true` on the shell level for have max logging
# add LEGACY_SYNC=true before bash or run `export LEGACY_SYNC=true` on the shell level to switch to the legacy sync
./test/debug.sh ./test/configs/local-multi-bls-multi-ext-node.txt &
echo sleep 10s before creating the external validator
sleep 10
./test/wait_rpc_ready.sh http://localhost:9500 60 1
bash ./test/build-localnet-validator.sh

clean:
Expand Down Expand Up @@ -271,4 +267,4 @@ debug-delete-log:


docker-go-test:
docker run --rm -it -v "$PWD":/go/src/github.com/harmony-one/harmony frozen621/harmony-test bash -c 'make go-test'
docker run --rm -it -v "$PWD":/go/src/github.com/harmony-one/harmony frozen621/harmony-test bash -c 'make go-test'
102 changes: 57 additions & 45 deletions internal/configs/sharding/localnet.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,9 @@ const (

localnetEpochBlock1 = 5

localnetBootstrapEpochBlocks = 4
localnetBootstrapEpochs = 3

localnetVdfDifficulty = 5000 // This takes about 10s to finish the vdf
)

Expand Down Expand Up @@ -80,70 +83,79 @@ func (ls localnetSchedule) BlocksPerEpoch() uint64 {
return localnetConfig.BlocksPerEpochV2
}

func (ls localnetSchedule) blocksInEpoch(epochNum uint64) uint64 {
if epochNum > 0 &&
localnetBootstrapEpochBlocks > 0 &&
localnetBootstrapEpochs > 0 &&
epochNum <= localnetBootstrapEpochs {
return localnetBootstrapEpochBlocks
}
if params.LocalnetChainConfig.IsTwoSeconds(big.NewInt(int64(epochNum))) {
return ls.BlocksPerEpoch()
}
return ls.BlocksPerEpochOld()
}

func (ls localnetSchedule) twoSecondsFirstBlock() uint64 {
if params.LocalnetChainConfig.TwoSecondsEpoch.Uint64() == 0 {
return 0
}
return (params.LocalnetChainConfig.TwoSecondsEpoch.Uint64()-1)*ls.BlocksPerEpochOld() + localnetEpochBlock1
return ls.EpochLastBlock(params.LocalnetChainConfig.TwoSecondsEpoch.Uint64()-1) + 1
}

func (ls localnetSchedule) CalcEpochNumber(blockNum uint64) *big.Int {
firstBlock2s := ls.twoSecondsFirstBlock()
switch {
case blockNum < localnetEpochBlock1:
if blockNum < localnetEpochBlock1 {
return big.NewInt(0)
case blockNum < firstBlock2s:
return big.NewInt(int64((blockNum-localnetEpochBlock1)/ls.BlocksPerEpochOld() + 1))
default:
extra := uint64(0)
if firstBlock2s == 0 {
blockNum -= localnetEpochBlock1
extra = 1
}
low, high := uint64(1), uint64(1)
for ls.EpochLastBlock(high) < blockNum {
low = high + 1
high *= 2
}
for low < high {
mid := low + (high-low)/2
if ls.EpochLastBlock(mid) < blockNum {
low = mid + 1
} else {
high = mid
}
return big.NewInt(int64(extra + (blockNum-firstBlock2s)/ls.BlocksPerEpoch() + params.LocalnetChainConfig.TwoSecondsEpoch.Uint64()))
}
return big.NewInt(int64(low))
}

func (ls localnetSchedule) IsLastBlock(blockNum uint64) bool {
switch {
case blockNum < localnetEpochBlock1-1:
return false
case blockNum == localnetEpochBlock1-1:
return true
default:
firstBlock2s := ls.twoSecondsFirstBlock()
switch {
case blockNum >= firstBlock2s:
if firstBlock2s == 0 {
blockNum -= localnetEpochBlock1
}
return ((blockNum-firstBlock2s)%ls.BlocksPerEpoch() == ls.BlocksPerEpoch()-1)
default: // genesis
blocks := ls.BlocksPerEpochOld()
return ((blockNum-localnetEpochBlock1)%blocks == blocks-1)
}
}
epoch := ls.CalcEpochNumber(blockNum).Uint64()
return blockNum == ls.EpochLastBlock(epoch)
}

func (ls localnetSchedule) EpochLastBlock(epochNum uint64) uint64 {
switch {
case epochNum == 0:
if epochNum == 0 {
return localnetEpochBlock1 - 1
default:
switch {
case params.LocalnetChainConfig.IsTwoSeconds(big.NewInt(int64(epochNum))):
blocks := ls.BlocksPerEpoch()
firstBlock2s := ls.twoSecondsFirstBlock()
block2s := (1 + epochNum - params.LocalnetChainConfig.TwoSecondsEpoch.Uint64()) * blocks
if firstBlock2s == 0 {
return block2s - blocks + localnetEpochBlock1 - 1
}
return firstBlock2s + block2s - 1
default: // genesis
blocks := ls.BlocksPerEpochOld()
return localnetEpochBlock1 + blocks*epochNum - 1
}

bootstrapEpochs := uint64(0)
if localnetBootstrapEpochBlocks > 0 && localnetBootstrapEpochs > 0 {
bootstrapEpochs = localnetBootstrapEpochs
if bootstrapEpochs > epochNum {
bootstrapEpochs = epochNum
}
}

blocks := bootstrapEpochs * localnetBootstrapEpochBlocks
regularEpochs := epochNum - bootstrapEpochs
twoSecondsEpoch := params.LocalnetChainConfig.TwoSecondsEpoch.Uint64()
oldEpochs := uint64(0)
if regularEpochs > 0 && twoSecondsEpoch > bootstrapEpochs+1 {
oldEpochs = twoSecondsEpoch - bootstrapEpochs - 1
if oldEpochs > regularEpochs {
oldEpochs = regularEpochs
}
}
v2Epochs := regularEpochs - oldEpochs

blocks += oldEpochs * ls.BlocksPerEpochOld()
blocks += v2Epochs * ls.BlocksPerEpoch()
return localnetEpochBlock1 + blocks - 1
}

func (ls localnetSchedule) VdfDifficulty() int {
Expand Down
55 changes: 47 additions & 8 deletions internal/configs/sharding/localnet_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ func TestLocalnetEpochCalculation(t *testing.T) {

//test config init
lnConfig := GetLocalnetConfig()
localnetBlocksPerEpoch := lnConfig.BlocksPerEpoch
localnetBlocksPerEpochV2 := lnConfig.BlocksPerEpochV2

//test epoch calculations
Expand Down Expand Up @@ -40,19 +39,59 @@ func TestLocalnetEpochCalculation(t *testing.T) {
backup := params.LocalnetChainConfig.TwoSecondsEpoch
params.LocalnetChainConfig.TwoSecondsEpoch = big.NewInt(0)
check(0, localnetEpochBlock1-1)
check(1, localnetEpochBlock1+localnetBlocksPerEpochV2-1)
check(2, localnetEpochBlock1+localnetBlocksPerEpochV2*2-1)
check(1, localnetEpochBlock1+localnetBootstrapEpochBlocks-1)
check(2, localnetEpochBlock1+localnetBootstrapEpochBlocks*2-1)
check(3, localnetEpochBlock1+localnetBootstrapEpochBlocks*3-1)
check(4, localnetEpochBlock1+localnetBootstrapEpochBlocks*3+localnetBlocksPerEpochV2-1)

params.LocalnetChainConfig.TwoSecondsEpoch = big.NewInt(1)
check(0, localnetEpochBlock1-1)
check(1, localnetEpochBlock1+localnetBlocksPerEpochV2-1)
check(2, localnetEpochBlock1+localnetBlocksPerEpochV2*2-1)
check(1, localnetEpochBlock1+localnetBootstrapEpochBlocks-1)
check(2, localnetEpochBlock1+localnetBootstrapEpochBlocks*2-1)
check(3, localnetEpochBlock1+localnetBootstrapEpochBlocks*3-1)
check(4, localnetEpochBlock1+localnetBootstrapEpochBlocks*3+localnetBlocksPerEpochV2-1)

params.LocalnetChainConfig.TwoSecondsEpoch = big.NewInt(2)
check(0, localnetEpochBlock1-1)
check(1, localnetEpochBlock1+localnetBlocksPerEpoch-1)
check(2, localnetEpochBlock1+localnetBlocksPerEpoch+localnetBlocksPerEpochV2-1)
check(3, localnetEpochBlock1+localnetBlocksPerEpoch+localnetBlocksPerEpochV2*2-1)
check(1, localnetEpochBlock1+localnetBootstrapEpochBlocks-1)
check(2, localnetEpochBlock1+localnetBootstrapEpochBlocks*2-1)
check(3, localnetEpochBlock1+localnetBootstrapEpochBlocks*3-1)
check(4, localnetEpochBlock1+localnetBootstrapEpochBlocks*3+localnetBlocksPerEpochV2-1)

params.LocalnetChainConfig.TwoSecondsEpoch = backup
}

func TestLocalnetBootstrapEpochCalculation(t *testing.T) {
InitLocalnetConfig(16, 16)

backupTwoSecondsEpoch := params.LocalnetChainConfig.TwoSecondsEpoch
params.LocalnetChainConfig.TwoSecondsEpoch = big.NewInt(2)
defer func() { params.LocalnetChainConfig.TwoSecondsEpoch = backupTwoSecondsEpoch }()

check := func(epoch, expected uint64) {
if got := LocalnetSchedule.EpochLastBlock(epoch); got != expected {
t.Fatalf("wrong EpochLastBlock at epoch %d. expected: %d got: %d.", epoch, expected, got)
}
if !LocalnetSchedule.IsLastBlock(expected) {
t.Fatalf("%d is not LastBlock", expected)
}
epochStart := uint64(0)
if epoch > 0 {
epochStart = LocalnetSchedule.EpochLastBlock(epoch-1) + 1
}
for blockNo := epochStart; blockNo <= expected; blockNo++ {
if isLastBlock := LocalnetSchedule.IsLastBlock(blockNo); isLastBlock != (blockNo == expected) {
t.Fatalf("IsLastBlock for %d is wrong. expected %v got %v", blockNo, blockNo == expected, isLastBlock)
}
if got := LocalnetSchedule.CalcEpochNumber(blockNo).Uint64(); got != epoch {
t.Fatalf("CalcEpochNumber for %d is wrong. expected %d got %d", blockNo, epoch, got)
}
}
}

check(0, localnetEpochBlock1-1)
check(1, localnetEpochBlock1+localnetBootstrapEpochBlocks-1)
check(2, localnetEpochBlock1+localnetBootstrapEpochBlocks*2-1)
check(3, localnetEpochBlock1+localnetBootstrapEpochBlocks*3-1)
check(4, localnetEpochBlock1+localnetBootstrapEpochBlocks*3+16-1)
}
2 changes: 1 addition & 1 deletion internal/params/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -382,7 +382,7 @@ var (
DevnetExternalEpoch: EpochTBD,
TestnetExternalEpoch: EpochTBD,
TimestampValidationEpoch: big.NewInt(0),
IsOneSecondEpoch: big.NewInt(4),
IsOneSecondEpoch: big.NewInt(0),
EIP2537PrecompileEpoch: EpochTBD,
EIP1153TransientStorageEpoch: EpochTBD,
EIP7939CLZEpoch: EpochTBD,
Expand Down
10 changes: 2 additions & 8 deletions scripts/api_test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,10 +69,7 @@ while getopts "lbvp" OPTION; do
--data "{\"jsonrpc\":\"2.0\",\"method\":\"hmy_sendRawTransaction\",\"params\":[\""$SIGNED_RAW_TRANSACTION"\"],\"id\":1}" | jq -r '.result')
echo "TRANSACTION_HASH:"
echo $TRANSACTION_HASH
sleep 20s
TRANSACTION=$(curl --location --request POST "http://l0.b.hmny.io:9500" \
--header "Content-Type: application/json" \
--data "{\"jsonrpc\":\"2.0\",\"method\":\"hmy_getTransactionByHash\",\"params\":[\"$TRANSACTION_HASH\"],\"id\":1}")
TRANSACTION=$(./test/wait_tx_receipt.sh "http://l0.b.hmny.io:9500" "$TRANSACTION_HASH" 60 1)

echo "TRANSACTION:"
echo "$TRANSACTION"
Expand Down Expand Up @@ -113,10 +110,7 @@ while getopts "lbvp" OPTION; do
--data "{\"jsonrpc\":\"2.0\",\"method\":\"hmy_sendRawTransaction\",\"params\":[\""$SIGNED_RAW_TRANSACTION"\"],\"id\":1}" | jq -r '.result')
echo "TRANSACTION_HASH:"
echo $TRANSACTION_HASH
sleep 20s
TRANSACTION=$(curl --location --request POST "http://localhost:9500" \
--header "Content-Type: application/json" \
--data "{\"jsonrpc\":\"2.0\",\"method\":\"hmy_getTransactionByHash\",\"params\":[\"$TRANSACTION_HASH\"],\"id\":1}")
TRANSACTION=$(./test/wait_tx_receipt.sh "http://localhost:9500" "$TRANSACTION_HASH" 60 1)

echo "TRANSACTION:"
echo "$TRANSACTION"
Expand Down
Loading
Loading