Suspected Wrapped Call Handing Back Wrong Data at the 32-bit Boundary

[shakeheartbreak]

So I've hit a bug: When you reach the first fork in Outrun 2, playing under the setup you've made possible, the fork section never loads. The road stops drawing and the car can only drive into a void:

In addition to play behaviour, this also reproduces in attract mode. The first track itself loads fine.

Things I've tested:

• The game runs correctly on x86 with same ROM set, so this is ARM/box64-specific.
• Not GPU: reproduces with LIBGL_ALWAYS_SOFTWARE=1 and TU_DEBUG=sysmem
• Not dynarec: survives STRONGMEM=3/BIGBLOCK=0/SAFEFLAGS=2 and full BOX64_DYNAREC=0.
• Not fopen: the fork's course files open fine (logged my_fopen64); the only failures are optional files my working x86 sets also lack.

So it looks downstream of the open and in the wrapped read/data path. My guess is it's a wrapped call handing back wrong data at the 32-bit boundary. I have a reliable repro here, I'm happy to log whatever call you suggest, test a patch, or send the setup.

This is a relatively vague report on a specific setup, so I mean you no offence by asking.

Any help deeply appreciated.

[ptitSeb]

Ok, that's a strange mis-behaviour! I'll probably have to try to reproduce this myself to understand what is going one, I suspect it will not be easy to just have an idea of what is going wrong here.

[shakeheartbreak]

Thank you as always - but it's not fair to expect you to reproduce a whole one-game setup just to chase this down for me. I'll try to get more detail to bring the fault to you instead.

My C skills aren't there yet - and I'm still finding my way around how box64 wraps things too - but I'd rather put the work in here than just drop a problem on you.

My plan: hand-wrap fread so I can log what it hands back for the course files at the fork, and come back with the specific call and how it's misbehaving. Then you'd have a concrete lead, instead of a repro to chase.

Please do let me know if there's a call you'd point me at first and I can start there. Otherwise, I'll report back when I can. Thanks again.

[shakeheartbreak]

Update: I wrote the wrapper. GOW(fread, LEpLLS) -> GOWM(fread, LEEpLLS) plus a my32_fread that logs every read on the course files.

fread is clear. The course data reaches memory correctly and the fault is downstream of the read.

I wanted to test libm next by forcing it to emulation, but BOX64_EMULATED_LIBS fails on current Debian i386 libraries:

Error: RELR type of Relocation unsupported (only REL)
Error: relocating symbols in elf libm.so.6

..which cascades into libGL failing to initialise and an abort.

Is there a way round that, or a wrapped call you would point me at instead?

[ptitSeb]

Using emulated version of libc, libm and pthread is not supported with box64. while libm can probably be doable, adding other type of relocation is a big task. Maybe try to gather some older version of libm?

Still, I suspect it's not a wrapping issue but an emulation issue. But it would be something in both Interpreter & Dynarec.. But maybe some x87 behaviour? or something that relly on 80bits full precision?

[shakeheartbreak]

So I've narrowed it, though not to a named call.

The fork transition does no file I/O - all stage data loads during START and is byte-correct in memory forty seconds before the failure. The game logs nothing at the failure either, no error or warning. So it is a pure in-memory computation producing a wrong but plausible result, which fits your precision suggestion.

Flags tried, all confirmed accepted and none changed anything: BOX64_DYNAREC_X87DOUBLE=1 and =2, BOX64_X87_NO80BITS=1, and BOX64_DYNAREC_STRONGMEM=4 with WEAKBARRIER=0 and ALIGNED_ATOMICS=1.

So, given this is in-memory (and silent), how would you narrow it further? Is there a way to trace x87 operations or a build option that would help? Your help is immense.

[ptitSeb]

Well, the first question is: does the emulator use x87 or only SSE/AVX? (I suppose a DUMP of the generated code, with BOX64_DYNAREC_DUMP=2 woul dbe usefull (if you have a TRACE enabled build, also use BOX64_TRACE=0-1 for better x86 opcode decoding with libZyDis)

[shakeheartbreak]

Answering the first question.. it uses both.

From a dynarec dump, SSE scalar is the bulk of the float work: MOVSS 9259, MULSS 3669, ADDSS 1963, UCOMISS 815, DIVSS 553, CVTSI2SS 363. There's a fair amount of x87 as well: FSTP 2586, FLD 1915, FMUL 492, FXCH 461, FLDCW 420, FILD 356, FISTP 207.

One thing I noticed but can't really interpret: FLDCW appears 420 times against 207 FISTP, and every FISTP I looked at sits in the same sequence: MOV, FLDCW, FISTP, FLDCW, MOV.

No inline transcendentals at all, no FSIN, FCOS or FYL2X. The trig comes through wrapped libm.

FASTROUND=0 and SYNC_ROUNDING=1 both still produce the error sadly. I've also rebuilt with HAVE_TRACE=ON and libzydis, so I can produce a trace if that would help.

Do you know if FISTP honours the rounding mode from a preceding FLDCW (in both the interpreter and the dynarec)?

[ptitSeb]

Yes FISTP honnor the rounding mode. Note that there are 3 version of FISTP: 16bits / 32bits & 64bits integer...

I assume the emulator is running on Windows. If that's the case, you might want also to use BOX64_DYNAREC_DUMP_RANGE to dump only the part from the main binary (and maybe it's dll) instead of everything (as wine also have SSE & x87 operations)

[shakeheartbreak]

So there's no Wine for this one, it's using Sega's x86 Linux arcade platform. So it's a 32-bit Linux ELF. Your point still helped though - the dump was mixing in the loader's own x86 libraries alongside the game itself.

Split by address space, the game's own binary does use both. SSE scalar for most of the float work, plus 59 FISTP of its own - the remaining 148 are in SDL3 and libstdc++.

The variant split across the whole dump is FISTP Ed 137, i64 63, Ew 7. All 7 of the 16-bit ones are in the game binary, none in the libraries.

Is the 16-bit FISTP path as well covered as the 32 and 64-bit ones? 🙏🏼

[shakeheartbreak]

Eurgh. I screwed up. Those 16-bit FISTP all turned out to be the ADX audio codec 😫

One thing that might be more useful though is that the interpreter computes FPATAN and FSINCOS on doubles via the host libm:

x64rund9.c:150 ST1.d = atan2(ST1.d, ST0.d);
x64rund9.c:259 sincos(ST1.d, &ST1.d, &ST0.d);

Checking the game's addresses against its symbols, its FPATAN and FSINCOS are all in matrix and angle code - utiSolveMatrixRotation, Vec2Rad, mxRotateX/Y/Z.

Is double close enough there, or could that matter?

[ptitSeb]

Yeah, double should be close enough. I don't think a precision difference in a trig function would be enough for such a dramatic effect. I would suspect more a C0..C3 x87 flag difference, or some NAN subtelties