Add Japanese v1.1 as a fourth build target - #777
Open
quarrel07 wants to merge 41 commits into
Open
Conversation
MegaMech
requested changes
Aug 1, 2026
Collaborator
|
Wow... lgtm |
Collaborator
|
Does US still build if you make clean and build US after building EU? |
Contributor
Author
Yes. Tested it two ways just now: Worth saying why that is not automatic: |
Collaborator
|
Sorry, I meant JP not EU |
Contributor
Author
All good! Just ran that one in a single reused tree:
Worth noting
The one difference is |
Collaborator
|
Lets remove the macos IDO Recomp binaries for now and figure something out later for that. |
macOS iconv converts a backslash that follows Japanese text into the fullwidth reverse solidus (0xA1C0) rather than leaving it as 0x5C. In src/cpu_vehicles_camera_path.c that turns a "\n" escape inside a string literal into a literal two-byte character, adding 26 bytes, which grows .main and shifts the whole ROM. No version can byte-match on macOS as a result. Python's euc_jp codec produces the bytes the cartridge actually has, and is identical to GNU iconv's output for the other two EUC-JP sources. PYTHON is already a required build dependency.
The group JSONs under assets/ describe one version only, so a second version needs a whole duplicate file selected by an ifeq in every .mk. Nearly all of an asset is version independent though: of the Japanese assets measured against their US counterparts, 97% differ in rom_offset and nothing else. Let an asset instead carry an override block keyed by version name, holding only the keys that differ. new_extract_assets.py folds it in before anything reads the asset, so the rest of the pipeline is unchanged, and the version comes from one --version flag on ASSET_EXTRACT rather than a switch in each .mk. No asset carries an override yet, so this is a no-op. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Assets extract to one shared path regardless of version, and an asset that already exists on disk is never extracted again. Building a second version therefore reuses the first version's bytes, silently, and they end up in the ROM. `make clean` does not help: it clears build/, not the asset tree. Record the version alongside the asset list in .assets-local.txt and re-extract when it changes. The revision ID goes to 2 so an existing tree is refreshed once rather than trusted. This matters little for EU, where 31 assets carry their own offsets, and a great deal for a version that differs more widely. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Torch generated into assets/code/<name> for every version, so building one version left its generated C in the tree for the next version to compile. Route it through ASSET_CODE_DIR, which the linker script now receives as a define, and drop the assets/code/ prefix from the rainbow_road includes so they resolve through the include path instead of a fixed location. EU has no config.yml entry and is built on top of the US assets, so it keeps reading the US output. torch.hash.yml is not version aware and would skip regeneration after a switch, so the assets target clears it first; Torch takes under a second. Also mark extracted assets .PRECIOUS. They come out of the baserom, and make's habit of deleting a target whose recipe failed leaves a tree that only re-extraction can repair. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
macOS still ships GNU make 3.81 as `make`, and tools/Makefile generates its per-tool rules with `define COMPILE =`, which is 3.82 syntax. Under 3.81 those rules do not take effect and make falls back to its built-in %: %.c rule, so each tool is linked from its first source file alone. n64graphics is built from n64graphics.c and utils.c, so it fails with an undefined _g_verbosity rather than anything that points at the cause. The docs already tell macOS users to build with gmake, but this line then calls bare make and undoes that. This makefile needs 4.0+ for != in the first place, so $(MAKE) is always a new enough make.
Makefile option and expected checksum only; the asset map and the code differences follow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
An asset is described once and usually only moves between versions, but a few also change shape: seven menu textures are a different width in the Japanese release, and one audio sequence a different length. Neither can be said today, because meta is shared by every version. Let meta hold an "overrides" map listing just the keys that differ for a version. Nesting them keeps a version name from colliding with a real meta key, and an asset with no override behaves exactly as before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
877 assets carry a jp.v11 offset, and seven also carry a dims override where the Japanese menu text is a different width than the US artwork. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
12379 group assets carry a jp.v11 override. 12060 of them differ from the US entry in nothing but rom_offset, which is why these are overrides rather than a second copy of every group file. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Registers the Japanese 1.1 cartridge by checksum with its own segment table, and adds the yamls describing the segments it relocates. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 39 tkmk00 menu textures the Japanese build needs are sliced straight out of the cartridge, the same way the US blobs already are, so nothing has to be checked in for them. Offsets were derived by locating each blob in the cartridge; every one occurs exactly once, and all 39 agree with the offsets recorded when the match was originally worked out. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every difference is behind #ifdef VERSION_JP, so the other three versions compile exactly as before. Track path data moves out of course_data.c into .inc.c files so the two versions can share it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A tkmk00 block is described twice in assets.json at the same ROM offset: the .tkmk00 blob under bin/ is the build input, and the .png beside it under textures/ is a decoded copy. Extraction groups entries by offset and then decided whether to decode by looking at the first one in the group, so whichever was listed first won. The .tkmk00 entries come first in the file, so all 63 of the textures/ PNGs were written straight from the compressed bytes and came out as noise. They only ever looked right on an incremental run that already had the blob on disk. Decode alongside instead: the blob keeps the compressed bytes it needs and the png gets the decoded image. Nothing in the build consumes those PNGs, so no ROM changes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
JP ships a different build of this course. Its packed display-list stream is disassembled back into Gfx macros here, so the JP lists are ordinary committed source like every other course instead of a checked-in blob. The disassembly is verified two ways. Re-emitting US from the US blob reproduces the committed course_displaylists.inc.c character for character, and the JP output compiles and repacks to the 13726 bytes that sit at rom 0x90604C in the cart. A single #ifdef VERSION_JP after the includes selects the JP file, so us, eu.v10 and eu.v11 compile from exactly the bytes they did before. This replaces course_displaylists.jp.v11.s, which was never wired into the build (ASM_DIRS does not cover courses/). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Group assets extract to one shared path regardless of version, guarded by a `.export` sentinel that did not name the version. After building one version the sentinel looks current, so building a second skips extraction entirely and compiles the first version's artwork into the ROM. That is silent: wrong artwork compiles fine, and the damage only shows up as a ROM hash mismatch far downstream, cascaded by the size change. It matters now because 194 assets genuinely differ in bytes between us and jp.v11 rather than merely moving, which is a case EU never exercised: only 31 assets in the repo carry EU entries and EU art is US-identical. Name the sentinel .export.$(VERSION), the same way .assets-local.txt already records the extracted version for assets.json. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
An uncompressed asset uses the baserom itself as its source block, and asset_from_block closes the block when it is done. That closes the handle the caller still owns, so every later export_bin dies on a closed file. extract_asset already worked around it by reopening, which is why images survive and only raw `bin` assets fail. jp.v11 has enough of them to stop the build outright. Close the block only when it is a temporary one. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
course_offsets.c may guard its texture table with #ifdef VERSION_JP, and the texture order in that table decides the segment 5 layout. The generator already resolves the conditional, but nothing passed it the version, so it always took the #else and a JP build linked its display lists against the US texture addresses. The generated files also live at one shared path while their contents are version-specific, and course_offsets.c does not change when the built version does. Stamp the version so a switch regenerates them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Group assets extract to one shared path whatever the version is, guarded
by a .export sentinel that only depended on the group's json. Switching
version therefore kept the previous version's images and compiled them
into the ROM, silently: wrong artwork builds fine and only shows up as a
ROM hash mismatch, cascaded by the size change.
EU never exposed this. Its art is US-identical and merely relocated, so
US <-> EU has no asset whose bytes differ. jp.v11 has 194 that do.
Three things were needed, and each is useless alone:
- Depend the sentinel on a stamp holding the version currently on disk.
Naming the sentinel per version instead looks right but is not: it
records that a version was extracted once, which stays true after the
other version overwrites the shared files.
- Give the image rules a real recipe rather than `;`. An empty recipe
marks the target updated without re-stat'ing it, so the .bin and .mio0
built from a re-extracted png kept the old bytes until a second make.
- Extract EU's group assets from the US cart. None of the 12,521 group
assets carries an eu offset, so reading the EU cart at US offsets
returns garbage. EU relied on US files happening to be left on disk.
ASSET_VERSION mirrors TORCH_VERSION, which folds EU to US already.
Verified by building jp.v11 -> us -> eu.v10 -> eu.v11 -> jp.v11 with no
cleaning: all three byte-match, and JP re-extracts its own assets after.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
JP ships its own audio: every bank differs, the sample table is 0x180 larger, and 5 of the 30 sequences are different music. Map them instead of carrying the blobs. The banks and sample table get their own entries, the way eu.v10 already has bin/audiobanks.eu.bin, because their contents differ rather than merely moving. The sequences do not need that. 29 of the 30 already carried a jp.v11 offset, and extraction writes the built version's bytes to the shared music/*.m64 path, which is what the other 25 sequences have always relied on. So the five .ifdef VERSION_JP blocks pointing at .jp.v11.m64 paths were doing nothing the offset did not already do; drop them and let all thirty read alike. 00_seq_00 needed the offset plus a size override, being 0x50 bytes longer in JP. Same for the onomatopoeia TLUT, which turns out to be identical in both versions anyway: 512 values, byte for byte. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
JP shows three forms of each course name where US shows one: romanised ascii, japanese, and japanese with one name shortened to fit. menu_items.c already expected three files; nothing produced them. Torch's mk64:metadata factory writes exactly one names table per config entry, from the fixed `name` field, alongside nineteen other tables built from the same course metadata, and its parser requires every field of every course. Three names tables would therefore mean three copies of all twenty-one fully populated course yamls, differing in a single string and re-emitting nineteen identical tables each time. That is the per-version duplication this branch removes everywhere else. So the twenty names live in tools/jp_course_names.json and a small script writes the three tables in EUC-JP. They are hand-authored map data exactly like `name: banshee boardwalk` already is in the course yamls; they just do not need the twenty-table machinery. If the factory ever takes the names field from its config entry, the json folds back into yamls/courses and no source changes. Also defines the two symbols the JP link was missing. D_0B002900 is the segment-0B texture US has at 0x0B002A00 and belongs beside it. gCreditsTextExtra is different in kind and is called out in the file: JP's mirror-mode credits table is not decompiled, so it is an address rather than source, and it is the one place this target is not built from the repo. With these, jp.v11 links and produces a ROM. It does not match yet. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
JP packs both credit sets into one table: the english names occupy the first 63 entries of gCreditsText and the japanese ones the second 63, starting at エグゼクティブ プロデューサー. Mirror mode shows the japanese half, which is what gCreditsTextExtra points at - the same table entered at its midpoint rather than a second table. It was a bare address, which read as data the repo does not build. It is not: gCreditsText is ordinary source in src/ending/credits.c and this is an offset into it. Say so, and derive the symbol from it. The ROM is byte for byte identical either way. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Completes the jp.v11 side of assets.json. Every asset that needs a JP offset now has one; what is left without is US-only or EU-only, and has its own jp.v11 entry elsewhere. Two groups. The 36 common textures share one MIO0 block, which sits at 0x1323A0 in JP against 0x132B50 in US and decompresses to the same 184664 bytes, so the layout is unchanged and every asset in it moves by a constant 0xA40. Each was located by finding its bytes uniquely inside the decompressed block and the delta checked across all 36. The other 39 are the decoded copies of the tkmk00 blobs, sharing their offsets, so they take the offsets the blobs already use. Neither group is a build input: the ROM takes the common textures from Torch's assets/code/<version>/common_data and the tkmk00 art from the bin/ blobs. These entries are what make `extract_assets.py jp.v11` give a JP player their own artwork instead of the American, which is the point of the map. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The twenty course previews are the same images in both versions but sit 0x56D0 later in the JP cart. Without an override the JP build read the JP cart at the US offsets, got noise, and MIO0 grew it rather than shrinking it: the previews alone were 210,792 bytes of the ROM's overflow. Each offset was found by decompressing every MIO0 block in the region and matching it against the extracted image, so the shift is measured per asset rather than assumed; all twenty agree on 0x56D0. The ROM is now exactly 12,582,912 bytes, the size of the cart. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The eight blue shell frames sit 0xAEE0 later in the JP cart, the same shift the lakitu frames use. Each was matched by decompressing every MIO0 block in the region and comparing against the reference's extracted data, so the shift is measured rather than carried over. Their shared palette lives in the common textures block, which moves to 0x1323A0, but at the SAME offset within it. The block is not uniformly shifted: the assets earlier in it do not move and the later ones drop by 0xA40, so a block_offset has to be checked per asset rather than adjusted wholesale. That matters here because these are ci8: the .bin the ROM gets is the png round-tripped through its palette, so a wrong tlut corrupts the indices even when the image offsets are right. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A ci4/ci8 image reaches the ROM as png -> indices, remapped through its palette. That is lossless only while every index the ROM uses is the first one holding its colour. Several JP palettes repeat a colour, so the trip back picks the earlier index and quietly rewrites the image: correct offset, correct palette, wrong bytes. It cost 36 files across three courses and 25,664 bytes of the ROM. `type: bin` is the existing escape and is how the lakitu frames are done, but it reads the ROM directly, so it only works where the asset is stored uncompressed. These are MIO0 and it would copy compressed bytes. The jp.v11 entries already said `"compressed": true` in the hope that something honoured it; nothing did. Now export_bin decompresses the block and reads from there. That exposed the other half of the png rule bug: an asset extracted raw has no png, so the .bin rule handed n64graphics a missing file, and make deleted the .bin extraction had just written. The rule now depends on the extraction and converts only when a png is really there. Verified: those three courses go from 36 differing files to 0 against the byte-perfect JP reference, and us/eu.v10/eu.v11 still byte-match. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`make VERSION=jp.v11` cannot build after a us build: it dies converting gTextureCactus1Left.png. Those five are extracted raw for jp.v11 because their palette repeats a colour, and 370109612 guarded their rule on the png being present. A us build leaves its png behind, so the guard passes, n64graphics runs it against the JP palette, and it fails over the bytes extraction had just written. Extraction now removes a png it supersedes, so absence is the signal that an asset is raw. The eight lakitu rules then have to stop naming the png as a prerequisite, and take the shape the three course rules already have. They were not failing before only because extraction runs at parse time and leaves the .bin newer than the stale png: true today, but not something a rule should rest on. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
yamls/jp.v11/common_data.yml was the us yaml with a different segment address, so all 385 of its offsets still described the us layout. The two versions do not share one: us puts the CPU behaviour tables and the target speed tables ahead of the textures, JP puts the textures first and packs both at the end of the segment. Every texture therefore sits 0xA40 lower than its us offset, and those 21 entries sit 0x23800 higher. With the offsets corrected the built segment is byte-identical to the cartridge's, 184,664 bytes, and the 98 textures that were wrong come out of Torch already right. config.yml sets `sort: OFFSET`, so the entries do not need reordering and this file stays comparable with the us one line for line. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Segment 0x0B is the same 36,232 bytes in both versions and identical up to 0x6BB8, but JP arranges the tail differently: the four podium_ceremony_path tables move from the end of the segment to just ahead of light1, and everything between them shifts up by 0x2E0. yamls/jp.v11/ceremony_data.yml still described the us arrangement, so Torch read the display lists out of what is texture data in JP. A `gfx` entry is sized by scanning for its end opcode rather than from the yaml, so each one ran past its real end and the segment came out 1,680 bytes over. With the offsets corrected the built segment is byte-identical to the cartridge's. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
JP ships different geometry for this course: 5,949 vertices against the us table's 5,936, with 674 matching at the head and 1,254 at the tail. That is the same difference the packed display lists carry, and it is why they hold 434 differing gsSPVertex operands. Separate file under one guard, the shape 6b464168e used for the display lists. A file that currently produces byte-perfect us/eu.v10/eu.v11 cannot regress while the us table sits untouched on the #else branch. The table was read from the course geography block in the JP cart with a generator checked both ways against upstream's own us file: the us source compiles to the same bytes the us build produces, and those bytes re-emit that file character for character. This is the last piece: make VERSION=jp.v11 now reproduces the cart. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every version's ROM rule runs doxygen_symbol_gen.py, but the script hardcodes build/us/mk64.us.map. A fresh clone that builds anything other than us first therefore fails on the last line of the recipe, after the ROM it just produced is already correct. This is reachable on master today with `make VERSION=eu.v10`; it is not specific to the version this branch adds. The script now takes the map as an argument and still defaults to the us one, so anything calling it bare keeps working. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
EU has no entry in config.yml and builds on the us Torch output, which is what ASSET_CODE_DIR and ASSET_VERSION already arrange. The assets target still passed $(BASEROM), so `make assets VERSION=eu.v11` handed Torch the EU cart, matched no sha1, and silently wrote nothing; the link then failed on a missing assets/code/us object. Found by building the four versions in the other order from a fresh clone. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
134 assets carried a jp.v11 `type: bin` override on the theory that their palette repeats a colour, so the png round trip would remap indices and rewrite the image. Removing all of them and rebuilding says otherwise: 129 come out byte-identical through the normal image path. The override was standing in for offsets that are now correct, not for a palette problem. The five kalimari cacti are real, and for a different reason. gTLUTCactus has no jp.v11 entry and its palette does not appear anywhere in the JP cart, so there is no palette to convert through. Those keep the retype, and kalimari_desert.mk keeps the guard that goes with it. The other ten .mk files return to upstream's plain rule. All four ROMs still byte-identical, and the build log confirms the png path really ran rather than reusing a leftover .bin. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
567 places had a guard block closing and the identical guard reopening on the very next line, one display list entry at a time. Merging them removes 1,134 lines without touching a line of guarded code: the same entries survive under the same condition, in the same order. Only blocks with no `#else` are merged. With one, the merge would have to interleave the two branches, and declaration order is data layout here. The 33 pairs of that shape are left alone. All four ROMs still byte-identical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The note asked for the same arrangement in data, rodata and bss. Those blocks are already there, so the TODO reads as unfinished work when nothing is outstanding. Say what the block is for instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
README gains the fourth row. compiling.md gains a "Building JP" section next to "Building EU", and it has to say two things that section does not: Torch writes its generated C per version, so `make assets` needs the version too, and unlike EU, JP does not need us built first. It extracts everything from the Japanese cart, so a fresh clone builds it on its own. first-diff.py and diff_settings.py learn `--jp11` / `-jp11`, so the commands the new section points at exist. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
clang-format wraps this array differently now that the table is a line longer. Whitespace only: the non-whitespace bytes are identical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The wildcard was for the `.export.$(VERSION)` scheme this branch replaced with one shared sentinel guarded by assets/.version. Nothing writes a suffixed one now, so match the file that actually exists. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Revision 1.1 of the December 1996 Japanese release, not a 1997 version. The neighbouring lines are 1997 releases and this one inherited that by proximity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
git-clang-format against 230fe2e with the pinned clang-format 14, so only lines this branch added are considered. A stray blank line and one continuation alignment. The two other files it flags are left alone deliberately. src/audio/external.c would be re-indented for the JP branch, which nests a level deeper around a body both branches share, so the result is correct for JP and wrong for us. src/menu_items.c holds EUC-JP text and clang-format counts bytes rather than columns, so its wrapping there is not meaningful. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review feedback: the case labels do not need duplicating, only the two assignments differ between versions. Same preprocessed output either way. All four ROMs still byte-identical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both said the US revision replaced the JP code. It is the other way round: the December 1996 JP 1.0 cartridge has yoshi_valley_cpu_path and the update_path_index recovery chains, byte for byte the same as US, so JP 1.1 is the revision that replaced them. Comments only. jp.v11 still byte-identical. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Author
|
Sounds good, I'll pull them out of #777 and #772. queueRAM's ido-static-recomp submodule idea looks like the better fix to me anyway. I tried it here just now, make setup plus make VERSION=5.3 was about 8 seconds and mk64 builds a byte-matching US ROM off the result. Happy to do that as its own PR if it would help. |
Collaborator
|
Resolves #715 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds revision 1.1 of the original December 1996 Japanese release, next to
us,eu.v10andeu.v11. Drop in your ownbaserom.jp.v11.z64, runmake assets -j VERSION=jp.v11thenmake -j VERSION=jp.v11, and it builds byte for byte.No game data is added. 994 of the 1,067
assets.jsonentries gain ajp.v11offset, 12,408 group assets gain ajp.v11override, and there are 8 new Torch yamls underyamls/jp.v11/.git ls-filesstill shows exactly the three ROM-derived files this repo already had, which are the documentation images.What actually differs
Most of this is arrangement rather than content, but not all of it. The CPU code genuinely diverges.
Gameplay and CPU behavior
gPlayerPathIndex = random_int(4)when a CPU crosses the finish line, in place of theyoshi_valley_cpu_pathpath-tracking that every other version uses, which runs per path point. The JP 1.1 code sits under one guard; the existing function is untouched.caselabels are in the US and EU builds with empty bodies, so their CPUs fall through to generic behavior.src/audio/external.creadsif ((x > 0.5f) || (x < 0.5f))for US and JP 1.0, which is true for every value except exactly 0.5. JP 1.1 reads< -0.5f, as does EU. The US side is left as-is, since it is what the cartridge does.gsSPDisplayListcalls than the US and EU builds over the same sub-models: 1,742 fewer in total. Mario Raceway is the cleanest case at 1,996 against their 2,123, with every other command type matching in count. Same models, invoked less often. JP 1.0 has not been counted, so this may go back further than 1.1.Different data
gTLUTCactushas no JP counterpart anywhere in the cart. That palette is simply absent, which is why those five cactus textures are the one place this PR reads indices raw instead of converting a png.Same data, moved
This is the category that made the work hard, because a wrong offset and wrong content look identical until you decompress and compare.
light1, and the 736 bytes between them shift up by 0x2E0.Text, credits and leftovers
迷路に突入！(entering the maze),迷路から出た！,バイパス切り替え エラー!!!andOGAWA DEBUG DRAWare in every other version but not JP 1.1, going with the Yoshi Valley path code above. In their place JP 1.1 hasチョコマウンテン踏み外し！！！, belonging to a Choco Mountain recovery path new in that revision.gCreditsTextis 126 entries either way, English in the first 63 and Japanese in the second, sogCreditsTextExtrais not a second table: it isgCreditsText + 0xFC, the same table entered at its midpoint. 11 of the 126 entries differ. JP credits six voice actors where US and EU credit three and leave three empty strings in the slots the others came out of; JP saysnintendo e.a.d.where they sayphil sandhop; andrare ltd.where they have the lawyeredDonkey Kong 3-D Model Provided Courtesy of Rare U.K.mario circuit), Japanese, and Japanese withドンキージャングルパークshortened toドンキージャングルto fit. One name of twenty differs between the last two.サンプリンング ボイス, where JP 1.1 hasサンプリング. Both JP revisions spellcharles martinee, missing the final t, where US and EU havecharles martinet.gRefreshRate = 60where US and JP 1.0 switch onosTvTypefor NTSC, PAL and MPAL.What is in the diff
40 commits, 250 files, +48,913 / -24,295.
The asset map churn looks larger than it is. Nearly every one of those lines is a
jp.v11override added inline to an existing entry, in the same idiomassets.jsonalready used. No existing US offset changes value; 8 of the 1,026 entries gain a nestedmeta.overridesblock, and that is the whole of the US side's exposure.Five assets read
"type": "bin"for JP rather than going through the normal image path: the kalimari cacti, for the palette reason above.Source differences are
#ifdef VERSION_JP, matching the existing#ifdef VERSION_EUstyle: 1,678 guard sites across 52 files. 9 of them read#if defined(VERSION_EU) || defined(VERSION_JP), where EU and JP agree and US is the odd one out. That is also why this does not adopt oot's ordered version integers, which assume a linear lineage that mk64 does not have.luigi_raceway
The vertex table and the display lists both go in as separate files under a single
#ifdef VERSION_JP(course_vertices.jp.v11.inc.candcourse_displaylists.jp.v11.inc.c) rather than inline guards. Inline would need roughly 284 of them sitting inside array bodies, in files that today produce byte-perfectus,eu.v10andeu.v11. A separate file cannot regress those.Both were produced by disassemblers that are checked against your own US files first: emit the US table from the US cart and the result matches the committed file character for character. So the JP output is in your format rather than a new one, and both files are clang-format clean.
Why tools/generate_jp_course_names.py exists
JP shows three forms of each course name where US shows one: romanized, Japanese, and Japanese with one name shortened to fit. Torch's
mk64:metadatafactory writes one names table per config entry from the fixednamefield, and emits twenty files from the same course metadata with every field required.Three names tables that way means three copies of all twenty-one fully populated course yamls, differing by a single string each, re-emitting nineteen identical tables alongside. So the three tables come from a small json plus a generator instead. If the factory ever takes the names table's source field from its config entry, that json folds back into
yamls/coursesand nothing else changes. Happy to do it the other way if you would rather.Verification
Fresh clone, only the four cartridge dumps copied in, every version built and compared byte for byte against its own dump. Not just hashed. Run in both orders, because building JP first in a clean tree is what catches assets left behind by a previous version.
usfirstjp.v11firstus579c48e211ae952530ffc8738709f078d5dd215eeu.v10a729039453210b84f17019dda3f248d5888f7690eu.v11f6b5f519dd57ea59e9f013cc64816e9d273b2329jp.v119f439457585146a4e1da7e1dd9104f7f94381688git statusis empty after all four in both orders, so nothing the build needs is untracked.jp.v11builds byte-correct as the first thing in a fresh clone, withoutusbuilt first.Why the asset pipeline commits are here
Several commits make the asset pipeline version-aware rather than adding JP data. They are not separable from this PR: JP is the first version whose asset bytes differ rather than merely moving, so without them a
VERSIONswitch silently compiles whatever the previous build extracted. It is also why the verification above is run in both orders.One of them turns out to fix something on master.
assets: re-extract the group assets when the built version changesaddsASSET_VERSION := $(TORCH_VERSION), folding EU's asset extraction to US the way its Torch output already did. Take230fe2ec1, cherry-pick only the three macOS build fixes so it compiles on a Mac, then follow the documentedmake assets && make VERSION=eu.v10in a fresh clone: you getc600563dc5abed8ed616bcab3beb0ba4bd3dc3barather thana729039453.... None of the 12,521 group assets carries an EU offset, so the moment extraction actually runs for EU it reads the EU cart at US positions. A warm tree hides it, because a previous US build has already left the right files on disk.Notes for the review
Formatting was applied with
git-clang-formatagainst230fe2ec1using the pinned clang-format 14, so only lines this branch touched were considered. That came to a stray blank line and one continuation alignment. A baremake formatwould have rewritten 341 lines in files this PR barely touches, since the tree is not currently clean under its own pinned version.Two files are deliberately left unformatted.
src/audio/external.cwould be re-indented to suit the JP branch, which nests a level deeper around a body both branches share, so the result is right for JP and wrong for US.src/menu_items.cis EUC-JP and clang-format counts bytes rather than columns, so its wrapping there is not meaningful. Both generated luigi files are clang-format clean.CI is not wired up for
jp.v11, since that needs an encrypted JP ROM and a matching secret only you can add. Happy to do it if you want it.JP 1.0, the launch revision, is a separate and larger job: measured against two reference points it lands closer to the US delta than to the gap between the two EU revisions. Later PR.
Depends on
#772 for the macOS build fixes, and #776 for the doxygen step. Two commits here are #772's and one is #776's; all three drop out as those merge.