Skip to content

Fix non base 10 number parsing#529

Merged
TomWright merged 7 commits intomasterfrom
fix-yaml-int-parsing
Apr 23, 2026
Merged

Fix non base 10 number parsing#529
TomWright merged 7 commits intomasterfrom
fix-yaml-int-parsing

Conversation

@TomWright
Copy link
Copy Markdown
Owner

Fixes #525

@TomWright
Copy link
Copy Markdown
Owner Author

Holding this here for a short while. I need to add additional tests in other parsers/funcs

@TomWright TomWright changed the title Fix non base 10 number parsing issues in YAML Fix non base 10 number parsing Mar 10, 2026
Comment thread execution/func_to_int.go
@TomWright TomWright force-pushed the fix-yaml-int-parsing branch from 09642a1 to 01434f5 Compare March 30, 2026 18:44
@TomWright TomWright requested a review from ccoVeille March 30, 2026 18:52
Comment thread parsing/yaml/yaml_test.go
return "", model.NewFloatValue(f), nil
case unstable.Integer:
i64, err := strconv.ParseInt(string(n.Data), 10, 64)
i64, err := strconv.ParseInt(string(n.Data), 0, 64)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not an expert in TOML, but I took a quick look at the spec

  • Please add tests for these:

Valid

42, -42, 0x12, 0o7, 0b111, 12_000, -12_000

Valid but as string: "042"

Invalid

-0x12, -0o7, -0b111, 042

All these values are valid with base 0 with ParseInt but not in TOML

Here are also feedbacks provided by an LLM

# ==========================================================
# HEXADECIMAL, OCTAL, AND BINARY BASES
# ==========================================================

# --- Valid (Case Insensitivity for Prefixes and Values) ---
hex_lowercase = 0xff12       # Standard hex
hex_uppercase = 0XFF12       # Uppercase prefix and values are VALID

bin_lowercase = 0b1101
bin_uppercase = 0B1101       # Uppercase 'B' is VALID
oct_lowercase = 0o755
oct_uppercase = 0O755        # Uppercase 'O' is VALID

# --- Invalid ---
# Negative signs are NOT supported for non-decimal bases
invalid_hex = -0xFF12        
invalid_oct = -0o755         
invalid_bin = -0b1101        

# ==========================================================
# SCIENTIFIC NOTATION (FLOATS)
# ==========================================================

# --- Valid ---
standard_sci = 1e6           # 1,000,000
positive_exp = 5e+10         # 50,000,000,000
negative_exp = -1.5E-4       # -0.00015 (Uppercase 'E' is VALID)
large_float  = 6_022.140e23  # Avogadro's number

# --- Invalid ---
# Decimal points must always have digits on both sides
invalid_point_1 = .1e2       
invalid_point_2 = 1.e2       

# Underscores are NOT allowed in the exponent part
invalid_underscore = 1e1_0   

…sion

- Handle underscores in YAML decimal integers (e.g. 1_000)
- Handle signed prefixed numbers in YAML (+0x10, -0o10, etc.)
- Add TOML test coverage for hex, octal, binary, and underscore numbers
- Revert parse_literal.go to base 10 to prevent 010 being parsed as octal
@ccoVeille
Copy link
Copy Markdown

Good additions.

Is there anything else you want to change?

Should I wait before reviewing again?

@TomWright

@TomWright
Copy link
Copy Markdown
Owner Author

Good additions.

Is there anything else you want to change?

Should I wait before reviewing again?

@TomWright

I think that's all I've got for the time being. If you see any major issues please flag them, otherwise I'm happy to get the main issues resolved and follow-up with minor fixes for specific cases at a later date.

It should be ready for you, thank you!

@TomWright TomWright merged commit f5916c2 into master Apr 23, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Failure to parse hexadecimal and octal numbers in YAML

2 participants