Skip to content

feat(git): add inline diff syntax highlighting with tokenizer and CSS classes#296

Open
DeryFerd wants to merge 2 commits into
gnoviawan:devfrom
DeryFerd:feat/git-diff-syntax-highlight
Open

feat(git): add inline diff syntax highlighting with tokenizer and CSS classes#296
DeryFerd wants to merge 2 commits into
gnoviawan:devfrom
DeryFerd:feat/git-diff-syntax-highlight

Conversation

@DeryFerd

@DeryFerd DeryFerd commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Problem

Git diffs in termul show code additions and deletions with colour-coded background lines, but the code itself was plain white text — no syntax highlighting. When reviewing a diff with multiple changed functions, variables, or string literals, you had to mentally parse each line to tell what's a keyword, a string, or a comment.

This adds a lightweight tokenizer that runs on every content line in the diff (context, addition, deletion) and wraps recognised tokens in <span className="hl-..."> elements with colour classes.

What changed

New file — src/renderer/lib/diff-syntax-highlight.ts

A standalone tokenizer with no external dependencies. It walks a line character by character and classifies tokens into:

  • keyword — language keywords (TypeScript, Rust, Python, Go, etc.)
  • string — double-quoted, single-quoted, and backtick template literals
  • comment//, #, and /* */ styles
  • number — integers, floats, scientific notation
  • punctuation — brackets, operators, semicolons, colons
  • plain — everything else (identifiers, whitespace, etc.)

The keyword list covers roughly 80 keywords across the languages most commonly seen in a polyglot terminal environment.

New file — src/renderer/lib/diff-syntax-highlight.test.ts

15 tests covering keywords, strings (single, double, template), numbers (int, float), comments (//, #, /* */), escape sequences inside strings, Rust-specific keywords, the empty string, and a combined expression.

Modified — src/renderer/components/git/GitDiffView.tsx

Both InlineDiff and SplitCell now pass their content through renderHighlighted(), which calls highlightLine() and maps the returned token array to React nodes. Header and meta lines (file paths, chunk headers) are left unhighlighted since they aren't source code.

Modified — src/renderer/index.css

Five new @layer components rules added:

Class Colour Token
.hl-keyword #c792ea (purple) Keywords
.hl-string #c3e88d (green) Strings
.hl-comment #676e95 (grey italic) Comments
.hl-number #f78c6c (orange) Numbers
.hl-punctuation #89ddff (cyan) Punctuation

These match the project's existing dark-theme Material palette. No new dependencies, no theme system changes.

Why not use a library

No existing dependency (Prism, Shiki, highlight.js) was already in the project. Pulling one in for a single feature adds bundle weight, a new license, and maintenance surface. The tokenizer is ~130 lines of logic — enough to make diffs significantly more readable without the weight.

Validation

  • bun run typecheck passes
  • bun run test — 15 new tests pass covering all token types
  • The highlighting only fires for context, addition, and deletion lines; header/meta lines remain plain
  • The tokenizer is pure function — no state, no DOM access, no side effects
  • No visual regression: unchanged lines keep their existing background colour, highlighted tokens sit on top

Summary by CodeRabbit

Release Notes

  • New Features
    • Syntax highlighting has been added to diff views, improving code readability. The feature supports highlighting keywords, strings, comments, numeric literals, and punctuation across various programming languages.

@coderabbitai

coderabbitai Bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@DeryFerd, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 23 minutes and 28 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e5f37c23-543a-45c4-99db-7d4a1f1ff693

📥 Commits

Reviewing files that changed from the base of the PR and between a1a0771 and 8f2b344.

📒 Files selected for processing (1)
  • src/renderer/lib/diff-syntax-highlight.ts
📝 Walkthrough

Walkthrough

Introduces a new diff-syntax-highlight.ts library with a character-scanning tokenizeLine function and HighlightToken interface. Adds highlightLine and lineToHtml exports, Vitest test coverage, CSS token classes, and integrates highlightLine into GitDiffView for both inline and split diff rendering.

Changes

Diff Syntax Highlighting

Layer / File(s) Summary
HighlightToken interface and tokenizeLine core
src/renderer/lib/diff-syntax-highlight.ts
Defines HighlightToken with a type union, a keyword set, regex/constants, and a single-pass tokenizeLine loop covering whitespace, single-line/block comments, strings with escape handling, words/keywords, numbers, and punctuation. Exports highlightLine as a public wrapper and lineToHtml for HTML rendering.
highlightLine test suite
src/renderer/lib/diff-syntax-highlight.test.ts
Adds Vitest tests with per-type token constructors asserting correct tokenization of keywords, strings, numbers, comment styles (//, #, /* */), empty input, escape sequences, keyword-like literals, and Rust-specific tokens.
CSS token highlight classes
src/renderer/index.css
Adds .hl-keyword, .hl-string, .hl-comment, .hl-number, .hl-punctuation inside @layer components with specific foreground colors and italic style for comments.
GitDiffView integration
src/renderer/components/git/GitDiffView.tsx
Imports highlightLine, adds renderHighlighted to convert tokens into React nodes (plain passthrough, others as span.hl-{type}), and applies it to context/addition/deletion lines in both inline and split rendering modes.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related issues

Possibly related PRs

  • gnoviawan/termul#236: Modifies the same GitDiffView.tsx diff-line rendering path that this PR also changes to apply renderHighlighted output.

Poem

🐰 Hop, hop, through the diff I go,
Keywords gleam and strings aglow,
Comments slant in italic grace,
Punctuation finds its rightful place.
Each token painted, line by line—
Even Rust code looks divine! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately and concisely describes the main change: adding syntax highlighting to inline diffs with a tokenizer and CSS classes.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@gnoviawan

Copy link
Copy Markdown
Owner

@CodeRabbit review

@coderabbitai

coderabbitai Bot commented Jun 14, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
src/renderer/lib/diff-syntax-highlight.ts (1)

85-86: ⚡ Quick win

Consider removing unused PUNCTUATION_RE constant.

The PUNCTUATION_RE regex is defined but never referenced. Line 208 marks any remaining character as punctuation without validating against this pattern. If the regex is intended for future use, consider adding a comment; otherwise, remove it to reduce maintenance overhead.

♻️ Proposed fix
 const NUMBER_RE = /^-?\d+(\.\d+)?([eE][+-]?\d+)?$/
-const PUNCTUATION_RE = /^[{}()[\];:,.<>+\-*/%=!&|^~@#?]$/
 
 function tokenizeLine(line: string): HighlightToken[] {
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/renderer/lib/diff-syntax-highlight.ts` around lines 85 - 86, The
PUNCTUATION_RE regex constant defined at the beginning of the file is never used
anywhere in the code, which adds unnecessary maintenance overhead. Remove the
PUNCTUATION_RE constant declaration entirely unless it is explicitly intended
for future use, in which case add a clear comment explaining the intended
purpose. Since line 208 already handles punctuation classification without
reference to this pattern, the constant can be safely deleted.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/renderer/lib/diff-syntax-highlight.ts`:
- Around line 6-79: The KEYWORDS set in the diff-syntax-highlight.ts file
contains a duplicate entry for the keyword 'let', which appears twice in the set
definition. Remove one of the duplicate 'let' entries from the KEYWORDS set to
eliminate unnecessary duplication and improve code clarity for future
maintainers.

---

Nitpick comments:
In `@src/renderer/lib/diff-syntax-highlight.ts`:
- Around line 85-86: The PUNCTUATION_RE regex constant defined at the beginning
of the file is never used anywhere in the code, which adds unnecessary
maintenance overhead. Remove the PUNCTUATION_RE constant declaration entirely
unless it is explicitly intended for future use, in which case add a clear
comment explaining the intended purpose. Since line 208 already handles
punctuation classification without reference to this pattern, the constant can
be safely deleted.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 8512b17d-c071-459d-ba5d-d7fd4015f5bb

📥 Commits

Reviewing files that changed from the base of the PR and between 466d73a and a1a0771.

📒 Files selected for processing (4)
  • src/renderer/components/git/GitDiffView.tsx
  • src/renderer/index.css
  • src/renderer/lib/diff-syntax-highlight.test.ts
  • src/renderer/lib/diff-syntax-highlight.ts

Comment thread src/renderer/lib/diff-syntax-highlight.ts
@DeryFerd

Copy link
Copy Markdown
Contributor Author

Fixed - Code cleanup

Removed code quality issues identified by CodeRabbit:

  1. Removed duplicate keyword: The keyword appeared twice in the KEYWORDS set (lines 27 and 46). Removed the duplicate.

  2. Removed unused constant: regex was defined but never referenced in the code. Line 208 already handles punctuation classification without needing this pattern.

Changes:

  • KEYWORDS set now has 73 unique keywords (was 74 with duplicate)
  • Removed 1 line of unused code (PUNCTUATION_RE declaration)
  • All 15 existing tests still pass ✅

No functional changes - pure code cleanup.

@gnoviawan gnoviawan closed this Jun 18, 2026
@gnoviawan gnoviawan reopened this Jun 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants