Diff & Text Analysis

This page documents the auxiliary features of CmpStr for text analysis and diffing. While not part of the core string similarity pipeline, these utilities provide convenient, dependency-free solutions for common text processing and inspection tasks.

Both tools — TextAnalyzer and DiffChecker — are included in the CmpStr package, exported separately and can be used independently of the main comparison API. For convenience, they can also be instantiated via CmpStr.analyze() and CmpStr.diff().

Text Analysis

The TextAnalyzer class provides a comprehensive set of methods for extracting statistical, structural, and readability-related information from a text. It is intended for lightweight linguistic analysis, readability estimation, and preprocessing support.

Instantiation

import { TextAnalyzer } from 'cmpstr';

const analyzer = new TextAnalyzer(`'This is a sample text. It contains several sentences.' );

Basic Statistics

• getLength()
  Returns the total number of characters.

• getWordCount()
  Returns the total number of words.

• getSentenceCount()
  Returns the number of detected sentences.

• getAvgWordLength()
  Returns the average word length.

• getAvgSentenceLength()
  Returns the average sentence length in words.

Frequency Analysis

• getWordHistogram()
  Returns a word frequency histogram.

• getMostCommonWords( limit = 5 )
  Returns the most frequent words up to the given limit.

• getHapaxLegomena()
  Returns all words that occur exactly once.

• getCharFrequency()
  Returns a character frequency histogram.

• getUnicodeCodepoints()
  Returns the frequency of Unicode codepoints in the text.

Word and Syllable Analysis

• getLongWordRatio( len = 7 )
  Ratio of words with a length ≥ len.

• getShortWordRatio( len = 3 )
  Ratio of words with length ≤ len.

• getSyllablesCount()
  Estimates the total number of syllables.

• getMonosyllabicWordCount()
  Number of words with exactly one syllable.

• getMinSyllablesWordCount( min )
  Number of words with at least min syllables.

• getMaxSyllablesWordCount( max )
  Number of words with at most max syllables.

• getAvgSyllablesPerWord()
  Average syllables per word in the text.

• getMedianSyllablesPerWord()
  Median syllables per word in the text.

Readability and Richness

• getHonoresR()
  Calculates Honore’s R (lexical richness).

• getReadingTime( wpm = 200 )
  Estimates reading time in minutes based on words per minute.

• getReadabilityScore( metric = 'flesch' )
  Calculates readability scores:

  • flesch: Flesch Reading Ease
  • fleschde: Flesch Reading Ease (German)
  • kincaid: Flesch-Kincaid Grade Level

• getLIXScore()
  Calculates the LIX readability index.

• getWSTFScore()
  Returns an array of four Wiener Sachtextformel (WSTF) scores.

Other Utilities

• hasNumbers()
  Returns true if the text contains digits.

• getUpperCaseRatio()
  Ratio of uppercase letters to total letters.

Minimal Example

import { TextAnalyzer } from 'cmpstr';

const analyzer = new TextAnalyzer( 'The quick brown fox jumps over the lazy dog. 12345!' );

console.log( analyzer.getWordCount() );           // 9
console.log( analyzer.getAvgWordLength() );       // 3.888…
console.log( analyzer.getMostCommonWords( 3 ) );  // [ 'the', 'quick', 'brown' ]
console.log( analyzer.getReadabilityScore() );    // 98.8675 (Flesch)
console.log( analyzer.getLIXScore() );            // 4.5
console.log( analyzer.hasNumbers() );             // true

Try yourself at OneCompiler

Diff Tool

The DiffChecker class provides functionality for comparing two texts and extracting their differences. It supports both line-based and word-based diffing, optional context lines, grouping of adjacent changes, and two output formats: plain ASCII and CLI-colored output.

Instantiation

import { DiffChecker } from 'cmpstr';

const a = `The quick brown fox
jumps over the lazy dog.
This is the original text.`;

const b = `The quick brown fox
leaps over the lazy dog.
This is the changed text.`;

const diff = new DiffChecker( a, b, { mode: 'word', contextLines: 1 } );

Output Methods

• getStructuredDiff()
  Returns an array of structured line-level diff objects.

• getGroupedDiff()
  Returns grouped diff objects with adjacent changes merged.

• getASCIIDiff()
  Returns a unified diff as plain text using -[deleted] and +[inserted].

• getCLIDiff()
  Returns the same unified diff enriched with ANSI color codes for terminal output.

Example Output

Given the example above, calling:

console.log( diff.getASCIIDiff() );

Produce the following output:

       @@ -2,13 +2,12 @@ +--
   1   The quick brown fox
   2 - -[jumps] over the lazy dog.
     + +[leaps] over the lazy dog.
   3 - This is the -[original] text.
     + This is the +[changed] text.

Try yourself at OneCompiler

Using getCLIDiff() yields the same structure, with colored highlights for insertions and deletions in compatible terminals.