-
Notifications
You must be signed in to change notification settings - Fork 2
WEBDEV-8507: Migrate field-parsers into elements #51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
jbuckner
wants to merge
9
commits into
main
Choose a base branch
from
WEBDEV-8507-migrate-field-parsers
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 3 commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
1025db9
WEBDEV-8507: Migrate field-parsers into elements
jbuckner ddb7680
Merge branch 'main' into WEBDEV-8507-migrate-field-parsers
jbuckner 58cdc68
WEBDEV-8507: Make EpochParser tests timezone-independent
jbuckner 0394738
WEBDEV-8507: Base field-parsers on upstream main, dropping epoch WIP
jbuckner 72056ff
WEBDEV-8507: Add interactive field-parsers demo story
jbuckner 59dac11
WEBDEV-8507: Drop '.' list separator and its test to match main
jbuckner c3c26d9
WEBDEV-8507: Render field-parsers demo via story-template
jbuckner 78e283e
WEBDEV-8507: Support a plain heading for non-component story-template…
jbuckner 2fd984d
WEBDEV-8507: Don't show non-component demo stories as <tag> in the nav
jbuckner File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| export type FieldParserRawValue = string | number | boolean; | ||
|
|
||
| export interface FieldParserInterface<T> { | ||
| /** | ||
| * Parse the raw value and return a value of type T or undefined if unparseable | ||
| * | ||
| * @param rawValue T | undefined | ||
| */ | ||
| parseValue(rawValue: FieldParserRawValue): T | undefined; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| import { describe, expect, test } from 'vitest'; | ||
|
|
||
| import { BooleanParser } from './boolean'; | ||
|
|
||
| describe('BooleanParser', () => { | ||
| test('can parse string number truthy', () => { | ||
| const parser = new BooleanParser(); | ||
| const response = parser.parseValue('1'); | ||
| expect(response).toBe(true); | ||
| }); | ||
|
|
||
| test('can parse string number falsy', () => { | ||
| const parser = new BooleanParser(); | ||
| const response = parser.parseValue('0'); | ||
| expect(response).toBe(false); | ||
| }); | ||
|
|
||
| test('can parse words truthy', () => { | ||
| const parser = new BooleanParser(); | ||
| const response = parser.parseValue('true'); | ||
| expect(response).toBe(true); | ||
| }); | ||
|
|
||
| test('can parse words falsy', () => { | ||
| const parser = new BooleanParser(); | ||
| const response = parser.parseValue('false'); | ||
| expect(response).toBe(false); | ||
| }); | ||
|
|
||
| test('can parse date truthy', () => { | ||
| const parser = new BooleanParser(); | ||
| const response = parser.parseValue(Date()); | ||
| expect(response).toBe(true); | ||
| }); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| import { | ||
| FieldParserInterface, | ||
| FieldParserRawValue, | ||
| } from '../field-parser-interface'; | ||
|
|
||
| export class BooleanParser implements FieldParserInterface<boolean> { | ||
| // use a shared static instance for performance instead of | ||
| // instantiating a new instance for every use | ||
| static shared = new BooleanParser(); | ||
|
|
||
| /** @inheritdoc */ | ||
| parseValue(rawValue: FieldParserRawValue): boolean { | ||
| if ( | ||
| typeof rawValue === 'string' && | ||
| (rawValue === 'false' || rawValue === '0') | ||
| ) { | ||
| return false; | ||
| } | ||
| return Boolean(rawValue); | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| import { describe, expect, test } from 'vitest'; | ||
|
|
||
| import { ByteParser } from './byte'; | ||
|
|
||
| describe('ByteParser', () => { | ||
| test('can parse int strings', () => { | ||
| const parser = new ByteParser(); | ||
| const response = parser.parseValue('3'); | ||
| expect(response).toBe(3); | ||
| }); | ||
|
|
||
| test('can parse float strings', () => { | ||
| const parser = new ByteParser(); | ||
| const response = parser.parseValue('3.14'); | ||
| expect(response).toBe(3.14); | ||
| }); | ||
|
|
||
| test('returns undefined if the number is not a number', () => { | ||
| const parser = new ByteParser(); | ||
| const response = parser.parseValue('qab'); | ||
| expect(response).toBeUndefined(); | ||
| }); | ||
|
|
||
| test('returns the number if a number is passed', () => { | ||
| const parser = new ByteParser(); | ||
| const response = parser.parseValue(5.67); | ||
| expect(response).toBe(5.67); | ||
| }); | ||
|
|
||
| test('returns undefined if a boolean is passed in', () => { | ||
| const parser = new ByteParser(); | ||
| const response = parser.parseValue(true); | ||
| expect(response).toBeUndefined(); | ||
| }); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| import { | ||
| FieldParserInterface, | ||
| FieldParserRawValue, | ||
| } from '../field-parser-interface'; | ||
| import { NumberParser } from './number'; | ||
|
|
||
| /** | ||
| * A Byte is a unit-specific `number`, in bytes. | ||
| */ | ||
| export type Byte = number; | ||
|
|
||
| /** | ||
| * The ByteParser is a unit-specific NumberParser | ||
| * that returns a value in bytes | ||
| * | ||
| * @export | ||
| * @class ByteParser | ||
| * @implements {FieldParserInterface<Byte>} | ||
| */ | ||
| export class ByteParser implements FieldParserInterface<Byte> { | ||
| // use a shared static instance for performance instead of | ||
| // instantiating a new instance for every use | ||
| static shared = new ByteParser(); | ||
|
|
||
| /** @inheritdoc */ | ||
| parseValue(rawValue: FieldParserRawValue): Byte | undefined { | ||
| const parser = NumberParser.shared; | ||
| return parser.parseValue(rawValue); | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,137 @@ | ||
| import { describe, expect, test } from 'vitest'; | ||
|
|
||
| import { DateParser } from './date'; | ||
|
|
||
| describe('DateParser', () => { | ||
| test('can parse date-only strings', () => { | ||
| const parser = new DateParser(); | ||
| const response = parser.parseValue('2020-06-20'); | ||
| const response2 = parser.parseValue('06/20/2020'); | ||
| const expected = new Date(); | ||
| expected.setHours(0); | ||
| expected.setMinutes(0); | ||
| expected.setSeconds(0); | ||
| expected.setMilliseconds(0); | ||
| expected.setMonth(5); | ||
| expected.setDate(20); | ||
| expected.setFullYear(2020); | ||
| expect(response?.getTime()).toBe(expected.getTime()); | ||
| expect(response2?.getTime()).toBe(expected.getTime()); | ||
| }); | ||
|
|
||
| test('can parse date-time strings', () => { | ||
| const parser = new DateParser(); | ||
| const response = parser.parseValue('2020-06-20 3:46:23'); | ||
| const response2 = parser.parseValue('2020-06-20 03:46:23'); | ||
| const expected = new Date(); | ||
| expected.setHours(3); | ||
| expected.setMinutes(46); | ||
| expected.setSeconds(23); | ||
| expected.setMilliseconds(0); | ||
| expected.setMonth(5); | ||
| expected.setDate(20); | ||
| expected.setFullYear(2020); | ||
| expect(response?.getTime()).toBe(expected.getTime()); | ||
| expect(response2?.getTime()).toBe(expected.getTime()); | ||
| }); | ||
|
|
||
| test('can parse date-time strings different string test', () => { | ||
| const parser = new DateParser(); | ||
| const response = parser.parseValue('2020-09-20 05:12:38'); | ||
| const expected = new Date(); | ||
| expected.setHours(5); | ||
| expected.setMinutes(12); | ||
| expected.setSeconds(38); | ||
| expected.setMilliseconds(0); | ||
| expected.setMonth(8); | ||
| expected.setDate(20); | ||
| expected.setFullYear(2020); | ||
| expect(response?.getTime()).toBe(expected.getTime()); | ||
| }); | ||
|
|
||
| test('can parse ISO8601 strings without time zones', () => { | ||
| const parser = new DateParser(); | ||
| const response = parser.parseValue('2020-06-20T13:37:15'); | ||
| const expected = new Date(); | ||
| expected.setHours(13); | ||
| expected.setMinutes(37); | ||
| expected.setSeconds(15); | ||
| expected.setMilliseconds(0); | ||
| expected.setMonth(5); | ||
| expected.setDate(20); | ||
| expected.setFullYear(2020); | ||
| expect(response?.getTime()).toBe(expected.getTime()); | ||
| }); | ||
|
|
||
| test('can parse ISO8601 strings with explicit time zones', () => { | ||
| const parser = new DateParser(); | ||
| const response = parser.parseValue('2020-06-20T13:37:15Z'); | ||
| const response2 = parser.parseValue('2020-06-20T13:37:15-00:00'); | ||
| const response3 = parser.parseValue('2020-06-20T13:37:15+00:00'); | ||
| const expected = new Date(); | ||
| expected.setUTCHours(13); | ||
| expected.setUTCMinutes(37); | ||
| expected.setUTCSeconds(15); | ||
| expected.setUTCMilliseconds(0); | ||
| expected.setUTCMonth(5); | ||
| expected.setUTCDate(20); | ||
| expected.setUTCFullYear(2020); | ||
| expect(response?.getTime()).toBe(expected.getTime()); | ||
| expect(response2?.getTime()).toBe(expected.getTime()); | ||
| expect(response3?.getTime()).toBe(expected.getTime()); | ||
| }); | ||
|
|
||
| test('can parse "c.a. yyyy" formatted dates', () => { | ||
| const parser = new DateParser(); | ||
| const response = parser.parseValue('c.a. 2020'); | ||
| const expected = new Date(); | ||
| expected.setHours(0); | ||
| expected.setMinutes(0); | ||
| expected.setSeconds(0); | ||
| expected.setMilliseconds(0); | ||
| expected.setMonth(0); | ||
| expected.setDate(1); | ||
| expected.setFullYear(2020); | ||
| expect(response?.getTime()).toBe(expected.getTime()); | ||
| }); | ||
|
|
||
| test('can parse "ca yyyy" formatted dates', () => { | ||
| const parser = new DateParser(); | ||
| const response = parser.parseValue('ca 2020'); | ||
| const expected = new Date(); | ||
| expected.setHours(0); | ||
| expected.setMinutes(0); | ||
| expected.setSeconds(0); | ||
| expected.setMilliseconds(0); | ||
| expected.setMonth(0); | ||
| expected.setDate(1); | ||
| expected.setFullYear(2020); | ||
| expect(response?.getTime()).toBe(expected.getTime()); | ||
| }); | ||
|
|
||
| test('can parse "[yyyy]" formatted dates', () => { | ||
| const parser = new DateParser(); | ||
| const response = parser.parseValue('[2020]'); | ||
| const expected = new Date(); | ||
| expected.setHours(0); | ||
| expected.setMinutes(0); | ||
| expected.setSeconds(0); | ||
| expected.setMilliseconds(0); | ||
| expected.setMonth(0); | ||
| expected.setDate(1); | ||
| expected.setFullYear(2020); | ||
| expect(response?.getTime()).toBe(expected.getTime()); | ||
| }); | ||
|
|
||
| test('returns undefined if it is a bad date', () => { | ||
| const parser = new DateParser(); | ||
| const response = parser.parseValue('absjkdvfnskj'); | ||
| expect(response).toBeUndefined(); | ||
| }); | ||
|
|
||
| test('returns undefined if passed a non-string value', () => { | ||
| const parser = new DateParser(); | ||
| const response = parser.parseValue(123); | ||
| expect(response).toBeUndefined(); | ||
| }); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| import { | ||
| FieldParserInterface, | ||
| FieldParserRawValue, | ||
| } from '../field-parser-interface'; | ||
|
|
||
| export class DateParser implements FieldParserInterface<Date> { | ||
| // use a shared static instance for performance instead of | ||
| // instantiating a new instance for every use | ||
| static shared = new DateParser(); | ||
|
|
||
| /** @inheritdoc */ | ||
| parseValue(rawValue: FieldParserRawValue): Date | undefined { | ||
| // try different date parsing | ||
| return this.parseJSDate(rawValue) || this.parseBracketDate(rawValue); | ||
| } | ||
|
|
||
| // handles "[yyyy]" format | ||
| private parseBracketDate(rawValue: FieldParserRawValue): Date | undefined { | ||
| if (typeof rawValue !== 'string') return undefined; | ||
| const yearMatch = rawValue.match(/\[([0-9]{4})\]/); | ||
| if (!yearMatch || yearMatch.length < 2) { | ||
| return undefined; | ||
| } | ||
| return this.parseJSDate(yearMatch[1]); | ||
| } | ||
|
|
||
| private parseJSDate(rawValue: FieldParserRawValue): Date | undefined { | ||
| if (typeof rawValue !== 'string') return undefined; | ||
| let parsedValue = rawValue; | ||
|
|
||
| // fix for Safari not supporting `yyyy-mm-dd HH:MM:SS` format, insert a `T` into the space | ||
| if ( | ||
| parsedValue.match( | ||
| /^[0-9]{4}-[0-9]{2}-[0-9]{2}\s{1}[0-9]{2}:[0-9]{2}:[0-9]{2}$/, | ||
| ) | ||
| ) { | ||
| parsedValue = parsedValue.replace(' ', 'T'); | ||
| } | ||
|
|
||
| const parsed = Date.parse(parsedValue); | ||
| if (Number.isNaN(parsed)) { | ||
| return undefined; | ||
| } | ||
| let date = new Date(parsedValue); | ||
| // The `Date(string)` constructor parses some strings as UTC and some in the local timezone. | ||
| // This attempts to detect cases that get parsed as UTC but should be parsed as local. | ||
| // Note that this does _not_ include cases with an explicit time zone specified, which | ||
| // should generally be parsed as-is and not converted to local time. | ||
| const isUTCTimeZoneInferred = | ||
| parsedValue.match(/^[0-9]{4}$/) || // just the year, ie `2020` | ||
| parsedValue.match(/^[0-9]{4}-[0-9]{2}-[0-9]{2}$/); // YYYY-MM-DD format | ||
| if (isUTCTimeZoneInferred) { | ||
| date = new Date(date.getTime() + date.getTimezoneOffset() * 1000 * 60); | ||
| } | ||
| return date; | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| import { describe, expect, test } from 'vitest'; | ||
|
|
||
| import { DurationParser } from './duration'; | ||
|
|
||
| describe('DurationParser', () => { | ||
| test('can parse mm:ss format', () => { | ||
| const parser = new DurationParser(); | ||
| const response = parser.parseValue('45:23'); | ||
| const expected = 23 + 45 * 60; | ||
| expect(response).toBe(expected); | ||
| }); | ||
|
|
||
| test('can parse hh:mm:ss format', () => { | ||
| const parser = new DurationParser(); | ||
| const response = parser.parseValue('3:45:23'); | ||
| const expected = 23 + 45 * 60 + 3 * 60 * 60; | ||
| expect(response).toBe(expected); | ||
| }); | ||
|
|
||
| test('returns undefined for a non-number component in hh:mm:ss format', () => { | ||
| const parser = new DurationParser(); | ||
| const response = parser.parseValue('3:AB:23'); | ||
| expect(response).toBeUndefined(); | ||
| }); | ||
|
|
||
| test('can parse decimal format', () => { | ||
| const parser = new DurationParser(); | ||
| const response = parser.parseValue('345.23'); | ||
| expect(response).toBe(345.23); | ||
| }); | ||
|
|
||
| test('returns undefined for non-numeric numbers', () => { | ||
| const parser = new DurationParser(); | ||
| const response = parser.parseValue('abc.de'); | ||
| expect(response).toBeUndefined(); | ||
| }); | ||
|
|
||
| test('returns the number if passed a number', () => { | ||
| const parser = new DurationParser(); | ||
| const response = parser.parseValue(345.23); | ||
| expect(response).toBe(345.23); | ||
| }); | ||
|
|
||
| test('returns undefined if passed a boolean', () => { | ||
| const parser = new DurationParser(); | ||
| const response = parser.parseValue(true); | ||
| expect(response).toBeUndefined(); | ||
| }); | ||
| }); |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
QUESTION: Are there any month/year formats we need to support? (BTW: @ximm is a good reference for dates we support throughout the system.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Potentially? This PR is just for migrating from a standalone repo into elements, not modifying any functionality. I filed https://webarchive.jira.com/browse/WEBDEV-8535 to investigate that.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can talk about dates, but would need context.
(In brief, forgiving raw string formats with enormous variability and many styles are explicitly allowed in the
datefield in item metadata; these are normalized in search documents into ISO8601 timestamps, andyearderived when possible. Related, I just merged a chance today that will extend the representation of dates in all search documents, to exprsss a range, with a best-effort precision and uncertainty assignment.)