Skip to content

feat: add parser for JSON with JS comment #137

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lopuhin merged 7 commits into from
Sep 1, 2020
Merged

feat: add parser for JSON with JS comment #137

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
c7a8759
feat: add parser for JSON with JS comment
May 11, 2020
3c0665e
fix: remove the regex of JS comment
May 12, 2020
5bb1a87
fix: adjust package
May 15, 2020
8dc25e1
Update extruct/jsonld.py
Gallaecio May 19, 2020
09d7f47
Update extruct/jsonld.py
Gallaecio May 19, 2020
3970f79
fix: handle unexpected jsonld format
May 20, 2020
8e5a603
feat: add unittest
May 20, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions extruct/jsonld.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,14 @@
"""
JSON-LD extractor
"""

import jstyleson
import json
Comment thread
Gallaecio marked this conversation as resolved.
import re

import lxml.etree
Comment thread
Gallaecio marked this conversation as resolved.
Outdated

from extruct.utils import parse_html

HTML_OR_JS_COMMENTLINE = re.compile(r'^\s*(//.*|<!--.*-->)')


class JsonLdExtractor(object):
Expand All @@ -34,8 +33,7 @@ def _extract_items(self, node):
data = json.loads(script, strict=False)
except ValueError:
# sometimes JSON-decoding errors are due to leading HTML or JavaScript comments
data = json.loads(
HTML_OR_JS_COMMENTLINE.sub('', script), strict=False)
data = jstyleson.loads(script, strict=False)
if isinstance(data, list):
return data
elif isinstance(data, dict):
Expand Down
1 change: 1 addition & 0 deletions requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,4 @@ mf2py>=1.1.0
six>=1.11
w3lib
html-text
jstyleson