Open
Conversation
last maintenance of 2022
Remove Twitter
fix add-by-webpage
add three more providers to harvest
improve springer descriptions, autoactivate sitemap harvest
- FAQ: Replace three-campaign-types intro with Thanks-for-Ungluing focus, add short legacy note for Pledge/B2U. Remove Pledge Campaign FAQ, Premiums FAQ, B2U-specific ebook Q&As, and detailed Rights Holder pledge/B2U sections. Simplify Funding and Conversion sections. Fix stale campaign_list links (pledge/b2u → t4u). - learn_more.html: Remove Buy-to-Unglue and Pledge-to-Unglue program rows from landing page "how it works" panel (keep Thanks only). - programs.html: Remove Pledge-to-Unglue and Buy-to-Unglue program terms (keep Thanks-for-Ungluing terms only). - faq_b2u.html: Replace detailed B2U FAQ with short legacy note. Part of #1081 (strip REWARDS/B2U campaign types). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Revise FAQ and landing page for post-campaign era
- Delete sysadmin/ (superseded by regluit-provisioning/Ansible) - Delete test/selenium-server-standalone-2.24.1.jar (obsolete binary) - Gitignore: settings/prod.py, settings/local.py, settings/aws.py (Ansible-generated or local-only, should never be committed) - Gitignore: credential-adjacent files (iam_keys, *.pem, *.der, etc.) - Gitignore: IDE configs (.idea/, *.komodoproject) - Gitignore: test drivers/binaries, test-data/, venv/, deploy/prod.wsgi Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Strip REWARDS + BUY2UNGLUE: Phase 1 — routes, views, templates
limit=None causes 'int >= NoneType' TypeError in load_doab_oai(), crashing every unattended run. Guard the comparison so limit=None means no cap (original intent). Fixes #1096 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When DOAB returns 429 Too Many Requests, pyoai raises urllib.error.HTTPError which previously propagated unhandled, crashing the management command without printing the "loaded N records" summary. The Retry-After header (typically 86400s = 24h) was silently discarded. Now catches 429 specifically, logs the Retry-After value and how many records were harvested before being cut off, then returns normally. Any other HTTPError is still re-raised. Fixes #1100 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Changed from 15s to 60s read timeout per Eric's feedback that 15s is too aggressive for some publisher servers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three pre-existing bugs in the DOAB loader:
1. groups(1) -> group(1) in Springer FTP cover handling
re.Match.groups() returns a tuple; .format(tuple) produces a broken
URL like '.../('filename.jpg',)/...'. group(1) returns the string.
2. Non-FTP 302 redirect fetched url (original) instead of redirurl
(the redirect destination). Now correctly follows the redirect.
3. item_type filter was case-sensitive and checked only the first list
element via unlist(). Records where DOAB returns ['Book'] or
['peer-reviewed', 'book'] were silently skipped. Now checks the
full list case-insensitively.
Fixes #1102
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…_streamdata Two reliability improvements: 1. Per-record exception isolation in load_doab_oai() add_by_doab() was called with no try/except. A DB constraint violation or unexpected metadata error at any record aborted the entire remaining harvest window silently. Now catches Exception, logs the DOAB ID and error, and continues to the next record. 2. Explicit 429 handling in get_streamdata() Previously a 429 response caused response.json() to raise ValueError (DOAB returns HTML on rate-limit), logged only as "decoder error" — masking the true cause. Now checks status code before calling .json() and logs a clear rate-limit error with the Retry-After value. Note: surfacing partial 429 harvests in management command output requires coordination with PR #1101 and is tracked in #1105. Fixes #1105 (partial — management command visibility pending #1101) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ContentTyper: timeout=(5, 60) on all 4 requests calls - load_ebookfile: timeout=(10, 60) on 3 requests calls (file downloads) - get_soup: timeout=(10, 30) + Timeout exception handler Read timeout changed from 15s to 60s per Eric's feedback. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Merge as part of DOAB harvester fix rollout (Wave 1)
Add 60s timeout to DOAB cover image HTTP requests
Handle DOAB OAI HTTP 429 rate-limit gracefully
Fix Springer cover URL bug, redirect target bug, and item_type filter
Add timeouts to ContentTyper, load_ebookfile, and get_soup HTTP calls
DOAB harvest reliability: per-record exception isolation and 429 in get_streamdata
Remove django-tastypie and python-mimeparse from requirements, delete api/resources.py (Tastypie resource definitions), remove tastypie from INSTALLED_APPS, clean up tastypie imports and API key signal/retrieval code from core/signals.py, api/views.py, and frontend/views/__init__.py. Update api/urls.py to remove v1_api registration. Update api_help.html to remove REST API documentation. Remove Tastypie-specific tests while preserving all OPDS, ONIX, widget, AllowedRepo, and webhook functionality. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file is generated by Ansible during deployment and contains server-specific config. It was already in .gitignore but was still tracked, requiring a stash/pop cycle on every production merge. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove Booxtream watermarking code paths that were specific to BUY2UNGLUE campaigns, which no longer exist: - Remove B2U transaction handling branch in signals.py (provisioning books, watermark_acq call, gift handling for B2U purchases) - Remove Campaign.watermark_success() and Campaign.make_unglued_ebf() methods (only called for B2U campaign success) - Remove B2U-specific template blocks in manage_campaign.html (watermark checkbox) and edition_uploads.html (B2U upload UI) - Clean up unused imports (BUY2UNGLUE, Library, ungluify, date_today) Preserved all library lending Booxtream integration: - Acq.get_watermarked(), Acq.borrow() watermark_acq calls - BooXtream class, Boox model, watermark_acq task - Campaign.do_watermark field (used by library lending path) - All Booxtream settings and configuration Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Makes the Turnstile challenge invisible unless user interaction is actually needed, reducing visual clutter. Applied to all 4 templates that use the cf-turnstile widget: - base.html (search bar) - registration/welcome.html (welcome search) - supporter.html (supporter page search) - add_your_books.html (book submission search) Fixes #1092 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Eric approved during 2026-03-19 meeting
Eric approved during 2026-03-19 meeting
Tested on test.unglue.it, Eric approved direction, ry:approved. Deploying to production.
Tested on test.unglue.it, Eric approved direction, ry:approved. Deploying to production.
Eric approved.
Rebased on fix/all-facet-alias-1117 which handles the 'all' base token. This commit adds keyword isolation on top: - get_other_groups(): keyword active → no compounds; non-keyword active → keywords excluded from sidebar/OPDS - get_facet_object(): raises InvalidFacetCombination for keyword compounds → 404 in web, OPDS, OPDS-JSON, ONIX views - explore.html: removed per-subject links from sidebar - 10 new tests (frontend + API) for isolation + 404 behavior Refs #1110, #1095 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Eric approved. Merging keyword facet isolation (Phase 1 of #1110).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.