Add Supertonic defaults and smarter Threads settings#2550
Open
hongphuc5497 wants to merge 25 commits into
Open
Add Supertonic defaults and smarter Threads settings#2550hongphuc5497 wants to merge 25 commits into
hongphuc5497 wants to merge 25 commits into
Conversation
…ture
## Summary
Implements multi-platform support for VideoMakerBot, starting with Meta Threads as a new content source alongside Reddit. Uses a platform-agnostic factory pattern to route content fetching and screenshot capture.
## Changes
### New Files
- platforms/__init__.py: Factory dispatch for platform selection
- platforms/threads/__init__.py: Threads package marker
- platforms/threads/fetcher.py: Threads Graph API integration
- platforms/threads/screenshot.py: Playwright-based Threads screenshotter
- CLAUDE.md: Comprehensive development guide
- AGENT.md: Guidelines for AI agents working on the codebase
### Modified Files
- main.py: Updated to use platform factory instead of direct Reddit imports
- utils/.config.template.toml: Added [settings].platform, [settings].post_lang, [threads.*] sections
- utils/videos.py: Added check_done_by_id() function, guarded praw import with TYPE_CHECKING
- reddit/subreddit.py: Added thread_category field to content dict
- TTS/engine_wrapper.py: Fixed post_lang to use fallback chain
- video_creation/final_video.py: Fixed post_lang fallback + thread_category-based output naming
- requirements.txt: Fixed yt-dlp version to 2025.10.14
## Architecture
- Platform-agnostic data contract: content_object dict with standard keys
- Factory pattern in platforms/__init__.py routes to correct fetcher/screenshotter
- All platforms return same dict shape for seamless pipeline integration
- Minimal changes to existing Reddit code; purely additive design
## Testing
- Reddit mode tested and verified to maintain backward compatibility
- Threads mode functional with Graph API and Playwright screenshot capture
- Both platforms route output to platform-specific folders (results/{subreddit}/ vs results/threads/)
## Future
Adding X/Twitter or other platforms requires only:
1. New platform module (fetcher + screenshot)
2. Config section in .config.template.toml
3. Two elif branches in platforms/__init__.py
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Build one shared container image for the Flask GUI and CLI pipeline, with Playwright, FFmpeg, and spaCy preinstalled so first runs are reliable. Add bootstrap logic for missing runtime files, bind the GUI to 0.0.0.0 in containers, and preserve state through a repo mount. Constraint: Local development needs a single image that supports both entrypoints without introducing extra services or dependencies. Rejected: Separate GUI and CLI images | duplicated maintenance and no runtime benefit for this repo. Confidence: high Scope-risk: moderate Directive: Keep runtime state creation in the container bootstrap layer; do not reintroduce host-specific assumptions into GUI startup. Tested: docker compose build; docker compose run --rm gui python -c '...'; docker compose run --rm cli python -c 'import main'; docker compose up -d gui; curl -I http://localhost:4000 Not-tested: Full end-to-end video generation with live credentials in this environment.
Document the Docker Compose workflow, persistent runtime paths, and container-specific GUI binding so future work on this branch follows the implemented setup rather than the old direct-Python assumptions. Constraint: The repo now supports both host and container execution paths, and the agent guidance needs to reflect the new operational defaults. Rejected: Leave AGENTS.md untouched | it would continue pointing contributors at stale runtime behavior. Confidence: high Scope-risk: narrow Directive: Treat the Docker Compose commands as the default local workflow for GUI/CLI work on this branch. Tested: Reviewed AGENTS.md against the implemented Docker files and runtime bootstrap. Not-tested: No code-path changes; documentation-only update.
Keep the third-party bot in a dedicated submodule so its history and update cadence stay isolated from the main repo while still making it available in-tree for local workflows. Constraint: The upstream code should remain separately updatable without copying its files into this repository. Rejected: Copy the project into the tree directly | duplicated history and harder future syncing. Confidence: high Scope-risk: narrow Directive: Update the submodule by moving it deliberately; do not edit its contents from the parent repo without planning that workflow. Tested: git submodule add; git submodule status; inspected .gitmodules Not-tested: Upstream repo build/runtime inside this checkout.
…be upload wiring Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…av migration - Web scraper (platforms/threads/scraper.py) with div-based card parsing - Multi-source discovery: For You feed + configurable search queries - Engagement filtering (min_engagement) and post age filter (max_post_age) - Shared Playwright auth module (platforms/threads/auth.py) - Migrated ffmpeg-python to av (PyAV) for in-process media probing - Video composition uses subprocess ffmpeg (av filter graph segfault workaround) - Updated CLAUDE.md with Threads scraping and macOS-specific notes Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add /create page with pipeline stage polling, /video/<id> route for safe file serving, modernized Tailwind/DaisyUI UI, and pytest regression tests. Consolidate AGENT.md + AGENTS.md into CLAUDE.md. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add keywords input to Create page to override search_queries per run. Split /create into two panels: left controls + progress, right real-time scraper activity feed with stage diagram and typed event cards. Emit structured events from scraper and auth modules. Add blocked_words fields to Settings page Content tab. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…mance - Reply screenshots now target by comment_id instead of .first (was capturing main post) - TTS engine returns actual count (idx+1) instead of last index - Background chop uses ffmpeg stream-copy instead of moviepy re-encode - Merged prepare_background crop+scale into overlay filter graph (single encode pass) - Added -preset veryfast -crf 23 to overlay renders - Platform-conditional title image (no Reddit template on Threads) Co-Authored-By: RuFlo <ruv@ruv.net>
Replace eval() with safe type-coercion dicts in console/settings/gui_utils.
Replace os.system() with subprocess.run() in TTS engine_wrapper.
Remove shell=True from all subprocess/Popen calls in main + ffmpeg_install.
Redact credentials from error logs and settings page HTML.
Fix 6 bare except clauses across the codebase.
Bug fixes:
- Config overwrite crash: set config={} after writing empty file
- Playwright TimeoutError: import correct exception class
- Lambda closure: default arg captures loop variable value
- Redundant ffmpeg: single concat run after all segments generated
- Audio IndexError: explicit check before accessing clips_durations[0]
- NSFW selector: use generic role-based button instead of hardcoded post ID
- Dead macOS branch: sys.platform == "darwin" instead of os.name == "mac"
Hardening:
- Flask secret_key from env var, rotate per startup
- Docker non-root user (appuser)
- CSRF check via Origin header on mutating requests
- Security headers: X-Content-Type-Options, X-Frame-Options
- Citation path traversal sanitization
- Temp file cleanup in ProgressFfmpeg.__exit__
Co-Authored-By: RuFlo <ruv@ruv.net>
Co-Authored-By: RuFlo <ruv@ruv.net>
- M1: version check resilient to network errors, configurable upstream repo - M2: extract get_output_path() in final_video.py, reuse from main.py - M3: replace backtracking URL regex with atomic https?://\S+ pattern - M4: replace fixed 6s delay with page.wait_for_url() post-login - L1: TOTP secret security warning in config template - L2-L3: Dockerfile comments for digest pinning + pip version pinning Co-Authored-By: RuFlo <ruv@ruv.net>
… errors - Replace missed os.system() in posttextparser with subprocess.run - Add truncate() after json.dump in videos.py save_data (trailing bytes bug) - Replace exit() with sys.exit() in subreddit.py - Add retry depth limit (50) to prevent infinite recursion in get_subreddit_threads - Log search-scraping errors instead of silent except: pass in scraper.py Co-Authored-By: RuFlo <ruv@ruv.net>
… deps CLAUDE.md: - Add security section (12 rules: no eval, no os.system, no shell=True, etc.) - Update key files table with new functions and patterns - Document login flow change (event-wait instead of fixed delay) - Add 7 new debugging tips for recently fixed issues - Add Recent Changes section with all 18 fixes from May 2026 Dependencies bumped: - boto3 1.42.94 → 1.43.3, botocore 1.42.94 → 1.43.3 - playwright 1.58.0 → 1.59.0 - requests 2.32.5 → 2.33.1 - pyttsx3 2.98 → 2.99 - clean-text 0.6.0 → 0.7.1 - yt-dlp 2025.10.14 → 2026.3.17 Python version: main.py error message simplified, CLAUDE.md reflects host 3.14 + Docker 3.10 (stable). Co-Authored-By: RuFlo <ruv@ruv.net>
Bump base image from 3.10 to 3.14 to match host Python version. Add pip cache purge after install to reduce image size. Note: spaCy remains commented out (no 3.14 wheel yet). All other dependencies verified working on Python 3.14.4. Co-Authored-By: RuFlo <ruv@ruv.net>
Co-Authored-By: RuFlo <ruv@ruv.net>
- Enable spaCy 3.8.13 (3.8.14 was not on PyPI) - Remove `pip cache purge` — conflicts with PIP_NO_CACHE_DIR=1 - Model en_core_web_sm auto-downloads at runtime Co-Authored-By: RuFlo <ruv@ruv.net>
Preserve masked secrets on settings save, tolerate malformed background add requests, escape background catalog values, and skip terminal clearing when TERM is unset. Tested: rtk docker compose run --rm test
Refresh the GUI templates around the desktop-first neo-brutalist direction so the library, create, background, and settings screens share one visual system. Tested: rtk docker compose run --rm test Tested: Playwright desktop screenshots for /, select mode, /backgrounds, and /settings
Use the explicit base64url-like character class for YouTube IDs so background validation no longer accepts punctuation matched by A-z ranges. Tested: rtk docker compose run --rm test
Add Supertonic as the default TTS provider, introduce browser backend plumbing, make the Settings UI template-driven, and tighten Threads discovery/auth behavior with regression coverage.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Code review
Testing
Notes