Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,10 @@
/tmp/pids/*
!/tmp/pids/.keep

# Ignore the Postgres bind mount created by the compose db service. It is
# root-owned, so a rebuild after first boot cannot read the build context.
/data

# Ignore storage (uploaded files in development and any SQLite databases).
/storage/*
!/storage/.keep
Expand Down
13 changes: 13 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,19 @@ POSTGRES_DB=streamvault
# Host port to expose the app on (container always listens on 80).
PORT=3000

# ── Background jobs ─────────────────────────────────────────────
# Jobs run in the `worker` compose service, so the web container leaves
# SOLID_QUEUE_IN_PUMA unset. Set it to true only for a single-container
# deployment where Puma should run the supervisor itself.
SOLID_QUEUE_IN_PUMA=
# Worker processes; each one runs three threads (see config/queue.yml).
JOB_PROCESSES=1

# The web process stops expired HLS sessions and removes their segments,
# because it owns the PIDs and tmp/hls paths they reference. Set to false
# only if you clean tmp/hls yourself; no other service can do it.
HLS_CLEANUP_IN_WEB=true

# ── External APIs ────────────────────────────────────────────────
# Torrentio API base URL. Default is the public instance, but it may
# be blocked by Cloudflare. Consider running a self-hosted proxy
Expand Down
8 changes: 7 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,11 @@ The database lives in `./data` and survives rebuilds. Containers restart on cras
| `STREAM_PROVIDER` | `torrentio`, `comet`, or `auto` | `torrentio` |
| `TMDB_READ_ACCESS_TOKEN` | Recommendations and the fallback certification source for child accounts | empty |
| `SUBDL_API_KEY` | External subtitle fallback | empty |
| `FORCE_SSL` | Redirect to HTTPS and mark cookies Secure; `false` for plain HTTP on a trusted network | `true` |
| `ASSUME_SSL` | Trust `X-Forwarded-Proto` from a TLS-terminating proxy; `false` when Rails is reached directly | `true` |
| `SOLID_QUEUE_IN_PUMA` | Run jobs inside the web process instead of the `worker` service | unset |
| `JOB_PROCESSES` | Worker processes started by the `worker` service; each runs three threads | `1` |
| `HLS_CLEANUP_IN_WEB` | Web process cleans up expired HLS sessions; `false` disables it | `true` |

The full list, provider modes, Cloudflare proxy setups, and reverse proxy notes are in [docs/CONFIGURATION.md](docs/CONFIGURATION.md).

Expand All @@ -220,12 +225,13 @@ The full list, provider modes, Cloudflare proxy setups, and reverse proxy notes
| Playback buffers during transcoding | CPU saturation in the web logs. Try a 1080p H.264 source before 4K, HEVC, or AV1 |
| A resumed stream is CPU-heavy | Expected: a non-zero seek may re-encode to keep audio and video aligned |
| Sign-in does not persist behind a proxy | HTTPS on the public URL, and the proxy must forward the original scheme and host |
| Jobs never run, or recommendations stay stale | The `worker` service is up (`docker compose ps`, `docker compose logs -f worker`); the web container does not execute jobs unless `SOLID_QUEUE_IN_PUMA` is set |

Logs and health:

```bash
docker compose ps
docker compose logs --tail=200 -f web
docker compose logs --tail=200 -f web worker
curl -I http://localhost:${PORT:-3000}/up
```

Expand Down
50 changes: 50 additions & 0 deletions config/initializers/hls_session_cleanup.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# frozen_string_literal: true

# HLS sessions are created, transcoded and served by the web process: their
# rows carry PIDs and tmp/hls paths that exist nowhere else. Solid Queue's
# scheduler lives in the worker when a separate worker container is deployed,
# and the single-container Procfile deployment has no worker at all, so the
# cleanup runs in the web process in both layouts.
module HlsCleanup
INTERVAL = 5.minutes
LOCK_PATH = "tmp/hls-cleanup.lock"

class << self
# Called from config/puma.rb's on_booted hook, so only the web server
# process starts a loop: bin/jobs, the console and rake tasks never load
# that config, and a phased restart cannot start a second one.
def start
return if @thread
return unless Rails.env.production?
return if ENV["HLS_CLEANUP_IN_WEB"].to_s.strip.downcase == "false"

lock = File.open(Rails.root.join(LOCK_PATH), File::RDWR | File::CREAT, 0o644)
@lock = lock
Rails.logger.info("[HlsSessionCleanup] starting web-side HLS cleanup loop")

@thread = Thread.new do
loop do
run_once(lock)
sleep INTERVAL
end
end
end

private

# The lock is taken per run and released afterwards, so a replacement
# process takes over on the next cycle and clustered Puma workers never
# run the cleanup at the same time.
def run_once(lock)
return unless lock.flock(File::LOCK_EX | File::LOCK_NB)

begin
Rails.application.executor.wrap { HlsSessionManager.cleanup_expired }
ensure
lock.flock(File::LOCK_UN)
end
rescue StandardError => e
Rails.logger.warn("[HlsSessionCleanup] #{e.class}: #{e.message}")
end
end
end
16 changes: 15 additions & 1 deletion config/puma.rb
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,22 @@
plugin :tmp_restart

# Run the Solid Queue supervisor inside of Puma for single-server deployments.
plugin :solid_queue if ENV["SOLID_QUEUE_IN_PUMA"]
# Opt in by setting the variable; every string is truthy, so the value is also
# compared against falsey spellings to let a two-container deployment turn the
# supervisor off while a separate worker owns job execution.
# ActiveSupport is not loaded when Puma reads this file, so test the string
# directly instead of relying on #present?.
solid_queue_in_puma = ENV["SOLID_QUEUE_IN_PUMA"].to_s.strip.downcase
unless solid_queue_in_puma.empty? || %w[0 false no off].include?(solid_queue_in_puma)
plugin :solid_queue
end

# Specify the PID file. Defaults to tmp/pids/server.pid in development.
# In other environments, only set the PID file if requested.
pidfile ENV["PIDFILE"] if ENV["PIDFILE"]

# HLS sessions belong to this process: their rows carry PIDs and tmp/hls
# paths that exist only here, so the cleanup loop starts with the web
# server and never in a worker, console or rake task.
# Puma 8 renamed on_booted to after_booted.
after_booted { HlsCleanup.start }
2 changes: 1 addition & 1 deletion config/queue.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ default: &default
workers:
- queues: "*"
threads: 3
processes: <%= ENV.fetch("JOB_CONCURRENCY", 1) %>
processes: <%= ENV.fetch("JOB_PROCESSES", 1) %>
polling_interval: 0.1

development:
Expand Down
10 changes: 4 additions & 6 deletions config/recurring.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,7 @@ production:
clear_solid_queue_finished_jobs:
command: "SolidQueue::Job.clear_finished_in_batches(sleep_between_batches: 0.3)"
schedule: every hour at minute 12
# Stop inactive/over-age HLS sessions and remove stale untracked
# segment directories. Active playlist and segment requests refresh
# session activity, while an absolute lifetime still bounds storage.
cleanup_hls_sessions:
command: "HlsSessionManager.cleanup_expired"
schedule: every 5 minutes
# HLS cleanup deliberately does not live here: it stops processes by PID
# and deletes tmp/hls directories that only exist in the web container, so
# it runs from the web process instead. See
# config/initializers/hls_session_cleanup.rb.
25 changes: 24 additions & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,30 @@ services:
environment:
RAILS_ENV: production
DATABASE_URL: postgres://${POSTGRES_USER:-streamvault}:${POSTGRES_PASSWORD}@db:5432/${POSTGRES_DB:-streamvault}
SOLID_QUEUE_IN_PUMA: "true"
# Background jobs run in the `worker` service below, so the web
# container must not start its own Solid Queue supervisor.
SOLID_QUEUE_IN_PUMA: "false"
# Both default to true, which is right behind a TLS-terminating proxy.
# Set both to false in .env to serve plain HTTP on a trusted network.
FORCE_SSL: "${FORCE_SSL:-true}"
ASSUME_SSL: "${ASSUME_SSL:-true}"
depends_on:
db:
condition: service_healthy
restart: unless-stopped

# Runs the Solid Queue supervisor, dispatcher and workers. The image's
# default command is the web server, so the command override is required.
worker:
build:
context: .
env_file: .env
environment:
RAILS_ENV: production
DATABASE_URL: postgres://${POSTGRES_USER:-streamvault}:${POSTGRES_PASSWORD}@db:5432/${POSTGRES_DB:-streamvault}
# Worker processes; each one runs three threads (config/queue.yml).
JOB_PROCESSES: "${JOB_PROCESSES:-1}"
command: [ "bundle", "exec", "bin/jobs" ]
Comment thread
vitobotta marked this conversation as resolved.
Resolved
depends_on:
db:
condition: service_healthy
Expand Down
Loading