Skip to content

Restore IPTV automatically after UniFi OS upgrades - #449

Open
kjanat wants to merge 16 commits into
fabianishere:masterfrom
kjanat:upstream/firmware-lifecycle
Open

kjanat wants to merge 16 commits into
fabianishere:masterfrom
kjanat:upstream/firmware-lifecycle

Conversation

@kjanat

@kjanat kjanat commented Aug 30, 2026 •

Copy link
Copy Markdown

What this changes

  • a UniFi OS upgrade no longer leaves IPTV offline: udm-iptv reinstalls itself, restores its configuration, and starts the IPTV service without manual intervention
  • GitHub Actions now runs a firmware-upgrade reproduction built from Ubiquiti's own .bin firmware files for every supported model

Closes #120.

Firmware-upgrade reproduction

The workflow queries Ubiquiti's firmware catalog at runtime, selects the two newest releases that existed when the pull request was opened, and verifies their SHA-256 checksums. It extracts their root filesystems from the official firmware files, installs the Debian package under the older firmware userspace, then starts the newer userspace under Docker and systemd with the persisted /data state and restore unit.

The test then exercises the packaged maintainer scripts, dpkg locking, restore helper, systemd unit, and IPTV proxy. It is a container reproduction of the firmware userspace transition, not hardware emulation; the real-router validation below covers the device itself.

How restoration works

  • persist the installed package, restore helper, configuration, and debconf answers under /data/udm-iptv
  • keep a restore unit under /etc/systemd/system, where it survives the firmware upgrade
  • run restoration only when the daemon is missing, so an ordinary reboot remains a no-op
  • wait for UniFi OS to release the dpkg lock before reinstalling
  • reinstall from the saved package without network access, with a bounded network fallback when no package was saved
  • restore the saved answers and configuration, then verify that the service remains running
  • disable automatic restoration on removal and delete persisted state on purge

Related changes

  • add explicit persist, restore, upgrade, and uninstall lifecycle commands
  • support installation and upgrades from releases, local packages, pull requests, and workflow runs
  • reuse matching private firmware images and build only missing images
  • harden the shell scripts and release workflow, including build attestations

Validation

  • a real UDM Pro firmware upgrade restored udm-iptv 3.0.6 automatically; the service was active with zero restarts, improxy was running, the IPTV interface and route returned, and live television worked
  • the five-model reproduction exercised UniFi OS 5.1.26 to 5.1.31 for UDM, UDM Pro, UDM SE, UDM Pro Max, and UDM Beast
  • Build, packaging, and the UniFi OS matrix pass on the current head: Build CI matrix from UniFi firmware catalog kjanat/udm-iptv#10
  • shellcheck -x -o all, Actionlint, dprint, Debian package building, and Lintian complete successfully

For the time being

I have setup releases1 from the fork repo that are fully functional. Install instructions2 in the readme, in case anybody would like to try it out.

Footnotes

  1. https://github.com/kjanat/udm-iptv/releases ↩

  2. https://github.com/kjanat/udm-iptv#configuring-helper-tool ↩

kjanat added 6 commits August 28, 2026 19:52
#1)

* Guard profile sourcing during preconfiguration

apt runs the config script before unpacking, so
/usr/lib/udm-iptv/profiles does not exist during that pass and sourcing
the selected profile aborts the script under sh -e. dpkg runs the config
script again at configure time, after unpacking, where the profile is
sourced successfully.

* Allow the installer to be pointed at another package

The version and repository were hardcoded, so installing a build other
than the latest upstream release required bypassing the installer.

UDM_IPTV_VERSION, UDM_IPTV_REPOSITORY and UDM_IPTV_PACKAGE now override
the download, and UDM_IPTV_PACKAGE also accepts a local path. Defaults
resolve to the same URL as before.

curl gains -f so an HTTP error is reported at download time instead of
being written to disk and failing later as a corrupt package.

* Persist the configuration across firmware updates

A firmware update replaces the root filesystem, removing the package, its
service unit and /etc/udm-iptv.conf. /data survives.

The debconf answers and the configuration are now copied to
/data/udm-iptv after every successful configuration, so no action is
required before an update. After one, `udm-iptv restore` reinstalls from
that copy and reapplies both. `udm-iptv persist` writes the copy on
demand.

restore waits for the package manager, because UniFi OS reinstalls its
own packages for several minutes after the first boot and holds the dpkg
lock throughout.

Refs fabianishere#120

* Install from a workflow artifact

Only releases could be installed, so a build that had not been released
was reachable only by downloading the package by hand.

UDM_IPTV_PR takes a pull request, as a number or as owner/repo#number,
and installs the artifact its head commit produced. UDM_IPTV_RUN takes a
workflow run, or "latest" for the most recent successful one. Either
waits up to UDM_IPTV_TIMEOUT for the run to finish, so the installer can
be started before the build is.

Listing pull requests, runs and artifacts needs no credentials;
downloading an artifact does, so UDM_IPTV_TOKEN (or GITHUB_TOKEN)
supplies it and its absence is reported before the download rather than
as a 401.

Settings that the configuration file may legitimately omit are now
dereferenced with a default, as is RET when no profile has been selected.

apt-get output was discarded with "2>&1 1>/dev/null", which sends stderr
to the terminal and stdout to /dev/null by way of an ordering that reads
like the opposite. It is now "> /dev/null", which is the same thing.

* Document the installer variables and update the actions

The variables that select which build to install were undocumented, so
nothing but the latest upstream release was discoverable.

The workflow actions were several major versions behind: checkout and
upload-artifact at v4, download-artifact at v4 and action-gh-release at
v2.

* Restore the saved configuration when reinstalling

persist ran unconditionally after configuration, so reinstalling after a
firmware update wrote the freshly defaulted answers over the ones being
kept, and restore then declined because the service it had just started
was running.

A first install that finds saved answers now applies them before the
configuration is written, which makes reinstalling the restore. restore
reinstalls and leaves the configuration to postinst.

* Reinstall automatically after a firmware update

A firmware update replaces the read-only layer of the root overlay and
discards the writable layer below /usr, which removes the package, its
service unit and the udm-iptv command. Recovering meant running
`udm-iptv restore` by hand, which is one of the things the update had
just removed.

udm-iptv-restore.service is installed in /etc/systemd/system, which
survives, and runs /data/udm-iptv/udm-iptv-restore, which persist keeps
alongside the answers. A condition on the missing daemon makes it a
no-op on an ordinary boot. It waits for UniFi OS to release the dpkg
lock and installs the copy of the package that the installer now leaves
in the state directory, so no network is needed; without that copy it
falls back to downloading the installer. postinst enables the unit and
postrm disables it on remove, since a package that reinstalls itself
after removal is not removed, and purge clears the state directory.
`udm-iptv restore` runs the same script.

curl follows redirects everywhere. A 3xx is not an error for -f, so
without -L the redirect body was what reached `sh -c`.

shellcheck -x -o all reported 431 findings and now reports none. Two of
them were bugs. _network_teardown read a target that was local to
_network_setup, so `udm-iptvd stop` deleted its iptables rules with an
empty -o and silently matched nothing; both functions now ask
_wan_target. custom.profile reported an unknown state by printing STATE
rather than the P_STATE it loops on.

`local` is gone from every script, which meant giving the variables that
shared a name across functions distinct ones. The two disable=SC2086
directives are gone with the string of arguments they covered, replaced
by positional parameters, and the DHCP options keep their word splitting
through a loop rather than an unquoted expansion. run_scripts, which
nothing called, is deleted.

Refs fabianishere#120

* Update installer and CI workflows

- Pin GitHub Actions to specific SHAs for improved security
- Refactor `install.sh` with better API error handling and tab indentation
- Fix a potential logic error in `udm-iptv` debconf parsing
- Restrict permissions on the generated preseed file to 600

* Refactor release workflow and simplify build step

- Use `working-directory` for the build step instead of `cd`.
- Simplify release name formatting and environment variable usage.
- Use the built-in `github.token` instead of `secrets.GITHUB_TOKEN`.

* Fix the regressions from the shellcheck sweep

Quoting ${staticroutes} in udhcpc.hook turns the whole option 121 list
into a single argument, so $2 is empty, the loop over destination and
gateway pairs never runs, and no classless static route is installed at
all. On a KPN connection that is the route IPTV travels over. The list
keeps its word splitting through positional parameters, the way the DHCP
options already do.

debian/config runs under sh -e, where a standalone assignment carries
the exit status of its command substitution. `readlink` of an interface
that is no longer present therefore ends the script before it asks its
first question, and `udm-iptv configure` is the documented way to
correct exactly that interface. Both assignments fall back to an empty
link, and `basename` runs only when there is one, so the stray newline
goes with the abort.

`db_set` returns the status debconf reports, and that is 10 for a
question the installed templates do not define. A preseed saved by a
different version could end postinst there and leave the package half
configured.

purge took its delete path from the environment of whoever invoked dpkg,
where UDM_IPTV_STATE_DIR=/data would have taken the whole partition.

A profile that cannot be read is still skipped rather than fatal, but it
says so now. udm-iptv-restore waits for the dpkg lock with `fuser`,
which ships in psmisc.
* Add UniFi OS firmware images and persist/restore CI

A firmware update drops /usr. Restore has to be tested by installing on
one official image and booting a newer one with the same /etc and /data.

firmware.tsv is the catalog. CI converts it with column(1) and yq, builds
arm64 images from 5.1.26 and 5.1.31, publishes them privately to ghcr.io,
and asserts udm-iptv is enabled and active after the upgrade reboot.

* Skip install tests until firmware images are published

Pull requests skip the publish job, so docker run against ghcr.io
exited 125 and fail-fast cancelled the rest of the matrix.

test-install now needs a successful publish. unsquashfs drops xattrs
so a non-root extract is not a failure.

* Build firmware images in the install test job

Each matrix job extracts the 5.1.26/5.1.31 pair, imports it, and runs
the upgrade reboot against those local tags. ghcr.io is written only
after that passes on workflow_dispatch.

* Refactor GitHub Actions workflows for better readability

- Use grouped redirections for GITHUB_OUTPUT in release.yml.
- Clean up whitespace and formatting in unifi-os.yml.
- Set working-directory for test-install job to simplify script paths.
- Consolidate environment variables at the job level.

* Keep the new firmware /etc on upgrade

Mounting the 5.1.26 /etc over 5.1.31 hid the new systemd and the
container exited. Only /data and the restore unit persist across the
upgrade boot.

* Update CI workflows and issue templates

- Add timeouts to UniFi OS build jobs
- Refactor yq commands for better readability
- Standardize quote usage in bug report template

* Add concurrency control to UniFi OS workflow

* Add concurrency and clean up build workflow

- Add concurrency grouping to prevent redundant CI runs
- Simplify YAML syntax for runs-on and step properties
- Use working-directory for the build step instead of manual cd

* Update concurrency logic for UniFi OS workflow

Use PR numbers for group isolation and enable cancellation for manual
workflow dispatches.

* Stop dpkg from waiting on udm-iptv.service

postinst hangs in deb-systemd-invoke start after enabling the unit.
policy-rc.d 101 lets the install finish; the test starts the service
itself with a 30s cap.

* Wait for restore before asserting the upgraded root

assert_restored ran the moment systemd was up, so /usr still lacked
the package and the job died in three seconds. It now starts the
restore unit, waits for the files, then starts udm-iptv.

build-deb timeout is 10 minutes so lintian is not a cancel.

* Improve UniFi OS integration test reliability

- Introduce a custom `udm-iptv-test.target` to speed up and stabilize
  systemd boot in Docker.
- Add GitHub Actions workflow commands for better error reporting and
  log grouping.
- Enhance diagnostic output when tests fail.

* Include workflow name in concurrency group

This prevents different workflows from canceling each other when triggered
by the same pull request or reference.

* Improve test logging and service startup in UniFi OS

Add debug logs to polling loops to track progress during package and
service waits. Switch systemctl commands to non-blocking to prevent
timeouts from stalling the test script.

* Suppress stderr from systemctl in test-install.sh

* Improve UniFi OS integration tests and debconf restoration

- Implement caching for firmware images in GitHub Actions to speed up
  integration tests.
- Add support for restoring debconf answers from `/data` during firmware
  restoration, ensuring configuration persists across updates.
- Refactor `test-install.sh` to use Bash and improve error handling.
- Optimize Docker build script to reuse cached firmware binaries.

* Add missing-profile warning check to test-install.sh
* Add uninstall --keep-data instructions to README

* Enhance upgrade and uninstall commands

- Add support for specific versions, packages, PRs, and workflow runs
  to the `upgrade` command.
- Implement `uninstall` with `--purge` and `--keep-data` options.
- Update `postrm` to correctly handle service cleanup during removal.
- Include `install.sh` in the Debian package for offline upgrades.
- Add integration tests for package upgrades and uninstallation.

* Use xz compression for test-install deb packages

* Dump Docker logs on test failure in cleanup script

* Add retry logic for systemd service activation

The restore script now retries enabling and restarting the service
for a configurable timeout period to handle transient systemd reloads.
* Suppress ShellCheck SC2154 warnings in profiles

Add a ShellCheck disable directive for the RET variable in all
profile scripts, as it is populated by the db_get command.

* Defer profile application during package installation

On first install, the debconf config script runs before the package
payload is unpacked, making profiles unavailable. This change defers
profile application by caching the selection and applying it during
the post-installation phase once the files are present.

* Fix debconf persistence during installation

Pass current debconf session values directly to the persist command
during postinst. This ensures that answers provided during the active
installation transaction are saved to disk, as the debconf database
on disk is not updated until the script exits.
Includes necessary permission updates for GitHub Actions to generate and
store attestations for the Debian packages.
Introduces a `log_config` helper to dump `/etc/udm-iptv.conf` at key
stages of the installation, persistence, and upgrade lifecycle.
@kjanat
kjanat marked this pull request as draft August 30, 2026 12:25
kjanat added 8 commits August 30, 2026 15:35
Exercise package restoration through the persisted systemd unit and the
real firmware proxy instead of recreating state and starting services in
the harness. Preserve firmware xattrs, test offline and locked restore
paths, and keep saved-package restoration independent of apt metadata.
Let saved-package restoration start without network-online, retry the network-only fallback with a bound, and keep firmware-only units out of the reduced test target. Preserve firmware xattrs on hosts without passwordless sudo by extracting through Docker.
Use an exit trap for the installer-owned temporary directory so failed
artifact downloads and package installations do not leave files behind.
Keep fallback downloads within the configured deadline, give locked
restores their full test budget, and observe service stability long
enough for CI to reject restart loops.
Pull catalog-matched firmware images in parallel and build only missing
ones. Keep image defaults fork-neutral and distinguish the Docker host's
multicast limitation from an unexpected service restart.
- Enable `cancel-in-progress` for pull requests in build workflows
- Pin `softprops/action-gh-release` to a specific version hash
Build CI matrix from UniFi firmware catalog
@kjanat
kjanat marked this pull request as ready for review August 30, 2026 21:48
@kjanat kjanat changed the title Persist and automatically restore configuration after firmware updates Restore IPTV automatically after UniFi OS upgrades Aug 30, 2026
kjanat added 2 commits August 31, 2026 10:05
Stop same-version release upgrades before downloading, add a force path
that downloads and reinstalls deliberately, and let package postinst own
configuration persistence so reconfigure saves answers once.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Installation not persistent on UDR / UDM SE

1 participant