Tube Archivist

Hotfix published

There is a hotfix published for ARM64 builds where the ffmpeg binaries weren’t downloaded correctly into the container. If you are affected, pull again.

Project updates

• This ships various fixes for playlist refresh limitations. Also see the docs here clarifying that functionality.
• This fixes a comment extraction problem due to a change on YT
  • See below for a manual fix for your comments
• This also partially fixes failed channel extraction, we are waiting for upstream fix, that’s tracked here
  • See below to manually trigger a refresh of your failed channels
• This ships a new schedule management system, fixing various previous limitations. Your existing schedules and notification URLs should migrate automatically at first start, monitor your startup logs and validate on your schedule settings page that everything transferred correctly.

Added

• Added delete and ignore for videos, docs #286
• Added new notification hook for add to queue task, #714
• Added thumbs clean up in Thumbnail Check task
• Added 404 handler for detail views
• Added better error handling for invalid urls
• Added a few unit tests

Changed

• Changed build to use python script to download best ffmpeg build
• Changed schedule store to use beats model, no longer requiring restart to apply schedule, docs
• Changed task notification management, docs
• Changed playlist matching logic in download post processing, docs
• Changed queue handling to use Redis as temporary store, #500
• Changed queue handling progress and position tracking to use score in ordered set
• Changed to cleaner and more concise seconds to date parser

Fixed

• Fixed comment extraction
• Fixed playlist subscription bool logic in template, #684
• Fixed playlist rescanning logic, ensuring refresh, #686
• Partial fix channel subs extraction with workaround, #701, upstream link
• Fixed adding existing playlist to queue again, #634
• Fixed playlist matching not removing deleted videos
• Fixed rescan negative total notification
• Fixed schedule cron parser, you are no longer able to enter an invalid schedule
• Fixed logout
• Fixed application startup dependency making tests unrunnable, #564

Manual updates

You don’t need to run these commands, the regular metadata refresh should fix these problems, at least for active videos. That’s just if you want to fix it now.

First command will “relink” comments with your video. Due to a failed comment extraction, that link got lost, but all your comments are still here. Run python manage.py ta_fix_comment_link in your Tube Archivist container, e.g.:

docker exec -it tubearchivist python manage.py ta_fix_comment_link

Depending on your index size and system performance, expect this to take around 15 seconds for 10k videos. You will see ✓ task completed once that’s done.

Second command will trigger a reindex for channels that have failed to extract before, run python manage.py ta_fix_channels in your Tube Archivist container, e.g.:

docker exec -it tubearchivist python manage.py ta_fix_channels

This will scan your channels and add faulty channels to the regular reindex queue. That should return in a few seconds, you will see ✓ task queued. Then you can monitor the reindex progress on your settings page and in your docker logs.