Tube Archivist

Project Update

• This ships various fixes provided by the amazing yt-dlp team, working around some YT shenaningans.
• TA can now integrate with bgutil-ytdlp-pot-provider, created by Brainicism.
  • For how to install and configure, check out our docs
  • A PO token can in some cases work around 403 errors or other bot detection mechanism. For additional context see the yt-dlp wiki
• This fixes a validation issue for the date published on the download queue. If you have faulty items indexed, see here for steps on how to rectify that.
• This brings much better serialization and data validation over all. At first start, depending on your index size and hardware, expect this to take a few minutes as there are various migration steps to fix and standardize mapping and index.
• This now introduces index rebuild from embedded TA metadata.
  • As we have now much better validated index metadata, run Embed metadata into media file again to apply this to the embedded metadata too.

EDIT: If you see a ValueError: Incorrect timezone setting at first startup, you are likely using a “linked” timezone alternative. They were available in the previous base image of Python, but have been removed. You can find the actual timezone in the list of this wiki page there you can for example see that US/Central points to America/Chicago.

Added

• Added tini to Docker CMD for better signal handling, by @geicht
• Added channel filtering in fulltext search, by @DySprozin, docs
• Added serialization check for adding to queue, by @bbilly1
• Added serialization for indexes, by @bbilly1
• Added metadata embed during manual import and filesystem rescan import, by @bbilly1
• Added index from embed for shared offline files, by @bbilly1
• Added channel and playlist artwork embedding, by @bbilly1
• Added version number to embedded metadata, by @bbilly1
• Added artwork restore from embedded metadata, by @bbilly1
• Added error handling control for filesystem rescan and manual import, by @bbilly1, docs and docs
• Added playlist indexing from embedded metadata, by @bbilly1
• Added potoken provider by @bmcdonough and @bbilly1, docs
• Added temporary cookie store from yt-dlp for less frequent player ID recreation, by @bbilly1, #1081
• Added cache folder for yt-dlp internal use, by @bbilly1, #1081
• Added socket status for membership connection, by @bbilly1

Changed

• Changed old now redundant whitenoise static file handling, by @PawsFunctions
• Changed subtitle fetching to use cookie if enabled, by @bbilly1
• Removed separate thumb embedding task in favor of unified embedding task, by @bbilly1
• Changed index mapping update logic, switch to alias for faster index rebuild, by @bbilly1, docs
• Changed index mapping definition, define all fields for better serialization, by @bbilly1

Fixed

• Fixed locking down permissions on API docs, by @bbilly1
• Fixed home page link in custom playlists, by @bbilly1, #1088
• Fixed local playlist matching without remote fetch, by @bbilly1 #1092
• Fixed invalid timestamp parsing in add to queue and indexing, by @isaacsanders and @bbilly1, #1094
• Fixed null parsing for deactivated appsettings, by @bbilly1
• Fixed comment tree indexing and parsing, by @bbilly1, #1102
• Fixed duplicate cache expire headers in nginx, by @mmomjian, #1111
• Fixed handling for utf8 errors in metadata embedding, #1113