ArchiveBox

WIP ALPHA pre-release for the upcoming ArchiveBox v0.8 release.

[!CAUTION] This was an ALPHA pre-release. We were promoting it a little earlier than usual because it contains ✨ lots of big new features ✨ and we want brave early adopters to help us test it!

Highlights

• New REST API built with django-ninja (thanks @Brandl!)
• New ability to send outgoing webhooks triggered by archiving events
• new support for S3/B2/Google Drive/etc. remote storage using Docker + rclone
• new ability to manage ArchiveBox config in Admin UI (read-only for now, ability to edit coming soon…)
• new noVNC remote viewing support for ArchiveBox browser (grab the updated docker-compose.yml first!)
• upgraded to Django 5.0 internally (thanks @jimwins!)
• add new *_EXTRA_ARGS options (thanks @benmuth!) and new unified USER_AGENT option
• add new generic_jsonl parser (thanks @jimwins!)
• switch to feedparser for RSS parsing (thanks @jimwins!)
• remember Snapshot detail page header expanded/collapsed state

Expand to see see more...

• add gitea and other domains to default GIT_DOMAINS list to run git archiving on
• check /, /data, and /data/archive in Docker and warn if running low on disk space
• Add COOKIES_FILE support for singlefile extractor by @naoph in https://github.com/ArchiveBox/ArchiveBox/pull/1372
• Use COOKIES_FILE to fetch page titles by @benmuth in https://github.com/ArchiveBox/ArchiveBox/pull/1364
• Fallback to not chown’ing ./data/archive dir if it’s a network mount that prevents ownership changes by @gnattu in https://github.com/ArchiveBox/ArchiveBox/pull/1312
• Show the upgrade notification only in specific views by @benmuth in https://github.com/ArchiveBox/ArchiveBox/pull/1314
• ability to populate is_staff and is_superuser flags at LDAP authentication by @vladimirdulov in https://github.com/ArchiveBox/ArchiveBox/pull/1335
• Make it a little easier to run specific tests by @jimwins in https://github.com/ArchiveBox/ArchiveBox/pull/1371
• disable chrome automatic self-updating when running headless
• Add ability to populate is_staff and is_superuser flags during LDAP first auth
• allow more restrictive NFS permission coercion on ./data/archive
• bump yt-dlp, singlefile, wget, curl, and chrome versions
• fix RESOLUTION being ignored when using Chrome headless in Docker
• fix sorting by Size / Files in the Admin Snapshots list page UI
• fix spinner icon showing on some Snapshots instead of favicon when only a few extractors are enabled
• fix yt-dlp sometimes failing to archive media due to filenames being too long or containing special characters
• fix wget extractor not finding output when :80 or :443 port is present in the original URL
• fix /var/spool/cron/crontabs permissions when mounting it via Docker
• fix /browsers chown on Docker armv7 entrypoint failing

COMING SOON: new sci-dl scientific paper downloader being worked on by @benmuth

New Contributors

• @Brandl made their first contribution in https://github.com/ArchiveBox/ArchiveBox/pull/1397
• @tqobqbq made their first contribution in https://github.com/ArchiveBox/ArchiveBox/pull/1396
• @gnattu made their first contribution in https://github.com/ArchiveBox/ArchiveBox/pull/1312
• @speerer made their first contribution in https://github.com/ArchiveBox/ArchiveBox/pull/1323
• @neel-suthar made their first contribution in https://github.com/ArchiveBox/ArchiveBox/pull/1330
• @jimwins made their first contribution in https://github.com/ArchiveBox/ArchiveBox/pull/1365
• @naoph made their first contribution in https://github.com/ArchiveBox/ArchiveBox/pull/1372
• @rdela made their first contribution in https://github.com/ArchiveBox/ArchiveBox/pull/1374
• @n-hebert made their first contribution in https://github.com/ArchiveBox/ArchiveBox/pull/1382

Full Changelog: https://github.com/ArchiveBox/ArchiveBox/compare/v0.7.2…v0.8.0-rc