We are very excited to announce the 0.6 release of Kopia! This is a big milestone on the way to an upcoming 1.0 release and there have not only been a large number of changes across the stack but also growth of the community.

This version brings manny performance, usability and stability improvements listed below, adds supports for new providers and CLI options and introduces major new features described at https://kopia.io/docs/release-notes/v0.6.0/

Upgrade notes:

• Upgrading from 0.5.x is supported, make sure to follow instructions in #503 to enable maintenance tasks.
• Upgrading from 0.4.0 or earlier is not officially supported, it may work but use at your own risk. It’s strongly recommended to create new repository using v0.6.0 and migrate existing snapshots as outlined in the documentation.

Changes in this release:

core

* big performance improvements when snapshotting large directories (#331)
* improvements for dealing with eventually-consistent stores (S3) (#437)
* GC safety improvements to resolve race condition when content is re-referenced when about to be deleted (#420)
* added OpenCensus (#339)
* introduced explicit maintenance operations that perform periodic repository cleanup/compaction (#411)
* disabled automatic compaction on repository opening - moved to maintenance tasks
* added AsyncWrites to ObjectWriter, which improves performance… (#369)
* object: ensure that all I objects have a content prefix which improves locality by putting them in q packs
* deduplicate multiple policies for the same source in policy manager, fixes #391
* fixed race condition during Open() where we may read incomplete file
* deprecated NONE algorithm, will not be available for new repositories (#395)
* server: automatically flush the repository after setting or deleting a policy (#489)
* snapshot checkpointing (#410)
* moved creating cache directory from connect to first use (#450)
* persist relative path to cache if possible, this allows config directory to be partially portable
* server: implemented 'flush' and 'refresh' API
* experimental support for remote repository (#427)
* repo: refactored public API (#318)
* upload: auto-ignore kopia cache directories when creating snapshots (#524)

KopiaUI

 * support for multiple repositories + portability (#398)
 * highlight snapshot errors (#376)

Providers

* support for gather writes (#373)
* b2: added provider for backblaze b2
* sftp: add missing options for configuring sftp known_hosts
* s3: add CLI option for disabling tls verification while connecting to s3
* filesystem: added retry which addresses the macOS race condition

CLI Improvements:

* added flags to control progress output
* 'kopia server' made --ui default (#452)
* allow override of snapshot start time and end time
* improved 'snapshot delete' usage (#436)
* Remove legacy flags from snapshot create command (#441)
* support for zip, tar and tar.gz restore outputs (#482)
* Support for repository sync (#522)

Infrastructure:

* macOS and Windows KopiaUI builds are now signed
* robustness testing framework
* testing: added blob.Storage wrapper that simulates eventual consistency (#434)
* switched back to using v-prefixed tag names.
* tests: added smoke test that exercises all combinations of encryption and hashing

Changelog

98448090 Updated documentation for v0.6.0 release (#525) bb2434dc upload: auto-ignore kopia cache directories when creating snapshots (#524) 505ab92e Support for repository sync (#522) d0e631a7 fix typo in url (#523) b995a1d0 app: Disabled progress percentage display above 100% (#521) cfa30f2e switched back to using v-prefixed tag names. ff244b54 go.mod: upgrade github.com/klauspost/compress to v1.10.10 8ead49b7 restore: support for zip, tar and tar.gz restore outputs (#482) 52e76315 kopia-ui: fixed missing ‘log’ when clicking ‘Launch At Startup’ 7e9ce61f server: automatically flush the repository after setting or deleting a policy (#489) a544e0c3 kopia-ui: Added logging when waiting for notarization response from Apple. 2c21582e kopia-ui: added option to disable TLS and force HTTP 73ed5adb kopia-ui: cleaned up console logging 8c941ccd Fix BUILD.md (#481) 0b2d12df kopia-ui: build deb and rpm packages ce5e6dcd [Robustness] Add first robustness tests 82a2fa0e [Robustness] Add test engine to manage snapshot verification testing (#468) d44c01fa content: do not short circuit compaction if there’s one index but there are contents to delete 64a6cb42 parallelwork: fixed error handling, which caused parallel work to never finish on any error 1c253419 cli: fixed panic() during snapshot verification caused by refactoring to new Repository interface 3843ed67 cli: added content verify —include-deleted flag ed9511ba b2: fix GetMetadata() implementation 25934a54 content: fixed deletion of cleanup blobs (#462) 79757672 server: implemented ‘flush’ and ‘refresh’ API 293cb104 cosmetic changes to maintenance behavior (#461) c13b5f82 Remove extra whitespaces and fix minor typos (#460) 85aa4ae6 Add ‘repositories’ page (#454) deb1099c Trivial: remove unused logging.Option type 6bc47cb3 content: emit warning whenever the number of index blobs gets too high, which is an indication that maintenance has not run in a long time 960c3347 maintenance: disabled automatic compaction on repository opening 6cb970f1 Revert “Fix #360: Leverage Go 1.14 testing.T.Cleanup() (#455)” (#458) 73b913c4 Fix #360: Leverage Go 1.14 testing.T.Cleanup() (#455) b30da511 Wire up snapshot-store-compare behavior d68273a5 Improvements for dealing with eventually-consistent stores (S3) (#437) 9123e3eb cli: ‘kopia server’ made —ui default (#452) 130712a1 Remove unused content.Manager.closed channel (#451) e445dc5e repo: moved creating cache directory from connect to first use (#450) fff2b44e Upgrade AWS SDK to 1.31.3 from 1.31.1 0642b261 Upgrade github.com/klauspost/compress to 1.10.6 0afd1a54 Upgrade github.com/minio/minio 97e04d53 Upgrade golang.org/x dependencies 0875939d dep: upgrade protobuf dependents (#442) 23c12125 Build test/tools in Darwin as well (#447) 0c2ad997 Upgrade github.com/minio/minio-go/v6 53b82c93 dep: upgrade various dependencies (#445) 0a6a0d4f Upgrade AWS SDK to 1.31.1 ea681e40 deps: Upgrade compress package to 1.10.5 (#443) e4931772 Remove legacy flags from snapshot create command (#441) 8c4fb53c blob: support for GetMetadata() to get server-side timestamp and blob length (#440) ca284697 cli: improved ‘snapshot delete’ usage (#436) 7e5fc52c removal of unnecessary extra )} f3359d78 add sftp and webdav as repositories to “Getting started” documentation page, “Setting Up Repository” chapter 993c3bfb linter: fixed how linter is installed on Darwin and Linux since it was getting wrong version d6574158 testing: added blob.Storage wrapper that simulates eventual consistency (#434) 200ef57c app: portability fixes (#431) be4b8975 Support for remote repository (#427) 1377d057 Maintenance changes (#423) 4f6dd947 Maintenance: report and record record maintenance run timings (#422) a462798b content: added blob-level metadata cache (#421) d707f361 GC content repair (#420) 2b8e882b filesystem: added retry which addresses the macOS race condition 4b4628a2 Repository maintenance support (#411) 573d1042 object: ensure that all I objects have a content prefix b671bda1 cli: add missing options for configuring sftp known_hosts 14fb49a8 content: asynchronous cache sweep e50b5ec2 Trivial: Add package alias to silence IDE/gopls warning 1f1682b2 Snapshot checkpointing (#410) 70d4c876 cli: improvements to content selection for list/rewrite/stats/verify (#409) 057c2789 Kopia UI: support for multiple repositories + portability (#398) 45195fab appveyor: moved WINDOWS_CERT_SHA1 to secure variable so that we don’t sign PRs c89b1a7c appveyor: added GH_TOKEN variable, disabled RDP 01f52f8c appveyor: fixed secure variable 24f6720c enable RDP on appveyor worker until it’s stable 0017f59a Makefile: fix travis-setup on Windows 2055fe3c Makefile: support Windows Appveyor environment (#407) 16275a15 Makefile: changed how build number is calculated 37b69d8e site: tweaked layout and options for cookie consent popup e0984641 improved logging for sign.js 6ad92738 travis: setup tools to support signing Windows binaries 05ea35fc testing: retry around B2 test 567bd33e added b2 to repository creation ui 4bed4511 implemented b2 backend 3924fb24 Minor cleanup for snapshot time override (#392) 6cb9b8fa repo: refactored public API (#318) 10bb4929 repo: deprecated NONE algorithm, will not be available for new repositories (#395) 60977812 Support for gather writes (#373) b08d3948 policy: deduplicate multiple policies for the same source in policy manager, fixes #391 393d273e Policy set: fix nil pointer dereference #385 (#387) ea510f30 repo: fixed race condition during Open() where we may read incomplete file f2e6d515 repo: persist relative path to cache if possible, this allows config directory to be partially portable 9b68a631 Highlight snapshot errors in the UI and CLI (#376) 8687f1c0 object: added AsyncWrites to ObjectWriter, which improves performance… (#369) 2e7fe7c2 travis: switch osx to xcode11.3 e86e8970 fix makefile for windows (#377) 6789f8e6 cli: allow override of snapshot start time and end time a1f7a068 Add CLI option for disabling tls verification while connecting to s3 (#370) 1df5c81d correct a typo in Doc 73e21e01 correct a typo in kopia UI 239d8090 performance: introduced buf.Pool which helps reuse memory buffers (#345) becf08af testing: added missing condition variable broadcast after finishing flushing, fixes timeouts on Travis 4a444e6c content: reduce allocations in index scan 169f25c8 s3: pass blob length to minio in PutBlob() b5d44215 upload: added missing writer.Close() b1020941 splitter: exposed MaxSegmentSize() for splitters 4564ed5a encryption: added MaxOverhead to encryptor c9877bf1 performance: refactored content manager to avoid copying data 53a59263 content: wait for cache sweep goroutine to close (#361) d930f06a cli: added flags to control progress output 06fbaba1 testing: ensure cleanup of content managers, fixes #356 272b6575 upgrade all dependencies to latest versions a8ef0b2b kopia-ui: fixed auto-update 4088eb51 site: updated CDN links for cookie consent c1507671 site: added cookie consent form 674543eb site: added simple privacy policy 331ff076 cli: added number of files that are in the process of being hashed 78a921ab Rename Simple mutex 60d1cc82 [robustness testing] Implement metadata store for test metadata using kopia 6b04b427 tests: added smoke test that exercises all combinations of encryption and hashing e2a169d2 hotfix: kopia-0.5.0 did not properly support 256-bit hashes 8d452a82 performance: improvements to object manager (#336) 526445a9 CLI: pre-allocated buffers for crypto benchmark (#343) e80f5536 performance: plumbed through output buffer to encryption and hashing,… (#333) 0d1a627d Improved upload performance and memory usage (#331) 89c0c6ba Refactor CLI stats (#341) 71e55630 Fix download link (#342) 6ce8410a Added OpenCensus (#339) edc87fcc Refactor content stats (#340) 9c3d419b Fixed social site logo - alternative #2 (#335) 514df69a performance: added wrapper around io.Copy() ec56ed3d Makefile: fixed layering test f07d1648 Makefile: added layering-test, fixes #324 05852322 Add snapshotter interface and kopia implementation d9ce3d0a Inject time in Kopia components (#314) 5f96b024 testing: added retry helper bd3cace7 site: changed Community page to make it more obvious how to get to Slack and GitHub 88ce341a Trivial cleanup for internal diff (#316) 3ebef0ad snapshot: removed content stats from snapshot.Stats, which are confusing and incorrect if snapshots are done in parallel d862bc2d site: delete news section since it’s empty 7638d4a9 site: clarified docs based on Reddit questions, removed stale link to user mailing list which is not in use