Main changes:
* Change the listing order of files/directories so that the local and remote
snapshots can be compared on-the-fly.
* Introduce a new struct called EntryList that maintains a list of
files/directories, which are kept in memory when the number is lower, and
serialized into a file when there are too many.
* EntryList can also be turned into an on-disk incomplete snapshot quickly,
to support fast-resume on next run.
* ChunkOperator can now download and upload chunks, thus replacing original
ChunkDownloader and ChunkUploader. The new ChunkDownloader is only used
to prefetch chunks during the restore operation.
This is to avoid the read-after-rename consistency issue where the effect
of renaming may not be observed by the subsequent attempt to download the
just renamed chunk.
A new variable was added previosuly which caused a 64-bit variable to be not
aligned on a 8-byte boundary. Go still can't handle such variables on 32-bit
OS.
* Restore/check should report an error instead of a success at the end if there
were any errors and -persist is specified
* Don't compute the file hash before passing the file to the chunk maker; this is
redundant as the chunk maker will produce the file hash
* Add a LOG_WERROR function to switch between LOG_WARN and LOG_ERROR dynamically
This option will download and verify every chunk. Unlike the -files option,
this option only downloads each chunk once. There is also a new -threads
option to use multiple threads to download chunks.
Wasabi's GetObject occasionally (approximately 2% of the time in my testing) returns objects whose contents disagree with what has been stored in Wasabi. These cause errors when chunks are downloaded (during restore, for example). Previously, these errors would abort the restore, requiring that it be started over from the beginning. This made it effectively impossible to complete any normally-sized restore where the cumulative chance of encountering such an error approaches unity.
With this change Duplicacy will retry up to three times if it can't decrypt the downloaded chunk, or if the downloaded chunk's ID doesn't agree with a chunk ID computed from the downloaded chunk's content.
