Commit Graph

9 Commits

Author SHA1 Message Date
Gilbert Chen
bc2d762e41 Add -rewrite to the check command to fix corrupted chunks
This option is useful only when erasure coding is enabled.  It will
download and re-upload chunks that contain corruption but are
generally recoverable.  It can also be used to fix chunks that
are created by 3.0.1 on arm64 machines with wrong hashes.
2022-11-15 11:47:02 -05:00
Gilbert Chen
54952cef26 Fixed a bug that referenced uninitialized operator.snapshotCache 2022-07-10 12:23:11 -04:00
Gilbert Chen
fc2386f9cc Initialize startTime correctly in CreateChunkOperator 2022-06-09 23:28:29 -04:00
Gilbert Chen
d9f6545d63 Rewrite the backup procedure to reduce memory usage
Main changes:

* Change the listing order of files/directories so that the local and remote
  snapshots can be compared on-the-fly.

* Introduce a new struct called EntryList that maintains a list of
  files/directories, which are kept in memory when the number is lower, and
  serialized into a file when there are too many.

* EntryList can also be turned into an on-disk incomplete snapshot quickly,
  to support fast-resume on next run.

* ChunkOperator can now download and upload chunks, thus replacing original
  ChunkDownloader and ChunkUploader.  The new ChunkDownloader is only used
  to prefetch chunks during the restore operation.
2021-10-24 23:34:49 -04:00
Michael Cook
741644b575 spelling 2018-12-29 13:04:40 +01:00
Gilbert Chen
93cc632021 Record deleted snapshots in the fossil collection and if any deleted snapshot still exist nude the fossil collection 2018-08-04 22:59:25 -04:00
Gilbert Chen
fce4234861 Rearrange struct members to avoid 64-bit int alignment issues 2018-07-26 21:19:03 -04:00
Gilbert Chen
f68eb13584 A few fixes for multi-threaded pruning 2018-06-05 16:09:12 -04:00
Gilbert Chen
dd53b4797e Implement multi-threaded pruning 2018-06-04 21:52:07 -04:00