18 Commits

Author SHA1 Message Date
Gilbert Chen
d9f6545d63 Rewrite the backup procedure to reduce memory usage
Main changes:

* Change the listing order of files/directories so that the local and remote
  snapshots can be compared on-the-fly.

* Introduce a new struct called EntryList that maintains a list of
  files/directories, which are kept in memory when the number is lower, and
  serialized into a file when there are too many.

* EntryList can also be turned into an on-disk incomplete snapshot quickly,
  to support fast-resume on next run.

* ChunkOperator can now download and upload chunks, thus replacing original
  ChunkDownloader and ChunkUploader.  The new ChunkDownloader is only used
  to prefetch chunks during the restore operation.
2021-10-24 23:34:49 -04:00
gilbertchen
3fd3f6b267 Merge pull request #606 from gilbertchen/erasure_coding
Implement Erasure Coding
2020-09-25 14:30:39 -04:00
Gilbert Chen
923a6fbc5b Implement Erasure Coding 2020-09-03 12:54:48 -04:00
Tet Woo Lee
4ae16dec7f add -persist in check and restore mode (for PR) 2020-05-06 18:39:52 +12:00
Gilbert Chen
6699e2f440 Fixed a bug that disabled RSA when copying from a non RSA-encrypted storage.
When copying to an RSA-encrypted storage, we relied on the RSA encryption
version to determine if a chunk is a snapshot chunk or a file chunk.  This is
wrong when the source storage is not encrypted or not RSA-encrypted.  There
is a more reliable to determine if a chunk is a snapshot chunk or not.
2020-03-13 20:13:27 -04:00
Gilbert Chen
cc88abd547 Fixed a bug that caused all copied chunks to be RSA encrypted
The field encryptionVersion in the Chunk struct is supposed to pass the status
of RSA encrytpion from a source chunk to a destination chunk in a copy command.
This field needs to be a 3-state boolean in order to pass the status correctly.
2020-02-13 14:03:07 -05:00
Gilbert Chen
9a0d60ca84 Store the public key in the config to ensure one key policy.
Also make sure that RSA encrpytion works with the copy command.
2019-09-23 12:53:43 -04:00
Gilbert Chen
90833f9d86 Implement RSA encryption
This is to support public key encryption in the backup operation.  You can use
the -key option to supply the public key to the backup command, and then the
same option to supply the private key when restoring a previous revision.

The storage must be encrypted for this to work.
2019-09-20 14:19:18 -04:00
Michael Cook
0762c448c4 gofmt -s 2018-12-29 13:20:10 +01:00
Michael Cook
741644b575 spelling 2018-12-29 13:04:40 +01:00
Gilbert Chen
21b3d9e57f Padding size was incorrect -- didn't pad to multiples of 256 2018-11-03 11:42:03 -04:00
Gilbert Chen
b1c1b47983 Add an env var DUPLICACY_DECRYPT_WITH_HMACSHA256 to force using HMAC-SHA256 for encryption key in order to be able to manage backups created by Vertical Backup 2018-05-02 22:57:47 -04:00
Gilbert Chen
923cd0aa63 Run goimports on all source files 2017-09-20 23:07:43 -04:00
Gilbert Chen
7f04a79111 Replace Fair Source 5 with our own free-for-personal-use license 2017-07-13 23:33:14 -04:00
Gilbert Chen
09f695b3e1 Verify chunk id for snapshot chunks 2017-07-07 12:08:38 -04:00
Gilbert Chen
42337d84c3 Move source files into src for a better looking front page 2017-06-07 14:04:11 -04:00
Gilbert Chen
4a1dc01ff4 Reorganize src directory 2017-06-02 16:33:36 -04:00
Gilbert Chen
bdf017e552 Move source files to src/ 2017-05-23 10:25:00 -04:00