jkl1337/duplicacy
1
0
Fork 0
mirror of https://github.com/jkl1337/duplicacy.git synced 2026-01-03 04:04:45 -06:00
Code Issues Packages Projects Releases Wiki Activity
757 Commits 11 Branches 57 Tags
1661caeb9240aead114ad8cd0eb9242fb5021b12
Commit Graph

398 Commits

Author SHA1 Message Date
Mark Feit
1661caeb92 More fixups 2021-12-04 10:37:11 -05:00
Mark Feit
041ba944c4 Typo 2021-12-04 10:24:31 -05:00
Mark Feit
934c2515cc Import context 2021-12-04 10:23:42 -05:00
Mark Feit
c363d21954 First cut of Swift v2 2021-12-04 10:11:19 -05:00
Gilbert Chen
e43e848d47 Find the storage path in shared folders first when connecting to Google Drive 
When connecting to Google Drive with a service account key, only files in the
service account's own hidden drive space are listable.  This change finds
the given storage path among shared folders first so that folders from the user
space can be made accessible via service account.
 2021-03-09 22:46:23 -05:00
gilbertchen
fd1b7e1d20 Merge pull request #612 from gilbertchen/gcd_impersonate 
Support GCD impersonation via modified service account file
 2021-03-09 10:24:08 -05:00
Gilbert Chen
f83e4f3c44 Fix SNAPSHOT_INACTIVE log message 2021-01-28 00:06:49 -05:00
Gilbert Chen
4cfecf12f8 Show the path in the error when a subdirectory can't be listed 2021-01-04 10:17:11 -05:00
Gilbert Chen
41a8f657c4 Add test storage for StorageMadeEasy's File Fabric 2020-11-23 14:38:15 -05:00
Gilbert Chen
474f07e5cc Support GCD impersonation via modified service account file 2020-11-23 09:44:10 -05:00
Gilbert Chen
bec3a0edcd Fixed a bug that caused a fresh restore to fail without the -overwrite option 
When restoring a file that doesn't exit locally, if the file is large (>100M)
Duplicacy will create an empty sparse file.  But this newly created file will
be mistaken for a local copy and hence the restore will fail with a message
suggesting the -overwrite option.
 2020-11-03 10:57:47 -05:00
Gilbert Chen
b392302c06 Use github.com/pkg/xattr for reading/writing extended attributes. 
The one previously used, github.com/redsift/xattr, is old and can only process
user-defined extended attributes, not system ones.
 2020-10-16 21:16:05 -04:00
Gilbert Chen
7c36311aa9 Change snapshot source path from / to /System/Volumes/Data 
Also use a regex to extract the snapshot date from tmutil output.
 2020-10-11 15:23:09 -04:00
Gilbert Chen
7f834e84f6 Don't attemp to load verified_chunks when it doesn't exist. 2020-10-09 14:22:45 -04:00
Gilbert Chen
d7c1903d5a Skip chunks already verified in previous runs for check -chunks. 
This is done by storing the list of verified chunks in a file
`.duplicacy/cache/<storage>/verified_chunks`.
 2020-10-08 19:59:39 -04:00
Gilbert Chen
0da55f95ab Fixed a 64-bit integer alighment problem on 32-bit OS 
A new variable was added previosuly which caused a 64-bit variable to be not
aligned on a 8-byte boundary.  Go still can't handle such variables on 32-bit
OS.
 2020-10-01 10:35:59 -04:00
Gilbert Chen
2f407d6af9 Another change needed for symlinked chunk subdirectories 2020-10-01 09:27:20 -04:00
Gilbert Chen
7e372edd68 Allow chunks subdirectories in the disk storage to be symlinks. 2020-09-29 13:15:45 -04:00
Gilbert Chen
836a785798 Add src/duplicacy_utils_freebsd.go 2020-09-26 21:22:49 -04:00
Gilbert Chen
6ad698328f Fixed test build errors caused by previous merges 2020-09-26 12:01:38 -04:00
Gilbert Chen
ace1ba5848 Remove runtime OS check for excluding by attributes 2020-09-25 22:37:54 -04:00
gilbertchen
1fedfd1b1a Merge branch 'master' into mac-exclude 2020-09-25 20:13:43 -04:00
gilbertchen
3fd3f6b267 Merge pull request #606 from gilbertchen/erasure_coding 
Implement Erasure Coding
 2020-09-25 14:30:39 -04:00
Gilbert Chen
e3e3e97046 Improvements for the WebDAV backend 
* CreateDirectory() looks up the directory in the cache first and don't create
  it if found in cache
* ListFiles() puts subdirectories in the cache
* If CreateDirectory() encounters EOF, assume the directory already exists
 2020-09-25 13:56:44 -04:00
Gilbert Chen
3f29ec2ffb File metedata must be restored even if the content is unchanged. 2020-09-24 22:44:43 -04:00
Gilbert Chen
947006411b Improvements for the copy command 
* Added a `-download-threads` option for specifying the number of downloading
  threads
* Show progress log messages during copy
 2020-09-24 14:56:19 -04:00
Gilbert Chen
6841c989c6 Fixed a bug that caused check -chunks -persist to succeed with broken chunks 
The bug was not setting the `isBroken` flag in WaitForChunk()
 2020-09-24 14:53:42 -04:00
Gilbert Chen
d0b3b5dc2e Print progress logs when verifying chunks (check -chunks) 2020-09-23 09:02:53 -04:00
Gilbert Chen
73ae3f809e Revert "Add a -max-list-rate option to backup to slow down the listing" 
This reverts commit 67a3103467.
 2020-09-22 22:08:43 -04:00
Gilbert Chen
67a3103467 Add a -max-list-rate option to backup to slow down the listing 
This option sets the maximum number of files that can be listed in one
second.
 2020-09-22 08:27:09 -04:00
Gilbert Chen
6ee01a2e74 Allow RSA keys to be passed directly via CML instead of a file 
This change is intended to be used by the web GUI to create an RSA encrypted
storage.
 2020-09-20 20:03:52 -04:00
Gilbert Chen
b7d820195a Remove a debug log message accidentally checked in 2020-09-18 14:59:44 -04:00
Gilbert Chen
16d2c14c5a Follow-up changes for the -persist PR 
* Restore/check should report an error instead of a success at the end if there
  were any errors and -persist is specified
* Don't compute the file hash before passing the file to the chunk maker; this is
  redundant as the chunk maker will produce the file hash
* Add a LOG_WERROR function to switch between LOG_WARN and LOG_ERROR dynamically
 2020-09-18 11:23:35 -04:00
Gilbert Chen
eecbb8fa99 Fix OneDrive Business and improve retry mechanism. 
* Switch to the upload-by-session api for OneDrive Bussiness as their servers
may keep incomplete files when an upload is aborted when the simple upload API
is used

* Use the delay value in the http Retry-After header if there is one

* Decorate the https traffic in the hope of less rate limiting.
 2020-09-18 10:19:03 -04:00
Gilbert Chen
97bae5f1a3 Close the response body when 301 is returned in the WebDAV backend 2020-09-14 11:36:22 -04:00
Gilbert Chen
40243fb043 Fixed a compile error in previous checkin 2020-09-12 11:39:46 -04:00
Gilbert Chen
403df1fd06 Added a call to discard http response where it was missed in previous fixes. 
Also added a missing import "io/ioutil".
 2020-09-09 21:47:01 -04:00
gilbertchen
4369bcfc0b Merge pull request #549 from Jos635/master 
Improve WebDAV performance
 2020-09-09 16:02:36 -04:00
gilbertchen
d2b08aebee Merge pull request #594 from alecuyer/fix/swift 
Call Authenticate() before using Swift storage
 2020-09-09 15:48:36 -04:00
gilbertchen
ca4d004aca Merge branch 'master' into add_persist_pr 2020-09-09 15:42:01 -04:00
Gilbert Chen
ce472fe375 Show erasure coding/rsa encryption if enabled for backup and copy 2020-09-04 10:43:37 -04:00
Gilbert Chen
923a6fbc5b Implement Erasure Coding 2020-09-03 12:54:48 -04:00
Gilbert Chen
fd469bae9e Check the returned value of Close() when uploading a chunk file via SFTP. 2020-08-29 22:22:51 -04:00
Gilbert Chen
1eb1fb14a8 Don't throw an error on 0-byte chunk files with suffix '.tmp'. 2020-07-07 23:25:09 -04:00
Gilbert Chen
1da7e2b536 Fix a crash when a username is not specified with the WebDAV backend 2020-07-03 12:29:53 -04:00
Gilbert Chen
ed8b4393be Add a new backend for StorageMadeEasy's File Fabric storage. 
The storage url is fabric://username@storagemadeeasy.com/path/to/storage.
 2020-07-03 12:07:33 -04:00
Gilbert Chen
f2f07a120d Retry on "unexpected EOF" errors for the webdav backend. 2020-07-03 11:48:30 -04:00
Gilbert Chen
153f6a2d20 Use multiple threads to list the chunks directory for Google Drive 2020-06-15 12:49:13 -04:00
Gilbert Chen
5d45999077 Clear the loaded content after a snapshot has been verified 
The snapshot content is loaded before verifying the snapshot, but after that
it isn't used anymore so it should be released to save memory.
 2020-06-10 10:08:53 -04:00
Gilbert Chen
1adcf56890 Add an SFTP backend that supports more ciphers and kex algorithms. 
"sftpc://" supports all algorithms implemented in golang.org/x/crypto/ssh,
especially including those weak ones that are excluded from the defaults.
 2020-06-08 11:24:20 -04:00