jkl1337/duplicacy
1
0
Fork 0
mirror of https://github.com/jkl1337/duplicacy.git synced 2026-01-04 04:34:45 -06:00
Code Issues Packages Projects Releases Wiki Activity
757 Commits 11 Branches 57 Tags
1661caeb9240aead114ad8cd0eb9242fb5021b12
Commit Graph

398 Commits

Author SHA1 Message Date
Philipp Bandow
5821cad8c5 Add BucketName to API call in FindBucket function 2019-06-28 12:15:45 +02:00
gilbertchen
6efcd37c5c Merge pull request #562 from gilbertchen/azure_retry 
Retry on broken pipe in Azure backend
 2019-06-20 12:14:48 -04:00
Gilbert Chen
045be3905b Better handling of B2 authorization failures 
This commit fixed 2 issues wrt Backblaze B2 authorization:
* every thread may call b2_authorize_account at the same time when there
are 401 errors
* if B2 has a login outage, then all threads will call b2_authorize_account
repeatedly without delay

A simple solution is to limit one b2_authorize_account call to once every
30 second regardless of how many threads there are.  If the call to
b2_authorize_account is not allowed, the random exponential backoff will
be performed.
 2019-06-13 22:43:07 -04:00
Gilbert Chen
4da7f7b6f9 Check -files may download a chunk multple times 
This commit fixed a bug that caused 'check -files' to download the same chunk
multiple times if shared by multiple small files.
 2019-06-13 14:47:21 -04:00
Gilbert Chen
9d4ac34f4b Don't compare hashes of empty files in the diff command 
Empty files may or may not have a hash depending if the -hash option is used
during backup.
 2019-06-06 12:35:34 -04:00
Gilbert Chen
47c4c25d8b Fixed a bug that restoring files doesn't work due to missing parent directory 
The root cause was path.Dir can't handle Windows paths that use \ as the
separator.
 2019-06-04 21:57:10 -04:00
Gilbert Chen
6e7d45caac Add a TRACE message when skipping a file to be restored 2019-05-22 12:03:21 -04:00
Gilbert Chen
8e9caea201 Retry on broken pipe in Azure backend 
Azure sometimes disconnect the connection randomly when uploading files.  The
returned error was 'broken pipe' but this error is wrapped deep in multiple
levels of errors so we have to check the error string instead.
 2019-05-07 22:35:51 -04:00
Gilbert Chen
458687d543 The cat command doesn't need to load the entire file into memory 
It can print out the chunk as soon as a chunk is retrieved.  This avoids
reconstructing the file in the memory which can be an issue with large files.
 2019-05-03 11:33:16 -04:00
Gilbert Chen
57a408a577 Rework the Backblaze B2 backend 
* All APIs include UploadFile are done via the call() function
* New retry mechanism limiting the maximum backoff each time to 1 minute
* Add an env var DUPLICACY_B2_RETRIES to specify the number of retries
* Handle special/unicode characters in repositor ids
* Allow a directory in a bucket to be used as the storage destination
 2019-04-30 23:31:57 -04:00
gilbertchen
e56efc1d3a Merge pull request #554 from arikorn/ask_b2_application_key 
Request B2 "Backblaze Account or Application ID"
 2019-04-27 10:55:12 -04:00
gilbertchen
bb58f42a37 Merge pull request #529 from turtleleo/patch-1 
Retry on 408 error from Google Drive (Update to duplicacy_gcdstorage.go)
 2019-04-27 10:53:26 -04:00
Gilbert Chen
4eb174cec5 Remove a few util functions that aren't necessary 2019-04-26 23:47:25 -04:00
gilbertchen
6fd3fbd568 Merge pull request #514 from a-s-z-home/filter_extension 
Filter extension: @ to include another file
 2019-04-26 21:56:42 -04:00
Gilbert Chen
a6fe3d785e Fixed a MoveFile bug in Wasabi when the storage is at the root of a bucket 
When the storage dir is empty, the destination path passed to the MOVE api starts
with a / which causes Wasabi to fail silently.
 2019-04-24 16:48:25 -04:00
Gilbert Chen
1da151f9d9 Add an additional lookup for a chunk that isn't in the chunk list 
A chunk not in the chunk list may actually exists in two scenarios:
* the chunk may be a special snapshot chunk that contains the chunk sequence,
  so it may be resurrected by the chunk downloader if it had been turned into
  a fossil before
* if the API to list all chunks doesn't return the complete list due to some
  bug

This additional lookup avoid reporting the missing chunk prematurely.
 2019-04-21 20:32:21 -04:00
Gilbert Chen
4b69c1162e Fix a memory issue that check -tabular uses too much memory with many revisions 
The call to GetSnapshotChunks in ShowStatisticsTabular sets keepChunkHashes to
true -- this can cause too much memory consumption with hundreds of revisions.
 2019-04-20 22:47:03 -04:00
Gilbert Chen
abcb4d75c1 Fixed a bug where filenames starting with i or e are mistakenly interpreted as regex 2019-04-07 22:43:36 -04:00
Ari Kornfeld
10d2058738 Request B2 "Backblaze Account or Application ID" (rather than "Account ID") 
fixes #539 (Duplicacy init for B2 storage still ask for account ID)
 2019-04-02 22:20:29 -07:00
Gilbert Chen
43a5ffe011 Fixed a bug where a wrong variable is used as the number of threads 2019-03-13 15:38:26 -04:00
Gilbert Chen
d16273fe2b Set the content length for upload 2019-03-04 15:34:32 -05:00
Jos
2eb8ea6094 Improve WebDAV performance 2019-03-01 19:41:00 +01:00
Philipp Bandow
a55ac1b7ad Add option to use a ssh key signed with a certificate to authenticate 2019-02-28 01:37:14 +01:00
Gilbert Chen
2b56d576c7 Fixed a webdav compatibility issue with rclone and other bugs 2019-02-26 14:00:02 -05:00
turtleleo
82c6c15f1c Update duplicacy_gcdstorage.go 
Add automatic retry on receiving error 408 (request timeout) from Google Drive.
 2019-01-16 13:12:46 -05:00
gilbertchen
bebd7c4b77 Merge pull request #495 from plasticrake/environment-variables 
Replace special characters in environment variable name with underscores
 2019-01-04 17:04:29 -05:00
gilbertchen
46376d82ed Merge pull request #489 from gilbertchen/sftp_retry 
Retry on EOF errors in the SFTP backend
 2019-01-04 13:53:44 -05:00
gilbertchen
31c25e98f7 Merge branch 'master' into master 2019-01-04 13:48:44 -05:00
gilbertchen
242db8377e Merge pull request #447 from s4y/patch-1 
Acknowledge malware/spam warnings from GCD
 2019-01-04 13:33:11 -05:00
Gilbert Chen
bb652d0a8c Add a Sync call before close when uploading a file to local storage 2019-01-03 12:44:50 -05:00
Michael Cook
4b9524bd43 go vet: unreachable code 2018-12-29 13:20:11 +01:00
Michael Cook
a782d42ad6 go vet: result of fmt.Errorf call not used 2018-12-29 13:20:10 +01:00
Michael Cook
0762c448c4 gofmt -s 2018-12-29 13:20:10 +01:00
Michael Cook
741644b575 spelling 2018-12-29 13:04:40 +01:00
a-s-z-home
df7487cc0b Merge remote-tracking branch 'origin/master' into filter_extension 2018-11-15 01:40:39 +01:00
Gilbert Chen
8aa67c8162 Support ssh private key files encrypted by passphrases 2018-11-09 14:17:56 -05:00
Gilbert Chen
53548a895f Add the \?\ prefix to all paths on Windows 2018-11-08 21:29:02 -05:00
a-s-z-home
5e8baab4ec - Reverted changes to exclude mechanism of .duplicacy directory. 2018-11-05 22:39:11 +01:00
a-s-z-home
aaebf4510c - Replaced static check for .duplicacy directory with usage of predefined filters. 
Do not "misuse" property nobackupFile to trigger this feature.
- Restructured ProcessFilterFile function and splitted it in smaller parts.
- Prepare usage of new filter syntax for arguments of restore command.
 2018-11-05 00:32:12 +01:00
a-s-z-home
96dd28995b Added an include mechanism for filter file. 
- Using @<filename>, you can now include other files. Relative paths are supported.
  This is useful, if you have several repositories with some different filters and a common filter base set.
 2018-11-03 20:39:03 +01:00
a-s-z-home
166f6e6266 Added string array helper functions Contains and Find. 2018-11-03 20:20:00 +01:00
a-s-z-home
86c89f43a0 Automatically exclude .duplicacy directory only, if nobackup_file is not 
set.
 2018-11-03 20:13:43 +01:00
Gilbert Chen
21b3d9e57f Padding size was incorrect -- didn't pad to multiples of 256 2018-11-03 11:42:03 -04:00
Gilbert Chen
244b797a1c Print the number of files if available in the snapshot file 2018-11-03 10:38:35 -04:00
Gilbert Chen
073292018c Don't show snapshots whose tags don't match the given one 2018-10-28 23:30:22 -04:00
Gilbert Chen
15f15aa2ca Show more statistics in the check command 2018-10-28 23:27:36 -04:00
Gilbert Chen
d8e13d8d85 Benchmark may incorrectly list the chunks directory when looking for previous temporary files 2018-10-22 09:11:15 -04:00
Gilbert Chen
bfb4b44c0a Optimizating restore to avoid reading newly created sparse file 2018-10-21 22:43:24 -04:00
Patrick Seal
a1efbe3b73 Add exclude_by_attribute preference 2018-09-21 21:35:40 -07:00
Patrick Seal
cce798ceac Replace special characters in environment variable name with underscores 2018-09-18 11:16:31 -07:00