Functions for producing signatures and deltas of directories
Note that the main processes of this module have two parts. In the first, the signature or delta is constructed of a ROPath iterator. In the second, the ROPath iterator is put into tar block form.
- class duplicity.diffdir.DeltaTarBlockIter(input_iter)
TarBlockIter that yields parts of a deltatar file
Unlike SigTarBlockIter, the argument to __init__ is a delta_path_iter, so the delta information has already been calculated.
Return pair (next data block, boolean last data block)
Get a tarblock from delta_ropath
Return next volume in multivol diff or snapshot
- exception duplicity.diffdir.DiffDirException
- duplicity.diffdir.DirDelta(path_iter, dirsig_fileobj_list)
Produce tarblock diff given dirsig_fileobj_list and pathiter
dirsig_fileobj_list should either be a tar fileobj or a list of those, sorted so the most recent is last.
- duplicity.diffdir.DirDelta_WriteSig(path_iter, sig_infp_list, newsig_outfp)
Like DirDelta but also write signature into sig_fileobj
Like DirDelta, sig_infp_list can be a tar fileobj or a sorted list of those. A signature will only be written to newsig_outfp if it is different from (the combined) sig_infp_list.
Return a tarblock full backup of items in path_iter
A full backup is just a diff starting from nothing (it may be less elegant than using a standard tar file, but we can be sure that it will be easy to split up the tar and make the volumes the same sizes).
- duplicity.diffdir.DirFull_WriteSig(path_iter, sig_outfp)
Return full backup like above, but also write signature to sig_outfp
Alias for SigTarBlockIter below
- class duplicity.diffdir.DummyBlockIter(input_iter)
TarBlockIter that does no file reading
Get a fake tarblock from delta_ropath
- class duplicity.diffdir.FileWithReadCounter(infile)
File-like object which also computes amount read as it is read
- class duplicity.diffdir.FileWithSignature(infile, callback, filelen, *extra_args)
File-like object which also computes signature as it is read
- __init__(infile, callback, filelen, *extra_args)
The object will act like infile, but whenever it is read it add infile’s data to a SigGenerator object. When the file has been read to the end the callback will be called with the calculated signature, and any extra_args if given.
filelen is used to calculate the block size of the signature.
- blocksize = 32768
- class duplicity.diffdir.SigTarBlockIter(input_iter)
TarBlockIter that yields blocks of a signature tar from path_iter
Return associated signature TarBlock from path
- class duplicity.diffdir.TarBlock(index, data)
Contain information to add next file to tar
- __init__(index, data)
TarBlock initializer - just store data
- class duplicity.diffdir.TarBlockIter(input_iter)
A bit like an iterator, yield tar blocks given input iterator
Unlike an iterator, however, control over the maximum size of a tarblock is available by passing an argument to next(). Also the get_footer() is available.
Return closing string for tarfile, reset offset
Return index of last tarblock, or None if no previous index
Turn next value of input_iter into a TarBlock
Get more tarblocks
If processing val above would produce more than one TarBlock, get the rest of them by calling process_continue.
Next time next() is called, we will return data instead of processing
Retrieve index remembered with remember_next_index
When called, remember the index of the next block iterated
- tarinfo2tarblock(index, tarinfo, file_data=b'')
Make tarblock out of tarinfo and file data
- duplicity.diffdir.collate2iters(riter1, riter2)
Collate two iterators.
The elements yielded by each iterator must be have an index variable, and this function returns pairs (elem1, elem2), (elem1, None), or (None, elem2) two elements in a pair will have the same index, and earlier indicies are yielded later than later indicies.
Produce new iterator by combining the iterators in path_iter_list
This new iter will iterate every path that is in path_iter_list in order of increasing index. If multiple iterators in path_iter_list yield paths with the same index, combine_path_iters will discard all paths but the one yielded by the last path_iter.
This is used to combine signature iters, as the output will be a full up-to-date signature iter.
- duplicity.diffdir.delta_iter_error_handler(exc, new_path, sig_path, sig_tar=None)
Called by get_delta_iter, report error in getting delta
Return a reasonable block size to use on files of length file_len
If the block size is too big, deltas will be bigger than is necessary. If the block size is too small, making deltas and patching can take a really long time.
Return path iter combining signatures in list of open sig files
- duplicity.diffdir.get_delta_iter(new_iter, sig_iter, sig_fileobj=None)
Generate delta iter from new Path iter and sig Path iter.
For each delta path of regular file type, path.difftype with be set to “snapshot”, “diff”. sig_iter will probably iterate ROPaths instead of Paths.
If sig_fileobj is not None, will also write signatures to sig_fileobj.
- duplicity.diffdir.get_delta_path(new_path, sig_path, sigTarFile=None)
Return new delta_path which, when read, writes sig to sig_fileobj, if sigTarFile is not None
- duplicity.diffdir.log_delta_path(delta_path, new_path=None, stats=None)
Look at delta path and log delta. Add stats if new_path is set
Convert signature tar file object open for reading into path iter
- duplicity.diffdir.write_block_iter(block_iter, out_obj)
Write block_iter to filename, path, or file object