[Gllug] DD Ext3 Move

[Christian Smith]

On Tue, 7 Mar 2006, Daniel P. Berrange wrote:

>On Tue, Mar 07, 2006 at 02:52:30PM +0000, Christian Smith wrote:
>> On Wed, 1 Mar 2006, Daniel P. Berrange wrote:
>>
>> >On Wed, Mar 01, 2006 at 08:41:37PM +0000, Nix wrote:
>> >> On Tue, 28 Feb 2006, Bruce Richardson gibbered uncontrollably:
>> >> > On Wed, Feb 08, 2006 at 05:37:57PM +0000, Nix wrote:
>> >> >>
>> >> >> I used to move filesystems around via a pair of tars to and from stdout
>> >> >> connected via ssh, but for my most recent major move I just used `cp
>> >> >> -av' as root.
>> >> >
>> >> > Does that preserve hard links properly?  Doesn't always matter,
>> >> > sometimes really does.
>> >>
>> >> Yes, it does. (It doesn't preserve sparseness perfectly but that's
>> >> impossible to do reliably in any case. It tries as well as it can.)
>> >
>> >Impossible with current support in the Linux kernel, but I'm told that
>> >Solaris has an ioctl() command which lets an app query the sparse regions
>> >within a file.
>>
>>
>> Not impossible.
>
>Ok, s/impossible/practical/.

Did  you mean s/impossible/impractical/?

Perfectly practical for 99% of data files.


>
>> The app has to read the file in order to write it back out. While in
>> memory, just scan the current block, and if it's all zeros, simply don't
>> write it and seek to the next block. The benefits being less data written,
>> the possibility of making sparse a previously non-sparse file, and no need
>> for special system calls or ioctls.
>
>I've seen multiple TB sparse files on 64-bit systems. Reading several
>terrabytes of data to determine where the zeros are is crazy. Of course
>the app creating the multi-TB sparse file is pretty crazy too, but since
>the kernel knows the allocation pattern it ought not to be too difficult
>to let userspace know too, to avoid having to byte scan.



Most likely database data files? If so, then using the database dump tools
is probably a better solution anyway, which should result in non-sparse
files.

But for most people, multi TB sparse files are not a real concern.

Also, that multi TB read will mostly be memory->memory reads, so not
nearly as bad as reading from disk. At current memory rates, you could
probably read a multi TB sparse file in the order of one or two hours. OK,
not as fast as knowing where the fragments of data are, but not
impractical for an overnight backup.


>
>Dan.
>

Christian

-- 
    /"\
    \ /    ASCII RIBBON CAMPAIGN - AGAINST HTML MAIL
     X                           - AGAINST MS ATTACHMENTS
    / \
-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug