Reliable file transfer?

[Alex LeDonne]

On 9/12/06, Victor Odhner <vodhner at cox.net> wrote:
> Is there a file transfer protocol that will verify
> the data as it comes across, and retry any failed
> blocks?  Apparently sftp is not doing this.
>
> I have a 1.5 GB .bz2 file on one computer that I am
> trying to transfer to another nearby, via a local
> router.  I don't have DVD drives.  (Both boxes have
> USB ports, would that work better?)
>
> In two attempts (30 minutes each), the number of
> bytes received at least has been right.  But the
> files arrive broken (according to bunzip2 -t) and
> md5sum has given non-matching results.
>
> The two wrong results have differed from each other,
> so I will try once again . . .  Each try takes
> 30+ minutes.
>
> Thanks,
>
> Vic

To reduce the time for retries, you could use split on the original,
get the md5sum for each part, transfer the parts, verify them, and
then cat them back together.

To reduce the overhead, you could NFS (or SMB) mount the dir with the
file on the target box, then just use cp to copy. Given the local
router, is encrypting the transfer necessary? Does the source box have
an ftp or http server running? If so you could transfer the file that
way.

I know none of these ideas actually do the block-by-block verify that
you asked about, but maybe they'll help nonetheless... at least with
speed.

-Alex