On Friday 17 Feb 2006 08:00, Andrew wrote: > I just need to transfer a couple hundred thousand files off a **REALLY** > slow server to a faster server and to be able to document that file x on > slow server is identical to file x on faster server. It is not a forensics > problem per se -- but I do need to prove/document the process -- so I'm > thinking md5sums. > > /** begin simplification **/ > > Procedure 1: The slow option > > [slow server] calc md5sums > list && nc list > faster_server > [slow server] rsync files to faster@server over a 10T crossover cable > [faster server] check md5sums against list > This means you have to boot the OS on the slower machine. NO. The suspicious minded will say "What if the person who controlled that machine left a software bomb thats ready to go off if the machine is booted?" Script pseudocode 1] Start program when machine boots 2] wait 10 minutes 3] if someone with the right permissions doesn't notice me and kill me first, delete all those suspicious files. I have something similar on a laptop designed to be used by a Partner at a law firm. Script runs even in level 1. If anyone other than her logs in, OR she doesn't log in within 10 minutes after the system boots, her entire home directory gets encrypted with gpg. If she forgets, tough .. I have her gpg keyring backed up :p, she has to come to me to get her data back. Not perfect, I know, since I could strip the entire home partition in 30 secs (as I'm sure most of the posters here could do), but it does stop the data being at risk from the casual thief who is more likely to nick the laptop, turn it on and go browsing around. > Procedure 2: The faster option (repeated for each hard drive) > > [slow server] halt > pull hard drives off slow_server and place in removable tray on > faster_server > [faster server] dd if= /dev/slowdrive ibs=4096 conv=notruc,noerror > > slowdrive.img > [faster server] mount slowdrive.img && calc md5sums > list > [faster server] rsync /mnt/slowdrive/ /newdir/ > [faster server] cd /newdir/ && check md5sums against list > > /** end simplification **/ > Better. Make sure that the physical move from one machine to the next is documented and witnessed, preferrably by that suspicious guy who otherwise would be the one asking all the questions <g> > -The slow server is 6 years old -- no cdrom, no floppy :-) > -Data corruption is totally unacceptable. Question : How much of this data is there? the entire disk? Why not install a CD(1) and CD/DVD writer(2), boot from a live-CD in (1), mount the hdd read-only and write the data to (2)? You could then run the checksums against the individual files on the old drive, write the checksum to another CD, and run the checksum comparisons against the CD copies of the data. This would mean that the copies, if taken onto ReadOnly media, are automatically protected from being amended during the checking. You coul do the same procedure mounting the old data drives (read only of course!) in a newer system already equipped with a CD writer, but with NO OS drives ... run the new system from a live-CD. This way the OS used to make the copies (the liveCD) is also available to be checked, so you can convince people that you weren't running anything in the background to alter the data. > Questions: > 1) Theory question: will Option 2 produce identical results to Option 1 > only quicker ?? No idea. Comments above refer to procedure, not to scripting. > 2) Practical question: is Option 2 "too risky" given the age of the drives > ?? I have this fear of possibly harming a hard drive no matter how careful > .. is my fear justified or baseless ?? Put the old drives on a cable on their own, do not change the jumper status. ie connect a slave drive to the slave end of a drive cable with no master. etc etc. I'm always moving drives. I have a drawer full of old drives. I tend to use them as "I might need this data one day, but probably not" storage. I've not, so far, had a problem. Any time I upgrade a system, I copy all the user data to one of these drives, label it and stuff the old drive away. Thats in addition to all the normal backup and restore procedures. Its just a safety net, but it has come in useful several times ... I hadn't heard from an old friend for years, had deleted all her contact details from all 'live' systems, then was told she was back in the country. Dug out an old drive, got her mobile phone number, and took her out for a meal last night in fact! > 3) The better idea is to ... I'm always up for the better idea !!! Everything depends on just what this procedure is is designed to do, and whether your setup will satisfy other people, if there are others involved. 1] Run a checksum on the entire drive, mirror it to a new hdd/CD (not a mounted partition), then run a checksum on the mirror image. Compare. 2] If/when you have to mount the drive with the old data on it, mount it read-only. Log all commands relating to that data automatically, so others can 'see' each step. > Thanks again and best wishes to everyone. > > Andrew -- Tony Dietrich PGP Key at hkp://wwwkeys.pgp.net ID D4055CE3
