On Sun, 2007-09-02 at 21:23 +0100, Ian Malone wrote: > Karl Larsen wrote: > > > I will wear it and just read info dd for about the 4th time in 5 > > days. The first time I read it I was not aware that dd sends Everything > > including the file system. The info dd does not say this. > > > > "Copy a file, converting and formatting according to the operands." > You applied it to a block device. Maybe it would have been worth > knowing what you were doing. > > > I tried to copy this f7 which is about 8 Gbytes to another partition > > that was 10 Gbytes. The transfer failed because the 8 Gbyte f7 was from > > a 40 Gbyte partition :-) > > If you read everything in info dd you will not see this listed. > > > > 10 < 40. Not sure what more needs to be said here. > > > So it may be time to get the author of the dd info to add something > > about large copies and the use of the Rescue CD. > > > > You're serious? > > -- > imalone > Hi, Ian, This is one of the aspects of computers that someone with experience understands and someone else may not. Karl, The disk contains not only the partition table, files and data, but also many blocks of data that contain information about how the files are stored, the superblocks. These are linked. If one changes, they all change. The dd utility looks at the disk and finds its size, then copys that to the output file system. This will include the superblocks and everything else that is tied to the disk structure that is visible to the outside world. There is also data on the disk that is not visible to the outside world that is used by the interface, but dd won't see that, and won't copy it (doesn't have access). The disk structure then is spread out over the full size of the disk. Moreover, when you edit or update a file, it is not generally done "in place", but rather a new file is created, then the old one is erased. This means a disk can have voids in its cylinders (not a bad thing, because this process prevents excessive writes to the same areas of the disk to some extent, thus prolonging the life of the disk slightly). As a result, the "clone tools" such as dd and a few others that I used to use, will copy the disk literally. As long as the disks are the same basic physical structure this works. But if the size is different, the cylinder size is different, or the sector size is different then it will not work well, although it may complete successfully. If the cylinders have more sectors, some will be left blank, and due to the design of the superblocks, not accessible after the clone is complete. If the cylinders have fewer sectors (not a likely scenario today, but used to happen), then the clone would overwrite on successive sectors past the physical number unless it detected the error. If the disk is faster or slower in rotation, it can also cause problems, as can relative bandwidth of the heads, but generally the controller will attempt to set this correctly. Thus dd has evolved over the years to detect most of the possible errors and prompt you to not continue. However, since dd is also a catastrophe tool, it will proceed if told to do so, so that someone in a bad situation may be able to clone the disk to a larger disk and hopefully recover their data. It has been pointed out in this thread that this is also sometimes the process used by computer forensics folks to clone the disk to avoid tampering with its contents, and then work only on the clone. This is one of those tools that can do a lot of damage if used unwisely, and so as many folks here told you, using the same identical hardware is critical to ensuring success in the most basic manner. Another issue that arises in dynamic systems that are interrupt and demand driven like all modern multithreaded systems is the possibility of file modifications. This would result in a "snap shot" of a file system in transition, which could possibly cause catastrophic failure should the cloned system be used to boot. One of the responders I felt gave a really good example of a book being written as it was copied. And, no, the man and info pages do not detail exactly what dd does or the potential interactions. However I do believe that they say to not perform the operation on mounted systems, and that the systems should be the same or at least the same size. In short, man pages are abbreviated to "just the facts, Ma'am" as Joe Friday used to say. Therefore using commands just based upon man one should always follow very carefully every argument, every precaution, and not do anything else. Otherwise, you need more information. This is true of most every *nix command, as most have evolved over the years to much greater capability than was originally envisioned. The original mantra was for each command to do one thing, but do it well and be capable of chaining to other command through pipes or I/O redirection to accomplish complex tasks. However as systems have grown, our ability to segment the complex tasks is being overwhelmed, so commands get more and more capabilities, necessitating manuals for some of the commands, and more understanding of the OS and system functions and processes. The command dd falls into this category, but so do most other commands. I do not believe that man should be your only source, but I do agree that one should read and follow the man recommendations religiously. This applies to old hands as well as new. I have seen "we've always done it this way" go too far wrong in many cases. Be careful. And by the way, Karl, you did the right thing. Write down what you did. Get it clarified, and improve it, whether for yourself or for wider dissemination. That is the true spirit of open source. The gentleman here said make a contribution to man. I believe you have made a contribution, perhaps not to man, but to several folks out there who did not understand what was happening or why. I hope I have helped here. But if not... at least I tried. Good luck to you both. Regards, Les H
