Re: DAT/DDS tape backups...

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Friday 30 December 2005 03:52, Angus MacGyver wrote:
>Ok, this seems like a simple question, tho I cannot find an answer
>anywhere...
>
>
>Take one DDS-4 tape drive, in this case a Compaq TSL-10000 which is a
>DDS-4 drive complete with a 8 tape cassette autoloader.
>(basically a re-badged Sony 10000 autoloader)
>
>These devices are supposed to be 20G/40G.
>
>Granted, I am probably never gonna get 40G on one of these tapes, and
> i can live with that....
>....but surely I should be able to get 20G...
>
>Setup is as follows.
>
>On board SCSI Card, external TSL-10000 autoloader.
>Cabling is perfect, and terminated correctly (I do a lot of work with
>SCSI at home and at work, I know what I'm doing there)
>
>Stick a DDS2 tape in the drive, and do a backup using dump, I get
>DUMP: Volume 1 3924430 blocks (3832.45MB)
>
>OK, fairly reasonable I guess, would prefer closer to the actual
> value of 4G, but...
>
>Turn the compression on via mt,
>[root@hostname ~]# mt -f /dev/st0 compression 1
>
>And do the same thing to the SAME TAPE I just dumped to
>and ....
>
>DUMP: Volume 2 3543529 blocks (3460.48MB)
>
>
>That is less data, on a tape that I have just put more on..
>
>I was
>1) Expecting more data,
>... and
>2) Expecting it to go faster...
> (i.e. send more data to the drive, the drive compresses it and put
> to tape)
>
>At least that is how the specifications for these drives (and all
>DAT/DDS drives is written)
>
>
>When I use DDS-4 tapes, basically the same thing happens.
>
>With compression turned off, I get 19459MB to tape, with it on, I get
>18559MB...
>
>I don't mind hugely that I get slightly less on a tape that spec'ed
>(i.e. 19.5 GB compared to the spec'ed 20GB) due to the manufacturer's
>statement of 20G means 20,000,000,000 not 20*1024*1024*1024......
>
>..... but what i do mind is that whilst I understand that the
>compression is never going to be 2:1 as the
>salesman^H^H^H^H^H^H^H^HManufacturer states, I do mind that I can get
>LESS on with the compression turned on!!...
>
>Google has been somewhat less than helpful, and I have spent a good
> few days at this.
>
>man mt, man st really all don't shed light on the "issue"
>
>That actual data are the directories data/ and mail/, and before
> anyone asks, the tar.gz's below were created to proove that the data
> did at least compress by some amount (I know if you compress already
> compressed data it can get bigger)
>
>[root@hostname data_backups]# du -sh *
>19G     data
>17G     data.tar.gz
>1.9G    mail
>494M    mail.tar.gz
>
>There is only actually 21G in the directories of data/ and mail/ and
> I kinda think i should be able to get 21G on to a tape with
> compression turned on...

Here is a classic example of salesspeak not making any sense in the 
real world.

The problems you are seeing are those we see by newbies all the time on 
the amanda mailing lists, amanda being the only backup worth wasting 
the space on the drive for IMO, but I'm admittedly biased.

Here it is in simple terms.  Already compressed material, compressed by 
a decent compressor, will overwhelm the basic rll encoding of the 
hardware compressor AND IT WILL GROW.

So ypou have your choice.  Don't feed the hardware compressor already 
compressed material, or put up with the apparent lack of honesty on 
the part of the drive peddlars.  But they aren't being dishonest in 
most cases, but remember, tho those folks a megabyte is 1,000,000 
bytes, to use its a somewhat larger figure based on 1024 bytes per 
kilobyte.

Over on the amanda lists, we discourage the use of the hardware 
compressors in tape drives for one good reason.

Amanda keeps track of the bytes fed down the cable after any optional 
software compression has been done, and without the hardware 
compressor enabled, amanda can know within 1% or less how much tape is 
left.  With hardware smunching on, one has to reduce the estimated 
tape size by 10-15% to help keep amanda from hitting the EOT of the 
tape.

I quite regularly put 11 or more GB of raw data on DDS2 tapes when I 
was using tape, but the tape format I could afford was also a very 
undependable format, and I'd place the current crop of DDS4 drives in 
this same category, so I'm now using virtual tapes on a large hd and 
having .000001% of the trouble I had with tape.  Until that 200GB 
drive dies there will not be any errors not of MY doing.

But the fact that software compression can beat hardware hands down in 
anything but speed is best demoed by a snip from a backup report from 
amanda it emails me every night, running the most recent 2.4.5p1 
snapshot, about 2 weeks old now:
-------------
STATISTICS:
                          Total       Full      Incr.
                        --------   --------   --------
Estimate Time (hrs:min)    0:31
Run Time (hrs:min)         2:05
Dump Time (hrs:min)        2:44       2:33       0:11
Output Size (meg)        6217.8     5618.3      599.4
Original Size (meg)     16706.2    15137.1     1569.2
Avg Compressed Size (%)    35.8       35.6       37.7   
(level:#disks ...)
Filesystems Dumped           44         11         33   (1:29 2:2 3:2)
Avg Dump Rate (k/s)       648.1      627.7      931.6

Tape Time (hrs:min)        0:04       0:03       0:00
Tape Size (meg)          6217.8     5618.4      599.4
Tape Used (%)              73.7       66.2        7.4   
(level:#disks ...)
Filesystems Taped            44         11         33   (1:29 2:2 3:2)
Avg Tp Write Rate (k/s) 28871.2    28897.7    28624.3
-----------------
This is from a mixture of uncompressed and compressed src data, with 
the archive stuffs (the rpms/tar.gz etc) not being compressed because 
they already are, and normal data which can often be squeezed down to 
less than 10% of its original size.

>Whilst I am here, i can also say that it isn't the hardware, as i
> have another tape drive, a DDS-5 (36/72) that does exactly the same
> thing with these tapes..

Correct, its an artifact of trying to compress already compressed data.

-- 
Cheers, Gene
People having trouble with vz bouncing email to me should add the word
'online' between the 'verizon', and the dot which bypasses vz's
stupid bounce rules.  I do use spamassassin too. :-)
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2005 by Maurice Eugene Heskett, all rights reserved.


[Index of Archives]     [Current Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Yosemite News]     [Yosemite Photos]     [KDE Users]     [Fedora Tools]     [Fedora Docs]

  Powered by Linux