On Friday 30 December 2005 03:52, Angus MacGyver wrote:
>Ok, this seems like a simple question, tho I cannot find an answer
>anywhere...
>
>
>Take one DDS-4 tape drive, in this case a Compaq TSL-10000 which is a
>DDS-4 drive complete with a 8 tape cassette autoloader.
>(basically a re-badged Sony 10000 autoloader)
>
>These devices are supposed to be 20G/40G.
>
>Granted, I am probably never gonna get 40G on one of these tapes, and
> i can live with that....
>....but surely I should be able to get 20G...
>
>Setup is as follows.
>
>On board SCSI Card, external TSL-10000 autoloader.
>Cabling is perfect, and terminated correctly (I do a lot of work with
>SCSI at home and at work, I know what I'm doing there)
>
>Stick a DDS2 tape in the drive, and do a backup using dump, I get
>DUMP: Volume 1 3924430 blocks (3832.45MB)
>
>OK, fairly reasonable I guess, would prefer closer to the actual
> value of 4G, but...
>
>Turn the compression on via mt,
>[root@hostname ~]# mt -f /dev/st0 compression 1
>
>And do the same thing to the SAME TAPE I just dumped to
>and ....
>
>DUMP: Volume 2 3543529 blocks (3460.48MB)
>
>
>That is less data, on a tape that I have just put more on..
>
>I was
>1) Expecting more data,
>... and
>2) Expecting it to go faster...
> (i.e. send more data to the drive, the drive compresses it and put
> to tape)
>
>At least that is how the specifications for these drives (and all
>DAT/DDS drives is written)
>
>
>When I use DDS-4 tapes, basically the same thing happens.
>
>With compression turned off, I get 19459MB to tape, with it on, I get
>18559MB...
>
>I don't mind hugely that I get slightly less on a tape that spec'ed
>(i.e. 19.5 GB compared to the spec'ed 20GB) due to the manufacturer's
>statement of 20G means 20,000,000,000 not 20*1024*1024*1024......
>
>..... but what i do mind is that whilst I understand that the
>compression is never going to be 2:1 as the
>salesman^H^H^H^H^H^H^H^HManufacturer states, I do mind that I can get
>LESS on with the compression turned on!!...
>
>Google has been somewhat less than helpful, and I have spent a good
> few days at this.
>
>man mt, man st really all don't shed light on the "issue"
>
>That actual data are the directories data/ and mail/, and before
> anyone asks, the tar.gz's below were created to proove that the data
> did at least compress by some amount (I know if you compress already
> compressed data it can get bigger)
>
>[root@hostname data_backups]# du -sh *
>19G data
>17G data.tar.gz
>1.9G mail
>494M mail.tar.gz
>
>There is only actually 21G in the directories of data/ and mail/ and
> I kinda think i should be able to get 21G on to a tape with
> compression turned on...
Here is a classic example of salesspeak not making any sense in the
real world.
The problems you are seeing are those we see by newbies all the time on
the amanda mailing lists, amanda being the only backup worth wasting
the space on the drive for IMO, but I'm admittedly biased.
Here it is in simple terms. Already compressed material, compressed by
a decent compressor, will overwhelm the basic rll encoding of the
hardware compressor AND IT WILL GROW.
So ypou have your choice. Don't feed the hardware compressor already
compressed material, or put up with the apparent lack of honesty on
the part of the drive peddlars. But they aren't being dishonest in
most cases, but remember, tho those folks a megabyte is 1,000,000
bytes, to use its a somewhat larger figure based on 1024 bytes per
kilobyte.
Over on the amanda lists, we discourage the use of the hardware
compressors in tape drives for one good reason.
Amanda keeps track of the bytes fed down the cable after any optional
software compression has been done, and without the hardware
compressor enabled, amanda can know within 1% or less how much tape is
left. With hardware smunching on, one has to reduce the estimated
tape size by 10-15% to help keep amanda from hitting the EOT of the
tape.
I quite regularly put 11 or more GB of raw data on DDS2 tapes when I
was using tape, but the tape format I could afford was also a very
undependable format, and I'd place the current crop of DDS4 drives in
this same category, so I'm now using virtual tapes on a large hd and
having .000001% of the trouble I had with tape. Until that 200GB
drive dies there will not be any errors not of MY doing.
But the fact that software compression can beat hardware hands down in
anything but speed is best demoed by a snip from a backup report from
amanda it emails me every night, running the most recent 2.4.5p1
snapshot, about 2 weeks old now:
-------------
STATISTICS:
Total Full Incr.
-------- -------- --------
Estimate Time (hrs:min) 0:31
Run Time (hrs:min) 2:05
Dump Time (hrs:min) 2:44 2:33 0:11
Output Size (meg) 6217.8 5618.3 599.4
Original Size (meg) 16706.2 15137.1 1569.2
Avg Compressed Size (%) 35.8 35.6 37.7
(level:#disks ...)
Filesystems Dumped 44 11 33 (1:29 2:2 3:2)
Avg Dump Rate (k/s) 648.1 627.7 931.6
Tape Time (hrs:min) 0:04 0:03 0:00
Tape Size (meg) 6217.8 5618.4 599.4
Tape Used (%) 73.7 66.2 7.4
(level:#disks ...)
Filesystems Taped 44 11 33 (1:29 2:2 3:2)
Avg Tp Write Rate (k/s) 28871.2 28897.7 28624.3
-----------------
This is from a mixture of uncompressed and compressed src data, with
the archive stuffs (the rpms/tar.gz etc) not being compressed because
they already are, and normal data which can often be squeezed down to
less than 10% of its original size.
>Whilst I am here, i can also say that it isn't the hardware, as i
> have another tape drive, a DDS-5 (36/72) that does exactly the same
> thing with these tapes..
Correct, its an artifact of trying to compress already compressed data.
--
Cheers, Gene
People having trouble with vz bouncing email to me should add the word
'online' between the 'verizon', and the dot which bypasses vz's
stupid bounce rules. I do use spamassassin too. :-)
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2005 by Maurice Eugene Heskett, all rights reserved.
