----- Original Message -----
From: "John Austin" <ja@xxxxxxxxxx>
Newsgroups: gmane.linux.redhat.fedora.general
To: "For users of Fedora" <fedora-list@xxxxxxxxxx>
Sent: Wednesday, October 24, 2007 6:15 AM
Subject: Re: USB drive on server?
On Tue, 2007-10-23 at 16:32 -0700, Ted Marshall wrote:
From: "John Austin" <ja@xxxxxxxxxx>
Newsgroups: gmane.linux.redhat.fedora.general
To: <tim@xxxxxxxxxxxxxxxxxxxxxx>; "For users of Fedora"
<fedora-list@xxxxxxxxxx>
Sent: Tuesday, October 23, 2007 9:40 AM
Subject: Re: USB drive on server?
> Dup from other thread
>
>> Over 10 year life time
>
> http://www.corsairmemory.com/_faq/FAQ_flash_drive_wear_leveling.pdf
The problem with the statistics given for Dynamic Wear Leveling is that
the
flash controller must be able to know which blocks are not in use by the
filesystem. AFAIK (and this is very difficult to verify one way or the
other), consumer grade devices (at least) are only aware of FAT
filesystems
and only if there is only one partition on the device. If you use ext2/3
or
have multiple partitions (both typical for Linux) the controller cannot
track blocks being freed by the filesystem and so any block ever written
by
the host will looks busy to the flash controller. Therefore, writes will
only rotate through the (small) pool of extra blocks maintained by the
flash
controller.
This is where I hope you are wrong !!!!!!!! See below
Also, AFAIK, few if any consumer-grade and, I suspect, few
professional-grade flash devices do Static Wear Leveling.
The result is that the flash drive will burn out much faster than the
statistics quoted. How much faster, I am unable to compute. It may
still
be adequate for your needs. I don't know.
You have made me have a very careful read of the the Corsair info
and to query my understanding!!
Assuming the Corsair info true then my understanding is as follows
The Flash Voyager GT uses Dynamic Wear Levelling
I am not convinced that it knows about VFAT, NTFS, ext2 ...
The stick does not know about which is static data and which is dynamic,
all its knows is that if it is written to then it has to find
somewhere to put it.
The Corsair info suggests that there is more memory on the stick than
you can get at, See Fig 3, I do not know if this is true.
The stick does not need to know which bits are freed by the
operating system, only which bits have been freed by the
wear levelling look up table.
When the operating system deletes a file then it will free up
a certain number of cylinders/tracks/sectors as "it" understands them.
At a later time the operating system will write to those
cylinders/tracks/sectors again.
At that time the stick will remap them to different locations on the
stick,
freeing up the previously used physical memory.
My F7 stick layout is shown below.
I conclude that there is approx 6GB of "Static" data and 2GB of
Dynamic data (Free Space as far as Linux is concerned).
Thus if there is 2GB "free" as understood by the operating system then
this
is "almost" equivalent to 2GB being "free" to be mapped by the stick.
The bit that gets the hammering will be the superblocks - every time
the inode access time is updated for either read or write.
The superblock data will be moving around on the stick as
a result of the wear levelling, however the wear levelling
will only be applied over 2GB instead of 8GB.
If I remember correctly then the superblock (or hopefully only part of it)
will be written every 30 seconds.
http://www.storagesearch.com/siliconsys-art1.html
http://www.stec-inc.com/downloads/AN-0702_STEC_SMALL_CARDS_WEAR_LEVELING_LIFETIME_CALCULATOR.pdf
both give a formulae to compute the lifetime of wear levelled devices.
Filling in the first as follows gives a very large number of years!!
The 100,000 write cycle figure comes from the Corsair info
A bit of superblock = 10kB ????????
[(8000 - 6000)*100,000*(1 - 0)]/[(0.01*2 writes/min)*525,600]
= 19,026 years.
I conclude:
You should NEVER use the stick for long periods when it is
close to full as the wear levelling will only be able
to act over the remaining memory available at that time
and will become ineffective.
I would be very grateful for feed back on this !!
I still have a 10 year warranty to fall back on !!
As an aside
Is it a good idea to run "/" mounted without "atime" ??
John
[root@corsair ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda2 7549152 4869528 2296148 68% /
/dev/sda1 41053 19301 19632 50% /boot
tmpfs 1684780 0 1684780 0% /dev/shm
[root@corsair ~]#
Disk /dev/sda: 8287 MB, 8287944704 bytes
241 heads, 32 sectors/track, 2098 cylinders
Units = cylinders of 7712 * 512 = 3948544 bytes
Device Boot Start End Blocks Id System
/dev/sda1 1 11 42400 83 Linux
/dev/sda2 * 12 2000 7669584 83 Linux
/dev/sda3 2001 2098 377888 82 Linux swap /
Solaris
--
fedora-list mailing list
fedora-list@xxxxxxxxxx
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
Blocks that have been freed in the filesystem are still static data as far
as the flash controller is concerned, unless the controller can understand
the FS metadata. Yes, when they are reallocated by the FS, the controller
will be able to move that to another free flash block, freeing the old
block. However, as long as they just sit unused by the FS, they are
unavailable to the flash controller for wear leveling unless "static" wear
leveling is used. As I see it, the worst case is where the filesystem has
touched all the blocks that the controller makes visible and then one
single-block file (or the superblock) is constantly rewritten in place. The
controller will only be able to wear level between that block and its
spare-block pool.
So the question is "how big is the spare-block pool"?
http://www.sandisk.com/Assets/File/OEM/WhitePapersAndBrochures/RS-MMC/WPaperWearLevelv1.0.pdf
was published by Sandisk 4 years ago. Here's an early paragraph:
"Each memory chip is divided into blocks. A block is an array of memory
cells organized as
sectors. The number of blocks and sectors vary from product to product. The
minimum unit for a
write or read operation is a page (or sector). The minimum unit for an erase
operation is a block.
Physical blocks are logically grouped into zones. For the current
technology, a typical zone size is
4 MB. However, this may change from product to product. Wear leveling is
done within a zone.
The current firmware does not spread the wear across the capacity of the
card. Each zone has
about 3% additional "spare blocks" beyond what is assigned to meet the
logical capacity of the
flash card. This group of blocks is commonly referred to as the "Erase Pool"."
On the other hand, your Corsair PDF talks of a 2GB device having a write
page size of 2KB, an erase block size of 128KB (64 pages) and a leveling
management unit of 128MB (1024 blocks). Unfortunately, it does not tell us
how many spare blocks are in a leveling unit. I'll assume the 3% from
Sandisk and this gives us about 32 spare blocks.
Next, we have to ask what happens if only one page is changes in the block?
I'm going to assume the worst case that this causes the whole block to be
moved to a new block and the old one freed. Corsair states that they use
SLC technology for 100,000 erase cycle life instead of MLC with 10,000 so I
will use this assumption. (I have read specifications for other devices for
10,000 cycle lifetime so I'd but that the cheaper devices d ouse MLC).
Ok, so assume that the superblock does get rewritten every 30 seconds (I do
not know this for a fact). The lifetime of the wear management unit
containing the superblock is 32 * 100000 writes or 3 years. OK, that may be
ok for you. On the other hand, if the device uses MLC technology, devide by
10 and you only get 3.6 months!!
Now it may be that some devices use better algorithms and will give you a
longer life. (I would expect that a US$5000 "sold-state disk" will have
implemented things better and would have a much longer life.) On the other
hand, if you run out to Fry's and buy a $10 1GB USB stick, don't expect it
to last vey long.
One problem I've seen is that no manufacturer gives you all of the
information you need. Also, what is true for one device may not be true for
another.
It seems to me that one way to extend the life is to use ext3 with a large
"ordered" journal that will spread out the writes to the superblock
throughout the journal (I assume the superblock is journaled, anyone know
for sure?). Also, turn off atime on the filesystem.
What we need is a version of JFFS that can be used on an external device.
Assume that the device does not do wear leveling and do it in the OS.
