Re: [Linux-usb-users] USB storage: corrupted data transfers

[Alan Stern <stern@rowland.harvard.edu>]

On Sat, 10 Nov 2007, Mark Lord wrote:

> Something may be broken in USB / usb-storage land.
> 
> I've got a 2GB USB stick here.
> I want to copy it to an image file on my hard drive:
> 
>     cat /dev/sdb > usbkey.image1
> 
> Make a second copy, with or without unplugging/replugging the stick:
> 
>     cat /dev/sdb > usbkey.image2
> 
> After doing this, the two copies *differ*.
> 
> So I wrote a little program to compare them and determine *how* they differ,
> and the result is very interesting.
> 
> Periodically, or or the other file has 8192 *zero* bytes inserted
> in place of it's data.   Not replacing it's data, just inserted
> into the stream, causing the real data (which follows) to be offset by 8192.

That's strange.  It happens periodically, you say?  What's the period?

> If one skips over the inserted 8192 bytes in the file, the data from that point
> onward matches the other file, until another 8192 zeros are encountered.
> 
> The total sizes of the two image files match each other,
> but that's probably just due to the logic in the 'cat' program.
> 
> I wonder where those extraneous pairs of zero pages are coming from?

Me too.  It seems rather unlikely that they are coming from the USB
device or the transport, since at that level everything is addressed in
terms of sector numbers.  I don't see how 16 extra sectors could just
get inserted.  More believable would be a problem in the block layer,
the filesystem, or the application.

> This makes USB drives somewhat unreliable for backups and the like,
> which unfortunately is exactly what just about everybody uses them for.

Or it makes all drives somewhat unreliable, depending on just where the 
problem is.

> This (so far) is with 2.6.23.1 (w/slab allocater), and 2.6.24-rc2,
> on two different machines (both Intel based, but different chipset, CPU, RAM, ...).
> 
> I suspect not all brands/models of USB sticks would give the same results,
> but it is rather worrisome that it happens at all.

You could try doing a more specific test.  Write a program to access
the block device sector-by-sector and store a distinct recognizable
byte pattern to each sector.  (Maybe just repeat the 4-byte sector 
number over and over.)  Then write another program to read the
data back and see what bytes ended up in which sectors.

At the same time, use the usbmon facility (see
Documentation/usb/usbmon.txt) to record exactly what data gets sent
to/from the stick.  That should at least be sufficient to determine the
problem is in the stick itself, in the SCSI/USB layers, or someplace
higher up.

Alan Stern

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/