Archive for the 'sysadmin' Category

Brief shell hack makes “/dev/ones” for erasing USB flash

Wednesday, January 5th, 2011

Since USB flash devices don’t support Secure Erase yet, I’ve been doing what I think is the next best thing: overwriting every block in them with all 1’s.  I think it’s better to write ones rather than zeros, because flash memory devices traditionally erase to ones and then subsequent writes turn some of those ones to zeroes.  (It would’ve been simpler if the first guy to implement flash devices just put an inverter in there, so that the erased state would be zeroes, but they didn’t.)  By erasing to ones rather than zeroes, I try to lengthen the lifetime of the flash memory cells in the device.

So the problem is how to quickly gin up a large supply of ones (byte values of 0xFF) when I need to copy them, e.g. to a USB memory stick.  For an infinite supply of zeroes we have /dev/zero, but nobody has bothered to create /dev/ones.  Here’s the quick Bash shell command hack that I use:

echo $'xFF' | dd bs=1 count=1  >/tmp/ones cat /tmp/ones{,,,,,,,} >/tmp/o2; mv /tmp/o2 /tmp/ones; ls -sh /tmp/ones

Then repeat the previous command as many times as needed, until you have enough ones.The first command makes a 1-byte file containing the 0xFF byte value, using the Bash $’string’ extension.  The next command concatenates that 8 times (using Bash {}-expansion} in a second file, then moves that file back to the original name, and prints out how big it is.  Do that the first time, you get an 8-byte file.  The second, a 64-byte file.  Third, 512.  A few more times, and due to the power of exponential growth, you’re up to a billion-byte file.

Then at the end, I do a little trick to avoid needing quite as much disk space or time.  When e.g. erasing a 16GB USB memory, I repeat the above until it shows me that I have a 1GB file, then I do:

cat /tmp/ones{,,,,,,,,,,,,,,,} | dd bs=64M of=/dev/whatever

This concatenates 16 copies of that 1GB file and, without writing the result to the hard drive, feeds them to “dd” to write them directly to the USB device.  I use a 64-megabyte block size so that the writes to flash don’t get done 512 bytes at a time, which would be incredibly slow.  After a long pause, dd complains about “No space left on device”, and I know it’s done.  (There’s no space because a “16GB” memory stick is 16.0 billion bytes, which is shorter than the 16 mebibytes that I wrote to it.  The binary doubling strategy produced mebibytes, not decimal megabytes.)  Then I have a nice empty flash drive; I can put it in my pocket empty, or reformat the filesystem on that drive and I’m done.

I would be interested in test results (from probing the flash chips) to see how much data is left after this sequence.  I’m also interested in finding out whether my zeroes-vs-ones supposition has any practical value.

PS:  If the USB drive was previously mounted, don’t forget to unmount /dev/whatever before scribbling all over its filesystem.

USB storage can never be securely erased

Saturday, December 25th, 2010

USB devices that implement the Mass Storage command set don’t provide a standard way to securely erase the contents of the storage. People often want to do this, for example before giving the storage device to someone else (without compromising your personal or business records that used to be on it), or before crossing an international border where normal legal protections against intrusive searches or “fishing expeditions” do not apply.

The lack of a standard command for this prevents USB-attached disk drives from being erased using the ATA Secure Erase command. (Sometimes you can open the plastic, extract the SATA drive from inside the USB disk drive case, attach that drive to a computer via SATA, and then erase it securely, but this is painful). It also means that there’s no reliable way to remove information from a USB flash memory stick, which is more serious.

If you merely delete a file from such a memory stick, its contents probably still exist in the flash chips, and it can be read out by anyone who pops open the plastic and connects to the flash chip itself. An upcoming paper from the USENIX FAST conference details how 14 different conventional attempts to securely erase a file all failed to erase the contents of the file from a variety of USB memory sticks. (See the slides.)

Clearly this oversight in the USB Storage command set should be remedied. Who knows somebody who’s on the relevant standards committee?