« The World of Stuff goes to the movies
2008: A Personal Odyssey? »

Hidden in plain sight

Tue Jul 01, 2008 21:48 (UTC -5)

Here’s something more people need to know: when you delete a file from your hard drive, it could still remain there, inaccessible through normal means, for a long time. Here’s an analogy: Think of a hard drive like a book in which each file occupies one or more pages. The pages may or may not be in order, so there’s a table of contents to let you know what’s there and how to find it. Deleting a file only removes it from the table of contents. When files are added later, they are added to pages that aren’t listed in the table of contents, an act which may or may not overwrite old, de-listed files. Well, it’s something like that.

These deleted files aren’t normally accessible. But there are programs that scan hard drives for unlisted, “deleted” files. If you’re going to be selling a hard drive or flash drive, the security implications are enormous. There are numerous accounts of people buying old hard drives or camera memory cards on eBay and using “undelete” utilities to find photos and even credit card numbers. Today I decided to try out one such program to find files I had deleted from my flash drives. The program I used is called Magic Rescue, and it’s available as magicrescue in Ubuntu‘s repositories and probably other Linux distributions. So this is a Linux program, but there are similar tools for other operating systems.

Magic Rescue works by scanning a drive for certain file types that you specify and outputting any that it finds into a separate folder (which should be on a separate drive to prevent the program from duplicating the same files over and over again and entering an infinite loop). It works on any filesystem or lack thereof. To use magicrescue, you need to know the device name of the drive you’re going to work on. It should be in /dev. For me, it was /dev/sdb1 whenever I had one flash drive plugged in. I was able to find this by going to the GNOME Partition Editor (gparted), but there’s probably a better way. In any case, do find out the right device name before you run the program, or else infinite loops might happen and the world will explode. You might also want to unmount the drive. I’m not sure if it’s necessary, but it couldn’t hurt.

The folder /usr/share/magicrescue/recipes contains “recipes,” or brief scripts for recognizing certain types of files. If you need a recipe that’s not there, you might be able to find it on the Internet or, if you’re really good, write it yourself. Some of them require programs that you may not have; to extract JPEGs, you need jpegtran (which I was able to get by installing libjpeg-progs). You use the -r flag to specify any or all of the recipes when doing a search. The -d flag says where you want to place any files that Magic Rescue finds. The final argument is the location of the drive being searched. Here’s what I ran when I searched a “blank” flash drive for some file types that I’d be likely to put on there:

sudo magicrescue -r avi -r gimp-xcf -r gzip -r mp3-id3v2 -r msoffice -r zip -r png -r jpeg-exif -r jpeg-jfif -d ~/found-new /dev/sdb1

Maybe it wasn’t necessary to include the -r a million times? But it worked anyway, and I found some pretty interesting stuff. It tended to find more recent files, which, in the case of one of my flash drives, were projects from my senior year of high school. Many different revisions of a few Microsoft Office files were retrieved, which gives some indication about how they’re saved. My camera’s memory cards revealed more secrets. Again, most of the photos it found were recent, but I recognized one as being from April 2005. I went back to my folder of April 2005 photos just to be sure, and… it wasn’t there. I knew I had taken 62 photos that particular day, and here was a final 63rd that the camera never told me about.

So you see, deleting files doesn’t delete them for good, and in fact, they can remain for years. So you’re going to want to get rid of them for good. Luckily, this is possible. Just as there are programs that can scour drives for deleted files, there are others that can overwrite them so they would be very hard (if not impossible) to recover. The GNU utility for this is called shred. (Windows has a similar tool whose name escapes me.) shred can obscure individual files or an entire drive by writing random data or zeroes. By default it does this 25 times.

I took one of my “blank” flash drives (that had 45 recoverable files on it) and used shred to overwrite the whole drive with random data five times and once more with zeroes. Again, the drive had been unmounted. And again, if you try this, be very, very sure of which device you’re shredding.

sudo shred -n5 -z /dev/sdb1

This took about 10 minutes to complete on a 64 MB flash drive. Once it was done, there was absolutely nothing on the drive, not even a filesystem. When I ran Magic Rescue on it again, it found nothing. I reformatted it to make it usable again, and it was ready to go, as though the old files had never existed. (Just to be sure, I ran Magic Rescue again after reformatting, and it still didn’t find anything.)

So before you give away that old computer or camera, remember that reformatting the hard drive or memory card isn’t enough. Your data could still be retrievable using simple tools. You have to actually overwrite the old data, preferably a large number of times, so your sensitive financial information and/or drunken party photos don’t get into the wrong hands.

Back in the ’50s, being a paperboy was a big deal. Check out the Cleveland Press‘s Carrier’s Handbook from back in the day. Later, the newspapers figured out they could save money by just throwing papers out of a truck.

As most older Americans can tell you where they were when John F. Kennedy was assassinated on November 22, 1963, longtime South Floridians can tell you what they were doing on January 19, 1977, when it snowed here for the first time in recorded history. Wikipedia has a comprehensive list of snow events in Florida.

How much do you know about the Nobel Prizes? Here are 12 Things You May Not Know About the Nobel Prizes.

1 comment

Thank you for the link to my Fun Friday Post (12 Things You May Not Know About the Nobel Prizes). While my blog is geared towards philatelists (stamp collectors), others may find some interesting entries in my “Fun Friday” posts.

Thanks again,

#1 by Tony Servies: Wed Jul 02, 2008 10:00 (UTC -5)

« The World of Stuff goes to the movies
2008: A Personal Odyssey? »