Aug 12

Recovering from an external drive failure on Mac

A few weeks ago I plugged the external USB hard drive that holds my photo library into my Mac, and was greeted with the following message:

"The disk you inserted was not readable by this computer."

“The disk you inserted was not readable by this computer.”

 

Bummer. As it happened, I didn’t have an up-to-date backup of the drive, having made some recent updates. I really needed to recover the contents of the drive. I tried the drive in another Mac, and got the same message. I tried it in a Windows machine, and got a message about an error with the drive.

I plugged it back into the Mac and had a look at Disk Utility. It showed the drive in the left pane, but with no associated volumes. On the First Aid tab, the Verify Disk and Repair Disk items were grayed out. In the disk info section at the bottom, the following telltale appeared: “Partition Map Scheme: Unformatted”. It was clear that the partition table had been corrupted somehow.

I gave up on the GUI tools and switched to diskutil, which is OSX’s tool to manage disks, volumes, and partitions.

 

The list command shows the available drives:

[21:07:16] mini-me ~ $ diskutil  list
/dev/disk0
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *120.0 GB   disk0
   1:                        EFI                         209.7 MB   disk0s1
   2:                  Apple_HFS MacintoshHD             119.2 GB   disk0s2
   3:                 Apple_Boot Recovery HD             650.0 MB   disk0s3
/dev/disk1
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *1.5 TB     disk1
   1:                        EFI                         209.7 MB   disk1s1
   2:                  Apple_HFS BigMedia                1.5 TB     disk1s2
/dev/disk2
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                                                   *1.0 TB     disk2

 

Lines 13-15 (disk2) represent the drive in question. Note that there are no partition types or volume names, nor are there any partitions. diskutil has several repair verbs. One of them, repairDisk, exists specifically to fix a busted partition map:

Repair the components of a partition map of a disk

 

I am certain that this drive used to be partitioned using the GUID partition scheme (the default under OSX), so answered yes to the confirmation question:

[21:08:23] mini-me ~ $ diskutil repairDisk /dev/disk2
Nonexistent, unknown, or damaged partition map scheme
If you are sure this disk contains a (damaged) APM, MBR, or GPT partition map,
you can hereby try to repair it enough to be recognized as a map; another
"diskutil repairDisk /dev/disk2" might then be necessary for further repairs
Proceed? (y/N) y
Partition map repair complete; you might now want to repeat the
verifyDisk or repairDisk verbs to perform further checks and repairs

 

That seems to have fixed it; diskutil now recognized the two partitions from the external drive:

[21:08:52] mini-me ~ $ diskutil list
/dev/disk0
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *120.0 GB   disk0
   1:                        EFI                         209.7 MB   disk0s1
   2:                  Apple_HFS MacintoshHD             119.2 GB   disk0s2
   3:                 Apple_Boot Recovery HD             650.0 MB   disk0s3
/dev/disk1
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *1.5 TB     disk1
   1:                        EFI                         209.7 MB   disk1s1
   2:                  Apple_HFS BigMedia                1.5 TB     disk1s2
/dev/disk2
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *1.0 TB     disk2
   1:                        EFI                         209.7 MB   disk2s1
   2:                  Apple_HFS TimeMachine             150.0 GB   disk2s2
   3:                  Apple_HFS Mercury                 849.7 GB   disk2s3

 

To confirm that the partition really was restored, mount up the disk:

[21:09:47] mini-me ~ $ diskutil mountDisk /dev/disk2
Volume(s) mounted successfully

[21:09:53] mini-me ~ $ mount
/dev/disk0s2 on / (hfs, local, journaled)
devfs on /dev (devfs, local, nobrowse)
map -hosts on /net (autofs, nosuid, automounted, nobrowse)
map auto_home on /home (autofs, automounted, nobrowse)
/dev/disk2s2 on /Volumes/TimeMachine (hfs, local, nodev, nosuid, journaled, noowners)
/dev/disk2s3 on /Volumes/Mercury (hfs, local, nodev, nosuid, journaled, noowners)

 

I manually checked the contents of the two partitions, and they appeared to be correct. All the files were readable and I didn’t notice anything missing. Disaster averted!

If diskutil had been unable to recover my files, there are some alternatives:

  • TeskDisk: Data recovery software to rebuild or recover from partition/filesystem problems
  • PhotoRec: Block-by-block recovery software for known file types
  • GParted: graphical partition editor

In addition, there are several commercial disk-recovery applications available.

 

A note on backups: as I mentioned at the beginning of this post, my backup of the data on this disk was out of date.  As I tried to recover them, I realized just how important that data was.  To avoid the recurrence of this problem in the future, I implemented a backup strategy for irreplaceable data:

  1. Master data stored on my machine’s internal drive
  2. Frequent backups, using <code>rsync</code>, to a local external backup drive
  3. Less frequent backups to an EBS volume in Amazon EC2-land (again using <code>rsync</code>)

A future move will be to switch to RAID1 or RAID5 storage locally, to be able to handle a hardware failure.

Feb 01

Essential tools for web developers

I’m a firm believer in the principle of using the right tool for the job.  It applies equally well when building software as it does building a house or fixing a car.  There are literally thousands of tools available to do the job, and it can be hard finding the right one.  This post focuses on web development tools, specifically Firefox extensions.

  1. Firefox: while it’s not an extension, the browser itself deserves mention.  Primary web development should happen in a standards-compliant environment, and Firefox is a great platform for that.  There are certainly other browsers that fit the bill (Safari and Opera are candidates), but neither has the wealth of extensions that Firefox offers, nor the developer community behind them.

  2. Firebug: the tagline at the Firebug site is “web development evolved”, and that is a true statement.  Firebug may well be the largest single innovation in web development, ever.  That’s a bold statement, but it provides so much functionality, you hardly need other tools.  Some of it’s best features include:
    • Visual DOM exploration: mouse over nodes in the DOM tree, see them highlighted in the browser window.  Conversely, inspect an element in the browser window and access it in the DOM tree
    • Realtime CSS editing: changes to styles take effect immediately
    • Javascript console: execute arbitrary javascript in the context of the page
    • Network monitor: see full details on every request and response the browser makes, as they happen
    • Javascript debugging: a full-featured debugger and profiler for javascript code
    • Realtime reporting of Javascript and CSS errors: you’ll know when something’s not right

    If you don’t have Firebug yet, go get it.  Now.

  3. HTML Validator: invalid markup is a web developer’s nightmare — if your markup isn’t right, how can anything else be expected to function or look right?  Yet, unless you’re extremely well disciplined (and even then, sometimes), you’ll make mistakes.  This extension adds on-the-fly validation to Firefox, letting you know right away when there’s an error, indicating where it is, and even offering suggestions on how to fix it.  The validator has multiple validation engines; for best effect choose the serial algorithm (w3C’s SGML parser first, then HTML Tidy).  There’s no longer an excuse for invalid markup.

  4. Web Developer Toolbar: Chris Pederick’s toolbar was one of the first Firefox extensions targeted at web developers, and it still offers plenty of functionality.  Besides offering shortcuts to oft-used Firefox features, it works with cookies, forms, window resizing, HTML validation and more.  It also provides lots of information about the page and it’s elements.

  5. Tamper Data: when you need detailed information about HTTP requests and responses, the Net tab in Firebug is one place to look.  Another is Tamper Data, which provides an easily-filtered interface for inspecting HTTP traffic.  However, Tamper Data also allows the user to “tamper” with the request before it’s dispatched to the server, an incredibly useful trick when debugging misbehaving web applications.

Using the right tools will not only make your job easier, it’ll make you a better developer.  These tools can alert you to mistakes in your code, and you’ll learn to avoid repeating them in the future.

Everyone has their own favorite tools in their toolbox.  These are my favorites, and I ones I believe no professional web developer should be without.

Jul 18

Introducing Omnibug

Omnibug is a tool for web developers.  I wrote it because debugging an Omniture implementation is… painful.

The idea is that web metrics (or webanalytics, if you’re so inclined) systems generally make an HTTP request (usually an image) in order to pass along tracking information.  The URLs contain lots of parameters, conveniently URL-encoded so you can’t read them easily.

Omnibug is an extension to Firebug (without a doubt the best Firefox add-on available, driving a revolution in web UI development).  It adds a new panel with the decoded output of each such HTTP request, making it a breeze to see exactly what values were sent.

Though it was designed with Omniture in mind, it will work with other systems (also tested with Moniforce).  The patterns it looks for are fully configurable, so in theory it should work with any similar system.

An additional feature is the ability to log matching requests to the local filesystem.  While this feature was intended to support automated testing of metrics implementations, it may have other uses.

See the Omnibug page for downloads and full documentation.

Apr 23

Breaking teh sudoers file on OS X

Not having anything better to do, I horked my sudoers file while modifying an entry, thereby preventing myself from sudoing at all (and subsequently fixing it). Weak.

Turns out there’s a pretty easy fix for this using the Finder, so long as your account has administrator privileges:

Steps:

  1. In the Finder, go to /etc (or do <cmd>-<shift>-g and enter /etc, if you don’t see it in the Finder)
  2. Get Info on sudoers
  3. In the Info window’s Sharing & Permissions section, add an entry for your local user with read and write access
  4. With a text editor, edit /etc/sudoers and fix the problem.
  5. Fix the permissions on /etc/sudoers back to 0440.
  6. Enjoy your newly-regained sudo power!

Of course, this can be used to overcome other permissions problems too.

If only visudo had a validator, along with a syntax checker…

Jan 24

SEO headings bookmarklet

I wrote this simple bookmarklet to highlight each heading (h1, h2 … h6) on the page with a 3px border, and add a tooltip with the tag name in it.

Drag the Headings bookmarklet to your bookmarks bar, or bookmark it, or even click it to try it on this page.

May 21

Load flags

Mozilla’s load flags defined (useful in tamper data):

http://www.xulplanet.com/references/xpcomref/ifaces/nsIRequest.html