Google Maps vs. Virtual Earth: A geocoding accuracy showdown in West Palm Beach
By William M. Hartnett on Jan 15, 2008 in featured, geekery
Careful readers will have noticed I have a certain admiration for Microsoft’s Live Search Maps and the broader Virtual Earth platform. Not only is it a worthy competitor to Google Maps, but in some respects it’s a superior product. Mileage may vary depending on your location and needs, but I have crazy love for Virtual Earth/Live Maps down here in South Florida. Even casual readers will know that I find Virtual Earth’s oblique aerial imagery mind-blowingly awesome, whereas I think Google’s Street View is merely awesome. Like in a “check out my awesome four-slice toaster” sort of way. Too, most of Microsoft’s imagery in South Florida appears to have been shot in early 2007, while Google’s looks 2005-ish at best.
Image quality and recentness aside, colleague and all-around brainy fella Matt Wensing last week ran across another issue that I haven’t seen mentioned before: Geocoding accuracy. As in, Google isn’t very good at it, at least compared to Virtual Earth, and at least in Palm Beach County. Matt passed 400 random West Palm Beach addresses to both Google Maps and Virtual Earth. We happen to know the actual center point of these parcels, so he compared the distance from that exact point to the coordinates returned by each mapping service’s big, black and seemingly unknowable geocoding box. Hit the jump to read about the results and download the test points. Or be totally lame and don’t hit the jump. See if I care.
In the head-to-head, closest to the pin contest, Microsoft’s geocoder beat Google’s in 341 of 400 cases, with four ties. That’s an 85.25 percent winning percentage for Virtual Earth, people. Not too shabby for a mapping service about which almost no one outside mapping circles appears to have heard. Google’s average error was 509 feet, with a median of 149 feet. Microsoft’s geocoder was off by an average of 366 feet, with a median of 83 feet.
Here’s one not even particularly dramatic example. The true property is highlighted in red, with Google’s pin six houses down the street.
And here’s how Microsoft did. That’s the actual house obscured by the pin.
So, what say you, super mapping geek who actually read to the end of this post? Are we missing something here? Or should more developers look at Google Maps not as the default in this space, but merely one of several serious options?
Mash your mouse here to download the test points and results (Excel worksheet, 121 KB)
Matthew Wensing | Jan 15, 2008 | Reply
I also wonder why Google’s ortho imagery is so much more saturated and grainy than Microsoft’s.
Morten | Jan 15, 2008 | Reply
“I also wonder why Google’s ortho imagery is so much more saturated and grainy than Microsoft’s.”
That’s a direct result of analog imagery vs. Microsoft’s digital cameras. You will still find images on Live Maps that were shot with analog and has the same problem.
Regarding geocoding accuracy: It’s not fair to do this in one small area. Both GE and VE often get their data from local vendors, and for this area Microsoft probably just happened to get the data from the best vendor, whereas as far as I can tell, GE just uses interpolated adresses along the street centerlines for this particular area.
As with their aerial imagery in some areas one is better than the other, and vice-versa.
Matthew Wensing | Jan 15, 2008 | Reply
I think it’s fair, because it’s our market. I don’t care how good GE is doing in Calcutta if I’m trying to map homes in Boca Raton, you know? I think that’s all that’s being said.
William M. Hartnett | Jan 15, 2008 | Reply
@Morten: All true and valid points. But we’re really only interested in image quality and recentness and geocoding accuracy in our own small area. Google might blow VE away in Dallas and Helsinki, but that doesn’t do us any good here in South Florida.
Jim Pollock | Jan 16, 2008 | Reply
William,
Very interesting! Just out of curiosity… of the 400 points, I assume all 400 gave you street locations in Google? How many were roof-tops vs. streets in VE? Were there any wrong roof-tops in VE, or were the wrong ones all street locations? How big of an area in West Palm did your sample cover? Just wondering if you kept such notes. That would provide a lot of insite into the schema that’s going on.
Also, were your 400 addresses all residential? If you did commercial, did you notice any shift in accuracy for comm vs residential in either system?
Sorry for being anal! I can’t help it.
Best,
Jim
William M. Hartnett | Jan 16, 2008 | Reply
@Jim: All 400 addresses were residential, and only within incorporated West Palm Beach, not the larger metro area. I just added an Excel file of Matt’s test points to the bottom of the post, so feel free to download and play. And let us know what you think!
wlrock | Jan 18, 2008 | Reply
No mystery here - there have been multiple posts in the last several months that Microsoft uses the Group 1 rooftop geocoder. All other mapping systems use street interpolation, which puts a pushpin approximately where the number of the house/business should be. In most populated areas (at least in the US), the Virtual Earth geocoder is bound to be MUCH more accurate.
Mr. Hartnett, I commend you for finding one of the many key areas that Microsoft is investing in to provide the best digital earth product available. They just don’t seem to make a lot of noise about it, but I know they’ve invested heavily in better aerial imagery, the exclusive Pictometry Bird’s Eye 45 degree shots, the Vexcel massively scalable 3D models capability (see the 75,000+ modeled buildings in Florida lately?)and even Navteq road data, available to developers through the APIs (Google uses Navteq on their own site, but only permits access to lesser quality roads via their API so they don’t have to pay Navteq).
Unfortunately, the user interface is not always the best, which pushes people to other, more intuitive services.
People who pay for a quality and reliable service will choose Microsoft. Those looking to create a free mashup can choose either, but will be more familiar with Google. See the weather.com interactive weather map (they pay Microsoft for Virtual Earth access on the site), vs. the weather they put on Google MyMaps, which they don’t have to pay for. It’s basically free advertising to them.
Lots of other differences, but you and your readers are 100% correct - it only matters if it matters to YOU in your area.
Kiefer Richardson | Feb 14, 2008 | Reply
Google earth’s footage looks more like 2004 to me, which my neighborhood and things change a lot. there’s a whol new part of my neighborhood that cant be seen on google earth because its outdated. However Microsoft not only has 2007 or 2006 pictures, but it also has a feature called “birds eye view”. ABSOLUTELY SCARY, amazing how close; you can read letters on buidings and recognize your car on the road! INCREDIBLE
Venky Krishnan | May 30, 2008 | Reply
Take a look at the City of Miami’s Virtual Earth, Awesome use of this technology to help the community develop and for first responders and planners to get a view of the city.
http://www.miamigis.com/cityofmiamive/
William M. Hartnett | May 30, 2008 | Reply
Thanks for the link to Miami’s Virtual Earth-powered mapping app. Pretty impressive work!
Scott | Aug 1, 2008 | Reply
It’s amazing the accuracy of Microsoft’s 3D imaging as well. I compared a photo I took in Montana with Microsoft Virtual Earth’s rendering of the same view. See for yourself!
William M. Hartnett | Aug 1, 2008 | Reply
That is a pretty good rendering, not to mention a very cool photo on your part.