Comments on: Even Google has bugs? http://blog.bartjellema.com/2009/04/18/even-google-has-bugs/ My message to the void Fri, 23 Dec 2011 15:35:41 +0000 hourly 1 http://wordpress.com/ By: tjoos http://blog.bartjellema.com/2009/04/18/even-google-has-bugs/#comment-14 Tue, 04 Aug 2009 16:28:04 +0000 http://blog.bartjellema.com/?p=32#comment-14 Ok, got a quick response… and having a closer look showed me that there are actually only 154 results, not 41,900 as reported on page 1.

It seems the GoogleBot accepts URLs with a bracket as a valid URL and our server was setup to resolve *.tjoos.com to our IP, not just http://www.tjoos.com. Hence “(www.tjoos.com” got indexed alongside “www.tjoos.com” because of an erroneous link.

Since I’ve fixed the DNS it seems that the pages are slowly but surely disappearing from the Google index.

I’m still a bit surprised Google would index URLs that are not RFC compliant and that most browsers can’t display. But I guess that the DNS resolves (for some DNS lookup tools) and the website does exist at the resolved IP and responds with a page, so it’s part of the internet and should be indexed. Kind of makes sense.

]]> By: tjoos http://blog.bartjellema.com/2009/04/18/even-google-has-bugs/#comment-13 Mon, 03 Aug 2009 02:12:22 +0000 http://blog.bartjellema.com/?p=32#comment-13 I did contact Google at the time of the post and Google responded saying that this was working correctly. Apparently there has been a link with “(www.tjoos.com” on our site at some point in the past and this caused it to be indexed. And at the time it was true that if you ignored the fact that it’s an illegal URL, it did resolve in DNS.

So I removed the *.tjoos.com DNS entry and now it doesn’t resolve anymore. This was 2 months ago and a search for ‘site:(www.tjoos.com’ now returns 41,900 results. So even though the domain is now invalid and unresolvable, it continues to be indexed. :(

I would use the webmaster tools to remove the content, but that’s impossible because it correctly says: “URL contains illegal characters”.

I’ll contact Google again. Maybe I’m still missing something?

]]> By: Leo http://blog.bartjellema.com/2009/04/18/even-google-has-bugs/#comment-12 Thu, 30 Jul 2009 03:57:03 +0000 http://blog.bartjellema.com/?p=32#comment-12 Find any more information on this?

I’d guess there are links been generated somewhere (possibly within your site) that incorrectly writes the parenthesis into the URL. GoogleBot then ignores the parenthesis when resolving the URL, but still retains it in it’s database.

You might want to see if there are such links or perhaps even contact Google.

]]>