Google SiteMap doesn't like custom 404 pages

[Steve Rindsberg]

When you sign up for Google SiteMap, they have you put a specially named file on your site (I suppose to verify that you've got rights to the site) then they have you click a Verify button, which presumably checks for the existence of the file.

It must check some other things, specifically looking for files that aren't there because I found that if I had a custom 404 page, Google would complain about it. It barked because it got a 200 ("everything's fine, file was found") message when it was looking for a 404.

I *think* it's reasonable that it would get a 200 whenever there's a custom error page. The web server is *supposed* to serve up a page rather than handing back a 404 and letting the browser deal with it.

If it handed back a 404 AND a custom error page, it wouldn't make sense, unless the browser's supposed to be smart enough to recognize such an odd situation.

Surely google's run across custom 404 pages before? It seems very odd that they can't cope.

IAC, the workaround's simple enough: open an ftp window on the site, rename .htaccess to something else temporarily, click the verify button on the google Sitemap page, it works, rename something else back to .htaccess and everybody's happy.

Renaming the custom error page instead *might* work but it can also trigger a bug in Apache:

Receive request for page
Look for page
Page isnt there
Check .htaccess
It says to display custom error page
Request custom error page, which takes us back to the beginning of the loop.

One or two browsers left open doing this dance can *really* run up your hit count at the ISP. ;-)