All My Brain

During my experiments with WP-Super-cache, I noticed a strange thing happen to my Adsense ads. A short while after getting gzip compression to work properly, all my ad content had foreign characters and strange seemingly unrelated content.

Having changed nothing on my blog except for installing WP-super-cache, I decided to add an additional check to my .htaccess. Here is a modified snippet that disallows Google’s Adsense bot from receiving the gzipped page:

RewriteCond %{HTTP_COOKIE} !^.*comment_author_.*$
RewriteCond %{HTTP_COOKIE} !^.*wordpressuser.*$
RewriteCond %{HTTP_COOKIE} !^.*wp-postpass_.*$
RewriteCond %{HTTP_USER_AGENT} !Google*
RewriteCond %{HTTP:Accept-Encoding} gzip
RewriteCond %{DOCUMENT_ROOT}/wp-content/cache/supercache/%{HTTP_HOST}/$1index.ht ml.gz -f
RewriteRule ^(.*) /wp-content/cache/supercache/%{HTTP_HOST}/$1index.html.gz [L]


Notice the new line that says the User Agent can’t have Google in it’s description.

Sure enough, ads are back to normal. I’m not sure how exactly Google’s crawlers handle gzip compressed pages. They are sending an “Accept-Encoding” header that includes gzip or the page wouldn’t be served to them in the first place. Judging from the change in my Ads however, I’d suspect that the bot isn’t uncompressing the received file.