An interesting project to start the day with today, Web server caching and compression. This is probably something that should have been investigated a while back, but better late than never. The whole idea being that if you compress the data at both ends (server and client) then you are transmitting less data and saving bandwidth, and the bigger the site, the more the improvement you will notice. However, the compression of course adds overhead and means more processing power on the server is required to do all this compression, again, the bigger the site, the more compression that will need to be done. Static files that don’t change only need to be compressed once and then can be stored on the server in their compressed form until they change, this will save on the processing power required, this is no good for dynamic content though.
The caching on the other hand basically means that the client keeps a local copy of some parts of the site (bits that don’t change that often, i.e. backgrounds etc) and so each time you visit a page within a site, you are not constantly downloading the same files, saving a lot of bandwidth and from the clients point of view, the site loads a lot quicker. Of course your ISP may cache some stuff to reduce its bandwidth, and then you might go out via a more local cache, i.e. Squid which will also cache data.
With recent changes to Google, sites will now be ranked based on their speed, so this is another very important reason to speed it up as much as possible (See here). Anyway, this is a pretty big subject and I’m only going to scratch the surface here. For more detailed reading, please check here.
You will also want to get Firefox (here), and then firebug (here) and lastly Yahoo’s yslow for firebug (here).
Firefox is just a better browser than IE, nuff said. Firebug is a very useful plug-in for Firefox, very handy when diagnosing WebPages, and yslow is an add-on for that which allows you to quickly asses the speed of a page and work out where you can tweak it to make it better. It also gives each page a handy score so it can easily be ranked.
Next, get a copy of fiddler (here) which is a web debugging proxy. It’s basically a local proxy which makes it easy to keep track of the all the requests going back and forth from your browser.
I ran a test on my poker league site (www.ddpt.co.uk, written in PHP) before I had applied any compression or caching, and got a score of 71, Grade C. Not too bad, lots of A’s in there, but still room for improvement. One key area (Graded F) was “Compress Components with gzip”.
As with most things in IT, it’s a fine line between making it fast and also up to date. I.e., caching everything and setting it never to check for updates would make it sooper fast, but then, your content would always be out of date. So there’s a bit more to this than you might originally think.
In IIS (I’m running V7 at home) there are a few things to check. It turns out that by default, static compression was on, although upon investigation this was only for files larger than 256 bytes and this basically meant the compression of only 3 CSS files which didn’t make a stack of difference. I wanted to apply dynamic content but this option was not available. It turns out that this feature of IIS is not installed by default, so it was off to the Control Panel to add this.
Once added, I enabled it and then headed over to HTTP response Headers. In there you can set the Common Headers and I set the Web content to expire after 1 day. I then ran the yslow test again and got a score of 80, grade B. So, a bit better. There are still areas that can be improved. It told me I should host some stuff on a CDN (Content Delivery Network, read more on them here) but I can’t afford that (having only made £12 in 18 months from Google Ads on the poker site!). I got a grade E for 4 expires Headers, although 3 of these are googlesyndication pages, so out of my control. I should minify the JavaScript and CSS too, although It makes it a pain in the arse to edit so I probably won’t bother; on a commercial release I would definitely look at this. Another area is configuring entity tags (Etags, read more on them here). ETags are basically a response header to check for a newer file. There are a few major problems with Etags though, firstly, with an IIS restart the tags change and secondly, in a server farm situation where multiple boxes are serving requests, the Etag on one server would be different to another, rendering it a bit useless, although a fix can be found here. See here for a guide on disabling Etags.
My poker site doesn’t really get that many hits so the benefit was pretty low, but that was merely a test for our company’s corporate website (Running on IIS6 right now). Although the development of the site is out of my control, the IIS setup is well within my control, so it was time to speed it up. The initial score before I changed anything was… 55, Grade E, not very good I think you’ll agree. IIS6 is slightly different to IIS7 so these are the steps I took to speed it up:
- Enable content expiration (1 Day) – No Change
- Enable content expiration (30 Days) – Score 56, grade E. Not too keen on the 30 day expiration, so I’ll set this back to 1 Day.
- Under Web Sites Properties, enable “Compress Application Files” and “Compress static Files” – No Change. Basically, if we take a closer look at the metabase (%SystemRoot%\System32\Inetsrv\MetaBase.xml) we see this right at the very bottom (Make sure that “Edit Direct Metabase Edit” is enabled in IIS):
<IIsCompressionScheme Location =”/LM/W3SVC/Filters/Compression/deflate”
HcCompressionDll=”%windir%\system32\inetsrv\gzip.dll”
HcCreateFlags=”0″
HcDoDynamicCompression=”TRUE”
HcDoOnDemandCompression=”TRUE”
HcDoStaticCompression=”FALSE”
HcDynamicCompressionLevel=”0″
HcFileExtensions=”htm
html
txt”
HcOnDemandCompLevel=”10″
HcPriority=”1″
HcScriptFileExtensions=”asp
dll
exe”
>
</IIsCompressionScheme>
<IIsCompressionScheme Location =”/LM/W3SVC/Filters/Compression/gzip”
HcCompressionDll=”%windir%\system32\inetsrv\gzip.dll”
HcCreateFlags=”1″
HcDoDynamicCompression=”TRUE”
HcDoOnDemandCompression=”TRUE”
HcDoStaticCompression=”TRUE”
HcDynamicCompressionLevel=”0″
HcFileExtensions=”htm
html
txt”
HcOnDemandCompLevel=”10″
HcPriority=”1″
HcScriptFileExtensions=”asp
dll
exe”
>
</IIsCompressionScheme>
<IIsCompressionSchemes Location =”/LM/W3SVC/Filters/Compression/Parameters”
HcCacheControlHeader=”max-age=86400″
HcCompressionBufferSize=”8192″
HcCompressionDirectory=”%windir%\IIS Temporary Compressed Files”
HcDoDiskSpaceLimiting=”FALSE”
HcDoDynamicCompression=”TRUE”
HcDoOnDemandCompression=”TRUE”
HcDoStaticCompression=”TRUE”
HcExpiresHeader=”Wed, 01 Jan 1997 12:00:00 GMT”
HcFilesDeletedPerDiskFree=”256″
HcIoBufferSize=”8192″
HcMaxDiskSpaceUsage=”99614720″
HcMaxQueueLength=”1000″
HcMinFileSizeForComp=”1″
HcNoCompressionForHttp10=”TRUE”
HcNoCompressionForProxies=”TRUE”
HcNoCompressionForRange=”FALSE”
HcSendCacheHeaders=”FALSE”
>
</IIsCompressionSchemes>
There are 3 bits here really. Firstly the deflate options, then the gzip options and lastly the parameters. It’s worth noting here the you must restart IIS after making any changes to the metabase, and always always backup the metabase file before you touch it as its very easy to break!
Basically, deflate and gzip work slightly differently. Deflate uses the zlib (see here http://www.zlib.net) format and is essentially just a very slightly different compression method to gzip, which uses the gzip format. This is all a bit more confusing that it should be, and if you want somebody to blame for that, then look here. There are arguments as to which is better, gzip is apparently zlib with headers (see here, so it’s slightly larger, making zlib smaller and better. However, it also seems that gzip is more popular).
Anyway, this is getting a bit deeper than I really wanted to go, so, back to the metabase and we can see that both the gzip and deflate methods are checking only 3 file extensions:
- .htm
- .html
- .txt
And only 3 script extensions:
- .asp
- .dll
- .exe
Which in short explains why it wasn’t too effective earlier. There are a couple of settings I’m going to change (Based on information found here). Firstly, setting HcDynamicCompressionLevel to 9 (It’s 0 by default, which again is not very effective, the max is 10) and then also adding in some extra file and script extensions:
File extensions:
- .htm
- .html
- .txt
- .js
- .css
Script extensions:
- .asp
- .aspx
- .asmx
- .php
- .dll
- .exe
Image files are generally already compressed so extra compression won’t usually help. I noticed 13% improvement on average for .gif’s and a 0.4% improvement on average for .jpg’s. Caching of those files though will make a big difference. Really, all these sort of files should be compressed before being uploaded, this needs to be looked at by the web developers for future releases. It’s worth noting that smush.it provide a pretty decet service to shrink images for you, for free!
IIS7 does this all a bit differently, it uses a list of Mime Types to trigger compression. A future project will be moving our server from 2k3 to 2k8 boxes with IIS7, I’ll look into this next month I think.
So, was all this hard work worth the effort? I ran yslow again on our public website and the score came back at… drumroll… 63, Grade D. So, a bit better, still some room for improvement though. Another good tool to test your site can be found here, it had some nice graphs and time lines so you can easily see where your site is slow.
An in depth guide on everything to do with making your site quicker can be found here.
Update…
So, not content with the D grade, Pete spent a bit of time minifying the CSS and JavaScript and also combined a lot of the separate CSS and JavaScript files so there were less requests going back and forth. He also used CSS Sprites which combine the images, so again there are less requests. All this resulted in a B grade with a score of 80 – pretty damn good and hopefully worth all the effort!