I have recently installed the W3 Total Cache plugin, and it has made a tremendous difference. I have no specific metrics, but the feeling I have is as if it loads at least three times faster!! WOW!
What makes the biggest contribution is CDN (content delivery network), which I set up on a subdomain. It is not the easiest thing to do and it took a couple hours of tweaking to get it just right. I mostly followed the instructions found here:
I also got some hints from How To Install W3 Total Cache Tutorial – Setup Guide and How To Install W3 Total Cache 0.9 Setup Guide.
In order to make it work for me, I specifically did the following things slightly differently: