No matter how fast Internet speeds get, if you serve every single HTTP request directly from your server, it will choke under load. Here are a few of my personal tips to improve caching, based on experience in both small and large environments:
1. Use a tmpfs for your nginx/varnish cache whenever possible.
Most webservers these days rarely take advantage of the amount of RAM in the system.
This won't work for shared hosting, but if you have the capability (i.e. you are the root user on your dedicated server or VPS), you should be running a caching proxy (nginx or varnish, I prefer the former) in front of your website. But don't stop there! With modern SSDs, your disk I/O performance has probably increased, but it's still nowhere near the performance of RAM. Use a bit of old tech -- a RAMDisk -- to increase the performance of your web cache (in some cases, massively). In Linux, this is achieved by mounting a tmpfs partition (for small sites, 256mb or 512mb should suffice; I've used up to 8GB on larger sites). You simply then set your varnish's storage.bin location or your nginx static cache dir to a writable directory on the tmpfs partition, and ensure that your cache size (specified in the config) is smaller than the available space on the tmpfs partition. Presto! You have a cache that never hits your disk.
The caveat is that since a tmpfs is gone at reboot time, you need to set it up in your fstab (with the proper permissions) so that it mounts automatically at boot time, before your application starts. You'll take a performance hit in the few minutes after rebooting while the cache repopulates, but it's worth it.
2. Use memcached or redis for PHP session storage.
Once again, RAM is much faster than disks, especially for accessing small chunks of data. Using file or database storage for sessions *drastically* increases the load on, and decreases the lifetime of, your disks -- and your performance suffers.
3. Use memcached or redis for caching repeated SQL queries, with an indexed-table fallback, before executing potentially slow queries.
But take it a step further -- you can (and should) use memcached within your application whenever database queries are repetitive and don't require real-time updates. I've done this by implementing a 2-level cache system. My personal database access (MySQL) uses an extension of PHP's mysqli class, that takes a hash of the SQL being executed and stores that hash (along with its results, if they're not tremendously huge) in memory. In addition to that, for complex queries (i.e. JOINs and fulltext searches on large tables) it stores that same hash and results in a database table, with the index being the hash, along with an expiration time (longer than the memcache/redis time, but still within reason depending on the application). This results in your application checking, in order: RAM (memcached/redis); a fast, indexed SQL query, and then falling back to your slow ass text-match or JOIN.
UPDATE: I have recently ported this site to use redis rather than memcached due to the requirement of sharing sessions between PHP and Node.js. If your application doesn't require this functionality, memcached is still easier (and better documented) for simple caching. However, since my server resources for this site are limited, I didn't want to use two caching engines, and ported all the caching code to use redis.
4. Use the end-user's computer to store as much as possible, for as long as possible.
Evaluate what of the data being sent to the client REALLY needs to come from the server, and what data being sent from the client REALLY needs to be sent to the server. Unless your site changes format daily (even with a team of developers, this is unlikely), determine what JS and CSS and images need to be updated and break them into two groups: those that are OK up to a week before refreshing, and those that essentially last forever (for example, my photo gallery images). Set far-future Expires: headers on those long ones (30 days or more), and 7days headers on everything else. Remember, if you really need to push an update *now*, you control your cache (whether it's a CDN or a local cache).
5. Use a CDN and manage your URLs wisely.
I personally use the free plan from CloudFlare (https://www.cloudflare.com), which provides the ability to create up to three custom "Page Rules" that determine content caching, SSL, etc. While this is an awesome deal, it does present a bit of a caching problem with the way most sites are laid out, in terms of directory structure. Solution: code your site with this in mind. In my case (if you view source), you can see that all CSS, JS, and images are stored under a subdirectory called /lib. You could call it /s or /assets or /dickface or whatever you want, but remember -- these are LONG-TERM STATIC files that don't change often. Simply create a page rule with a long expiration for /lib/* and you're done -- no need to upgrade to a paid plan unless you need a lot of granularity. Then every file you put in that directory or any of its subdirectories is assured to have a proper long-term expiration on THEIR edge servers -- and you won't be delivering it from your own machine/bandwidth.