Caching plugins can be easily installed and will cache your WordPress posts and pages as static files. These static files are then served to users, reducing the processing load on the server. This can improve performance several hundred times over for fairly static pages.
When combined with a system level page cache such as Varnish, this can be quite powerful. If your posts/pages have a lot of dynamic content configuring caching can be more complex.
Web server caching' is more complex but is used in very high traffic sites. A wide range of options are available, beyond the scope of this article. The simplest solutions start with the server caching locally while more complex and involved systems may use multiple caching servers (also known as reverse proxy servers) "in front" of web servers where the WordPress application is actually running. Adding an opcode cache like Alternative PHP Cache (APC) to your server will improve PHP's performance by many times.
Varnish Cache stores pre-built pages in memory and serve them quickly without requiring execution of the Apache, PHP, WordPress stack.
As described within, using a plugin for comments such as Disqus instead of native WordPress comments can assist Varnish by not requiring your readers to login to WordPress and increasing the number of page views that Varnish can serve out of the cache.
Look into HTTP Cache-Control (specifically max-age) and Expires headers, as well as Entity Tags for more information.