How to Install ‘Varnish’ (HTTP Accelerator) and Perform Load Testing Using Apache Benchmark

Configuring Varnish

There are two configuration files you’ll need to become familiar with while working with Varnish:

1. The parameters configuration file for Varnish:

  1. /etc/sysconfig/varnish (on RHEL/CentOS 6.x/Fedora machines) or /etc/default/varnish (on Ubuntu and Debian systems (Varnish v3.0), and
  2. /etc/varnish/varnish.params on RHEL/CentOS 7 and Fedora 21 (Varnish 4.0).

Unless we explicitly say so, we will leave this file untouched.

2. The VCL (Varnish Configuration Language) configuration file, /etc/varnish/default.vcl
The first one is an environment configuration file that contains all the information needed by Varnish to work properly, whereas the second one allows you to define your caching policy.

As we will be dealing with different Varnish versions in this article, we will outline the common configuration tasks and indicate when to refer to the documentation for further information about each version (3.0 and 4.0).

As we said earlier, Varnish stands in the middle of external clients and the web server. For that reason, and in order for the caching to become transparent to end users, we will need to:

1. change the default port where Varnish listens on to 80,

2. change the default port where the Web server listens on to 8080 instead of the default 80, and finally,

3. redirect incoming traffic from Varnish to the web server. Fortunately, Varnish does this automatically after we have completed 1) and 2) steps.

The following table lists the configuration files and the changes that need to be made per version:

Version Distribution File Change or edit
3 Debian and Ubuntu /etc/default/varnish VARNISH_VCL_CONF=/etc/varnish/default.vcl
VARNISH_LISTEN_ADDRESS=
VARNISH_LISTEN_PORT=80
VARNISH_MIN_THREADS=1
VARNISH_MAX_THREADS=1000
VARNISH_THREAD_TIMEOUT=120
VARNISH_STORAGE_SIZE=512M
VARNISH_STORAGE=”malloc,${VARNISH_STORAGE_SIZE}”
VARNISH_SECRET_FILE=/etc/varnish/secret
VARNISH_TTL=120
Uncomment the DAEMON_OPTS line following the above lines.
  Fedora-based /etc/sysconfig/varnish
4 CentOS 7 /etc/varnish/varnish.params VARNISH_STORAGE=”malloc,512M”

All the settings are well commented in the configuration files. However, I think it doesn’t hurt to explain that:

VARNISH_STORAGE="malloc,${VARNISH_STORAGE_SIZE}"

or

VARNISH_STORAGE="malloc,512M"

means Allocate Memory Block of 512 MB for caching. You can change the amount of memory as per your hardware resources or alternatively choose to save cached files to disk.

In the second case (which we will not explain here) you can just use the default VARNISH_STORAGE line that appears by default in the configuration file.

Now let’s edit the web server configuration file:

Web Server Distribution Configuration file Edit
Apache Debian and Ubuntu /etc/apache2/ports.conf NameVirtualHost *:8080
Listen 8080
    /etc/apache2/sites-available/default
  Fedora-based /etc/httpd/conf/httpd.conf Listen 8080
Nginx Debian and Ubuntu /etc/nginx/sites-enabled/default listen 8080 in the server block.
  Fedora-based /etc/nginx/nginx.conf

and don’t forget to restart Varnish and the web server after you’re done!

Testing Varnish

The first test consists in making a HTTP request via curl and verifying that it is handled by Varnish:

# curl -I http://localhost
Check Varnish Performance
Check Varnish Cache Status

Varnish also includes two handy tools too:

  1. view in real time Varnish logs, intuitively called varnishlog,
  2. display Varnish cache statistics, called varnishstat.

As a second test, in the following screencast we will display both the logs and the statistics on a CentOS 7 server (dev1, upper half of the screen) and on a Debian 7.5 server (dev2, lower half of the screen) as HTTP requests are sent and responses received.

Our third and final test will consist in benchmarking both the web server and Varnish with the ab tool and compare the response times and the CPU load in each case. In this particular example we will use the CentOS 7 server, but you can use any distribution and obtain similar results. Watch the load average in top and the Requests per second line in the output of ab.

With ab, we will sent 50 (-c 50) concurrent requests at the same time and repeat the test 1000000 times (indicated by -n 1000000). Varnish will return a higher number of requests per second and a much lower load average.

Important: Please remember that Varnish is listening on port 80 (the default HTTP port), while Apache is listening on port 8080. You can also take note of the amount of time required to complete each test.

# ab -c 50 -n 100000 http://localhost/index.html

Conclusion

In this article we have discussed how to set up Varnish cache in front of a web server, Apache or Nginx. Note that we have not digged deep into the default.vcl configuration file, which allows us to customize the caching policy further.

You may now want to refer to the official documentation (see the links at the beginning of this article where I listed the configuration files for Varnish) for further examples or leave a comment using the form below.

If you liked this article, then do subscribe to email alerts for Linux tutorials. If you have any questions or doubts? do ask for help in the comments section.

If You Appreciate What We Do Here On TecMint, You Should Consider:

TecMint is the fastest growing and most trusted community site for any kind of Linux Articles, Guides and Books on the web. Millions of people visit TecMint! to search or browse the thousands of published articles available FREELY to all.

If you like what you are reading, please consider buying us a coffee ( or 2 ) as a token of appreciation.

Support Us

We are thankful for your never ending support.

37 thoughts on “How to Install ‘Varnish’ (HTTP Accelerator) and Perform Load Testing Using Apache Benchmark”

  1. I followed all your steps but kept getting 503 service unavailable after reboot. I tried few combinations ( setting KeepAlive off for Apache; increasing the timeout etc. )but that didn’t help. Ultimately had to go back to apache listening on port 80.

    Reply
    • Nginx and Varnish are not the same thing… you should read about them a bit! But you could use them on the same server. But with benchmark Apache and varnish is better than Nginx and varnish… but lots of things also depends on number of visitor, server configurations and server materials ;)

      Reply
  2. We have configure SSL so what setting we have to do in below content ?

    backend default {
    .host = “127.0.0.1”;
    .port = “80”;
    }

    OR
    backend default {
    .host = “SERVER-IP-ADD”;
    .port = “443”;
    }

    Reply
  3. Is this just for hosting a website? OR can it be used to cache all pages users are browsing. I.e users inside a LAN browsing out to the WWW?
    Also can it pre-fetch? I.e. every hour get the pages users have visited on facebook, youtube, (or whatever changes regularly) and update the cache with the newly added posts or videos.

    Thanks

    Reply

Got something to say? Join the discussion.

Have a question or suggestion? Please leave a comment to start the discussion. Please keep in mind that all comments are moderated and your email address will NOT be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.