How to Install ‘Varnish’ (HTTP Accelerator) and Perform Load Testing Using Apache Benchmark

Configuring Varnish

There are two configuration files you’ll need to become familiar with while working with Varnish:

1. The parameters configuration file for Varnish:

  1. /etc/sysconfig/varnish (on RHEL/CentOS 6.x/Fedora machines) or /etc/default/varnish (on Ubuntu and Debian systems (Varnish v3.0), and
  2. /etc/varnish/varnish.params on RHEL/CentOS 7 and Fedora 21 (Varnish 4.0).

Unless we explicitly say so, we will leave this file untouched.

2. The VCL (Varnish Configuration Language) configuration file, /etc/varnish/default.vcl
The first one is an environment configuration file that contains all the information needed by Varnish to work properly, whereas the second one allows you to define your caching policy.

As we will be dealing with different Varnish versions in this article, we will outline the common configuration tasks and indicate when to refer to the documentation for further information about each version (3.0 and 4.0).

As we said earlier, Varnish stands in the middle of external clients and the web server. For that reason, and in order for the caching to become transparent to end users, we will need to:

1. change the default port where Varnish listens on to 80,

2. change the default port where the Web server listens on to 8080 instead of the default 80, and finally,

3. redirect incoming traffic from Varnish to the web server. Fortunately, Varnish does this automatically after we have completed 1) and 2) steps.

The following table lists the configuration files and the changes that need to be made per version:

Version Distribution File Change or edit
3 Debian and Ubuntu /etc/default/varnish VARNISH_VCL_CONF=/etc/varnish/default.vcl
VARNISH_LISTEN_ADDRESS=
VARNISH_LISTEN_PORT=80
VARNISH_MIN_THREADS=1
VARNISH_MAX_THREADS=1000
VARNISH_THREAD_TIMEOUT=120
VARNISH_STORAGE_SIZE=512M
VARNISH_STORAGE=”malloc,${VARNISH_STORAGE_SIZE}”
VARNISH_SECRET_FILE=/etc/varnish/secret
VARNISH_TTL=120
Uncomment the DAEMON_OPTS line following the above lines.
  Fedora-based /etc/sysconfig/varnish
4 CentOS 7 /etc/varnish/varnish.params VARNISH_STORAGE=”malloc,512M”

All the settings are well commented in the configuration files. However, I think it doesn’t hurt to explain that:

VARNISH_STORAGE="malloc,${VARNISH_STORAGE_SIZE}"

or

VARNISH_STORAGE="malloc,512M"

means Allocate Memory Block of 512 MB for caching. You can change the amount of memory as per your hardware resources or alternatively choose to save cached files to disk.

In the second case (which we will not explain here) you can just use the default VARNISH_STORAGE line that appears by default in the configuration file.

Now let’s edit the web server configuration file:

Web Server Distribution Configuration file Edit
Apache Debian and Ubuntu /etc/apache2/ports.conf NameVirtualHost *:8080
Listen 8080
    /etc/apache2/sites-available/default
  Fedora-based /etc/httpd/conf/httpd.conf Listen 8080
Nginx Debian and Ubuntu /etc/nginx/sites-enabled/default listen 8080 in the server block.
  Fedora-based /etc/nginx/nginx.conf

and don’t forget to restart Varnish and the web server after you’re done!

Testing Varnish

The first test consists in making a HTTP request via curl and verifying that it is handled by Varnish:

# curl -I http://localhost
Check Varnish Performance

Check Varnish Cache Status

Varnish also includes two handy tools too:

  1. view in real time Varnish logs, intuitively called varnishlog,
  2. display Varnish cache statistics, called varnishstat.

As a second test, in the following screencast we will display both the logs and the statistics on a CentOS 7 server (dev1, upper half of the screen) and on a Debian 7.5 server (dev2, lower half of the screen) as HTTP requests are sent and responses received.

Our third and final test will consist in benchmarking both the web server and Varnish with the ab tool and compare the response times and the CPU load in each case. In this particular example we will use the CentOS 7 server, but you can use any distribution and obtain similar results. Watch the load average in top and the Requests per second line in the output of ab.

With ab, we will sent 50 (-c 50) concurrent requests at the same time and repeat the test 1000000 times (indicated by -n 1000000). Varnish will return a higher number of requests per second and a much lower load average.

Important: Please remember that Varnish is listening on port 80 (the default HTTP port), while Apache is listening on port 8080. You can also take note of the amount of time required to complete each test.

# ab -c 50 -n 100000 http://localhost/index.html

Conclusion

In this article we have discussed how to set up Varnish cache in front of a web server, Apache or Nginx. Note that we have not digged deep into the default.vcl configuration file, which allows us to customize the caching policy further.

You may now want to refer to the official documentation (see the links at the beginning of this article where I listed the configuration files for Varnish) for further examples or leave a comment using the form below.

Best Affordable Linux and WordPress Services For Your Business
Outsource Your Linux and WordPress Project and Get it Promptly Completed Remotely and Delivered Online.

If You Appreciate What We Do Here On TecMint, You Should Consider:

  1. Stay Connected to: Twitter | Facebook | Google Plus
  2. Subscribe to our email updates: Sign Up Now
  3. Get your own self-hosted blog with a Free Domain at ($3.45/month).
  4. Become a Supporter - Make a contribution via PayPal
  5. Support us by purchasing our premium books in PDF format.
  6. Support us by taking our online Linux courses

We are thankful for your never ending support.

Gabriel Cánepa

Gabriel Cánepa is a GNU/Linux sysadmin and web developer from Villa Mercedes, San Luis, Argentina. He works for a worldwide leading consumer product company and takes great pleasure in using FOSS tools to increase productivity in all areas of his daily work.

Your name can also be listed here. Got a tip? Submit it here to become an TecMint author.

RedHat RHCE and RHCSA Certification Book
Linux Foundation LFCS and LFCE Certification Preparation Guide

You may also like...

37 Responses

  1. Andy says:

    I followed all your steps but kept getting 503 service unavailable after reboot. I tried few combinations ( setting KeepAlive off for Apache; increasing the timeout etc. )but that didn’t help. Ultimately had to go back to apache listening on port 80.

  2. Mayuresh Mulye says:

    What do you think about Nginx vs Varnish ?

    • Steacy says:

      Nginx and Varnish are not the same thing… you should read about them a bit! But you could use them on the same server. But with benchmark Apache and varnish is better than Nginx and varnish… but lots of things also depends on number of visitor, server configurations and server materials ;)

  3. reza says:

    hello

    it’s good idea but when we install the varnish , that give high resource.
    my idea is xcache installation in cpanel.
    and yours?
    thanks

  4. Mayuresh Mulye says:

    Excellent article !!!
    Thanks you very much….was fun testing on AWS too :)

  5. shraddhesh says:

    We have configure SSL so what setting we have to do in below content ?

    backend default {
    .host = “127.0.0.1”;
    .port = “80”;
    }

    OR
    backend default {
    .host = “SERVER-IP-ADD”;
    .port = “443”;
    }

  6. Hadi says:

    Hello,
    It was very usefull article;
    I have installed on my server and work great for me
    http://www.irwebhost.com
    Thank you

  7. RS says:

    Is this just for hosting a website? OR can it be used to cache all pages users are browsing. I.e users inside a LAN browsing out to the WWW?
    Also can it pre-fetch? I.e. every hour get the pages users have visited on facebook, youtube, (or whatever changes regularly) and update the cache with the newly added posts or videos.

    Thanks

Got something to say? Join the discussion.

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.