There are two options to get and manage Varnish activity data:

• Varnish Plus, which is a paid service that offers a VAC (Varnish Administration Console). It allows the visualization of real time stats on Varnish usage.
• collectd, wich is daemon that generates servers usage graphs taht posesses a Varnish plugin.

The former option isn’t free while the latter feels a little bit difficult to implement and is limited by the type of data collected.

My two big issues were: how do I get Varnish usage data i.e. HIT and MISS, and whiel being at it, get the URL lists for the MISS to identify cache problems. Therefore I decided to use Google Analytics to meet my needs.

To store these data in Google Analytics, a custom dimension has to be created and configure to get the Varnish data sent there. Varnish will register the HIT or MISS in a cookie whose value will be sent by the browser to Google.

This video describes the Varnish subroutines and how to set the proper configurations to be deployed.

Varnish groundwork

Install vmods, an extension that expands Varnish foncitonnalities including the option to set cookies.
First, install Varnish sources to compile the module.

apt-get install varnish-dev

Download the archive including the sources and compile everything.

wget https://download.varnish-software.com/varnish-modules/varnish-modules-0.11.0.tar.gz
tar xvzf varnish-modules-0.11.0.tar.gz
cd varnish-modules-0.11.0
./configure
make
make check # optional
sudo make install

You can check that the installition successfully finished by listing the /usr/lib/varnish/vmods/ directory that should include all the new modules.

Varnish configuration

Edit the configuration file to include the module header.

import header;

Inject code in vcl_deliver, the latest subroutine called in the Varnish process during a HIT or MISS. The reasons that lead to a MISS, stored in the X-Pass header will be retrieved. Set the cookie with a header.append.

if (obj.hits > 0) {
   set resp.http.X-Cache = "HIT";
} else {
    set resp.http.X-Cache = "MISS: " + req.http.X-Pass;
}


header.append(resp.http.Set-Cookie, "VarnishCache="+resp.http.X-Cache+";path=/");

The X-Pass is set in vlc_recv. Here is an example:

# Don't cache some URLs (security in case missing header in PHP)
if (req.url ~ "/(.*)sitemap(.*).xml") {
   set req.http.X-Pass = "Urls";
   return(pass);
}

Google Analytics

The creation of a custom variable and configuration of Google Analytics will be explained in the following video. The code used is set right after.

The following javascript function get the value of a cookie.

function getCookie(cname) {
    var name = cname + "=";
    var decodedCookie = decodeURIComponent(document.cookie);
    var ca = decodedCookie.split(';');
    var clen = ca.length;
    for(var i = 0; i < clen; i++) {
        var c = ca[i];
        while (c.charAt(0) == ' ') {
            c = c.substring(1);
        }
        if (c.indexOf(name) == 0) {
            return c.substring(name.length, c.length);
        }
    }
    return "";
}
var varnishCache = getCookie('VarnishCache');

The following snippet set the value of a custom dimension. The value now sent to Google is made during the pageview. In my example, there is a single custom dimension hence the index 1 appended to the word dimension.

//get('set', 'dimension[index]', value)
ga('set', 'dimension1', varnishCache);
ga('send', 'pageview');

Creation of a custom dimension

You can create up to 20 dimensions. tha can be activated an deactivated but cannot be delete.

Reach the administration page of your google analytics account.

In the PROPERTY column, click Custom Definitions > Custom Dimensions.
Click New Custom Dimension.
Add a Name.

To set up custom metrics for the HIT and MISS, go to Customization > +New Custom Report. Add a title.

Here is an example of a custom report that will be created. It shows the number of pages viewed based on the HIT and MISS (coupled with the URLs).

Here is the report that has been generated.

Thanks to this new report that gives you HIT and MISS stats, you are now able to detect any cache issue and correct it when necessary.