Nginx Memory Minicache

Dec 052012

Introducing the Nginx Memory Minicache, this is a combination of two concepts the nginx microcache and nginx on tmpfs.

The following information is based on our experience of using an nginx minicache stored in memory on a tmpfs mount point. The website environment is a Drupal 7 based multilayered architecture hosting a live production website.

The nginx memory minicache currently sits in front of a malloc varnish instance (OCD – Obsessive Caching Disorder), and is configured as a 10s minicache based on the 1s microcache configuration supplied by Fenn Bailey, ~~Microcaching: Speed your app up 250x with no new code~~. This link is deprecated, see below for the original nginx microcaching config example.

This configuration extends the nginx microcache concept by storing the nginx minicache in memory using a tmpfs mount point. From fstab:

tmpfs    /var/cache/nginx    tmpfs    size=96M    0    0

or for testing:

 /etc/init.d/nginx stop; mount -t tmpfs -o size=96m tmpfs /var/cache/nginx/; /etc/init.d/nginx start

Edit the minicache nginx config to limit the max_size and change the proxy_cache_valid time:

keys_zone=minicache:5m max_size=80m

proxy_cache_valid 200 10s;

The the only thing left to do is to choose your nginx memory cache level:

Microcache = 1s, Minicache = 10s, Maxicache = 1m, Megacache = 10m !!

The following graph shows nginx memory minicache activity in yellow on the tmpfs mount point. Fenn Bailey’s original nginx microcaching config example can be found below the graph.

Original Nginx Microcaching Config Example:

# Set cache dir</span>
<pre>proxy_cache_path /var/cache/nginx levels=1:2
                 keys_zone=microcache:5m max_size=1000m;

# Virtualhost/server configuration
server {
    listen   80;
    server_name  yourhost.domain.com;

    # Define cached location (may not be whole site)
    location / {

        # Setup var defaults
        set $no_cache "";

        # If non GET/HEAD, don't cache & mark user as uncacheable for 1 second via cookie
        if ($request_method !~ ^(GET|HEAD)$) {
            set $no_cache "1";
        }

        # Drop no cache cookie if need be
        # (for some reason, add_header fails if included in prior if-block)
        if ($no_cache = "1") {
            add_header Set-Cookie "_mcnc=1; Max-Age=2; Path=/";
            add_header X-Microcachable "0";
        }

        # Bypass cache if no-cache cookie is set
        if ($http_cookie ~* "_mcnc") {
            set $no_cache "1";
        }

        # Bypass cache if flag is set
        proxy_no_cache $no_cache;
        proxy_cache_bypass $no_cache;

        # Point nginx to the real app/web server
        proxy_pass http://appserver.domain.com;

        # Set cache zone
        proxy_cache microcache;

        # Set cache key to include identifying components
        proxy_cache_key $scheme$host$request_method$request_uri;

        # Only cache valid HTTP 200 responses for 1 second
        proxy_cache_valid 200 1s;

        # Serve from cache if currently refreshing
        proxy_cache_use_stale updating;

        # Send appropriate headers through
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        # Set files larger than 1M to stream rather than cache
        proxy_max_temp_file_size 1M;

        # Custom logging
        log_format custom '$remote_addr - $remote_user [$time_local]  '
                    '"$request" $status $body_bytes_sent '
                    '"$http_referer" "$http_user_agent" nocache:$no_cache';
        access_log  /var/log/nginx/microcache.log custom;

    }

}

Source: https://gist.github.com/fennb/1283573