.SHARP{C#DERS}

As more and more server architecture moves to the cloud the need for distributed caching continues to increase. Scaling horizontally with many tiny virtual web servers makes sense in a lot of scenarios, but without distributed caching quickly runs into diminishing returns. Going from one server to ten small servers can actually decrease performance as each server has to fetch from the database and cache locally. 

There are certainly tools out there to solve this problem, notably memcached. If you’re using Azure they also have their own distributed cache service. In order to use the Azure distributed cache service you can’t use their XS size servers, though. This restriction, along with wanting to set up distributed caching for our primary product at work, led me in search of a solution. We had just set up a Redis server to sync signalR connections across servers, so rather than adding another technology to the stack I decided to use Redis to sync caches as well.

This all led to me buidling CacheSleeve.

We were already using Marc Gravell’s BookSleeve library for interacting with Redis and is also where the project got its name.

Getting Started

The first thing to do was to create a common interface for caching:

public interface ICacher
{
    T Get<T>(string key);
    bool Set<T>(string key, T value, string parentKey = null);
    bool Set<T>(string key, T value, DateTime expiresAt, string parentKey = null);
    bool Set<T>(string key, T value, TimeSpan expiresIn, string parentKey = null);
    bool Remove(string key);
    void FlushAll();
}

This basic interface pretty much defines everything we should need to do for caching. Using generics with T lets us cache any primitive or serializable object.

An interface isn’t all that useful without an implementation so next came HttpContextCacher and RedisCacher. HttpContextCacher is pretty basic and simply uses HttpContext.Cache to implement the ICacher methods. The RedisCacher is a little more complex, but not too bad until you get into child/parent relationships. Here’s an example of using BookSleeve to interact with Redis:

using (var conn = new RedisConnection(_cacheSleeve.RedisHost, _cacheSleeve.RedisPort, -1, _cacheSleeve.RedisPassword))
{
    conn.Open();
    var result = conn.Strings.Get(Db, “user.1”).Result;
}

This bit of code will pull the value for the key “user.1” out of Redis.

At this point all we have is two basic cachers behind the same interface. Good if we’re looking to do some basic caching, but nothing distributed here yet. The real magic happens in the ICacher implementation HybridCacher. The HybridCacher uses the two other ICacher implementations, RedisCacher and HttpContextCacher, to provide a distributed cache with distributed invalidation.

Going Distributed

When getting items from the cache using the HybridCacher we do the normal caching dance of checking if we have it locally, if so use that, if not then go get it from Redis. This means once we’ve gotten an item on Server A, that item is cached locally and Server A won’t have to go back to get it from Redis until it’s invalidated. 

The fun begins when we start invalidating keys. It’s not enough to remove an item from Server A and Redis. All the other web servers might have local copies which they would happily continue to use. To force Servers B to get a fresh copy we need to invalidate the item in its local cache as well. This is where pub/sub comes to the rescue.

Redis allows you to subscribe to keys or patterns of keys. We can use this pub/sub feature to watch for messages about items being removed from the cache. Each time we manually invalidate a cache item we’ll also publish a message telling all of the connected servers to do the same locally. Using BookSleeve we can subscribe to a key pattern like so:

var connection = new RedisConnection(redisHost, redisPort, -1, redisPassword);
var channel = connection.GetOpenSubscriberChannel();
channel.PatternSubscribe("cacheSleeve.remove.*", (key, message) => LocalCacher.Remove(GetString(message)));

Now every time we see a published message that starts with “cacheSleeve.remove.” the LocalCacher (HttpContextCacher) will remove the item locally. To publish the message we just do the following whenever we delete or update an item using HybridCacher:

_remoteCacher.PublishToKey("cacheSleeve.remove." + key, key);

We do cache invalidation for items that are invalidated due to a date/time by simply checking if a key has an expiration date in Redis and setting that same expiration locally whenever we do a Get.

Hard Mode: Cache Dependencies

Just when I thought I was done I realized we really needed cache dependencies. This allows you to give an item a parent whenever you set it. If the parent is invalidated then so should all of its children.

You get cache dependencies for free with HttpContext.Cache, but there is no concept of it in Redis. So I implemented it manually like so:

• When an item is added with a parent we create two links between that item and the parent
• We add the key of the child to a list of children in Redis with a key like [parentKey].children
• We add another item to Redis that points to the parent with a key like [childKey].parent
• Now that we have these references, whenever we delete an item we can check if it has any children and recursively delete

There’s a lot more to it with invalidating the references themselve, date/time invalidation, etc. but you can look at the code on GitHub if you’re interested.

Sample Usage 

Before you can start using the distributed cache you will need to initialize the CacheSleeve CacheManager with connection details for your Redis server. This should only be done once per server connecting to the distributed cache, usually in Application_Start.

If you're running Redis on localhost with default settings the following is all you need to get the cache manager running:

CacheSleeve.CacheManager.Init("localhost");

Once the cache manager has been initialized You can use the HybridCacher to interact with the distributed cache.

Server A

var cacher = new CacheSleeve.HybridCacher();
cacher.Set("key", "value");

This item is now in the distributed cache and will be synced/invalidated across all connected servers.

Server B

var cacher = new CacheSleeve.HybridCacher();
var item = cacher.Get<string>("key");

Here item is equal to "value" and the cache item has also been stored in Server B's in-memory cache for subsequent requests.

Server A (again)

cacher.Remove("key");

This invalidates the item with the key "key" in Server A's in-memory cache, the distributed cache, Server B's in-memory cache, and any other connected servers in-memory cache.

Server B (again)

var item = cacher.Get<string>("key");

Here item is now null because it was invalidated on Server A.

Dependency Injection 

You can also use the HybridCacher without hard coding new instances into your code. HybridCacher implements the CacheSleeve.ICacher interface which you can use to inject the dependency.

Example dependency setup (StructureMap):

For<ICacher>().Use(new HybridCacher());

You can then use constructor injection (or property injection) to inject the dependency.