Caching with Radiant CMS, lighttpd and Lua

Lighttpd is now one of the top five web servers: very lightweight and efficient, with FastCGI support – which is how my half-dozen Radiant websites are served – and a clever built-in Lua interpreter, allowing you to script modifications to requests in ways beyond Apache’s mod_rewrite. For each request, Radiant can cache the complete rendered page in a pair of files (one for the body, another for metadata) – with a short Lua script, you can then arrange for Lighttpd to serve up recently generated cache files directly without ever touching the Radiant process – an enormous performance gain.



if (string.sub(lighty.env[“request.uri”],-1)==“/”) then

        lighty.env[“request.uri”]=string.sub(lighty.env[“request.uri”],1,-2)

end

local file = “/radiant/cache/” .. lighty.env[“uri.authority”] .. lighty.env[“request.uri”] .. “.data”

local attr = lighty.stat(file)

if attr and (os.time()-attr[“st_mtime”])<3600 then

        lighty.header[“Cache-Control”] = “max-age=3600, public”

        lighty.header[“Expires”] = os.date (“!%a, %d %b %Y %H:%M:%S GMT”,os.time()+3600)

        lighty.header[“Content-Type”]=getmime(lighty.env[“request.uri”])
— Now load any header lines defined in the yml file:
meta=io.open(“/radiant/cache/” .. lighty.env[“uri.authority”] .. lighty.env[“request.uri”] .. “.yml”)
if (meta ~= nil) then
for line in meta:lines() do
— Extract header fields:
string.gsub(line,“^%s+([A-Z][A-Za-z-]):%s(.*)$”,function(key,val)
lighty.header[key]=val
end)
end
io.close(meta)
end
lighty.content = { { filename = file } }
print("HIT for: "..file)
return 200
else

        print("MISS for: "..file)

end

The 3600 on line 6 is the TTL: if the cached copy is less than 3,600 seconds (one hour) old, it will be served rather than re-rendering the page. If you have a single Radiant installation where all content updates are made through that server’s web interface, you can use a much higher value (obsolete cached files get purged immediately by Radiant when you edit that page) – Radiant doesn’t know to purge files which are indirectly edited, however (for example by editing a snippet), and if you replicate on the database level as I do, with a MySQL replication ring, the replicated Radiant instances will not be aware of the update, so they never purge those files. Lines 7 and 8 insert HTTP headers which convey a similar message to clients and proxy servers, indicating that this file is not expected to change within the next hour. If you want a more aggressive caching strategy, I would increase the first number (serving older cached copies from the server itself, if present, relying on Radiant purging the cache as needed) but leave the other two. The later part of this Lua script loads any other headers which Radiant has defined in the .yml file, most importantly Content-Type (in my early days using Radiant, I was bitten by this: serving up CSS files as text/html will make some, but not all, browsers ignore the CSS content completely). The regular expression deliberately limits headers to those beginning with a capital letter in order to disregard the “cookie: []” entry from the yml files.

Even the relatively conservative settings I have here make a significant difference – in particular, if a single popular page gets “Slashdotted”, almost every single hit will be served as a single item of static content: Radiant itself will only see one hit per hour for any given page, however popular.

I intend to try increasing the TTL significantly in the near future, having switched to a non-replicated configuration (more specifically, the content is still replicated on the database level between two nodes, but only one is actively serving content at any time) – I also have grander long term plans which should give me the best of both worlds, ensuring that obsolete content gets purged on replicas as well. In the shorter term, I would like to implement handling of If-Modified-Since requests: if the timestamp for Last-Modified in the cached object is older than the timestamp in the If-Modified-Since field of the request, I could return a 304 Not Modified status code and skip serving the content entirely. Compared to the performance difference between rendering a page through Radiant and serving a cached page as static content, however, this will be a trivial extra gain.