BB.Net / ramblings / tags / Pylons - Justin's Ramblings

dynamic ikiwiki pages

The static pages that ikiwiki generates are great, but I want to have some dynamic content here as well.

If this works, this page should include the servers uptime.

yay

So how does that work?

first configure nginx as follows

server {
    listen       80;
    server_name  bouncybouncy.net  *.bouncybouncy.net web;

    location / {
        root   /home/justin/bbdotnet/static/;
        index  index.html index.htm;
        ssi on;
    }
    location /dyn {
        # All POST requests go to pylons directly
        include /usr/local/nginx/conf/proxy.conf;
        proxy_redirect  default; 
        if ($request_method = POST) {
            proxy_pass  http://127.0.0.1:5000;
            break;
        }
        default_type text/html; 

        set $memcached_key "$uri";
        memcached_pass localhost:11211;

        proxy_intercept_errors  on;

        # If no info would be found in memcache or memecache would be dead, go to real dynamic location
        error_page 404 502 = @dynamic_request;
    }
    location @dynamic_request{
        # This means, that we can't get to this location from outside - only by internal redirect
        internal;

        include /usr/local/nginx/conf/proxy.conf;
        proxy_redirect  default; 
        proxy_pass  http://127.0.0.1:5000;
    }

}

Pylons is setup to run on port 5000 as usual, nothing fancy there.

Then anywhere we want some dynamic content we can simply do

<!--# include virtual="/dyn/demo/uptime" -->

For now, you have to disable the htmlscrubber plugin for this to work. There is probably a better solution. I think this would simply involve a plugin that could run after htmlscrubber to insert the include, then you would only need to have something like [[include virtual="/dyn/demo/uptime"]] in your pages.

If you did not mind requring javscript, you could use HInclude instead of SSI.

To keep things running fast, we enable to caching on the pylons controller. using a modified version of the beakercache decorator. The following lines are inserted at the end of the createfunc method, which causes the page result to be cached in memcache as well as in beaker.

url = pylons.request.path_info
if pylons.request.params:
    url += "?" + pylons.request.environ['QUERY_STRING']

mc = memcache.Client(['localhost'])
mc.set(url, result, cache_expire)

The only remaining problem I see is a small race condition. If the cache expires, and 20 concurrent requests all come in for the page, most of them will end up hitting python instead of waiting for the memcache key to appear. This might actually work better using varnish or apache2 with mod_disk_cache, but the last time I tried I could not get varnish to work at all, and apache2 (I think) still does not support PURGE.