Tue 12 Feb 2008

Django Tip: Application-level context processors

Whilst going through my feed reading this morning, I read the application-level context processors were not possible. This came as a bit of a shock, since I've been using them for a while now. :-)

It's not hard and there's an interesting conclusion to be drawn about the power of scoping rules (plus a reprise on my thoughts on why putting everything under the sun into a framework isn't a great idea).

Some Context

This post is essentially a reply to a reply to a post about framework design. I'm certainly not about to belittle the efforts of either James or Eric here, since anybody who posts their thoughts, wishes and problems in a coherent and fair fashion on the net deserves respect. Rather, I want to point add to the body of knowledge about what is possible, in a concrete fashion. I also want to try and throw in some advice on technique and how these solutions can be rediscovered.

Context Processors

Context processors in Django are functions that run just before a template is rendered. A template is given a bag of data (the context) and context processors can add to that bag of data. A context processor is only given the request object; no further arguments. The idea is that it should be suitable to be run for every template and only adds data that is useful in most cases. The idea is that you're not obliged to use each piece of data in the template, obviously, but you've decided it makes sense to have it available as a sort of global value each time.

Django also provides the ability to provide extra context processors directly to the RequestContext object. Sometimes this is used to run a particular context processor for a single view or a small subset of views. The drawback (unless you step back for a minute) is that you have to specify these extra processors each time.

Well, no. You don't.

Writing a replacement RequestHandler

The first time you try to reuse some context processors over and over for a particular set of views, the repetition became obvious. So let's do the natural thing and factor out the repetition. In the solution that follows, I am intentionally not calling my class RequestContext. You want to have clues when you're reading the code that this is different from the normal RequestContext. When you're reading this code in 6, 12 or 24 months, you will need the reminder that this is your own variant and not something that behaves exactly as the Django standard.

Drop this into a file somewhere:

from django import template

def make_request_context(extra_processors):
   class MyRequestContext(template.RequestContext):
      def __init__(*args, **kwargs):
         processors = list(kwargs.pop('processors', ())
         super(MyRequest, self).__init__(processors=extra_processors + processors,
               *args, **kwargs)
   return MyRequestContext

Now, whenever you want a RequestContext-like class that always runs a specific set of context processors, you call make_request_context(), passing it a list of the extra processors.

For example, if you have one file with all your views in it, you could put this at the top:

MY_CONTEXT_PROCESSORS = (
    super.secret.special.processor.stuff,
    ...
)
RequestContext = make_request_context(MY_CONTEXT_PROCESSORS)

and your code will work transparently (that's why I called it RequestContext here, so that existing code won't require any change. Slightly contradicts the above advice and I wouldn't personally do this, but it's your call to make).

For bonus points, you could alter my function a little bit to cache the classes it creates and use the Borg pattern. That way if somebody wants to reuse the class with the same list of extra context processors, you don't create yet another object. A little more efficient and probably worth it if you're using this in multiple view files. I haven't shown that code here since it's pretty easy to implement and this post is long enough without it, but that's actually the approach I've got in my own little library of extra bits I use in a lot of Django projects.

Aside: Why Not Use a Metaclass?

Now, it's possible to write make_request_context() using a metaclass and a __new__() method. But why bother?? That just adds extra complexity because you have to remember the signature for __new__() and puzzle out when it gets called, etc. It's much harder to read .People seem to leap to meta-programming a bit too fast in Python circles these days; particularly people who are just getting started in the language. Factory functions are often a lot simpler to read when they're only doing simple things. Meta-programming has its place, but it shouldn't necessarily be your tool of first resort. First check that you really need to use a class (it's simplest to do so in this case, since we're replacing a class), then check if a simple function can create the class you need. Only then, if there are lots of parameters required or massive customisation, is it worth leaping into metaclassing. Keep in mind the documentation for the __new__() method in the Python docs: it was originally created so that it was possible to subclass built-in types like int and str. Everything else was already possible.

Again, sometimes a metaclass makes things a little neater. Sometimes it's a line ball. Often, it's just showing off.

Conclusion: Put It In Django Core?!

[Pure personal opinion here. I do not have my Django maintainers hat on, outside of the first sentence of the next paragraph.]

I'd vote against including this for the simple reason that it's not needed in core. Things should be added to core if they're really being used all over the place, are complicated to write, or cannot be written without modifying core. If it went in, I wouldn't cry, but it's just not something that's needed and the idea isn't to increase Django to 500,000 lines of code anytime soon.

It really isn't hugely necessary to have application-level context processors. They might be useful from time to time, but it's usually not harmful to just put them into the global list if you don't want to think about them. Secondly, what people want to do with application-level adjustments is quite varied, so it's pretty hard to write something that is both simple to use and covers the broad use-cases.

Adding things to the core of a framework makes the core bigger. That means there is more that people have to learn. There is already a lot of evidence that people don't read documentation. So adding more code and more documentation and more things to learn isn't really going to lower the anxiety levels much. It also adds something we have to maintain forever. There is more and more a trend in library and framework design to create small things that are functionally consistent and complete and provide something for everybody else to build on. This is not the same as containing every feature that person X might want (or even large group G might want).

What to add and what to leave out, feature-wise from a framework is always going to be a judgement call and is always, guaranteed, going to leave some people feeling they were left unheard or ignored. Including everything under the sun has its own drawbacks: code size, steeper learning curve, a much greater level of people wanting help from the same-sized base of people answering questions, ... . People should keep suggesting features that might be useful. Sometimes it will be added to the core framework (history shows I'm very far out on the conservative side of what to accept, by the way), sometimes it will trigger a post from some joker showing that it's already possible in five lines of code, sometimes you'll just have to add it to your own personal utility library (you do have a little utility library of things you use, right?). That's life in open source framework development land. Don't take it too personally, don't hold grudges (being alternately right and wrong is easy; trust me, I've got experience there) and don't stop experimenting and learning.

Topics: software/django/tips