Wed 14 Nov 2007

Django Tips: Source Code Filenames

Following on from last week's post about writing code that isn't tied to a project structure in Django, here are a couple of other quick thoughts about file naming and layout.

(Apologies to the half dozen or so people who read this blog for the "anything but Django" content. I've been spending all my free time in the last week working on Django, so it's buzzing around in my head at the moment.)

The Executive Summary

I mentioned last week that your settings file doesn't have to be called settings.py and your root URL configuration (or any other URL configuration) file doesn't have to be called urls.py. In a similar vein, it's worth remembering that almost nothing has to be called by a particular name or put in a particular place. The number of required files are few enough we can list all of them:

1. Django applications, which are listed in the INSTALLED_APPLICATIONS list in your settings file, must be an importable Python module. Mostly for things like the automatic interface and constructing reverse relations between models (if model A refers to model B, then model B can also refer back to model A without any extra annotation). So Django must be able to do the equivalent of from app_name import ... for any app_name in your application list.
2. If your application contains any models, they must be in a models namespace in the application directory. This means either models.py or a directory called models/, containing all you models. We'll talk about this a bit more below.
3. Any template tags you write must live under a directory called templatetags in your application directory. You can have subdirectories underneath the templatetags/ directory, but that must be the root of the hierarchy.

Everything else is unrestricted (constrained only by Python's syntax and semantics).

The Restrictions

Models

If you only have a few models, a single models.py file in your application is fine. When you start to have many models or lots of custom model managers and supporting functions, a single file becomes cumbersome. So you want to start using normal practices and breaking things out into their own files and sub-directories. If you do this, you need to remember to import each model directly into the model namespace by writing things like this in models/__init__.py:

from file1 import ModelA, ModelB
from subdir.file2 import ModelC

Secondly, each model needs to set the Meta.app_label attribute to the name of the application is you're using this directory method. In this fashion, you can happily use as many files and subdirectories as you like under models/

At the moment, this is a little messy — needing to specify app_label and import each model explicitly. Fortunately, I have a feeling it's probably not too hard to fix when we get down to it (although lot of attempted fixes keep overlooking things like nested directories and files that only contain utility functions or model managers). Adrian has a plan that would seem to fix a few of these related problems, which he explained to me at one point, although I'm not completely clear on all the details. So I would expect this slight wart to go away in the near future.

Template Tags

Although custom template tags are required to live under a templatetags/ hierarchy, that's as far as the restriction goes. You can use whatever file names you want (subject to Python's rules again). You can also use sub-modules (sub-directories, each containing an __init__.py file). So if you have app_name/templatetags/foo/bar/baz.py, you can write

{% load foo.bar.baz %}

in a template and it load your template tags from baz.py.

The Non-Restrictions

All other file and directory names are completely unrestricted. This freedom is apparently a source of confusion because of the tendency to use common names (views.py, urls.py, settings.py) for the same concepts. However, that's mostly just a shorthand so that we can get to the meat of the issue without having to lay the ground rules all the time. Unless Django needs to automatically import something (which means models and template tags only), you can use any name you want.

Some quick notes on a few of the cases that cause confusion for those who still aren't believers...

VIews

A Django view is just a Python callable that accepts at least one argument (the HttpRequest object as the first positional argument) and returns an HttpResponse. Any function that meets that interface is sufficient. You can store it anywhere. You can load it however you like. It doesn't even have to be a function (a class with a __call__ method is used in some cases in the Django source, for example).

Remember that you import the views in your URL configuration, so the imports are entirely under your control. Feel free (and be encouraged to) put your views in different files, grouped by purpose, so that you don't have multi-thousand line files to plough through when working on your code.

Templates

Almost certainly by this point, somebody thinks I've made a mistake because templates must go in the templates/ directory of your application so they can be loaded, right? Wrong! That is only true if you decide to use the app_directories template loader. This is a handy loader in a lot of cases, since you can don't need to make any extra configuration changes when you add new templates and it's a loader that is available in the default configuration. But don't confuse possible with compulsory.

For all its benefits, be aware that the app_directories loader does have some drawbacks: it collapses all of your templates/ directories (for each application) into one, in effect. So name collisions are possible and not always easy to avoid if you're lazy and pick really common names (like index.html). This has led to the practice of people always putting a sub-directory under their app_name/templates/ directory, also called app_name, so that each application's templates are loaded as app_name/index.html, etc. That is better and works very well. Then you only have to worry about application name collisions, but it's impossible to entirely avoid name collisions at some level (you can't have two files with the same name, for example), so although there are design changes proposed here, too, the current system is hardly onerous.

What is possibly not widely realised is that the first template loader Django tries to use is the filesystem loader. This looks in the template directories listed in the TEMPLATE_DIRS setting for templates to load. It's used a lot to create site-wide templates, but can also be handily used for loading application templates: just append your application's root template directory into TEMPLATE_DIRS. This also means that even if somebody writing a third-party app has forgotten you might not be using the app_directories loader, you can still put app_name/templates/ into TEMPLATE_DIRS and have the filesystem loader do the work.

I'm slowly becoming more and more of a fan of just using the filesystem loader for a few reasons like this. I suspect that, long-term, it makes managing configuration and debugging problems easier for large installations. However, that's just personal opinion and my preference. As always, whatever works for you is certainly a feasible alternative.

Conclusion

Everything I've written here is quite possibly of no use to you. Django's defaults and conventions work quite well for a lot of situations. Now and again, though, I see mailing list posts that make me realise some people handicap themselves by trying to stick too closely to convention without good reason. Part of any learning curve is stepping and reviewing and revising your assumptions periodically. Maybe this has helped you do just that.

Topics: software/django/tips