Python template language performance (Chameleon is cool!)

I really like the Genshi templating library. However, it really doesn't perform nearly as well as something like ClearSilver. Out of curiosity I updated the benchmarks included with Genshi to include the chameleon.genshi package. It performs amazingly well.

Genshi tag builder 378.72 ms
Genshi template 213.89 ms
Genshi text template 146.23 ms
Genshi template + tag builder 404.52 ms
ClearSilver 65.23 ms
Django template 423.26 ms
Chameleon Genshi Template 26.02 ms

These numbers are from the genshi bigtable benchmark which generates
a 1000x10 table (1000 rows of 10 numbers each). The load and creation
of the template and the construction of the list of dicts that make
up the 1000 rows of data are all done outside of the timing. All this
benchmark tests is the time to pump this data into the template and render.

I'm still suprised by the fact that chameleon beats clearsilver here.

Ah. The ClearSilver benchmark looks something like:

def test_clearsilver():
"""ClearSilver"""
hdf = neo_util.HDF()
for i, row in enumerate(table):
for j, c in enumerate(row.values()):
hdf.setValue("rows.%d.cell.%d" % (i, j), cgi.escape(str(c)))

cs = neo_cs.CS(hdf)
cs.parseStr("""
""")
cs.render()

So now I understand why the clearsilver test is taking so long. That benchmark code transforms
the list of dicts into an HDF dataset and does some string escapes and so on. It's easy to
confirm that this transformation is the majority of the runtime of the ClearSilver benchmark.

Moving that construction of the HDF dataset outside of the timed benchmark function shows us that the majority of the time ClearSilver spends is not rendering your content but building an HDF dataset.

ClearSilver 3.41 ms
Chameleon Genshi Template 26.70 ms

However, it is still very impressive that chameleon runs as fast as it does given that it is able to work with the unmodified python data structure rather then an optimized HDF dataset. And honestly in our system I don't think we can move the construction of the HDF out of the rendering pipeline so that ~60ms rendering time with ClearSilver is actually going to be slower then Chameleon. Combine the speed of Chameleon with Genshi's syntax and you've got a pretty sweet templating system.

Of course, Chameleon's default template language (TAL) has many of the same aspects I like about Genshi.

So let's get a number for TAL too:

ClearSilver 65.60 ms
chameleon.genshi 25.77 ms
chameleon (TAL) 26.25 ms

Messing with google wave

New Tutorial Toolchain

Previously I generated my tutorial pages (Python Tutorial) by hand writing *.shtml files.  I used Apache SSI (server side includes) as a very basic templating system to construct pages from basic building blocks.  For each code sample I create a *py file.

A makefile ties everything together by running a perl utility, code2html over all the python files to create *.py.html files.  These code snippets are then inserted into the tutorial pages with include directives.

I'm currently in the process of migration to a scheme that uses a real templating system (Genshi) and the Pygments library to build the complete tutorial.  The finished scripts should allow me to work directly with HTML and Python files on disk and compile the whole thing into a complete document.

Reasons for migration:

• The toolchain will be all python eases migration to Django/Twisted web layer
• Genshi templates will help ensure that the markup is correct and make it considerably easier to modify the entire tutorials look and feel going forward.  (Or reuse these work for other tutorials I'd like to work on in the future).  
• Pygments is an excellent syntax highligher with support for a ton of languages.

Unfortunately:

• There is still a build step
• Creating tutorials still involves writing html by hand
• No web interface to edit pages or create new ones

Emacs 23 On Leopard

Just a quick note on building emacs-cvs (emacs23+) on Mac OS 10.6.

First, check emacs out:

$ mkdir Emacs; cd Emacs  

$ cvs -z3 -d:pserver:anonymous@cvs.savannah.gnu.org:/cvsroot/emacs co emacs-cvs

Now create a scratch directory:

$ mkdir emacs-build

$ cd emacs-build

To build the mac App bundle of emacs:

$ ../emacs-cvs/configure --with-ns

$ make install 

$ open nextstep/Emacs.app # just a test to confirm it works before installing

$ sudo cp -r src/Emacs.app /Applications

To build a more old school x11/commandline emacs as well as replacing the emacs22 that ships as the
default on Snow Leopard.  This is my preferred install as I tend to stick to the commandline whenever possible:

$ ../emacs-cvs/configure --prefix=/usr --with-x-toolkit=no --with-jpeg=no --with-png=no --with-gif=no --with-tiff=no

$ make

$ ./src/emacs -q # just a test to confirm it's working before installing

$ sudo make install

Favicon!

Check out my new penguin/dino favicon!

(Belated) Spring Cleaning

I appreciate all the e-mails from folks who've found the Python tutorials useful.  I'm finally starting to clean up the site as it's kind of a mess at the moment.

There's a long todo list and cleaning up the main page of the site is just the logical place to start.

I'll be updating minor problems in the simple HTML tutorials but will I don't plan on making significant changes or additions until I've got a better way to edit and update that content.  Most likely this means that I'll be plugging in a real CMS to better manage it.  I've some interest in creating a custom engine in Django or Twisted, which might be a really nice example for the tutorials as well as an improvement for me in terms of easy editing and improving the quality of the code running the site currently.

Thanks again for all the encouragement to get back to work on the site.

I'll keep you posted.

Penzilla Project on Google Code

Anyone interested in my Python tutorials (penzilla.net/tutorials/python) can now check out the mercurial repository including the code and HTML here at google code.