This document is intended to help you understand how the various components of TurboGears 2 work together, and what happens to a web-request on the way into your controller code.
It may seem like there are a lot of layers here, and there are, but most of the time you don’t need to know anything about how they work, just that they are there to do work for you.
The first thing that happens is that some WSGI (web server gateway interface) compliant HTTP server recieves an HTTP message from somebody, and it calls your TG application which is a WSGI app.
For those new to WSGI, it’s a very simple interface that defines how web servers interact with python methods or functions, or really any callable.
The basic WSGI interface is this:
def simple_app(environ, start_response):
"""Simplest possible application object"""
status = '200 OK'
response_headers = [('Content-type','text/plain')]
start_response(status, response_headers)
return ['Hello world!\n']
Fundamentally, WSGI means your python function gets called with two things, an environ dictionary, and a start_response callable. Before your function returns, you have to pass a status, and a set of headers to the start_response method, and then you’re free to return a list (or any itterable) of strings as the response body.
The environ dictionary, is a copy of the CGI spec’s ENVIRON (http://hoohoo.ncsa.uiuc.edu/cgi/env.html). And it has everything you need to know about the incoming request.
One more thing to know about WSGI is that it’s easy for a python function or method to take and environ and start_response to do some stuff, and then to call another function that’s also a WSGI app (meaning it takes an environ and a start response). When an application like this sits between the “real” webserver and another WSGI app, we call it middleware.
The TurboGears request/response cycle is composed of various bits of middleware that help make writing web applicaitons easier for you.
Here’s a quick outline of the stack, but we’ll be going through the pieces in a bit more detail as we go.
WSGI Server
PasteCascade - serves one of a list of WSGI apps.
StaticFile Server - serves static files from /public
OR
TurboGears Application: - the TG stack
Registry Manager - sets up the request proxy, etc.
Error Middleware - if the path goes to _debug handle the request
Database Session Manager - setup the DBSession
Transaction Manager -
Authentication - add info to the environ if user is authenticated
Authorization - add more info to the environ for authorization.
ToscaWidgets - nothing on the way in.
Cache - sets up the cache
Session - sets up the web session
Routes - parses the URL and adds info to environ
Custom Middleware - User defined middleware
TurboGearsApp -- calls WSGI style controller
ObjectDispatchController -- gets params, do validation, etc
Your Controller Code -- does anything!
ObjectDispatchController -- renders response, etc.
ToscaWidgets - injects resources used by widgets
Transaction Manager - commits or rolls back transaction
Database Session Manager - cleans up the DBSession
Error Middleware - displays error pages, etc
In total, this stack provides automatic database helpers, sessions, authentication, authorization, caching, sessions, URL based dispatch, and injection of CSS and JS resources into your app as required, and generally makes web development easier.
WSGI Server
PasteCascade - Tries one app then the next
StaticFile Server
OR
TurboGears Application Stack
The first thing that gets called by the WSGI server on the way into the TG stack is the PasteCascade. Paste’s Cascade app tries several WSGI apps in order, if the first app returns an HTTP Not Found (404) status code, it moves on to the second, and so on. In the default TG configuration the Cascade does two things 1) tries the StaticFile Server which serves up static files from your public directory, 2) tries the main TurboGears application.
Registry Manager - sets up the request proxy, etc.
Error Middleware - if the path goes to _debug handle the request.
{{ lots of stuff }}
Error Middlware - redirect to nice pages on HTTP error codes, and produce debug pages/email for python errors.
The next thing on the stack is the Registry Manager which sets up some
global objects that proxy to the current thread, and the current
request. This is what allows you to do from tg import request and
then use that to manipulate just the current request. It also has a
less-often used but still useful feature which allows you to put one
TurboGears application inside of another, and still have different
config objects, etc. If you hear anybody talking about “Stacked
Object Proxies” or SOP’s that’s what this is.
The next layer on the stack is the error handling middleware. This is
there to provide debugging helpers when python exceptions or other
application errors happen. In debug mode this provides you with the
nice interactive debugger, and in production mode it’s what logs
errors and sends out e-mails about the failures. Whenever a request
comes in that lives on the _debug path, the error handler
middleware looks up the info and responds directly.
Other than that error handling middleware doesn’t do much on the way
in to the stack, but on the way out it catches errors saves data, and
does the right thing when _debug requests come in for that info.
Inside the error handler, the next thing we setup is a couple of database helpers:
Database Session Manager - creates a DBSession for the request
Transaction Manager - regesters a TransactionManager for the request.
{{ lots of stuff }}
Transaction Manager - Commit the transaction
Database Session Manager - Clear the DBSession.
Inside the error handling middleware is a tiny little piece of middleware that sets up a SQLAlchemy database session for this request on the way in, and clears it out on the way out of the stack. This means that in TG2 by default you get a new DBSession for every request, and everything is cleared away when you’re done with it. This keeps requests isolated, and matches the “stateless” pattern of HTTP.
And inside that is the middleware portion of the automatic transaction
system. When a request has updated the DBSession in any way (the in
memory copies of database data) a transaction is automatically
registered, and the Transaction Manager will handle it. If a python
exception happens, an HTTP Error Code is returned,or
transaction.doom() is called during the request, the transaction
will be rolled back on the way out.
There’s a lot more to the transaction manager than just that, because you can setup new TransactionManager classes for whatever you want. You can write an e-mail module that does not send e-mail until the database transaction is committed. And if you have a database that supports two-phase commits you can write transactions that span multiple data sources.
ToscaWidgets - nothing much on the way in.
{{ lots of stuff }}
ToscaWidgets - inject resources into the generated
Nothing much on the way in. Inject JS, and CSS resources used by widgets in the main app.
Cache - sets up the cache
Session - sets up the web session
Routes - parses the URL and adds info to environ
The middleware outside of Core Middleware is optional and can often be configured out via special config values in app_cfg.py, and can be manipulated in any way you can imagine by subclassing AppConfig and replacing the methods that set it up. TurboGears itself has code that requires that the core middleware be in place, so you won’t want to mess with this stuff without a good reason. This is particularly true of Routes which can only be configured out of your app if you reimplement TGApp. Please see App Config General Options for more information on how to modify the core middleware.
The Cache middleware sets up a reference to the threadlocal cache manager that turbogears uses to interface to whatever backend you’re using for caching. The cache manager is injected into the environ so that it’s available to anything that happens in the request. In the future it’s possible that this will no longer be middleware, and will simply become another global object that is configured separately from the WSGI stack.
The Session middleware also sets up a reference to a threadlocal session manager, and at the moment both Session and Cache use the same back-ends based on Beaker.
Finally the Routes middleware inspects the URL of the request, and tries to map it to a series of “routes” which explain what controller and controller method should be called to handle that request. The Routes middleware then puts this information into the controller so that the TGApp can call the right method.
By default TG is setup with one route, that goes to the “routes_placeholder” method on your RootController in the root.py module. This is a hint to TG’s object dispatch controller to take over and do dispatch to the right controller method in root’s object hierarchy.
You can define custom middlware that does whatever you want it to do
and pass into the application constructor in middleware.py.
To use a middleware before the TurboGears stack is processed you can
wrap the application returned by make_base_app function:
app = make_base_app(global_conf, full_stack=True, **app_conf)
app = MyMiddleware(app)
return app
To use a middleware that is TurboGears specific and wants to have the full
TurboGears stack available (session, authentication, database, etc…)
you can pass the middleware class to the make_base_app and it
will be created wrapping the application:
app = make_base_app(global_conf, full_stack=True, wrap_app=MyMiddleware, **app_conf)
return app
If you prefer to have more control over where your middleware is placed in the stack, you can do that by subclassing AppConfig or overriding methods on the base_config object.
Looks up a WSGIController object based on the info from Routes and calls it. By default this is an ObjectDispatchController that’s pulled into your app from lib/base so that you can override it if you need to.
But if necessary, you can replace with something more application specific.
The ObjectDispatchController’s job is to take the WSGI interface and adapt it to the way TG methods behave (dealing with templates and returned dicts, etc), and to do object based dispatch like CherryPy did in TurboGears 1.
The ObjectDispatchController’s functionality is broken into three
basic pieces. The root PylonsController implements a WSGI interface,
and actually calls the controller methods with params from routes. TG
provides a DecoratedController. Decorated Controller allows
you to use TG1 style decorations (@expose(), @validate etc.)
on your controller methods, but does nothing for dispatch.
All of the dispatch is done by the Object Dispatch Controller and some associated
functions that help with lookup.
The @expose and @validate decorators in TG2 are not function wrappers
in the same way that they were in TG1. They merely register
information about how that method ought to be called in it’s
associated decorator diagram. This is brought up here because they
influence the way that the Controller calls your code and handles the
response. Expose determines how the dictonary returned by the
controller is rendered into a WSGI response. If you return a string,
or a WebOb webob.Response object, expose will not change
your returned results at all.
The @validate in turn makes sure the form post or get query
parameters are converted to python objects on the way in, or it will
redirect the request to an optional error handler method.
All this is covered in much more depth in the Writing Controller Methods methods doc.
At this point we’ve arrived at your controller code, and it’s run. The details of all of this are covered here: Writing Controller Methods
Hopefully this helps you understand the flow of the request through the stack, and gives you some hints on how you can modify or customize the stack to meet your needs.
For details on exactly how the stack is configured take a look at the configuration docs at TurboGears 2 Configuration.