From SQLAlchemy to Ming with TurboGears2

From SQLAlchemy to Ming with
TurboGears2

Alessandro Molina
alessandro.molina@axant.it
@__amol__

Simple, Stupid, Serviceable
● We wanted to offer to development team
members the opportunity to take a first look at
MongoDB.
● We wanted to do this by letting them write an
actual web app using Python.
● We wanted a real use case: “expenses and
incomings” tracking (serviceable)
● We wanted everyone to understand how it is
expected to work (stupid)
● Must be blazing fast to implement, we don't have
a lot of time (simple)

Involved Technologies

● Python → http://www.python.org
● TurboGears2 → http://www.turbogears.org
● SQLAlchemy → http://www.sqlalchemy.org
● Ming → http://merciless.sourceforge.net/index.html
● Genshi → http://genshi.edgewall.org
● ToscaWidgets → http://toscawidgets.org
● Sprox → http://www.sprox.org

Python Technologies
● Why? Needed a WebApp ready in 5 minutes while being
possible to quickly play with the code
● Python: Our team is manly python developers based
● TurboGears2: Our team is used to TG. Object Dispatch is really
comfortable and the framework is incredibly fast to start with
while still making possible to quickly get under the hood.
● Genshi: clean and plain xhtml, even designers can understand it,
also it came by default
● ToscaWidgets: Can do most of the HTML for us
● Sprox: Can do most of the CRUD for us

Quickstarting Our Project
(tg2dev)Quasar-2:tuts amol$ paster quickstart mt_sqla
● Project Structure Enter package name [mt_sqla]: paster quickstart mt_sqla
(tg2dev)Quasar-2:tuts amol$
Enter package name [mt_sqla]:
Would you prefer mako templates? (yes/[no]):
Do you need prefer mako templates? (yes/[no]):
Would you authentication and authorization in this project? ([yes]/no):

with Controllers Selected need implied templates: authorization in this project? ([yes]/no):
Do you and authentication and
Selected and implied templates:
tg.devtools#turbogears2 TurboGears 2. Standard Quickstart Template

for Index Page (tg2dev)Quasar-2:mt-sqla amol$ python setup.py develop
(tg2dev)Quasar-2:mt-sqla amol$ python setup.py develop
paster srunning develop
Installed /Users/amol/wrk/tuts/mt-sqla
Installed /Users/amol/wrk/tuts/mt-sqla

● Models for Users, (tg2dev)Quasar-2:mt-sqla amol$ paster setup-app development.ini
(tg2dev)Quasar-2:mt-sqla amol$ paster setup-app development.ini
Running setup_config() from mt_sqla.websetup
Running setup_config() from mt_sqla.websetup
Creating tables

Groups and
Creating tables
(tg2dev)Quasar-2:mt-sqla amol$ paster serve development.ini
(tg2dev)Quasar-2:mt-sqla amol$ paster serve development.ini
Starting server in PID 23939.

Permission
serving on server in PID 23939.
Starting http://127.0.0.1:8080
serving on http://127.0.0.1:8080

● Authentication (tg2dev)Quasar-2:tuts amol$ paster quickstart –ming mt_ming
Enter package name [mt_ming]: paster quickstart –ming mt_ming
(tg2dev)Quasar-2:tuts amol$
Enter package name [mt_ming]:

and Authorization
Would you prefer mako templates? (yes/[no]):
Do you need prefer mako templates? (yes/[no]):
Would you authentication and authorization in this project? ([yes]/no):
Selected need implied templates: authorization in this project? ([yes]/no):
Do you and authentication and
Selected and implied templates:

● Both on SQLA (tg2dev)Quasar-2:mt-ming amol$ python setup.py develop
(tg2dev)Quasar-2:mt-ming amol$ python setup.py develop
Installed /Users/amol/wrk/tuts/mt-ming

and Ming
Installed /Users/amol/wrk/tuts/mt-ming
(tg2dev)Quasar-2:mt-ming amol$ paster setup-app development.ini
(tg2dev)Quasar-2:mt-ming amol$ paster setup-app development.ini
Running setup_app() from mt_ming.websetup
Running setup_app() from mt_ming.websetup
(tg2dev)Quasar-2:mt-ming amol$ paster serve development.ini
(tg2dev)Quasar-2:mt-ming amol$ paster serve development.ini
Starting server in PID 23939.
serving on server in PID 23939.
Starting http://127.0.0.1:8080
serving on http://127.0.0.1:8080

Overview

Browser Genshi

ToscaWidgets
TurboGears

Controller Code Sprox (TGAdmin)

Ming SQLAlchemy

SQLAlchemy
● Supports pratically everything: PostgresSQL,
MySQL, SQLite, Firebird, Oracle, MSSQL, Sybase,
DB2, Informix, SAPDB, MSAccess
● Declarative ORM layer with access to a powerful and
flexible query builder
● Unlike many tools, it never enforces schemas or
relies on naming conventions of any kind.
● Unit Of Work system organizes pending
insert/update/delete operations into queues and
flushes them all in one batch (Patterns of Enterprise
Application Architecture - Martin Fowler)

Python MongoDB Ecosystem
● PyMongo: Driver for MongoDB, everything is a dictionary
● MongoEngine: ORM, query language is familiar to Django users
● Ming: ORM, query language is familiar to SQLAlchemy users
● MongoKit: ORM, written to be as simple and light as possible

MongoKit Ming
class BlogPost(Document): class WikiComment(MappedClass):
structure = { class __mongometa__:
'title':unicode, session = session
'body':unicode, name = 'wiki_comment'
'author':unicode,
'date_creation':datetime.datetime, _id = FieldProperty(schema.ObjectId)
'rank':int text=FieldProperty(str, if_missing='')
} page_id = ForeignIdProperty('WikiPage')
page=RelationProperty('WikiPage')
tutorial.BlogPost.find({'body': {'$exists': True}})
WikiComment.query.find({'text':{'$exists':True}})

MongoEngine PyMongo
class User(Document): post = {"author": "Mike",
email = StringField(required=True) ... "text": "My first blog post!",
first_name = StringField(max_length=50) ... "tags": ["mongodb", "python"],
last_name = StringField(max_length=50) ... "date": datetime.datetime.utcnow()}

Users.objects(age__lte=18) db.posts.find({"author": "Mike"})

Ming
● SQLAlchemy inspired, easy to catch up if you are
used to SQLAlchemy
● Declarative ORM layer, use ORM through objects
and get direct access to mongo through collections
● Integrated Migrations (deprecated attributes and
even on-demand lazy migration code)
● Unit of Work, we tried it, we loved it, we wanted it
even on MongoDB
● Mongo In Memory, complete implementation of
MongoDB in memory for unit testing

Some Little Magic (TGAdmin)

● It did CRUD for us.
● Works both with SQLAlchemy and Ming
● Comes for free with TurboGears2
● Sprox based, deeply customizable

Storing our Data
from sqlalchemy import Table, Column from ming import schema as s
from sqlalchemy import Table, Column
from sqlalchemy.types import Unicode, Integer, from ming import schema as s
from ming.orm import FieldProperty, Mapper
Float, sqlalchemy.types import Unicode, Integer,
from DateTime from ming.orm import FieldProperty, Mapper
from ming.orm.declarative import MappedClass
Float, DateTime
from datetime import datetime from ming.orm.declarative import MappedClass
from session import DBSession
from datetime import datetime from session import DBSession
from datetime import datetime
from datetime import datetime
class MoneyTransfer(DeclarativeBase): class MoneyTransfer(MappedClass):
class MoneyTransfer(DeclarativeBase):
__tablename__ = 'money_transfers' class MoneyTransfer(MappedClass):
class __mongometa__:
__tablename__ = 'money_transfers' class __mongometa__:
session = DBSession
session = DBSession
name = 'money_transfers'
name = 'money_transfers'
uid = Column(Integer, autoincrement=True, _id = FieldProperty(s.ObjectId)
uid = Column(Integer,primary_key=True)
autoincrement=True, _id = FieldProperty(s.ObjectId)
date = FieldProperty(s.DateTime,
primary_key=True)
date = Column(DateTime, default=datetime.now) date = FieldProperty(s.DateTime,
if_missing=datetime.now)
sign = =Column(Integer, default=1)
date Column(DateTime, default=datetime.now) if_missing=datetime.now)
sign = FieldProperty(s.Int, if_missing=1)
sign = Column(Integer, default=1)
description = Column(Unicode(255)) sign = FieldProperty(s.Int, if_missing=1)
description = FieldProperty(s.String)
description = Column(Unicode(255))
value = Column(Float, default=0) description = FieldProperty(s.String)
value = FieldProperty(s.Float, if_missing=0)
value = Column(Float, default=0) value = FieldProperty(s.Float, if_missing=0)

Retrieving and Displaying Data
from tw.forms import DataGrid from tw.forms import DataGrid
from tw.forms.datagrid DataGrid
from tw.forms import import Column from tw.forms.datagrid DataGrid
from tw.forms import import Column
from tw.forms.datagrid import Column from tw.forms.datagrid import Column
transactions_grid = DataGrid(fields=[ transactions_grid = DataGrid(fields=[
transactions_grid 'date'),
Column('Date', = DataGrid(fields=[ transactions_grid 'date'),
Column('Date', = DataGrid(fields=[
Column('Date', 'date'),
Column('Description', 'description'), Column('Date', 'date'),
Column('Description', 'description'),
Column('Description', 'description'),
Column('Value', lambda row:row.value*row.sign)]) Column('Description', 'description'),
Column('Value', lambda row:row.value*row.sign)])
Column('Value', lambda row:row.value*row.sign)]) Column('Value', lambda row:row.value*row.sign)])
class RootController(BaseController): class RootController(BaseController):
class RootController(BaseController):
@expose('mt_sqla.templates.index') class RootController(BaseController):
@expose('mt_ming.templates.index')
@expose('mt_sqla.templates.index')
def index(self): @expose('mt_ming.templates.index')
def index(self):
def index(self): DBSession.query(MoneyTransfer).all()
transactions = def index(self): MoneyTransfer.query.find().all()
transactions =
transactions = DBSession.query(MoneyTransfer).all()
total = sum((t.value*t.sign for t in transactions)) transactions = MoneyTransfer.query.find().all()
total = sum((t.value*t.sign for t in transactions))
total = sum((t.value*t.sign for t in transactions))
return dict(page='index', total = sum((t.value*t.sign for t in transactions))
return dict(page='index',
return dict(page='index',
transactions=transactions, return dict(page='index',
transactions=transactions,
transactions=transactions,
transactions_grid=transactions_grid, transactions=transactions,
transactions_grid=transactions_grid,
transactions_grid=transactions_grid,
total=total) transactions_grid=transactions_grid,
total=total)
total=total) total=total)

<div style="width:650px; margin-top:20px;">
<div style="width:650px; margin-top:20px;">
<a href="${tg.url('/admin/moneytransfers/new')}">Add New Transaction</a>
<a href="${tg.url('/admin/moneytransfers/new')}">Add New Transaction</a>
${transactions_grid(transactions)}
${transactions_grid(transactions)}
<div style="text-align:right;"><strong>AVAILABLE:</strong> ${total}</div>
<div style="text-align:right;"><strong>AVAILABLE:</strong> ${total}</div>
</div>
</div>

What changed
● Template remained the same. Thanks god we
have an ORM.
● Had to change the model.
● Had to change just 1 line of controller code to
retrieve the data. Ming and SQLA are similar
enough
● Did we really need to change that line? We just
want to get a list of objects...

Sprox, abstraction over abstraction
● ORMProvider, provides an abstraction over the ORM
● ORMProviderSelector, automatically detects the provider to
use from a model.
● Mix those together and you have a db independent layer.
● Provider.query(self, entity, limit=None, offset=0, limit_fields=None,
order_by=None, desc=False) → get all objects of a collection
● Provider.get_obj(self, entity, params) → get an object
● Provider.update(self, entity, params) → update an object
● Provider.create(self, entity, params) → create a new object
SQLAlchemy
Sprox
count, transactions = provider.query(MoneyTransfer)
Ming count, transactions = provider.query(MoneyTransfer)

transactions = MoneyTransfer.query.find().all()
transactions = MoneyTransfer.query.find().all()

Experiments Experience

● Starting with a really simple use case made people
get comfortable with MongoDB and feel confident
enough to start learning more complex features.
● The idea of being possible to use sprox to abstract
over the db, making possible to switch back anytime,
created a “safety net” idea in people.
● When people feel safe they start experimenting a lot
more and learn by themselves.

From SQLAlchemy to Ming with TurboGears2

More Related Content

What's hot (20)

Similar to From SQLAlchemy to Ming with TurboGears2 (20)

More from Alessandro Molina (12)

Recently uploaded (20)

From SQLAlchemy to Ming with TurboGears2