Materialized views and django
Download as PPTX, PDF3 likes4,869 views
The document discusses using materialized views in PostgreSQL to improve query performance on large, growing datasets with complex joins. It describes what materialized views are, how to create them from Django queries, and a technique to manually format the queries and parameters to create the view. It also discusses updating and refreshing the view, creating a base and materialized model in Django, and using South migrations to update the view when the base data changes.
![What are materialized views
a materialized view is a
database object that contains the
results of a query
REFRESH MATERIALIZED VIEW [ CONCURRENTLY ]
name
[ WITH [ NO ] DATA ]
Updating the view is
simple](https://image.slidesharecdn.com/materializedviewsanddjango-150414011429-conversion-gate01/85/Materialized-views-and-django-3-320.jpg)
![Creating a view from django
CREATE MATERIALIZED VIEW table_name
[ (column_name [, ...] ) ]
[ WITH ( storage_parameter [= value] [, ... ] ) ]
[ TABLESPACE tablespace_name ]
AS query
[ WITH [ NO ] DATA ]
How do I get this
From this?
Car.objects.filter(
manufacturer__some_complicated_tbl__some_
crazy_value=10,
engine__valve__name=’Steam’
)](https://image.slidesharecdn.com/materializedviewsanddjango-150414011429-conversion-gate01/85/Materialized-views-and-django-4-320.jpg)
![The Dirty Trick
Part 1 – Creating the view
raw_query, params = qs.query.sql_with_params()
As django rely on the driver to format the query
and parameters, we have to manually do it
ourselves
Reformat the parameters
params = tuple([("'%s'" % unicode(p).replace('%', '%%'))
for p in params])
Get the SQL query and parameters from query
set](https://image.slidesharecdn.com/materializedviewsanddjango-150414011429-conversion-gate01/85/Materialized-views-and-django-5-320.jpg)
Materialized views and django
- 2. The Problem A real time query with growing data Doing a lot of joins including to itself Performance was degrading with data increase The solution …
- 3. What are materialized views a materialized view is a database object that contains the results of a query REFRESH MATERIALIZED VIEW [ CONCURRENTLY ] name [ WITH [ NO ] DATA ] Updating the view is simple
- 4. Creating a view from django CREATE MATERIALIZED VIEW table_name [ (column_name [, ...] ) ] [ WITH ( storage_parameter [= value] [, ... ] ) ] [ TABLESPACE tablespace_name ] AS query [ WITH [ NO ] DATA ] How do I get this From this? Car.objects.filter( manufacturer__some_complicated_tbl__some_ crazy_value=10, engine__valve__name=’Steam’ )
- 5. The Dirty Trick Part 1 – Creating the view raw_query, params = qs.query.sql_with_params() As django rely on the driver to format the query and parameters, we have to manually do it ourselves Reformat the parameters params = tuple([("'%s'" % unicode(p).replace('%', '%%')) for p in params]) Get the SQL query and parameters from query set
- 6. Execute the query sql = ("DROP MATERIALIZED VIEW IF EXISTS table_materialized;n“ "CREATE MATERIALIZED VIEW table_materialized ASn"+ raw_query) cursor = connection.cursor() cursor.execute(sql) raw_query = raw_query.replace('%s','{}') .format(*params) .replace('%', '%%') Reformat the query
- 7. The Dirty Trick Part 2 – django model Base Model Class BaseCar(Model): class Meta: abstract = True total_km = models.IntegerField(default=0) num_of_windows = models.IntegerField("Num of Windows", default=6) …
- 8. Real Model Class Car(BaseCar): @classmethod def create_materialized(cls): … @classmethod def refresh_materialized(cls): sql = "REFRESH MATERIALIZED VIEW car_materialized“ cursor = connection.cursor() cursor.execute(sql)
- 9. Materialized Model Class MaterializedCar(Model): class Meta: db_table = ‘car_materialized‘ managed = False last_day_km = models.IntegerField(default=0) …
- 10. South Migration If the base data changes we need to remember to update the view. ./manage.py schemamigration roadtripapp update_materialized_car –empty Call create_materialized in the forward method
- 11. Thank You XtremIO is looking for: Python developers for backend Angular developers for frontend