Architecture at PBS
Download as PPTX, PDF3 likes1,164 views
PBS operates as a federation of public television stations with a focus on technology leadership and development of core applications. The document details the architecture of PBS.org, including the Cove API and Merlin API, highlighting their technology stacks and best practices. It also discusses lessons learned in system management, data handling, and scaling using AWS infrastructure.
![API - Piston/Haystack/Solr
class WebObjectIndexHandler(BaseHandler):
...
def get_queryset(self):
...
return PistonSearchQuerySet().models(*models)
from haystack.query import SearchQuerySet
class PistonSearchQuerySet(SearchQuerySet):
...
def __getitem__(self, k):
...
return [IndexSerializer(i) for i in
super(PistonSearchQuerySet, self).__getitem__(k)]](https://image.slidesharecdn.com/dcpython-pbs-20110607-120205204632-phpapp02/85/Architecture-at-PBS-20-320.jpg)
Architecture at PBS
- 1. Architecture of PBS.org DCPython - June 7, 2011
- 2. PBS is… • PBS is a national federation of independently owned and operated public television stations and producers – Each with their own management and development resources • 1500+ highly trafficked websites: – http://www.pbs.org/ – http://www.pbs.org/nova/ – http://pbskids.org/ – http://pbskids.org/sesame/ – http://video.pbs.org/ • Enterprise services/APIs
- 3. PBS is not! • Radio is easy… We do television! • Or any of the other ~200 local stations.
- 4. What we do • Technology leadership within public broadcasting community • Distribution of national programming content • Services to local stations • Core application development. Yeah!!!
- 5. A few of our sites
- 6. History of PBS.org Early 1990’s: Hand rolled static html Late 1990’s: Hand crafted static html + CGI! Most of 2000’s: Zope/Plone CMS generated static html 2008-10: Django generated static html Launched Oct 2010: Django all the way
- 7. COVE API • Contains the metadata for all PBS videos online including pointers to streaming video • Needed to be: – Secure – Fast – Scalable
- 8. COVE API – Technology Stack • Amazon Elastic Cluster Computing (EC2) • Amazon Relational Database Service (RDS) • Linux • Python • Django • Piston for REST API
- 9. COVE API - Architecture Internet Elastic Load Balancer Auto Scale Array App Server 1 … App Server N HA Proxy S3 RDS Master RDS Slave 11 RDS Slave RDS Slave 1 Backups App Sync Server
- 10. COVE API – Management Tools • Amazon Web Service Console • RightScale • Splunk
- 11. COVE API – Interesting Stuff • Easy to load test – Duplicate environment for several days • Easy to scale – Autoscale array grows automatically • Easy to upgrade – Each server built from vanilla base
- 12. COVE API – Lessons learned • Use normalized data for administration and de- normalized data for API
- 13. COVE API – Lessons learned • Piston is fine, but lacks flexibility without significant customization – TastyPie? • JSON is probably good enough • Don’t get fancy with your endpoints • Stick to REST principles • Don’t get fancy with your authentication – Use OAuth2 or simple token
- 14. PBS.org and Merlin API • PBS.org – Slim, fast layer – Pulls data from Merlin API – Uses memcache extensively – Currently Django, but could be anything (Flask?) • Merlin API – Aggregate content from distributed CMSes – Expose via standardized API – Power PBS.org and more
- 15. Merlin API – Technology stack • Python • Amazon Web Services (“cloud”) • Django – EC2 • MySQL – RDS - Relational Database Service – ELB - Elastic Load Balancing • Piston – Cloudfront CDN • Solr – S3Storage • Celery • RabbitMQ
- 16. Data flow RSS Feed Standardized Ingestor API
- 17. Merlin API architecture API Endpoint – Django Piston Search service Indexing service Django-haystack Solr Data layer – MySQL (RDS) Administration Feed ingestion Django admin Celery
- 18. Merlin API server topology Internet Elastic Load Balancer Autoscaling App #N App #N Solr Master array App #N Celery App #n Index DB RDS S3 backups
- 19. Merlin API – Management Tools • Amazon Web Service Console • RightScale • Splunk
- 20. API - Piston/Haystack/Solr class WebObjectIndexHandler(BaseHandler): ... def get_queryset(self): ... return PistonSearchQuerySet().models(*models) from haystack.query import SearchQuerySet class PistonSearchQuerySet(SearchQuerySet): ... def __getitem__(self, k): ... return [IndexSerializer(i) for i in super(PistonSearchQuerySet, self).__getitem__(k)]
- 21. Feed ingestor - Celery from celery.decorators import task, periodic_task @periodic_task(run_every=timedelta(seconds=300)) def update_webobject_states(): ... solr_visible = WebObject.children.filter(visible=True) solr_visible = solr_visible.exclude( flag__api_visible=True, available__isnull=True) ... updated = solr_visible.update(visible=False, is_indexed = False) ... signals.bulk_update.send('tasks.update_webobject_states')
- 22. Merlin API - Lessons learned • Memcached was not necessary • Denormalized search data via Solr index is much faster than querying database • Asynchronous task delegation is awesome • Celery prone to memory leaks • App server array for easy horizontal scaling – Even if not autoscaling, increase min servers • Never trust data you don’t control (validate!)
- 23. Resources • http://lucene.apache.org/solr/ • http://haystacksearch.org/ • http://celeryproject.org/ • http://celeryproject.org/docs/django-celery/ • http://aws.amazon.com/
- 24. PBS Developer Community • Dedicated to making open.PBS the industry standard in open development communities. http://open.pbs.org/ https://github.com/pbs open@pbs.org
- 25. Questions? Drew Engelson drew@engelson.net http://tomatohater.com Edgar Roman emroman@pbs.org