SlideShare a Scribd company logo
Python             应
         2011.12
About Me
•          / hongqn

•   2002              Python

•   2004                  Python



•   2006                现
      构师

•   http://www.douban.com/
    people/hongqn/
Python and Douban
Python在豆瓣的应用
Python在豆瓣的应用
Python在豆瓣的应用
Python在豆瓣的应用
Python in Douban
协   发环
协   发环




+
协   发环




+    +
协           发环



    Face to Face
        +
+                  +
术团队
             动   发   术

     产   发

QA
动   发   术

     产   发

QA
Mako
Mako




• A very fast template engine
Mako




• A very fast template engine
• Mako IS Python
态               处
static():   动处       态      URL

     <link href="${static('/css/douban.css')}"
     rel="stylesheet" type="text/css">
态                  处
static():   动处         态        URL

     <link href="${static('/css/douban.css')}"
     rel="stylesheet" type="text/css">




  发环        http://dev.douban.com/css/douban.css
态                  处
static():   动处         态        URL

     <link href="${static('/css/douban.css')}"
     rel="stylesheet" type="text/css">




  发环        http://dev.douban.com/css/douban.css
线   环       http://img3.douban.com/css/packed_douban9922754105.css
联

...html code....
<script>
${istatic('/js/sns/tribe/manage_members.js')|n}
</script>
预编译
@import
预编译
@import         pyScss
动   发   术

     产   发

QA
APNS Agent
APNS Agent

•           应        Apple Push Notification
    Service (APNS)      讯
APNS Agent

•           应        Apple Push Notification
    Service (APNS)      讯

• gevent
APNS Agent

•           应        Apple Push Notification
    Service (APNS)      讯

• gevent
• bottle
APNS Agent

•           应        Apple Push Notification
    Service (APNS)      讯

• gevent
• bottle
• APNSWrapper
动   发   术

     产   发

QA
Quixote
Quixote

• Pretty old (we are using version 1.2)
Quixote

• Pretty old (we are using version 1.2)
• Simple
Quixote

• Pretty old (we are using version 1.2)
• Simple
• Fast
Quixote

• Pretty old (we are using version 1.2)
• Simple
• Fast
• Stable
Quixote

• Pretty old (we are using version 1.2)
• Simple
• Fast
• Stable
• Traversal based
Traversal
http://www.douban.com/group/python/members
Traversal
http://www.douban.com/group/python/members


         /
Traversal
http://www.douban.com/group/python/members


         /
             import luz.group

      group
Traversal
http://www.douban.com/group/python/members


         /
             import luz.group

      group
             _q_lookup(request, 'python')

      python
Traversal
http://www.douban.com/group/python/members


         /
             import luz.group

      group
             _q_lookup(request, 'python')

      python
             .members(request)

    members
Traversal
http://www.douban.com/group/python/members


         /
             import luz.group

      group
             _q_lookup(request, 'python')

      python
             ._q_access(request)
             .members(request)

    members
Traversal
http://www.douban.com/group/python/members


         /
             import luz.group

      group
                         raise DeletedGroupError
             _q_lookup(request, 'python')
Traversal
http://www.douban.com/group/python/members


         /
             import luz.group

      group                                  raise DeletedGroupError
             _q_lookup(request, 'python')   _q_exception_handler(request, e)
Traversal
http://www.douban.com/group/python/members


         /
             import luz.group

      group                                   raise DeletedGroupError
             _q_lookup(request, 'python')    _q_exception_handler(request, e)



                                            route based url dispatch
                                                          录级          进
Decorators

@need_post
@require_login
@check_switch('annotation')
@check_permission('admin')
@jsonize
def preview(self, request):
Generators
Generators
feed       feed     feed
Generators
feed         feed              feed

  iterator     iterator
                          iterator
Generators
feed          feed              feed

  iterator      iterator
                           iterator

             imerge

                iterator
Generators
feed          feed              feed

  iterator      iterator
                           iterator

             imerge

                iterator


             pager
                list
Meta Programming
class MetaPage(object):
    __metaclass__ = MetaObserver

class Vote(Commentable, MetaPage):
    kind = K_VOTE
    kind_name = 'vote'

assert MetaObserver.get_class_by_name('vote') is Vote
OneRing

   • Build Desktop
       Application Using Web
       Technology
   •   HTML5 + CSS + JS
   •   WebKit + Qt + Python
动   发   术

     产   发

QA
Restful MongoDB
         Service

• 给 业             储

• django-piston
• sleepy
动   发   术

     产   发

QA
pylint

         • 码 态检查
         • SQL inject
            检查

         • XSS   检查
onimaru
          • 错误
                统
          • Based on
           django-sentry
release manager



            • 动 tag
            • 认测试结
            •发       线报
            • django powered
动   发   术

     产   发

QA
cmy - MySQL Client
• MySQL 户       Python/C API编

•      select 语

•
•      访问时      python对

•        时 MySQLdb
C++ Intergration
    • cos_sim load_data.py
def load():
    for user, subject, rating in CmyResult(
            "select user_id, subject_id, rating from user_interest"):
        cos_sim_io.add_edge(user, subject, rating)




        Python 载
        C++计
动   发   术

     产   发

QA
Online Profiler
                   12164 function calls (12109 primitive calls) in 0.250 CPU seconds

Ordered by: internal time, call count
List reduced from 274 to 60 due to restriction <60>

ncalls   tottime     percall   cumtime   percall   filename:lineno(function)
  1000     0.098       0.000     0.098     0.000   cmemcached.py:22(restore)
    61     0.064       0.001     0.070     0.001   cmemcached.py:64(get)
    14     0.026       0.002     0.121     0.009   cmemcached.py:68(get_multi)
   908     0.010       0.000     0.017     0.000   decorator.py:22(gen_key)
   908     0.005       0.000     0.033     0.000   decorator.py:39(_)
    67     0.004       0.000     0.199     0.003   debug.py:26(_)
   898     0.004       0.000     0.004     0.000   format.py:57(_)
   953     0.003       0.000     0.095     0.000   cmemcached.py:70(<genexpr>)
     1     0.003       0.003     0.149     0.149   __init__.py:1866(get_loc_period_events)
   908     0.002       0.000     0.007     0.000   format.py:10(format)
     2     0.002       0.001     0.143     0.071   __init__.py:346(gets)
   963     0.002       0.000     0.075     0.000   wrapper.py:138(get)
    67     0.001       0.000     0.001     0.000   debug.py:36(format_call)
   878     0.001       0.000     0.001     0.000   __init__.py:1240(is_published)
     8     0.001       0.000     0.128     0.016   wrapper.py:146(get_multi)
   233     0.001       0.000     0.001     0.000   cgi.py:1031(escape)
DoubanService
•     thrift   RPC 务

•   PasteScript实现     码                   务

•   https://github.com/twitter/thrift_client
        Python 负载              failover

•   nagios munin       监
Dpark
•               计

•   Modeled after Spark, but using Python
•       mesos实现资 调


        dpark = DparkContext()
        rdd = dpark.textFile(filepath)

        def filter(line):
            return pattern in line

        for line in rdd.filter(filter).collect():
            sys.stdout.write(line)
pypcap + dpkt

        • Warm up
          MySQL slaves
        • Analyze MySQL
          issue with real-
          time queries
DAE
• Modeled after Google AppEngine, but for
  internal use only
• virtualenv to manage dependencies
• gunicorn + gevent
• Best practice built-in
• Developing...
编 语   in
编 语   in
编 语   in
编 语   in
编 语   in
编 语   in
编 语   in
编 语   in
Q &A
Thanks

More Related Content

PDF
Doxygen 사용법
PPTX
Neovim으로 생산성 퀀텀점프하기 by 이재열
PDF
How the Postgres Query Optimizer Works
 
PDF
Plotly dash and data visualisation in Python
PPT
14. Defining Classes
PDF
PostgreSQL 15 and its Major Features -(Aakash M - Mydbops) - Mydbops Opensour...
PDF
Webscale PostgreSQL - JSONB and Horizontal Scaling Strategies
PDF
Hearts Of Darkness - a Spring DevOps Apocalypse
Doxygen 사용법
Neovim으로 생산성 퀀텀점프하기 by 이재열
How the Postgres Query Optimizer Works
 
Plotly dash and data visualisation in Python
14. Defining Classes
PostgreSQL 15 and its Major Features -(Aakash M - Mydbops) - Mydbops Opensour...
Webscale PostgreSQL - JSONB and Horizontal Scaling Strategies
Hearts Of Darkness - a Spring DevOps Apocalypse

What's hot (11)

PDF
MariaDB ColumnStore
PPTX
APOC Pearls - Whirlwind Tour Through the Neo4j APOC Procedures Library
PDF
쿠키런 1년, 서버개발 분투기
PPTX
차곡차곡 쉽게 알아가는 Elasticsearch와 Node.js
PDF
Practical Object Oriented Models In Sql
PDF
Performance tuning ColumnStore
PDF
Hibernate start (하이버네이트 시작하기)
PDF
Using Optimizer Hints to Improve MySQL Query Performance
PDF
Binlog Servers 구축사례
PDF
Reproducible AI using MLflow and PyTorch
PDF
적당한 스터디 발표자료 만들기
MariaDB ColumnStore
APOC Pearls - Whirlwind Tour Through the Neo4j APOC Procedures Library
쿠키런 1년, 서버개발 분투기
차곡차곡 쉽게 알아가는 Elasticsearch와 Node.js
Practical Object Oriented Models In Sql
Performance tuning ColumnStore
Hibernate start (하이버네이트 시작하기)
Using Optimizer Hints to Improve MySQL Query Performance
Binlog Servers 구축사례
Reproducible AI using MLflow and PyTorch
적당한 스터디 발표자료 만들기
Ad

Viewers also liked (20)

PDF
Python高级编程(二)
PDF
豆瓣技术架构的发展历程 @ QCon Beijing 2009
PDF
Python于Web 2.0网站的应用 - QCon Beijing 2010
PDF
服务框架: Thrift & PasteScript
KEY
New Design of OneRing
PDF
OneRing @ OSCamp 2010
PDF
合久必分,分久必合
PDF
29c3 OpenBTS workshop - Mini-Workshop
PPT
How To Automate Part 1
PDF
Don Bailey - A Million Little Tracking Devices
PPTX
DAB+ for local and community radio
PPTX
Linux内存管理
PPT
Erlang高级原理和应用
PDF
Taobao图片存储与cdn系统到服务
PPT
Erlang及其应用
PDF
Inside Erlang Vm II
PPTX
Linux内存管理
PDF
LVS development and experience
PDF
Creating Excel files with Python and XlsxWriter
Python高级编程(二)
豆瓣技术架构的发展历程 @ QCon Beijing 2009
Python于Web 2.0网站的应用 - QCon Beijing 2010
服务框架: Thrift & PasteScript
New Design of OneRing
OneRing @ OSCamp 2010
合久必分,分久必合
29c3 OpenBTS workshop - Mini-Workshop
How To Automate Part 1
Don Bailey - A Million Little Tracking Devices
DAB+ for local and community radio
Linux内存管理
Erlang高级原理和应用
Taobao图片存储与cdn系统到服务
Erlang及其应用
Inside Erlang Vm II
Linux内存管理
LVS development and experience
Creating Excel files with Python and XlsxWriter
Ad

Similar to Python在豆瓣的应用 (20)

PDF
Python Load Testing - Pygotham 2012
PDF
Django at Scale
PDF
Django
KEY
PyCon AU 2012 - Debugging Live Python Web Applications
PDF
Pycon2017 instagram keynote
PDF
Django - 次の一歩 gumiStudy#3
PDF
gumiStudy#3 Django – 次の一歩
PDF
обзор Python
PPTX
Python at Facebook
PDF
Python for Penetration testers
PDF
Web Development with Python and Django
PPTX
Thrift+scribe实现分布式日志收集,并与log4j集成
PDF
Lego: A brick system build by scala
PDF
Python & Django TTT
PDF
Tools for Development and Debugging in Python
PDF
Chef on Python and MongoDB
PDF
Building Scalable Web Apps
KEY
Inside Of Mbga Open Platform
PDF
Python RESTful webservices with Python: Flask and Django solutions
PDF
Extensions on PostgreSQL
Python Load Testing - Pygotham 2012
Django at Scale
Django
PyCon AU 2012 - Debugging Live Python Web Applications
Pycon2017 instagram keynote
Django - 次の一歩 gumiStudy#3
gumiStudy#3 Django – 次の一歩
обзор Python
Python at Facebook
Python for Penetration testers
Web Development with Python and Django
Thrift+scribe实现分布式日志收集,并与log4j集成
Lego: A brick system build by scala
Python & Django TTT
Tools for Development and Debugging in Python
Chef on Python and MongoDB
Building Scalable Web Apps
Inside Of Mbga Open Platform
Python RESTful webservices with Python: Flask and Django solutions
Extensions on PostgreSQL

Recently uploaded (20)

PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
PPTX
Understanding_Digital_Forensics_Presentation.pptx
PDF
Diabetes mellitus diagnosis method based random forest with bat algorithm
PPTX
Big Data Technologies - Introduction.pptx
PDF
CIFDAQ's Market Insight: SEC Turns Pro Crypto
PDF
Approach and Philosophy of On baking technology
PPTX
A Presentation on Artificial Intelligence
PDF
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
PDF
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
PPTX
Cloud computing and distributed systems.
PDF
Chapter 3 Spatial Domain Image Processing.pdf
PDF
Empathic Computing: Creating Shared Understanding
PDF
Spectral efficient network and resource selection model in 5G networks
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
Understanding_Digital_Forensics_Presentation.pptx
Diabetes mellitus diagnosis method based random forest with bat algorithm
Big Data Technologies - Introduction.pptx
CIFDAQ's Market Insight: SEC Turns Pro Crypto
Approach and Philosophy of On baking technology
A Presentation on Artificial Intelligence
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
20250228 LYD VKU AI Blended-Learning.pptx
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
Agricultural_Statistics_at_a_Glance_2022_0.pdf
Dropbox Q2 2025 Financial Results & Investor Presentation
Mobile App Security Testing_ A Comprehensive Guide.pdf
Cloud computing and distributed systems.
Chapter 3 Spatial Domain Image Processing.pdf
Empathic Computing: Creating Shared Understanding
Spectral efficient network and resource selection model in 5G networks

Python在豆瓣的应用

Editor's Notes