How we do python
- 1. How We Do Python Python, a way © BestofMedia Grenoble, Science Team
- 2. ● Python Runtime ● Development environment ○ bom_utils ○ Zero-Application-Server ○ AWS ● Packaging ● CI ● Deployment How does the science team speak python? © BestofMedia Grenoble, Science Team
- 3. ● VirtualEnv vs GlobalEnv ● GlobalEnv ○ tainted on every install ○ high risk of base installation instability (python is highly used as system scripting) ● VirtualEnv & PYTHONPATH ○ Sandboxed: don't break the box ○ Can be legion Python Runtime © BestofMedia Grenoble, Science Team
- 4. ● Ubuntu/Debian box with python2.6 ● Targeted sci-env ● Eclipse with pydev Et voilà! Development environment © BestofMedia Grenoble, Science Team
- 5. ● timothy (easy hadoop using dumbo - to be open-sourced) ● common libs ○ bss (our proprietary documents warehouse) ○ ldap ○ mail ○ ... bom_utils © BestofMedia Grenoble, Science Team
- 6. ● Boto (aws python lib) encapsulated into bom_utils ○ And now for some names dropping: ■ SNS, SQS, S3, EC2, EMR, CloudWatch AWS © BestofMedia Grenoble, Science Team
- 7. ● One Application to serve them all ● Automatic logging configuration ● On AWS: ○ Automatic Monitoring & alerting ● to be open-sourced Zero-Application-Server © BestofMedia Grenoble, Science Team
- 8. ● Once, there was Maven ○ easy to put to Jenkins/Hudson ○ BOM was all Maven world ○ but: ■ Verbose ■ not really suited for python ■ Heavy & sloooooooow ● Then, bdist-deb came to the rescue ○ pythonic ○ light ○ just a line of freestyle Jenkins job ○ to be open-sourced Packaging © BestofMedia Grenoble, Science Team
- 9. ● One commit job, one release job ● unittest, pylint, coverage CI © BestofMedia Grenoble, Science Team
- 10. ● One project: ○ one egg ○ one deb (can contain other eggs, like projects shipping their version of internal libs) ● Deb deployed using old internal conf server ○ migration to our new in house system in the pipe Deployment © BestofMedia Grenoble, Science Team
- 11. Any more questions? © BestofMedia Grenoble, Science Team