python-csp: bringing OCCAM to Python
0 likes1,460 views
This document discusses the python-csp project, which brings the concepts of Communicating Sequential Processes (CSP) to Python. It covers the history of CSP from its origins in Tony Hoare's work, through OCCAM and Transputers. It then summarizes key features of python-csp like process creation, parallel and sequential processes, and communication through channels. Finally, it mentions future challenges for the project.
![Process creation
Ancient history Parallel, sequential and repeated processes
python-csp: features and idioms Communication via channel read / writes
Using built-in processes More channels: ALTing and choice
Future challenges More channels: poisoning
Producer / consumer or worker / farmer models
@process
d e f main ( IMSIZE ) :
channels = [ ]
# One p r o d u c e r + c h a n n e l f o r e a c h
image column .
f o r x i n r a n g e ( IMSIZE [ 0 ] ) :
c h a n n e l s . append ( C h a n n e l ( ) )
p r o c e s s e s . append ( m a n d e l b r o t ( x ,
. . . channels [ x ]) )
p r o c e s s e s . i n s e r t ( 0 , consume ( IMSIZE ,
channels ) )
mandel = Par ( ∗ p r o c e s s e s )
mandel . s t a r t ( )
Sarah Mount python-csp: bringing OCCAM to Python](https://image.slidesharecdn.com/pythoncsp-100727083606-phpapp02/85/python-csp-bringing-OCCAM-to-Python-27-320.jpg)
![Ancient history
python-csp: features and idioms
Using built-in processes
Future challenges
@process
def O s c i ll o s c o p e ( inchan ) :
# I n i t i a l i s e PyGame
ydata = [ ]
w h i l e n o t QUIT :
y d a t a . append ( i n c h a n . r e a d ( ) )
y d a t a . pop ( 0 )
# B l i t data to s c r e e n . . .
# Deal with e v e n t s
Sarah Mount python-csp: bringing OCCAM to Python](https://image.slidesharecdn.com/pythoncsp-100727083606-phpapp02/85/python-csp-bringing-OCCAM-to-Python-31-320.jpg)
![Ancient history
python-csp: features and idioms
Using built-in processes
Future challenges
from c s p . b u i l t i n s i m p o r t S i n
def t r a c e s i n () :
chans = Channel ( ) , Channel ( )
Par ( G e n e r a t e F l o a t s ( c h a n s [ 0 ] ) ,
Sin ( chans [ 0 ] , chans [ 1 ] ) ,
O s c i l l o s c o p e ( chans [ 1 ] ) ) . s t a r t ()
return
Sarah Mount python-csp: bringing OCCAM to Python](https://image.slidesharecdn.com/pythoncsp-100727083606-phpapp02/85/python-csp-bringing-OCCAM-to-Python-32-320.jpg)
![Ancient history
python-csp: features and idioms
Using built-in processes
Future challenges
def trace mux () :
chans = [ Channel ( ) f o r i i n range (6) ]
p a r = Par ( G e n e r a t e F l o a t s ( c h a n s [ 0 ] ) ,
Delta2 ( chans [ 0 ] , chans [ 1 ] ,
chans [ 2 ] ) ,
Cos ( c h a n s [ 1 ] , c h a n s [ 3 ] ) ,
Sin ( chans [ 2 ] , chans [ 4 ] ) ,
Mux2 ( c h a n s [ 3 ] , c h a n s [ 4 ] ,
chans [ 5 ] ) ,
O s c i l l o s c o p e ( chans [ 5 ] ) )
par . s t a r t ()
Sarah Mount python-csp: bringing OCCAM to Python](https://image.slidesharecdn.com/pythoncsp-100727083606-phpapp02/85/python-csp-bringing-OCCAM-to-Python-34-320.jpg)
python-csp: bringing OCCAM to Python
- 1. Ancient history python-csp: features and idioms Using built-in processes Future challenges python-csp: bringing OCCAM to Python Sarah Mount Europython 2010 Sarah Mount python-csp: bringing OCCAM to Python
- 2. Ancient history python-csp: features and idioms Using built-in processes Future challenges 1 Ancient history 2 python-csp: features and idioms Process creation Parallel, sequential and repeated processes Communication via channel read / writes More channels: ALTing and choice More channels: poisoning Producer / consumer or worker / farmer models 3 Using built-in processes 4 Future challenges Sarah Mount python-csp: bringing OCCAM to Python
- 3. Ancient history python-csp: features and idioms Using built-in processes Future challenges This talk. . . is all about the python-csp project. There is a similar project called PyCSP, these will merge over the next few months. Current code base is here: http://code.google.com/p/python-csp Please do contribute bug fixes / issues / code / docs / rants / . . . Sarah Mount python-csp: bringing OCCAM to Python
- 4. Ancient history python-csp: features and idioms Using built-in processes Future challenges INMOS B0042 Board c David May The Transputer was the original “multicore” machine and was built to run the OCCAM language – a realisation of CSP. http://www.cs.bris.ac.uk/~dave/transputer.html Sarah Mount python-csp: bringing OCCAM to Python
- 5. Ancient history python-csp: features and idioms Using built-in processes Future challenges Tony Hoare Invented CSP, OCCAM, Quicksort and other baddass stuff. You may remember him from such talks as his Europython 2009 Keynote. Sarah Mount python-csp: bringing OCCAM to Python
- 6. Ancient history python-csp: features and idioms Using built-in processes Future challenges OCCAM OCCAM lives on in it’s current implementation as OCCAM-π http://www.cs.kent.ac.uk/projects/ofa/kroc/ It can run on Lego bricks, Arduino boards and other things. Like Python it uses indentation to demark scope. Unlike Python OCCAM is MOSTLY IN UPPERCASE. Sarah Mount python-csp: bringing OCCAM to Python
- 7. Ancient history python-csp: features and idioms Using built-in processes Future challenges CSP and message passing This next bit will be revision for many of you! Sarah Mount python-csp: bringing OCCAM to Python
- 8. Ancient history python-csp: features and idioms Using built-in processes Future challenges Most of us think of programs like this http://xkcd.com/205/ Sarah Mount python-csp: bringing OCCAM to Python
- 9. Ancient history python-csp: features and idioms Using built-in processes Future challenges This is the old “standard” model! Multicore will soon take over the world (if it hasn’t already). Shared memory concurrency / parallelism is hard: Race hazards Deadlock Livelock Tracking which lock goes with which data and what order locks should be used. Operating Systems are old news! Sarah Mount python-csp: bringing OCCAM to Python
- 10. Ancient history python-csp: features and idioms Using built-in processes Future challenges Message passing Definition In message passing concurrency (or parallelism) “everything” is a process and no data is shared between processes. Instead, data is “passed” between channels which connect processes together. Think: OS processes and UNIX pipes. The actor model (similar to Kamaelia) uses asynchronous channels. Processes write data to a channel and continue execution. CSP (and π-calculus) use synchronous channels, or rendezvous, where a process writing to a channel has to wait until the value has been read by another process before proceeding. Sarah Mount python-csp: bringing OCCAM to Python
- 11. Ancient history python-csp: features and idioms Using built-in processes Future challenges Process diagram Sarah Mount python-csp: bringing OCCAM to Python
- 12. Ancient history python-csp: features and idioms Using built-in processes Future challenges Why synchronous channels? Leslie Lamport (1978). Time, clocks, and the ordering of events in a distributed system”. Communications of the ACM 21 (7) Sarah Mount python-csp: bringing OCCAM to Python
- 13. Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models Hello World! from c s p . c s p p r o c e s s i m p o r t ∗ @process def h e l l o () : p r i n t ’ H e l l o CSP w o r l d ! ’ hello () . start () Sarah Mount python-csp: bringing OCCAM to Python
- 14. Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models Example of process creation: web server @process def s e r v e r ( host , port ) : sock = socket . socket ( . . . ) # S e t up s o c k e t s w h i l e True : conn , a d d r = s o c k . a c c e p t ( ) r e q u e s t = conn . r e c v ( 4 0 9 6 ) . s t r i p ( ) i f r e q u e s t . s t a r t s w i t h ( ’GET ’ ) : ok ( r e q u e s t , conn ) . s t a r t ( ) else : n o t f o u n d ( r e q u e s t , conn ) . s t a r t ( ) Sarah Mount python-csp: bringing OCCAM to Python
- 15. Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models Example of process creation: web server @process d e f n o t f o u n d ( r e q u e s t , conn ) : page = ’<h1>F i l e n o t found </h1> ’ conn . s e n d ( r e s p o n s e ( 4 0 4 , ’ Not Found ’ , page ) ) conn . shutdown ( s o c k e t . SHUT RDWR) conn . c l o s e ( ) return Sarah Mount python-csp: bringing OCCAM to Python
- 16. Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models Combining processes: in parallel # With a Par o b j e c t : Par ( p1 , p2 , p3 , . . . pn ) . s t a r t ( ) # With s y n t a c t i c s u g a r : p1 // ( p2 , p3 , . . . pn ) # RHS must be a s e q u e n c e t y p e . Sarah Mount python-csp: bringing OCCAM to Python
- 17. Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models Combining processes: repeating a process sequentially @process def h e l l o () : p r i n t ’ H e l l o CSP w o r l d ! ’ if name == ’ m a i n ’ : hello () ∗ 3 2 ∗ hello () Sarah Mount python-csp: bringing OCCAM to Python
- 18. Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models Channels @process d e f c l i e n t ( chan ) : p r i n t chan . r e a d ( ) @process d e f s e r v e r ( chan ) : chan . w r i t e ( ’ H e l l o CSP w o r l d ! ’ ) if name == ’ m a i n ’ : ch = C h a n n e l ( ) Par ( c l i e n t ( ch ) , s e r v e r ( ch ) ) . s t a r t ( ) Sarah Mount python-csp: bringing OCCAM to Python
- 19. Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models What you need to know about channels Channels are currently UNIX pipes This will likely change Channel rendezvous is “slow” compared with JCSP: 200µ s vs 16µ s This will be fixed by having channels written in C Because channels are, at the OS level, open “files”, there can only be a limited number of them (around 300 on my machine) This makes me sad :-( Sarah Mount python-csp: bringing OCCAM to Python
- 20. Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models Message passing an responsiveness This isn’t about speed. Sometimes, you have several channels and reading from them all round-robin may reduce responsiveness if one channel read blocks for a long time. This is BAD: @process def foo ( channels ) : f o r chan i n c h a n n e l s : p r i n t chan . r e a d ( ) Sarah Mount python-csp: bringing OCCAM to Python
- 21. Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models ALTing and choice Definition ALTing, or non-deterministic choice, selects a channel to read from from those channels which are ready to read from. You can do this with (only) two channels: @process d e f f o o ( c1 , c2 ) : p r i n t c1 | c2 p r i n t c1 | c2 Sarah Mount python-csp: bringing OCCAM to Python
- 22. Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models ALTing proper (many channels) @process d e f f o o ( c1 , c2 , c3 , c4 , c5 ) : a l t = A l t ( c1 , c2 , c3 , c4 , c5 ) # Choose random a v a i l a b l e o f f e r print alt . se le ct () # Avoid l a s t s e l e c t e d channel print alt . f a i r s e l e c t () # Choose c h a n n e l w i t h l o w e s t i n d e x print alt . p r i s e l e c t () Sarah Mount python-csp: bringing OCCAM to Python
- 23. Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models Syntactic sugar for ALTing @process d e f f o o ( c1 , c2 , c3 , c4 , c5 ) : gen = A l t ( c1 , c2 , c3 , c4 , c5 ) p r i n t gen . n e x t ( ) p r i n t gen . n e x t ( ) p r i n t gen . n e x t ( ) p r i n t gen . n e x t ( ) p r i n t gen . n e x t ( ) Sarah Mount python-csp: bringing OCCAM to Python
- 24. Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models ALTing: when you really, really must return right now @process d e f f o o ( c1 , c2 , c3 ) : # Skip () w i l l / always / complete # a read immediately gen = A l t ( c1 , c2 , c3 , S k i p ( ) ) p r i n t gen . n e x t ( ) ... Sarah Mount python-csp: bringing OCCAM to Python
- 25. Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models Channel poisoning Definition Idiomatically in this style of programming most processes contain infinite loops. Poisoning a channel asks all processes connected to that channel to shut down automatically. Each process which has a poisoned channel will automatically poison any other channels it is connected to. This way, a whole graph of connected processes can be terminated, avoiding race conditions. @process def foo ( channel ) : ... channel . poison () Sarah Mount python-csp: bringing OCCAM to Python
- 26. Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models A bigger example: Mandelbrot generator Sarah Mount python-csp: bringing OCCAM to Python
- 27. Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models @process d e f main ( IMSIZE ) : channels = [ ] # One p r o d u c e r + c h a n n e l f o r e a c h image column . f o r x i n r a n g e ( IMSIZE [ 0 ] ) : c h a n n e l s . append ( C h a n n e l ( ) ) p r o c e s s e s . append ( m a n d e l b r o t ( x , . . . channels [ x ]) ) p r o c e s s e s . i n s e r t ( 0 , consume ( IMSIZE , channels ) ) mandel = Par ( ∗ p r o c e s s e s ) mandel . s t a r t ( ) Sarah Mount python-csp: bringing OCCAM to Python
- 28. Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models @process def mandelbrot ( xcoord , cout ) : # Do some maths h e r e cout . w r i t e ( xcoord , column data ) Sarah Mount python-csp: bringing OCCAM to Python
- 29. Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models @process d e f consumer ( c i n s ) : # I n i t i a l i s e PyGame . . . gen = l e n ( c i n s ) ∗ A l t ( ∗ c i n s ) f o r i in range ( len ( c i n s ) ) : x c o o r d , column = gen . n e x t ( ) # B l i t t h a t column t o s c r e e n f o r e v e n t i n pygame . e v e n t . g e t ( ) : i f e v e n t . t y p e == pygame . QUIT : for channel in cins : channel . poison () pygame . q u i t ( ) Sarah Mount python-csp: bringing OCCAM to Python
- 30. Ancient history python-csp: features and idioms Using built-in processes Future challenges Oscilloscope Sarah Mount python-csp: bringing OCCAM to Python
- 31. Ancient history python-csp: features and idioms Using built-in processes Future challenges @process def O s c i ll o s c o p e ( inchan ) : # I n i t i a l i s e PyGame ydata = [ ] w h i l e n o t QUIT : y d a t a . append ( i n c h a n . r e a d ( ) ) y d a t a . pop ( 0 ) # B l i t data to s c r e e n . . . # Deal with e v e n t s Sarah Mount python-csp: bringing OCCAM to Python
- 32. Ancient history python-csp: features and idioms Using built-in processes Future challenges from c s p . b u i l t i n s i m p o r t S i n def t r a c e s i n () : chans = Channel ( ) , Channel ( ) Par ( G e n e r a t e F l o a t s ( c h a n s [ 0 ] ) , Sin ( chans [ 0 ] , chans [ 1 ] ) , O s c i l l o s c o p e ( chans [ 1 ] ) ) . s t a r t () return Sarah Mount python-csp: bringing OCCAM to Python
- 33. Ancient history python-csp: features and idioms Using built-in processes Future challenges Process diagram Sarah Mount python-csp: bringing OCCAM to Python
- 34. Ancient history python-csp: features and idioms Using built-in processes Future challenges def trace mux () : chans = [ Channel ( ) f o r i i n range (6) ] p a r = Par ( G e n e r a t e F l o a t s ( c h a n s [ 0 ] ) , Delta2 ( chans [ 0 ] , chans [ 1 ] , chans [ 2 ] ) , Cos ( c h a n s [ 1 ] , c h a n s [ 3 ] ) , Sin ( chans [ 2 ] , chans [ 4 ] ) , Mux2 ( c h a n s [ 3 ] , c h a n s [ 4 ] , chans [ 5 ] ) , O s c i l l o s c o p e ( chans [ 5 ] ) ) par . s t a r t () Sarah Mount python-csp: bringing OCCAM to Python
- 35. Ancient history python-csp: features and idioms Using built-in processes Future challenges Wiring Sarah Mount python-csp: bringing OCCAM to Python