Concurrent Programming in Python
0 likes383 views
This document discusses concurrent programming in Python. It covers threading, processes, subprocesses, and concurrent frameworks. Threading uses thread-local data, locks, and events to coordinate between threads. Processes allow spawning child processes and interprocess communication. Subprocesses run other scripts. Concurrent frameworks like Curio use futures to allow asynchronous function programming. The document provides examples of using processes and subprocesses in Python and recommends concurrent frameworks for all but simple applications.
![@adam_englander
Subprocess Example
import subprocess
print("nnManaged exec error")
p = subprocess.Popen(
[“/not/a/valid/script"],
shell=True,
stderr=subprocess.PIPE)
p.wait()
print(p.returncode, p.stderr.read())](https://image.slidesharecdn.com/concurrentprogramminginpython-170416042535/85/Concurrent-Programming-in-Python-8-320.jpg)
Concurrent Programming in Python
- 1. @adam_englander Concurrent Programming in Python Programming the multi-verse
- 2. @adam_englander Threading • Thread-Local can be used for global data only accessible by a single thread. • Semaphores and locks prevent simultaneous access of memory. Python’s with statement makes it a lot cleaner than many languages. • Events and Barriers allow for easy interprocess coordination.
- 3. @adam_englander Processes • Process class allows for spawning and forking of processes from a parent process. • Popular for multi-tasking in scripts as you are simply executing a command rather Threading which runs code. • Provides pipes, queues, shared memory, and server processes for interprocess communication. • Allows for worker pools to manage resource utilization.
- 4. @adam_englander Subprocesses • Run other scripts from a main script • Uses popen to open other scripts • Allows for better error handling of subscripts • Allows for waiting on the subscript to complete
- 5. @adam_englander Concurrent • Uses Futures • Can use Thread Pools or Process Pools • Allows for asynchronous function programming • Works just like any other language with Futures/Promises
- 6. @adam_englander Suggestions • For scripts, uses Subprocess • For all but the simplest of applications, use a concurrent framework like curio, Tornado, Twisted, etc.
- 7. @adam_englander Process Example from multiprocessing import Process import os from random import randint from time import sleep def start(): print("Hello from pid", os.getpid()) sleep(randint(1, 3)) print("Goodbye from pid", os.getpid()) exit() print("Hello from master pid", os.getpid()) for i in range(1, 5): p = Process(target=start) p.start() print("Goodbye from master pid", os.getpid())
- 8. @adam_englander Subprocess Example import subprocess print("nnManaged exec error") p = subprocess.Popen( [“/not/a/valid/script"], shell=True, stderr=subprocess.PIPE) p.wait() print(p.returncode, p.stderr.read())
- 9. @adam_englander Curio • High concurrency framework with no asynchronous I/O • Utilizes core asynchronous and concurrency libraries available in Python 3.5+ • Simplifies concurrent programming • Highly performant