Python multithreading
0 likes686 views
- Python allows programmers to write multithreaded programs that run multiple threads concurrently for more efficient use of resources and responsiveness. - The main benefits of multithreading over multiple processes are that threads within a process can share data and resources more easily. Threads also have less memory overhead than processes. - There are two main modules for multithreading in Python - the older thread module and the newer threading module, which provides more powerful features and methods for managing threads.
![http://www.tutorialspoint.com/python/python_multithreading .htm Copyright © tutorialspoint.com
PYTHON MULTITHREADED PROGRAMMING
Running severalthreads is similar to running severaldifferent programs concurrently, but withthe following
benefits:
Multiple threads withina process share the same data space withthe mainthread and cantherefore share
informationor communicate witheachother more easily thanif they were separate processes.
Threads sometimes called light-weight processes and they do not require muchmemory overhead; they
care cheaper thanprocesses.
A thread has a beginning, anexecutionsequence, and a conclusion. It has aninstructionpointer that keeps track
of where withinits context it is currently running.
It canbe pre-empted (interrupted)
It cantemporarily be put onhold (also knownas sleeping) while other threads are running - this is called
yielding.
Starting a New Thread:
To spawnanother thread, youneed to callfollowing method available inthread module:
thread.start_new_thread ( function, args[, kwargs] )
This method callenables a fast and efficient way to create new threads inbothLinux and Windows.
The method callreturns immediately and the child thread starts and calls functionwiththe passed list of agrs.
Whenfunctionreturns, the thread terminates.
Here, args is a tuple of arguments; use anempty tuple to callfunctionwithout passing any arguments. kwargs is
anoptionaldictionary of keyword arguments.
Example:
#!/usr/bin/python
import thread
import time
# Define a function for the thread
def print_time( threadName, delay):
count = 0
while count < 5:
time.sleep(delay)
count += 1
print "%s: %s" % ( threadName, time.ctime(time.time()) )
# Create two threads as follows
try:
thread.start_new_thread( print_time, ("Thread-1", 2, ) )
thread.start_new_thread( print_time, ("Thread-2", 4, ) )
except:
print "Error: unable to start thread"
while 1:
pass
Whenthe above code is executed, it produces the following result:
Thread-1: Thu Jan 22 15:42:17 2009
Thread-1: Thu Jan 22 15:42:19 2009
Thread-2: Thu Jan 22 15:42:19 2009](https://image.slidesharecdn.com/pythonmultithreading-190712152108/85/Python-multithreading-1-320.jpg)
![Thread-1: Thu Jan 22 15:42:21 2009
Thread-2: Thu Jan 22 15:42:23 2009
Thread-1: Thu Jan 22 15:42:23 2009
Thread-1: Thu Jan 22 15:42:25 2009
Thread-2: Thu Jan 22 15:42:27 2009
Thread-2: Thu Jan 22 15:42:31 2009
Thread-2: Thu Jan 22 15:42:35 2009
Althoughit is very effective for low-levelthreading, but the thread module is very limited compared to the newer
threading module.
The Threading Module:
The newer threading module included withPython2.4 provides muchmore powerful, high-levelsupport for
threads thanthe thread module discussed inthe previous section.
The threading module exposes allthe methods of the thread module and provides some additionalmethods:
threading.activeCount(): Returns the number of thread objects that are active.
threading.currentThread(): Returns the number of thread objects inthe caller's thread control.
threading.enumerate(): Returns a list of allthread objects that are currently active.
Inadditionto the methods, the threading module has the Thread class that implements threading. The methods
provided by the Thread class are as follows:
run(): The run() method is the entry point for a thread.
start(): The start() method starts a thread by calling the runmethod.
join([time]): The join() waits for threads to terminate.
isAlive(): The isAlive() method checks whether a thread is stillexecuting.
getName(): The getName() method returns the name of a thread.
setName(): The setName() method sets the name of a thread.
Creating Thread using Threading Module:
To implement a new thread using the threading module, youhave to do the following:
Define a new subclass of the Thread class.
Override the __init__(self [,args]) method to add additionalarguments.
Then, override the run(self [,args]) method to implement what the thread should do whenstarted.
Once youhave created the new Thread subclass, youcancreate aninstance of it and thenstart a new thread by
invoking the start(), whichwillinturncallrun() method.
Example:
#!/usr/bin/python
import threading
import time
exitFlag = 0
class myThread (threading.Thread):
def __init__(self, threadID, name, counter):
threading.Thread.__init__(self)
self.threadID = threadID
self.name = name
self.counter = counter](https://image.slidesharecdn.com/pythonmultithreading-190712152108/85/Python-multithreading-2-320.jpg)
![# Get lock to synchronize threads
threadLock.acquire()
print_time(self.name, self.counter, 3)
# Free lock to release next thread
threadLock.release()
def print_time(threadName, delay, counter):
while counter:
time.sleep(delay)
print "%s: %s" % (threadName, time.ctime(time.time()))
counter -= 1
threadLock = threading.Lock()
threads = []
# Create new threads
thread1 = myThread(1, "Thread-1", 1)
thread2 = myThread(2, "Thread-2", 2)
# Start new Threads
thread1.start()
thread2.start()
# Add threads to thread list
threads.append(thread1)
threads.append(thread2)
# Wait for all threads to complete
for t in threads:
t.join()
print "Exiting Main Thread"
Whenthe above code is executed, it produces the following result:
Starting Thread-1
Starting Thread-2
Thread-1: Thu Mar 21 09:11:28 2013
Thread-1: Thu Mar 21 09:11:29 2013
Thread-1: Thu Mar 21 09:11:30 2013
Thread-2: Thu Mar 21 09:11:32 2013
Thread-2: Thu Mar 21 09:11:34 2013
Thread-2: Thu Mar 21 09:11:36 2013
Exiting Main Thread
Multithreaded Priority Queue:
The Queue module allows youto create a new queue object that canhold a specific number of items. There are
following methods to controlthe Queue:
get(): The get() removes and returns anitemfromthe queue.
put(): The put adds itemto a queue.
qsize() : The qsize() returns the number of items that are currently inthe queue.
empty(): The empty( ) returns True if queue is empty; otherwise, False.
full(): the full() returns True if queue is full; otherwise, False.
Example:
#!/usr/bin/python
import Queue
import threading
import time
exitFlag = 0](https://image.slidesharecdn.com/pythonmultithreading-190712152108/85/Python-multithreading-4-320.jpg)
![class myThread (threading.Thread):
def __init__(self, threadID, name, q):
threading.Thread.__init__(self)
self.threadID = threadID
self.name = name
self.q = q
def run(self):
print "Starting " + self.name
process_data(self.name, self.q)
print "Exiting " + self.name
def process_data(threadName, q):
while not exitFlag:
queueLock.acquire()
if not workQueue.empty():
data = q.get()
queueLock.release()
print "%s processing %s" % (threadName, data)
else:
queueLock.release()
time.sleep(1)
threadList = ["Thread-1", "Thread-2", "Thread-3"]
nameList = ["One", "Two", "Three", "Four", "Five"]
queueLock = threading.Lock()
workQueue = Queue.Queue(10)
threads = []
threadID = 1
# Create new threads
for tName in threadList:
thread = myThread(threadID, tName, workQueue)
thread.start()
threads.append(thread)
threadID += 1
# Fill the queue
queueLock.acquire()
for word in nameList:
workQueue.put(word)
queueLock.release()
# Wait for queue to empty
while not workQueue.empty():
pass
# Notify threads it's time to exit
exitFlag = 1
# Wait for all threads to complete
for t in threads:
t.join()
print "Exiting Main Thread"
Whenthe above code is executed, it produces the following result:
Starting Thread-1
Starting Thread-2
Starting Thread-3
Thread-1 processing One
Thread-2 processing Two
Thread-3 processing Three
Thread-1 processing Four
Thread-2 processing Five
Exiting Thread-3
Exiting Thread-1
Exiting Thread-2
Exiting Main Thread](https://image.slidesharecdn.com/pythonmultithreading-190712152108/85/Python-multithreading-5-320.jpg)
Python multithreading
- 1. http://www.tutorialspoint.com/python/python_multithreading .htm Copyright © tutorialspoint.com PYTHON MULTITHREADED PROGRAMMING Running severalthreads is similar to running severaldifferent programs concurrently, but withthe following benefits: Multiple threads withina process share the same data space withthe mainthread and cantherefore share informationor communicate witheachother more easily thanif they were separate processes. Threads sometimes called light-weight processes and they do not require muchmemory overhead; they care cheaper thanprocesses. A thread has a beginning, anexecutionsequence, and a conclusion. It has aninstructionpointer that keeps track of where withinits context it is currently running. It canbe pre-empted (interrupted) It cantemporarily be put onhold (also knownas sleeping) while other threads are running - this is called yielding. Starting a New Thread: To spawnanother thread, youneed to callfollowing method available inthread module: thread.start_new_thread ( function, args[, kwargs] ) This method callenables a fast and efficient way to create new threads inbothLinux and Windows. The method callreturns immediately and the child thread starts and calls functionwiththe passed list of agrs. Whenfunctionreturns, the thread terminates. Here, args is a tuple of arguments; use anempty tuple to callfunctionwithout passing any arguments. kwargs is anoptionaldictionary of keyword arguments. Example: #!/usr/bin/python import thread import time # Define a function for the thread def print_time( threadName, delay): count = 0 while count < 5: time.sleep(delay) count += 1 print "%s: %s" % ( threadName, time.ctime(time.time()) ) # Create two threads as follows try: thread.start_new_thread( print_time, ("Thread-1", 2, ) ) thread.start_new_thread( print_time, ("Thread-2", 4, ) ) except: print "Error: unable to start thread" while 1: pass Whenthe above code is executed, it produces the following result: Thread-1: Thu Jan 22 15:42:17 2009 Thread-1: Thu Jan 22 15:42:19 2009 Thread-2: Thu Jan 22 15:42:19 2009
- 2. Thread-1: Thu Jan 22 15:42:21 2009 Thread-2: Thu Jan 22 15:42:23 2009 Thread-1: Thu Jan 22 15:42:23 2009 Thread-1: Thu Jan 22 15:42:25 2009 Thread-2: Thu Jan 22 15:42:27 2009 Thread-2: Thu Jan 22 15:42:31 2009 Thread-2: Thu Jan 22 15:42:35 2009 Althoughit is very effective for low-levelthreading, but the thread module is very limited compared to the newer threading module. The Threading Module: The newer threading module included withPython2.4 provides muchmore powerful, high-levelsupport for threads thanthe thread module discussed inthe previous section. The threading module exposes allthe methods of the thread module and provides some additionalmethods: threading.activeCount(): Returns the number of thread objects that are active. threading.currentThread(): Returns the number of thread objects inthe caller's thread control. threading.enumerate(): Returns a list of allthread objects that are currently active. Inadditionto the methods, the threading module has the Thread class that implements threading. The methods provided by the Thread class are as follows: run(): The run() method is the entry point for a thread. start(): The start() method starts a thread by calling the runmethod. join([time]): The join() waits for threads to terminate. isAlive(): The isAlive() method checks whether a thread is stillexecuting. getName(): The getName() method returns the name of a thread. setName(): The setName() method sets the name of a thread. Creating Thread using Threading Module: To implement a new thread using the threading module, youhave to do the following: Define a new subclass of the Thread class. Override the __init__(self [,args]) method to add additionalarguments. Then, override the run(self [,args]) method to implement what the thread should do whenstarted. Once youhave created the new Thread subclass, youcancreate aninstance of it and thenstart a new thread by invoking the start(), whichwillinturncallrun() method. Example: #!/usr/bin/python import threading import time exitFlag = 0 class myThread (threading.Thread): def __init__(self, threadID, name, counter): threading.Thread.__init__(self) self.threadID = threadID self.name = name self.counter = counter
- 3. def run(self): print "Starting " + self.name print_time(self.name, self.counter, 5) print "Exiting " + self.name def print_time(threadName, delay, counter): while counter: if exitFlag: thread.exit() time.sleep(delay) print "%s: %s" % (threadName, time.ctime(time.time())) counter -= 1 # Create new threads thread1 = myThread(1, "Thread-1", 1) thread2 = myThread(2, "Thread-2", 2) # Start new Threads thread1.start() thread2.start() print "Exiting Main Thread" Whenthe above code is executed, it produces the following result: Starting Thread-1 Starting Thread-2 Exiting Main Thread Thread-1: Thu Mar 21 09:10:03 2013 Thread-1: Thu Mar 21 09:10:04 2013 Thread-2: Thu Mar 21 09:10:04 2013 Thread-1: Thu Mar 21 09:10:05 2013 Thread-1: Thu Mar 21 09:10:06 2013 Thread-2: Thu Mar 21 09:10:06 2013 Thread-1: Thu Mar 21 09:10:07 2013 Exiting Thread-1 Thread-2: Thu Mar 21 09:10:08 2013 Thread-2: Thu Mar 21 09:10:10 2013 Thread-2: Thu Mar 21 09:10:12 2013 Exiting Thread-2 Synchronizing Threads: The threading module provided withPythonincludes a simple-to-implement locking mechanismthat willallow you to synchronize threads. A new lock is created by calling the Lock() method, whichreturns the new lock. The acquire(blocking) method of the new lock object would be used to force threads to runsynchronously. The optionalblocking parameter enables youto controlwhether the thread willwait to acquire the lock. If blocking is set to 0, the thread willreturnimmediately witha 0 value if the lock cannot be acquired and witha 1 if the lock was acquired. If blocking is set to 1, the thread willblock and wait for the lock to be released. The release() method of the the new lock object would be used to release the lock whenit is no longer required. Example: #!/usr/bin/python import threading import time class myThread (threading.Thread): def __init__(self, threadID, name, counter): threading.Thread.__init__(self) self.threadID = threadID self.name = name self.counter = counter def run(self): print "Starting " + self.name
- 4. # Get lock to synchronize threads threadLock.acquire() print_time(self.name, self.counter, 3) # Free lock to release next thread threadLock.release() def print_time(threadName, delay, counter): while counter: time.sleep(delay) print "%s: %s" % (threadName, time.ctime(time.time())) counter -= 1 threadLock = threading.Lock() threads = [] # Create new threads thread1 = myThread(1, "Thread-1", 1) thread2 = myThread(2, "Thread-2", 2) # Start new Threads thread1.start() thread2.start() # Add threads to thread list threads.append(thread1) threads.append(thread2) # Wait for all threads to complete for t in threads: t.join() print "Exiting Main Thread" Whenthe above code is executed, it produces the following result: Starting Thread-1 Starting Thread-2 Thread-1: Thu Mar 21 09:11:28 2013 Thread-1: Thu Mar 21 09:11:29 2013 Thread-1: Thu Mar 21 09:11:30 2013 Thread-2: Thu Mar 21 09:11:32 2013 Thread-2: Thu Mar 21 09:11:34 2013 Thread-2: Thu Mar 21 09:11:36 2013 Exiting Main Thread Multithreaded Priority Queue: The Queue module allows youto create a new queue object that canhold a specific number of items. There are following methods to controlthe Queue: get(): The get() removes and returns anitemfromthe queue. put(): The put adds itemto a queue. qsize() : The qsize() returns the number of items that are currently inthe queue. empty(): The empty( ) returns True if queue is empty; otherwise, False. full(): the full() returns True if queue is full; otherwise, False. Example: #!/usr/bin/python import Queue import threading import time exitFlag = 0
- 5. class myThread (threading.Thread): def __init__(self, threadID, name, q): threading.Thread.__init__(self) self.threadID = threadID self.name = name self.q = q def run(self): print "Starting " + self.name process_data(self.name, self.q) print "Exiting " + self.name def process_data(threadName, q): while not exitFlag: queueLock.acquire() if not workQueue.empty(): data = q.get() queueLock.release() print "%s processing %s" % (threadName, data) else: queueLock.release() time.sleep(1) threadList = ["Thread-1", "Thread-2", "Thread-3"] nameList = ["One", "Two", "Three", "Four", "Five"] queueLock = threading.Lock() workQueue = Queue.Queue(10) threads = [] threadID = 1 # Create new threads for tName in threadList: thread = myThread(threadID, tName, workQueue) thread.start() threads.append(thread) threadID += 1 # Fill the queue queueLock.acquire() for word in nameList: workQueue.put(word) queueLock.release() # Wait for queue to empty while not workQueue.empty(): pass # Notify threads it's time to exit exitFlag = 1 # Wait for all threads to complete for t in threads: t.join() print "Exiting Main Thread" Whenthe above code is executed, it produces the following result: Starting Thread-1 Starting Thread-2 Starting Thread-3 Thread-1 processing One Thread-2 processing Two Thread-3 processing Three Thread-1 processing Four Thread-2 processing Five Exiting Thread-3 Exiting Thread-1 Exiting Thread-2 Exiting Main Thread