SlideShare a Scribd company logo
Elegant
Solutions for
Everyday
Python
Problems
Nina Zakharenko
@nnja
h!p://bit.ly/elegant-
python
ℹ There are links in these slides.
slides: h!p://bit.ly/elegant-python
This talk is for you if:
You're an intermediate python programmer
You're coming to python from another language
You want to learn about language features like: magic
methods, iterators, decorators, and context managers
@nnja
What
is
elegant code?
@nnja
How do we make code elegant?
We pick the right tool for the job!
Resources for converting from Python 2 -> 3
Beauty is
in the eye of
the beholder
magic
methods
Magic methods start and end with a double underscore
(dunder)
By implementing a few straightforward magic methods,
you can make your objects behave like built-ins such as:
numbers
lists
dictionaries
and more...
@nnja
class Money:
currency_rates = {
'$': 1,
'€': 0.88,
}
def __init__(self, symbol, amount):
self.symbol = symbol
self.amount = amount
def __str__(self):
return '%s%.2f' % (self.symbol, self.amount)
@nnja
class Money:
currency_rates = {
'$': 1,
'€': 0.88,
}
def __init__(self, symbol, amount):
self.symbol = symbol
self.amount = amount
def __str__(self):
return '%s%.2f' % (self.symbol, self.amount)
@nnja
class Money:
# defined currency_rates, __init__, and str above...
def convert(self, other):
"""Convert other amount to our currency"""
new_amount = (
other.amount / self.currency_rates[other.symbol]
* self.currency_rates[self.symbol])
return Money(self.symbol, new_amount)
@nnja
__str__ in action
>>> soda_cost = Money('$', 5.25)
>>> print(soda_cost)
$5.25
>>> pizza_cost = Money('€', 7.99)
>>> print(pizza_cost)
€7.99
@nnja
class Money:
def __add__(self, other):
""" Add 2 Money instances using '+' """
new_amount = self.amount + self.convert(other).amount
return Money(self.symbol, new_amount)
@nnja
>>> soda_cost = Money('$', 5.25)
>>> pizza_cost = Money('€', 7.99)
>>> print(soda_cost + pizza_cost)
$14.33
More on Magic Methods: Dive into Python3 - Special Method
Names
>>> soda_cost = Money('$', 5.25)
>>> pizza_cost = Money('€', 7.99)
>>> print(soda_cost + pizza_cost)
$14.33
>>> print(pizza_cost + soda_cost)
€12.61
More on Magic Methods: Dive into Python3 - Special Method
Names
some magic methods map to symbols
>>> d = {'one': 1, 'two': 2}
>>> d['two']
2
>>> d.__getitem__('two')
2
@nnja
other magic methods map to built-in functions
class SquareShape:
def __len__(self):
""" Return the number of sides in our shape """
return 4
>>> my_square = SquareShape()
>>> len(my_square)
4
@nnja
custom
iterators
image source
Making classes iterable
In order to be iterable, a class needs to implement
__iter__()
__iter__() must return an iterator
In order to be an iterator a class needs to implement
__next__() which must raise StopIteration when
there are no more items to return
Great explanation of iterable vs. iterator vs. generator
Scenario..
We have a Server instance running services on
different ports.
Some services are active, some are inactive.
When we loop over our the Server instance, we only
want to loop over active services.
@nnja
class IterableServer:
services = [
{'active': False, 'protocol': 'ftp', 'port': 21},
{'active': True, 'protocol': 'ssh', 'port': 22},
{'active': True, 'protocol': 'http', 'port': 80},
]
@nnja
class IterableServer:
def __init__(self):
self.current_pos = 0
def __iter__(self):
# can return self, because __next__ implemented
return self
def __next__(self):
while self.current_pos < len(self.services):
service = self.services[self.current_pos]
self.current_pos += 1
if service['active']:
return service['protocol'], service['port']
raise StopIteration
@nnja
class IterableServer:
def __init__(self):
self.current_pos = 0
def __iter__(self):
# can return self, because __next__ implemented
return self
def __next__(self):
while self.current_pos < len(self.services):
service = self.services[self.current_pos]
self.current_pos += 1
if service['active']:
return service['protocol'], service['port']
raise StopIteration
@nnja
>>> for protocol, port in IterableServer():
print('service %s on port %d' % (protocol, port))
service ssh on port 22
service http on port 80
loops over all active services ... not bad
@nnja
tip: use a generator
when your iterator doesn't need to
maintain a lot of state
(which is most of the time)
@nnja
class Server:
services = [
{'active': False, 'protocol': 'ftp', 'port': 21},
{'active': True, 'protocol': 'ssh', 'port': 22},
{'active': True, 'protocol': 'http', 'port': 21},
]
def __iter__(self):
for service in self.services:
if service['active']:
yield service['protocol'], service['port']
@nnja
class Server:
services = [
{'active': False, 'protocol': 'ftp', 'port': 21},
{'active': True, 'protocol': 'ssh', 'port': 22},
{'active': True, 'protocol': 'http', 'port': 21},
]
def __iter__(self):
for service in self.services:
if service['active']:
yield service['protocol'], service['port']
@nnja
Why does this work?
use single parenthesis ( ) to create a generator
comprehension
^ technically, a generator expression but I like this term better, and so does Ned
Batchelder
>>> my_gen = (num for num in range(1))
>>> my_gen
<generator object <genexpr> at 0x107581bf8>
@nnja
An iterator must implement __next__()
>>> next(my_gen) # __next__() maps to built-in next()
0
and raise StopIteration when there are no more elements
>>> next(my_gen)
... StopIteration Traceback (most recent call last)
see itertools for working with iterators
✨
Method
✨
Magic
@nnja
alias methods
class Word:
def __init__(self, word):
self.word = word
def __repr__(self):
return self.word
def __add__(self, other_word):
return Word('%s %s' % (self.word, other_word))
# Add an alias from method __add__ to the method concat
concat = __add__
@nnja
We can add an alias from __add__ to concat because
methods are just objects
>>> # remember, concat = __add__
>>> first_name = Word('Max')
>>> last_name = Word('Smith')
>>> first_name + last_name
Max Smith
>>> first_name.concat(last_name)
Max Smith
>>> Word.__add__ == Word.concat
True
@nnja
Dog class
>>> class Dog:
sound = 'Bark'
def speak(self):
print(self.sound + '!', self.sound + '!')
>>> my_dog = Dog()
>>> my_dog.speak()
Bark! Bark!
read the docs
getattr(object, name, default)
>>> class Dog:
sound = 'Bark'
def speak(self):
print(self.sound + '!', self.sound + '!')
>>> my_dog = Dog()
>>> my_dog.speak()
Bark! Bark!
>>> getattr(my_dog, 'speak')
<bound method Dog.speak of
<__main__.Dog object at 0x10b145f28>>
>>> speak_method = getattr(my_dog, 'speak')
>>> speak_method()
Bark! Bark!
getattr(object, name, default)
>>> class Dog:
sound = 'Bark'
def speak(self):
print(self.sound + '!', self.sound + '!')
>>> my_dog = Dog()
>>> my_dog.speak()
Bark! Bark!
>>> getattr(my_dog, 'speak')
<bound method Dog.speak of
<__main__.Dog object at 0x10b145f28>>
>>> speak_method = getattr(my_dog, 'speak')
>>> speak_method()
Bark! Bark!
Example: command line tool with dynamic commands
class Operations:
def say_hi(self, name):
print('Hello,', name)
def say_bye(self, name):
print('Goodbye,', name)
def default(self, arg):
print('This operation is not supported.')
if __name__ == '__main__':
operations = Operations()
# let's assume error handling
command, argument = input('> ').split()
getattr(operations, command, operations.default)(argument)
read the docs
Output
$ python demo.py
> say_hi Nina
Hello, Nina
> blah blah
This operation is not supported.
✨
additional reading - inverse of getattr() is setattr()
functools.partial(func, *args, **kwargs)
Return a new partial object which behaves like func
called with args & kwargs
if more arguments are passed in, they are appended
to args
if more keyword arguments are passed in, they extend
and override kwargs
read the docs on partials
functool.partial(func, *args, **kwargs)
# We want to be able to call this function on any int
# without having to specify the base.
>>> int('10010', base=2)
18
>>> from functools import partial
>>> basetwo = partial(int, base=2)
>>> basetwo
<functools.partial object at 0x1085a09f0>
>>> basetwo('10010')
18
read the docs on partials
functool.partial(func, *args, **kwargs)
# We want to be able to call this function on any int
# without having to specify the base.
>>> int('10010', base=2)
18
>>> from functools import partial
>>> basetwo = partial(int, base=2)
>>> basetwo
<functools.partial object at 0x1085a09f0>
>>> basetwo('10010')
18
read the docs on partials
functool.partial(func, *args, **kwargs)
# We want to be able to call this function on any int
# without having to specify the base.
>>> int('10010', base=2)
18
>>> from functools import partial
>>> basetwo = partial(int, base=2)
>>> basetwo
<functools.partial object at 0x1085a09f0>
>>> basetwo('10010')
18
read the docs on partials
magic methods
& method magic
in action!
photo credit
library I
!
: github.com/mozilla/agithub
agithub is a (poorly named) REST API client with
transparent syntax which facilitates rapid prototyping
— on any REST API!
Implemented in ~400 lines.
Add support for any REST API in ~30 lines of code.
agithub knows everything it needs to about protocol
(REST, HTTP, etc), but assumes nothing about your
upstream API.
@nnja
define endpoint url & other connection properties
class GitHub(API):
def __init__(self, token=None, *args, **kwargs):
props = ConnectionProperties(
api_url = kwargs.pop('api_url', 'api.github.com'))
self.setClient(Client(*args, **kwargs))
self.setConnectionProperties(props)
@nnja
then, start using the API!
>>> gh = GitHub('token')
>>> status, data = gh.user.repos.get()
>>> # ^ Maps to GET /user/repos
>>> data
... ['tweeter', 'snipey', '...']
github.com/mozilla/agithub
404 if we provide a path that doesn't exist:
>>> gh = GitHub('token')
>>> status, data = gh.this.path.doesnt.exist.get()
>>> status
... 404
github.com/jpaugh/agithub
magic!
but, how?...
@nnja
class API:
def __getattr__(self, key):
return IncompleteRequest(self.client).__getattr__(key)
__getitem__ = __getattr__
class IncompleteRequest:
def __getattr__(self, key):
if key in self.client.http_methods:
htmlMethod = getattr(self.client, key)
return partial(htmlMethod, url=self.url)
else:
self.url += '/' + str(key)
return self
__getitem__ = __getattr__
class Client:
http_methods = ('get') # ...
def get(self, url, headers={}, **params):
return self.request('GET', url, None, headers)
github.com/jpaugh/agithub source: base.py
class API:
def __getattr__(self, key):
return IncompleteRequest(self.client).__getattr__(key)
__getitem__ = __getattr__
class IncompleteRequest:
def __getattr__(self, key):
if key in self.client.http_methods:
htmlMethod = getattr(self.client, key)
return partial(htmlMethod, url=self.url)
else:
self.url += '/' + str(key)
return self
__getitem__ = __getattr__
class Client:
http_methods = ('get') # ...
def get(self, url, headers={}, **params):
return self.request('GET', url, None, headers)
github.com/jpaugh/agithub source: base.py
class API:
def __getattr__(self, key):
return IncompleteRequest(self.client).__getattr__(key)
__getitem__ = __getattr__
class IncompleteRequest:
def __getattr__(self, key):
if key in self.client.http_methods:
htmlMethod = getattr(self.client, key)
return partial(htmlMethod, url=self.url)
else:
self.url += '/' + str(key)
return self
__getitem__ = __getattr__
class Client:
http_methods = ('get') # ...
def get(self, url, headers={}, **params):
return self.request('GET', url, None, headers)
github.com/jpaugh/agithub source: base.py
Context
Managers
@nnja
When should I use one?
Need to perform an action before and/or after an
operation.
Common scenarios:
Closing a resource after you're done with it (file,
network connection)
Perform cleanup before/after a function call
@nnja
Example Problem: Feature Flags
Turn features of your application on and off easily.
Uses of feature flags:
A/B Testing
Rolling Releases
Show Beta version to users opted-in to Beta Testing
Program
More on Feature Flags
class FeatureFlags:
SHOW_BETA = 'Show Beta version of Home Page'
flags = {
SHOW_BETA: True
}
@classmethod
def is_on(cls, name):
return cls.flags[name]
@classmethod
def toggle(cls, name, value):
cls.flags[name] = value
feature_flags = FeatureFlags()
@nnja
How do we temporarily turn features on and off when
testing flags?
Want:
with feature_flag(FeatureFlags.SHOW_BETA):
assert '/beta' == get_homepage_url()
@nnja
Using Magic Methods __enter__ and __exit__
class feature_flag:
""" Implementing a Context Manager using Magic Methods """
def __init__(self, name, on=True):
self.name = name
self.on = on
self.old_value = feature_flags.is_on(name)
def __enter__(self):
feature_flags.toggle(self.name, self.on)
def __exit__(self, *args):
feature_flags.toggle(self.name, self.old_value)
See: contextlib.contextmanager
The be!er way: using the contextmanager decorator
from contextlib import contextmanager
@contextmanager
def feature_flag(name, on=True):
old_value = feature_flags.is_on(name)
feature_flags.toggle(name, on)
yield
feature_flags.toggle(name, old_value)
See: contextlib.contextmanager
The be!er way: using the contextmanager decorator
from contextlib import contextmanager
@contextmanager
def feature_flag(name, on=True):
""" The easier way to create Context Managers """
old_value = feature_flags.is_on(name)
# behavior of __enter__()
feature_flags.toggle(name, on)
yield
# behavior of __exit__()
feature_flags.toggle(name, old_value)
See: contextlib.contextmanager
Note: yield?
from contextlib import contextmanager
@contextmanager
def feature_flag(name, on=True):
""" The easier way to create Context Managers """
old_value = feature_flags.is_on(name)
feature_flags.toggle(name, on) # behavior of __enter__()
yield
feature_flags.toggle(name, old_value) # behavior of __exit__()
See: contextlib.contextmanager
either implementation
def get_homepage_url():
""" Returns the path of the page to display """
if feature_flags.is_on(FeatureFlags.SHOW_BETA):
return '/beta'
else:
return '/homepage'
def test_homepage_url_with_context_manager():
with feature_flag(FeatureFlags.SHOW_BETA):
# saw the beta homepage...
assert get_homepage_url() == '/beta'
with feature_flag(FeatureFlags.SHOW_BETA, on=False):
# saw the standard homepage...
assert get_homepage_url() == '/homepage'
either implementation
def get_homepage_url():
""" Returns the path of the page to display """
if feature_flags.is_on(FeatureFlags.SHOW_BETA):
return '/beta'
else:
return '/homepage'
def test_homepage_url_with_context_manager():
with feature_flag(FeatureFlags.SHOW_BETA):
assert get_homepage_url() == '/beta'
print('seeing the beta homepage...')
with feature_flag(FeatureFlags.SHOW_BETA, on=False):
assert get_homepage_url() == '/homepage'
print('seeing the standard homepage...')
Decorators
The simple explanation:
Syntactic sugar that allows modification of an underlying
function.
@nnja
Decorators:
Wrap a function in another function.
Do something:
before the call
after the call
with provided arguments
modify the return value or arguments
@nnja
class User:
is_authenticated = False
def __init__(self, name):
self.name = name
Throw an exception if trying to access data only for
logged in users:
def display_profile_page(user):
""" Display profile page for logged in User """
if not user.is_authenticated:
raise Exception('User must login.')
print('Profile: %s' % user.name)
def enforce_authentication(func):
def wrapper(user):
if not user.is_authenticated:
raise Exception('User must login.')
return func(user)
return wrapper
the important logic:
def enforce_authentication(func):
def wrapper(user):
if not user.is_authenticated:
raise Exception('User must login.')
return func(user)
return wrapper
@nnja
Using enforce_authentication without a decorator:
enforce_authentication(display_profile_page)(some_user)
Or, as a decorator:
@enforce_authentication
def display_profile_page(user):
print('Profile: %s' % user.name)
Now this raises an Exception if unauthenticated:
user = User('nina')
display_profile_page(nina)
Problem: lost context using a decorator
>>> display_profile_page.__name__
'wrapper'
>>>> display_profile_page.__doc__
# ... empty
Solution: Use contextlib.wraps
from contextlib import wraps
def enforce_authentication(func):
@wraps(func)
def wrapper(user):
# ... rest of the code
Decorators: Common uses
logging
timing
validation
rate limiting
mocking/patching
@nnja
ContextDecorators
ContextManagers
+ Decorators combined.
@nnja
By using ContextDecorator you can easily write
classes that can be used both as decorators with @
and context managers with the with statement.
ContextDecorator is used by contextmanager(),
so you get this functionality automatically .
Or, you can write a class that extends from
ContextDecorator or uses ContextDecorator as a
mixin, and implements __enter__, __exit__ and
__call__
@nnja
Remember @contextmanager from earlier?
from contextlib import contextmanager
@contextmanager
def feature_flag(name, on=True):
old_value = feature_flags.is_on(name)
feature_flags.toggle(name, on)
yield
feature_flags.toggle(name, old_value)
@nnja
use it as a context manager
with feature_flag(FeatureFlags.SHOW_BETA):
assert get_homepage_url() == '/beta'
or use as a decorator
@feature_flag(FeatureFlags.SHOW_BETA, on=False)
def get_profile_page():
beta_flag_on = feature_flags.is_on(
FeatureFlags.SHOW_BETA)
if beta_flag_on:
return 'beta.html'
else:
return 'profile.html'
library I
!
: freezegun
lets your python tests ❇ travel through time! ❇
from freezegun import freeze_time
# use it as a Context Manager
def test():
with freeze_time("2012-01-14"):
assert datetime.datetime.now() == datetime.datetime(
2012, 1, 14)
assert datetime.datetime.now() != datetime.datetime(2012, 1, 14)
# or a decorator
@freeze_time("2012-01-14")
def test():
assert datetime.datetime.now() == datetime.datetime(2012, 1, 14)
read the source sometime, it's mind-bending!
NamedTuple
Useful when you need lightweight representations of
data.
Create tuple subclasses with named fields.
@nnja
Example
from collections import namedtuple
CacheInfo = namedtuple(
"CacheInfo", ["hits", "misses", "max_size", "curr_size"])
@nnja
Giving NamedTuples default values
RoutingRule = namedtuple(
'RoutingRule',
['prefix', 'queue_name', 'wait_time']
)
(1) By specifying defaults
RoutingRule.__new__.__defaults__ = (None, None, 20)
(2) or with _replace to customize a prototype instance
default_rule = RoutingRule(None, None, 20)
user_rule = default_rule._replace(
prefix='user', queue_name='user-queue')
NamedTuples can be subclassed and extended
class Person(namedtuple('Person', ['first_name', 'last_name'])):
""" Stores first and last name of a Person"""
def __str__(self):
return '%s %s' % (self.first_name, self.last_name)
>>> me = Person('nina', 'zakharenko')
>>> str(me)
'nina zakharenko'
>>> me
Person(first_name='nina', last_name='zakharenko')
New Tools
Magic Methods
make your objects behave like builtins (numbers,
list, dict, etc)
Method ❇Magic❇
alias methods
getattr
@nnja
ContextManagers
Manage resources
Decorators
do something before/after call, modify return value
or validate arguments
ContextDecorators
ContextManagers + Decorators in one
@nnja
Iterators & Generators
Loop over your objects
yield
NamedTuple
Lightweight classes
@nnja
"Perfection is achieved, not when
there is nothing more to add, but
when there is nothing left to take
away."
— Antoine de Saint-Exupery
@nnja
Be an
elegant
Pythonista!
@nnja
Thanks!
slides: bit.ly/elegant-python
python@ microso! aka.ms/python
@nnja
@nnja

More Related Content

PPTX
Parathyroid and calcium metabolism
PDF
Elegant Solutions For Everyday Python Problems - PyCon Canada 2017
PDF
Elegant Solutions For Everyday Python Problems - Nina Zakharenko
PDF
Python na Infraestrutura 
MySQL do Facebook

PPTX
Python_UNIT-I.pptx
PPTX
Iterarators and generators in python
PPTX
python ppt.pptx
PPTX
Advance python
Parathyroid and calcium metabolism
Elegant Solutions For Everyday Python Problems - PyCon Canada 2017
Elegant Solutions For Everyday Python Problems - Nina Zakharenko
Python na Infraestrutura 
MySQL do Facebook

Python_UNIT-I.pptx
Iterarators and generators in python
python ppt.pptx
Advance python

Similar to Elegant Solutions for Everyday Python Problems Pycon 2018 - Nina Zakharenko (20)

PPTX
Functions2.pptx
PDF
C++ code, please help! RESPOND W COMPLETED CODE PLEASE, am using V.pdf
PPTX
Python Learn Function with example programs
PDF
Python lecture 03
PDF
Cc code cards
PPTX
Python_Functions_Unit1.pptx
PPTX
Introduction to python programming 2
PPTX
Pythonlearn-04-Functions (1).pptx
PDF
Functions_19_20.pdf
PDF
C++ Course - Lesson 2
PDF
Advanced Web Technology ass.pdf
PDF
Functions2.pdf
PPTX
Generators & Decorators.pptx
PPTX
Python-Magic-Methodsndndnndndndndndnnddnndndndndndn
PPTX
Using-Python-Libraries.9485146.powerpoint.pptx
PPTX
Advance-Python-Iterators-for-developers.pptx
PDF
Functions_21_22.pdf
PDF
C++ Interview Question And Answer
PDF
C++ questions And Answer
Functions2.pptx
C++ code, please help! RESPOND W COMPLETED CODE PLEASE, am using V.pdf
Python Learn Function with example programs
Python lecture 03
Cc code cards
Python_Functions_Unit1.pptx
Introduction to python programming 2
Pythonlearn-04-Functions (1).pptx
Functions_19_20.pdf
C++ Course - Lesson 2
Advanced Web Technology ass.pdf
Functions2.pdf
Generators & Decorators.pptx
Python-Magic-Methodsndndnndndndndndnnddnndndndndndn
Using-Python-Libraries.9485146.powerpoint.pptx
Advance-Python-Iterators-for-developers.pptx
Functions_21_22.pdf
C++ Interview Question And Answer
C++ questions And Answer
Ad

More from Nina Zakharenko (7)

PDF
Recovering From Git Mistakes - Nina Zakharenko
PDF
Code Review Skills for Pythonistas - Nina Zakharenko - EuroPython
PDF
How to successfully grow a code review culture
PDF
Memory Management In Python The Basics
PDF
Djangocon 2014 - Django REST Framework - So Easy You Can Learn it in 25 Minutes
PDF
Nina Zakharenko - Introduction to Git - Start SLC 2015
PDF
Djangocon 2014 angular + django
Recovering From Git Mistakes - Nina Zakharenko
Code Review Skills for Pythonistas - Nina Zakharenko - EuroPython
How to successfully grow a code review culture
Memory Management In Python The Basics
Djangocon 2014 - Django REST Framework - So Easy You Can Learn it in 25 Minutes
Nina Zakharenko - Introduction to Git - Start SLC 2015
Djangocon 2014 angular + django
Ad

Recently uploaded (20)

PDF
SM_6th-Sem__Cse_Internet-of-Things.pdf IOT
PPTX
Lecture Notes Electrical Wiring System Components
PPTX
MET 305 2019 SCHEME MODULE 2 COMPLETE.pptx
PDF
Enhancing Cyber Defense Against Zero-Day Attacks using Ensemble Neural Networks
PPTX
OOP with Java - Java Introduction (Basics)
PDF
Mitigating Risks through Effective Management for Enhancing Organizational Pe...
PDF
R24 SURVEYING LAB MANUAL for civil enggi
PDF
Embodied AI: Ushering in the Next Era of Intelligent Systems
DOCX
ASol_English-Language-Literature-Set-1-27-02-2023-converted.docx
PPT
Mechanical Engineering MATERIALS Selection
PPTX
Construction Project Organization Group 2.pptx
PPT
CRASH COURSE IN ALTERNATIVE PLUMBING CLASS
PPTX
FINAL REVIEW FOR COPD DIANOSIS FOR PULMONARY DISEASE.pptx
PDF
BMEC211 - INTRODUCTION TO MECHATRONICS-1.pdf
PDF
keyrequirementskkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk
PPTX
web development for engineering and engineering
PPTX
CYBER-CRIMES AND SECURITY A guide to understanding
PPTX
UNIT-1 - COAL BASED THERMAL POWER PLANTS
PPTX
additive manufacturing of ss316l using mig welding
PPTX
Infosys Presentation by1.Riyan Bagwan 2.Samadhan Naiknavare 3.Gaurav Shinde 4...
SM_6th-Sem__Cse_Internet-of-Things.pdf IOT
Lecture Notes Electrical Wiring System Components
MET 305 2019 SCHEME MODULE 2 COMPLETE.pptx
Enhancing Cyber Defense Against Zero-Day Attacks using Ensemble Neural Networks
OOP with Java - Java Introduction (Basics)
Mitigating Risks through Effective Management for Enhancing Organizational Pe...
R24 SURVEYING LAB MANUAL for civil enggi
Embodied AI: Ushering in the Next Era of Intelligent Systems
ASol_English-Language-Literature-Set-1-27-02-2023-converted.docx
Mechanical Engineering MATERIALS Selection
Construction Project Organization Group 2.pptx
CRASH COURSE IN ALTERNATIVE PLUMBING CLASS
FINAL REVIEW FOR COPD DIANOSIS FOR PULMONARY DISEASE.pptx
BMEC211 - INTRODUCTION TO MECHATRONICS-1.pdf
keyrequirementskkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk
web development for engineering and engineering
CYBER-CRIMES AND SECURITY A guide to understanding
UNIT-1 - COAL BASED THERMAL POWER PLANTS
additive manufacturing of ss316l using mig welding
Infosys Presentation by1.Riyan Bagwan 2.Samadhan Naiknavare 3.Gaurav Shinde 4...

Elegant Solutions for Everyday Python Problems Pycon 2018 - Nina Zakharenko

  • 2. slides: h!p://bit.ly/elegant-python This talk is for you if: You're an intermediate python programmer You're coming to python from another language You want to learn about language features like: magic methods, iterators, decorators, and context managers @nnja
  • 4. How do we make code elegant? We pick the right tool for the job! Resources for converting from Python 2 -> 3
  • 5. Beauty is in the eye of the beholder
  • 7. Magic methods start and end with a double underscore (dunder) By implementing a few straightforward magic methods, you can make your objects behave like built-ins such as: numbers lists dictionaries and more... @nnja
  • 8. class Money: currency_rates = { '$': 1, '€': 0.88, } def __init__(self, symbol, amount): self.symbol = symbol self.amount = amount def __str__(self): return '%s%.2f' % (self.symbol, self.amount) @nnja
  • 9. class Money: currency_rates = { '$': 1, '€': 0.88, } def __init__(self, symbol, amount): self.symbol = symbol self.amount = amount def __str__(self): return '%s%.2f' % (self.symbol, self.amount) @nnja
  • 10. class Money: # defined currency_rates, __init__, and str above... def convert(self, other): """Convert other amount to our currency""" new_amount = ( other.amount / self.currency_rates[other.symbol] * self.currency_rates[self.symbol]) return Money(self.symbol, new_amount) @nnja
  • 11. __str__ in action >>> soda_cost = Money('$', 5.25) >>> print(soda_cost) $5.25 >>> pizza_cost = Money('€', 7.99) >>> print(pizza_cost) €7.99 @nnja
  • 12. class Money: def __add__(self, other): """ Add 2 Money instances using '+' """ new_amount = self.amount + self.convert(other).amount return Money(self.symbol, new_amount) @nnja
  • 13. >>> soda_cost = Money('$', 5.25) >>> pizza_cost = Money('€', 7.99) >>> print(soda_cost + pizza_cost) $14.33 More on Magic Methods: Dive into Python3 - Special Method Names
  • 14. >>> soda_cost = Money('$', 5.25) >>> pizza_cost = Money('€', 7.99) >>> print(soda_cost + pizza_cost) $14.33 >>> print(pizza_cost + soda_cost) €12.61 More on Magic Methods: Dive into Python3 - Special Method Names
  • 15. some magic methods map to symbols >>> d = {'one': 1, 'two': 2} >>> d['two'] 2 >>> d.__getitem__('two') 2 @nnja
  • 16. other magic methods map to built-in functions class SquareShape: def __len__(self): """ Return the number of sides in our shape """ return 4 >>> my_square = SquareShape() >>> len(my_square) 4 @nnja
  • 18. Making classes iterable In order to be iterable, a class needs to implement __iter__() __iter__() must return an iterator In order to be an iterator a class needs to implement __next__() which must raise StopIteration when there are no more items to return Great explanation of iterable vs. iterator vs. generator
  • 19. Scenario.. We have a Server instance running services on different ports. Some services are active, some are inactive. When we loop over our the Server instance, we only want to loop over active services. @nnja
  • 20. class IterableServer: services = [ {'active': False, 'protocol': 'ftp', 'port': 21}, {'active': True, 'protocol': 'ssh', 'port': 22}, {'active': True, 'protocol': 'http', 'port': 80}, ] @nnja
  • 21. class IterableServer: def __init__(self): self.current_pos = 0 def __iter__(self): # can return self, because __next__ implemented return self def __next__(self): while self.current_pos < len(self.services): service = self.services[self.current_pos] self.current_pos += 1 if service['active']: return service['protocol'], service['port'] raise StopIteration @nnja
  • 22. class IterableServer: def __init__(self): self.current_pos = 0 def __iter__(self): # can return self, because __next__ implemented return self def __next__(self): while self.current_pos < len(self.services): service = self.services[self.current_pos] self.current_pos += 1 if service['active']: return service['protocol'], service['port'] raise StopIteration @nnja
  • 23. >>> for protocol, port in IterableServer(): print('service %s on port %d' % (protocol, port)) service ssh on port 22 service http on port 80 loops over all active services ... not bad @nnja
  • 24. tip: use a generator when your iterator doesn't need to maintain a lot of state (which is most of the time) @nnja
  • 25. class Server: services = [ {'active': False, 'protocol': 'ftp', 'port': 21}, {'active': True, 'protocol': 'ssh', 'port': 22}, {'active': True, 'protocol': 'http', 'port': 21}, ] def __iter__(self): for service in self.services: if service['active']: yield service['protocol'], service['port'] @nnja
  • 26. class Server: services = [ {'active': False, 'protocol': 'ftp', 'port': 21}, {'active': True, 'protocol': 'ssh', 'port': 22}, {'active': True, 'protocol': 'http', 'port': 21}, ] def __iter__(self): for service in self.services: if service['active']: yield service['protocol'], service['port'] @nnja
  • 27. Why does this work? use single parenthesis ( ) to create a generator comprehension ^ technically, a generator expression but I like this term better, and so does Ned Batchelder >>> my_gen = (num for num in range(1)) >>> my_gen <generator object <genexpr> at 0x107581bf8> @nnja
  • 28. An iterator must implement __next__() >>> next(my_gen) # __next__() maps to built-in next() 0 and raise StopIteration when there are no more elements >>> next(my_gen) ... StopIteration Traceback (most recent call last) see itertools for working with iterators
  • 30. alias methods class Word: def __init__(self, word): self.word = word def __repr__(self): return self.word def __add__(self, other_word): return Word('%s %s' % (self.word, other_word)) # Add an alias from method __add__ to the method concat concat = __add__ @nnja
  • 31. We can add an alias from __add__ to concat because methods are just objects >>> # remember, concat = __add__ >>> first_name = Word('Max') >>> last_name = Word('Smith') >>> first_name + last_name Max Smith >>> first_name.concat(last_name) Max Smith >>> Word.__add__ == Word.concat True @nnja
  • 32. Dog class >>> class Dog: sound = 'Bark' def speak(self): print(self.sound + '!', self.sound + '!') >>> my_dog = Dog() >>> my_dog.speak() Bark! Bark! read the docs
  • 33. getattr(object, name, default) >>> class Dog: sound = 'Bark' def speak(self): print(self.sound + '!', self.sound + '!') >>> my_dog = Dog() >>> my_dog.speak() Bark! Bark! >>> getattr(my_dog, 'speak') <bound method Dog.speak of <__main__.Dog object at 0x10b145f28>> >>> speak_method = getattr(my_dog, 'speak') >>> speak_method() Bark! Bark!
  • 34. getattr(object, name, default) >>> class Dog: sound = 'Bark' def speak(self): print(self.sound + '!', self.sound + '!') >>> my_dog = Dog() >>> my_dog.speak() Bark! Bark! >>> getattr(my_dog, 'speak') <bound method Dog.speak of <__main__.Dog object at 0x10b145f28>> >>> speak_method = getattr(my_dog, 'speak') >>> speak_method() Bark! Bark!
  • 35. Example: command line tool with dynamic commands class Operations: def say_hi(self, name): print('Hello,', name) def say_bye(self, name): print('Goodbye,', name) def default(self, arg): print('This operation is not supported.') if __name__ == '__main__': operations = Operations() # let's assume error handling command, argument = input('> ').split() getattr(operations, command, operations.default)(argument) read the docs
  • 36. Output $ python demo.py > say_hi Nina Hello, Nina > blah blah This operation is not supported. ✨ additional reading - inverse of getattr() is setattr()
  • 37. functools.partial(func, *args, **kwargs) Return a new partial object which behaves like func called with args & kwargs if more arguments are passed in, they are appended to args if more keyword arguments are passed in, they extend and override kwargs read the docs on partials
  • 38. functool.partial(func, *args, **kwargs) # We want to be able to call this function on any int # without having to specify the base. >>> int('10010', base=2) 18 >>> from functools import partial >>> basetwo = partial(int, base=2) >>> basetwo <functools.partial object at 0x1085a09f0> >>> basetwo('10010') 18 read the docs on partials
  • 39. functool.partial(func, *args, **kwargs) # We want to be able to call this function on any int # without having to specify the base. >>> int('10010', base=2) 18 >>> from functools import partial >>> basetwo = partial(int, base=2) >>> basetwo <functools.partial object at 0x1085a09f0> >>> basetwo('10010') 18 read the docs on partials
  • 40. functool.partial(func, *args, **kwargs) # We want to be able to call this function on any int # without having to specify the base. >>> int('10010', base=2) 18 >>> from functools import partial >>> basetwo = partial(int, base=2) >>> basetwo <functools.partial object at 0x1085a09f0> >>> basetwo('10010') 18 read the docs on partials
  • 41. magic methods & method magic in action! photo credit
  • 42. library I ! : github.com/mozilla/agithub agithub is a (poorly named) REST API client with transparent syntax which facilitates rapid prototyping — on any REST API! Implemented in ~400 lines. Add support for any REST API in ~30 lines of code. agithub knows everything it needs to about protocol (REST, HTTP, etc), but assumes nothing about your upstream API. @nnja
  • 43. define endpoint url & other connection properties class GitHub(API): def __init__(self, token=None, *args, **kwargs): props = ConnectionProperties( api_url = kwargs.pop('api_url', 'api.github.com')) self.setClient(Client(*args, **kwargs)) self.setConnectionProperties(props) @nnja
  • 44. then, start using the API! >>> gh = GitHub('token') >>> status, data = gh.user.repos.get() >>> # ^ Maps to GET /user/repos >>> data ... ['tweeter', 'snipey', '...'] github.com/mozilla/agithub
  • 45. 404 if we provide a path that doesn't exist: >>> gh = GitHub('token') >>> status, data = gh.this.path.doesnt.exist.get() >>> status ... 404 github.com/jpaugh/agithub
  • 47. class API: def __getattr__(self, key): return IncompleteRequest(self.client).__getattr__(key) __getitem__ = __getattr__ class IncompleteRequest: def __getattr__(self, key): if key in self.client.http_methods: htmlMethod = getattr(self.client, key) return partial(htmlMethod, url=self.url) else: self.url += '/' + str(key) return self __getitem__ = __getattr__ class Client: http_methods = ('get') # ... def get(self, url, headers={}, **params): return self.request('GET', url, None, headers) github.com/jpaugh/agithub source: base.py
  • 48. class API: def __getattr__(self, key): return IncompleteRequest(self.client).__getattr__(key) __getitem__ = __getattr__ class IncompleteRequest: def __getattr__(self, key): if key in self.client.http_methods: htmlMethod = getattr(self.client, key) return partial(htmlMethod, url=self.url) else: self.url += '/' + str(key) return self __getitem__ = __getattr__ class Client: http_methods = ('get') # ... def get(self, url, headers={}, **params): return self.request('GET', url, None, headers) github.com/jpaugh/agithub source: base.py
  • 49. class API: def __getattr__(self, key): return IncompleteRequest(self.client).__getattr__(key) __getitem__ = __getattr__ class IncompleteRequest: def __getattr__(self, key): if key in self.client.http_methods: htmlMethod = getattr(self.client, key) return partial(htmlMethod, url=self.url) else: self.url += '/' + str(key) return self __getitem__ = __getattr__ class Client: http_methods = ('get') # ... def get(self, url, headers={}, **params): return self.request('GET', url, None, headers) github.com/jpaugh/agithub source: base.py
  • 51. When should I use one? Need to perform an action before and/or after an operation. Common scenarios: Closing a resource after you're done with it (file, network connection) Perform cleanup before/after a function call @nnja
  • 52. Example Problem: Feature Flags Turn features of your application on and off easily. Uses of feature flags: A/B Testing Rolling Releases Show Beta version to users opted-in to Beta Testing Program More on Feature Flags
  • 53. class FeatureFlags: SHOW_BETA = 'Show Beta version of Home Page' flags = { SHOW_BETA: True } @classmethod def is_on(cls, name): return cls.flags[name] @classmethod def toggle(cls, name, value): cls.flags[name] = value feature_flags = FeatureFlags() @nnja
  • 54. How do we temporarily turn features on and off when testing flags? Want: with feature_flag(FeatureFlags.SHOW_BETA): assert '/beta' == get_homepage_url() @nnja
  • 55. Using Magic Methods __enter__ and __exit__ class feature_flag: """ Implementing a Context Manager using Magic Methods """ def __init__(self, name, on=True): self.name = name self.on = on self.old_value = feature_flags.is_on(name) def __enter__(self): feature_flags.toggle(self.name, self.on) def __exit__(self, *args): feature_flags.toggle(self.name, self.old_value) See: contextlib.contextmanager
  • 56. The be!er way: using the contextmanager decorator from contextlib import contextmanager @contextmanager def feature_flag(name, on=True): old_value = feature_flags.is_on(name) feature_flags.toggle(name, on) yield feature_flags.toggle(name, old_value) See: contextlib.contextmanager
  • 57. The be!er way: using the contextmanager decorator from contextlib import contextmanager @contextmanager def feature_flag(name, on=True): """ The easier way to create Context Managers """ old_value = feature_flags.is_on(name) # behavior of __enter__() feature_flags.toggle(name, on) yield # behavior of __exit__() feature_flags.toggle(name, old_value) See: contextlib.contextmanager
  • 58. Note: yield? from contextlib import contextmanager @contextmanager def feature_flag(name, on=True): """ The easier way to create Context Managers """ old_value = feature_flags.is_on(name) feature_flags.toggle(name, on) # behavior of __enter__() yield feature_flags.toggle(name, old_value) # behavior of __exit__() See: contextlib.contextmanager
  • 59. either implementation def get_homepage_url(): """ Returns the path of the page to display """ if feature_flags.is_on(FeatureFlags.SHOW_BETA): return '/beta' else: return '/homepage' def test_homepage_url_with_context_manager(): with feature_flag(FeatureFlags.SHOW_BETA): # saw the beta homepage... assert get_homepage_url() == '/beta' with feature_flag(FeatureFlags.SHOW_BETA, on=False): # saw the standard homepage... assert get_homepage_url() == '/homepage'
  • 60. either implementation def get_homepage_url(): """ Returns the path of the page to display """ if feature_flags.is_on(FeatureFlags.SHOW_BETA): return '/beta' else: return '/homepage' def test_homepage_url_with_context_manager(): with feature_flag(FeatureFlags.SHOW_BETA): assert get_homepage_url() == '/beta' print('seeing the beta homepage...') with feature_flag(FeatureFlags.SHOW_BETA, on=False): assert get_homepage_url() == '/homepage' print('seeing the standard homepage...')
  • 61. Decorators The simple explanation: Syntactic sugar that allows modification of an underlying function. @nnja
  • 62. Decorators: Wrap a function in another function. Do something: before the call after the call with provided arguments modify the return value or arguments @nnja
  • 63. class User: is_authenticated = False def __init__(self, name): self.name = name Throw an exception if trying to access data only for logged in users: def display_profile_page(user): """ Display profile page for logged in User """ if not user.is_authenticated: raise Exception('User must login.') print('Profile: %s' % user.name)
  • 64. def enforce_authentication(func): def wrapper(user): if not user.is_authenticated: raise Exception('User must login.') return func(user) return wrapper the important logic: def enforce_authentication(func): def wrapper(user): if not user.is_authenticated: raise Exception('User must login.') return func(user) return wrapper @nnja
  • 65. Using enforce_authentication without a decorator: enforce_authentication(display_profile_page)(some_user) Or, as a decorator: @enforce_authentication def display_profile_page(user): print('Profile: %s' % user.name) Now this raises an Exception if unauthenticated: user = User('nina') display_profile_page(nina)
  • 66. Problem: lost context using a decorator >>> display_profile_page.__name__ 'wrapper' >>>> display_profile_page.__doc__ # ... empty Solution: Use contextlib.wraps from contextlib import wraps def enforce_authentication(func): @wraps(func) def wrapper(user): # ... rest of the code
  • 69. By using ContextDecorator you can easily write classes that can be used both as decorators with @ and context managers with the with statement. ContextDecorator is used by contextmanager(), so you get this functionality automatically . Or, you can write a class that extends from ContextDecorator or uses ContextDecorator as a mixin, and implements __enter__, __exit__ and __call__ @nnja
  • 70. Remember @contextmanager from earlier? from contextlib import contextmanager @contextmanager def feature_flag(name, on=True): old_value = feature_flags.is_on(name) feature_flags.toggle(name, on) yield feature_flags.toggle(name, old_value) @nnja
  • 71. use it as a context manager with feature_flag(FeatureFlags.SHOW_BETA): assert get_homepage_url() == '/beta' or use as a decorator @feature_flag(FeatureFlags.SHOW_BETA, on=False) def get_profile_page(): beta_flag_on = feature_flags.is_on( FeatureFlags.SHOW_BETA) if beta_flag_on: return 'beta.html' else: return 'profile.html'
  • 72. library I ! : freezegun lets your python tests ❇ travel through time! ❇ from freezegun import freeze_time # use it as a Context Manager def test(): with freeze_time("2012-01-14"): assert datetime.datetime.now() == datetime.datetime( 2012, 1, 14) assert datetime.datetime.now() != datetime.datetime(2012, 1, 14) # or a decorator @freeze_time("2012-01-14") def test(): assert datetime.datetime.now() == datetime.datetime(2012, 1, 14) read the source sometime, it's mind-bending!
  • 73. NamedTuple Useful when you need lightweight representations of data. Create tuple subclasses with named fields. @nnja
  • 74. Example from collections import namedtuple CacheInfo = namedtuple( "CacheInfo", ["hits", "misses", "max_size", "curr_size"]) @nnja
  • 75. Giving NamedTuples default values RoutingRule = namedtuple( 'RoutingRule', ['prefix', 'queue_name', 'wait_time'] ) (1) By specifying defaults RoutingRule.__new__.__defaults__ = (None, None, 20) (2) or with _replace to customize a prototype instance default_rule = RoutingRule(None, None, 20) user_rule = default_rule._replace( prefix='user', queue_name='user-queue')
  • 76. NamedTuples can be subclassed and extended class Person(namedtuple('Person', ['first_name', 'last_name'])): """ Stores first and last name of a Person""" def __str__(self): return '%s %s' % (self.first_name, self.last_name) >>> me = Person('nina', 'zakharenko') >>> str(me) 'nina zakharenko' >>> me Person(first_name='nina', last_name='zakharenko')
  • 77. New Tools Magic Methods make your objects behave like builtins (numbers, list, dict, etc) Method ❇Magic❇ alias methods getattr @nnja
  • 78. ContextManagers Manage resources Decorators do something before/after call, modify return value or validate arguments ContextDecorators ContextManagers + Decorators in one @nnja
  • 79. Iterators & Generators Loop over your objects yield NamedTuple Lightweight classes @nnja
  • 80. "Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away." — Antoine de Saint-Exupery @nnja