Some Pitfalls with Python and Their Possible Solutions v1.0

5/296
The Bad
The Good
The Weird

6/296
The Bad
The Good
The Weird
The Good

Yann-Gaël Guéhéneuc
Yann-Gaël Guéhéneuc
(/jan/, he/il)
Work licensed under Creative Commons
BY-NC-SA 4.0 International
Python: The Good,
the Bad, the Weird
yann-gael.gueheneuc@concordia.ca
Version 1.0
2024/05/31

10/296
Not a eulogy
Not a rant

11/296
Not a eulogy
Not a rant
Not a
course

12/296
Outline
1. Background
2. Quality Criteria
3. Special Objects
4. Collection API
5. Attributes
6. Methods
7. Resources
8. Polymorphism
9. Inheritance
10. Misc.
11. (Im)Mutability
12. Metaclasses
13. Conclusion

13/296
Outline
1. Background
2. Quality Criteria
3. Special Objects
4. Collection API
5. Attributes
6. Methods
7. Resources
8. Polymorphism
9. Inheritance
10. Misc.
11. (Im)Mutability
12. Metaclasses
13. Conclusion

14/296
BACKGROUND
Section Background

15/296
Background
 Educated with
– Scheme (a Prolog interpreter)
– C (system/network programming)
• And C++, but only for reflection
– Smalltalk (OO design)
– Java (since c. 1996)

17/296
Background
– Ptidej Tool Suite
• 550,000+ Java LOC

18/296
Background
– Ptidej Tool Suite
• 550,000+ Java LOC
– AmiModRadio, others
• 40,000+ C code

19/296
Disclaimer





20/296
Disclaimer
 Not a Python programmer ⠦
 But an old programmer ѿ



21/296
Disclaimer
 Not a Python programmer ⠦
 But an old programmer ѿ
 Not a course on Python
 Nor a review of its libraries

22/296
Resources
 https://github.com/ptidejteam/tutorials-
PythonPitfalls
 yann-gael.gueheneuc@concordia.ca

23/296
Outline
1. Background
2. Quality Criteria
3. Special Objects
4. Collection API
5. Attributes
6. Methods
7. Resources
8. Polymorphism
9. Inheritance
10. Misc.
11. (Im)Mutability
12. Metaclasses
13. Conclusion

24/296
QUALITY CRITERIA
Section Quality Criteria

25/296
Content
 Definition
– Defensive programming
– Principle of least surprise
– Principle of locality
– Present and future productivity
 Evaluation
– Quality criteria
– Likert scale

26/296
Content
 Definition
 Evaluation
– Quality criteria
– Likert scale

27/296
Defensive Programming
 Typing and execution
https://www.cleverti.com/blog/computer-science/strongly-vs-weakly-typed-languages/

28/296
https://www.cleverti.com/blog/computer-science/strongly-vs-weakly-typed-languages/
Compiled
Interpreted
Erlang
Clojure
Python
Perl
VB
Groovy
Ruby
Magik
PHP
JavaScript F#
Java
C C++
Scala
Haskell
Python PyPy
Python PyPy
Python PyPy
Python PyPy Ruby YJIY
Ruby YJIY
Ruby YJIY
Ruby YJIY
JS V8
JS V8
JS V8
JS V8 C#

29/296
 Catching more errors before execution
requires more effort upfront
– Less “freedom” for developers
 Giving more “freedom” to developers
– Requires catching more errors during execution

32/296
Here ͦ

33/296
 “Defensive programming is an approach in
which the programmer assumes that there
may be undetected faults or inconsistencies
in code.”
 Also, it assumes that users call the API
– With the “wrong” parameter values
– In the “wrong” order

34/296
Principle of Least Surprise
Advances in Computers, volume 12, 1972, pages 175-284

35/296

36/296

37/296
Here ͦ

38/296
 Principle of least astonishment / surprise
– “[T]he designers of a systems programming
language should obey the “Law of Least
Astonishment”.”
– “[E]very construct in the system should behave
exactly as its syntax suggests.”
– “Widely accepted conventions should be
followed whenever possible, and exceptions to
[…] rules of the language should be minimal.”
Also from "Law of Least Astonishment" appeared in the PL/I Bulletin in 1967

39/296
Principle of Locality
 Principle of locality
– Process
– Implementation

40/296
– Process
– Implementation

41/296
– Process
– Implementation
• “The primary feature for
easy maintenance is
locality: Locality is that
characteristic of source
code that enables a
programmer to
understand that source
by looking at only a small
portion of it.”
https://dev.to/ralphcone/new-hot-trend-locality-of-behavior-1g9k

42/296
Present and Future Productivity

43/296
 A Message to the Future
(by Linda Rising)
– “Think of every line of
code you write as a
message for someone in
the future”
• “Pretend you're explaining
to this smart [programmer]
how to solve this tough
problem”
– “That the smart
programmer in the
future [should] see your
code and say, 'Wow!
This is great!”
• “I can understand
perfectly what's been
done here and I'm
amazed at [its elegance]”
• “I'm going to show the
other folks on my team”

44/296
 Write Code as If You Had to Support It for
the Rest of Your Life (by Yuriy Zubarev)
– “[H]ow do you keep up with all the best practices
you've learned and how do you make them an
integral part of your programming practice?”
– “I think the answer lies in your frame of mind or,
more plainly, in your attitude.”

45/296
Content
 Definition
 Evaluation
– Criteria
– Scale

46/296
Evaluation
Criteria
 Four quality criteria
– Present and future
productivity
Scale (3-point Likert scale)
 Good
 Bad
 Weird

47/296
Outline
1. Background
2. Quality Criteria
3. Special Objects
4. Collection API
5. Attributes
6. Methods
7. Resources
8. Polymorphism
9. Inheritance
10. Misc.
11. (Im)Mutability
12. Metaclasses
13. Conclusion

48/296
SPECIAL OBJECTS
Section Special Objects

49/296
SPECIAL OBJECTS
Also the name of the
corresponding project
in the GitHub repo.
Section Special Objects

50/296
__builtins__ Module
 On Windows, with Python v3.12.1
– 3.12.1 (tags/v3.12.1:2305ca5, Dec 7 2023,
22:03:25) [MSC v.1937 64 bit (AMD64)]





51/296
__builtins__ Module
– 3.12.1 (tags/v3.12.1:2305ca5, Dec 7 2023,
22:03:25) [MSC v.1937 64 bit (AMD64)]
 158 built-in objects (!)
 97 built-in classes
 44 built-in functions
 17 others objects

52/296
__builtins__ Module

53/296
__builtins__ Module
– Values can be changed!?

54/296
__builtins__ Module
def testBuiltIBooleans2(self):
for bi in __builtins__:
if isinstance(__builtins__[bi], bool):
__builtins__[bi] = True
numberOfBooleans = 0
booleans = []
for bi in __builtins__.values():
if isinstance(bi, bool):
booleans.append(bi)
numberOfBooleans += 1
self.assertEqual(numberOfBooleans, 3)
self.assertEqual(booleans[0], True)
self.assertEqual("Hello" == "Hello", True)
self.assertEqual("Hello" == "World", False)

55/296
__builtins__ Module
booleans = []
booleans.append(bi)

56/296
__builtins__ Module
booleans = []
booleans.append(bi)
Magic?

57/296
__builtins__ Module
booleans = []
booleans.append(bi)
Magic?
Keywords!

58/296
__builtins__ Module
booleans = []
booleans.append(bi)
Magic?
Keywords!
Defensive programming
Principle of least surprise
Principle of locality
Present and future productivity

59/296
__builtins__ Module
 Recommendation
– Do not access (read, write) __builtins__

60/296
Special Methods
 Function vs. Methods
print(len("Hello, World!"))
class MyClass:
def __len__(self):
return 42
m = MyClass()
print(len(m))
print(m.__len__())

61/296
Special Methods
class MyClass:
def __len__(self):
return 42
m = MyClass()
print(len(m))
print(m.__len__())
Function call
Method call

62/296
Special Methods
class MyClass:
def __len__(self):
return 42
m = MyClass()
print(len(m))
print(m.__len__())
# print(m.len()) # AttributeError: 'MyClass' object has no attribute 'len'
# print(m.length()) # AttributeError: 'MyClass' object has no attribute 'length'
Function call
Method call

63/296
Special Methods
def testSpecialMethods(self):
x = [0, 1, 4, 3, 2, 1]

64/296
Special Methods
x = [0, 1, 4, 3, 2, 1]
self.assertEqual(x.count(1), 2)
self.assertEqual(len(x), 6)
self.assertTrue(isinstance(any(x), int))

65/296
Special Methods
x = [0, 1, 4, 3, 2, 1]
Method and
functions

66/296
Special Methods
x = [0, 1, 4, 3, 2, 1]
self.assertEqual(x[0], 0)
x.reverse()
Method and
functions

67/296
Special Methods
x = [0, 1, 4, 3, 2, 1]
x.reverse()
Method and
functions
Method mutates
receiver

68/296
Special Methods
x = [0, 1, 4, 3, 2, 1]
x.reverse()
x2 = list(reversed(x))
self.assertEqual(x2[0], 0)
Method and
functions
Method mutates
receiver

69/296
Special Methods
x = [0, 1, 4, 3, 2, 1]
x.reverse()
Method and
functions
Method mutates
receiver
Function create
new object

70/296
Special Methods
x = [0, 1, 4, 3, 2, 1]
x.reverse()
x.sort()
Method and
functions
Method mutates
receiver
Function create
new object

71/296
Special Methods
x = [0, 1, 4, 3, 2, 1]
x.reverse()
x.sort()
Method and
functions
Method mutates
receiver
Function create
new object

72/296
Special Methods
x = [0, 1, 4, 3, 2, 1]
x.reverse()
x.sort()
x2 = list(sorted(x))
Method and
functions
Method mutates
receiver
Function create
new object

73/296
Special Methods
x = [0, 1, 4, 3, 2, 1]
x.reverse()
x.sort()
x2 = list(sorted(x))
Method and
functions
Method mutates
receiver
Function create
new object

74/296
Special Methods
class MyClass:
def __len__(self):
return 42
m = MyClass()
print(len(m))
print(m.__len__())

75/296
Special Methods
class MyClass:
def __len__(self):
return 42
m = MyClass()
print(len(m))
print(m.__len__())

76/296
Special Methods
class MyClass:
def __len__(self):
return 42
m = MyClass()
print(len(m))
print(m.__len__())

77/296
Outline
1. Background
2. Quality Criteria
3. Special Objects
4. Collection API
5. Attributes
6. Methods
7. Resources
8. Polymorphism
9. Inheritance
10. Misc.
11. (Im)Mutability
12. Metaclasses
13. Conclusion

78/296
COLLECTION API
Section Collection API

79/296
Working with Lists
 Comprehensions and variants

80/296
Working with Lists
def testConditionalRanges(self):
result = [i for i in range(10) if i % 2 == 0]
expected = [0, 2, 4, 6, 8]
self.assertEqual(result, expected)

81/296
Working with Lists
expected = [0, 2, 4, 6, 8]
def testExpressionRanges(self):
result = [i + 1 for i in range(10)]
expected = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

82/296
Working with Lists
expected = [0, 2, 4, 6, 8]
def testExpressionRanges(self):
result = [i + 1 for i in range(10)]
expected = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
def testDoubleRanges(self):
result = [(a, b) for a in range(10) for b in range(10)]
expected = [(0, 0), (0, 1), (0, 2), (0, 3), (0, 4), (0, 5), (0, 6), (0, 7),
(0, 8), (0, 9), (1, 0), (1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (1, 6),
(1, 7), (1, 8), (1, 9), (2, 0), (2, 1), (2, 2), (2, 3), (2, 4), (2, 5),
(2, 6), (2, 7), (2, 8), (2, 9), (3, 0), (3, 1), (3, 2), (3, 3), (3, 4),
(3, 5), (3, 6), (3, 7), (3, 8), (3, 9), <Many…> (9, 0), (9, 1), (9, 2),
(9, 3), (9, 4), (9, 5), (9, 6), (9, 7), (9, 8), (9, 9)]

83/296
Working with Lists
 Combining, pivoting, and enumerating

84/296
Working with Lists
def testCombining(self):
a = ["x", "y", "z"] # Strings (three)
b = [3, 5, 7, 9] # Integers (four)
c = [2.1, 2.5] # Floats (two)
result = list(zip(a, b, c))
expected = [("x", 3, 2.1), ("y", 5, 2.5)]

85/296
Working with Lists
b = [3, 5, 7, 9] # Integers (four)
c = [2.1, 2.5] # Floats (two)
expected = [("x", 3, 2.1), ("y", 5, 2.5)]
def testPivoting(self):
a = [['x', 3, 2.1], ['y', 5, 2.5], ['z', 7, 2.9]]
result = list(zip(*a))
expected = [('x', 'y', 'z'), (3, 5, 7), (2.1, 2.5, 2.9)]

86/296
Working with Lists
b = [3, 5, 7, 9] # Integers (four)
c = [2.1, 2.5] # Floats (two)
expected = [("x", 3, 2.1), ("y", 5, 2.5)]
a = [['x', 3, 2.1], ['y', 5, 2.5], ['z', 7, 2.9]]
expected = [('x', 'y', 'z'), (3, 5, 7), (2.1, 2.5, 2.9)]
def testEnumerating(self):
a = ["quick", "brown", "fox", "jumps", "over"]
result = list(enumerate(a))
expected = [(0, "quick"), (1, "brown"), (2, "fox"), (3, "jumps"), (4, "over")]

87/296
Working with Lists
b = [3, 5, 7, 9] # Integers (four)
c = [2.1, 2.5] # Floats (two)
expected = [("x", 3, 2.1), ("y", 5, 2.5)]
a = [['x', 3, 2.1], ['y', 5, 2.5], ['z', 7, 2.9]]
expected = [('x', 'y', 'z'), (3, 5, 7), (2.1, 2.5, 2.9)]
def testEnumerating(self):
a = ["quick", "brown", "fox", "jumps", "over"]
result = list(enumerate(a))
expected = [(0, "quick"), (1, "brown"), (2, "fox"), (3, "jumps"), (4, "over")]
Unpacking operator

88/296
Working with Lists
 Slices
def setUp(self):
unittest.TestCase.setUp(self)
self.l = [1, 2, 3]
def testAppending(self):
self.l[len(self.l):] = [4]
self.assertEqual(self.l, [1, 2, 3, 4])
def testPrepending(self):
self.l[:0] = [0]
def testReplacing(self):
self.l[:2] = ['a', 'b', 'c’]
self.assertEqual(self.l, ['a', 'b', 'c', 3])
def testInserting(self):
self.l[2:2] = ['a', 'b’]
self.assertEqual(self.l, [1, 2, 'a', 'b', 3])

89/296
Working with Lists
 Slices
def setUp(self):
unittest.TestCase.setUp(self)
self.l = [1, 2, 3]
def testAppending(self):
self.l[len(self.l):] = [4]
def testPrepending(self):
self.l[:0] = [0]
def testReplacing(self):
self.l[:2] = ['a', 'b', 'c’]
self.assertEqual(self.l, ['a', 'b', 'c', 3])
def testInserting(self):
self.l[2:2] = ['a', 'b’]
self.assertEqual(self.l, [1, 2, 'a', 'b', 3])

90/296
Comprehensions vs. Generators
Generators Comprehensions

91/296
def testComprehension(self):
start = time.time()
result = [i for i in range(10000000)]
end = time.time()
time_diff = end – start
self.assertEqual(len(str(result)), 88888890)
self.assertTrue(time_diff > 0.5) # Seconds

92/296
start = time.time()
end = time.time()
def testGenerator(self):
start = time.time()
result = (i for i in range(10000000))
end = time.time()
self.assertEqual(len(str(list(result))), 88888890)
self.assertTrue(time_diff < 0.1) # Seconds

93/296
start = time.time()
end = time.time()
def testGenerator(self):
start = time.time()
result = (i for i in range(10000000))
end = time.time()
self.assertEqual(len(str(list(result))), 88888890)
self.assertTrue(time_diff < 0.1) # Seconds

94/296
Collection API
http://dailusia.blog.fc2.com/blog-entry-451.html

95/296
Collection API
https://sangmoonoh.medium.com/just-class-diagram-for-python-3-collections-abstract-base-classes-e1eafde6ad25

96/296
Collection API
 Multiple implementations of queues
from collections import deque
def testCollectionsDeque(self):
q = deque([1, 2, 3])
q.append(4)
self.assertEqual(list(q), [1, 2, 3, 4])
q.appendleft(0)
self.assertEqual(list(q), [0, 1, 2, 3, 4])
result = q.pop()
self.assertEqual(result, 4)
result = q.popleft()
self.assertEqual(list(q), [1, 2, 3])
q.insert(3, 4)
q.remove(3) self.assertEqual(list(q), [1, 2, 4])
q.extend([5, 6])
q.extendleft(['a', 'b', 'c’])
self.assertEqual(list(q), ['c', 'b', 'a', 1, 2, 4, 5, 6])

97/296
Collection API
q = deque([1, 2, 3])
q.append(4)
q.appendleft(0)
result = q.pop()
q.insert(3, 4)
q.extend([5, 6])
from queue import Queue
def testQueueQueue(self):
q = Queue(8)
q.put(1)
q.put(2)
q.put(3)
q.put(4)
self.assertEqual(list(q.queue), [1, 2, 3, 4])
result = q.get()
self.assertEqual(list(q.queue), [2, 3, 4])

98/296
Collection API
q = deque([1, 2, 3])
q.append(4)
q.appendleft(0)
result = q.pop()
q.insert(3, 4)
q.extend([5, 6])
from queue import Queue
def testQueueQueue(self):
q = Queue(8)
q.put(1)
q.put(2)
q.put(3)
q.put(4)
self.assertEqual(list(q.queue), [1, 2, 3, 4])
result = q.get()
self.assertEqual(list(q.queue), [2, 3, 4])
Not really a queue

99/296
Collection API
 But some inconsistencies
 And no alternatives APIs,
no alternative implementations?
some_dict = {}
some_dict['foo'] = { 'bar': 2 }
value = some_dict.get('foo', {}).get('bar', {})
print(value)
some_dict['foo'] = { 'bar': [1, 2, 3] }
value = some_dict.get('foo', {}).get('bar', []).get(10)
print(value)
https://mail.python.org/archives/list/python-ideas@python.org/thread/LLK3EQ3QWNDB54SEBKJ4XEV4LXP5HVJS/

100/296
Collection API
some_dict = {}
print(value)
some_dict['foo'] = { 'bar': [1, 2, 3] }
print(value)

101/296
Collection API
some_dict = {}
print(value)
some_dict['foo'] = { 'bar': [1, 2, 3] }
print(value)
Pandas
Polars
…

102/296
Outline
1. Background
2. Quality Criteria
3. Special Objects
4. Collection API
5. Attributes
6. Methods
7. Resources
8. Polymorphism
9. Inheritance
10. Misc.
11. (Im)Mutability
12. Metaclasses
13. Conclusion

103/296
ATTRIBUTES
Section Attributes

104/296
All Attributes Are Dynamic
class A:
pass
a = A()

105/296
class A:
pass
a = A()
print()
print("A.attr = "1"")
A.attr = "1"
print(f"A.attr = {A.attr} (id = {id(A.attr)})")
print(f"a.attr = {a.attr} (id = {id(a.attr)})")

106/296
class A:
pass
a = A()
A.attr = "1"
A.attr = 1 (id = 140736437823448)
a.attr = 1 (id = 140736437823448)
print()
A.attr = "1"

107/296
class A:
pass
a = A()
A.attr = "1"
A.attr = 1 (id = 140736437823448)
a.attr = 1 (id = 140736437823448)
print()
print("a.attr = "2"")
a.attr = "2"
print()
A.attr = "1"

108/296
class A:
pass
a = A()
A.attr = "1"
A.attr = 1 (id = 140736437823448)
a.attr = 1 (id = 140736437823448)
print()
print("a.attr = "2"")
a.attr = "2"
a.attr = "2"
A.attr = 1 (id = 140736437823448)
a.attr = 2 (id = 140736437823496)
print()
A.attr = "1"

109/296
 Python automagically assign the value of a
class attribute to the instance attribute of the
same name

110/296
 Python automagically assign the value of a
class attribute to the instance attribute of the
same name

111/296
class B:
pass
b = B()
print()
print("b.attr = "1"")
b.attr = "1"
print(f"b.attr = {b.attr} (id = {id(b.attr)})")
print(f"B.attr = {B.attr} (id = {id(B.attr)})")
print()
B.attr = "2"

112/296
class B:
pass
b = B()
print()
b.attr = "1"
print()
B.attr = "2"
Throws an exception

113/296
class B:
pass
b = B()
print()
b.attr = "1"
try:
except:
print("AttributeError: type object 'B' has no attribute 'attr'")
print()
B.attr = "2"

114/296
class B:
pass
b = B()
print()
b.attr = "1"
try:
except:
print()
B.attr = "2"
b.attr = "1"
b.attr = 1 (id = 140736437823448)
<What error can it be?>
b.attr = "1"
b.attr = 1 (id = 140736437823448)
B.attr = 2 (id = 140736437823496)

115/296
class B:
pass
b = B()
print()
b.attr = "1"
try:
except:
print()
B.attr = "2"
b.attr = "1"
b.attr = 1 (id = 140736437823448)
AttributeError: type object 'B' has no attribute 'attr'
b.attr = "1"
b.attr = 1 (id = 140736437823448)
B.attr = 2 (id = 140736437823496)

116/296
 Even popular questions with popular
answers on StackOverflow confuses
class and instance variables!
https://stackoverflow.com/questions/6760685/what-is-the-best-way-of-implementing-singleton-in-python

117/296
 Even popular questions with popular
answers on StackOverflow confuses
class and instance variables!
https://stackoverflow.com/questions/6760685/what-is-the-best-way-of-implementing-singleton-in-python

118/296
 Read/Write accesses on classes behave as
expected in any other language
 Write accesses on instances behave
differently and shadow the class variable!
https://stackoverflow.com/questions/3434581/how-do-i-set-and-access-attributes-of-a-class
A.a_var1 = "New value for a_var1"
print(f"A.a_var1 = {A.a_var1} (id = {id(A.a_var1)})")
print(f"C.a_var1 = {C.a_var1} (id = {id(C.a_var1)})")
print(f"a1.a_var1 = {a1.a_var1} (id = {id(a1.a_var1)})")
a1.a_var1 = "Another value for a_var1"
A.a_var1 = New value for a_var1 (id = 2238584427760)
C.a_var1 = New value for a_var1 (id = 2238584427760)
a1.a_var1 = New value for a_var1 (id = 2238584427760)
a1.a_var1 = Another value for a_var1 (id = 2238584286432)

119/296
 Read/Write accesses on classes behave as
expected in any other language
 Write accesses on instances behave
differently and shadow the class variable!
https://stackoverflow.com/questions/3434581/how-do-i-set-and-access-attributes-of-a-class
A.a_var1 = "New value for a_var1"
a1.a_var1 = "Another value for a_var1"
a1.a_var1 = Another value for a_var1 (id = 2238584286432)
Same name, but now
an instance variable!

120/296
class A():
cls_var1 = 1
cls_var2 = 2
def __init__(self):
self.ins_var1 = "a"
self.ins_var2 = "b"
class B(A):
cls_var2 = 3
cls_var3 = 4
def __init__(self):
self.ins_var2 = "c"
self.ins_var3 = "d"

121/296
class A():
cls_var1 = 1
cls_var2 = 2
def __init__(self):
self.ins_var1 = "a"
self.ins_var2 = "b"
class B(A):
cls_var2 = 3
cls_var3 = 4
def __init__(self):
self.ins_var2 = "c"
self.ins_var3 = "d"
def testClassVariablesAccess(self):
self.assertEqual(A.cls_var1, 1)
self.assertEqual(A.cls_var2, 2)
self.assertEqual(B.cls_var1, 1)

122/296
class A():
cls_var1 = 1
cls_var2 = 2
def __init__(self):
self.ins_var1 = "a"
self.ins_var2 = "b"
class B(A):
cls_var2 = 3
cls_var3 = 4
def __init__(self):
self.ins_var2 = "c"
self.ins_var3 = "d"

123/296
class A():
cls_var1 = 1
cls_var2 = 2
def __init__(self):
self.ins_var1 = "a"
self.ins_var2 = "b"
class B(A):
cls_var2 = 3
cls_var3 = 4
def __init__(self):
self.ins_var2 = "c"
self.ins_var3 = "d"
def testInstanceVariablesAccess(self):
# Class variables
a = A()
self.assertEqual(a.cls_var1, 1)
self.assertEqual(a.cls_var2, 2)
b = B()
# Instance variables
self.assertEqual(a.ins_var1, "a")
self.assertEqual(a.ins_var2, "b")
self.assertEqual(b.ins_var2, "c")
self.assertEqual(b.ins_var3, "d")

124/296
class A():
cls_var1 = 1
cls_var2 = 2
def __init__(self):
self.ins_var1 = "a"
self.ins_var2 = "b"
class B(A):
cls_var2 = 3
cls_var3 = 4
def __init__(self):
self.ins_var2 = "c"
self.ins_var3 = "d"

125/296
class A():
cls_var1 = 1
cls_var2 = 2
def __init__(self):
self.ins_var1 = "a"
self.ins_var2 = "b"
class B(A):
cls_var2 = 3
cls_var3 = 4
def __init__(self):
self.ins_var2 = "c"
self.ins_var3 = "d"
def testInstanceVariablesDynamicCreation(self):
a1 = A()
a1.ins_varNew1 = "w"
self.assertEqual(a1.ins_var1, "a")
self.assertEqual(a1.ins_var2, "b")
self.assertEqual(a1.ins_varNew1, "w")
a2 = A()
a2.ins_varNew2 = "x"
self.assertEqual(a2.ins_var1, "a")
self.assertEqual(a2.ins_var2, "b")
self.assertEqual(a2.ins_varNew2, "x")
self.assertFalse(hasattr(a2, 'ins_varNew1’))
b = B()
self.assertFalse(hasattr(b, 'ins_varNew1’))
self.assertFalse(hasattr(b, 'ins_varNew2'))

126/296
class A():
cls_var1 = 1
cls_var2 = 2
def __init__(self):
self.ins_var1 = "a"
self.ins_var2 = "b"
class B(A):
cls_var2 = 3
cls_var3 = 4
def __init__(self):
self.ins_var2 = "c"
self.ins_var3 = "d"

127/296
class A():
cls_var1 = 1
cls_var2 = 2
def __init__(self):
self.ins_var1 = "a"
self.ins_var2 = "b"
class B(A):
cls_var2 = 3
cls_var3 = 4
def __init__(self):
self.ins_var2 = "c"
self.ins_var3 = "d"
def testClassVariablesDynamicCreation(self):
A.cls_varNew1 = "y"
self.assertTrue(hasattr(A, 'cls_varNew1’))
self.assertTrue(hasattr(B, 'cls_varNew1’))
b = B()
self.assertTrue(hasattr(b, 'cls_varNew1’))
A.cls_varNew2 = "z"
self.assertTrue(hasattr(b, 'cls_varNew2'))

128/296
 Recommendations
– Be carful when defining classes
– Be careful when defining instances
– Use immutable classes, instances

129/296
 Recommendations
– Be carful when defining classes
– Be careful when defining instances
– Use immutable classes, instances

130/296
No Attribute Protection
class A():
def __init__(self):
super().__init__()
self.x = 1
self._y = 2
class B():
def __init__(self):
self.__z = 3
class C(A, B):
pass

131/296
class A():
def __init__(self):
super().__init__()
self.x = 1
self._y = 2
class B():
def __init__(self):
self.__z = 3
class C(A, B):
pass
def testA(self):
a = A()
self.assertEqual(a.__dict__.__len__(), 2)
self.assertTrue(isinstance(a.x, int))
self.assertTrue(isinstance(a._y, int))

132/296
class A():
def __init__(self):
super().__init__()
self.x = 1
self._y = 2
class B():
def __init__(self):
self.__z = 3
class C(A, B):
pass
def testA(self):
a = A()
Name mangling,
no enforcing

133/296
class A():
def __init__(self):
super().__init__()
self.x = 1
self._y = 2
class B():
def __init__(self):
self.__z = 3
class C(A, B):
pass
def testA(self):
a = A()
def testC(self):
c = C()
self.assertEqual(c.__dict__.__len__(), 3)
self.assertTrue(isinstance(c._B__z, int))
Name mangling,
no enforcing

134/296
 But some logic associated with underscores

135/296
 But some logic associated with underscores
Some enforcing

136/296
 Controversial topic
– https://stackoverflow.com/questions/1301346/what-is-
the-meaning-of-single-and-double-underscore-before-
an-object-name
– Accepted answer with 1,622 votes
• “This answer is extremely misleading […] as explained here by
Raymond Hettinger, who explicitly states that dunderscore is
incorrrectly [sic] used to mark members private, while it was
designed to be the opposite of private.” —Markus Meskanen

137/296
 Controversial topic
– https://stackoverflow.com/questions/1301346/what-is-
the-meaning-of-single-and-double-underscore-before-
an-object-name
– Accepted answer with 1,622 votes
• “This answer is extremely misleading […] as explained here by
Raymond Hettinger, who explicitly states that dunderscore is
incorrrectly [sic] used to mark members private, while it was
designed to be the opposite of private.” —Markus Meskanen
A Python core developer

138/296
 Recommendations
– Follow naming conventions
– Use a linter to enforce conventions

139/296
 Recommendations
– Follow naming conventions

140/296
Outline
1. Background
2. Quality Criteria
3. Special Objects
4. Collection API
5. Attributes
6. Methods
7. Resources
8. Polymorphism
9. Inheritance
10. Misc.
11. (Im)Mutability
12. Metaclasses
13. Conclusion

141/296
METHODS
Section Methods

142/296
Everything Is A Method
 Python includes
– Instance methods
– Class methods
– Static methods

143/296
class A:
def instanceMethod(self):
print(f"A.instanceMethod({self})")
@classmethod
def classMethod(cls):
print(f"A.classMethod({cls})")
@staticmethod
def staticMethod():
print("A.staticMethod()")

144/296
class A:
@classmethod
@staticmethod
def staticMethod():
print("On A")
A.instanceMetod(A())
A.classMethod()
A.staticMethod()

145/296
class A:
@classmethod
@staticmethod
def staticMethod():
On A
A.instanceMethod(<__main__.A object at 0x...>)
A.classMethod(<class '__main__.A'>)
A.staticMethod()
print("On A")
A.classMethod()
A.staticMethod()

146/296
class A:
@classmethod
@staticmethod
def staticMethod():
On A
A.staticMethod()
print("On A")
A.classMethod()
A.staticMethod()
print("On a = A()")
a = A()
a.instanceMethod()
a.classMethod()
a.staticMethod()

147/296
class A:
@classmethod
@staticmethod
def staticMethod():
On A
A.staticMethod()
print("On A")
A.classMethod()
A.staticMethod()
print("On a = A()")
a = A()
a.instanceMethod()
a.classMethod()
a.staticMethod()
On a = A()
A.staticMethod()

148/296
class A:
@classmethod
@staticmethod
def staticMethod():
On A
A.staticMethod()
print("On A")
A.classMethod()
A.staticMethod()
print("On a = A()")
a = A()
a.instanceMethod()
a.classMethod()
a.staticMethod()
On a = A()
A.staticMethod()

149/296
class A:
@classmethod
@staticmethod
def staticMethod():
class B(A):
print(f"B.instanceMethod({self})")
@classmethod
print(f"B.classMethod({cls})")
@staticmethod
def staticMethod():
print("B.staticMethod()")
print("On B")
B.instanceMethod(B())
B.instanceMethod(A())
B.classMethod()
B.staticMethod()
print("On b = B()")
b = B()
b.instanceMethod()
b.classMethod()
b.staticMethod()

150/296
class A:
@classmethod
@staticmethod
def staticMethod():
class B(A):
@classmethod
@staticmethod
def staticMethod():
print("On B")
B.classMethod()
B.staticMethod()
print("On b = B()")
b = B()
b.instanceMethod()
b.classMethod()
b.staticMethod()
On B
B.instanceMethod(<__main__.B object at 0x...>)
B.instanceMethod(<__main__.A object at 0x...>)
B.classMethod(<class '__main__.B'>)
B.staticMethod()
On b = B()
B.staticMethod()

151/296
class A:
@classmethod
@staticmethod
def staticMethod():
class B(A):
@classmethod
@staticmethod
def staticMethod():
print("On B")
B.classMethod()
B.staticMethod()
print("On b = B()")
b = B()
b.instanceMethod()
b.classMethod()
b.staticMethod()
On B
B.instanceMethod(<__main__.A object at 0x...>)
B.staticMethod()
On b = B()
B.staticMethod()

152/296
 All methods are overloadable
Class methods are methods
Therefore, class methods are overloadable
 Same goes for static methods!
https://en.wikipedia.org/wiki/Syllogism

153/296
 All methods are overloadable
Class methods are methods
Therefore, class methods are overloadable
 Same goes for static methods!
https://en.wikipedia.org/wiki/Syllogism

154/296
 The decorations @classmethod and
@staticmethod are decorators
 The decorations @classmethod and
@staticmethod are about bindings
– Not about the receiver / call site
– Not the object model (i.e., metaclasses)

155/296
class C(A):
print(f"C.instanceMethod({self})")
super().instanceMethod()
def classMethod(self):
print(f"C.classMethod({self})")
super().classMethod()
def staticMethod(self):
print("C.staticMethod()")
super().staticMethod()
print("On C")
C.instanceMethod(C())
C.instanceMethod(A())
C.classMethod(C())
C.staticMethod(C())
print("On c = C()")
c = C()
c.instanceMethod()
c.classMethod()
c.staticMethod()

156/296
class C(A):
print("On C")
C.classMethod(C())
C.staticMethod(C())
print("On c = C()")
c = C()
c.instanceMethod()
c.classMethod()
c.staticMethod()
No more decorations
All instance methods

157/296
class C(A):
try:
except:
print("TypeError: super(type, obj): obj must be an instance or subtype of type")
print("On C")
C.classMethod(C())
C.staticMethod(C())
print("On c = C()")
c = C()
c.instanceMethod()
c.classMethod()
c.staticMethod()

158/296
class C(A):
try:
except:
print("On C")
C.classMethod(C())
C.staticMethod(C())
print("On c = C()")
c = C()
c.instanceMethod()
c.classMethod()
c.staticMethod()
On C
C.instanceMethod(<__main__.C object at 0x...>)
A.instanceMethod(<__main__.C object at 0x...>)
C.instanceMethod(<__main__.A object at 0x...>)
<What error can it be?>
C.classMethod(<__main__.C object at 0x...>)
A.classMethod(<class '__main__.C'>)
C.staticMethod()
A.staticMethod()
On c = C()
C.staticMethod()
A.staticMethod()

159/296
class C(A):
try:
except:
print("On C")
C.classMethod(C())
C.staticMethod(C())
print("On c = C()")
c = C()
c.instanceMethod()
c.classMethod()
c.staticMethod()
On C
C.instanceMethod(<__main__.A object at 0x...>)
TypeError: super(type, obj): obj must be an…
C.staticMethod()
A.staticMethod()
On c = C()
C.staticMethod()
A.staticMethod()

160/296
 Python 3 super() is equivalent to Python 2
super(__class__, <firstarg>)
– “where __class__ is the class [in which] the
method was defined, and <firstarg> is the first
parameter of the method (normally self for
instance methods, and cls for class methods).”
 Contravariance on <firstarg>
– Obviously!
https://peps.python.org/pep-3135/

161/296
class C(A):
...
class D(C):
pass
print("On C")
C.instanceMethod(D())
C.classMethod(C())
C.staticMethod(C())
print("On d = D()")
d = D()
d.instanceMethod()
d.classMethod()
d.staticMethod()

162/296
class C(A):
...
class D(C):
pass
print("On C")
C.instanceMethod(D())
C.classMethod(C())
C.staticMethod(C())
print("On d = D()")
d = D()
d.instanceMethod()
d.classMethod()
d.staticMethod()
On C
C.instanceMethod(<__main__.D object at 0x...>)
A.instanceMethod(<__main__.D object at 0x...>)
C.staticMethod()
A.staticMethod()
On d = D()
C.instanceMethod(<__main__.D object at 0x...>)
A.instanceMethod(<__main__.D object at 0x...>)
C.classMethod(<__main__.D object at 0x...>)
A.classMethod(<class '__main__.D'>)
C.staticMethod()
A.staticMethod()

163/296
Nested Functions and Closures
 Nested function declarations
 Access to outer variables
 First-class functions
 Closures

164/296
 Closures

165/296
class A(object):
def __init__(self):
self.ins_var = 1
def foo1(self):
print("foo1")
a = 1
def bar():
print("foo1.bar")
a = 2
return a
bar()
return a
def foo2(self):
print("foo2")
a = 1
def bar():
nonlocal a
print("foo2.bar")
a = 2
return a
bar()
return a

166/296
class A(object):
def __init__(self):
self.ins_var = 1
def foo1(self):
print("foo1")
a = 1
def bar():
print("foo1.bar")
a = 2
return a
bar()
return a
def foo2(self):
print("foo2")
a = 1
def bar():
nonlocal a
print("foo2.bar")
a = 2
return a
bar()
return a
No outer access

167/296
class A(object):
def __init__(self):
self.ins_var = 1
def foo1(self):
print("foo1")
a = 1
def bar():
print("foo1.bar")
a = 2
return a
bar()
return a
def foo2(self):
print("foo2")
a = 1
def bar():
nonlocal a
print("foo2.bar")
a = 2
return a
bar()
return a
No outer access
Outer access
but no closure

168/296
 Closures

169/296
– Assigned to any variables
– Returned by other functions
 Objects created at runtime
https://www.geeksforgeeks.org/defining-a-python-function-at-runtime/
def testFunctionAndMethodTypes(self):
self.assertEqual(str(type(foo)), "<class 'function'>")
self.assertEqual(str(type(A.foo1)), "<class 'function'>")
a = A()
self.assertEqual(str(type(a.foo1)), "<class 'method'>")
from types import FunctionType
f_code = compile('def gfg(): return "GEEKSFORGEEKS"', "<string>", "exec")
f_func = FunctionType(f_code.co_consts[0], globals(), "gfg")
print(f_func())

170/296
 Closures
– Closure
• Lexical closure or function closure
• Lexically-scoped name binding
– Some examples
• Lambdas in Scheme
• Blocks in Smalltalk
• Functions in JavaScript and Python
https://en.wikipedia.org/wiki/Closure_(computer_programming)

171/296
class A(object):
def __init__(self):
self.ins_var = 1
def foo1(self):
print("foo1")
a = 1
def bar():
print("foo1.bar")
a = 2
return a
bar()
return a
def foo2(self):
print("foo2")
a = 1
def bar():
nonlocal a
print("foo2.bar")
a = 2
return a
bar()
return a
def foo4(self, aParam):
print("foo3")
a = 42
def bar(anotherParam):
nonlocal a
print("foo2.bar")
return a * aParam + anotherParam + self.ins_var
return bar

172/296
class A(object):
def __init__(self):
self.ins_var = 1
def foo1(self):
print("foo1")
a = 1
def bar():
print("foo1.bar")
a = 2
return a
bar()
return a
def foo2(self):
print("foo2")
a = 1
def bar():
nonlocal a
print("foo2.bar")
a = 2
return a
bar()
return a
print("foo3")
a = 42
nonlocal a
print("foo2.bar")
return bar
No outer access

173/296
class A(object):
def __init__(self):
self.ins_var = 1
def foo1(self):
print("foo1")
a = 1
def bar():
print("foo1.bar")
a = 2
return a
bar()
return a
def foo2(self):
print("foo2")
a = 1
def bar():
nonlocal a
print("foo2.bar")
a = 2
return a
bar()
return a
print("foo3")
a = 42
nonlocal a
print("foo2.bar")
return bar
No outer access
Outer access
but no closure

174/296
class A(object):
def __init__(self):
self.ins_var = 1
def foo1(self):
print("foo1")
a = 1
def bar():
print("foo1.bar")
a = 2
return a
bar()
return a
def foo2(self):
print("foo2")
a = 1
def bar():
nonlocal a
print("foo2.bar")
a = 2
return a
bar()
return a
print("foo3")
a = 42
nonlocal a
print("foo2.bar")
return bar
No outer access
Outer access
but no closure
Outer access, with
closures on outer and
instance variables

175/296
 Example of closure
def multiplier(factor):
f = factor * 2
def multiply(x):
nonlocal f
return x * f
return multiply
class Test(unittest.TestCase):
def testDouble(self):
double = multiplier(2)
self.assertEqual(double(10), 40)
def testTriple(self):
double = multiplier(3)

176/296
 Recommendation
– Use closures when appropriate

177/296
 Recommendation
– Use closures when appropriate

178/296
Decorators
 Definition

179/296
Decorators
 Definition
def intercept_return(func):
def _intercept_return(*args):
ret = func(*args)
return "Intercepted: " + ret + ", returning Bye!"
return _intercept_return
@intercept_return
def greet():
return "Hello, world!"
class A:
@intercept_return
def greet(self):

180/296
Decorators
 Definition
def intercept_return(func):
def _intercept_return(*args):
ret = func(*args)
return "Intercepted: " + ret + ", returning Bye!"
return _intercept_return
@intercept_return
def greet():
class A:
@intercept_return
def greet(self):
def testAddMessageToFunction(self):
self.assertEqual(greet(),
"Intercepted: Hello, world!, returning Bye!")
def testAddMessageToMethod(self):
a = A()
self.assertEqual(a.greet(),
"Intercepted: Hello, world!, returning Bye!")

181/296
Decorators
 Example
https://stackoverflow.com/questions/71357736/how-does-python-decorator-work-under-the-hood

182/296
Decorators
 Example
def logger(func):
def _logger(*args, **kwargs):
result = func(*args, **kwargs)
with open('log.txt', 'w') as f:
f.write(str(result))
return result
return _logger
@logger
def summator(numlist):
return sum(numlist)

183/296
Decorators
 Example
def logger(func):
return result
return _logger
@logger
return sum(numlist)
def testLogger(self):
self.assertEqual(summator([1, 2, 3, 4, 5]), 15)
log_file = Path("log.txt")
self.assertTrue(log_file.is_file())

184/296
def logger(func):
return result
return _logger
@logger
return sum(numlist)
Decorators
 Implementation
def logger(func):
return result
return _logger
@logger
return sum(numlist)
def summator2(numlist):
return sum(numlist)
summator2 = logger(summator2)

185/296
def logger(func):
return result
return _logger
@logger
return sum(numlist)
Decorators
 Implementation
def logger(func):
return result
return _logger
@logger
return sum(numlist)
return sum(numlist)
def testLogger2(self):
self.assertEqual(summator2([1, 2, 3, 4, 5]), 15)

186/296
def logger(func):
return result
return _logger
@logger
return sum(numlist)
Decorators
 Implementation
def logger(func):
return result
return _logger
@logger
return sum(numlist)
return sum(numlist)
def testLogger2(self):
self.assertEqual(summator2([1, 2, 3, 4, 5]), 15)
Wrapper function
@ is syntactic sugar

187/296
Decorators
 Recommendation
– Use Decorator rather than modifying functions

188/296
Decorators
 Recommendation
– Use Decorator rather than modifying functions

189/296
One-expression Lambdas
 Lambda functions (anonymous functions)
can only contain one expression
def testLambda1(self):
l = lambda x: x + 1
self.assertEqual(l(1), 2)
l = lambda a, b: a * b
self.assertEqual(l(2, 3), 6)

190/296
One-expression Lambdas
 Lambda functions (anonymous functions)
can only contain one expression
l = lambda x: x + 1
self.assertEqual(l(1), 2)
l = lambda a, b: a * b
self.assertEqual(l(2, 3), 6)

191/296
Outline
1. Background
2. Quality Criteria
3. Special Objects
4. Collection API
5. Attributes
6. Methods
7. Resources
8. Polymorphism
9. Inheritance
10. Misc.
11. (Im)Mutability
12. Metaclasses
13. Conclusion

192/296
RESOURCES
Section Resources

193/296
Safe Access
 Python offers the with… as… statement to
manage resources safely

194/296
Safe Access
def testBad(self):
file = open('log1.txt', 'w’)
file.write('hello world !’)
file.close()

195/296
Safe Access
def testBad(self):
file.close()
def testGood(self):
try:
file.write('hello world’)
finally:
file.close()

196/296
Safe Access
def testBad(self):
file.close()
def testGood(self):
try:
file.write('hello world’)
finally:
file.close()
def testBetter(self):
with open('log3.txt', 'w') as file:
file.write('hello world !')

197/296
Iterable Files
def testIterableFiles(self):
result = open("Roy's Last Words")
self.assertEqual(type(result), _io.TextIOWrapper)
result2 = []
for l in result:
result2 += [l]
self.assertEqual(len(result2), 7)

198/296
Iterable Files
result2 = []
for l in result:
result2 += [l]
Implements the
Iterator protocol

199/296
Iterable Files
result2 = []
for l in result:
result2 += [l]
Implements the
Iterator protocol

200/296
Resources
 Recommendation
– Make your own objects safe and iterable

201/296
Resources
 Recommendation
– Make your own objects safe and iterable

202/296
Outline
1. Background
2. Quality Criteria
3. Special Objects
4. Collection API
5. Attributes
6. Methods
7. Resources
8. Polymorphism
9. Inheritance
10. Misc.
11. (Im)Mutability
12. Metaclasses
13. Conclusion

203/296
POLYMORPHISM
Section Polymorphism

204/296
Definition
 “the provision of a single interface
to [objects] of different types.”
https://en.wikipedia.org/wiki/Polymorphism_(computer_science)
def testPolymorphism(self):
str1 = "Hello, "
str2 = "World!"
self.assertEqual(str1 + str2, "Hello, World!")
num1 = 40
num2 = 2
self.assertEqual(num1 + num2, 42)

205/296
Definition
str1 = "Hello, "
str2 = "World!"
num1 = 40
num2 = 2
Different types,
same method

206/296
Definition
str1 = "Hello, "
str2 = "World!"
num1 = 40
num2 = 2
Different types,
same method
(The method is
__add__())

207/296
Multiple Inheritance
 “Python supports a form of multiple
inheritance as well.”
https://docs.python.org/3/tutorial/classes.html
https://medium.com/@touahartoufik/extending-classes-with-mixins-with-python-2aad5c6997cc
from xmlrpc.server import SimpleXMLRPCServer
from socketserver import ThreadingMixIn
class ThreadedXMLRPCServer(ThreadingMixIn, SimpleXMLRPCServer):
pass

208/296
Duck Typing
 Earlier we discussed generators
 The class Generator subclasses
Iterator, which subclasses Iterable,
which declares __iter__() and
__next__()
As of 25/05/28: https://github.com/python/cpython/blob/main/Lib/_collections_abc.py#L348

209/296
Duck Typing
 Earlier with discussed files
 The class TextIOWrapper, which
subclasses TextIOBase, which
subclasses IOBase
As of 25/05/28: https://github.com/python/cpython/blob/main/Lib/_pyio.py#L1966

210/296
Duck Typing
 Iterable is not in the hierarchy of
TextIOWrapper
 But IOBase declares __iter__() and
__next__() and, thus, files can be iterated
As of 24/05/28: https://github.com/python/cpython/blob/main/Lib/_pyio.py#L554

212/296
Duck Typing
 If it looks like a duck,
swims like a duck, and
quacks like a duck,
then it is a duck


213/296
Duck Typing
 If it looks like a duck,
swims like a duck, and
quacks like a duck,
then it is a duck
 IOBase “looks” like
Iterable and, by
abductive reasoning,
is Iterable

214/296
Duck Typing

 IOBase “looks” like
Iterable and, by
abductive reasoning,
is Iterable

215/296
Polymorphism
 Combining
– Multiple inheritance
– Duck typing
Without “rules”
– Generator
– TextIOWrapper

216/296
Polymorphism
 Recommendation
– Use multiple inheritance rather than duck typing

217/296
Polymorphism
 Recommendation
– Use multiple inheritance rather than duck typing

218/296
Outline
1. Background
2. Quality Criteria
3. Special Objects
4. Collection API
5. Attributes
6. Methods
7. Resources
8. Polymorphism
9. Inheritance
10. Misc.
11. (Im)Mutability
12. Metaclasses
13. Conclusion

219/296
INHERITANCE
Section Inheritance

220/296
Inheritance Is Just A Suggestion
 Java
– “[The] super keyword is used to access methods of the parent class
while this is used to access methods of the current class.”
 Smalltalk
– “self is used when an object wishes to refer to itself, and super is
used to refer to the superclass of the object.”
 C++
– “this is a keyword that refers to the current instance of the class.”
– There is no super keyword in (standard) C++
 Python
– “self is a reference to the object instance […]. super allows you to
access attributes (methods, members, etc.) of an ancestor type.”
https://www.geeksforgeeks.org/super-and-this-keywords-in-java/
https://courses.cs.washington.edu/courses/cse505/99au/oo/smalltalk-concepts.html
https://www.javatpoint.com/cpp-this-pointer
https://stackoverflow.com/questions/72705781/difference-between-self-and-super

221/296
 Single inheritance
– Java
– Smalltalk
 super refers to the (direct) superclass of a
class so an object can access the methods
and fields of the superclass of its class

222/296
 Multiple inheritance
– C++
– Python
 Two different approaches
– C++ Ἢ
– Python ⏏

223/296
Inheritance Is Just
A Suggestion
 C++ Ἢ
– virtual keyword in
base clause
– Base initialisation in the
member initializer list
class Object {
public:
Object(int c) {
printf("Objectn");
a = 0;
}
int a;
};
class Object1 : public virtual Object {
public:
Object1(int c) : Object(c) {
printf("Object1n");
a1 = 0;
a = 1;
}
int a1;
};
class Object2 : public virtual Object {
public:
Object2(int c) : Object(c) {
printf("Object2n");
a2= 0;
a = 2;
}
int a2;
};
class Object4 : public Object1, public Object2 { public:
Object4(int c) : Object(c), Object1(c), Object2(c) {
printf("Object4n");
a4 = 0;
a = 4;
}
int a4;
};
https://cplusplus.com/forum/general/1414/
https://cplusplus.com/forum/general/1420/

224/296
Inheritance Is Just
A Suggestion
 Python ⏏
– Method Resolution Order
• C3 algorithm
https://dl.acm.org/doi/10.1145/236337.236343
https://www.python.org/download/releases/2.3/mro/
class A:
def __init__(self):
print("A")
super().__init__()
def output(self):
print("A.output()")
class B(A):
def __init__(self):
print("B")
super().__init__()
def output(self):
print("B.output()")
super().output()
class C(B):
def __init__(self):
print("C")
super().__init__()
def output(self):
print("C.output()")
super().output()
class D(A):
def __init__(self):
print("D")
super().__init__()
def output(self):
print("D.output()")
super().output()
class E(C, D):
def __init__(self):
print("E")
super().__init__()
def output(self):
print("E.output()")
super().output()

225/296
Inheritance Is Just
A Suggestion
 Python ⏏
• C3 algorithm
class A:
def __init__(self):
print("A")
super().__init__()
def output(self):
print("A.output()")
class B(A):
def __init__(self):
print("B")
super().__init__()
def output(self):
print("B.output()")
super().output()
class C(B):
def __init__(self):
print("C")
super().__init__()
def output(self):
print("C.output()")
super().output()
class D(A):
def __init__(self):
print("D")
super().__init__()
def output(self):
print("D.output()")
super().output()
class E(C, D):
def __init__(self):
print("E")
super().__init__()
def output(self):
print("E.output()")
super().output()
e = E()
e.output()

226/296
Inheritance Is Just
A Suggestion
 Python ⏏
• C3 algorithm
class A:
def __init__(self):
print("A")
super().__init__()
def output(self):
print("A.output()")
class B(A):
def __init__(self):
print("B")
super().__init__()
def output(self):
print("B.output()")
super().output()
class C(B):
def __init__(self):
print("C")
super().__init__()
def output(self):
print("C.output()")
super().output()
class D(A):
def __init__(self):
print("D")
super().__init__()
def output(self):
print("D.output()")
super().output()
class E(C, D):
def __init__(self):
print("E")
super().__init__()
def output(self):
print("E.output()")
super().output()
e = E()
e.output()
E
C
B
D
A
E.output()
C.output()
B.output()
D.output()
A.output()

227/296
 Python ⏏
• C3 algorithm
print(E.mro())
print(D.mro())
print(C.mro())
print(B.mro())
print(A.mro())
[<class '__main__.E'>, <class ‘….C'>, <class ‘….B'>, <class ‘….D'>, <class ‘….A'>, <class 'object'>]
[<class '__main__.D'>, <class '__main__.A'>, <class 'object'>]
[<class '__main__.C'>, <class '__main__.B'>, <class '__main__.A'>, <class 'object'>]
[<class '__main__.B'>, <class '__main__.A'>, <class 'object'>]
[<class '__main__.A'>, <class 'object'>]

228/296
Inheritance Is Just
A Suggestion
 Python ⏏
• C3 algorithm
class A:
def __init__(self):
print("A")
super().__init__()
def output(self):
print("A.output()")
class B(A):
def __init__(self):
print("B")
super().__init__()
def output(self):
print("B.output()")
super().output()
class C(B):
def __init__(self):
print("C")
super().__init__()
def output(self):
print("C.output()")
super().output()
class D(A):
def __init__(self):
print("D")
super().__init__()
def output(self):
print("D.output()")
super().output()
class E(C, D):
def __init__(self):
print("E")
super().__init__()
def output(self):
print("E.output()")
super().output()

229/296
Inheritance Is Just
A Suggestion
 Python ⏏
• C3 algorithm
e = E()
e.output()
class A:
def __init__(self):
print("A")
super().__init__()
def output(self):
print("A.output()")
class B(A):
def __init__(self):
print("B")
super().__init__()
def output(self):
print("B.output()")
super().output()
class C(B):
def __init__(self):
print("C")
super().__init__()
def output(self):
print("C.output()")
super().output()
class D(A):
def __init__(self):
print("D")
super().__init__()
def output(self):
print("D.output()")
super().output()
class E(C, D):
def __init__(self):
print("E")
super().__init__()
def output(self):
print("E.output()")
super().output()

230/296
Inheritance Is Just
A Suggestion
 Python ⏏
• C3 algorithm
e = E()
e.output()
class A:
def __init__(self):
print("A")
super().__init__()
def output(self):
print("A.output()")
class B(A):
def __init__(self):
print("B")
super().__init__()
def output(self):
print("B.output()")
super().output()
class C(B):
def __init__(self):
print("C")
super().__init__()
def output(self):
print("C.output()")
super().output()
class D(A):
def __init__(self):
print("D")
super().__init__()
def output(self):
print("D.output()")
super().output()
class E(C, D):
def __init__(self):
print("E")
super().__init__()
def output(self):
print("E.output()")
super().output()
E
C
B
D
A
E.output()
C.output()
B.output()
D.output()
A.output()

231/296
 Python ⏏
• C3 algorithm
https://news.ycombinator.com/item?id=24255334
class A(object):
def __init__(self):
self.x = 1
class B(object):
def __init__(self):
self.y = 2
class C(A, B):
pass
print(C().y)

232/296
 Python ⏏
• C3 algorithm
class A(object):
def __init__(self):
self.x = 1
class B(object):
def __init__(self):
self.y = 2
class C(A, B):
pass
print(C().y)
Traceback (most recent call last):
File “…Inheritance2.py", line 14, in <module>
print(C().y)
^^^^^
AttributeError: 'C' object has no attribute 'y'

233/296
 Python ⏏
• C3 algorithm
class A(object):
def __init__(self):
super().__init__()
self.x = 1
class B(object):
def __init__(self):
self.y = 2
class C(A, B):
pass
print(C().y)

234/296
 Python ⏏
• C3 algorithm
class A(object):
def __init__(self):
super().__init__()
self.x = 1
class B(object):
def __init__(self):
self.y = 2
class C(A, B):
pass
print(C().y)
2

235/296
 Python ⏏
• C3 algorithm

236/296
 Reminder
– Principle of least astonishment / surprise
• “Transparency is a passive quality. A program is
transparent when it is possible to form a simple
mental model of its behavior that is actually predictive
for all or most cases, because you can see through
the machinery to what is actually going on.”
—Eric Raymond
(Emphasis mine)
https://wiki.c2.com/?PrincipleOfLeastAstonishment

237/296
 Reminder
• “[A]n error is local in time if it is discovered very soon
after it is created; an error is local in space if it is
identified very close (at) the site where the error
actually resides.”
(Emphasis mine)
https://beza1e1.tuxen.de/articles/principle_of_locality.html
https://wiki.c2.com/?CeeVsAdaStudy

238/296
https://stackoverflow.com/questions/3277367/how-does-pythons-super-work-with-multiple-inheritance

239/296
Inheritance in Python involves explicit
delegations and an obscure algorithm

240/296
Inheritance in Python involves explicit
delegations and an obscure algorithm
Exercise utmost caution

241/296
 Recommendations
– Always add super().__init__()

242/296
 Recommendations
– Always add super().__init__()

243/296
Outline
1. Background
2. Quality Criteria
3. Special Objects
4. Collection API
5. Attributes
6. Methods
7. Resources
8. Polymorphism
9. Inheritance
10. Misc.
11. (Im)Mutability
12. Metaclasses
13. Conclusion

245/296
Content
 Chaining comparisons
 For loop else clause
 Imaginary numbers
 String interpolations
 Parsing inputs

246/296
Content
 Parsing inputs

247/296
Content
 Parsing inputs

248/296
Chaining Comparisons
def testChainingComparisons(self):
i = 5
j = 10
k = 15
self.assertTrue(i < j < k)
self.assertFalse(i < k < j)

249/296
For Loop Else Clause
def testForLoopElse(self):
for x in range(6):
if x == 7: break
pass
else:
self.assertTrue(True)
for x in range(6):
if x == 3: break
pass
else:
self.assertTrue(False)

250/296
Imaginary Numbers
https://www.sciencedirect.com/topics/engineering/imaginary-number
def testImaginaryNumbers(self):
v1 = 5 + 2j
v2 = 10 + 3j
v = v1 + v2
self.assertEqual(v, 15 + 5j)
self.assertEqual(v.real, 15)
self.assertEqual(v.imag, 5)

251/296
Imaginary Numbers
https://www.sciencedirect.com/topics/engineering/imaginary-number
def testImaginaryNumbers(self):
v1 = 5 + 2j
v2 = 10 + 3j
v = v1 + v2
self.assertEqual(v, 15 + 5j)
self.assertEqual(v.real, 15)
self.assertEqual(v.imag, 5)
Imaginary marker

252/296
Content
 Parsing inputs

253/296
Content
 Parsing inputs Defensive programming

254/296
String Interpolation
def testJoinMethod(self):
question = 'Life, the Universe, and Everything’
answer = 42
text = " ".join([str(answer), "is the Answer to the Ultimate Question of", question])
self.assertEqual(text,
"42 is the Answer to the Ultimate Question of Life, the Universe, and Everything")
def testPercentageOperator(self):
answer = 42
text = "%s is the Answer to the Ultimate Question of %s" % (answer, question)
def testFormatMethod(self):
answer = 42
text = "{in2} is the Answer to the Ultimate Question of {in1}".format(in1=question, in2=answer)
def testFStrings(self):
answer = 42
self.assertEqual(f'{answer} is the Answer to the Ultimate Question of {question}’,

255/296
Parsing Inputs
 No “native” parser
– No scanf (C)
– No cin (C++)
– No Scanner (Java)
 But, the dedicated
module parse
– https://pypi.org/
project/parse
from parse import parse
# ...
def testParser1(self):
result = parse('{} fish', '1’)
self.assertEqual(result, None)
result = parse('{}', '1’)
self.assertEqual(result[0], '1')
result = parse('{}, World!', 'Hello, World!’)
self.assertEqual(result[0], 'Hello')
result = parse('{}, {}!', 'Hello, World!’)
self.assertEqual(result[0], 'Hello’)
self.assertEqual(result[1], 'World')

256/296
Outline
1. Background
2. Quality Criteria
3. Special Objects
4. Collection API
5. Attributes
6. Methods
7. Resources
8. Polymorphism
9. Inheritance
10. Misc.
11. (Im)Mutability
12. Metaclasses
13. Conclusion

257/296
(IM)MUTABILITY
Section (Im)Mutability

258/296
Definitions
 “[A]n immutable object (unchangeable
object) is an object whose state cannot be
modified after it is created.”
– “[A]n object that uses memoization to cache the
results of expensive computations could still be
considered an immutable object.”
 Variables, attributes, references
 Weak, strong

259/296
Definitions
 Weak, strong immutability
– Weak if some attributes of an object are mutable
– Immutable if all attributes are immutable
– Strong if all attributes are immutable and it
cannot be extended

261/296
Variables
def testConstant(self):
MAX_SPEED: Final[int] = 300
self.assertEqual(MAX_SPEED, 300)
MAX_SPEED = 500
def testMutableParameters(self):
a = A()
l = []
a.foo3('Hello', l)
self.assertEqual(len(l), 1)
a.foo3('World', l)
def testImmutableParameters(self):
a = A()
l = []
a.foo3('Hello', frozenset(l))
a.foo3('World', frozenset(l))

262/296
Variables
MAX_SPEED = 500
a = A()
l = []
a.foo3('Hello', l)
a.foo3('World', l)
a = A()
l = []
Final is only an
annotation for a type
checker, e.g., mypy

263/296
Variables
MAX_SPEED = 500
a = A()
l = []
a.foo3('Hello', l)
a.foo3('World', l)
a = A()
l = []
Final is only an
annotation for a type
checker, e.g., mypy
The caller must
enforce the invariant,
not the receiver

264/296
Instances
 Immutable objects
class ImmutableClassMetaClass3(type):
def __new__(cls, name, bases, attrs):
attrs['__slots__'] = {}
cls = type.__new__(cls, name, bases, attrs)
return cls
class ImmutableWithMC3(metaclass=ImmutableClassMetaClass3):
pass
def testImmutableWithMC3(self):
a = ImmutableWithMC3()
try:
a.inst_var3 = 42
except AttributeError:

265/296
Instances
 Immutable objects
attrs['__slots__'] = {}
return cls
pass
try:
a.inst_var3 = 42
The metaclass adds
__slots__ to its
instances (classes)

266/296
Classes
 Immutable classes
https://stackoverflow.com/questions/9654133/metaclasses-and-slots
https://stackoverflow.com/questions/56579348/how-can-i-force-subclasses-to-have-slots

267/296
Classes
 Immutable classes
https://stackoverflow.com/questions/9654133/metaclasses-and-slots
https://stackoverflow.com/questions/56579348/how-can-i-force-subclasses-to-have-slots

268/296
Classes and Instances
 Using a well-defined metaclass
self.assertEqual(ImmutableWithMC5.cls_var, 42)
try:
ImmutableWithMC4.cls_var5 = 42
self.assertEqual(a.ins_var, 24)
try:
a.inst_var5 = 42

269/296
 Using a well-defined metaclass
self.assertEqual(ImmutableWithMC5.cls_var, 42)
try:
ImmutableWithMC4.cls_var5 = 42
self.assertEqual(a.ins_var, 24)
try:
a.inst_var5 = 42
class ImmutableWithMC5(...):
cls_var = 42
def __init__(self, *args, **kwargs):
super().__init__()
self.ins_var = 24

270/296
def __setattr__(self, name, value):
raise AttributeError(“Cannot add class variables to this class")
def __setattr__for_class(self, name, value):
if inspect.stack()[1].function != "__init__":
raise AttributeError(“Cannot add instance variables to this class")
else:
object.__setattr__(self, name, value)
super().__setattr__('__setattr__’, ImmutableClassMetaClass5.__setattr__for_class)
return cls

271/296
cls_var = 42
def __init__(self, *args, **kwargs):
super().__init__()
self.ins_var = 24
def __setattr__(self, name, value):
raise AttributeError(“Cannot add class variables to this class")
def __setattr__for_class(self, name, value):
if inspect.stack()[1].function != "__init__":
raise AttributeError(“Cannot add instance variables to this class")
else:
object.__setattr__(self, name, value)
super().__setattr__('__setattr__’, ImmutableClassMetaClass5.__setattr__for_class)
return cls

272/296
Outline
1. Background
2. Quality Criteria
3. Special Objects
4. Collection API
5. Attributes
6. Methods
7. Resources
8. Polymorphism
9. Inheritance
10. Misc.
11. (Im)Mutability
12. Metaclasses
13. Conclusion

273/296
METACLASSES
Section Metaclasses

274/296
Metaclasses Always Come Last
https://blog.invisivel.net/2012/04/10/pythons-object-model-explained/

275/296
Metaclasses
Always
Come
Last

276/296
 Caveats
– Cannot have both a class and an instance
__new__() method in the same class
– The class Object defines a static (à la Python)
__new__() method that hides any __new__()
method from a metaclass

277/296
 Very promising…
…But limited by dynamicity of Python
 Workaround with __call__()

278/296
 Class creation
– __new__() instantiates a
class
– __init__() initialises
variables
– __prepare__() defines
the class namespace
passed to the metaclass
__new__ and __init__
methods
 Instance creation
– __call__() invoked
after the __new__ and
__init__
– Only because classes
are callable objects
• Instance of a class with a
__call__ method
• Anything with a non-null
tp_call (C struct)
https://elfi-y.medium.com/python-metaclass-7cb56510845
https://stackoverflow.com/questions/111234/what-is-a-callable

279/296
 Using __call__() for reference counting
class ReferenceCountingMetaClass(type):
def __init__(self, name, bases, namespace):
self._instances = 0
def __call__(self):
newInstance = super().__call__()
self._instances = self._instances + 1
return newInstance
def getNumberOfInstances(self):
return self._instances

280/296
class ReferenceCountingMetaClass(type):
def __init__(self, name, bases, namespace):
self._instances = 0
def __call__(self):
newInstance = super().__call__()
self._instances = self._instances + 1
return newInstance
def getNumberOfInstances(self):
return self._instances
Override the __call__ metaclass instance method
Define the get…() metaclass instance method

281/296
class C(metaclass=ReferenceCountingMetaClass):
pass
class D():
pass
x = C()
print(C.getNumberOfInstances())
y = C()
z = C()
x = D()
y = D()

282/296
class C(metaclass=ReferenceCountingMetaClass):
pass
class D():
pass
x = C()
y = C()
z = C()
x = D()
y = D()
1
2
3

283/296
 Recommendations
– Use metaclasses carefully
– Test your code extensively

284/296
 Recommendations
– Use metaclasses carefully
– Test your code extensively

285/296
Outline
1. Background
2. Quality Criteria
3. Special Objects
4. Collection API
5. Attributes
6. Methods
7. Resources
8. Polymorphism
9. Inheritance
10. Misc.
11. (Im)Mutability
12. Metaclasses
13. Conclusion

286/296
CONCLUSION
Section Conclusion

287/296
Conclusion
 Defensive programming
 Principle of least surprise
 Present and future productivity

288/296
Conclusion
 Python has many good qualities
– Also several bad ones
– Also few weird ones
 Proceed with caution!

289/296
Conclusion
Compiled
Interpreted
Erlang
Clojure
Python
Perl
VB
Groovy
Ruby
Magik
PHP
JavaScript F#
Java
C C++
Scala
Haskell
Python PyPy
Python PyPy
Python PyPy
Python PyPy Ruby YJIY
Ruby YJIY
Ruby YJIY
Ruby YJIY
JS V8
JS V8
JS V8
JS V8 C#

290/296
Conclusion
https://devopedia.org/duck-typing

291/296
Conclusion
https://devopedia.org/duck-typing

292/296
Static Checkers
 Many checkers exist
– MyPY (http://mypy-lang.org/)
– PyLint (https://pylint.org/)
– PyFlakes (https://pypi.org/project/pyflakes/)
– PyCodeStyle (https://pypi.org/project/pycodestyle/)
– Flake8 (http://flake8.pycqa.org/en/latest/)
– Prospector (https://prospector.landscape.io/en/master)
– Bandit (https://github.com/PyCQA/bandit)
 They do the work of a type system / a compiler

293/296
Coincidence ࢓
 JavaScript also needs
static checkers,
similar to Python

294/296
References
 Images credits in order of appearance (?)
– //www.nautiljon.com/asian_movies/the+good,+the+bad,+the+weird.html
– //www.academymuseum.org/en/programs/detail/the-good-the-bad-the-weird-018b021a-602c-
3e29-9388-6a07e825751a
– //asianmoviepulse.com/2021/04/film-review-the-good-the-bad-the-weird-2008-by-kim-jee-woon/
– //www.dvdbeaver.com/film2/DVDReviews46/good_bad_weird_blu-ray.htm
• //www.dvdbeaver.com/film2/DVDReviews46/good_bad_weird_blu-ray/800_good_bad_weird_blu-ray_12.jpg
• http://www.dvdbeaver.com/film2/DVDReviews46/good_bad_weird_blu-ray/800_good_bad_weird_blu-
ray_13.jpg
– //www.doblu.com/2010/08/25/the-good-the-bad-the-weird-review/
• //media.doblu.com/wp-content/uploads/2010/08/goodbadweird14902.jpg
– //www.flaticon.com/free-icons/tower (by Freepik – Flaticon)
– //www.flaticon.com/free-icons/box (by Becris – Flaticon)
– //www.flaticon.com/free-icons/local (by srip – Flaticon)
– //www.flaticon.com/free-icons/efficiency (by HAJICON – Flaticon)
– https://arvinf07.medium.com/duck-typing-7f93896dc893

295/296
References
 Sources of some of the advantages, in no order
– //machinelearningmastery.com/some-language-features-
in-python/
– //therenegadecoder.com/code/coolest-python-
programming-language-features/
– //www.quora.com/What-are-the-10-best-features-of-
Python
– //sahandsaba.com/thirty-python-language-features-and-
tricks-you-may-not-know.html

296/296
References
 Sources of some of the disadvantages, in no order
– //softwareengineering.stackexchange.com/questions/15468/what-
are-the-drawbacks-of-python
– //serokell.io/blog/python-pros-and-cons
– //www.linkedin.com/pulse/advantages-disadvantages-python-aj-p/
– //www.linode.com/docs/guides/pros-and-cons-of-python/
– //webandcrafts.com/blog/advantages-and-disadvantages-of-python
– //data-flair.training/blogs/advantages-and-disadvantages-of-python/
– //thecodest.co/blog/pros-and-cons-of-python/
– //unstop.com/blog/advantages-and-disadvantages-of-python
– //www.pixelcrayons.com/blog/software-development/python-pros-
and-cons/

Some Pitfalls with Python and Their Possible Solutions v1.0

More Related Content

Similar to Some Pitfalls with Python and Their Possible Solutions v1.0 (20)

More from Yann-Gaël Guéhéneuc (20)

Recently uploaded (20)

Some Pitfalls with Python and Their Possible Solutions v1.0