Advanced Developer Testing

Three Advanced 
DeveloperTestingTechniques
Alistair McKinnell
@amckinnell

How do we  
change code safely?

Changing Code
Existing Behaviour New Behaviour

Characterization
Testing
Approval
Testing
Code
Coverage
Mutation
Testing

1 class GildedRose
2 attr_reader :items
3
4 def initialize(items)
5 @items = items
6 end
7
8 def update_quality
9 @items.each do |item|
10 if item.name != 'Aged Brie' and
11 item.name != 'Backstage passes to a TAFKAL80ETC concert'
12 if item.quality > 0
13 if item.name != 'Sulfuras, Hand of Ragnaros'
14 item.quality = item.quality - 1
15 end
16 end
17 else
18 if item.quality < 50
19 item.quality = item.quality + 1
20 if item.name == 'Backstage passes to a TAFKAL80ETC concert'
21 if item.sell_in < 11
24 end
25 end
29 end
30 end
31 end
32 end
33 end
35 item.sell_in = item.sell_in - 1
36 end
38 if item.name != 'Aged Brie'
39 if item.name != 'Backstage passes to a TAFKAL80ETC concert'
40 if item.quality > 0
42 item.quality = item.quality - 1
43 end
44 end
45 else
46 item.quality = item.quality - item.quality
47 end
48 else
51 end
52 end
53 end
54 end
55 end
56
57 end
!

“Conjured” items degrade  
twice as fast as normal items.
New Behaviour

Existing Behaviour 
(Refactored)
New Behaviour 
(Conjured Item)
Changing Code

Are we ready to  
change code safely?

Nope
Refactoring safely 
requires tests.

describe GildedRose do
it 'knows how to update quality for items' do
items = [Item.new('Mail Armour', 10, 20)]
subject = GildedRose.new(items)
subject.update_quality
expect(subject.items[0].to_s).to eq(nil)
end
end

end
end
expected: nil
got: "Mail Armour, 9, 19"

expect(subject.items[0].to_s).to eq('Mail Armour, 9, 19')
end
end

items = [Item.new(‘Mail Armour', 10, 20)]
end
end

end
end
expected: nil
got: "Mail Armour, 8, 18"

end
end

items = [
Item.new('Mail Armour', 10, 20),
Item.new('Aged Brie', 4, 9)
]
end
end

items = [
Item.new(‘Aged Brie', 4, 9)
]
end
end

items = [
]
expect(subject.items[1].to_s).to eq(‘Aged Brie, 3, 10')
end
end

items = [
]
end
end
Let’s organize our test

characterization = characterize(subject, 2)
expect(characterization).to eq(expected)
end
end

end
end
def items
end
def characterize(subject, days)
end
def expected
end

end
end
def items
item_attributes = [
['Mail Armour', 10, 20],
['Aged Brie', 4, 9]
]
item_attributes.map { |args| Item.new(*args) }
end

end
end
characterization = []
(1..days).each do
subject.items.each { |item| characterization << item.to_s }
end
characterization
end

end
end
def expected
[
'Mail Armour, 9, 19',
'Aged Brie, 3, 10',
'Aged Brie, 2, 11'
]
end

Nope
Characterization 
is incomplete.

if item.name == 'Backstage passes to a TAFKAL80ETC concert'
if item.sell_in < 11
if item.quality < 50
item.quality = item.quality + 1
end
end
if item.sell_in < 6
end
end
end

end
end
if item.sell_in < 6
end
end
end
def items
item_attributes = [
['Aged Brie', 4, 9]
]
end

end
end
def items
item_attributes = [
['Aged Brie', 4, 9],
['Backstage passes to a TAFKAL80ETC concert', 15, 17]
]
end

def items
item_attributes = [
['Backstage passes to a TAFKAL80ETC concert', 15, 17]
]
end
end
end

def expected
[
'Aged Brie, 3, 10',
'Backstage passes to a TAFKAL80ETC concert, 14, 18',
'Aged Brie, 2, 11',
]
end
end
end

end
end
end
end
if item.sell_in < 6
end
end
end

end
end

end
end
This is what  
we want to do

end
end
def expected
[
'Aged Brie, 3, 10',
'Aged Brie, 2, 11',
'Backstage passes to a TAFKAL80ETC concert, 13, 19’,
and many more expected results
]
end

end
end
(1..days).each do
characterization.concat(subject.items.map(&:to_s))
end
Digest::SHA2.hexdigest(characterization.join)
end

end
end
def expected
[
'Aged Brie, 3, 10',
'Aged Brie, 2, 11',
'Backstage passes to a TAFKAL80ETC concert, 13, 19’,
]
end

end
end
def expected
nil
end

end
end
def expected
nil
end
expected: nil
got: “c869cb44cd36e1553d18 … d958277a43b1adc8e8e5”

end
end
def expected
‘c869cb44cd36e1553d18 … d958277a43b1adc8e8e5’
end

end
end
def expected
end

end
end
expected: “c869cb44cd36e1553d18 … d958277a43b1adc8e8e5”
got: “fb61e7fdfffcba653fec … 9ae0512470a974b1561e"
def expected
end

end
end
def expected
‘fb61e7fdfffcba653fec … 9ae0512470a974b1561e’
end

end
end
def items
item_attributes = [
['Aged Brie', 1, 49],
['Backstage passes to a TAFKAL80ETC concert', 15, 17],
['Sulfuras, Hand of Ragnaros', -1, 80]
]
end

class GildedRose
attr_reader :items
def initialize(items)
@items = items
end
def update_quality
@items.each do |item|
…
if item.name != 'Sulfuras, Hand of Ragnaros'
item.sell_in = item.sell_in - 1
end
…
end
end
end

class GildedRose
attr_reader :items
@items = items
end
def update_quality
…
# if item.name != 'Sulfuras, Hand of Ragnaros'
# end
…
end
end
end

class GildedRose
attr_reader :items
@items = items
end
def update_quality
…
# end
…
end
end
end
Comment out  
two lines

Test still passes?
class GildedRose
attr_reader :items
@items = items
end
def update_quality
…
# end
…
end
end
end

def items
item_attributes = [
]
end
end
end

Mutant configuration:
Subjects: 2 
Mutations: 476
Kills: 454
Alive: 22
Coverage: 95.38%
Expected: 100.00%

- if (item.quality > 0)
+ if (item.quality > 1)
- if (item.quality < 50)
+ if (item.quality < 49)
+ if (item.quality < 51)
+ if true
+ if 50
- if (item.name == "Backstage passes to a TAFKAL80ETC concert")
+ if item.name.eql?("Backstage passes to a TAFKAL80ETC concert")

def items
item_attributes = [
['Aged Brie', 1, 49],
]
end
end
end

Subjects: 2
Mutations: 476
Kills: 454
Alive: 22
Coverage: 95.38%
Expected: 100.00%
Before After
Subjects: 2
Mutations: 476
Kills: 474
Alive: 4
Coverage: 99.16%
Expected: 100.00%

New BehaviourExisting Behaviour
Changing Code

1 class GildedRose
2 attr_reader :items
3
4 def initialize(items)
5 @items = items
6 end
7
8 def update_quality
9 @items.each { |item| update_item_quality(item) }
10 end
11
12 private
13
14 def update_item_quality(item)
15 return if item.name == 'Sulfuras, Hand of Ragnaros'
16
17 perform_inventory_rollover(item)
18 perform_inventory_expiration(item)
19 end
20
21 def perform_inventory_rollover(item)
22 item.sell_in -= 1
23
24 case item.name
25 when 'Aged Brie'
26 increase_quality(item)
27 when 'Backstage passes to a TAFKAL80ETC concert'
29 increase_quality(item) if item.sell_in < 10
30 increase_quality(item) if item.sell_in < 5
31 else
32 decrease_quality(item)
33 end
34 end
35
36 def perform_inventory_expiration(item)
37 return unless expired?(item)
38
39 case item.name
40 when 'Aged Brie'
42 when 'Backstage passes to a TAFKAL80ETC concert'
43 writeoff(item)
44 else
45 decrease_quality(item)
46 end
47 end
48
49 def expired?(item)
50 item.sell_in < 0
51 end
52
53 def decrease_quality(item)
54 item.quality -= 1 if 0 < item.quality
55 end
56
57 def increase_quality(item)
58 item.quality += 1 if item.quality < 50
59 end
60
61 def writeoff(item)
62 item.quality = 0
63 end
64
65 end

def perform_inventory_rollover(item)
case item.name
when 'Conjured Mana'
decrease_quality(item)
end
end
def perform_inventory_expiration(item)
case item.name
end
end

def items
item_attributes = [
['Aged Brie', 1, 49],
['Sulfuras, Hand of Ragnaros', -1, 80],
[‘Conjured Mana', 13, 50]
]
end
end
end

end
end
def expected
‘1768fa473f323772588a … 13eab4018717ea78ea0c’
end

end
end
expected: “1768fa473f323772588a … 13eab4018717ea78ea0c”
got: “6cf1111d5865232381f1 … 9d00ff8e831395de5420"
def expected
‘1768fa473f323772588a … 13eab4018717ea78ea0c’
end

Nope
Can’t distinguish existing
and new behaviour.

Whatever the code does.
Existing Behaviour

“Conjured” items degrade 
twice as fast as normal items.
New Behaviour

verify { characterize(subject, 20) }
end
end
(1..days).each_with_index do |day|
characterization << "Day #{day} of #{days}"
subject.items.each { |item| characterization <<
" #{item.to_s}" }
end
characterization.join("n")
end

end
end

end
end
Approvals::ApprovalError: Approval Error: Approval file  
“spec/fixtures/approvals/gildedrose/  
knows_how_to_update_quality_for_items.approved.txt" 
not found.

knows_how_to_update_quality_for_items.received.txt
1 Day 1 of 20
2 Mail Armour, 9, 19
3 Mail Armour, 9, 0
4 Aged Brie, 3, 10
5 Aged Brie, 0, 50
6 Backstage passes to a TAFKAL80ETC concert, 14, 18
8 Sulfuras, Hand of Ragnaros, -1, 80
9 Day 2 of 20
…
153 Day 20 of 20
154 Mail Armour, -10, 0
156 Aged Brie, -16, 45
157 Aged Brie, -19, 50
158 Backstage passes to a TAFKAL80ETC concert, -5, 0

knows_how_to_update_quality_for_items.approved.txt
1 Day 1 of 20
3 Mail Armour, 9, 0
4 Aged Brie, 3, 10
5 Aged Brie, 0, 50
9 Day 2 of 20
…
153 Day 20 of 20
156 Aged Brie, -16, 45
157 Aged Brie, -19, 50

def perform_inventory_rollover(item
case item.name
end
end
def perform_inventory_expiration(item)
case item.name
end
end

end
end
Approvals::ApprovalError: Approval Error: Received file
“spec/fixtures/approvals/gildedrose/
knows_how_to_update_quality_for_items.received.txt”
does not match approved
“spec/fixtures/approvals/gildedrose/
knows_how_to_update_quality_for_items.approved.txt".

4 Aged Brie, 3, 10
5 Aged Brie, 0, 50
9 Conjured Mana, 12, 48
10 Day 2 of 20
12 …
4 Aged Brie, 3, 10
5 Aged Brie, 0, 50
9 Day 2 of 20
11 …

10 Day 2 of 20
13 Aged Brie, 2, 11
14 Aged Brie, -1, 50
19 Day 3 of 20
22 Aged Brie, 1, 12
28 Day 4 of 20
30 …

10 Day 2 of 20
13 Aged Brie, 2, 11
19 Day 3 of 20
22 Aged Brie, 1, 12
28 Day 4 of 20
30 …

We win.
We preserved 
existing behaviour and
added new behaviour.

Changing Code
New Behaviour 
(ApprovalTests)
Existing Behaviour 
(CharacterizationTests)

Be amazing at 
changing code safely.

github.com/amckinnell/developer-testing

Advanced Developer Testing

More Related Content

Similar to Advanced Developer Testing (20)

More from Alistair McKinnell (10)

Recently uploaded (20)

Advanced Developer Testing