The Functional Programmer's Toolkit (NDC London 2019)

The
Functional Programming
Toolkit
(NDC London 2019)
@ScottWlaschin
fsharpforfunandprofit.com

Why do functional programmers
use so many strange words?

Functional programming is scary

Object oriented programming is scary

These aren't just academic concepts.
They are actually useful tools!

The FunctionalToolbox
• Composition
• Combination/Aggregation
• Iteration
• Working with effects
– Mixing effects and non-effects
– Chaining effects in series
– Working with effects in parallel
– Pulling effects out of a list

• Composition: compose
• Iteration: fold
• Combination/Aggregation: combine & reduce
– Mixing effects and non-effects: map & return
– Chaining effects in series: bind/flatMap
– Working with effects in parallel: apply & zip
– Pulling effects out of a list: sequence & traverse

Functional Toolbox (FP jargon version)
• Combination/Aggregation: Monoid
– Mixing effects and non-effects: Functor
– Chaining effects in series: Monad
– Working with effects in parallel: Applicative

This talk
A whirlwind tour of many sights
Don't worry if you don't understand everything

Core principles of
statically-typed FP

Core principles of (statically-typed) FP
Function
Functions are things
Composition everywhere

Core FP principle:
Functions are things
Function

The Tunnel of
Transformation
Function
apple -> banana
A function is a thing which
transforms inputs to outputs

A function is a standalone thing,
not attached to a class
It can be used for inputs and outputs
of other functions

input
A function can be an output
A function can be an output

output
A function can be an input
A function can be an input

input output
A function can be a parameter
A function can be a parameter
You can build very complex systems
from this simple foundation!
Most of the tools in the functional toolbox are "function
transformers" that convert functions to functions

Core FP principle:
Composition everywhere

Lego Philosophy
1. All pieces are designed to be connected
2. Connect two pieces together and get
another "piece" that can still be connected
3. The pieces are reusable in many contexts

All pieces are designed to be connected

Connect two pieces together and
get another "piece" that can still be connected

The pieces are reusable in different contexts

Make big things from small things in the same way

The Functional Programmer's Toolkit (NDC London 2019)

Functional Programming Philosophy
• Design functions that do one thing well
– Functions can be reused in different contexts
• Design functions to work together
– Expect the output of every function to become
the input to another, as yet unknown, function
• Use types to ensure that inputs match outputs

Function 1
apple -> banana
Function 2
banana -> cherry

>>
Function 1
apple -> banana
Function 2
banana -> cherry

New Function
apple -> cherry
Can't tell it was built from
smaller functions!
Where did the banana go?

Http
Response
Http
Request
A web application built from
functions only (no classes!)

Http
Response
Http
Request
I have a whole talk on "The Power of Composition" at
fsharpforfunandprofit.com/composition
A web application built from
functions only (no classes!)

Combining things with Monoids
Tool #1

1 + 2 = 3
1 + (2 + 3) = (1 + 2) + 3
1 + 0 = 1
0 + 1 = 1

1 + 2 = 3
Some things
A way of combining
them

2 x 3 = 6
Some things
A way of combining
them

"a" + "b" = "ab"
Some things
A way of combining
them

concat([a],[b]) = [a; b]
Some things
A way of combining
them

1 + 2
1 + 2 + 3
1 + 2 + 3 + 4
Is an integer
Is an integer
A pairwise operation has
become an operation that
works on lists!

1 + (2 + 3) = (1 + 2) + 3
Order of combining doesn’t matter
1 + 2 + 3 + 4
(1 + 2) + (3 + 4)
((1 + 2) + 3) + 4
All the same

1 - (2 - 3) = (1 - 2) - 3
Order of combining does matter

1 + 0 = 1
0 + 1 = 1
A special kind of thing that when
you combine it with something, just
gives you back the original
something

42 * 1 = 42
1 * 42 = 42
A special kind of thing that when
you combine it with something, just
gives you back the original
something

"" + "hello" = "hello"
"hello" + "" = "hello"
“Zero” for strings

• You start with a bunch of things, and some way of
combining them two at a time.
• Rule 1 (Closure):The result of combining two things is
always another one of the things.
• Rule 2 (Associativity):When combining more than
two things, which pairwise combination you do first
doesn't matter.
• Rule 3 (Identity element):There is a special thing
called "zero" such that when you combine any thing
with "zero" you get the original thing back. A monoid!

Hey, this is just the
kind of jargon that
makes my head hurt!
Show me why this is
useful!

• Rule 1 (Closure):The result of combining two
things is always another one of the things.
• Benefit: converts pairwise operations into
operations that work on lists.

1 + 2 + 3 + 4
[ 1; 2; 3; 4 ] |> List.reduce (+)
We'll see
this a lot!
Pairwise operations can be converted into
pairwise op

1 * 2 * 3 * 4
[ 1; 2; 3; 4 ] |> List.reduce (*)
pairwise op

"a" + "b" + "c" + "d"
[ "a"; "b"; "c"; "d" ] |> List.reduce (+)
pairwise op

Benefit of Associativity: Divide and conquer,
parallelization, and incremental accumulation.

Parallelization:
1 + 2 + 3 + 4

Parallelization:
(1 + 2) (3 + 4)
3 + 7

(1 + 2 + 3)

(1 + 2 + 3) + 4

(6) + 4

• How can I use reduce on an empty list?
• In a divide and conquer algorithm, what should I
do if one of the "divide" steps has nothing in it?
• When using an incremental algorithm, what
value should I start with when I have no data?

Identity element
• Benefit: Initial value for empty or missing data

Tip:
Simplify aggregation code with monoids

type OrderLine = {Qty:int; Total:float}
let orderLines = [
{Qty=2; Total=19.98}
{Qty=1; Total= 1.99}
{Qty=3; Total= 3.99} ]
How to add them up?

type OrderLine = {Qty:int; Total:float}
let combine line1 line2 =
let newQty = line1.Qty + line2.Qty
let newTotal = line1.Total + line2.Total
{Qty=newQty; Total=newTotal}
orderLines |> List.reduce combine
// {Qty=6; Total= 25.96}
Any combination
of monoids is
also a monoid
Write a pairwise combiner
Profit!

Pattern:
Convert non-monoids to monoids

Customer
+
Customer
+
Customer
Customer Stats
+
Customer Stats
+
Customer Stats
Reduce
Map
Not a monoid A monoid
Customer Stats
(Total)

Hadoop make me a sandwich
https://twitter.com/daviottenheimer
/status/532661754820829185

Beware: You will soon start seeing monoids everywhere
Monoids
Everywhere

Metrics guideline:
Use counters rather than rates
Alternative metrics guideline:
Make sure your metrics are monoids
• incremental updates
• can handle missing data

Many "design patterns" are monoids
• Composite Pattern
• Null Object Pattern
• Composable commands
• Closure of operations (DDD)
Mark Seemann: http://blog.ploeh.dk/2019/01/28/better-abstractions-revisited/

Is function composition
a monoid?

>>
Function 1
apple -> banana
Function 2
banana -> cherry
New Function
apple -> cherry
Not the same
thing.
Not a monoid 

>>
Function 1
apple -> apple
Same thing
Function 2
apple -> apple
Function 3
apple -> apple
A monoid! 

Monoid summary
• A set of values and a combining function:
combine (aka concat, plus, <>, <+> )
• Closed and associative and has a zero value
• Uses:
– Reduce a list of values to a single value
– Do parallel calculations
– Do incremental calculations

What is an effect?
• A generic type
List<_>
• A type enhanced with extra data
Option<_>, Result<_>
• A type that can change the outside world
Async<_>, Task<_>, Random<_>
• A type that carries state
State<_>, Parser<_>

What is an effect?
• A generic type
List<_>
• A type enhanced with extra data
Option<_>, Result<_>
• A type that can change the outside world
Async<_>, Task<_>, Random<_>
• A type that carries state
State<_>, Parser<_>
We'll focus on
three for this talk

"Normal" world vs.
"Effects" world

"Normal" world
int
string
bool
int -> string
int -> bool

"Option" world
Option<int>
Option<string>
Option<bool>
Option<int> -> Option<string>
Option<int> -> Option<bool>

"List" world
List<int>
List<string>
List<bool>
List<int> -> List<string>
List<int> -> List<bool>

"Async" world
Async<int>
Async<string>
Async<bool>
Async<int> -> Async<string>
Async<int> -> Async<bool>

"Effects" world
E<int>
E<string>
E<bool>
E<int> -> E<string>
E<int> -> E<bool>

How to do stuff in an effects
world?
Problem:

World of normal values
int string bool
World of options
Option<int> Option<string> Option<bool>

World of options
int string bool


World of options

int string bool

let add42 x = x + 42
add42 1 // 43
add42 (Some 1) // error

let add42ToOption opt =
if opt.IsSome then
let newVal = add42 opt.Value
Some newVal
else
None 

World of options
add42 

World of options
add42

Moving functions between
worlds with "map"
Tool #2

World of options
Option<T> -> -> Option
Option.map
T -> -> U

let add42 x = ...
let add42ToOption = Option.map add42
add42ToOption (Some 1) // Some 43


let add42 x = ...
(Option.map add42) (Some 1)

Example:
Working with List world

let add42ToEach list =
let newList = new List()
for item in list do
let newItem = add42 item
newList.Add(newItem)
// return
newList


World of lists
add42 

World of lists
List.map
List<T> -> -> List
T -> -> U

let add42 x = ...
let add42ToEach = List.map add42
add42ToEach [1;2;3] // [43;44;45]


World of async
async<T> -> -> async
Async.map
T -> -> U

Guideline:
Most wrapped generic types
have a “map”. Use it!

Guideline:
If you create your own generic type,
create a “map” for it.

FP terminology
A functor is
i. An effect type
– e.g. Option<>, List<>, Async<>
ii. Plus a "map" function that "lifts" a function to
the effects world
– a.k.a. select, lift
iii. And it must have a sensible implementation
– the Functor laws

Moving values between worlds
with "return"
Tool #3

World of options
Option<int>
Option.return
int

let x = 42
let intOption = Some x

World of lists
List<int>
List.return
int

Chaining world-crossing
functions with "bind"
Tool #4

What's a
world-crossing function?

let range max = [1..max]
// int -> List<int>

ListWorld
Normal world
A world crossing function
List<int>
int
range

let getCustomer id =
if customerFound then
Some customerData
else
None
// CustomerId -> Option<CustomerData>

OptionWorld
Normal world
A world crossing function
Option<CustomerData>
CustomerId
getCustomer

Problem:
How do you chain
world-crossing functions?

let example input =
let x = doSomething input
if x.IsSome then
let y = doSomethingElse (x.Value)
if y.IsSome then
let z = doAThirdThing (y.Value)
if z.IsSome then
let result = z.Value
Some result
else
None
else
None
else
None

let taskExample input =
let taskX = startTask input
taskX.WhenFinished (fun x ->
let taskY = startAnotherTask x
taskY.WhenFinished (fun y ->
let taskZ = startThirdTask y
taskZ.WhenFinished (fun z ->
z // final result
)
)
)

let example input =
if x.IsSome then
if y.IsSome then
if z.IsSome then
// do something with z.Value
// in this block
else
None
else
None
else
None
Let's fix this!
There is a pattern we can exploit...

let example input =
if x.IsSome then
if y.IsSome then
if z.IsSome then
// do something with z.Value
// in this block
else
None
else
None
else
None

let example input =
if x.IsSome then
if y.IsSome then
// do something with y.Value
// in this block
else
None
else
None

let example input =
if x.IsSome then
// do something with x.Value
// in this block
else
None
Can you see the pattern?

if opt.IsSome then
//do something with opt.Value
else
None

let ifSomeDo f opt =
if opt.IsSome then
f opt.Value
else
None

let example input =
doSomething input
|> ifSomeDo doSomethingElse
|> ifSomeDo doAThirdThing
|> ifSomeDo ...
let ifSomeDo f opt =
if opt.IsSome then
f opt.Value
else
None

Some
None
Input ->
Let's use a railway analogy

>> >>
Composing one-track functions is fine...

>> >>
... and composing two-track functions is fine...

 
... but composing switches is not allowed!

How to combine the
mismatched functions?

“Bind” is the answer!
Bind all the things!

Two-track input Two-track input
One-track input Two-track input



Two-track input
Slot for switch function
Two-track output

Two-track input Two-track output

let bind nextFunction optionInput =
match optionInput with
| Some s -> nextFunction s
| None -> None
Two-track input Two-track output

Pattern:
Use bind to chain options

let example input =
if x.IsSome then
if y.IsSome then
if z.IsSome then
let result = z.Value
Some result
else
None
else
None
else
None
Before

let bind f opt =
match opt with
| Some v -> f v
| None -> None
After

Pattern:
Use bind to chain tasks
a.k.a "promise" "future"

let taskX = startTask input
taskX.WhenFinished (fun x ->
let taskY = startAnotherTask x
taskY.WhenFinished (fun y ->
let taskZ = startThirdTask y
taskZ.WhenFinished (fun z ->
z // final result
)
)
)
Before

let taskBind f task =
task.WhenFinished (fun taskResult ->
f taskResult)
startTask input
|> taskBind startAnotherTask
|> taskBind startThirdTask
|> taskBind ...
After

Problem:
How to handle errors elegantly?

string UpdateCustomerWithErrorHandling()
{
var request = receiveRequest();
validateRequest(request);
canonicalizeEmail(request);
db.updateDbFromRequest(request);
smtpServer.sendEmail(request.Email)
return "OK";
}

{
var isValidated = validateRequest(request);
if (!isValidated) {
return "Request is not valid"
}
db.updateDbFromRequest(request);
return "OK";
}

{
if (!isValidated) {
}
var result = db.updateDbFromRequest(request);
if (!result) {
return "Customer record not found"
}
return "OK";
}

{
if (!isValidated) {
}
try {
if (!result) {
}
} catch {
return "DB error: Customer record not updated"
}
return "OK";
}

{
if (!isValidated) {
}
try {
if (!result) {
}
} catch {
return "DB error: Customer record not updated"
}
if (!smtpServer.sendEmail(request.Email)) {
log.Error "Customer email not sent"
}
return "OK";
}

Solution:
Use a Result type for error handling

Request SuccessValidate
Failure
type Result =
| Ok of SuccessValue
| Error of ErrorValue
Define a choice type

Request SuccessValidate
Failure
let validateInput input =
if input.name = "" then
Error "Name must not be blank"
else if input.email = "" then
Error "Email must not be blank"
else
Ok input // happy path

Why is bind so important?
It makes world-crossing functions
composable

EffectsWorld
Normal World
a
E
Before bind:
A diagonal function
(world crossing)
bind

bind
EffectsWorld
Normal World
a
E
EffectsWorld
Normal World
E<a> E
After bind:
A horizontal function
(all in E-world)

NormalWorld
EffectsWorld
"Diagonal" functions can't be composed

NormalWorld
EffectsWorld
"Horizontal" functions can be composed

FP terminology
A monad is
i. An effect type
ii. Plus a return function
– a.k.a. pure unit
iii. Plus a bind function that converts a "diagonal"
(world-crossing) function into a "horizontal" (E-
world-only) function
– a.k.a. >>= flatMap SelectMany
iv. And bind/return must have sensible implementations
– the Monad laws

TLDR: If you want to chain effects-
generating functions in series,
use a Monad

=
compose
with
Result is same kind of thing
Kleisli Composition

=>=>
Result is another monadic
function so you can repeat.
Kleisli Composition
Kleisli composition symbol
Sound familiar?

>=>
Order not
important
(Associative) Monoid!
>=>( )
( )>=> >=>

Combining effects in parallel
with applicatives
Tool #5

How to combine effects?
Option<T> Option+
Option<T,U>

Some 42 Some "hello"+
Some (42,"hello")
Combining options
This is what you expect!

Some 42 None+
None
Combining options

How to combine Lists?
List<T> List+
List<T,U>

[1,2,3] ["a","b","c"]+
[ (1,"a"), (1,"b"), (1,"c")
(2,"a"), (2,"b"), (2,"c")
(3,"a"), (3,"b"), (3,"c") ]
Combining lists (cross product)

[1,2,3] ["a","b","c"]+
[ (1,"a")
(2,"b")
(3,"c") ]
Combining lists (zip)

The general term for this is
"applicative functor"
Option, List, Async are all applicatives

FP terminology
A applicative (functor) is
i. An effect type
ii. Plus a return function
– a.k.a. pure unit
iii. Plus a function that combines two effects into one
– a.k.a. <*> apply pair
iv. And apply/return must have sensible implementations
– the Applicative Functor laws
So why is this useful?

Problem:
How to validate multiple fields
in parallel?

type Customer = {
Name : String50
Email : EmailAddress
Birthdate : Date
}
validateName validateEmail validateBirthdate
So we create some validation functions:
Each field must be validated

validateName validateEmail validateBirthdate
Problem: Validation done in series.
So only one error at a time is returned

type CustomerDto = {
name : string
email : string
birthdate : string
} validateName
validateEmail
validateBirthdate
Combine
output
Now we do get all
errors at once!
... But how to
combine them?

ResultWorld
R<name> R<email> R<bdate>

validName validEmail validDate
validCustomer
We know how to combine the normal values
(use a constructor)

ResultWorld
R<Customer>
The output is also in
Result world
Use the magic
of Applicatives!

let createValidCustomer (dto:CustomerDto) =
// get the validated values
let nameOrError = validateName dto.name
let emailOrError = validateEmail dto.email
let birthdateOrError =
validateBirthdate dto.birthdate
// call the constructor
makeCustomer
<!> nameOrError
<*> emailOrError
<*> birthdateOrError
// final output is Result<Customer,ErrMsg list>
Here's where the
magic happens!
What the code looks like
Monoids used here

• "combine"
– Combines two values to make another one of the same
kind
• "reduce"
– Reduces a list to a single value by using "combine"
repeatedly

• "map"
– Lifts functions into an effects world
• "return"
– Lifts values into an effects world
• "bind"
– Converts "diagonal" functions into "horizontal" ones so
they can be composed.
• "apply"
– Combines two effects in parallel
– "map2", "lift2", "lift3" are specialized versions of "apply“

Using all the tools together
Example

Example scenario
• Download a URL into a JSON file
• Decode the JSON into a Customer DTO
• Convert the DTO into a valid Customer
• Store the Customer in a database

NormalWorld
ResultWorld
AsyncWorld
Download the json file
Url
AsyncResult<Json>

ResultWorld
R<CustomerDto>
Json
Decode the json

ResultWorld
CustomerDto
Validate fields

Construct the customer
validCustomer

NormalWorld
ResultWorld
AsyncWorld
Customer
Store the customer
AsyncResult<unit>

How do we compose these
functions together?
None the worlds match up ...
... but we can use the functional toolkit!

ResultWorld
CustomerDto
Validate fields AND create a customer
validCustomer
We already did this one using applicatives (and monoids)

ResultWorld
CustomerDto
R<Customer>
Validate fields AND create a customer
We already did this one using applicatives, and monoids

ResultWorld
CustomerDto
R<Customer>
Parse json AND create a customer
R<CustomerDto>
Json
Use "bind" to turn the diagonal
functions into horizontal ones
Bind Bind

ResultWorld
R<Customer>
R<CustomerDto>R<Json>
Bind Bind

ResultWorld
R<Customer>R<Json>
Then compose them into one function

let processCustomerDto jsonOrError =
jsonOrError
|> Result.bind decodeCustomerDto
|> Result.bind createValidCustomer

NormalWorld
ResultWorld
AsyncWorld
R<Customer>R<Json>
Then lift it up to Async
world using map
Map

NormalWorld
ResultWorld
AsyncWorld
AsyncResult<Customer>AsyncResult<Json>

NormalWorld
AsyncWorld
Customer
Store the customer
Use "bind" to turn the
diagonal function horizontal
AsyncResult<unit>
Bind

NormalWorld
ResultWorld
AsyncWorld
AsyncResult<Customer> AsyncResult<unit>
Store the customer
Bind

NormalWorld
AsyncWorld
All steps are now composable
Url
AsyncResult<Json>
AsyncResult<Customer>AsyncResult<Json>
AsyncResult<Customer> AsyncResult<unit>

NormalWorld
AsyncWorld
All steps are now composable
Url
AsyncResult<Json> AsyncResult<Customer> AsyncResult<unit>

NormalWorld
AsyncWorld
All steps are now composable into one single function
Url
AsyncResult<unit>

let processCustomerDto jsonOrError =
jsonOrError
|> Result.bind decodeCustomerDto
|> Result.bind createValidCustomer
let downloadAndStoreCustomer url =
url
|> downloadFile
|> Async.map processCustomerDto
|> AsyncResult.bind storeCustomer
It takes much longer to explain it
than to write it!

In conclusion…
• FP jargon is not that scary
– Can you see why monads are useful?
• The FP toolkit is very generic
– FP's use these core functions constantly!
• You can now recognize "map", "apply" and "bind"
– Don’t expect to understand them all straight away.

"The Functional Programming Toolkit"
– Slides and video will be posted at
• fsharpforfunandprofit.com/fptoolkit
Related talks
– "Functional Design Patterns"
• fsharpforfunandprofit.com/fppatterns
– "The Power of Composition"
• fsharpforfunandprofit.com/composition
– "Domain Modeling Made Functional"
• fsharpforfunandprofit.com/ddd
Thanks!
Twitter: @ScottWlaschin

The Functional Programmer's Toolkit (NDC London 2019)

More Related Content

What's hot (20)

Similar to The Functional Programmer's Toolkit (NDC London 2019) (20)

More from Scott Wlaschin (13)

Recently uploaded (20)

The Functional Programmer's Toolkit (NDC London 2019)