Automatic Type Class Derivation with Shapeless

AUTOMATIC TYPE CLASS DERIVATION
WITH SHAPELESS
Shi Forward Tech Talks
July 6, 2017 / Joao Azevedo
The purpose of this talk is to give the basics on how to do automatic type class derivation using Shapeless. At the end of the
talk, attendees should be aware of the building blocks Shapeless provides and be able to apply some of the described patterns
to derive their own type classes.

TYPE CLASSES
A definition of behaviour in the form of operations that
must be supported by a given type.
A way to implement ad hoc polymorphism.

TYPE CLASSES
def mean[T](xs: Vector[T])(implicit ev: NumberLike[T]): T =
ev.divide(xs.reduce(ev.plus(_, _)), xs.size)

SHAPELESS
You must be shapeless, formless, like water.
When you pour water in a cup, it becomes
the cup. When you pour water in a bottle, it
becomes the bottle. When you pour water
in a teapot, it becomes the teapot. Water
can drip and it can crash. Become like
water my friend. -- Bruce Lee

SHAPELESS
A type class and dependent type based generic
programming library for Scala.
Generic programming provides ways to exploit similarities
between types to avoid repetition.
case class Employee(name: String, number: Int, manager: Boolean)
case class IceCream(name: String, numCherries: Int, inCone: Boolean)
Sometimes, types are too speciﬁc and we want to explore similarities in their shape to avoid repetition. In the presented
example, even though Employee and IceCream are different types, they share the same shape: they both contain three ﬁelds of
the same types: String, Int and Boolean.

A RANDOM EXAMPLE
We start with a practical example to show some of the shapeless concepts that are relevant for automatic derivation of type
classes.

A RANDOM EXAMPLE
trait Random[A] extends Function0[A]
object Random {
def apply[A](implicit r: Random[A]) = r()
}
Random[Int]
// 578073111
Random[Double]
// 0.48802405390668835
Random[String]
// mbbHlZAam
Random[Employee]
// Employee(Y7F,1976420522,true)
Random[IceCream]
// IceCream(CywOmpJd,-335712437,false)
The idea is to have a Random typeclass that is capable of producing "random" instances of a given type.

A RANDOM EXAMPLE
implicit val rInt: Random[Int] =
() => util.Random.nextInt
implicit val rDouble: Random[Double] =
() => util.Random.nextDouble
implicit val rString: Random[String] =
() => util.Random.alphanumeric.take(
util.Random.nextInt(10)).mkString
implicit val rBoolean: Random[Boolean] =
() => util.Random.nextBoolean
We can start by implementing Random instances for some base types. The String implementation is obviously incorrect, but the
speciﬁc implementation is not very relevant for now. We just want to make sure we have a typeclass for some base types.

A RANDOM EXAMPLE
implicit def rEmployee(implicit rs: Random[String],
ri: Random[Int],
rb: Random[Boolean]): Random[Employee] =
() => Employee(rs(), ri(), rb())
implicit def rIceCream(implicit rs: Random[String],
ri: Random[Int],
rb: Random[Boolean]): Random[IceCream] =
() => IceCream(rs(), ri(), rb())
Having implementations of Random for some base types, we can have implementations of Random for more complex types that
base themselves on the Random implementations for the types of ﬁelds that compose those types.
However, we still have two distinct implementations for Employee and IceCream even though they're remarkably similar. Can
we do better?

SHAPELESS TO THE RESCUE!
shapeless allows us to do better by taking advantage of the shape of the types and exploiting similarities between shapes.

HLISTS
"hello" :: 13 :: true :: HNil
: String :: Int :: Boolean :: HNil
The ﬁrst important concept from shapeless that is very relevant for the purpose of automatic derivation of type classes are
HLists. HLists are a generic representation of products, much like Scala's built-in tuples. They're a bit more powerful than
Scala's tuples because each size of tuple has an unrelated type and we can't represent 0-length tuples. HLists are similar to
Scala's Lists, but, at compile time, we know both their size and the type of each element that composes the list.

RANDOM HLISTS
implicit val rHNil: Random[HNil] = () => HNil
implicit def rHList[H, T <: HList](
implicit rh: Random[H], rt: Random[T]): Random[H :: T] =
() => rh() :: rt()
Knowing what HLists are, we can create an implementation of Random for HLists, provided that we have implementations of
Random for the type of each element that the list is composed of. This already provides us with a generic implementation of
Random for products.

GENERIC
Generic[Employee]
// shapeless.Generic[Employee]{
// type Repr = String :: Int :: Boolean :: shapeless.HNil} =
// anon$macro$16$1@3f09ba38
Generic[IceCream]
// shapeless.Generic[IceCream]{
// type Repr = String :: Int :: Boolean :: shapeless.HNil} =
// anon$macro$12$1@750eba8f
trait Generic[T] {
type Repr
def to(t: T): Repr
def from(r: Repr): T
}
shapeless provides a type class called Generic that allows us to go from an algebraic data type to its generic representation as
an HList (and vice-versa). The Generic instance has a type member Repr containing the type of its generic representation.

GENERIC
val hlist = "a" :: 1 :: true :: HNil
Generic[Employee].from(hlist)
// Employee(a,1,true)
Generic[IceCream].from(hlist)
// IceCream(a,1,true)
Using Generic, we can take advantage of the shape of an ADT and convert back and forth from different ADTs if they have the
same Repr.

RANDOM PRODUCTS
implicit def rProduct[T](
implicit g: Generic[T], rg: Random[g.Repr]): Random[T] =
() => g.from(rg())
Unfortunately this doesn't compile.
Taking this into account, we can derive an implementation of Random for products if we are able to derive an implementation of
Random for their generic representation (which we already did for HLists in a previous slide).
Unfortunately this doesn't compile because we can't reference a type member of a parameter within the same parameter list the
parameter is in, and all the implicits must go in one parameter list.

RANDOM PRODUCTS
object Generic {
type Aux[T, Repr0] = Generic[T] { type Repr = Repr0 }
}
implicit def rProduct[T, Repr](
implicit g: Generic.Aux[T, Repr], rg: Random[Repr]): Random[T] =
() => g.from(rg())
Random[Employee]
// Employee(ltdui,-443373978,false)
case class Foo(a: Employee, b: String)
Random[Foo]
// Diverging implicit expansion!
shapeless solves this problem by using the Aux pattern, which is nowadays a very common technique to expose type members
as type parameters. This ﬁnally allows us to have a generic implementation of Random for products.
Unfortunately, the compiler can run into cycles during implicit search, which can gives us a diverging implicit expansion error.

LAZY
Creates a macro that triggers implicit search for types
wrapped in `Lazy` only once.
implicit val rHNil: Random[HNil] = () => HNil
implicit def rHList[H, T <: HList](
implicit rh: Lazy[Random[H]],
rt: Lazy[Random[T]]): Random[H :: T] =
() => rh.value() :: rt.value()
implicit def rProduct[T, Repr <: HList](
implicit g: Generic.Aux[T, Repr],
rg: Lazy[Random[Repr]]): Random[T] =
() => g.from(rg.value())
To help us with diverging implicit expansion errors, shapeless provides us with the Lazy macro. The Lazy macro triggers the
implicit search for a given implicit and if this search triggers searches for types wrapped in Lazy then these will be done only
once and wrapped in a lazy val, which is returned as the corresponding value.

RANDOM COPRODUCTS
sealed trait Receptacle
case class Bottle(a: Int) extends Receptacle
case class Glass(a: String) extends Receptacle
case class Teapot(a: Boolean) extends Receptacle
Generic[Receptacle]
// shapeless.Generic[Receptacle]{
// type Repr = Bottle :+: Glass :+: Teapot :+: shapeless.CNil}
val g = Generic[Receptacle]
g.to(Glass("a"))
// Inr(Inl(Glass(a)))
g.to(Bottle(1))
// Inl(Bottle(1))
g.to(Teapot(true))
// Inr(Inr(Inl(Teapot(true))))
shapeless can also help us with coproducts (i.e. sealed families of case classes). The Generic Repr of coproducts is a
Coproduct instead of an HList. Coproduct is deﬁned in terms of Inr (which doesn't have a value and thus defers to the tail) and
Inl (which has a value and nothing else, thus guaranteeing that there's only one Inl in a Coproduct).

RANDOM COPRODUCTS
case class CoproductOptions[A, C <: Coproduct](
options: List[() => A] = Nil)
implicit def rCNil[A]: CoproductOptions[A, CNil] =
CoproductOptions[A, CNil](Nil)
implicit def rCP[A, H <: A, T <: Coproduct](
implicit rh: Lazy[Random[H]],
rt: Lazy[CoproductOptions[A, T]]): CoproductOptions[A, H :+: T] =
CoproductOptions[A, H :+: T](rh.value :: rt.value.options)
implicit def rCoproduct[T, Repr <: Coproduct](
implicit g: Generic.Aux[T, Repr],
rg: Lazy[CoproductOptions[T, Repr]]): Random[T] =
() => {
val choices = rg.value.options
choices(util.Random.nextInt(choices.length))()
}

RANDOM COPRODUCTS
Random[Receptacle]
// Bottle(2143179073)
Random[Receptacle]
// Glass(eMaxESFnD)
Random[Receptacle]
// Bottle(1915773318)
Random[Receptacle]
// Glass(4MX5xYi8v)
Random[Receptacle]
// Teapot(false)

SPRAY-JSON
sealed abstract class JsValue
case class JsObject(fields: Map[String, JsValue]) extends JsValue
case class JsArray(elements: Vector[JsValue]) extends JsValue
case class JsString(value: String) extends JsValue
case class JsNumber(value: BigDecimal) extends JsValue
sealed trait JsBoolean extends JsValue
case object JsTrue extends JsBoolean
case object JsFalse extends JsBoolean
case object JsNull extends JsValue
So far we haven't required the names of the fields of ADTs when deriving typeclasses. In order to show how to use them, we'll
be using spray-json as an example. A simplified version of the spray-json AST is defined as this.

JSONFORMAT
trait JsonFormat[T] {
def read(json: JsValue): T
def write(obj: T): JsValue
}
We want to use shapeless to derive instances of the JsonFormat type class.

SINGLETON TYPES
"bar".narrow : String("bar") // <: String
42.narrow : Int(42) // <: Int
'foo.narrow : Symbol('foo) // <: Symbol
true.narrow : Boolean(true) // <: Boolean
'a ->> "bar" : String with KeyTag[Symbol('a), String]
'b ->> 42 : Int with KeyTag[Symbol('b), Int]
'c ->> true : Boolean with KeyTag[Symbol('c), Boolean]
val a = implicitly[Witness[String("foo")]].value : String("foo")
field[Symbol('a)]("bar") : FieldType[Symbol('a), String]
field[Symbol('b)](42) : FieldType[Symbol('b), Int]
field[Symbol('c)](true) : FieldType[Symbol('c), Boolean]
shapeless introduces the concept of a singleton type, a construction that allows lifting a constant value to a type. The type of a
value that is narrowed is a subtype of the original type, but is reﬁned with a singleton instance of the type. The narrows get
erased at runtime, but allow us to work with them at compile time.
Singleton types are commonly used to add typelevel keys to a given type, and shapeless provides us with utilites to both add
keys and extract keys from a tagged type.

BACK TO HLISTS
('name ->> "foo") :: ('number ->> 42) :: ('manager ->> true) :: HNil
: FieldType[Symbol('name), String] ::
FieldType[Symbol('number), Int] ::
FieldType[Symbol('manager), Boolean] ::
HNil
case class Employee(name: String, number: Int, manager: Boolean)
Using the concept of FieldType previously introduced, we can extend our generic representation of a product to also include the
name of the ﬁelds the type is composed of.

implicit object HNilFormat extends JsonFormat[HNil] {
def read(j: JsValue) = HNil
def write(n: HNil) = JsObject()
}
implicit def hListFormat[Key <: Symbol, Value, Remaining <: HList](
implicit
key: Witness.Aux[Key],
jfh: JsonFormat[Value],
jft: JsonFormat[Remaining]
) = new JsonFormat[FieldType[Key, Value] :: Remaining] {
def write(hlist: FieldType[Key, Value] :: Remaining) =
JsObject(jft.write(hlist.tail).asJsObject.fields +
(key.value.name -> jfh.write(hlist.head)))
def read(json: JsValue) = {
val fields = json.asJsObject.fields
val head = jfh.read(fields(key.value.name))
val tail = jft.read(json)
field[Key](head) :: tail
}
}
We can also easily derive an implementation of JsonFormat for HLists of FieldTypes.

val employee =
('name ->> "foo") ::
('number ->> 42) ::
('manager ->> true) :: HNil
employee.toJson
// {"manager":true,"number":42,"name":"foo"}

LABELLEDGENERIC
LabelledGeneric[Employee]
// shapeless.LabelledGeneric[Employee]{
// type Repr = String with KeyTag[Symbol with Tagged[String("name")],String
// Int with KeyTag[Symbol with Tagged[String("number")],Int] ::
// Boolean with KeyTag[Symbol with Tagged[String("manager")],Bo
// = shapeless.LabelledGeneric$$anon$1@5832492c
LabelledGeneric[IceCream]
// shapeless.LabelledGeneric[Employee]{
// type Repr = String with KeyTag[Symbol with Tagged[String("name")],String
// Int with KeyTag[Symbol with Tagged[String("numCherries")],In
// Boolean with KeyTag[Symbol with Tagged[String("inCone")],Boo
// = shapeless.LabelledGeneric$$anon$1@17a5c45a
trait LabelledGeneric[T] {
type Repr
def to(t: T): Repr
def from(r: Repr): T
}
shapeless provides a type class called LabelledGeneric that allows us to go from an algebraic data type to its generic
representation as an HList of FieldTypes (and vice-versa). The LabelledGeneric instance has a type member Repr containing
the type of its generic representation.

LABELLEDGENERIC
val hlist = ('name ->> "foo") ::
('number ->> 42) ::
('manager ->> true) :: HNil
LabelledGeneric[Employee].from(hlist)
// Employee(foo,42,true)
LabelledGeneric[IceCream].from(hlist)
// Does not compile

LABELLEDGENERIC
implicit def productFormat[T, Repr <: HList](
implicit
gen: LabelledGeneric.Aux[T, Repr],
sg: JsonFormat[Repr]
): JsonFormat[T] = new JsonFormat[T] {
def read(j: JsValue): T = gen.from(sg.read(j))
def write(t: T): JsValue = sg.write(gen.to(t))
}
Employee("foo", 42, true).toJson
// {"manager":true,"number":42,"name":"foo"}
Taking this into account, we can derive an implementation of JsonFormat for products if we are able to derive an implementation
of JsonFormat for their generic representation (which we already did for HLists of FieldTypes in a previous slide).
We leave the derivation of JsonFormats for coproducts as an exercise for the reader.

REFERENCES
The Type Astronaut's Guide to Shapeless
spray-json-shapeless
Shapeless for Mortals

Automatic Type Class Derivation with Shapeless

More Related Content

What's hot (20)

Similar to Automatic Type Class Derivation with Shapeless (20)

Recently uploaded (20)

Automatic Type Class Derivation with Shapeless