SlideShare a Scribd company logo

Writing DSL with Applicative Functors

D
David Galichet

This document discusses writing a domain specific language (DSL) for data transformations using applicative functors in Scala. It introduces the concepts of Picker, Reader, and Result to parse heterogeneous data formats into a common format. Reader is defined as an applicative functor to allow combining multiple readers. Later, Reader is enhanced to take type parameters for both input and output to avoid reparsing data and support XML parsing. Type lambdas are used to make Reader work as an applicative functor.

Technology
1 of 43
Downloaded 24 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
Writing DSL with Applicative Functors David Galichet Freelance functional programmer ! twitter: @dgalichet
Content normalization • We want to parse heterogeneous data formats (CSV, XML …) and transform them to a pivot format (Scala object in our case) • The transformation should be described as a DSL • This DSL must be simple to use, and enable any kinds of data transformations or verifications
Expected DSL format val reader = ( ! Pick(0).as[String].map(_.capitalize) and ! Pick(1).as[Date].check(_.after(now())) ! ).reduce(FutureEvent)! ! reader("PSUG; 21/08/2014") // returns Success(FutureEvent("PSUG", date)) Inspired by Play2 Json API
tag: step1 Conventions & material • Code is available on Github : https://github.com/ dgalichet/PsugDSLWritingWithApplicative • Code revision is define on the top of any slides including source code (just checkout the specified tag)
tag: step1 Reading single entry • We have a CSV line and we want to read one column • We will introduce several abstractions: • Picker: fetch a data from CSV or XML • Result: either a Success or a Failure • Converter: convert value from Picker to Reader • Reader: container with methods to process its content
tag: step1 Introducing Picker case class Picker(p: String => Result[String]) {! def as[T](implicit c: Converter[T]): Reader[T] = c.convert(p)! }! ! object CsvPicker {! def apply[T](i: Int)(implicit separator: Char): Picker = Picker { s: String =>! val elems = s.trim.split(separator)! if (i > 0 && elems.size > i) Success(elems(i).trim)! else Failure(s"No column ${i} for ${s}")! }! }
tag: step1 Introducing Picker case class Picker(p: String => Result[String]) {! def as[T](implicit c: Converter[T]): Reader[T] = c.convert(p)! }! ! object CsvPicker {! def apply[T](i: Int)(implicit separator: Char): Picker = Picker { s: String =>! val elems = s.trim.split(separator)! if (i > 0 && elems.size > i) Success(elems(i).trim)! else Failure(s"No column ${i} for ${s}")! }! } Picker wraps a function from String to Result
The Result tag: step1 sealed trait Result[+T] case class Success[T](t: T) extends Result[T] case class Failure(error: String) extends Result[Nothing]!
The Converter tag: step1 trait Converter[T] { def convert(p: String => Result[String]): Reader[T] }! ! object Converter {! implicit val string2StringConverter = new Converter[String] {! override def convert(p: String => Result[String]) = Reader[String] (p)! // See code on Github for more converters! }
The Converter tag: step1 trait Converter[T] { def convert(p: String => Result[String]): Reader[T] }! ! Convert the content of the Picker to a Reader ! object Converter {! implicit val string2StringConverter = new Converter[String] {! override def convert(p: String => Result[String]) = Reader[String] (p)! // See code on Github for more converters! }
The Reader tag: step1 case class Reader[O](p: String => Result[O]) { def apply(s: String): Result[O] = p(s) } A Reader doesn’t contain a value but a process to transform original data (CSV line or XML) to a Result
Usage sample tag: step1 import Converter._ // import implicit converters! implicit val separator = ‘;’! ! CsvPicker(1).as[String].apply("foo;bar") === "bar"
tag: step2 Enhancing the Reader • The first defines a very simple Reader. We must add a method to combine two instances of Reader • We will also enhance Failure to store multiple error messages
tag: step2 Enhancing the Reader case class Reader[O](p: String => Result[O]) {! def apply(s: String): Result[O] = p(s)! ! def and[O2](r2: Reader[O2]): Reader[(O, O2)] = Reader { s: String =>! (p(s), r2.p(s)) match {! case (Success(s1), Success(s2)) => Success((s1, s2))! case (Success(_), Failure(f)) => Failure(f)! case (Failure(f), Success(_)) => Failure(f)! case (Failure(f1), Failure(f2)) => Failure(f1 ++ f2)! }! }! def map[T](f: O => T): Reader[T] = Reader { s: String =>! p(s) match {! case Success(o) => Success(f(o))! case f: Failure => f! }! }! def reduce[T] = map[T] _ // alias for map! }
tag: step2 Enhancing Result type sealed trait Result[+T]! case class Success[T](t: T) extends Result[T]! case class Failure(error: NonEmptyList[String]) extends Result[Nothing]! ! object Failure {! def apply(s: String): Failure = Failure(NEL(s))! }! ! case class NonEmptyList[T](head: T, tail: List[T]) { def toList = head::tail def ++(l2: NonEmptyList[T]): NonEmptyList[T] = NonEmptyList(head, tail ++ l2.toList) } object NEL { def apply[T](h: T, t: T*) = NonEmptyList(h, t.toList) }
Usage sample tag: step2 implicit val separator = ';' implicit val dtFormatter = new SimpleDateFormat("dd/MM/yyyy") import Converter.string2StringConverter import Converter.string2DateConverter! ! val reader = ( CsvPicker(1).as[String] and CsvPicker(2).as[Date] ).reduce { case (n, d) => FutureEvent(n, d) }! reader("foo;bar;12/10/2014") === Success(FutureEvent("bar", dtFormatter.parse("12/10/2014")))! ! case class FutureEvent(name: String, dt: Date)
tag: step2 Usability problem • The use of reduce (or map) method to transform a Reader[(0, 02)] into an instance of Reader[FutureEvent] for example is quite verbose • This will be even more verbose for instances of Reader[(0, (02, 03))] • We want the API to automatically bind tuple elements to a constructor as we can encounter in Play2 Json API
tag: step3 Applicative functors • To tackle our problem, we will use Applicative Functors and play2 functional library (and especially FunctionalBuilder) • This approach is inspired by @sadache (Sadek Drobi) article https://gist.github.com/sadache/ 3646092 • An Applicative Functor is a Type Class relying on ad-hoc polymorphism to extends a Class with some properties • Play2 functional library (or Scalaz) provides mechanism to compose Applicatives in a smart way
tag: step3 Applicative functors M is an Applicative Functor if there exists the following methods : def pure[A](a: A): M[A] def map[A, B](m: M[A], f: A => B): M[B] def apply[A, B](mf: M[A => B], ma: M[A]): M[B]! with the following Laws : • Identity: apply(pure(identity), ma) === ma where ma is an Applicative M[A] • Homomorphism: apply(pure(f), pure(a)) === pure(f(a)) where f: A => B and a an instance of A • Interchange: mf if an instance of M[A => B] apply(mf, pure(a)) === apply(pure {(g: A => B) => g(a)}, mf)! • Composition: map(ma, f) === apply(pure(f), ma)
tag: step3 Applicative functors trait Applicative[M[_]] { def pure[A](a: A): M[A] def map[A, B](m: M[A], f: A => B): M[B] def apply[A, B](mf: M[A => B], ma: M[A]): M[B] } Applicative is an Higher Kinded type (parameterized with M that take a single type parameter)
tag: step3 Reader is an Applicative case class Reader[O](p: String => Result[O]) { def apply(s: String): Result[O] = p(s)! def map[T](f: O => T): Reader[T] = Reader { s: String => p(s) match { case Success(o) => Success(f(o)) case f: Failure => f } }! }! object Reader { def map2[O, O1, O2](r1: Reader[O1], r2: Reader[O2])(f: (O1, O2) => O): Reader[O] = Reader { s: String => (r1.p(s), r2.p(s)) match { case (Success(s1), Success(s2)) => Success(f(s1, s2)) case (Success(_), Failure(e)) => Failure(e) case (Failure(e), Success(_)) => Failure(e) case (Failure(e1), Failure(e2)) => Failure(e1 ++ e2) } } …! }
tag: step3 Reader is an Applicative object Reader { … // map2 implicit val readerIsAFunctor: Functor[Reader] = new Functor[Reader] { override def fmap[A, B](m: Reader[A], f: (A) => B) = m.map(f) } implicit val readerIsAnApplicative: Applicative[Reader] = new Applicative[Reader] { override def pure[A](a: A) = Reader { _ => Success(a) } override def apply[A, B](mf: Reader[A => B], ma: Reader[A]) = map2(mf, ma)((f, a) => f(a)) override def map[A, B](m: Reader[A], f: A => B) = m.map(f) } }
Usage sample tag: step3 import Converter.string2StringConverter import Converter.string2DateConverter! import play.api.libs.functional.syntax._ import Reader.readerIsAnApplicative! ! implicit val separator = ';' implicit val dtFormatter = new SimpleDateFormat("dd/MM/yyyy")! ! val reader = ( CsvPicker(1).as[String] and CsvPicker(2).as[Date] )(FutureEvent) // here we use CanBuild2.apply reader("foo;bar;12/10/2014") === Success(FutureEvent("bar", dtFormatter.parse("12/10/2014")))!
Usage sample tag: step3 (errors accumulation) import Converter.string2StringConverter import Converter.string2DateConverter! import play.api.libs.functional.syntax._ import Reader.readerIsAnApplicative! ! implicit val separator = ';' implicit val dtFormatter = new SimpleDateFormat("dd/MM/yyyy")! ! val reader = ( CsvPicker(1).as[Int] and CsvPicker(2).as[Date] )((_, _)) reader(List("foo", "not a number", "not a date")) === Failure(NEL(! "Unable to format 'not a number' as Int", ! "Unable to format 'not a date' as Date"))!
Benefits tag: step3 • Making Reader an Applicative Functor give ability to combine efficiently instances of Reader • Due to Applicative properties, we still accumulate errors • Play2 functional builder give us a clean syntax to define our DSL
tag: step4 Introducing XML Picker case class Picker(p: String => Result[String]) { def as[T](implicit c: Converter[T]): Reader[T] = c.convert(p) }! ! object XmlPicker { def apply[T](query: Elem => NodeSeq): Picker = Picker { s: String => try { val xml = XML.loadString(s) Success(query(xml).text) } catch { case e: Exception => Failure(e.getMessage) } } }!
Usage sample tag: step4 import play.api.libs.functional.syntax._ import Reader.readerIsAnApplicative! import Converter._ implicit val dF = new SimpleDateFormat("dd/MM/yyyy")! val xml = """ <company name="Dupont and Co"> <owner> <person firstname="jean" lastname="dupont" birthdate="11/03/1987"/> </owner> </company>""" val r = ( XmlPicker(_ "person" "@firstname").as[String] and XmlPicker(_ "person" "@lastname").as[String] and XmlPicker(_ "person" "@birthdate").as[Date] )(Person)! r(xml) === Success(Person("jean","dupont",dF.parse("11/03/1987"))) case class Person(firstname: String, lastname: String, birthDt: Date)
tag: step4 Implementation problem • The Reader[O] takes a type argument for the output. The input is always a String • With this implementation, an XML content will be parsed (with XML.load) as many times as we use XmlPicker. This will cause unnecessary overhead • We will have the same issue (with lower overhead) with our CsvPicker
tag: step5 Introducing Reader[I, 0] To resolve this problem, we will modify Reader to take a type parameter for the input
tag: step5 Introducing Reader[I, 0] case class Reader[I, O](p: I => Result[O]) { def apply(s: I): Result[O] = p(s) def map[T](f: O => T): Reader[I, T] = Reader { s: I => p(s) match { case Success(o) => Success(f(o)) case f: Failure => f } }! }! object Reader { def map2[I, O, O1, O2](r1: Reader[I, O1], r2: Reader[I, O2])(f: (O1, O2) => O): Reader[I, O] = Reader { s: I => (r1.p(s), r2.p(s)) match { case (Success(s1), Success(s2)) => Success(f(s1, s2)) case (Success(_), Failure(e)) => Failure(e) case (Failure(e), Success(_)) => Failure(e) case (Failure(e1), Failure(e2)) => Failure(e1 ++ e2) } }
tag: step5 Introducing Reader[I, 0] object Reader {! implicit def readerIsAFunctor[I] = new Functor[({type λ[A] = Reader[I, A]})#λ] { override def fmap[A, B](m: Reader[I, A], f: (A) => B) = m.map(f) } implicit def readerIsAnApplicative[I] = new Applicative[({type λ[A] = Reader[I, A]})#λ] { override def pure[A](a: A) = Reader { _ => Success(a) } override def apply[A, B](mf: Reader[I, A => B], ma: Reader[I, A]) = map2(mf, ma)((f, a) => f(a)) override def map[A, B](m: Reader[I, A], f: (A) => B) = m.map(f) }
tag: step5 What are Type lambdas ? If we go back to Applicative definition, we can see that it’s an Higher Kinded type (same with Functor) : ! trait Applicative[M[_]] { … } // Applicative accept parameter M that take itself any type as parameter! ! Our problem is that Reader[I, 0] takes two parameters but Applicative[M[_]] accept types M with only one parameter. We use Type Lambdas to resolve this issue: ! new Applicative[({type λ[A] = Reader[I, A]})#λ]!
tag: step5 Go back to Reader[I, 0] object Reader {! implicit def readerIsAFunctor[I] = new Functor[({type λ[A] = Reader[I, A]})#λ] { override def fmap[A, B](m: Reader[I, A], f: A => B) = m.map(f) } implicit def readerIsAnApplicative[I] = new Applicative[({type λ[A] = Reader[I, A]})#λ] { override def pure[A](a: A) = Reader { _ => Success(a) } override def apply[A, B](mf: Reader[I, A => B], ma: Reader[I, A]) = map2(mf, ma)((f, a) => f(a)) override def map[A, B](m: Reader[I, A], f: A => B) = m.map(f) }
tag: step5 Go back to Reader[I, 0] object Reader {! import scala.language.implicitConversions // Here we help the compiler a bit. Thanks @skaalf (Julien Tournay) ! // and https://github.com/jto/validation implicit def fcbReads[I] = functionalCanBuildApplicative[({type λ[A] = Reader[I, A]})#λ] implicit def fboReads[I, A](a: Reader[I, A])(implicit fcb: FunctionalCanBuild[({type λ[x] = Reader[I, x]})#λ]) = new FunctionalBuilderOps[({type λ[x] = Reader[I, x]})#λ, A](a)(fcb)
Converter[I, T] tag: step5 trait Converter[I, T] { def convert(p: I => Result[String]): Reader[I, T] }! object Converter { implicit def stringConverter[I] = new Converter[I, String] { override def convert(p: I => Result[String]) = Reader[I, String](p) }! ! implicit def dateConverter[I](implicit dtFormat: DateFormat) = new Converter[I, Date] { override def convert(p: I => Result[String]) = Reader[I, Date] { s: I => p(s) match { case Success(dt) => try { ! Success(dtFormat.parse(dt)) } catch { case e: ParseException => Failure(s"...") } case f: Failure => f }}}
Picker[I] tag: step5 case class Picker[I](p: I => Result[String]) { def as[T](implicit c: Converter[I, T]): Reader[I, T] = c.convert(p) } object CsvPicker { def apply[T](i: Int): Picker[List[String]] = Picker { elems: List[String] => if (i > 0 && elems.size > i) Success(elems(i).trim) else Failure(s"No column ${i} found in ${elems.mkString(";")}") }} object XmlPicker { def apply[T](query: Elem => NodeSeq): Picker[Elem] = Picker { elem: Elem => try { Success(query(elem).text) } catch { case e: Exception => Failure(e.getMessage) }}}!
Usage sample tag: step5 import play.api.libs.functional.syntax._ import Reader._! import Converter._! implicit val dF = new SimpleDateFormat("dd/MM/yyyy")! val xml = XML.loadString(""" <company name="Dupont and Co"> <owner> <person firstname="jean" lastname="dupont" birthdate="11/03/1987"/> </owner> </company>""")! ! val r = ( XmlPicker(_ "person" "@firstname").as[String] and XmlPicker(_ "person" "@lastname").as[String] and XmlPicker(_ "person" "@birthdate").as[Date] )(Person)! r(xml) === Success(Person("jean", "dupont", dF.parse("11/03/1987")))
tag: step6 Adding combinators • We now add new abilities to Reader • We especially want a method to validate content
tag: step6 Adding combinators case class Reader[I, O](p: I => Result[O]) {! ! def flatMap[T](f: O => Reader[I, T]): Reader[I, T] = Reader { s: I => p(s) match { case Success(o) => f(o)(s) case f: Failure => f } } def verify(f: O => Result[O]): Reader[I, O] = flatMap { o: O => Reader( _ => f(o)) }! }
Usage sample tag: step6 val r: Reader[String, String] = Reader { Success(_) } r.verify { x => if (x == "OK") Success(x) else Failure("KO") }("OK") === Success("OK")
Conclusion • We have created a simple and powerful DSL for processing CSV and XML content • This DSL give us ability to Pick data, transform and verify it and also accumulate encountered errors • We have seen that making Reader an instance of the Applicative Functor Type Class add it new capabilities • Using ad-hoc polymorphism using Type Classes gives us ability to extends Reader without altering it
Follow-up • Type Classes are defined by functions that must be implemented with regards to their laws (left/right identity …) • Proving the correctness of Type Class Laws can be a bit tricky we usual approach • I will introduce the framework ScalaCheck at scala.io 2014, and show how to test them
Follow-up • In the roadmap that has been announced by @typesafe (http://scala-lang.org/news/roadmap-next), it seems that Scala 2.14 (aka « Don Giovanni ») will clean up lambda types syntax

More Related Content

PDF
Introducing Monads and State Monad at PSUG
David Galichet
 
PDF
Playing with State Monad
David Galichet
 
PPTX
Practical scalaz
oxbow_lakes
 
KEY
Scalaz
mpilquist
 
KEY
Deriving Scalaz
nkpart
 
PDF
Functor, Apply, Applicative And Monad
Oliver Daff
 
PDF
The Death of Final Tagless
John De Goes
 
PDF
Scalaz 8: A Whole New Game
John De Goes
 
Introducing Monads and State Monad at PSUG
David Galichet
 
Playing with State Monad
David Galichet
 
Practical scalaz
oxbow_lakes
 
Scalaz
mpilquist
 
Deriving Scalaz
nkpart
 
Functor, Apply, Applicative And Monad
Oliver Daff
 
The Death of Final Tagless
John De Goes
 
Scalaz 8: A Whole New Game
John De Goes
 

What's hot (20)

PDF
Principled Error Handling with FP
Luka Jacobowitz
 
PDF
Oh, All the things you'll traverse
Luka Jacobowitz
 
PDF
A Prelude of Purity: Scaling Back ZIO
Jorge Vásquez
 
PDF
Monad Transformers In The Wild
StackMob Inc
 
PDF
First-Class Patterns
John De Goes
 
PDF
Testing in the World of Functional Programming
Luka Jacobowitz
 
PDF
Refactoring Functional Type Classes
John De Goes
 
PDF
Side by Side - Scala and Java Adaptations of Martin Fowler’s Javascript Refac...
Philip Schwarz
 
PDF
Monoids, monoids, monoids
Luka Jacobowitz
 
PDF
Kotlin Basics - Apalon Kotlin Sprint Part 2
Kirill Rozov
 
PDF
Post-Free: Life After Free Monads
John De Goes
 
PDF
Quark: A Purely-Functional Scala DSL for Data Processing & Analytics
John De Goes
 
PDF
One Monad to Rule Them All
John De Goes
 
PDF
Why The Free Monad isn't Free
Kelley Robinson
 
PDF
Exploring type level programming in Scala
Jorge Vásquez
 
PDF
Exploring ZIO Prelude: The game changer for typeclasses in Scala
Jorge Vásquez
 
PDF
Atomically { Delete Your Actors }
John De Goes
 
PDF
Advanced Tagless Final - Saying Farewell to Free
Luka Jacobowitz
 
PDF
Blazing Fast, Pure Effects without Monads — LambdaConf 2018
John De Goes
 
PDF
Hammurabi
Mario Fusco
 
Principled Error Handling with FP
Luka Jacobowitz
 
Oh, All the things you'll traverse
Luka Jacobowitz
 
A Prelude of Purity: Scaling Back ZIO
Jorge Vásquez
 
Monad Transformers In The Wild
StackMob Inc
 
First-Class Patterns
John De Goes
 
Testing in the World of Functional Programming
Luka Jacobowitz
 
Refactoring Functional Type Classes
John De Goes
 
Side by Side - Scala and Java Adaptations of Martin Fowler’s Javascript Refac...
Philip Schwarz
 
Monoids, monoids, monoids
Luka Jacobowitz
 
Kotlin Basics - Apalon Kotlin Sprint Part 2
Kirill Rozov
 
Post-Free: Life After Free Monads
John De Goes
 
Quark: A Purely-Functional Scala DSL for Data Processing & Analytics
John De Goes
 
One Monad to Rule Them All
John De Goes
 
Why The Free Monad isn't Free
Kelley Robinson
 
Exploring type level programming in Scala
Jorge Vásquez
 
Exploring ZIO Prelude: The game changer for typeclasses in Scala
Jorge Vásquez
 
Atomically { Delete Your Actors }
John De Goes
 
Advanced Tagless Final - Saying Farewell to Free
Luka Jacobowitz
 
Blazing Fast, Pure Effects without Monads — LambdaConf 2018
John De Goes
 
Hammurabi
Mario Fusco
 
Ad

Viewers also liked (20)

PPTX
stenen aja
jonasvda
 
PPT
Eidn 6-simobe
liviacalmeida
 
PDF
The BIG Event - Canadian Mining Expo
primero_mining
 
PPT
Еремурус про екоосвіту_GreenDrinks 25.12.2012
Oleksandra Shandra
 
PPTX
Assignment 11 similar products, conventions and channel- my part
debbie14
 
PPTX
11 Telephone Phrases
Balqis Thaahaveettil
 
PPT
Fobba 2011 keynote handout
paiiceo
 
PPTX
Projects
Ashdeep Singh
 
PPTX
Planning for draft 3
debbie14
 
PDF
2012 aug 28 issue 60
Rotary Club of Blacktown City
 
PDF
10remarkableentrepreneurshipthoughts 131008125313-phpapp01
Gina Gu
 
PDF
Essays on economic analysis of competition law: theory and practice
Dr Danilo Samà
 
PDF
36kr no.94
Gina Gu
 
PPT
About iv network hawaii 2
Trevor Polischuk
 
PDF
10remarkableentrepreneurshipthoughts 131008125313-phpapp01
Gina Gu
 
PPTX
Business Model Test
Deon van Zyl
 
PDF
Corporate Presentation - BMO 2015 Global Metals & Mining Conference
primero_mining
 
PPTX
Media studies mark_scheme
debbie14
 
DOCX
Tabel analis resiko hiradc peledakan tambang
Sylvester Saragih
 
DOCX
Verb to be
Lourdes Ruales
 
stenen aja
jonasvda
 
Eidn 6-simobe
liviacalmeida
 
The BIG Event - Canadian Mining Expo
primero_mining
 
Еремурус про екоосвіту_GreenDrinks 25.12.2012
Oleksandra Shandra
 
Assignment 11 similar products, conventions and channel- my part
debbie14
 
11 Telephone Phrases
Balqis Thaahaveettil
 
Fobba 2011 keynote handout
paiiceo
 
Projects
Ashdeep Singh
 
Planning for draft 3
debbie14
 
2012 aug 28 issue 60
Rotary Club of Blacktown City
 
10remarkableentrepreneurshipthoughts 131008125313-phpapp01
Gina Gu
 
Essays on economic analysis of competition law: theory and practice
Dr Danilo Samà
 
36kr no.94
Gina Gu
 
About iv network hawaii 2
Trevor Polischuk
 
10remarkableentrepreneurshipthoughts 131008125313-phpapp01
Gina Gu
 
Business Model Test
Deon van Zyl
 
Corporate Presentation - BMO 2015 Global Metals & Mining Conference
primero_mining
 
Media studies mark_scheme
debbie14
 
Tabel analis resiko hiradc peledakan tambang
Sylvester Saragih
 
Verb to be
Lourdes Ruales
 
Ad

Similar to Writing DSL with Applicative Functors (20)

PPT
Extractors & Implicit conversions
Knoldus Inc.
 
PDF
Applicative Functor - Part 3
Philip Schwarz
 
PDF
Generic Functional Programming with Type Classes
Tapio Rautonen
 
PDF
Scala Validation with Functional Programming
Sukant Hajra
 
PDF
Scala or functional programming from a python developer's perspective
gabalese
 
PDF
TI1220 Lecture 9: Parsing & interpretation
Eelco Visser
 
PDF
여자개발자모임터 6주년 개발 세미나 - Scala Language
Ashal aka JOKER
 
PDF
Meet scala
Wojciech Pituła
 
PDF
From OOP To FP Through A Practical Case
Cristina Delgado Rodríguez
 
PDF
Scala for Java Developers
Martin Ockajak
 
PPTX
Improving Correctness with Types
Iain Hull
 
PDF
Talk - Query monad
Fabernovel
 
PDF
Scala Paradigms
Tom Flaherty
 
PPTX
Practically Functional
djspiewak
 
PDF
ScalaBlitz
Aleksandar Prokopec
 
PDF
Introduction To Scala
Innar Made
 
PDF
Practical cats
Raymond Tay
 
PDF
Fp in scala part 2
Hang Zhao
 
PPTX
Monads and friends demystified
Alessandro Lacava
 
PPT
Scala presentation by Aleksandar Prokopec
Loïc Descotte
 
Extractors & Implicit conversions
Knoldus Inc.
 
Applicative Functor - Part 3
Philip Schwarz
 
Generic Functional Programming with Type Classes
Tapio Rautonen
 
Scala Validation with Functional Programming
Sukant Hajra
 
Scala or functional programming from a python developer's perspective
gabalese
 
TI1220 Lecture 9: Parsing & interpretation
Eelco Visser
 
여자개발자모임터 6주년 개발 세미나 - Scala Language
Ashal aka JOKER
 
Meet scala
Wojciech Pituła
 
From OOP To FP Through A Practical Case
Cristina Delgado Rodríguez
 
Scala for Java Developers
Martin Ockajak
 
Improving Correctness with Types
Iain Hull
 
Talk - Query monad
Fabernovel
 
Scala Paradigms
Tom Flaherty
 
Practically Functional
djspiewak
 
ScalaBlitz
Aleksandar Prokopec
 
Introduction To Scala
Innar Made
 
Practical cats
Raymond Tay
 
Fp in scala part 2
Hang Zhao
 
Monads and friends demystified
Alessandro Lacava
 
Scala presentation by Aleksandar Prokopec
Loïc Descotte
 

Recently uploaded (20)

PPTX
Machine Learning_overview_presentation.pptx
kondavamsidharreddy2
 
PDF
Approach and Philosophy of On baking technology
30anthuong17
 
PDF
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
Libreria ERP
 
PDF
Diabetes mellitus diagnosis method based random forest with bat algorithm
IAESIJAI
 
PDF
MIND Revenue Release Quarter 2 2025 Press Release
MIND CTI
 
PPT
Teaching material agriculture food technology
LiaRayya
 
PDF
Encapsulation_ Review paper, used for researhc scholars
gurumoop
 
PDF
Machine learning based COVID-19 study performance prediction
IAESIJAI
 
PPTX
Cloud computing and distributed systems.
kkkkkhan
 
PDF
Assigned Numbers - 2025 - Bluetooth® Document
Bloombase
 
PPTX
Digital-Transformation-Roadmap-for-Companies.pptx
David Malick Dieng
 
PPTX
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
23bcla24
 
PDF
Unlocking AI with Model Context Protocol (MCP)
Brian McKeiver
 
PDF
Network Security Unit 5.pdf for BCA BBA.
Serpent6
 
PDF
Encapsulation theory and applications.pdf
gurumoop
 
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
PDF
Per capita expenditure prediction using model stacking based on satellite ima...
IAESIJAI
 
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Jian-Hong Pan
 
PPTX
Spectroscopy.pptx food analysis technology
nawahassan45
 
PPTX
Big Data Technologies - Introduction.pptx
kuthubussaman1
 
Machine Learning_overview_presentation.pptx
kondavamsidharreddy2
 
Approach and Philosophy of On baking technology
30anthuong17
 
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
Libreria ERP
 
Diabetes mellitus diagnosis method based random forest with bat algorithm
IAESIJAI
 
MIND Revenue Release Quarter 2 2025 Press Release
MIND CTI
 
Teaching material agriculture food technology
LiaRayya
 
Encapsulation_ Review paper, used for researhc scholars
gurumoop
 
Machine learning based COVID-19 study performance prediction
IAESIJAI
 
Cloud computing and distributed systems.
kkkkkhan
 
Assigned Numbers - 2025 - Bluetooth® Document
Bloombase
 
Digital-Transformation-Roadmap-for-Companies.pptx
David Malick Dieng
 
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
23bcla24
 
Unlocking AI with Model Context Protocol (MCP)
Brian McKeiver
 
Network Security Unit 5.pdf for BCA BBA.
Serpent6
 
Encapsulation theory and applications.pdf
gurumoop
 
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
Per capita expenditure prediction using model stacking based on satellite ima...
IAESIJAI
 
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Jian-Hong Pan
 
Spectroscopy.pptx food analysis technology
nawahassan45
 
Big Data Technologies - Introduction.pptx
kuthubussaman1
 

Writing DSL with Applicative Functors

  • 1. Writing DSL with Applicative Functors David Galichet Freelance functional programmer ! twitter: @dgalichet
  • 2. Content normalization • We want to parse heterogeneous data formats (CSV, XML …) and transform them to a pivot format (Scala object in our case) • The transformation should be described as a DSL • This DSL must be simple to use, and enable any kinds of data transformations or verifications
  • 3. Expected DSL format val reader = ( ! Pick(0).as[String].map(_.capitalize) and ! Pick(1).as[Date].check(_.after(now())) ! ).reduce(FutureEvent)! ! reader("PSUG; 21/08/2014") // returns Success(FutureEvent("PSUG", date)) Inspired by Play2 Json API
  • 4. tag: step1 Conventions & material • Code is available on Github : https://github.com/ dgalichet/PsugDSLWritingWithApplicative • Code revision is define on the top of any slides including source code (just checkout the specified tag)
  • 5. tag: step1 Reading single entry • We have a CSV line and we want to read one column • We will introduce several abstractions: • Picker: fetch a data from CSV or XML • Result: either a Success or a Failure • Converter: convert value from Picker to Reader • Reader: container with methods to process its content
  • 6. tag: step1 Introducing Picker case class Picker(p: String => Result[String]) {! def as[T](implicit c: Converter[T]): Reader[T] = c.convert(p)! }! ! object CsvPicker {! def apply[T](i: Int)(implicit separator: Char): Picker = Picker { s: String =>! val elems = s.trim.split(separator)! if (i > 0 && elems.size > i) Success(elems(i).trim)! else Failure(s"No column ${i} for ${s}")! }! }
  • 7. tag: step1 Introducing Picker case class Picker(p: String => Result[String]) {! def as[T](implicit c: Converter[T]): Reader[T] = c.convert(p)! }! ! object CsvPicker {! def apply[T](i: Int)(implicit separator: Char): Picker = Picker { s: String =>! val elems = s.trim.split(separator)! if (i > 0 && elems.size > i) Success(elems(i).trim)! else Failure(s"No column ${i} for ${s}")! }! } Picker wraps a function from String to Result
  • 8. The Result tag: step1 sealed trait Result[+T] case class Success[T](t: T) extends Result[T] case class Failure(error: String) extends Result[Nothing]!
  • 9. The Converter tag: step1 trait Converter[T] { def convert(p: String => Result[String]): Reader[T] }! ! object Converter {! implicit val string2StringConverter = new Converter[String] {! override def convert(p: String => Result[String]) = Reader[String] (p)! // See code on Github for more converters! }
  • 10. The Converter tag: step1 trait Converter[T] { def convert(p: String => Result[String]): Reader[T] }! ! Convert the content of the Picker to a Reader ! object Converter {! implicit val string2StringConverter = new Converter[String] {! override def convert(p: String => Result[String]) = Reader[String] (p)! // See code on Github for more converters! }
  • 11. The Reader tag: step1 case class Reader[O](p: String => Result[O]) { def apply(s: String): Result[O] = p(s) } A Reader doesn’t contain a value but a process to transform original data (CSV line or XML) to a Result
  • 12. Usage sample tag: step1 import Converter._ // import implicit converters! implicit val separator = ‘;’! ! CsvPicker(1).as[String].apply("foo;bar") === "bar"
  • 13. tag: step2 Enhancing the Reader • The first defines a very simple Reader. We must add a method to combine two instances of Reader • We will also enhance Failure to store multiple error messages
  • 14. tag: step2 Enhancing the Reader case class Reader[O](p: String => Result[O]) {! def apply(s: String): Result[O] = p(s)! ! def and[O2](r2: Reader[O2]): Reader[(O, O2)] = Reader { s: String =>! (p(s), r2.p(s)) match {! case (Success(s1), Success(s2)) => Success((s1, s2))! case (Success(_), Failure(f)) => Failure(f)! case (Failure(f), Success(_)) => Failure(f)! case (Failure(f1), Failure(f2)) => Failure(f1 ++ f2)! }! }! def map[T](f: O => T): Reader[T] = Reader { s: String =>! p(s) match {! case Success(o) => Success(f(o))! case f: Failure => f! }! }! def reduce[T] = map[T] _ // alias for map! }
  • 15. tag: step2 Enhancing Result type sealed trait Result[+T]! case class Success[T](t: T) extends Result[T]! case class Failure(error: NonEmptyList[String]) extends Result[Nothing]! ! object Failure {! def apply(s: String): Failure = Failure(NEL(s))! }! ! case class NonEmptyList[T](head: T, tail: List[T]) { def toList = head::tail def ++(l2: NonEmptyList[T]): NonEmptyList[T] = NonEmptyList(head, tail ++ l2.toList) } object NEL { def apply[T](h: T, t: T*) = NonEmptyList(h, t.toList) }
  • 16. Usage sample tag: step2 implicit val separator = ';' implicit val dtFormatter = new SimpleDateFormat("dd/MM/yyyy") import Converter.string2StringConverter import Converter.string2DateConverter! ! val reader = ( CsvPicker(1).as[String] and CsvPicker(2).as[Date] ).reduce { case (n, d) => FutureEvent(n, d) }! reader("foo;bar;12/10/2014") === Success(FutureEvent("bar", dtFormatter.parse("12/10/2014")))! ! case class FutureEvent(name: String, dt: Date)
  • 17. tag: step2 Usability problem • The use of reduce (or map) method to transform a Reader[(0, 02)] into an instance of Reader[FutureEvent] for example is quite verbose • This will be even more verbose for instances of Reader[(0, (02, 03))] • We want the API to automatically bind tuple elements to a constructor as we can encounter in Play2 Json API
  • 18. tag: step3 Applicative functors • To tackle our problem, we will use Applicative Functors and play2 functional library (and especially FunctionalBuilder) • This approach is inspired by @sadache (Sadek Drobi) article https://gist.github.com/sadache/ 3646092 • An Applicative Functor is a Type Class relying on ad-hoc polymorphism to extends a Class with some properties • Play2 functional library (or Scalaz) provides mechanism to compose Applicatives in a smart way
  • 19. tag: step3 Applicative functors M is an Applicative Functor if there exists the following methods : def pure[A](a: A): M[A] def map[A, B](m: M[A], f: A => B): M[B] def apply[A, B](mf: M[A => B], ma: M[A]): M[B]! with the following Laws : • Identity: apply(pure(identity), ma) === ma where ma is an Applicative M[A] • Homomorphism: apply(pure(f), pure(a)) === pure(f(a)) where f: A => B and a an instance of A • Interchange: mf if an instance of M[A => B] apply(mf, pure(a)) === apply(pure {(g: A => B) => g(a)}, mf)! • Composition: map(ma, f) === apply(pure(f), ma)
  • 20. tag: step3 Applicative functors trait Applicative[M[_]] { def pure[A](a: A): M[A] def map[A, B](m: M[A], f: A => B): M[B] def apply[A, B](mf: M[A => B], ma: M[A]): M[B] } Applicative is an Higher Kinded type (parameterized with M that take a single type parameter)
  • 21. tag: step3 Reader is an Applicative case class Reader[O](p: String => Result[O]) { def apply(s: String): Result[O] = p(s)! def map[T](f: O => T): Reader[T] = Reader { s: String => p(s) match { case Success(o) => Success(f(o)) case f: Failure => f } }! }! object Reader { def map2[O, O1, O2](r1: Reader[O1], r2: Reader[O2])(f: (O1, O2) => O): Reader[O] = Reader { s: String => (r1.p(s), r2.p(s)) match { case (Success(s1), Success(s2)) => Success(f(s1, s2)) case (Success(_), Failure(e)) => Failure(e) case (Failure(e), Success(_)) => Failure(e) case (Failure(e1), Failure(e2)) => Failure(e1 ++ e2) } } …! }
  • 22. tag: step3 Reader is an Applicative object Reader { … // map2 implicit val readerIsAFunctor: Functor[Reader] = new Functor[Reader] { override def fmap[A, B](m: Reader[A], f: (A) => B) = m.map(f) } implicit val readerIsAnApplicative: Applicative[Reader] = new Applicative[Reader] { override def pure[A](a: A) = Reader { _ => Success(a) } override def apply[A, B](mf: Reader[A => B], ma: Reader[A]) = map2(mf, ma)((f, a) => f(a)) override def map[A, B](m: Reader[A], f: A => B) = m.map(f) } }
  • 23. Usage sample tag: step3 import Converter.string2StringConverter import Converter.string2DateConverter! import play.api.libs.functional.syntax._ import Reader.readerIsAnApplicative! ! implicit val separator = ';' implicit val dtFormatter = new SimpleDateFormat("dd/MM/yyyy")! ! val reader = ( CsvPicker(1).as[String] and CsvPicker(2).as[Date] )(FutureEvent) // here we use CanBuild2.apply reader("foo;bar;12/10/2014") === Success(FutureEvent("bar", dtFormatter.parse("12/10/2014")))!
  • 24. Usage sample tag: step3 (errors accumulation) import Converter.string2StringConverter import Converter.string2DateConverter! import play.api.libs.functional.syntax._ import Reader.readerIsAnApplicative! ! implicit val separator = ';' implicit val dtFormatter = new SimpleDateFormat("dd/MM/yyyy")! ! val reader = ( CsvPicker(1).as[Int] and CsvPicker(2).as[Date] )((_, _)) reader(List("foo", "not a number", "not a date")) === Failure(NEL(! "Unable to format 'not a number' as Int", ! "Unable to format 'not a date' as Date"))!
  • 25. Benefits tag: step3 • Making Reader an Applicative Functor give ability to combine efficiently instances of Reader • Due to Applicative properties, we still accumulate errors • Play2 functional builder give us a clean syntax to define our DSL
  • 26. tag: step4 Introducing XML Picker case class Picker(p: String => Result[String]) { def as[T](implicit c: Converter[T]): Reader[T] = c.convert(p) }! ! object XmlPicker { def apply[T](query: Elem => NodeSeq): Picker = Picker { s: String => try { val xml = XML.loadString(s) Success(query(xml).text) } catch { case e: Exception => Failure(e.getMessage) } } }!
  • 27. Usage sample tag: step4 import play.api.libs.functional.syntax._ import Reader.readerIsAnApplicative! import Converter._ implicit val dF = new SimpleDateFormat("dd/MM/yyyy")! val xml = """ <company name="Dupont and Co"> <owner> <person firstname="jean" lastname="dupont" birthdate="11/03/1987"/> </owner> </company>""" val r = ( XmlPicker(_ "person" "@firstname").as[String] and XmlPicker(_ "person" "@lastname").as[String] and XmlPicker(_ "person" "@birthdate").as[Date] )(Person)! r(xml) === Success(Person("jean","dupont",dF.parse("11/03/1987"))) case class Person(firstname: String, lastname: String, birthDt: Date)
  • 28. tag: step4 Implementation problem • The Reader[O] takes a type argument for the output. The input is always a String • With this implementation, an XML content will be parsed (with XML.load) as many times as we use XmlPicker. This will cause unnecessary overhead • We will have the same issue (with lower overhead) with our CsvPicker
  • 29. tag: step5 Introducing Reader[I, 0] To resolve this problem, we will modify Reader to take a type parameter for the input
  • 30. tag: step5 Introducing Reader[I, 0] case class Reader[I, O](p: I => Result[O]) { def apply(s: I): Result[O] = p(s) def map[T](f: O => T): Reader[I, T] = Reader { s: I => p(s) match { case Success(o) => Success(f(o)) case f: Failure => f } }! }! object Reader { def map2[I, O, O1, O2](r1: Reader[I, O1], r2: Reader[I, O2])(f: (O1, O2) => O): Reader[I, O] = Reader { s: I => (r1.p(s), r2.p(s)) match { case (Success(s1), Success(s2)) => Success(f(s1, s2)) case (Success(_), Failure(e)) => Failure(e) case (Failure(e), Success(_)) => Failure(e) case (Failure(e1), Failure(e2)) => Failure(e1 ++ e2) } }
  • 31. tag: step5 Introducing Reader[I, 0] object Reader {! implicit def readerIsAFunctor[I] = new Functor[({type λ[A] = Reader[I, A]})#λ] { override def fmap[A, B](m: Reader[I, A], f: (A) => B) = m.map(f) } implicit def readerIsAnApplicative[I] = new Applicative[({type λ[A] = Reader[I, A]})#λ] { override def pure[A](a: A) = Reader { _ => Success(a) } override def apply[A, B](mf: Reader[I, A => B], ma: Reader[I, A]) = map2(mf, ma)((f, a) => f(a)) override def map[A, B](m: Reader[I, A], f: (A) => B) = m.map(f) }
  • 32. tag: step5 What are Type lambdas ? If we go back to Applicative definition, we can see that it’s an Higher Kinded type (same with Functor) : ! trait Applicative[M[_]] { … } // Applicative accept parameter M that take itself any type as parameter! ! Our problem is that Reader[I, 0] takes two parameters but Applicative[M[_]] accept types M with only one parameter. We use Type Lambdas to resolve this issue: ! new Applicative[({type λ[A] = Reader[I, A]})#λ]!
  • 33. tag: step5 Go back to Reader[I, 0] object Reader {! implicit def readerIsAFunctor[I] = new Functor[({type λ[A] = Reader[I, A]})#λ] { override def fmap[A, B](m: Reader[I, A], f: A => B) = m.map(f) } implicit def readerIsAnApplicative[I] = new Applicative[({type λ[A] = Reader[I, A]})#λ] { override def pure[A](a: A) = Reader { _ => Success(a) } override def apply[A, B](mf: Reader[I, A => B], ma: Reader[I, A]) = map2(mf, ma)((f, a) => f(a)) override def map[A, B](m: Reader[I, A], f: A => B) = m.map(f) }
  • 34. tag: step5 Go back to Reader[I, 0] object Reader {! import scala.language.implicitConversions // Here we help the compiler a bit. Thanks @skaalf (Julien Tournay) ! // and https://github.com/jto/validation implicit def fcbReads[I] = functionalCanBuildApplicative[({type λ[A] = Reader[I, A]})#λ] implicit def fboReads[I, A](a: Reader[I, A])(implicit fcb: FunctionalCanBuild[({type λ[x] = Reader[I, x]})#λ]) = new FunctionalBuilderOps[({type λ[x] = Reader[I, x]})#λ, A](a)(fcb)
  • 35. Converter[I, T] tag: step5 trait Converter[I, T] { def convert(p: I => Result[String]): Reader[I, T] }! object Converter { implicit def stringConverter[I] = new Converter[I, String] { override def convert(p: I => Result[String]) = Reader[I, String](p) }! ! implicit def dateConverter[I](implicit dtFormat: DateFormat) = new Converter[I, Date] { override def convert(p: I => Result[String]) = Reader[I, Date] { s: I => p(s) match { case Success(dt) => try { ! Success(dtFormat.parse(dt)) } catch { case e: ParseException => Failure(s"...") } case f: Failure => f }}}
  • 36. Picker[I] tag: step5 case class Picker[I](p: I => Result[String]) { def as[T](implicit c: Converter[I, T]): Reader[I, T] = c.convert(p) } object CsvPicker { def apply[T](i: Int): Picker[List[String]] = Picker { elems: List[String] => if (i > 0 && elems.size > i) Success(elems(i).trim) else Failure(s"No column ${i} found in ${elems.mkString(";")}") }} object XmlPicker { def apply[T](query: Elem => NodeSeq): Picker[Elem] = Picker { elem: Elem => try { Success(query(elem).text) } catch { case e: Exception => Failure(e.getMessage) }}}!
  • 37. Usage sample tag: step5 import play.api.libs.functional.syntax._ import Reader._! import Converter._! implicit val dF = new SimpleDateFormat("dd/MM/yyyy")! val xml = XML.loadString(""" <company name="Dupont and Co"> <owner> <person firstname="jean" lastname="dupont" birthdate="11/03/1987"/> </owner> </company>""")! ! val r = ( XmlPicker(_ "person" "@firstname").as[String] and XmlPicker(_ "person" "@lastname").as[String] and XmlPicker(_ "person" "@birthdate").as[Date] )(Person)! r(xml) === Success(Person("jean", "dupont", dF.parse("11/03/1987")))
  • 38. tag: step6 Adding combinators • We now add new abilities to Reader • We especially want a method to validate content
  • 39. tag: step6 Adding combinators case class Reader[I, O](p: I => Result[O]) {! ! def flatMap[T](f: O => Reader[I, T]): Reader[I, T] = Reader { s: I => p(s) match { case Success(o) => f(o)(s) case f: Failure => f } } def verify(f: O => Result[O]): Reader[I, O] = flatMap { o: O => Reader( _ => f(o)) }! }
  • 40. Usage sample tag: step6 val r: Reader[String, String] = Reader { Success(_) } r.verify { x => if (x == "OK") Success(x) else Failure("KO") }("OK") === Success("OK")
  • 41. Conclusion • We have created a simple and powerful DSL for processing CSV and XML content • This DSL give us ability to Pick data, transform and verify it and also accumulate encountered errors • We have seen that making Reader an instance of the Applicative Functor Type Class add it new capabilities • Using ad-hoc polymorphism using Type Classes gives us ability to extends Reader without altering it
  • 42. Follow-up • Type Classes are defined by functions that must be implemented with regards to their laws (left/right identity …) • Proving the correctness of Type Class Laws can be a bit tricky we usual approach • I will introduce the framework ScalaCheck at scala.io 2014, and show how to test them
  • 43. Follow-up • In the roadmap that has been announced by @typesafe (http://scala-lang.org/news/roadmap-next), it seems that Scala 2.14 (aka « Don Giovanni ») will clean up lambda types syntax