Let's refine your Scala Code

RefineyourScalaCode
TechTriveni2.0|NewDelhi
AjayViswanathan

Aboutme
Sr.SoftwareEngineer@Agoda
MyFPjourneyhastakenmethroughBigData,BackEndandFrontEnd
LearningFPandScalaforthepast5years
Reachouttomeonmyblogscala.ninja
ThisPresentationwaswrittenin markdown andcompiledusing marp-cli
Somecodeexamplesmayrequireyoutouse Scala 2.13 or Dotty

Motivationforthistalk
InmyexperienceofworkingwithteamsbuildingproductsinScala,my#1gripehasalways
been
Ifyouwantedtowriteitthatway,whydon'tyoudoitinJava?WhyevenuseScala?
Instead,Iwouldratherwriteitusing<insertXYZpattern/library>
Justlikeyoucanhavemultiplesolutionstoamathproblem,buttheelegantsolutionsare
sparse
Similarly,justidiomaticScalacodeisnotenough,especiallyifyoudon'tutilizethefull
potentialofwhatisavailable

Controversialstatementalert!
ScalaislikeaLiberalOligarchy
Liberal-becauseitacceptsawidevarietyofcodingstyleswithoutmuchcomplaint
Oligarchy-becausethecorelanguageisinthehandsoffewpeopleandcannotcope
upwiththeadvancementsallaround

Typelevel101
BeforeIheadintothemaintopicofthispresentation,allowmetointroduceafew
terminologiesthatwouldcomeinhandyinkeepingupwithsomeoftheconcepts.
Scalaispoweredbyanunder-appreciatedtype-system
WithoutgettingjumbledinwordslikeSKICombinatorCalculus,letmesaythatScala's
typesystemisTuringComplete
Thisfactcomesinhandywhenthecompilerhastomakeinferencesaboutyour
untypedcode.
ThereisnothingmagicalaboutworkingwithTypes.Itisbasedoninductivelogic
Imakenopretenseofanymathematicalformalityinthefollowingoversimplifications
I'mabouttoexplain

AstaticallytypedlanguagelikeScalahastwodomains-atypedomain,andavalue
domain.
case class Name(firstName: String, lastName: String)
val n1 = Name("ajay", "viswanathan")
// Name -> Type
// n1 -> Value
Likeclasses,typescanalsohaveconstructors
type IntMap[T] = Map[Int, T]

Thekeyconcepttonoteis
TypescanbereasonedaboutatCompileTime,whereasValuescanonlybe
reasonedaboutRuntime
Whatthisimpliesis
TypelevelprogrammingallowsustocatcherrorsatCompileTimewhichwould
otherwisebedealtwithatRuntime

DesignPattern#1:PhantomTypes
Anabstracttypethatisneverinitialized,hencehasnoeffectonruntime
Itisonlyusedtoprovestaticpropertiesusingtypeevidences
Thecompilererasestheseonceitcanprovethattheconstraintshold
sealed trait Status
sealed trait Red extends Status
sealed trait Orange extends Status
sealed trait Green extends Status
class TrafficSignal[T <: Status] {
private def to[U <: Status]: TrafficSignal[U] = this.asInstanceOf[TrafficSignal[U]]
def stop(implicit ev: T =:= Orange): TrafficSignal[Red] = to[Red]
def start(implicit ev: T =:= Red): TrafficSignal[Orange] = to[Orange]
def go(implicit ev: T =:= Orange): TrafficSignal[Green] = to[Green]
def slow(implicit ev: T =:= Green): TrafficSignal[Orange] = to[Orange]
}
val signal = new TrafficSignal[Red]
signal.stop // Compilation Error: Cannot prove that TrafficLight.Red =:= TrafficLight.Orange
signal.start.go // Compilation Successful

DesignPattern#2:Pathdependenttypes
Scalaallowsyoutodefinetypesinsideatype
Thisisusefulwhenonlypartialtypeinformationisavailableatcompiletime
Theruntimetypecanbeencapsulated
case class Cart(user: String, value: Int = 0) {
case class Item(name: String, amount: Int)
def add(item: this.Item): Cart = copy(value = value + item.amount)
}
val cart = Cart("suresh")
val item: cart.Item = cart.Item("perk", 2)
val valid = cart.add(item) // Cart(suresh, 2)
val invalid = Cart("ramesh").add(item) // Compilation Error

DesignPattern#2PathDependentTypesRedux:AUXpattern
Wedon'talwayshavetheluxuryofknowingcertaindatatypesatcompiletime
Whenwewantafunctiontodependentonaruntimetype,wecanuseTypeclassesto
workaroundthisproblem
Saythereexistsafunctionthatlooksforatypeclassbasedontheruntimevalue
trait Param[T, U] {
def convert(obj: T): Option[U]
}
trait Predicate[U] {
def eval(values: List[U], value: U): Boolean
}
def evaluate[T, U](request: T, values: List[U])(implicit param: Param[T, U], predicate: Predicate[U]): Boolean = {
param.convert(request).exists(v => predicate.eval(values, v))
}

implicit val stringPredicate = new Predicate[String] {
def eval(values: List[String], value: String) = values.contains(value)
}
implicit val intPredicate = new Predicate[Int] {
def eval(values: List[Int], value: Int) = values.forall(_ > value)
}
implicit val stringIntParam = new Param[String, Int] {
def convert(obj: String): Option[Int] = Try(obj.toInt).toOption
}
...
evaluate("3", List(2, 3, 4)) // false
evaluate("3", List(4, 5, 6)) // true

WecouldrewritethesameusingPathdependenttypesas
trait Param[T] {
type U
def convert(obj: T): Option[U]
}
def evaluate[T, U](request: T, values: List[U])(implicit param: Param[T], predicate: Predicate[param.U]) // Does not compile
Weintroducethe Aux patternheretohelpthecompilerreasonaboutthetypeinference
object Param {
type Aux[T0, U0] = Param[T0] {
type U = U0
}
}
def evaluate[T, U](request: T, values: List[U])(implicit ev: Param.Aux[T, U], predicate: Predicate[U]): Boolean = {
ev.convert(request).exists(v => predicate.eval(values, v))
}

DesignPattern#3:Singletontypes
EverySingletonobjectinScalaisitsowntype
Singletontypesbridgethegapbetweenthevaluelevelandthetypelevel
object Data {
val v1: String = "data"
}
object FakeData {
val v1: String = "virus"
}
def process(obj: Data.type) = println(obj.v1) // data

DesignPattern#3(Experimental):Literal-BasedSingletonTypes
SIP23-ProposaltointegratewithScala3
WillallowLiteralstoappearintype-position
val one: 1 = 1
Sampleuse-case:MatrixMultiplication
type Dim = Singleton & Int
case class Matrix[A <: Dim, B <: Dim](a: A, b: B) {
def *[C <: Dim](other: Matrix[B, C]): Matrix[A, C] = Matrix(a, other.b)
}

DesigningyourDataTypes
Problemswith Stringly definedcode
Pronetobadinputsifnoterror-checked
Havetohandleplentyofmeaninglessinputsasedge-cases
Muchofenterprisecodeendsuprelyingontrustofdeveloperthattheywillnotbreak
it
Solution:Makebadinputimpossibletoconstruct
Whyshouldwewritecodethathandleswhatitshouldn'tbehandling?
Itonlyleadstomoretests,morestupidtests
WhataretheissuesyoumayfacewithsuchaDatadomain?
case class Timestamp(date: String, hour: Int)

case class Timestamp(date: String, hour: Int)
Somanyquestions:
Whatisthedateformatexpected?
Areallformatshandled?
Day/Month/Year???
Hourin12/24hformat?
Houris0-indexed?

Let'strytofixthis
// With Self-checks
def verifyDate(date: String, format: String = "yyyyMMdd"): Boolean
case class Timestamp(date: String, hour: Int) {
require(hour > 0 && hour <= 24)
require(verifyDate(date))
}
// With Structural Enforcement
def apply(year: Int, month: Int, day: Int, hour: Int) =
Timestamp(s"$year-$month-$day", hour)

OpaquevsTransparentDesign
ThefirstconsiderationwhiledesigningaDataTypeisthelevelofflexibilityor
encapsulationyouwishtoprovide
Inotherwords,how opaque or transparent doyouwantyourDataTypetobe?
// opaque DataType
class ParseError(index: Int, input: String) {
def line(): Int
def position(): Int
def message(): String
}
// transparent DataType
case class ParseError(line: Int, position: Int, message: String)

OpacityandTransparencyisaSpectrum
Opacityenforcesinvariants
ParseError(-1,-3,"randomlog")vsParserError.message="[ERROR]Message"
OpacitycansaveonDefensiveness
Transparentdesigncanenforceinvariantsbutbyaddingassertsandotherchecks
usingSelf-checksorStructuralEnforcements
Transparencyreducescomplexity
Istheopaquedatacomputingatinitialization?recomputing?caching?Howmuch
memorydoesituseinstoringinput?

Exampleofbaddatadomain
case class Request(guid: String)

Approach#1:Usingtypealias
type GUID = String
case class Request(guid: GUID)
Doesnothingmorethanmakingiteasiertoread

Approach2:Usinga(value)class
case class GUID(value: String)
Compilercancheckfortherightinstance,butstillnothingintermsofsafetyofinput
Boxingoverhead
class GUID(val value: String) extends AnyVal
Stilldoesn'tsolvethevalidationproblem
Valueclassescanstillgetinitializedifusedincollections,methodsignaturesetc

Approach3:UsingSmartConstructors
case class GUID private (value: String)
object GUID {
def apply(value: String): Option[GUID] = if (value.startsWith("valid")) Some(new GUID(value)) else None
}
new GUID("valid") // Does not compile - Good!
GUID("valid") // GUID(valid) - Validations work - Great!!
GUID("valid").map(_.copy("invalid")) // GUID(invalid) - Copy constructor is vulnerable - Bad!!!
Bythistime,we'vealreadyintroducedimmensecomplexitytoasimpleDataType
ReturntypeofOptionisabitfrustratingtohandlefordevelopers

TheNarrow-Widenapproach
String => GUID notallstringsarevalid
String => Option[GUID] widentheoutputtoaccommodate
List[Char] => Option[GUID]
NonEmptyList[Char] => Option[GUID] slightrefinementtotheinput
NonEmptyList[HexDigit] => Option[GUID] furtherrefininingtheinput
ThirtyTwoList[HexDigit] => Option[GUID] => GUID totalrefinementofthe
input,plusun-wideningoftheoutput
Whenyouwanttonarrowtheinput,youendupWideningtheoutput,butifyounarrow
theinputenough,youcanun-widentheoutput

Approach#4:Refiningtheconstructor
Arefinementtypeisbasically Base Type + Predicate
Subtypingofrefinements
type T + P <: T
type T + P <: type T + Q if forall t in T + P, Q(t) is true
Eg.
A - Set of Int > 10
B - Set of Int > 5
A <: B

type HexDigit = Char Refined LetterOrDigit
type ThirtyTwoList[T] = List[T] Refined Size[32]
type GUID = ThirtyTwoList[HexDigit]
case class Request(guid: GUID)
val r1 = Request("2312k3j123...123dasd") // Compiles
val r2 = Request("asdas_asda_21#!@##$@#$...2234") // Does not compile
// At runtime
refineV[GUID]("asdasd2323...12312asd") // Either[String, GUID]
Typeserasedatcompiletimeleavingyouwithonlyprimitives
Atruntime,predicatescomputedonvaluesaswouldbethecasenormally

YoucandomuchmorewithRefined
Thelibrarycomeswiththesepredefinedpredicates
Boolean
True: constant predicate that is always true
False: constant predicate that is always false
Not[P]: negation of the predicate P
And[A, B]: conjunction of the predicates A and B
Or[A, B]: disjunction of the predicates A and B
Xor[A, B]: exclusive disjunction of the predicates A and B
Nand[A, B]: negated conjunction of the predicates A and B
Nor[A, B]: negated disjunction of the predicates A and B
AllOf[PS]: conjunction of all predicates in PS
AnyOf[PS]: disjunction of all predicates in PS
OneOf[PS]: exclusive disjunction of all predicates in PS

Char
Digit: checks if a Char is a digit
Letter: checks if a Char is a letter
LetterOrDigit: checks if a Char is a letter or digit
LowerCase: checks if a Char is a lower case character
UpperCase: checks if a Char is an upper case character
Whitespace: checks if a Char is white space

Collection
Contains[U]: checks if a Traversable contains a value equal to U
Count[PA, PC]: counts the number of elements in a Traversable which satisfy the predicate PA and passes the result to the predicate PC
Empty: checks if a Traversable is empty
NonEmpty: checks if a Traversable is not empty
Forall[P]: checks if the predicate P holds for all elements of a Traversable
Exists[P]: checks if the predicate P holds for some elements of a Traversable
Head[P]: checks if the predicate P holds for the first element of a Traversable
Index[N, P]: checks if the predicate P holds for the element at index N of a sequence
Init[P]: checks if the predicate P holds for all but the last element of a Traversable
Last[P]: checks if the predicate P holds for the last element of a Traversable
Tail[P]: checks if the predicate P holds for all but the first element of a Traversable
Size[P]: checks if the size of a Traversable satisfies the predicate P
MinSize[N]: checks if the size of a Traversable is greater than or equal to N
MaxSize[N]: checks if the size of a Traversable is less than or equal to N

Generic
Equal[U]: checks if a value is equal to U

Numeric
Less[N]: checks if a numeric value is less than N
LessEqual[N]: checks if a numeric value is less than or equal to N
Greater[N]: checks if a numeric value is greater than N
GreaterEqual[N]: checks if a numeric value is greater than or equal to N
Positive: checks if a numeric value is greater than zero
NonPositive: checks if a numeric value is zero or negative
Negative: checks if a numeric value is less than zero
NonNegative: checks if a numeric value is zero or positive
Interval.Open[L, H]: checks if a numeric value is in the interval (L, H)
Interval.OpenClosed[L, H]: checks if a numeric value is in the interval (L, H]
Interval.ClosedOpen[L, H]: checks if a numeric value is in the interval [L, H)
Interval.Closed[L, H]: checks if a numeric value is in the interval [L, H]
Modulo[N, O]: checks if an integral value modulo N is O
Divisible[N]: checks if an integral value is evenly divisible by N
NonDivisible[N]: checks if an integral value is not evenly divisible by N
Even: checks if an integral value is evenly divisible by 2
Odd: checks if an integral value is not evenly divisible by 2
NonNaN: checks if a floating-point number is not NaN

String
EndsWith[S]: checks if a String ends with the suffix S
IPv4: checks if a String is a valid IPv4
IPv6: checks if a String is a valid IPv6
MatchesRegex[S]: checks if a String matches the regular expression S
Regex: checks if a String is a valid regular expression
StartsWith[S]: checks if a String starts with the prefix S
Uri: checks if a String is a valid URI
Url: checks if a String is a valid URL
Uuid: checks if a String is a valid UUID
ValidByte: checks if a String is a parsable Byte
ValidShort: checks if a String is a parsable Short
ValidInt: checks if a String is a parsable Int
ValidLong: checks if a String is a parsable Long
ValidFloat: checks if a String is a parsable Float
ValidDouble: checks if a String is a parsable Double
ValidBigInt: checks if a String is a parsable BigInt
ValidBigDecimal: checks if a String is a parsable BigDecimal
Xml: checks if a String is well-formed XML
XPath: checks if a String is a valid XPath expression
Trimmed: checks if a String has no leading or trailing whitespace
HexStringSpec: checks if a String represents a hexadecimal number

TaggingTypes
Letstakeanothercommonexamplewhereaspecifictypeinadomaincanbecritical
def getWeight(): Double
def getHeight(): Double
def bmi(weight: Double, height: Double): Double // Let's assume SI units as input
bmi(getWeight(), getHeight()) // OK
bmi(getHeight(), getWeight()) // Still OK, but not ideal behavior

Besidesrefining,taggingtypesofferalow-costalternativetoValueclasses
type Tagged[U] = {
type Tag = U
}
type @@[T, U] = T with Tagged[U]
implicit class TaggedOps[T](val v: T) extends AnyVal {
@inline def taggedWith[U]: T @@ U = v.asInstanceOf[T @@ U]
@inline def @@[U]: T @@ U = taggedWith[U]
}

trait Kilogram
trait Metres
type Weight = Double @@ Kilogram
type Height = Double @@ Metres
def getWeight(): Weight
def getHeight(): Height
// Units are already encoded in the type now
def bmi(weight: Weight, height: Height): Double
bmi(getWeight(), getHeight()) // OK
bmi(getHeight(), getWeight()) // Compilation error, just like we wanted
Noruntimecostoftypetags
MinoroverheadofTaggedOps
Coderemainssaneandself-documenting
Scalaintroducing NewType and Opaque and Transparent typestoScalasoon
enough,whereTaggingandZero-overheadencapsulatedconstructorswouldbecome
partofthestandardlibrary

UseCase:LoadingConfigs
Whyconfigs?BecausewesanitizeeverythingelselikeDB,Userinput,butnever
configvalues
Dangeroustoblindlytrustyourdevelopers.Theyarehumansandmistakescanbe
madewhichcanbehardtofindandcostly
Validationofconfigshouldbemorethan value is present ,butthat value is
usable
case class Endpoint(host: String, port: Int)
// based on what we've learnt so far, let's rewrite that as
type Host = MatchesRegex["^[A-Z0-9-_]{1,255}$"]
type Port = Interval.Closed[1024, 65535]
case class Endpoint(host: String Refined Host, port: Int Refined Port)

Experimental:Addingside-effectstoRefinementtypes
case class OpenPort()
implicit val openPortValidate: Validate.Plain[Int, OpenPort] =
Validate.fromPartial(new ServerSocket(_).close(), "OpenPort", OpenPort())
type AvailablePort = Int Refined OpenPort

RefinedTypesforthewin
Typesafety++
Uniformvalidation
Compiletimerefinementofliteralvalues
Runtimesafeextractionofvalues
Self-documentedcodeandeasiertoreasonaboutanddebugerrors
Fewertestcasestowrite
Pushprimitivestotheedge-of-the-system

LimitationsofRefinementTypes
IntelliJsupportsucks
AvoidInfixnotations
String Refined XXX And YYY
String Refined And[XXX, YYY]
String Refined (XXX And YYY)
Refinedprimitivesarealwaysboxed,asisusualwithnormalScala
Validationerrorsnotclearalways
Increasedcompiletimes
Scalasupportstillnotverymature(Mightchangeafterv3)

References
https://github.com/fthomas/refined
https://slideslive.com/38908213/strings-are-evil-methods-to-hide-the-use-of-primitive-
types
http://www.lihaoyi.com/post/StrategicScalaStyleDesigningDatatypes.html
https://slideslive.com/38907881/literal-types-what-they-are-good-for
https://kwark.github.io/refined-in-practice-bescala/#94
https://meta.plasm.us/slides/circe/tour/#36
https://beyondthelines.net/programming/refined-types/
http://leifwickland.github.io/presentations/configBomb/#67
https://github.com/wjlow/blog/blob/3c27de716b40660801e68561252883fd0428395e
/Tests.md
https://gigiigig.github.io/posts/2015/09/13/aux-pattern.html
https://docs.scala-lang.org/sips/42.type.html

Let's refine your Scala Code

More Related Content

Similar to Let's refine your Scala Code (20)

More from Tech Triveni (20)

Recently uploaded (20)

Let's refine your Scala Code