Bootstrapping a Scala Mindset (Scala eXchange 2014)

Editor's Notes

• #3: “Scala is a Cliff” – that’s my quote – no-one other than me said it, but Chad Fowler got pretty close: "Scala is difficult to learn“ he wrote in his forward to Scala in Action, from 2013 “Scala is a lot to chew on. It’s got what seems way too many features.”
• #4: He continued: “You’re going to hurt yourself along the way” “[However] it’s these jagged edges that improve us. If you approach this climb properly, you’ll reach the top sharper, more open-minded, and, best of all, less afraid.“ I wish I’d paid more attention to this when I started learning Scala. This presentation pretty much deals with how I learned this the hard way.
• #5: * Senior Java Dev / Dev Lead * 5 or so years ago I “Learned” Ruby (ruby-eye-for-the-java-guy.blogspot.com) to get some DynaLang in my life (and done some Groovy-fiddling) * For the past few years I’ve been Learning Scala to get some Functional in my life As well as this I: * Stopped studying Maths at High School (Age 16 yrs) * And at university too the easy option and studied Psychology at University (fluffy!)
• #6: So given that, and assuming you’re not totally put off, whats my aim here? By telling you things I learned, I can reduce your pain (N.b. It’ll still hurt) How? * By mapping out the key stages involved in my moving from a Java to the beginnings of a Scala mindset * And by telling you some things few folks remember to tell you
• #7: And what do you get if you follow this path? Rather facetiously I’m gonna claim that “soon you’ll be using Monadic constructs without even feeling special.” But what this really means is you’ll have a better foundation mental model of the Scala language Which will allow you launch into the next challenges the language presents as you encounter more and more of its richness.
• #8: So are we ready? Lets go. Part 1: Everything you know is wrong – in which we learn just how far Andrew had to go when he started
• #9: First up, confession time. I was so clueless about Scala when I started I even had a misconception around the name: IS: * Scales nicely - Play and Reactive Programming * Parallelises well - Immutability & Actors * More productive - High Level so much less boiler plate * More team members - Statically Typed -> Refactorable BUT that's not where the name comes from…
• #10: In reality, it turns out that Scala is so named because it “grows with the demands of its users” (Programming in Scala 2nd Ed.) It’s layered – “Scala is much more like a bazaar than a cathedral” (Programming in Scala 2nd Ed.) It’s designed to be extended and adapted, in that it’s Scala all the way down, via libraries that nevertheless feel native. Consequently, you can apply it to a wide range of tasks. But you can also (must) bootstrap your way in. What do I mean by all this? Patience patience, we’ll soon see. But first, more misconceptions.
• #11: The next thing to bear in mind is that, whether you Like it or not: There Will Be Math(s)™ That's fine. But what you need to consider is that this maths is EVERYWHERE: * It inspired the much terser, and mathematical-idiom driven syntax * It means things are in the “wrong place”, such as something as fundamental as the parameter types and return type on method signatures - because they're in the "right place" for mathematical notation * and they expect operator overloading (i.e. more “weird looking” code) * And maths folks use a lot of chaining – so you’ll see a lot of that In short, you're stumbling into an entirely different world And that’s the first reason why I claim “everything you know is wrong” – everything I knew certainly was. Or it felt like that at one point.
• #12: There’s a reason for all this maths – Scala is Functional and not Imperative. And functional == maths I need to hammer this home (I wish someone had for me), and also unpick it a little: Scala is Functional not Imperative. (N.b. the fact its also Object Oriented matters not a jot here.) So before we complete this section, let’s have an overview of the specific aspects Functional compared to Imperative programming:
• #13: Here’s what underpins a functional mindset: 1. Consider what information and transformations are required on data (rather than the steps and resulting state changes) 2. That's because these state changes are non-existent (or mega-frowned upon) 3. Which means that the order of execution is of low importance 4. and the focus is primarily on functions and data as the primary manipulation units. That’s almost diametrically opposed to how I, and I guess you think when you code in Java (From: http://msdn.microsoft.com/en-GB/library/bb669144.aspx)
• #14: To move to this new mindset, which is essential in order to wrap your head round something like Scala, you must jettison a lot of what you currently hold dear: * the idiom (the “names of things”) * the familiar syntax (even the "places of things") * Pretty much your entire way of thinking about code: * No more State / Mutability (hands off!) * Worrying about how things work inside (you don't need to care) * And remembering the order in which they work (ditto) * There’s no more Looping for you ("iterate" / "traverse" instead) * Or functions only hanging off objects (a fundamental OO concept - now they can just float around...) But its not all bad. If you bear this list in mind as you go you might remember to think “I’m feeling uncomfortable / experiencing pain; this probably means I’m still holding onto an imperative mindset. Let go.... Learn to let go...
• #15: OK, so all we do is stop thinking in an imperative way and start learning to think functionally. Simple right? “Sure, I have my list right here…” Well no, because the Java-imperative baggage we have with us runs deep. This means that some stuff about Scala is seemingly so simple when you first come across it you’re like, “uhuh, got it. Next!” but then you need to build on this and it turns out this dismissive approach didn’t prepare you. Scala’s subtleties around things like its support for the Uniform Access Principle and related concepts is one of these key areas. To examine it we’ll revise the following: * First up: functions as first-class citizens * Next: “everything is an expression" * And finally: infix notation When you look at them in close proximity you have your first taster of functional programming: – look back at ourfour-point lists.: You don’t care how something is done, or in which order it’s done, you just care about the result This lets you /let /go of a lot of mental baggage. That's the subtle (but potentially spectacularly large) benefit. image "rock"
• #16: Lets start quite slowly. First up, it's reassuring to remember that in Scala, objects are still objects: In fact I discovered that far from being hard to grasp, the core class hierarchy in Scala was like a welcome friend – it gets so much more right than Java did: everything is an object
• #17: But did you spot the first subtlety slipping past you there? "Everything is an object". Naturally you'd expect this to mean that functions are objects too. And they are. Lets revise what this means with some code… image "pebbles"
• #18: Here’s some not very surprising code: apart from perhaps that foreach is a higher order function which means it takes a function as a parameter. The print(“> “ + n) is the (anonymous) function being passed to it.
• #19: And then just pass the new “show” function-object to foreach. Beautiful. Next: “Everything is an expression”
• #20: Now we've got our toe-hold in functions being first class, lets revise the next "tiny" change: Everything is an Expression. What? Well, in Java, we have "statements" - "a statement in a programming language does not produce a result. In order for the statement to do something interesting, it must change the state of its surroundings. Another way of putting this is “a statement is called for its side effects” (that is, what it does other than producing a result)." (from Atomic Scala v.1.1, Eckel and Marsh) As a memory aid Bruce and Dianne suggest we think: "A STATEMENT CHANGES STATE" And so what about expressions?: Again, as a memory aid Bruce and Dianne suggest we think "One definition of 'express' is 'to force or squeeze out,' as in 'to express the juice from an orange.' So: AN EXPRESSION EXPRESSES That is, it produces a result." (from Atomic Scala v.1.1, Eckel and Marsh)
• #21: Again, lets look at it in action: In Scala, “if” is an expression
• #22: As we can see here… “for” is an expression too, but we’ll get to them later on.
• #23: This means that “Try-catch” blocks are expressions
• #24: Ok. We’re still good – a hand-hold – Finally we’ll revise infix notation. image "stone"
• #25: Again, when glanced at cursorily, all you see here is "nice, no dots or parens - good for DSLs” (look at me, I've read a book on DSLs!). Wait! Come back! You missed something... Lets refresh our memories… Clearly you can drop the dots, and the parens, and you now only have to put the method name between the objects
• #26: (or after the first if there's only one object). Note, it also only works if there is zero or one argument - more than that and the compiler wouldn't know how to cope sadly. This is almost too simple, why am I telling you all this, and wasting your time? OK, we’ve got our 3 anchor points. Lets now move on and consider the Uniform Access Principle
• #27: We've now got everything in mind to be able to make real deep sense of the "Uniform Access Principle": Put forth by Bertrand Meyer. It boils down to: there should be no difference between working with an attribute, precomputed property, or method/query. (From: https://en.wikipedia.org/wiki/Uniform_access_principle) Lets have that again: "there should be NO DIFFRERENCE between working with an attribute, precomputed property, or method/query" To us this means that the code should look the same, whether you’re accessing a function or a value (val or var) – whatever the case, your client code should not be any different Let’s see how this works in Scala. image "rock"
• #28: Here we’ve got a class called simple. It has one constructor argument, which lets us set a single field: a String s. We’ve also got two functions, getA and getB and a property “getC” (so named for clarity – bear with me). We then instantiate simple with the constructor parameter “Hi”
• #29: Finally we’re going to test it by calling it’s various functions: First up, lets examine the parens: dot-getA-parens – gives us “Hi” as expected
• #30: GetA-no-parens – “Hi”
• #31: Next dot-getC-no-parens – also “Hiya” hmmm. That’s a val we’re accessing there, and the value is calculated at call time, but it looks like the method calls… Now lets drop the dots…
• #32: And so ladies and gentlemen, we now have your first bit of Scala power right here What? Infix notation? So what? Well, lets consider for a minute what we’re getting here: With infix we’re relieved from worrying about the implementation of something – is it a value or a function? Don’t care We get a similar thing with functions as objects – the difference between functions and other objects is diminished – you care less what something is too And again with everything is an expression? – again, everything returns something, and side effects are frowned on – again we let go of what “it” is a little more As a result of all of these things we’re helped to move towards functional thinking: * Functions and attributes become virtually the same – from client code anyway * Side-effects are down-graded Sound tenuous? Perhaps, but this for me was the first fundamental realisation around Scala and one which has since served most to kick me out of my Java-imperative torpor But I feel impelled to point out that this can be painful too. It takes trust to know that you can forego all this extra mental-loading. The letting go can lead to a feeling of mental vertigo as you lose so many of your Imperative-crutches. It is at this point that you realise that for some (myself included) the verbosity, and explicit-ness of Java served as a safety blanket. Don't underestimate the significance of this mental unloading. Also don't underestimate the power of the pull of the "old way" - when reading some Scala code hurts; stop, think - it might be your brain not letting go. image "boulder"
• #33: Ok, that’s the “watch out for the hidden subtleties in things that seem - on the surface - patently obvious” trip-up, and associated goodness once we realise and work through it. Next we get to consider a slightly more involved area: Imagine if you will that there is a feature of the Scala language which has been designed to do something, and do it well. Imagine also that the keywords chosen make this link supremely clear, and all the syntactic sugar built up around it makes sense too, but you hear about it far earlier in the learning process, as it turns out to be a selling point to moving to the language (it’s a sizzle all on its own). As a result, we (i.e. I) frequently fail to appreciate the full possibilities of pattern matching Case classes are one of these such circumstances. Not only that, but also when you grok them - and a supporting concept, PartialFunctions in this case - you get an early taste of the mental model and internal consistency of Scala – of the “Scala all the way down” I mentioned at the start. That’s where we want to get in this section. It’s a view that empowers and re-vitalises you, and is one that is worth savouring. We’ll use as our engine to drive this section the much discussed concept of Pattern Matching image "cliff"
• #34: Lets take this slowly. Lets not start with Case Classes. Lets instead introduce / refresh Partial Functions. Firstly, where does the name come from? From maths of course. image "climber"
• #35: def inc(i: Int) = i + 1 // this is a TOTAL function
• #36: def fraction(d: Int) = 42 / d // this is a PARTIAL FUNCTION as it is not defined for d== 0 – it will throw an exception if called like this REMEBER: A complete function can be treated as a partial function Here, the complier will complain the match is not exhaustiveIt'll blow up if you pass it an empty list
• #37: Now here’s the clever bit – at this stage of Scala-development learning, we likely don’t need to create explicit PartialFunctions which are a Trait offered by Scala, but we have been using them – because every time we put “{ case …” we have one. Lets look at this in more detail. Here’s a new function called second the body of which is a “{ case” It’s clearly a partial function as we’ve not covered the cases when the list is empty of only has one element. And indeed it is – “{ case … “ is a literal for a partial function.
• #38: In fact, that previous function is equivalent to this,
• #39: And in fact, courtesy of Scala’s translation work, our second instance gives us access to the isDefinedAt and apply methods which come from the PartialFunction trait without our having had to implement them
• #40: Before we move on from Partial Functions, lets make the final leap of this sub-section to a circumstance where we’ve likely seen the “case” keyword already and their existence as literals for PartialFunctions in this circumstance will seem quite sensible now:
• #41: We’ve seen them in catch blocks for example, as individual case expressions …
• #42: or, more likely, as “case sequences”. In these cases, our concept of these sequences as literal PartialFunctions, with an entry point per case to the function, makes perfect, logical sense. Knowing this little equivalence trick helped me greatly in grasping a lot of code as moved through the world of Scala. We’re almost ready to look at one of those instances now – Pattern Matching with the match keyword – but lets not leap into that without first reminding ourselves about case classes
• #43: Now there came a point when I was surprised to learn that Case Classes aren’t just a way to save a lot of typing, despite frequently being introduced in that way. In fact they’re designed “to allow pattern matching on objects without requiring a large amount of boilerplate” (Odersky, Venners, Spoon) Hence the name, which by now ought to have a little more resonance for you - adding the keyword “case” to definitions of classes means they can be used in pattern-matches image "climber"
• #44: As a refresher - here’s what one looks like. But behind the scenes you’re getting a lot of bang for your buck. Just like the case-based PartialFunction shorthand, the “case” keyword again gives us a lot for free. By adding it we get: * “Natural” implementations of toString, hashcode and equals. N.b. in Scala ‘==‘ always delegates to equals * All args in the parameter list implicitly get a “val” prefix so they are maintained as immutable fields * Which means that all fields in Case Classes are immutable by default * Factory methods with the same name as the class: No more “new”. NOTE: looks very nice when nested * A copy method, using named and default parameters This all means that when you write pattern matching code, you can fit a lot, in a small space - a small space like a pattern matching block. Ludicrously, deceptively, simple.
• #45: And so finally we get to Pattern Matching. My first encounter with this concept was when it was being touted as a major reason to dump everything and move to Scala. Perhaps that’s overstating the case, but folks get pretty excited about it. But it does bring great benefits - “Pattern matching is a generalization of switch from C/Java to class hierarchies”. (Coursera session 4-7) In short, it brings polymorphism to the concept of “switch”, which when understood clearly and used well can be incredibly powerful. image "ice climbing"
• #46: Lets do a little deck clearing yet again :- There are lots of flavours of patterns in pattern matching, but we only want to look at one here – case class matchers – so here are all the other major options to get them out of the way: matching on a constant, matching on a type, matching on a type with a guard, matching on a type with a guard and some nice value extraction, wildcard matching – we’ll not go into that. Now we know what we’re not thinking about, lets look very quickly at a Case Class matcher
• #47: Here’s the type of cleverness we can do when we’re matching with Case classes. This excerpt is the equivalent of saying: “match all cats with the name “Bob” and print their age”. Note we’re matching on the construction parameters that make up the Cat. Clearly, even from this little snippet its clear that this can get really powerful, but you’ll have to take my word for it – we’ve got to keep moving – on to the pros, cons, gotchas and most importantly our mindset lessons to learn. (But if you want to know more, take a look at the Pattern Matching chapter in “Programming in Scala”)
• #48: Any gotchas worth highlighting? Two here: Patterns are tried in the order they are coded (because they de-sugar into a load of nested "If"s) Partial Functions are NOT "partially applied functions" which seems to have confused more folks than just me, and its not helped by the fact that some folks Scala tutorials on the topic get it wrong. (Could someone tidy up the internet please?) image "climbing rope"
• #49: Finally we get to our mindset slide. The mindset that this deeper understanding of Scala’s Pattern Matching brings is for me at least the biggest benefit of all, far greater than the “polymorphic switch” I trailed at the top of this section. Because case lists are partial functions, and because we can group them together and use with “match”; and because Scala is internally consistent in the fact it maps these concepts to real objects, it takes the Pattern Matching and firmly embeds it into the bigger Object Oriented picture, lifting it far above simply being a syntactic construct. As someone coming from Java, I really appreciated that. It meant it stuck as a concept. Bear this “Scala’s made of Scala” benefit in mind because we’re now going to tackle our first peak – for-comprehensions – and this will help us a lot.
• #50: We’ve bootstrapped ourselves a long way in a short period of time. Now we’ve got enough under our belt to make a push at a peak – the for-comprehension. * We're seen so far that Scala is constructed, at least in part, from a lot of cross-cutting concepts, but we’ve not yet looked explicitly at de-sugaring? * I didn’t go into this aspect in the last section because, despite being “made up” of Partial Functions, when you desugar a pattern match (perhaps unexpectedly) we get a super-nested set of if-expressions With for-comprehensions we’re dealing with a concept that is not only made up of other, smaller chunks of Scala, but we’re also dealing with something which explicitly de-sugars a little more nicely But that alone isn’t reason enough to go through the palaver of desugaring – we’ll also find that unlike Pattern Matching, for-comprehensions work in a way completely different from that which you’d expect. And so by de-sugaring we’ll learn not only gain an understanding of the contributing constituent parts, but also the wider the construct in question. Not only this, but by understanding how for-comprehensions are constructed we’ll be able to start to get a glimpse of our future Scala selves In this final section we’ve a slightly convoluted path we’re going to follow, but its worth it we’ll be clear first about what we’re about to consider Then orient ourselves in our mental starting place – picking Option[T] as a nice way to do this We’ll then look at Option[T] in a for comprehension Before stepping back to refresh the functions under the covers Before finally facing the elements of the for head on, and mapping out under the covers elements to each of them
• #51: Before we move on, after the previous "partial" / "partially applied" stramash, and because it’s important to the next slides, here's the scoop: You will find the construct that we’re about to look at called variously: - a "for comprehension" - or simply a "comprehension" - or a "for-expression" - or even a "for-yield" (you can get for-loops too, but that's a special case and we'll get back to them at the end)
• #52: And what are they for? Just as with pattern Matching, falling into the “it’s the equivalent of Java’s ‘for’ ” trap will not serve you well. For me particularly, this caused immense pain which you can see written up on my Java-to-Scala blog. Despite this, I persisted, because For-Comprehensions are very powerful, and are commonly used to: - Iterate through things - filter something - produce a new collection
• #53: We can’t move on yet. I need to stress this point – you need to stop thinking of Java’s “for” and everything that comes with it right now – having “looping” in your mind (which this will summon up) will send you down the wrong path. Remember, to “Iterate” is not to “loop”
• #54: It’s far better to think of them in terms of Option[T] - But again, make sure you don't think of this as "a list of something or nothing“ which you’ll encounter some resources telling you to do. - Remember, Option[T] is more strongly typed than that - It can contain None or Some[T] - And there are methods provided to work on its contents - And these methods can be applied without “opening the box” - Think of it instead as a way of avoiding null (one of many) (Or if you want to look further, check out Future[T] - out of scope for this presentation)
• #55: Here’s code again – let’s step through things: We create some options first…
• #56: Now we add a Person case class and a for-comprehension to push the Options through and yield a person instance. Note we’re pushing both Options through Clearly the “for” is doing something here to unpack the Options, as we get a “None” out the bottom from our yield – the None in the maybeSurname must have caused problems… Note also we’re still typesafe – no null pointers here.
• #57: So now what happens if we have no “Nones” and two “Some[String]”s? We get a Some of a Person, constructed from the inputs to the for. Nice eh? And nothing to do with collections or iterating at all. So now we dive into the bowels of things right? Well, in a manner of speaking. Actually, it‘ll pay us now to step back a bit farther.
• #58: We’ve got a toe-hold. A nice solid one. Its a good place to consider what is a for comprehension made up of. Like I said previously, it turns out that this is the first case where something in Scala really is "made up" of things which it's worth us looking at; its worth considering it here because it helps us understand how the construct works and what it does. Lets start with the first building block: Map.
• #59: Point number 1: We’re *not* talking about the Map collection type – it’s the map function were thinking about here (Out! Out! Begone Imperative thinking!) In Scala, Map is another higher order function - A function that takes a function – we’ve seen them before remember back in sections 2 and 3? Now unhelpfully, the Scala map function (which is a concept which unsurprisingly comes from Maths) is usually encountered in the context of a collection hence the frequent confusion But it needn't be Remember Option[T]? Lets look at it in code: Note: def map[B](f: (A) => B): Option[B] .map - Returns a Some[T] containing the result of applying function f to this Option's value if this Option is nonempty. Otherwise return None
• #60: Lets make some options again, and then see what happens when we call Map on them When we call map-toUppercase on the non-empty Option we can see we get a Some of (uppercase) [“MARK”] When we call it on the empty Option we get a None Nice and simple, and also kind of expected based on what we saw with our for-example a minute ago
• #61: Next up, flatMap. Again, its not a form of collection But you're beginning to realise that Instead, its another higher order function, which is again usually encountered in the context of a collection hence the frequent confusion (again) And again it needn't be It's _similar_ to map It applies the function to each list element and returns the _concatenation_ of all function results def flatMap[B](f: (A) => Option[B]): Option[B] Returns the result of applying function f to this Option's value if this Option is nonempty. Returns None if this Option is empty Lets see some code again which is nice and clear.
• #62: To see flatmap in action we need something a little more elaborate as input. We’re starting here with a List of strings. Then, as a handy comparison, let’s first call map-toInt on this List Clearly some values in the List will be amenable to this function, but not all. We can see this in the output – we end up with a list of Option[Int], with Some[Int]s where things went well, and None’s where they didn’t. That’s good and everything has worked for us, but its not ideal and will require some hairy code to get the “good” bits out to work with them. Can we get the list with just the Ints in it nice and simply? Yes, Flatmap can help. When we call flatMap-toInt on the same original list, this same function is applied to each element, but at the end, the resulting multidimensional collection is “flattened” – resulting in the Some[int]’s being unpacked and the Nones being discarded. Note now we have a List of Int, not Option of Int. Super-useful.
• #63: Finally withfilter. This one is nice and easy WithFilter() is Like filter(), but avoids an unnecessary creation of an intermediate data structure So, for our purposes, dead like filter, and it does what is says on the tin Code again:
• #64: Here we’re setting up a bunch of Person objects. Note that Lara and Julie are female, and that Julie has two kids – Lara and Bob. Then we add them each to a collection Finally we filter the collection, applying the predicate !p.isMale to each, which strips out male Person elements. We’re left with a 2-element list, containing Lara and Julie (we’ve omitted her kids in the wrapped array for clarity)
• #65: OK. We’re done. Finally we’ve got enough mentally in place to get back to for-comprehensions. Perhaps now it’ll be no major surprise to learn that there are three main parts in the syntax: Generator Filter (aka "guard") Definitions (What about yield? We’ll get there…)
• #66: Here’s a simple example showing all three: … at the top is the generator … this feeds the rest of the comprehension: v (whatever it is) produces values n until it is exhausted
• #67: … The filter (aka guard) where we can examine n (or anything else for that matter) and filter out n-values we don’t like the look of …
• #68: And the Definition, where we can create comprehension-scoped variables to help keep our code clean We won’t cover these in the desugar (they don’t – they are what they are already)
• #69: And now that we know about our elements, we can finally move to the desugaring. Which is how we translate from map, flatmap and withFilter to generators and guards. For our first pass lets take a single generator and a filter
• #70: We can see the comprehension at the top of the slide Then at the bottom we can see the equivalent function, the one the compiler makes for us. And look: It turns into a map call on the value which feeds the generator, using the yield value as the function parameter. The result of this is then (perhaps unsurprisingly) passed to withfilter which contains the predicate from the filter expression in the for. I’ll say it again – the generator has become a map call with the yield-expression as it’s parameter - the filter has become a with filter which works on the output from map. These simple rules are used consistently across all flavours of for-comprehension. It sounds too good to be true, but it isn’t. So we’ve covered two (ok, three) of our concepts – map and withfilter. There’s one more to go – where’s flatMap?
• #71: To start seeing flatMap in action, we need to up the number of generators – you can have as many as you like, just as you can with filters and definitions. Lets take the filter out of this next example to keep things simple and help them fit on a screen (you can guess how it’s come into play anyway right?)
• #72: Here’s the code: We now have our two generators – the one we had before, and now a second, simpler one. When this evaluates, each combination of elements is fed in – for the first value x, we then generate each value y, before moving back to the next value x - and the results yielded. In this case the result is a Vector(true, false, false, false, false, false, false, false, false, false) And how does it desugar? We start with the expected map on the first generator input, taking the yield predicate as the function parameter. Next however, we flatmap the result of this map call and flatmaps it onto the second generator input. Why not map? Because we’d get a vector of vectors – Vector(Vector(true), Vector(false), Vector(false), Vector(false), Vector(false), Vector(false), Vector(false), Vector(false), Vector(false), Vector(false)). Flatmap flattens it all out for us. If you have more than one generator, the last one is always desugared into a flatmap, the rest into maps. And the best bit, this rule also holds true across all instances that you can come up with that match this pattern. That’s enough for now. Lets looks at the traditional gotchas and finally return to the mindset.
• #73: Here’s the code: We now have our two generators – the one we had before, and now a second, simpler one. When this evaluates, each combination of elements is fed in – for the first value x, we then generate each value y, before moving back to the next value x - and the results yielded. In this case the result is a Vector(true, false, false, false, false, false, false, false, false, false) And how does it desugar? We start with the expected map on the first generator input, taking the yield predicate as the function parameter. Next however, we flatmap the result of this map call and flatmaps it onto the second generator input. Why not map? Because we’d get a vector of vectors – Vector(Vector(true), Vector(false), Vector(false), Vector(false), Vector(false), Vector(false), Vector(false), Vector(false), Vector(false), Vector(false)). Flatmap flattens it all out for us. If you have more than one generator, the last one is always desugared into a flatmap, the rest into maps. And the best bit, this rule also holds true across all instances that you can come up with that match this pattern. That’s enough for now. Lets looks at the traditional gotchas and finally return to the mindset.
• #74: And Gotchas? There are a few minor ones: Firstly, the comprehension is executed *sequentially* It doesn't parallelize, unless you create your elements ahead of, outside the "for“ which to some is a bit non-intuitive. Secondly the type that goes in must come out - you're just filtering, mapping and flatmapping remember And a few more major ones: there are two valid forms of the comprehension: One with parens and semi-colons the other with curly braces Also, I lied - people do refer to the "for-loop“ When you only have a single generator, and no filters or definitions These desugar to a combo of withFilter and foreach
• #75: And so finally lets tie things up with the mindset – what’s the benefit to us of looking at for-comprehensions in such depth? Well, for-comprehensions illustrate the third type of creeps-up-on-you Scala-functional mind blend and that is: When you find that, beneath the covers, something is working in a way completely different to that which you might expect it to Then suddenly, this understanding of the de-sugaring helps you properly understand not only the construct in question, but also the contributing constituent parts It’s basically an enforced, but hand-held, letting go. When I personally “got” this, it put me in a place where I could look back on my many experiences spent getting to here, and a lot fell into place. In fact, it was when I got this that I realised I might have a talk – some kind of consistent narrative – that I could pull together for you here today.
• #76: So we’re almost done. I’m not going to claim that just by sitting and listening to this you’ll suddenly have the same dawning of realisation I’ve just talked about. I think you need to travel the path yourself, like thousands have before me. It’s hard, but it is worth it. And before I close I’d like to impart some insights that are my own, and I didn’t just read them or cut and paste them from the internet: General Rules: 1. Read closely – these maths guys cram a lot into a tiny amount of code / text 2. Don’t automatically trust what you know from Java 3. But don’t automatically discount it either 4. Unlearning is harder than you think. You’ll feel the side-effects of a lot of things you stopped being conscious of, as they’ll start getting in the way. When you understand what these are, then you can move past them / put them to work properly.
• #77: And if you want to retrace my footsteps, how might you get to here? Firstly I’d advise against reading Scala in Action if you’re at the stage I was (and still am). Try these instead, in this order, though you can (and should) overlap: * “Atomic Scala” (Eckel & Marsh) * The Scala Koans * “Scala for the Impatient” (Horstmann) * Twitter’s Scala School * “Introduction to Functional Programming” (Coursera)(Odersky) * “Programming in Scala (2nd Ed.)” (Odersky, Venners, Spoon) * The Scala Language Specification
• #78: And where next? * Firstly I’d encourage you to blog it as you progress – trying to explain what you think you understood helps you really understand it – You might even want to guest-post on my blog ;-) http://scalaeyeforthejavaguy.blogspot.com * And of course desugar more constructs for yourself – there’s lots more out there to consider And the next-step constructs beyond what we covered here? * Learn Implicits * Learn Type Classes * Learn Haskell! (“Learn you a Haskell…” is great for this) * Re-read things you thought you understood
• #79: Wait, what about those monadic constructs you mentioned as the goal right at the start? Oh, those? That’s map and flatmap. Without knowing it, you’re now very nicely set up to start getting your head round Monads too. Take a look and see…