How to extend map? Or why we need collections redesign? - Scalar 2017

HOW TO EXTEND MAP?
OR WHY WE NEED COLLECTIONS REDESIGN?
SZYMON MATEJCZYK, SCALAR 2017

HOW TO EXTEND MAP?… OR WHY WE NEED COLLECTIONS REDESIGN?
@SZYMONMATEJCZYK
▸ Engineer & data scientist.
▸ Scala fan since 2.8. Graphs lover.
▸ PhD student on leave of absence.
▸ OSS contributor (Cassovary,
Nebulostore).

GRID
 
type Grid[V] = Map[(Int, Int), V]
def row(i: Int): Seq[(Int, V)]

SCALA COLLECTIONS

SCALA COLLECTIONS - IMMUTABLE

IMMUTABLE MAP
package scala 
package collection 
package immutable 
 
import generic._
trait Map[K, +V] extends Iterable[(K, V)] 
// with GenMap[K, V] 
with scala.collection.Map[K, V] 
with MapLike[K, V, Map[K, V]
trait MapLike[K, +V, +This <: MapLike[K, V, This] with Map[K, V]] 
extends scala.collection.MapLike[K, V, This] 
with Parallelizable[(K, V), ParMap[K, V]]
package scala 
package collection
trait MapLike[K, +V, +This <: MapLike[K, V, This] with Map[K, V]] 
extends PartialFunction[K, V] 
with IterableLike[(K, V), This] 
with GenMapLike[K, V, This] 
with Subtractable[K, This] 
with Parallelizable[(K, V), ParMap[K, V]]

IMMUTABLE.MAP TRAITS HIERARCHY

LET’S START WITH A INTERFACE…
trait Grid[V] extends immutable.Map[(Int, Int), V] { 
def row(i: Int): Seq[(Int, V)] 
}
class NestedGrid[V](
val underlying: immutable.Map[Int, immutable.Map[Int, V]]
) 
extends Grid[V] {
...
}

IMPLEMENTATION
override def row(i: Int): Seq[(Int, V)] =
underlying.get(i).toSeq.flatten 
 
override def get(key: (Int, Int)): Option[V] = { 
for { 
inner <- underlying.get(key._1) 
res <- inner.get(key._2) 
} yield res 
}
override def iterator: Iterator[((Int, Int), V)] =
underlying.iterator.flatMap{ 
case (first, map) => map.iterator.map(x => ((first, x._1), x._2)) 
} 
 
override def -(key: (Int, Int)): NestedGrid[V] = { 
underlying.get(key._1) match { 
case Some(e) => new NestedGrid(underlying + ((key._1, e - key._2))) 
case None => this 
} 
}

IMPLEMENTATION
) extends Grid[V] {
override def +(kv: ((Int, Int), V)): NestedGrid[V] = { 
...
}
}

IMPLEMENTATION
) extends Grid[V] {
override def +(kv: ((Int, Int), V)): NestedGrid[V] = { 
...
}
}
Method + overrides nothing

FEW STEPS BACK
trait Grid[+V] extends immutable.Map[(Int, Int), V] { 
def row(i: Int): Seq[(Int, V)] 
} 
 
 
class NestedGrid[+V](
protected val underlying: immutable.Map[Int, immutable.Map[Int, V]]
) 
extends Grid[V] 
{
override def +[V1 >: V](kv: ((Int, Int), V1)): NestedGrid[V1] = { 
...
}
}

IMPLEMENTATION CONT’D
 
 
override def +[V1 >: V](kv: ((Int, Int), V1)): NestedGrid[V1] = { 
underlying.get(kv._1._1) match { 
case Some(e) => new NestedGrid( 
underlying + ((kv._1._1, e + ((kv._1._1, kv._2)))) 
) 
case None => new NestedGrid( 
underlying + ((kv._1._1, immutable.Map(kv._1._2 -> kv._2))) 
) 
} 
}

IMMUTABLE MAP COVARIANCE

SIMPLE
trait Key 
case class DoorKey(id: Int) extends Key 
case class EnglishKey(size: Int) extends Key 
 
import scala.collection.immutable.Map 
 
val cabinet = Map[Int, DoorKey]( 
1 -> DoorKey(7), 
3 -> DoorKey(8) 
) 
 
cabinet + (5 -> EnglishKey(77))

IMMUTABLE MAP COVARIANCE
=+

TEST
def sampleMap = { 
val map: NestedGrid[Figure] = NestedGrid.empty[Figure] 
map + (((1, 1), Bishop)) + (((2, 2), Knight)) 
}
 
"Grid" should { 
"contain elements" in { 
val map = sampleMap 
map((1, 1)) should equal (Bishop) 
map.toSeq.sortBy(_._1) should equal (Seq( 
((1, 1), Bishop), 
((2, 2), Knight) 
)) 
} 
 
"support iteration" in { 
map.iterator.toSeq should equal (Seq(((1, 1), Bishop), ((2, 2), Knight))) 
} 
 
"support adding superclass" in { 
val map: NestedGrid[Figure] = NestedGrid.empty[Figure] 
(map + (((2, 2), Pawn))) shouldBe an[Grid[Piece]] 
} 
 
"support row-iteration" in { 
map.row(1) should equal (Seq((1, Bishop))) 
} 
}

FILTER AND OTHERS
def filter(p: A => Boolean): Repr = { 
val b = newBuilder 
for (x <- this) 
if (p(x)) b += x 
b.result 
}
trait Map[K, +V] extends ... 
with MapLike[K, V, Map[K, V]
trait MapLike[K, +V, +This <: MapLike[K, V, This] extends … {
override protected[this] def newBuilder: Builder[(K, V), This] =
new MapBuilder[K, V, This](empty)
}
class NestedGrid[+V](
protected val underlying: immutable.Map[Int, immutable.Map[Int, V]]
) 
extends Grid[V] 
with immutable.MapLike[(Int, Int), V, NestedGrid[V]]

MAP AND OTHERS
def map[B, That](f: A => B)(
implicit bf: CanBuildFrom[Repr, B, That]
): That = { 
val b = bf(repr)
b.sizeHint(this) 
for (x <- this) b += f(x) 
b.result 
}
implicit def canBuildFrom[B]: CanBuildFrom[NestedGrid[B], ((Int, Int), B), NestedGrid[B]] = 
new CanBuildFrom[NestedGrid[_], ((Int, Int), B), NestedGrid[B]] { 
override def apply(from: NestedGrid[_]): mutable.Builder[((Int, Int), B), NestedGrid[B]] = 
newBuilder[B] 
override def apply(): mutable.Builder[((Int, Int), B), NestedGrid[B]] = newBuilder 
}
def newBuilder[B]: mutable.Builder[((Int, Int), B), NestedGrid[B]] = 
new mutable.MapBuilder[(Int, Int), B, NestedGrid[B]](empty)

DEFAULT
override def withDefault[V1 >: V](d: ((Int, Int)) => V1): NestedGrid[V1] =
new NestedGrid.WithDefault[V1](this, d) 
 
override def withDefaultValue[V1 >: V](d: V1): NestedGrid[V1] =
new NestedGrid.WithDefault[V1](this, x => d)
class WithDefault[+V](from: NestedGrid[V], d: ((Int, Int)) => V) extends NestedGrid[V](from.underlying) { 
override def empty = new WithDefault(from.empty, d) 
override def updated[V1 >: V](key: (Int, Int), value: V1): WithDefault[V1] = 
new WithDefault[V1](from.updated[V1](key, value), d) 
override def + [V1 >: V](kv: ((Int, Int), V1)): WithDefault[V1] = updated(kv._1, kv ._2) 
override def - (key: (Int, Int)): WithDefault[V] = new WithDefault(from - key, d) 
override def withDefault[V1 >: V](d: ((Int, Int)) => V1): NestedGrid[V1] = new WithDefault[V1](from, d) 
override def withDefaultValue[V1 >: V](d: V1): NestedGrid[V1] = new WithDefault[V1](from, x => d) 
override def default(x: (Int, Int)) = d(x) 
}

RECAP
1. Decipher hierarchy
2. Implement logic
3. Be careful about variance and type bounds.
4. Change type params in MapLike for ﬁlter, take,…
5. Add implicit CanBuildFrom for map, ﬂatMap, …
6. Manually create WithDefault implementation
(code duplication)

How to extend map? Or why we need collections redesign? - Scalar 2017

COLLECTIONS REDESIGN (ODERSKY)
•simplicity
•separate interface from implementation
•ﬁx complicated inheritance structure
•improve efﬁciency

COLLECTIONS REDESIGN (SPIEWAK)
•to generic interfaces (Seq)
•collection/mutable/immutable
•general implementations inefﬁcient
•extendability is important
•CanBuildFrom does more harm than good

BONUS - QUESTIONS
Did you spot a memory leak?
How would experienced Scala programmer tackle the
problem?
Is the extension of mutable.Map similar?

BONUS - ALTERNATIVES
•Implement row method outside of Map
•Implicit conversions
•Use composition, implement map, ﬁlter, default … on your
own

REFERENCES
•http://www.artima.com/scalazine/articles/scala_collections_architecture.html
•https://github.com/lampepﬂ/dotty/issues/818
•https://gist.github.com/djspiewak/2ae2570c8856037a7738
•Code: https://github.com/szymonm/scalar-grid
•@szymonmatejczyk

How to extend map? Or why we need collections redesign? - Scalar 2017

More Related Content

What's hot (20)

Similar to How to extend map? Or why we need collections redesign? - Scalar 2017 (20)

Recently uploaded (20)

How to extend map? Or why we need collections redesign? - Scalar 2017