Big Data

Scala

This lecture covers the Scala language that Spark uses: REPL, scripts, types, variables, strings, arrays, operators, loops, classes, functions and collections. After this page you can trace any slide program by hand and predict its exact output.

What Scala isSearch on YouTube

Scala is an object-oriented and functional programming language. Martin Odersky designed it. It appeared for the Java platform in early 2004 and for the .NET framework in June 2004.

Scala runs on the JVM, so Java libraries work from Scala code. Spark itself is written in Scala. That is why this paper teaches Scala before Spark.

Where Scala is usedSearch on YouTube

The slides name five use cases. Three are data processing with Apache Spark, web development with the Play Framework, and concurrent and parallel programming. The other two are robust backend systems and financial applications.

Frameworks and librariesSearch on YouTube

LibraryUse
Apache SparkLarge-scale data processing
AkkaConcurrent and distributed systems
Play FrameworkWeb development
ScalatraSimple web apps, in the style of Sinatra
CatsFunctional programming abstractions

IDEsSearch on YouTube

The slides name four IDEs. IntelliJ IDEA has a Scala plugin. Visual Studio Code has Scala plugins. Eclipse uses the Scala IDE plugin. Atom uses Scala plugins.

The REPLSearch on YouTube

The REPL (Read-Eval-Print Loop) is an interactive shell. You type a snippet and it runs at once. Start it in three steps.

  1. Open a terminal.
  2. Type scala and press Enter.
  3. Type Scala code and press Enter to see the result at once.

The slide session uses Scala 3.3.1 on Java 21.0.1. Each result gets a name (res0, res1) and a type.

Worked example. Three inputs from the slide session.

scala> println("Hello, tutorialspoint")
Hello, tutorialspoint
scala> 445
val res0: Int = 445
scala> 10/6
val res1: Int = 1

Step 1. println prints the string and returns no value. The output is exactly the text inside the quotes.

Step 2. 445 is an integer literal. The REPL stores it as res0 with type Int.

Step 3. 10/6 is integer division. Both sides are Int, so the answer is Int: 10/6=110/6 = 1 with remainder 4. The REPL stores 1 as res1.

Do not expect 10/6 to give 1.67. Two Int values divide to an Int. The fractional part is dropped, not rounded.

Scripts and setupSearch on YouTube

You can write Scala scripts in files with the .scala extension. The slide file is ScalaExample.scala.

interactive mode script mode terminal + scala HelloWorld.scala answer at once scalac, then scala
Interactive mode answers each line at once. Script mode compiles the file first, then runs it.
object ScalaExample {
  def main(args: Array[String]): Unit = {
    println("Hello Scala")
  }
}
scalac ScalaExample.scala
scala ScalaExample

scalac compiles the file. scala runs the compiled class. The output is Hello Scala. In Scala 2 style (the slides' style), a program needs an object with a main method that takes Array[String].

Environment setupSearch on YouTube

Step 1 is to verify Java. Run java --version on Windows or Linux. The slide outputs show Java 11 and Java 21. Scala needs a JDK first.

Step 2 is to install Scala. The slide commands use old installers:

java -jar scala-2.11.5-installer.jar
java -jar scala-2.9.0.1-installer.jar

The Linux installer prints a welcome message and unpacks one software package. Then verify the install:

scala -version
scala

The first command prints the runner version (3.3.1 on the slide). The second opens the REPL welcome text.

First programSearch on YouTube

A Scala program runs in two modes: interactive mode and script mode. In interactive mode you type scala, then println("Hello, Scala!"); prints Hello, Scala!. In script mode you save this file as HelloWorld.scala:

object HelloWorld {
  def main(args: Array[String]): Unit = {
    println("Hello, world!")
  }
}

Compile it with scalac HelloWorld.scala. Run it with scala HelloWorld.

Data typesSearch on YouTube

Scala has all the same data types as Java. The slide table is:

TypeSizeRange or meaning
Byte8 bit signed-128 to 127
Short16 bit signed-32768 to 32767
Int32 bit signed-2147483648 to 2147483647
Long64 bit signed-9223372036854775808 to 9223372036854775807
Float32 bit IEEE 754Single-precision decimal
Double64 bit IEEE 754Double-precision decimal
Char16 bit unsignedU+0000 to U+FFFF
StringSequence of CharText
Booleantrue or falseTruth value
UnitNo valueLike void in Java
NullNull referenceEmpty reference
NothingSubtype of every typeHolds no value

B-S-I-L, F-D, C-S, B-U-N-N

Byte (8), Short (16), Int (32), Long (64) grow by doubling bits. Float (32) and Double (64) are IEEE 754. Char and String hold text. Boolean holds truth. Unit, Null and Nothing mark absence of value.

VariablesSearch on YouTube

A mutable variable uses var. You can change its value later. An immutable variable uses val. You cannot change it after creation.

var myVar: String = "Foo"
var data: Int = 100
data = 101
val myVal: String = "Foo"

var declares a variable you can reassign, so data = 101 works. val declares a fixed value, so reassignment is an error. The type after the colon is optional when Scala can infer it.

Type inferenceSearch on YouTube

Scala infers the type when you skip it:

var myVar = 10;
val myVal = "Hello, Scala!";

Here myVar becomes Int and myVal becomes String by default. Inference never changes mutability: var stays mutable and val stays fixed.

Multiple assignmentsSearch on YouTube

A method or block can return a Tuple (a fixed group of objects with different types). You can assign the whole Tuple to val variables at once:

val (myVar1: Int, myVar2: String) = Pair(40, "Foo")
// or, with inference:
val (myVar1, myVar2) = Pair(40, "Foo")

Both forms give myVar1 = 40 and myVar2 = "Foo". The second form lets inference find the types. The slides use Pair(40, "Foo"). In Scala 2.13 and 3, write (40, "Foo") instead.

Worked example. Save this program in Demo.scala.

object Demo {
  def main(args: Array[String]): Unit = {
    var myVar: Int = 10;
    val myVal: String = "Hello Scala with datatype declaration.";
    var myVar1 = 20;
    val myVal1 = "Hello Scala new without datatype declaration.";
    println(myVar); println(myVal); println(myVar1); println(myVal1);
  }
}
scalac Demo.scala
scala Demo

Output. Four lines, in the order of the four println calls:

10
Hello Scala with datatype declaration.
20
Hello Scala new without datatype declaration.

StringsSearch on YouTube

Create a string with or without an explicit type:

var greeting = "Hello world!";
// or, with an explicit type:
var greeting: String = "Hello world!";

String lengthSearch on YouTube

object Demo {
  def main(args: Array[String]): Unit = {
    var palindrome = "Dot saw I was Tod";
    var len = palindrome.length();
    println("String Length is : " + len);
  }
}

The output is String Length is : 17. The count is 17. "Dot" gives 3. Two words add spaces and reach 8. "I" reaches 10. "was" reaches 14. "Tod" reaches 17.

Concatenating stringsSearch on YouTube

string1.concat(string2) returns a new string with string2 added at the end:

"My name is ".concat("Zara")

The + operator is more common:

"Hello," + " world" + "!"

Only "Hello," + " world" + "!" gives Hello, world!; the concat form gives My name is Zara. The slides also show these two string examples:

object Demo {
  def main(args: Array[String]): Unit = {
    var greeting: String = "Hello, world!";
    println(greeting);
  }
}

The output is Hello, world!. And:

object Demo {
  def main(args: Array[String]): Unit = {
    var str1 = "Dot saw I was ";
    var str2 = "Tod";
    println("Dot " + str1 + str2);
  }
}

The output is Dot Dot saw I was Tod.

ArraysSearch on YouTube

Declare an array with a fixed size, or from values directly:

var z: Array[String] = new Array[String](3)
// or, with inference:
var z = new Array[String](3)
// or, from values directly:
var z = Array("Zara", "Nuha", "Ayan")

Assign by index. The index can be any expression that yields a whole number:

z(0) = "Zara"; z(1) = "Nuha"; z(4/2) = "Ayan"

Here 4/2=24/2 = 2, so the third line sets index 2. The array is ["Zara", "Nuha", "Ayan"].

Multidimensional arraysSearch on YouTube

var myMatrix = Array.ofDim[Int](3, 3)

ofDim[Int](3, 3) makes a 3×33 \times 3 integer matrix. The general forms are Array.ofDim[T](n1), Array.ofDim[T](n1, n2) and Array.ofDim[T](n1, n2, n3). You need import Array._ for ofDim. You can also nest literals: Array(Array(e1, ...), Array(e2, ...)).

Worked example. Build the matrix column by column.

import Array._
object Demo {
  def main(args: Array[String]): Unit = {
    var myMatrix = ofDim[Int](3, 3)
    for (i <- 0 to 2) {
      for (j <- 0 to 2) {
        myMatrix(i)(j) = j;
      }
    }
    for (i <- 0 to 2) {
      for (j <- 0 to 2) {
        print(" " + myMatrix(i)(j));
      }
      println();
    }
  }
}

Trace. Each cell gets its column index j. Row 0 is 0 1 2. Row 1 is 0 1 2. Row 2 is 0 1 2.

Output.

 0 1 2
 0 1 2
 0 1 2

Processing arraysSearch on YouTube

Use loop control structures because all elements share one type and the size is known.

Worked example. Print, sum and find the maximum of Array(1.9, 2.9, 3.4, 3.5).

var myList = Array(1.9, 2.9, 3.4, 3.5)
for (x <- myList) {
  println(x)
}
var total = 0.0;
for (i <- 0 to (myList.length - 1)) {
  total += myList(i);
}
println("Total is " + total);
var max = myList(0);
for (i <- 1 to (myList.length - 1)) {
  if (myList(i) > max) max = myList(i);
}
println("Max is " + max);

Print step. The first loop prints each element: 1.9, 2.9, 3.4, 3.5.

Sum step.

total=0.0+1.9+2.9+3.4+3.5=11.7\begin{aligned} total &= 0.0 + 1.9 + 2.9 + 3.4 + 3.5\\ &= \mathbf{11.7} \end{aligned}

Maximum step. Start with max = 1.9. Compare 2.9 (larger, update), 3.4 (larger, update), 3.5 (larger, update). The final max is 3.5.

The slides link the full method list at https://www.tutorialspoint.com/scala/scala_arrays.htm.

OperatorsSearch on YouTube

The slides name five groups: arithmetic, relational, logical, bitwise and assignment operators. The worked demo uses arithmetic with a = 10, b = 20, c = 25, d = 25.

Worked example. Six println calls, one per line.

println("a + b = " + (a + b));
println("a - b = " + (a - b));
println("a * b = " + (a * b));
println("b / a = " + (b / a));
println("b % a = " + (b % a));
println("c % a = " + (c % a));
a+b=10+20=30a−b=10−20=−10a×b=10×20=200b/a=20/10=2b mod a=20 mod 10=0c mod a=25 mod 10=5\begin{aligned} a + b &= 10 + 20 = \mathbf{30}\\ a - b &= 10 - 20 = \mathbf{-10}\\ a \times b &= 10 \times 20 = \mathbf{200}\\ b / a &= 20 / 10 = \mathbf{2}\\ b \bmod a &= 20 \bmod 10 = \mathbf{0}\\ c \bmod a &= 25 \bmod 10 = \mathbf{5} \end{aligned}

Output. a + b = 30, a - b = -10, a * b = 200, b / a = 2, b % a = 0, c % a = 5.

If elseSearch on YouTube

The syntax is:

if (Boolean_expression) {
  // Runs when the expression is true
} else {
  // Runs when the expression is false
}

Worked example. var x = 30 with the test x < 20.

object Demo {
  def main(args: Array[String]): Unit = {
    var x = 30;
    if (x < 20) {
      println("This is if statement");
    } else {
      println("This is else statement");
    }
  }
}

Trace. 30<2030 \lt 20 is false, so the else branch runs. The output is This is else statement.

LoopsSearch on YouTube

LoopBehaviour
whileRepeats while a condition is true. It tests the condition before the body.
do-whileLike while, but it tests the condition at the end of the body. The body runs at least once.
forRepeats statements many times. It shortens the code that manages the loop variable.

For loop with rangesSearch on YouTube

for (a <- 1 to 10) {
  println("Value of a: " + a);
}

1 to 10 is an inclusive range. The output is ten lines: Value of a: 1 through Value of a: 10.

Multiple rangesSearch on YouTube

Separate two ranges with a semicolon:

for (a <- 1 to 3; b <- 1 to 3) {
  println("Value of a: " + a);
  println("Value of b: " + b);
}

The outer variable a takes 1, 2, 3. The inner variable b takes 1, 2, 3 for each a. That is 3×3=93 \times 3 = 9 pairs: (1,1), (1,2), (1,3), (2,1), (2,2), (2,3), (3,1), (3,2), (3,3). Each pair prints two lines, so 18 lines in all.

For loop with collectionsSearch on YouTube

val numList = List(1, 2, 3, 4, 5, 6);
for (a <- numList) {
  println("Value of a: " + a);
}

The output is six lines: Value of a: 1 through Value of a: 6.

For loop with yieldSearch on YouTube

yield builds a new collection from the loop instead of printing:

val numList = List(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
var retVal = for { a <- numList if a != 3; if a < 8 } yield a
for (a <- retVal) {
  println("Value of a: " + a);
}

With numList = List(1, 2, 3, 4, 5, 6, 7, 8, 9, 10), the guards drop 3 and drop every value ≥8\ge 8. The result is List(1, 2, 4, 5, 6, 7). The second loop prints those six values, one per line.

Classes and objectsSearch on YouTube

The slides cover five items: basic classes, anonymous objects, inheritance, implicit classes and singleton objects. The deck gives code for basic classes, anonymous objects and inheritance. It only names implicit classes and singleton objects. The outline also lists pattern matching, files I/O, closures and traits with no slides.

Basic classSearch on YouTube

class Student {
  var id: Int = 0;
  var name: String = null;
}
object MainObject {
  def main(args: Array[String]): Unit = {
    var s = new Student()
    println(s.id + " " + s.name);
  }
}

The class sets var id: Int = 0 and var name: String = null explicitly, so it prints 0 null.

A class can take constructor parameters and define methods:

class Student(id: Int, name: String) {
  def getRecord(): Unit = {
    println(id + " " + name);
  }
}
object MainObject {
  def main(args: Array[String]): Unit = {
    var student1 = new Student(101, "Raju");
    var student2 = new Student(102, "Martin");
    student1.getRecord();
    student2.getRecord();
  }
}

The output is 101 Raju then 102 Martin.

The Point class shows mutable state with a move method:

import java.io._
class Point(val xc: Int, val yc: Int) {
  var x: Int = xc
  var y: Int = yc
  def move(dx: Int, dy: Int): Unit = {
    x = x + dx
    y = y + dy
    println("Point x location : " + x);
    println("Point y location : " + y);
  }
}
object Demo {
  def main(args: Array[String]): Unit = {
    val pt = new Point(10, 20);
    pt.move(10, 10);
  }
}

Start is (10, 20). Add (10, 10). The output is Point x location : 20 and Point y location : 30.

Anonymous objectSearch on YouTube

An anonymous object has no reference name. Use it when you will not reuse it.

class Arithmetic {
  def add(a: Int, b: Int): Unit = {
    var add = a + b;
    println("sum = " + add);
  }
}
object MainObject {
  def main(args: Array[String]): Unit = {
    new Arithmetic().add(10, 10);
  }
}

new Arithmetic() creates the object and calls add at once. The output is sum = 20.

InheritanceSearch on YouTube

Scala allows inheritance from one class only. Point is the superclass. Location is the subclass. Location adds a z coordinate and overrides the constructor values with override val.

Point(x, y) move(dx, dy) Location(x, y, z) move(dx, dy, dz)
Location extends Point. It reuses x and y and adds z with a three-argument move.

Worked example. Build Location(10, 20, 15) and call move(10, 10, 5).

val loc = new Location(10, 20, 15);
loc.move(10, 10, 5);

Trace. x = 10 + 10 = 20. y = 20 + 10 = 30. z = 15 + 5 = 20.

Output.

Point x location : 20
Point y location : 30
Point z location : 20

Access modifiersSearch on YouTube

Members of packages, classes or objects can use private and protected. Without either keyword, access is public. The modifiers restrict which regions of code can reach the member.

class Outer {
  class Inner {
    private def f(): Unit = { println("f") }
    class InnerMost {
      f() // OK
    }
  }
  (new Inner).f() // Error: f is not accessible
}
package p {
  class Super {
    protected def f(): Unit = { println("f") }
  }
  class Sub extends Super {
    f()
  }
  class Other {
    (new Super).f() // Error: f is not accessible
  }
}

A private member is visible only inside the class or object that defines it. A protected member is visible only in subclasses of the defining class.

FunctionsSearch on YouTube

A function declaration names the function, lists parameters and states the return type. A function definition adds the body:

object add {
  def addInt(a: Int, b: Int): Int = {
    var sum: Int = 0
    sum = a + b
    return sum
  }
}

A function that returns nothing returns Unit (like void in Java). Such functions are called procedures:

object Hello {
  def printMe(): Unit = {
    println("Hello, Scala!")
  }
}

Worked example. Call addInt(5, 7) and print the result.

object Demo {
  def main(args: Array[String]): Unit = {
    println("Returned Value : " + addInt(5, 7));
  }
  def addInt(a: Int, b: Int): Int = {
    var sum: Int = 0
    sum = a + b
    return sum
  }
}

Trace. sum = 5 + 7 = 12. The function returns 12. The output is Returned Value : 12.

CollectionsSearch on YouTube

Scala has a rich collection library. Collections are containers of things. The containers can be sequenced, linear sets of items such as List, Tuple, Option and Map. A collection can hold any number of elements, or zero or one element (for example Option).

Traversable (trait) Iterable (abstract class) Set Seq Map HashSet SortedSet IndexedSeq LinearSeq - List HashMap SortedMap
Main branches of the slide hierarchy (simplified). Traversable sits at the top, then Iterable, then Set, Seq and Map with their implementations.
CollectionIdea
ListLinked list of one type List[T]
SetPairwise different elements of one type
MapKey and value pairs, looked up by key
TupleFixed group of mixed types, passed as a whole
OptionContainer for zero or one element of type T
IteratorOne-by-one access to a collection, not a collection itself

L-S-M-T-I-O: Lists Share, Maps Tag, Iterate Once

Lists hold one type in a linked list. Sets hold pairwise different elements. Maps tag each value with a unique key. Tuples mix types in a fixed group. Iterators walk elements one by one. Option holds zero or one element.

ListsSearch on YouTube

Scala Lists resemble arrays: all elements share one type. Two differences matter. Lists are immutable, so assignment cannot change an element. Lists are linked lists, while arrays are flat. A list of type T is written List[T].

Build a list with List(...) or with :: (prepend) and Nil (empty list):

val fruit: List[String] = List("apples", "oranges", "pears")
val nums: List[Int] = List(1, 2, 3, 4)
val empty: List[Nothing] = List()
val dim: List[List[Int]] = List(List(1, 0, 0), List(0, 1, 0), List(0, 0, 1))
val fruit = "apples" :: ("oranges" :: ("pears" :: Nil))
val nums = 1 :: (2 :: (3 :: (4 :: Nil)))
val empty = Nil

Key methods are head (first element), tail (all but first) and isEmpty.

Worked example. fruit = List("apples", "oranges", "pears"), nums = Nil.

println("Head of fruit : " + fruit.head)
println("Tail of fruit : " + fruit.tail)
println("Check if fruit is empty : " + fruit.isEmpty)
println("Check if nums is empty : " + nums.isEmpty)

Output.

Head of fruit : apples
Tail of fruit : List(oranges, pears)
Check if fruit is empty : false
Check if nums is empty : true

Concatenating listsSearch on YouTube

Use the ::: operator, the ::: method or List.concat. All three join two or more lists.

Worked example. fruit1 = List("apples", "oranges", "pears"), fruit2 = List("mangoes", "banana").

var fruit = fruit1 ::: fruit2
println("fruit1 ::: fruit2 : " + fruit)
fruit = fruit1.:::(fruit2)
println("fruit1.:::(fruit2) : " + fruit)
fruit = List.concat(fruit1, fruit2)
println("List.concat(fruit1, fruit2) : " + fruit)

Output.

fruit1 ::: fruit2 : List(apples, oranges, pears, mangoes, banana)
fruit1.:::(fruit2) : List(mangoes, banana, apples, oranges, pears)
List.concat(fruit1, fruit2) : List(apples, oranges, pears, mangoes, banana)

Order rule. ::: ends with a colon, so it binds to the right. fruit1.:::(fruit2) means fruit2 ::: fruit1. That is why the second line starts with mangoes.

SetsSearch on YouTube

A Scala Set holds pairwise different elements of one type:

var s: Set[Int] = Set()
var s: Set[Int] = Set(1, 3, 5, 7)
var s = Set(1, 3, 5, 7)

MapsSearch on YouTube

A Scala map holds key and value pairs. Any value is retrieved by its key. Keys are unique, but values need not be unique. Maps are also called hash tables. There are immutable and mutable kinds. An immutable object cannot change itself. By default Scala uses the immutable Map. For a mutable Map, import scala.collection.mutable.Map.

var A: Map[Char, Int] = Map()
val colors = Map("red" -> "#FF0000", "azure" -> "#F0FFFF", "peru" -> "#CD853F")
val nums: Map[Int, Int] = Map()

Worked example. Read keys, values and emptiness from colors above.

println("Keys in colors : " + colors.keys)
println("Values in colors : " + colors.values)
println("Check if colors is empty : " + colors.isEmpty)
println("Check if nums is empty : " + nums.isEmpty)

Output.

Keys in colors : Set(red, azure, peru)
Values in colors : MapLike(#FF0000, #F0FFFF, #CD853F)
Check if colors is empty : false
Check if nums is empty : true

Concatenate maps with ++ (operator or method). Duplicate keys collapse to one entry:

val colors1 = Map("red" -> "#FF0000", "azure" -> "#F0FFFF")
val colors2 = Map("blue" -> "#0033FF", "yellow" -> "#FFFF00")
var colors = colors1 ++ colors2
colors = colors1.++(colors2)

Print each key with its value:

colors.keys.foreach { i =>
  print("Key = " + i)
  println(" Value = " + colors(i))
}

The output is one line per pair: Key = red Value = #FF0000, then azure, then blue, then yellow.

TuplesSearch on YouTube

A Scala Tuple combines a fixed number of items so they travel as a whole. Unlike an array or list, a Tuple can hold objects with different types. Tuples are immutable. Access elements with t._1, t._2 and so on. Iterate with t.productIterator().

val t = (1, "hello", Console)
// or, with explicit syntax:
val t = new Tuple3(1, "hello", Console)

The first line is short syntax for the second.

Worked example. Sum the Tuple (4, 3, 2, 1).

val t = (4, 3, 2, 1)
val sum = t._1 + t._2 + t._3 + t._4
println("Sum of elements: " + sum)
sum=4+3+2+1=10\begin{aligned} sum &= 4 + 3 + 2 + 1\\ &= \mathbf{10} \end{aligned}

Iterate. t.productIterator.foreach { i => println("value = " + i) } prints value = 4, value = 3, value = 2, value = 1.

IteratorsSearch on YouTube

An iterator is not a collection. It is a way to access the elements of a collection one by one. The two basic operations on an iterator it are next and hasNext. A call to it.next() returns the next element and advances the iterator state. it.hasNext tells whether more elements remain.

Worked example. Walk Iterator("a", "number", "of", "words") to the end.

object Demo {
  def main(args: Array[String]): Unit = {
    val it = Iterator("a", "number", "of", "words")
    while (it.hasNext) {
      println(it.next())
    }
  }
}

Trace. Four elements, four loop passes. Pass 1 prints a, pass 2 prints number, pass 3 prints of, pass 4 prints words. Then hasNext is false and the loop stops.

Quick self-testSearch on YouTube

1. Differentiate var and val. Which one did the Demo program reassign?

var declares a mutable variable. val declares a fixed value. Neither was reassigned in the Demo program. data = 101 in the variables section shows a legal var reassignment.

2. The REPL shows val res1: Int = 1 for 10/6. Why 1 and not 1.67?

Both sides are Int, so Scala does integer division. The fractional part is dropped. 10/6=110/6 = 1 with remainder 4.

3. With a = 10, b = 20, c = 25, compute a * b, b % a and c % a.

a×b=200a \times b = 200. b mod a=20 mod 10=0b \bmod a = 20 \bmod 10 = 0. c mod a=25 mod 10=5c \bmod a = 25 \bmod 10 = 5.

4. numList is List(1 to 10). What does for { a &lt;- numList if a != 3; if a &lt; 8 } yield a return?

List(1, 2, 4, 5, 6, 7). The first guard drops 3. The second guard drops 8, 9 and 10. Six elements remain.

5. Array(1.9, 2.9, 3.4, 3.5): compute the total and the maximum.

Total=1.9+2.9+3.4+3.5=11.7Total = 1.9 + 2.9 + 3.4 + 3.5 = 11.7. Max=3.5Max = 3.5 (each element in turn beats the running maximum).

6. fruit1 is List(apples, oranges, pears) and fruit2 is List(mangoes, banana). What is the difference between fruit1 ::: fruit2 and fruit1.:::(fruit2)?

fruit1 ::: fruit2 is List(apples, oranges, pears, mangoes, banana). fruit1.:::(fruit2) means fruit2 ::: fruit1, which is List(mangoes, banana, apples, oranges, pears). Operators ending in a colon bind to the right.

7. What happens when two maps share a key and you join them with ++?

The duplicate key collapses to one entry. The slides state that adding maps removes duplicate keys.

8. Differentiate private and protected members. Where does each fail in the slide code?

private is visible only inside the defining class or object. (new Inner).f() fails. protected is visible only in subclasses. (new Super).f() in the unrelated Other class fails, while Sub can call f().

9. Location(10, 20, 15) calls move(10, 10, 5). What are the final x, y and z?

x=20x = 20, y=30y = 30, z=20z = 20. Each coordinate adds its own delta: 10+1010 + 10, 20+1020 + 10, 15+515 + 5.

10. Why is an iterator not a collection, and what ends the while loop over one?

An iterator only walks elements one by one with next and hasNext. It holds no elements itself. The loop ends when hasNext turns false after the last element.

  • Scala is an object-oriented and functional language by Martin Odersky (Java 2004, .NET June 2004). Spark is written in it.
  • The REPL answers each line at once (10/6 gives Int value 1). Scripts compile with scalac and run with scala.
  • var is mutable, val is fixed. Types can be inferred. A Tuple can assign several val variables at once.
  • Strings use length() and + or concat. Arrays use Array[T], indices, Array.ofDim for matrices and for loops for sums and maxima.
  • Arithmetic demo values: a+b=30a+b=30, a−b=−10a-b=-10, a×b=200a\times b=200, b/a=2b/a=2, b mod a=0b\bmod a=0, c mod a=5c\bmod a=5.
  • if-else picks a branch. while tests first, do-while tests last. for covers ranges, multiple ranges, collections and yield filters.
  • Classes need initialised fields. Location extends Point with override val (move gives 20, 30, 20). private stays inside its class, protected inside subclasses, else public.
  • Functions declare def name(params): returnType. Procedures return Unit. addInt(5, 7) returns 12.
  • Collections: List[T] (linked, immutable, head/tail, :::), Set (different elements), Map (unique keys, ++), Tuple (t._1, sum of (4,3,2,1) is 10), Option (zero or one), Iterator (next/hasNext).