Task: Install Spark, Set up SBT and Scala practice
Part 1: A show-and-tell program
Install Java JDK 7 and Spark 1.0 as in the lecture and in the lecture notes
Set up sbt as in the lecture and in the lecture notes.
Write a StateDiagram.scala program. Compile and run it using sbt.
In today’s lecture notes, we have shown you an example of using If-else control structure to produce an automatically generated story line. The story is randomly generated based on the following state diagram.
COSC 589 - Web Search and Sense-Making
The program would produce a story like:
Each time we run the program, it writes out a different story due to the randomness in the state diagram.
COSC 589 - Web Search and Sense-Making
Your task is to complete the story for a bigger graph, also using control structures. Your program should also be able to generate random stories based on this state diagram.
Note: Your story should be truly random and obey the probability in the diagram.
Part 2: Scala Practice
During lectures on Scala, we have learned function literals and closures. Please run the following exercises. For each of them, 1) screen capture the output on the screen, 2) ask a good question about the code 3) and provide your own answer to it. Your questions and answers will both be graded based on their quality and difficulty levels. Submit the screen captures in images. Submit the questions and answers in pdf.
1)
val num1 = 1 to 15
var num2 = 2
num1.foreach(num2 += _)
println(num2)
2)
COSC 589 - Web Search and Sense-Making
val strs = List (“test”, “mind-swap”, “swap-mind”)
def searchFrom(i: Int): Int =
if (i = strs.length) -1
else if (strs(i).startsWith("-")) searchFrom(i + 1)
else if (strs(i).endsWith(".scala")) i
else searchFrom(i + 1)
val i = searchFrom(0)
3)
def makeRowSeq(row: Int) =
for (col <- 1 to 10) yield {
val prod = (row * col).toString
val padding = " " * (4 - prod.length)
padding + prod
}
def makeRow(row: Int) = makeRowSeq(row).mkString
def multiTable() = {
val tableSeq =
for (row <- 1 to 10) yield makeRow(row)
tableSeq.mkString("\n")
}
4)
val f = _ + _
val f = (_: Int) + (_: Int)
f(5, 10)
5)
def sum(a: Int, b: Int, c: Int) = a + b + c
val a = sum _
a(1, 2, 9)
Part 3: Download Wikipedia Dump
Requirements: 100GB free disk space on your machine.
COSC 589 - Web Search and Sense-Making
Download the latest English Wikipedia dump. Follow the steps in the lecture notes to download the English Wikipedia Dump at http://dumps.wikimedia.org/enwiki/latest/enwiki-latest-pages-articles-multistream.xml.bz2
Uncompress the Wikipedia dump file and show the first 20 lines of the file on your screen, by following the instructions given in the lecture notes.
Make a screen capture of your results
What to Submit:
Part 1:
Your code
Screen captures of at least 4 random runs of the results
Part 2:
Screen captures of your outputs
Questions and answers in PDF
Part 3:
- Screen captures of the results
Where to submit:
Canvas
Note:
Your screen captures should include your screen name (better to show your name or netid) to show that they are captured from your computer.
