Queries with For

The for notation is essentially equivalent to the common operations of query languages for databases.

Example

Suppose that we have a database books, represented as a list of books.

case class Book(title: String, authors: List[String])

val books: List[Book] = List(
    Book(title = "Structure and Interpretation of Computer Programs",
        authors = List("Abelson, Harald", "Sussman, Gerald J.")),
    Book(title = "Introduction to Functional Programming",
        authors = List("Bird, Richard", "Wadler, Phil")),
    Book(title = "Effective Java",
        authors = List("Bloch, Joshua")),
    Book(title = "Java Puzzlers",
        authors = List("Bloch, Joshua", "Gafter, Neal")),
    Book(title = "Programming in Scala",
        authors = List("Odersky, Martin", "Spoon, Lex", "Venners, Bill")))

Usage:

// find the titles of books whose author’s name is "Bird":
for (
    b <- books; 
    a <- b.authors 
    if a startsWith "Bird,"
) yield b.title

// find all the books which have the word "Program" in the title:
for (
    b <- books 
    if b.title indexOf "Program" >= 0
) yield b.title

// find the names of all authors who have written at least two books present in the database:
for {
    b1 <- books
    b2 <- books
    if b1 != b2
    a1 <- b1.authors
    a2 <- b2.authors
    if a1 == a2
} yield a1

Distinct

In the last example, the output is:

> List(Bloch, Joshua, Bloch, Joshua)

This is since we flatly write out all authors who have authored atleast 2 books. So each author is added in the output each time he/she is encountered. How can we avoid this?

This can be resolved by using the distinct method on the sequence: // find the names of all authors who have written at least two books present in the database:

{
    for {
        b1 <- books
        b2 <- books
        if b1 != b2
        a1 <- b1.authors
        a2 <- b2.authors
        if a1 == a2
    } yield a1
}.distinct

or better yet, use sets.

{
    for {
        b1 <- books
        b2 <- books
        if b1 != b2
        a1 <- b1.authors
        a2 <- b2.authors
        if a1 == a2
    } yield a1
}.toSet
// Also can convert the books list to a set and use that instead, since the output of for will also be a set instead of a list

Generalization of For

As we saw in the last lecture, for is an implementation of map,flatMap and filter/withFilter.

Besides lists, sequences and even collections,you can use for on your own types as well – you must only define map, flatMap and withFilter for these types. There are many types for which this is useful: arrays, iterators, databases, XML data, optional values, parsers, etc.

For example, books might not be a list, but a database stored on some server. As long as the client interface to the database defines the methods map, flatMap and withFilter, we can use the for syntax for querying the database.

In the next lectures we will see how these 3 functions can be defined for: Generator, Option, try, which allows us to use for expressions on them.

This is the basis of the Scala data base connection frameworks ScalaQuery and Slick. Similar ideas underly Microsoft’s LINQ.