Map Flatmap and Options

Table of Content (Click to Expand)

tldr

map creates a new box, flatMap expects you to create the box and pass it along
Use map when the passed in function does not return a box
Use flatMap when the passed in function returns a box
The real power of Option is to chain map, flatMap so you can change things in the box and pass to the next

Overview

Back in 2020 when I joined a web team and the servers were written in Scala. I spent some time learning Scala and got confused about map and flatMap. I think I understood it at that time but since I didn’t write it down after all these years it became vague in my mind. Recently I’m picking up Flink and flatMap came up again. So I decided to re-learn and write up something.

The Problem of Representing Optional Fields

Say we have a User struct in Go and it looks like the following

type User struct {
	ID      int
	Email   string
	Address *string
	ZipCode *string
}

What I am trying to express with this User struct is:

A User has 4 fields, ID, Email, Address, ZipCode.
Among them ID and Email is required, Address and ZipCode is optional.

However when using the User struct the compiler doesn’t force me to do a nil pointer check, so in my code I always need to keep in mind to do a nil check before I reference the Address.

Let’s see how this is solved as a “builtin” in some other languages

Option

Scala has this type called Option.

The User struct above in Scala could be defined as

case class User(
  id: Int,
  email: String,
  address: Option[String],
  zipCode: Option[String]
)

Note that both optional fields are of type Option[String], this means the type of address is an Option which wraps over a String.

When you try to access address, you get an Option, if it contains the actual address (meaning it does exist), the address is Some(String), if it does not contain the actual address (meaning it does not exist), the address is None.

Some code snippets might be helpful

// A user with all fields populated
val user1 = User(
  id = 1,
  email = "alice@example.com",
  address = Some("123 Main St"),
  zipCode = Some("07901")
)

// A user missing the optional fields
val user2 = User(
  id = 2,
  email = "bob@example.com",
  address = None,
  zipCode = None
)

With Option here the compiler forces me to handle the Option at compile time.

The Other Problem of Representing Optional Fields

Referring to the same User struct in Go

type User struct {
	ID      int
	Email   string
	Address *string
	ZipCode *string
}

There is another problem using this struct. Given a User instance which has a nil Address you can not tell if the user did not set it, or if the user explicitly sets it to nil.

Why does this matter?

Let’s assume that in the database we have a User table where its Address and ZipCode column are both nullable.

So when you get a User instance like the following to write to the database the intent is ambiguous.

user := User{
		ID:    1,
		Email: "a@b.com",
	}

Both Address and ZipCode are nil. Should I set both of them to NULL in the database? Or should I leave the columns in the database alone? There isn’t a way to tell at this point.

The root cause is using a pointer in a struct is not enough to distinguish between

I intentionally set this field to nil
I did not provide this field so leave it alone

I made a big mistake here in the first version of this post.

I thought that Option[String] is going to solve the problem, where I could simply replace *string with Option[String]. But it does not.

Because what I needed is to express 3 states for a given field, using Address as an example. I want to know if the user

Value provided: Update the field to "123 Main St".
Intentionally set to NULL: The user wants to clear/delete their address.
Not provided (leave it alone): The user is only updating their Email, so they didn’t include an Address in the request.

Without creating custom abstractions, using Go to solve this problem requires doing a pointer to pointer

type User struct {
	ID      int
	Email   string
	Address **string
	ZipCode **string
}

Now we have a way to map our 3 states:

Not provided: The field is nil
Intentionally set to NULL: The field is a pointer to nil (*Address == nil)
Value provided: The field is a pointer to a pointer to a string (**Address == "123 Main St")

And because the compiler does not force you to check for nil before deref, it’s harder to maintain code using the struct above.

With Option in say Scala (without creating custom abstractions), we would mark the field as Option[Option[String]]. And because the compiler forces us to handle None cases, even though the code might look a little ugly we solved this problem at compile time.

case class User(
  id: Int,
  email: String,
  address: Option[Option[String]],
  zipCode: Option[Option[String]]
)

With nested Options, the 3 states are cleanly mapped:

Not provided: None
Intentionally set to NULL: Some(None)
Value provided: Some(Some("123 Main St"))

Basic Usages

The Option type has a method to tell if it was defined or not.

// anti-pattern code
val user2 = User(
  id = 2,
  email = "bob@example.com",
  address = None,
  zipCode = None
)

// code to print user2.address if was defined

if (user2.address.isDefined) {
    val addressString = user2.address.get
    println(s"$addressString")
}

The above is not how you use Option, imagine you need to get the user’s address and pass it to a service that takes in the address and returns the zipcode. Your code would be like

def getZipCodeByAddress(addr: String): Option[String] = {
    // A zipcode is not guaranteed by an address
    // Hence returning an Option[String]
}

// anti-pattern code
if (user2.address.isDefined) {
  // .get unwraps the Option, returning the raw String
  val addressString = user2.address.get
  val zipCodeOption = getZipCodeByAddress(addressString)
  if (zipCodeOption.isDefined) {
      val zipCode = zipCodeOption.get
      println(s"$zipCode")
  }
}

The code above works but it’s the typical example of “I’m from a certain language, I’m gonna write everything my way”.

In Scala there are multiple ways to write the above such as for-comprehension, but I want to focus on the less syntax sugary ones.

Pattern Matching (Not the Machine Learning One)

Given an Option, we could do Pattern Matching to match against an Option, this also exists in Rust.

// pattern matching

val someAddress = Some("123 Main St")

val displayAddress = someAddress match {
  case Some(address) => s"address is $address"
  case None          => "No address"
}

The above is fine but imagine the following.

Given an optional address, if the address exists call another function that returns an optional zipcode, and then print the zipcode if it exists.

// nested pattern matching

val someAddress = Some("123 Main St")

// Assuming getZipCodeByAddress returns an Option[String]
val displayMessage = someAddress match {
  case Some(address) =>
    getZipCodeByAddress(address) match {
      case Some(zipCode) => s"Found zip code: $zipCode"
      case None          => "Address exists, but zip code lookup failed"
    }
  case None =>
    "No address"
}

println(displayMessage)

This starts to feel very off, let’s see how map and flatMap could help.

map and flatMap

map

If you imagine Option as a box, in the examples above, we were trying to open (unwrap) the box every time.

What if there is a way that allows you to not open the box but change the thing?

map and flatMap is the tool.

Let’s start with map, what map does is it opens the box, applies the function you provided to the item, then creates a new box putting the new stuff in there.

1
2
3

// map for Option

def map[B](f: A => B): Option[B]

The way to read this is:

map is a function that has a type parameter B, it takes in a function f and returns an Option[B].
f takes in an argument of type A and returns a value of type B.

Say we have an Option[String], like Some("yo"), we could use map modify the box’s content from "yo" to "sup"

val x = Some("yo")

def sup(greet: String): String = {
    if (greet == "yo") "sup" else "hello"
}

val y = x.map(sup) // y's type is Option[String]

Note that using map we opened the Option[String] box, used the sup function to convert "yo" to "sup" and wrapped it in a new Option[String] box

Another example is to open the Option[String] box, get the length of the String and put the Int in a new box.

val x = Some("yo")

def getLength(s: String): Int = {
    s.length
}

val y = x.map(getLength) // y's type is Option[Int]

flatMap

1
2
3

// flatMap for Option

def flatMap[B](f: A => Option[B]): Option[B]

The way to read this is:

flatMap is a function that has a type parameter B, it takes in a function f and returns an Option[B].
f takes in an argument of type A and returns a value of type Option[B].

val x = Some("yo")

def maybeSup(greet: String): Option[String] = {
    if (greet == "yo") Some("sup") else None
}

val sup = x.flatMap(maybeSup) // sup's type is Option[String]

Note that using flatMap we opened the Option[String] box, used the maybeSup function to convert "yo" to Some("sup") and returned it.

map vs flatMap

If you look closer to the above examples you might feel confused, more specifically when you put map and flatMap side by side you feel like finding waldo.

The key difference is that map’s function takes in A and returns B, whereas flatMap’s function takes in A and returns Option[B].

Had we used map with maybeSup, the result would be Option[Option[String]]

val x = Some("yo")

def maybeSup(greet: String): Option[String] = {
    if (greet == "yo") Some("sup") else None
}

val sup = x.map(maybeSup) // sup's type is Option[Option[String]] because we used map

Hence the name “flat”, where it flattens nested Options

Observations

map unwraps Option, applies the function, then creates a new Option
flatMap unwraps Option, applies the function, and returns what the function is returning, since the function is required to return an Option, flatMap does not create an Option

The above could also be summarized as

When the function you want to apply does not return Option, use map.
When the function you want to apply returns Option[T], use flatMap.

But wait isn’t this the same on Lists

So far we have been talking about map and flatMap on Options, but for most people map and flatMap feel more natural on List.

The following applies a double function on each element in xs.

// map
val xs = List[Int](1, 2, 3)

def double(x: Int): Int = {
    x * 2
}

val ys = xs.map(double) // ys = List(2, 4, 6)

The following flattens nested Lists without modifying the elements

// flatMap

val xss: List[List[Int]] = List(List(1, 2), List(3), List(4, 5), List(6, 7))

def f(xs: List[Int]): List[Int] = {
    xs
}

val xs = xss.flatMap(f) // xs = List(1, 2, 3, 4, 5, 6, 7)

To be honest the above example isn’t a good one, I picked it because it demonstrates “flatten” a nested list to a single list.

It is confusing because f’s signature is List[Int] to List[Int] whereas flatMap for Option takes A to Option[B].

The reason this is correct is because the A that f takes is the List[Int] inside List[List[Int]].

The following scratch pad might help

List[List[Int]].flatMap(f: List[Int] => List[Int]): List[Int]

List[A].flatMap(f: A => List[B]): List[B]

A == List[Int]
B = Int

We can pick another example where I have a list of space separated strings and I want to return a List[String]

val sentences = List(
  "Option is a Monad",
  "Scala is strictly typed"
)

def splitWords(sentence: String): List[String] = {
  sentence.split(" ").toList
}

val words = sentences.flatMap(splitWords) // words is of type List[String]

// flatMap(f: String => List[String]): List[String]

The point I am trying to deliver is, map and flatMap work the same way on Option and List. Because Option and List both wrap around the inner type and support some specific operations.

Option in Rust

Rust uses Option a lot, let’s see how Rust does pattern matching for Option

let some_num = Some(123);

// if some_num contains a number, plus one to it
// else return 0 as default
let y = match some_num {
    Some(num) => num + 1,
    None => 0,
};

Rust also supports this if let syntax

let some_num = Some(123);

if let Some(num) = some_num {
    println!("{num}");
}

But in the section above we know that we could use map and flatMap to work with Option. How does it work in Rust?

map and flatMap (and_then) in Rust

The full doc is here.

I’ll point out that in Rust flatMap is called and_then.

It doesn’t feel like they are the same, but the signature of and_then is the same as flatMap

1
2
3

pub fn and_then<U, F>(self, f: F) -> Option<U>
where
    F: FnOnce(T) -> Option<U>,

f takes in T and returns Option<U>

It might feel a bit off to associate and_then with flatMap, but the way Rust handles its functions to work with types like Option focuses on boolean logic, whereas Scala focuses on the shape of the data structure.

It feels more natural when we use and_then on Option.

Let’s go back to the first example where we have address: Option<String> and try to use get_zipcode_by_address(addr: &str) -> Option<&str>

let some_address = Some("123 Main St");

fn get_zipcode_by_address(addr: &str) -> Option<&str> {
    Some("07901")
}

let some_zipcode = some_address.and_then(get_zipcode_by_address);
// some_zipcode's type is `Option<&str>`

The and in and_then checks to see if the Option contains an address; if it does, it passes it to the function and returns Option<&str>. Rust’s methods to work with Option focus on boolean logic. See doc here.

or_else is also interesting, it applies f when Option is None.

For completeness’s sake, I’ll add the map example with Rust here as well

let x = Some("yo");

fn sup(greet: &str) -> String {
    if greet == "yo" {
        return String::from("sup");
    }
    return String::from("hello");
}

let y = x.map(sup); // y's type is Option<String>

If you think about it, doing an and_then on a Vec doesn’t really make sense. Hence Rust actually uses flat_map on Vec. There is no and_then on Vec in Rust, even though they are essentially the same thing.

The example where we have a list of space separated strings and I want to return a Vec<&str> would be

let sentences = vec![
    "Vec is a Monad",
    "Rust is strictly typed",
];

fn split_words(sentence: &str) -> Vec<&str> {
    return sentence.split(" ").collect();
}

// Note: Rust uses into_iter().flat_map for collections instead of and_then
let words: Vec<&str> = sentences.into_iter().flat_map(split_words).collect();

Summary

What have we gone through so far

We started from Go to explain why nil is not enough and Option is needed.
Then went to Scala to learn about Option and Pattern Matching
Then learned about map and flatMap, what is the difference
Then went to Rust to see map and and_then in Rust

Option, List are called Monads

Here is how Haskell defines a Monad, tbh I can’t fully follow it.

My simpler (loose) version is the following.

A Monad is a type that wraps around another type that supports an interface/trait

The wrapper function to take the inner type and return the Monad type. Some("stuff")
Implements flatMap, e.g. Option[String].flatMap[B](f: String => Option[B]): Option[B]
Three rules, see Haskell doc here
- Left Identity
- Right Identity
- Associativity

Further things to think about

Future is also a Monad in both Rust and Scala
Although Promise in JavaScript feels like a Monad, it isn’t.
How other syntax sugars like async/await in Rust, for comprehension in Scala work.
Read Haskell’s Monad definition

Update: 2026-07-21

Tags:

<< MSB and LSB, MSb and LSb async await in asyncio >>