Rust Learning: Deref

tldr

• deref() returns reference because
  • Rust desugars *a to *a.deref()
  • Rust needs the size at compile time
  • We don’t want to consume by calling deref()
• deref coercion can happen
  • on function parameters
  • on method call expressions
  • on assignments with type annotations
• as_ref() and as_deref() always return references (or containers of references), which always implement Copy

Overview

I was practicing rust by doing this and this.

While chatting with Claude and trying to learn, I saw that there is this as_deref() call in the iterator chain.

This triggered some memory (trauma) about Deref and I decided to give it another shot with the help of Claude.

The following is (as usual), what I learned from teacher Claude.

The Confusion

When I first saw Deref trait, my instinct/past experience is telling me this is an “interface” that you could implement on your custom type so when you do *my_instance it triggers the specific “dereference” logic and gives you the underlying value.

Then if we look at Deref’s definition here, you can tell the required method is returning a reference instead of a concrete.

1
2
3
4
5
6

pub trait Deref {
    type Target: ?Sized;

    // Required method
    fn deref(&self) -> &Self::Target; // <----- here, returning a reference instead of Self::Target
}

So to summarize my confusion: Why is deref() in Deref trait returning a reference when it’s “dereferencing”?

The Answer

Apparently in Rust, when you call * on a type that implemented the Deref trait it would call deref() then still slap on the * operator

1
2
3
4
5

let boxed_value = Box::new(7);

println!("{}", *boxed_value);
// *boxed_value would be converted to *boxed_value.deref()
// where the desugar is *&i32, which gives i32

But Why?

The above seems to make sense but why can’t the deref() function return Target instead of &Target?

Claude explained that it’s because

1. Target might not be Sized, so its size might not be known at compile time. In Rust, types must be Sized to be passed or returned by value because the compiler needs to know how much stack space to allocate. For types like str or [T], they must live behind a pointer (like &str).
2. Returning Target would move it out from the containing struct, causing issues.

Deref Coercion

Now since we are reading/learning about Deref, the term Deref Coercion would be shown and we would be confused.

Deref Coercion in function parameters

Let’s look at an example below.

String implements Deref and its Target is str, its deref() returns &str.

yo() takes &str but passed &String, Rust would do a implicit conversion (coercion) from &String to &str.

1
2
3
4
5
6

fn yo(s: &str) {
    println!("{}", s);
}

let name = String::from("hi");
yo(&name); // here &String is coerced to &str

Note that Rust would do this as many times as needed. So if we have a Box that has a String in it, this would work the same.

1
2
3
4
5

fn yo(s: &str) {
    println!("{}", s);
}
let boxed_name = Box::new(String::from("hi"));
yo(&boxed_name); // Box<String>.deref() gets &String then String.deref() gets &str

Deref Coercion in method call expressions

Deref Coercion not only happens when passing function arguments, but also happens when we call methods.

1
2
3
4

let name = String::from("hi");

name.contains("h");
// .contains(&self, pat: P) exists on `str` but not `String`

The method call coercion doesn’t require you to add & yourself because in Method-call expressions the derefs are being considered.

The first step is to build a list of candidate receiver types. Obtain these by repeatedly dereferencing the receiver expression’s type, adding each type encountered to the list, then finally attempting an array unsized coercion at the end, and adding the result type if that is successful.

Deref Coercion in assignments with type annotations

Deref Coercion could also happen when doing an assignment with type annotations. This is kinda self explanatory so I’ll just skip.

1
2

let name = String::from("hi");
let s: &str = &name; // &String coerced to &str on assignment

1
2

let boxed_name = Box::new(String::from("hi"));
let s: &str = &boxed_name; // &Box<String> coerced to &String coerced to &str on assignment

as_deref() on Option

If you still remember, the reason I started this post is I saw this as_deref call in the iterator chain and got me thinking.

But all the discussions above don’t really help, do they?

Although the place where I was confused is on Vec, later practice shows that it might be more beneficial to understand as_deref on Options.

What does as_deref() on Option do?

Let’s look at as_ref() first

To speak about as_deref() on Options let’s have a look at as_ref() first.

Option’s as_ref says it

Converts from &Option to Option<&T>.

So given &Option<String> I can get an Option<&String>.

Yeah yeah sure sure, but what exactly does this mean?

Think about the following, if I have an Option<String>, and I have another function that takes in a &String. Is there a way for me to pass the String in the Option as a borrow to the function without consuming the original Option?

as_ref() is the answer, because it converts &Option<String> to Option<&String>, this gives you the ability to look inside the option without consuming it.

Let’s have a look at an example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

fn yo(s: &String) {
    println!("{}", s);
}

let value = Some("stuff".to_owned());
if let Some(inner) = value {
    yo(&inner);
    // value is consumed here, no longer available
    // inner's type is actually String, hence the need to borrow before passing to `yo`
}

let another = Some("other stuff".to_owned());
if let Some(inner) = another.as_ref() { // as_ref() takes &self, so another is borrowed here instead of consumed
    yo(inner);
    // here `inner`'s type is actually &String
}
// `another` is still available

Now deref() should feel natural

What if the function above requires &str instead of &String?

here is where as_deref() comes into play, it not only does what as_ref() does, it also calls deref() as it sees fit to do the conversions.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

fn yo(s: &str) {
    println!("{}", s);
}

let another = Some("other stuff".to_owned());
if let Some(inner) = another.as_deref() {
    // inner's type is actually `&str`
    yo(inner);
}
// another is still available here

Granted the as_deref() is not strictly needed in this specific case because &String would be coerced to &str anyways.

Then the natural question is, what is the case where I absolutely needs as_deref() over as_ref()?

The answer is when yo() requires Option<&str>

1
2
3
4
5
6
7
8
9

fn yo(x: Option<&str>) {
    println!("{:?}", x);
}

let another = Some("other stuff".to_owned());
yo(another.as_deref());
// another is still available
// side note: yo does consume the option but as_deref() provided a copy
// as_ref() and as_deref() always return references (or containers of references), which always implement Copy

But How are these implemented?

Now we know why does as_ref() and as_deref() exist, when and why to use them. The next question is, how are they implemented?

How are as_ref() implemented?

Let’s have a look at Option’s as_ref()

1
2
3
4
5
6
7

// &self is a short hand of self: &Self
pub const fn as_ref(&self) -> Option<&T> {
    match *self {
        Some(ref x) => Some(x),
        None => None,
    }
}

1
2

let value = Some("stuff".to_owned());
let x = value.as_ref(); // x's type is Option<&String>

• value.as_ref() is being called
• as_ref(&self) takes &self, which is a short hand of as_ref(self: &Self)
• Which means Self is Option<String>
• Which means self’s type is &Option<String>

Now we are clear about the types of self and Self. Let’s go to the match

• We are matching *self (note that this could be written better with matching ergonomics)
• As we said, self’s type is &Option<String> so matching *self means matching Option<String>
• Now we are looking at Some(ref x) matching on Option<String>
  • ref x in a pattern means “instead of moving the matched value, give me a reference to it”.
  • So we got a reference of String which is &String
  • And we returned it as Some(x) which is of type Option<&String>

1
2
3
4
5
6
7

// with matching ergonomics as_ref() could be written as
pub const fn as_ref(&self) -> Option<&T> {
    match self {
        Some(x) => Some(x), // because `self` is &Option<String>, `x` is automatically bound as &String
        None => None,
    }
}

How are as_deref() implemented?

Let’s see the code first.

1
2
3
4
5
6

pub const fn as_deref(&self) -> Option<&T::Target>
where
    T: [const] Deref,
{
    self.as_ref().map(Deref::deref)
}

And let’s use our example

1
2

let value = Some("stuff".to_owned());
let x = value.as_deref(); // x's type is Option<&str>

• value.as_deref() being called
• as_deref(&self) takes &self, which means self: &Self which means self’s type is &Option<String>
• self.as_ref() is called, we get Option<&String> back
• We call map with Deref::deref, so Option<&String> becomes Option<&str>
  • This is because String’s Deref Target is str
  • and deref() returns &Target so deref() returns &str

Bonus: DerefMut

DerefMut is the mut version of deref. If we have a &mut String, Rust can coerce it to &mut str if DerefMut is implemented.