Rust Learning: Modules

Overview

Rust modules is one of those things where I thought I learned it but it keeps giving me surprises.

I was recently contributing to an inner source rust project and got tripped by modules and realized that I don’t actually understand them.

So here I am trying to learn again and write things down.

I’ll go over the basics which explains the concept and syntax. Then I’ll show some idioms people use.

Might not be exhaustive but this is aligning with my understanding.

tldr

• Rust’s module tree is explicit, you have to write it yourself.
• mod is the keyword for you to build the module tree.
• pub mod abc declares module abc and makes it public.
• use is the keyword to “import” other modules, it creates shortcuts.
• pub use abc creates shortcuts and shares that with users of your module.
• To refer something in a sibling module, you need to use relative path super::sibling_name or absolute paths crate::path_to_sibling.

The Basics

Module Tree

I believe this is a rust concept, it means what module structure(tree) does your program has.

This isn’t a thing in languages like Python because in those languages the module tree maps to the file system.

Say in python I want to create my math package with four operators in it, I would do the following in mymath.py

mymath.py

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

# mymath.py

def add(a, b):
    return a + b

def minus(a, b):
    return a - b

def multiply(a, b):
    return a * b

def divide(a, b):
    return a / b

And in other places they could simply import mymath and use mymath.add() or from mymath import add then just use add()

In rust having mymath.rs is not enough, you need to explicitly tell the compiler you have a module.

mod keyword - to declare or define modules in the module tree

mod keyword is the keyword to declare or define modules in rust. Note that declare and define a module are slightly different.

When using the inline syntax mod defines the module, when using the file+folder way mod declares the module.

To define a module, there are three ways.

(1) Inline

src/main.rs

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

// src/main.rs
mod my_module {
    pub fn add(a: i32, b: i32) -> i32 {
        a + b
    }
}
fn main() {
    let three = my_module::add(1, 2);
    println!("Hello, world! {}", three);
}

In this way we defined and declared the module at the same time.

(2) File and Directory

The above feels like good tool to do ad-hoc things. Almost feels like in C++ where you open a namespace anywhere and starts adding things in it. A more natural way IMO would be using files and directories.

In rust there exists two ways, one of them being “legacy”, and the other one is the recommended way.

(2.1) [legacy] mod.rs + module_name/

You simply create a folder and name the folder name to be the module name.

Then you would need to have a mod.rs in the folder. You would use mod.rs to declare which modules are available to the outside world.

It could also re-export some internal modules to a different path. We’ll go over re-export in the use section.

folder structure

# folder structure
.
├── Cargo.lock
├── Cargo.toml
└── src
    ├── main.rs
    └── my_module
        ├── mod.rs
        └── operations.rs

3 directories, 5 files

src/my_module/mod.rs

1
2
3
4

// src/my_module/mod.rs

// this matches the file name operations.rs
pub mod operations;

src/my_module/operations.rs

1
2
3
4
5
6

// src/my_module/operations.rs

// pub here is needed otherwise this would be a private function
pub fn add(a: i32, b: i32) -> i32 {
    a + b
}

src/main.rs

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

// src/main.rs

// this is to declare the module
// module name is the directory name
mod my_module;

fn main() {
    let three = my_module::operations::add(1, 2);
    println!("{}", three);
}

(2.2) [recommended] module_name.rs + module_name/

The above is nice but there is one catch. In a big project there would be a lot of mod.rs files. Later a new way was introduced, essentially pulling mod.rs out of the folder and name it same as the folder (module name).

folder structure

# folder structure

.
├── Cargo.lock
├── Cargo.toml
└── src
    ├── main.rs
    ├── my_module
    │   └── operations.rs
    └── my_module.rs

3 directories, 5 files

I essentially pulled mod.rs out and renamed it. The rest is the same but for completeness sake I’ll put them below.

src/my_module.rs

1
2
3
4

// src/my_module.rs

// this matches the file name operations.rs
pub mod operations;

src/my_module/operations.rs

1
2
3
4
5
6

// src/my_module/operations.rs

// pub here is needed otherwise this would be a private function
pub fn add(a: i32, b: i32) -> i32 {
    a + b
}

src/main.rs

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

// src/main.rs

// this is to declare the module
// module name is the directory name
mod my_module;

fn main() {
    let three = my_module::operations::add(1, 2);
    println!("{}", three);
}

Visibility: public or private

You might have noticed that in the code example above there are some pub keywords here and there.

Everything in rust is defaulted to be private. Adding pub simply makes the declared module public and can be accessed outside.

Every module can have child modules, and in order to use something from the child module, the child module must make it visible for the parent.

Let’s look at a concrete example.

Say we have my_module and inside my_module we have operations, let’s say we need some utility functions that are only available for operations but we don’t want it to be exposed to users of my_module.

We would (1) create a sub module under operations, mark the functions as pub and then (2) declare the module in places we need to use. and (3) do not expose it in operations.rs

.
├── Cargo.lock
├── Cargo.toml
└── src
    ├── main.rs
    ├── my_module
    │   ├── operations
    │   │   └── utils.rs
    │   └── operations.rs
    └── my_module.rs

4 directories, 6 files

I’ll only pick the interesting files here

src/my_module/operations.rs

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

// src/my_module/operations.rs

// modules_playground::my_module::operations::utils
// this is to declare `utils` in the module tree.
// `utils` is a child of `operations`
mod utils;

pub fn add(a: i32, b: i32) -> i32 {
    utils::utility_func();
    a + b
}

src/my_module/operations/utils.rs

1
2
3
4

// src/my_module/operations/utils.rs
pub fn utility_func() {
    println!("I'm a utility function in my_module::operations::utils module");
}

The pub on utility_func in utils.rs is required, otherwise the function won’t be accessible in operations.rs.

The mod utils; in operations.rs is important, because it brought utils into the module tree. Mounted utils as a child of operations.

Note that in main.rs there is no access to utils::utility_func because it was not made public in operations.rs.

To make utils::utility_func available in main.rs, we could write pub mod utils instead of mod utils in operations.rs. The line pub mod utils does two things.

1. It mounts the module to the module tree.
2. It makes the module public for whoever is mounting the current module. (operations) in this case.

It can be confusing because sometimes when slapping on the pub keyword, a single line does two things.

There are also other fine grained control of visibility like pub(crate), pub(super) but we’ll go over them later.

Using modules from sibling

There is a tool called cargo modules, using that tool you can check the module tree.

Running cargo modules structure at the root of the project shows the following output

crate modules_playground
├── fn main: pub(crate)
└── mod my_module: pub(crate)
    └── mod operations: pub
        ├── fn add: pub
        └── mod utils: pub(self)
            └── fn utility_func: pub

The utils module we created is under operations, what if I want to have a utils module under my_module and share it across my_module?

In order to do this, let’s define the utils module under my_module by creating a utils.rs file inside my_module/ directory. Then in my_module.rs we need to declare it with mod utils;

.
├── Cargo.lock
├── Cargo.toml
└── src
    ├── main.rs
    ├── my_module
    │   ├── operations.rs
    │   └── utils.rs
    └── my_module.rs

3 directories, 6 files

src/my_module/utils.rs

1
2
3
4
5
6

// src/my_module/utils.rs

// this needs to be `pub` because it's used with other modules inside `my_module`
pub fn utility_func() {
    println!("I'm a utility function in my_module::operations::utils module");
}

src/my_module.rs

1
2
3
4
5

// src/my_module.rs
pub mod operations;

// we do not need to have `pub` here because this `utils` is private for my_module
mod utils;

And in order to use it from operations.rs, you need to either call it with a relative path, or call it with an absolute path.

To call it with a relative path, operations needs to go up to my_module then go down to utils.

To call it with an absolute path, operations can just specify the path from the root.

Let’s see both in action src/my_module/operations.rs

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

// src/my_module/operations.rs

pub fn add(a: i32, b: i32) -> i32 {
    // relative path
    super::utils::utility_func();

    // absolute path
    crate::my_module::utils::utility_func();
    a + b
}

Visibility: all pub keywords

So now we know that the mod keyword is to declare/define modules. And the pub keywork is to tweak the visibility. There are some variants of pub, I’ll list them below

• pub, public everywhere
• pub(crate), public in your current crate
• pub(super), public to your parent(no grandparents) and sibling and kids of siblings(nephews/nieces)
• pub(in path::to::module), public to the specific module path
• pub(self) or ignored, public to yourself, essentially private

pub(super) granting access to nephews/nieces seems strange but it’s because of the following rule.

When you grant visibility to a module, you automatically grant visibility to every single module nested inside of it.

mod Summary

So we know that mod keyword declares/defines a module in the module tree. And pub keyword can change the visibility of the module. If you write them in the same line it does two things with one line.

pub mod abc; means declaring abc in the current module and make it public to whoever uses the current module.

use Keyword

In the last example above (calling sibling modules), we saw how to call the function with different paths

src/my_module/operations.rs

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

// src/my_module/operations.rs

pub fn add(a: i32, b: i32) -> i32 {
    // relative path
    super::utils::utility_func();

    // absolute path
    crate::my_module::utils::utility_func();
    a + b
}

However it would be tedious to keep writing the path.

use keyword could be used to shorten the path. It feels similar like import in Python, or use in C++.

It’s easier to think that use creates shortcuts.

We could use till the module and call the function like utils::utility_func(). Similar to import utils in Python.

src/my_module/operations.rs

1
2
3
4
5
6
7

// src/my_module/operations.rs
use super::utils;

pub fn add(a: i32, b: i32) -> i32 {
    utils::utility_func();
    a + b
}

Or we could use the full path and call utility_func() without utils::. Similar to from utils import utility_func in Python.

src/my_module/operations.rs

1
2
3
4
5
6
7

// src/my_module/operations.rs
use super::utils::utility_func;

pub fn add(a: i32, b: i32) -> i32 {
    utility_func();
    a + b
}

pub use, aka the Facade Pattern

pub use is a way to share the shortcut use created. It’s known as re-export.

Let’s look at a deeply nested module tree example.

.
├── Cargo.lock
├── Cargo.toml
└── src
    ├── main.rs
    ├── my_module
    │   ├── aaa
    │   │   └── bbb.rs
    │   └── aaa.rs
    └── my_module.rs

4 directories, 6 files

crate modules_playground
├── fn main: pub(crate)
└── mod my_module: pub(crate)
    └── mod aaa: pub
        └── mod bbb: pub
            └── fn bbb_func: pub

Use this as an exercise to see if you can fill out what is needed in each file to create the module tree like this.

src/my_module.rs

1
2

// src/my_module.rs
pub mod aaa;

src/my_module/aaa.rs

1
2

// src/my_module/aaa.rs
pub mod bbb;

src/my_module/aaa/bbb.rs

1
2

// src/my_module/aaa/bbb.rs
pub fn bbb_func() {}

src/main.rs

1
2
3
4
5
6

// src/main.rs
mod my_module;

fn main() {
    my_module::aaa::bbb::bbb_func();
}

In main.rs, it is clear that bbb_func() is deeply nested, we could use pub use inside my_module.rs to create a shortcut and make that shortcut available, so clients (main.rs) don’t really need to know about this aaa::bbb:: module tree structure.

src/my_module.rs

1
2
3

// src/my_module.rs
mod aaa; // still need to declare `aaa`, but no need to make it public
pub use aaa::bbb; // creating the shortcut and sharing it

src/main.rs

1
2
3
4
5
6
7

// src/main.rs
mod my_module;

fn main() {
    // using the shortcut created in `my_module.rs`
    my_module::bbb::bbb_func();
}

If we wanted we could go one step further to create a shortcut of bbb_func on my_module.

src/my_module.rs

1
2
3

// src/my_module.rs
mod aaa; // still need to declare `aaa`, but no need to make it public
pub use aaa::bbb::bbb_func; // creating the shortcut and sharing it

src/main.rs

1
2
3
4
5
6
7

// src/main.rs
mod my_module;

fn main() {
    // using the shortcut created in `my_module.rs`
    my_module::bbb_func();
}

The Idioms

The pub use above is one of the commonly used idiom/pattern. Let’s go over a few that I saw. This might not be exhaustive but you gotta start somewhere.

using type vs using functions

In my understanding, the convention for functions is to have the module name, the convention for types is to import the full path.

1
2
3
4
5
6
7
8

use super::utils;
use super::utils::MyType;

fn foo() {
    utils::utility_func();

    let x: MyType = 0;
}

Prelude

Some packages have stuff that most people would like to use anyways, so they export them in a prelude module and users would use *. Examples like use rayon::prelude::*;, use bevy::prelude::*;, use std::io::prelude::*;

Facade Pattern

We already covered this when we go over pub use. This is essentially creating shortcuts and share them with the users of your module.

The pattern I saw

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

mod my_module {
    pub struct Type1{}
    pub struct Type2{}

    pub mod my_module {
        pub use super::Type1;
        pub use super::Type2;
    }
}

pub use self::my_module::*; // without `self` there would be ambiguity, the `my_module` in this line is referring to the outer one

I saw this code when working on the inner source project.

I was super confused then I learned that the reason the modules are structured like this, is to give clients the ability to either refer Type1 or refer it with my_module::Type1.

The gotcha I got was the last line didn’t have self::, so it had ambiguity where rust didn’t know which my_module it is referring to. Because the outer sibling module and the nested module share the same name, omitting self:: causes ambiguity for the compiler over which my_module is being targeted. Adding self:: explicitly tells the compiler to look at the local module in the current scope (the outer one), fixing the ambiguity.

Rust Modules are Completely Closed

Rust modules feels like C++’s namespaces (probably because of the use keyword). C++ namespaces could be “opened” anywhere and have things added to it. It then begs the question, can rust modules be “opened” anywhere and have things added to it?

No, modules are completely closed in rust.