title	subtitle	date	draft	tags
Another one bites the Rust	Lessons learned from my first Rust project	2019-08-25	false	dev programming rust

Rust, the (re)discovery 🗺️

Back to 2014, the year I wrote my first Rust program ! Wow, that was even before version 1.0 !

At my school, it was all about C, and 10 years of C is quite boring (because, yes, I started programming way before my engineering school). I wanted some fresh air, and even if I really liked Python back to that time, i'm more into statically typed programming languages.

I can't remember exactly what was my first contact with Rust, maybe a blog post or a reddit post, and my reaction was something like :

Now, let's dig some reasons about why Rust blows my mind.

Rust is a programming langage focused on safety, and concurrency. It's basically the modern replacement of C++, plus, a multi-paradigm approach to system programming development. This langage was created and designed by Graydon Hoare at Mozilla and used for the Servo browser engine, now embedded in Mozilla Firefox.

This system try to be as memory safe as possible :

• no null pointers
• no dandling pointers
• no data races
• values have to be initialized and used
• no need to free allocated data

Back in 2014, the first thing that come to my mind was :

Yeah, yeah, just yet another garbage collected langage

And I was wrong ! Rust uses other mechanisms such as ownership, lifetimes, borrowing, and the ultimate borrow checker to ensure that memory is safe and freed only when data will not be used anywhere else in the program (ie : when lifetime is over). This new memory management concepts ensure safety AND speed since there is no overhead generated by a garbage collection.

For me, as a young hardcore C programmer¹, this was literally heaven. Less struggling with calloc, malloc, free and valgrind, thanks god Mozilla ! Bless me !

But, I dropped it in 2015. Why ? Because this was, at this time, far from perfect. Struggling with all the new concepts was quite disturbing, add to that cryptic compilation errors and you open a gate to brainhell. My young me was not enough confident to learn and there was no community to help me understand things that was unclear to me.

Five years later, my programming skills are clearly not at the same level as before, learning and writing a lot of Golang and Javascript stuff, playing with Elixir and Haskell, completely changed how I manipulate and how I visualize code in day to day basis. It was time to give another chance to Rust.

Fediwatcher 📊

In order to practice my Rust skill, I wanted to build a concrete and useful (at least for me) project.

Inspired by the work of href on fediverse.network my main idea was to build a small app to fetch metrics from various instances of the fediverse and push it into an InfluxDB timeseries database.

Fediwatcher was born !

The code is available on a github repo associated with my github account.

If you're interested, check out the Fediwatcher public instance.

Ok, ok, enough personal promotion, now that you get the main idea, go for the technical part and all the lessons learn writing this small project !

Cargo, compiling & building 🏗️

cargo is the Rust standard packet manager created and maintained by the Rust project. This is the tool used by all rustaceans and that's a good thing. If you don't know it yet, I'm also a gopher, and package management with go is a big pile of shit. In 2019, leaving package management to the community is, I think, the biggest mistake you can make when creating a new programming language. So go for cargo !

cargo is used for :

• downloading app dependencies
• compiling app dependencies
• compiling your project
• running your project
• running tests on your project
• publishing you project to crates.io

All the informations are contained in the Cargo.toml file, no need for a makefile makes my brain happy and having a common way to tests code without external packages is pretty straightforward and a strong invitation to test your code.

All the standard stuff also means that default docker images contains all you need to setup a Continous Integration without the need to maintain specific container images for a specific test suite or whatever. Less time building stuff, means more productive time to code. With Rust, all the batteries are included.

About compiling, spoiler alert, Rust project compiling IS SLOW :

{{< highlight sh >}} cargo clean time cargo build Compiling autocfg v0.1.4 Compiling libc v0.2.58 Compiling arrayvec v0.4.10 Compiling spin v0.5.0 Compiling proc-macro2 v0.4.30 [...] Compiling reqwest v0.9.18 Compiling fediwatcher v0.1.0 (/Users/wilfried/code/github/fediwatcher) Finished dev [unoptimized + debuginfo] target(s) in 4m 25s cargo build 571,39s user 50,53s system 233% cpu 4:25,95 total {{< / highlight >}}

This is quite surprising if you compare it to a fast compiling language like go, but that's fair because the compiler have to check a bunch of things related to memory safety. With no garbage collector, speed at runtime and memory safety, you have to pay a price and this price is the compile time.

But I really think it's not a weakness for Rust because cargo caching is amazing and after the first compilation, iterations are pretty fast so it's not a real issue.

When it comes to building a Docker image, I learned a nice tip to optimize container image building with a clever use of layers. Here is the tip !

{{< highlight dockerfile "linenos=table" >}}

New empty project

RUN USER=root cargo new --bin fediwatcher WORKDIR /fediwatcher

Fetch deps list

COPY ./Cargo.lock ./Cargo.lock COPY ./Cargo.toml ./Cargo.toml

Step to build a default hello world project.

Since Cargo.lock and Cargo.toml are present,

all deps will be downloaded and cached inside this upper layer

RUN cargo build --release RUN rm src/*.rs

Now, copy source code

COPY ./src ./src

Build the real project

RUN rm ./target/release/deps/fediwatcher* RUN cargo build --release {{< / highlight >}}

Remember that dependencies are less volatile than code, and with containers this means get dependencies as soon as possible and copy code later ! In the Rust case, the first thing to do is creating an empty project using cargo new. This will create a default project with a basic hello world in main.rs file.

After that, copy all things related to dependencies (Cargo.toml and Cargo.lock files) and trigger a build, in this image layer, all the deps will be downloaded and compiled.

Now that there is a layer containing all the dependencies, copy the real source code and then compile the real project. With this technique, the dependencies layer will be cached and used in later build. Believe me, this a time saver !

Not lost yet ? Good, because there is more, so take a deep breath and go digging some Rust features.

Flow control 🛂

Rust takes inspiration from various programming language, mainly C++ a imperative language, but there is also a lot of features that are typical in functional programming. I already write some Haskell (because Xmonad ftw) and some Elixir but I don't feel very confident with functional programming yet.

I find this salad mix called as multi-paradigm programming very convenient to understand and try some functional way of thinking.

The top most functional feature of rust is the match statement. To me, this the most beautiful and clean way to handle multiple paths inside a program. For imperative programmers out there, a match is like a switch case on steroids. To illustrate, let's look at a simple example².

{{< highlight rust >}} let number = 2;

println!("Tell me something about {}", number);

match number { // Match a single value 1 => println!("One!"), // Match several values 2 | 3 | 5 | 7 | 11 => println!("This is a prime"), // Match an inclusive range 13..=19 => println!("A teen"), // Whatever _ => println!("Ain't special"), } {{< / highlight >}}

Here, all the cases are matched, but what if I removed the last branch ?

{{< highlight txt >}} help: ensure that all possible cases are being handled, possibly by adding wildcards or more match arms {{< / highlight >}}

See ? Rust violently pointing out missing stuff, and that's why it's a pleasant language to use.

A match statement can also be used to destructure a variable, a common pattern in functional programming. Destructuring is a process used to break a structure into multiple and independent variables. This can also be useful when you need only a part of a structure, making your code more comprehensive and readable³.

{{< highlight rust >}} struct Foo { x: (u32, u32), y: u32, }

// Try changing the values in the struct to see what happens let foo = Foo { x: (1, 2), y: 3 };

match foo { Foo { x: (1, b), y } => println!("First of x is 1, b = {}, y = {} ", b, y),

// you can destructure structs and rename the variables,
// the order is not important
Foo { y: 2, x: i } => println!("y is 2, i = {:?}", i),

// and you can also ignore some variables:
Foo { y, .. } => println!("y = {}, we don't care about x", y),
// this will give an error: pattern does not mention field `x`
//Foo { y } => println!("y = {}", y);

} {{< / highlight >}}

With match, I made my first step inside the functional programming way of thinking. The second one was iterators, functions chaining and closures, the perfect combo ! The idea is to chain function and pass input and output from one to another. Chaining using small scope functions made code more redable, more testable and more reliable. As always, an example !

{{< highlight rust >}} let iterator = [1,2,3,4,5].iter();

// fold, is also known as reduce, in other languages let sum = iterator.fold(0, |total, next| total + next);

println!("{}", sum); {{< / highlight >}}

The first line is used to create a iterator, a structure used to perform tasks on a sequence of items. Later on, a specific method associated with iterators fold is used to sum up all items inside the iterator and produce a single final value : the sum. As a parameter, we pass a closure (a function defined on the fly) with a total and a next arguments. The total variable is used to store current count status and next is the next value inside the iterator to add to total.

A non functional alternative as the code shown above will be something like :

{{< highlight rust >}} let collection = [1,2,3,4,5];

let mut sum = 0;

for elem in collection.iter() { sum += elem; }

println!("{}", sum); {{< / highlight >}}

With more complex data, more operations, removing for loops and chaining function using map, filter or fold really makes code cleaner and easier to understand. You just get important stuff, there is no distraction and a code without boiler plate lines is less error probes.

Flow control is a large domain and it contains error handling. In Rust there is two kind of generic errors : Option used to describe the possibility of absence and Result used as supersed of Option to handle the possibility of errors.

Here is the definition of an Option :

{{< highlight rust >}} enum Option { None, Some(T), } {{< / highlight >}}

Where None means "no value" and Some(T) means "some variable (of type T)"

An Option is useful if you, for example, search for a file that may not exists

{{< highlight rust >}} let file = "not.exists";

match find(file, '.') { None => println!("File not found."), Some(i) => println!("File found : {}", &file), } {{< / highlight >}}

If you need an explicit error to handle, go for Result :

{{< highlight rust >}} enum Result<T, E> { Ok(T), Err(E), } {{< / highlight >}}

Where Ok(T) means "everything is good for the value (of type T)" and Err(E) means "An error (of type E) occurs". To conclude, it's possible to define an Option like this :

{{< highlight rust >}} type Option = Result<T, ()>; {{< / highlight >}}

"An Option is a Result with an empty Err value". Q.E.D !

At this point of my journey (re)discovering Rust I was super happy with all this new concepts. As a gopher, I know how crappy error handling can be in other languages, so a clean and standard way to handle error, count me in.

So, what about composing functions that needs error handling ? Ahah ! Let's go :

{{< highlight rust "linenos=table, hl_lines=32-40 43-45 48-50" >}} // An example using music bands

// Allow dead code, for learning purpose #![allow(dead_code)]

#[derive(Debug)] enum Bands { AAL, Alcest, Sabaton, }

// But does it djent ? fn does_it_djent(b: Bands) -> Option { match b { // Only Animals As Leaders djents Bands::AAL => Some(b), _ => None, } }

// Do I like it ? fn likes(b: Bands) -> Option { // No, I do not like Sabaton match b { Bands::Sabaton => None, _ => Some(b), } }

// Do it djent and do I like it ? the match version ! fn match_likes_djent(b: Bands) -> Option { match does_it_djent(b) { Some(b) => match likes(b) { Some(b) => Some(b), None => None, }, None => None, } }

// Do it djent and do I like it ? the map version ! fn map_likes_djent(b: Bands) -> Option<Option> { does_it_djent(b).map(|b| likes(b)) }

// Do it djents and do I like it ? the and_then version ! fn and_then_likes_djent(b: Bands) -> Option { does_it_djent(b).and_then(|b| likes(b)) }

fn main() { let aal = Bands::AAL;

match and_then_likes_djent(aal) {
    Some(b) => println!("I like {:?} and it djents", b),
    None => println!("Hurgh, this band doesn't even djent !"),
}

} {{< / highlight >}}

On a first try, the basic solution is to use a series of match statements (line 32). With two functions, that's ok, but with 3 or more, this will be a pain in the ass to read. Searching for a cleaner way of handling stuff that returns an Option I find the associated map method. BUT using map with something that also return an Option leads to (function definition on line 43) :

an Option of an Option !

Is everything doomed ? No ! Because there is the god send and_then method (function starting on line 48). Basically, and_then ensure that we keep a "flat" structure and do not add an Option wrapping to an already existing Option. Lesson learned : if you have to deal with a lot of Options or Results, use and_then.

Last but not least, I also want to write about the ? operator for error handling. Since Rust version 1.13, this new operator removes a lot of boiler plate and redundant code.

Before 1.13, error handling will probably look like this :

{{< highlight rust >}} fn read_from_file() -> Result<String, io::Error> { let f = File::open("sample.txt"); let mut s = String::new();

let mut f = match f {
    Ok(f) => f
    Err(e) => return Err(e),
};


match f.read_to_string(&mut s) {
    Ok(_) => Ok(s),
    Err(e) => Err(e),
}

} {{< / highlight >}}

With 1.13 and later,

{{< highlight rust >}} fn read_from_file() -> Result<String, io::Error> { let mut s = String::new(); let mut f = File::open("sample.txt")?;

f.read_to_string(&mut s)?;

Ok(s)

} {{< / highlight >}}

Nice and clean ! Rust also experiment with a function name try, used like the ? operator, but chaining functions leads to unreadable and ugly code :

{{< highlight rust >}} try!(try!(try!(foo()).bar()).baz()) {{< / highlight >}}

To conclude, there is a lot of stuff here, to make code easy to understand and maintain. Flow control using match and functions combination may seems odd at the beggining but after some pratice and experiments I find quite pleasant to use. But there is (again), more, fasten your seatbelt, next section will blow your mind.

Ownership, borrowing, lifetimes 🤯

To be clear, the 10 first hours of Rust coding just smash my brains because of this three concepts. They are quite handy to understand at first, because they change the way we make and understand programs. With time, pratice and compiler help, the mist is replaced by a beautiful sunligth. There is plenty of other blog posts, tutorials and lessons about lifetimes, ownership and borrowing. I will add my brick to this wall, with my own understanding of it.

Let's start with ownership. Rust memory management is base on this concept. Every resources (variables, objects...) is own by a block of code. At the end of this block, resourses are destroyed. This is the standard predicatable, reproducible, behavior of Rust. For small stuff, that's easy to understand :

{{< highlight rust "linenos=table, hl_lines=11">}} fn main() { // create a block, or scope { // resource creation let i = 42; println!("{}", i); // i is destroyed by the compiler, and you have nothing else to do }

// fail, because i do not exists anymore 
println!("{}", i);

} {{< / highlight >}}

Compiling this piece of code will throw an error :

{{< highlight txt >}} error[E0425]: cannot find value i in this scope --> src/main.rs:11:20 | 11 | println!("{}", i); | ^ not found in this scope {{< / highlight >}}

To remove the error, just delete the line 11.

Ok, cool ! But what if I want to pass a resources to another block or even a function ?

{{< highlight rust "linenos=table" >}} fn priprint(val: int) { println!("{}", val); }

fn main() { let i = 42;

priprint(i);

println!("{}", i);

} {{< / highlight >}}

Here, this piece of code works because Rust copy the value of i into val when calling the priprint function. All primitve type in Rust works this way, but, if you want to pass, for example, a struct, Rust will move the resource to the function. By moving a resource, you transfer ownership to the receiver. So in the example below priprint will be responsible of the destruction of the struct passed to it.

{{< highlight rust "linenos=table, hl_lines=12" >}} struct Number { value: i32 }

fn priprint(n: Number) { println!("{}", n.value); }

fn main() { let n = Number{ value: 42 };

priprint(n);

println!("{}", n.value);

} {{< / highlight >}}

When compiling, Rust will not be happy :

{{< highlight txt >}} error[E0382]: borrow of moved value: n --> src/main.rs:14:20 | 10 | let n = Number{ value: 42 }; | - move occurs because n has type Number, which does not implement the Copy trait 11 | 12 | priprint(n); | - value moved here 13 | 14 | println!("{}", n.value); | ^^^^^^^ value borrowed here after move {{< / highlight >}}

After ownership comes borrowing. With borrowing our Rust program is able to have multiple references or pointers. Passing a reference to another block tells to this block, here is a borrow (mutable or imutable) do what you want with it but do not destroy it at the end of your scope. To pass references, or borrows, add the & operator to priprint argument and parameter.

{{< highlight rust "linenos=table, hl_lines=5 12" >}} struct Number { value: i32 }

fn priprint(n: &Number) { println!("{}", n.value); }

fn main() { let n = Number{ value: 42 };

priprint(&n);

println!("{}", n.value);

} {{< / highlight >}}

Seems cool no ? If a friend of mine borrow my car, I hope he will not return it in pieces.

Now, lifetimes ! Rust resources always have a lifetime associated to it. This means that resources the are accessible or "live" from the moment you declare it and the moment they are dropped. If you're familiar with other programming languages, think about extensible scopes. To me extensible scopes means that scopes can be move from one block of code to another. Simple, huh ? But things get complicated if you add references in the mix. Why ? Because references also have lifetime, and this lifetime, called associated lifetime, can be smaller than the lifetime pointed by the reference. Can this associated lifetime be longer ? No ! Because we want to access valid data ! In most cases, Rust compiler is able to guess how lifetimes are related. If not, it will explicitly ask you to annotate you're code with lifetimes specifiers. To dig this subject, a whole article is necessary and I don't find my self enough confident with lifetimes yet to explain it in details. This is clearly the hardest part when you learning Rust. If you don't understand what you're doing at the beginning, that's not a real problem. Don't give up, read, try and experiment, the reward worth it.

What's next ? 🔭

Thanks to Rust and my little project, I learned a bunch of new concepts related to programming.

Rust is a beautiful language. The first time I used it, many years ago, it was a bit odd to understand. Today, with more programming experiences, I really understand why it matters. To me 2019, will be the Rust year. A lots of Rust projects pops up on Github, and that's a good sign of how the language start to gain in popularity. Backed up with Mozilla and the community, I really believe that's it's the go to language for the next 10 years. Of course, Golang is also in this spectrum of new generation laguages but they complement one each other with various ways of thinking and programming. That's clear to me, I will continue to make Go AND Rust programs.

Now, I need to go deeper. On one hand, by adding new features to Fediwatcher I want to experiment around concurrency and how I can compare it to Golang.

On the other hand, I'm really, really interested by web assembly and I think Rust is a good bet to start exploring this new open field. Last but not least, all this skills will allow me to continue my contributions to Plume, a Rust federated blogging application, based on ActivityPub.

Let's go^Wrust !