23 KiB
title | subtitle | date | draft | tags | |||
---|---|---|---|---|---|---|---|
Another one bites the Rust | Lessons learned from my first Rust project | 2019-08-25 | false |
|
Rust, the (re)discovery 🗺️
Back to 2014, the year I wrote my first Rust program ! Wow, that was even before version 1.0 !
At my school, it was all about C, and 10 years of C is quite boring (because, yes, I started programming way before my engineering school). I wanted some fresh air, and even if I really liked Python back to that time, i'm more into statically typed programming languages.
I can't remember exactly what was my first contact with Rust, maybe a blog post or a reddit post, and my reaction was something like :
Now, let's dig some reasons about why Rust blows my mind.
Rust is a programming langage focused on safety, and concurrency. It's basically the modern replacement of C++, plus, a multi-paradigm approach to system programming development. This langage was created and designed by Graydon Hoare at Mozilla and used for the Servo browser engine, now embedded in Mozilla Firefox.
This system try to be as memory safe as possible :
- no null pointers
- no dandling pointers
- no data races
- values have to be initialized and used
- no need to free allocated data
Back in 2014, the first thing that come to my mind was :
Yeah, yeah, just yet another garbage collected langage
And I was wrong ! Rust uses other mechanisms such as ownership, lifetimes, borrowing, and the ultimate borrow checker to ensure that memory is safe and freed only when data will not be used anywhere else in the program (ie : when lifetime is over). This new memory management concepts ensure safety AND speed since there is no overhead generated by a garbage collection.
For me, as a young hardcore C programmer1, this was literally heaven. Less struggling
with calloc
, malloc
, free
and valgrind
, thanks god Mozilla ! Bless me !
But, I dropped it in 2015. Why ? Because this was, at this time, far from perfect. Struggling with all the new concepts was quite disturbing, add to that cryptic compilation errors and you open a gate to brainhell. My young me was not enough confident to learn and there was no community to help me understand things that was unclear to me.
Five years later, my programming skills are clearly not at the same level as before, learning and writing a lot of Golang and Javascript stuff, playing with Elixir and Haskell, completely changed how I manipulate and how I visualize code in day to day basis. It was time to give another chance to Rust.
Fediwatcher 📊
In order to practice my Rust skill, I wanted to build a concrete and useful (at least for me) project.
Inspired by the work of href on fediverse.network my main idea was to build a small app to fetch metrics from various instances of the fediverse and push it into an InfluxDB timeseries database.
Fediwatcher was born !
The code is available on a github repo associated with my github account.
If you're interested, check out the Fediwatcher public instance.
Ok, ok, enough personal promotion, now that you get the main idea, go for the technical part and all the lessons learn writing this small project !
Cargo, compiling & building 🏗️
cargo
is the Rust standard packet manager created and maintained by
the Rust project. This is the tool used by all rustaceans and that's a good
thing. If you don't know it yet, I'm also a gopher, and package management
with go
is a big pile of shit. In 2019, leaving package management to the
community is, I think, the biggest mistake you can make when creating a new
programming language. So go for cargo !
cargo
is used for :
- downloading app dependencies
- compiling app dependencies
- compiling your project
- running your project
- running tests on your project
- publishing you project to crates.io
All the informations are contained in the Cargo.toml
file, no need for a
makefile
makes my brain happy and having a common way to tests code without
external packages is pretty straightforward and a strong invitation to test
your code.
All the standard stuff also means that default docker images contains all you need to setup a Continous Integration without the need to maintain specific container images for a specific test suite or whatever. Less time building stuff, means more productive time to code. With Rust, all the batteries are included.
About compiling, spoiler alert, Rust project compiling IS SLOW :
{{< highlight sh >}} cargo clean time cargo build Compiling autocfg v0.1.4 Compiling libc v0.2.58 Compiling arrayvec v0.4.10 Compiling spin v0.5.0 Compiling proc-macro2 v0.4.30 [...] Compiling reqwest v0.9.18 Compiling fediwatcher v0.1.0 (/Users/wilfried/code/github/fediwatcher) Finished dev [unoptimized + debuginfo] target(s) in 4m 25s cargo build 571,39s user 50,53s system 233% cpu 4:25,95 total {{< / highlight >}}
This is quite surprising if you compare it to a fast compiling language like go
,
but that's fair because the compiler have to check a bunch of things related to
memory safety. With no garbage collector, speed at runtime and memory safety,
you have to pay a price and this price is the compile time.
But I really think it's not a weakness for Rust because cargo
caching is
amazing and after the first compilation, iterations are pretty fast so it's
not a real issue.
When it comes to building a Docker image, I learned a nice tip to optimize container image building with a clever use of layers. Here is the tip !
{{< highlight dockerfile "linenos=table" >}}
New empty project
RUN USER=root cargo new --bin fediwatcher WORKDIR /fediwatcher
Fetch deps list
COPY ./Cargo.lock ./Cargo.lock COPY ./Cargo.toml ./Cargo.toml
Step to build a default hello world project.
Since Cargo.lock and Cargo.toml are present,
all deps will be downloaded and cached inside this upper layer
RUN cargo build --release RUN rm src/*.rs
Now, copy source code
COPY ./src ./src
Build the real project
RUN rm ./target/release/deps/fediwatcher* RUN cargo build --release {{< / highlight >}}
Remember that dependencies are less volatile than code, and with containers
this means get dependencies as soon as possible and copy code later ! In the
Rust case, the first thing to do is creating an empty project using cargo new
.
This will create a default project with a basic hello world in
main.rs
file.
After that, copy all things related to dependencies
(Cargo.toml
and Cargo.lock
files) and trigger a build, in this image
layer, all the deps will be downloaded and compiled.
Now that there is a layer containing all the dependencies, copy the real source code and then compile the real project. With this technique, the dependencies layer will be cached and used in later build. Believe me, this a time saver !
Not lost yet ? Good, because there is more, so take a deep breath and go digging some Rust features.
Flow control 🛂
Rust takes inspiration from various programming language, mainly C++ a imperative language, but there is also a lot of features that are typical in functional programming. I already write some Haskell (because Xmonad ftw) and some Elixir but I don't feel very confident with functional programming yet.
I find this salad mix called as multi-paradigm programming very convenient to understand and try some functional way of thinking.
The top most functional feature of rust is the match
statement. To me,
this the most beautiful and clean way to handle multiple paths inside a
program. For imperative programmers out there, a match
is like a switch case
on steroids.
To illustrate, let's look at a simple example2.
{{< highlight rust >}} let number = 2;
println!("Tell me something about {}", number);
match number { // Match a single value 1 => println!("One!"), // Match several values 2 | 3 | 5 | 7 | 11 => println!("This is a prime"), // Match an inclusive range 13..=19 => println!("A teen"), // Whatever _ => println!("Ain't special"), } {{< / highlight >}}
Here, all the cases are matched, but what if I removed the last branch ?
{{< highlight txt >}} help: ensure that all possible cases are being handled, possibly by adding wildcards or more match arms {{< / highlight >}}
See ? Rust violently pointing out missing stuff, and that's why it's a pleasant language to use.
A match
statement can also be used to destructure a variable, a common
pattern in functional programming. Destructuring is a process used to
break a structure into multiple and independent variables. This can also be
useful when you need only a part of a structure, making your code more
comprehensive and readable3.
{{< highlight rust >}} struct Foo { x: (u32, u32), y: u32, }
// Try changing the values in the struct to see what happens let foo = Foo { x: (1, 2), y: 3 };
match foo { Foo { x: (1, b), y } => println!("First of x is 1, b = {}, y = {} ", b, y),
// you can destructure structs and rename the variables,
// the order is not important
Foo { y: 2, x: i } => println!("y is 2, i = {:?}", i),
// and you can also ignore some variables:
Foo { y, .. } => println!("y = {}, we don't care about x", y),
// this will give an error: pattern does not mention field `x`
//Foo { y } => println!("y = {}", y);
} {{< / highlight >}}
With match
, I made my first step inside the functional programming way
of thinking. The second one was iterators, functions chaining and
closures, the perfect combo ! The idea is to chain function and pass input
and output from one to another. Chaining using small scope functions made
code more redable, more testable and more reliable. As always, an example !
{{< highlight rust >}} let iterator = [1,2,3,4,5].iter();
// fold, is also known as reduce, in other languages let sum = iterator.fold(0, |total, next| total + next);
println!("{}", sum); {{< / highlight >}}
The first line is used to create a iterator
, a structure used to perform
tasks on a sequence of items. Later on, a specific method associated with
iterators fold
is used to sum up all items inside the iterator and produce
a single final value : the sum. As a parameter, we pass a closure
(a
function defined on the fly) with a total
and a next
arguments. The
total
variable is used to store current count status and next
is the
next value inside the iterator to add to total
.
A non functional alternative as the code shown above will be something like :
{{< highlight rust >}} let collection = [1,2,3,4,5];
let mut sum = 0;
for elem in collection.iter() { sum += elem; }
println!("{}", sum); {{< / highlight >}}
With more complex data, more operations, removing for loops and chaining
function using map
, filter
or fold
really makes code cleaner and easier
to understand. You just get important stuff, there is no distraction and
a code without boiler plate lines is less error probes.
Flow control is a large domain and it contains error handling. In Rust
there is two kind of generic errors : Option
used to describe the
possibility of absence and Result
used as supersed of Option
to handle
the possibility of errors.
Here is the definition of an Option
:
{{< highlight rust >}} enum Option { None, Some(T), } {{< / highlight >}}
Where None
means "no value" and Some(T)
means "some variable (of type T
)"
An Option
is useful if you, for example, search for a file that may not exists
{{< highlight rust >}} let file = "not.exists";
match find(file, '.') { None => println!("File not found."), Some(i) => println!("File found : {}", &file), } {{< / highlight >}}
If you need an explicit error to handle, go for Result
:
{{< highlight rust >}} enum Result<T, E> { Ok(T), Err(E), } {{< / highlight >}}
Where Ok(T)
means "everything is good for the value (of type T
)" and
Err(E)
means "An error (of type E
) occurs". To conclude, it's possible to
define an Option
like this :
{{< highlight rust >}} type Option = Result<T, ()>; {{< / highlight >}}
"An Option
is a Result
with an empty Err
value". Q.E.D !
At this point of my journey (re)discovering Rust I was super happy with all this new concepts. As a gopher, I know how crappy error handling can be in other languages, so a clean and standard way to handle error, count me in.
So, what about composing functions that needs error handling ? Ahah ! Let's go :
{{< highlight rust "linenos=table, hl_lines=32-40 43-45 48-50" >}} // An example using music bands
// Allow dead code, for learning purpose #![allow(dead_code)]
#[derive(Debug)] enum Bands { AAL, Alcest, Sabaton, }
// But does it djent ? fn does_it_djent(b: Bands) -> Option { match b { // Only Animals As Leaders djents Bands::AAL => Some(b), _ => None, } }
// Do I like it ? fn likes(b: Bands) -> Option { // No, I do not like Sabaton match b { Bands::Sabaton => None, _ => Some(b), } }
// Do it djent and do I like it ? the match version ! fn match_likes_djent(b: Bands) -> Option { match does_it_djent(b) { Some(b) => match likes(b) { Some(b) => Some(b), None => None, }, None => None, } }
// Do it djent and do I like it ? the map version ! fn map_likes_djent(b: Bands) -> Option<Option> { does_it_djent(b).map(|b| likes(b)) }
// Do it djents and do I like it ? the and_then version ! fn and_then_likes_djent(b: Bands) -> Option { does_it_djent(b).and_then(|b| likes(b)) }
fn main() { let aal = Bands::AAL;
match and_then_likes_djent(aal) {
Some(b) => println!("I like {:?} and it djents", b),
None => println!("Hurgh, this band doesn't even djent !"),
}
} {{< / highlight >}}
On a first try, the basic solution is to use a series of match
statements
(line 32). With two functions, that's ok, but with 3 or more, this
will be a pain in the ass to read. Searching for a cleaner way of handling
stuff that returns an Option
I find the associated map
method. BUT
using map
with something that also return an Option
leads to (function
definition on line 43) :
an Option of an Option !
Is everything doomed ? No ! Because there is the god send and_then
method
(function starting on line 48). Basically, and_then
ensure that we keep a
"flat" structure and do not add an Option
wrapping to an already existing
Option
. Lesson learned : if you have to deal with a lot of Option
s or
Result
s, use and_then
.
Last but not least, I also want to write about the ?
operator for error
handling. Since Rust version 1.13, this new operator removes a lot of
boiler plate and redundant code.
Before 1.13, error handling will probably look like this :
{{< highlight rust >}} fn read_from_file() -> Result<String, io::Error> { let f = File::open("sample.txt"); let mut s = String::new();
let mut f = match f {
Ok(f) => f
Err(e) => return Err(e),
};
match f.read_to_string(&mut s) {
Ok(_) => Ok(s),
Err(e) => Err(e),
}
} {{< / highlight >}}
With 1.13 and later,
{{< highlight rust >}} fn read_from_file() -> Result<String, io::Error> { let mut s = String::new(); let mut f = File::open("sample.txt")?;
f.read_to_string(&mut s)?;
Ok(s)
} {{< / highlight >}}
Nice and clean ! Rust also experiment with a function name try
, used like
the ?
operator, but chaining functions leads to unreadable and ugly code :
{{< highlight rust >}} try!(try!(try!(foo()).bar()).baz()) {{< / highlight >}}
To conclude, there is a lot of stuff here, to make code easy to understand and maintain. Flow control using match and functions combination may seems odd at the beggining but after some pratice and experiments I find quite pleasant to use. But there is (again), more, fasten your seatbelt, next section will blow your mind.
Ownership, borrowing, lifetimes 🤯
To be clear, the 10 first hours of Rust coding just smash my brains because of this three concepts. They are quite handy to understand at first, because they change the way we make and understand programs. With time, pratice and compiler help, the mist is replaced by a beautiful sunligth. There is plenty of other blog posts, tutorials and lessons about lifetimes, ownership and borrowing. I will add my brick to this wall, with my own understanding of it.
Let's start with ownership. Rust memory management is base on this concept. Every resources (variables, objects...) is own by a block of code. At the end of this block, resourses are destroyed. This is the standard predicatable, reproducible, behavior of Rust. For small stuff, that's easy to understand :
{{< highlight rust "linenos=table, hl_lines=11">}} fn main() { // create a block, or scope { // resource creation let i = 42; println!("{}", i); // i is destroyed by the compiler, and you have nothing else to do }
// fail, because i do not exists anymore
println!("{}", i);
} {{< / highlight >}}
Compiling this piece of code will throw an error :
{{< highlight txt >}}
error[E0425]: cannot find value i
in this scope
--> src/main.rs:11:20
|
11 | println!("{}", i);
| ^ not found in this scope
{{< / highlight >}}
To remove the error, just delete the line 11.
Ok, cool ! But what if I want to pass a resources to another block or even a function ?
{{< highlight rust "linenos=table" >}} fn priprint(val: int) { println!("{}", val); }
fn main() { let i = 42;
priprint(i);
println!("{}", i);
} {{< / highlight >}}
Here, this piece of code works because Rust copy the value of i
into
val
when calling the priprint
function. All primitve type in Rust
works this way, but, if you want to pass, for example, a struct, Rust
will move the resource to the function. By moving a resource, you
transfer ownership to the receiver. So in the example below priprint
will be responsible of the destruction of the struct passed to it.
{{< highlight rust "linenos=table, hl_lines=12" >}} struct Number { value: i32 }
fn priprint(n: Number) { println!("{}", n.value); }
fn main() { let n = Number{ value: 42 };
priprint(n);
println!("{}", n.value);
} {{< / highlight >}}
When compiling, Rust will not be happy :
{{< highlight txt >}}
error[E0382]: borrow of moved value: n
--> src/main.rs:14:20
|
10 | let n = Number{ value: 42 };
| - move occurs because n
has type Number
, which does not implement the Copy
trait
11 |
12 | priprint(n);
| - value moved here
13 |
14 | println!("{}", n.value);
| ^^^^^^^ value borrowed here after move
{{< / highlight >}}
After ownership comes borrowing. With borrowing our Rust
program is able to have multiple references or pointers. Passing a
reference to another block tells to this block, here is a borrow (mutable
or imutable) do what you want with it but do not destroy it at the end of your
scope. To pass references, or borrows, add the &
operator to priprint
argument and parameter.
{{< highlight rust "linenos=table, hl_lines=5 12" >}} struct Number { value: i32 }
fn priprint(n: &Number) { println!("{}", n.value); }
fn main() { let n = Number{ value: 42 };
priprint(&n);
println!("{}", n.value);
} {{< / highlight >}}
Seems cool no ? If a friend of mine borrow my car, I hope he will not return it in pieces.
Now, lifetimes ! Rust resources always have a lifetime associated to it. This means that resources the are accessible or "live" from the moment you declare it and the moment they are dropped. If you're familiar with other programming languages, think about extensible scopes. To me extensible scopes means that scopes can be move from one block of code to another. Simple, huh ? But things get complicated if you add references in the mix. Why ? Because references also have lifetime, and this lifetime, called associated lifetime, can be smaller than the lifetime pointed by the reference. Can this associated lifetime be longer ? No ! Because we want to access valid data ! In most cases, Rust compiler is able to guess how lifetimes are related. If not, it will explicitly ask you to annotate you're code with lifetimes specifiers. To dig this subject, a whole article is necessary and I don't find my self enough confident with lifetimes yet to explain it in details. This is clearly the hardest part when you learning Rust. If you don't understand what you're doing at the beginning, that's not a real problem. Don't give up, read, try and experiment, the reward worth it.
What's next ? 🔭
Thanks to Rust and my little project, I learned a bunch of new concepts related to programming.
Rust is a beautiful language. The first time I used it, many years ago, it was a bit odd to understand. Today, with more programming experiences, I really understand why it matters. To me 2019, will be the Rust year. A lots of Rust projects pops up on Github, and that's a good sign of how the language start to gain in popularity. Backed up with Mozilla and the community, I really believe that's it's the go to language for the next 10 years. Of course, Golang is also in this spectrum of new generation laguages but they complement one each other with various ways of thinking and programming. That's clear to me, I will continue to make Go AND Rust programs.
Now, I need to go deeper. On one hand, by adding new features to Fediwatcher I want to experiment around concurrency and how I can compare it to Golang.
On the other hand, I'm really, really interested by web assembly and I think Rust is a good bet to start exploring this new open field. Last but not least, all this skills will allow me to continue my contributions to Plume, a Rust federated blogging application, based on ActivityPub.
Let's go^Wrust !
-
I am not a C hardcore programmer anymore, beceause of Golang and Rust, of course. ↩︎
-
Taken and adapted from Rust documentation ↩︎
-
Taken from Rust documentation ↩︎