Programming Errors when Using References

We already saw that when you assign a variable to another variable, there are two cases: either their type is copyable, that is, it implements the Copy trait (and necessarily it implements the Clone trait too); or their type is not copyable, that is it does not implement the Copy trait (and it may implement the Clone trait or it may not).

In the first case, copy semantics is used. That means that a new object is created by that assignment; the original object remains represented by the source variable, and the new object is represented by the destination variable. Initially that new object has the same value of the original object. When each of these two variables gets out of their scope, the objects they represent are destroyed (a.k.a. dropped).

In the second case, move semantics is used. That means that, in an assignment, the source variable hands over its represented object to the destination variable. No object is created, and the source variable becomes no longer accessible. When the destination variable gets out of its scope, its represented object is destroyed. When the source variable gets out of its scope, nothing happens.

These behaviors guarantee the proper management of memory, as long as no references are used.

First, the ref_to_n variable is declared, but not initialized. Then, in an inner block, the n immutable variable is declared and initialized, and so it allocates a number in the stack, whose value is 12.

Then, the former variable is initialized with a reference to the n variable.

Then, the inner block ends, so the inner variable n ends its scope and its associated object is destroyed.

Then, the object referred to by the ref_to_n variable is printed. But that object has already been destroyed, and now it doesn’t exist anymore! Fortunately, the Rust compiler rejects this code, emitting the error message: `n` does not live long enough.

That message means that the n variable is dying at the end of the block in which it is declared, but there are still some references to the object it represents. In particular, the ref_to_n variable still references the object represented by the n variable. In addition, such reference is used later, at the last line of the program.

So, the n variable should definitely live longer; it should live at least as long as all the uses of the references to the objects it represents.

This program is accepted by all C and C++ compilers. The resulting program behaves in an unpredictable way (although usually it prints another number).

Let’s name this kind of programming error use after drop, in which an object is used (read or written) after it has already been destroyed, and so, in a way, it does not exist anymore.

Needless to say, the latter two programs are accepted by the respective compilers, even if their behavior is undefined. Instead, the Rust compiler rejects the first program, emitting the error message: cannot borrow `v` as mutable because it is also borrowed as immutable. Let’s see what’s wrong with this Rust program.

First, the v mutable variable is declared and initialized with a vector object containing only the number 12.

Then, the ref_to_first variable is declared and initialized with a reference to the first item of v. So, it is a reference to the object whose value is 12.

Then, another number is added to the vector, whose value is 13. But such an insertion could cause a reallocation in another place of the buffer containing the items of the vector. Even in such a case, the ref_to_first variable would continue to refer to the old, no longer valid, memory location.

At last, the old, possibly wrong, memory location is read and its value is printed, with unpredictable results.

This error has been caused by the fact that inserting items to or removing items from a vector invalidates all the references to that vector. In general, this error belongs to a broader category of errors, in which a data structure is accessible through several paths, or aliases; and when that data structure is changed using one alias, it cannot be properly used by another alias.

Let’s name this kind of programming error use after change by an alias.

To understand how you, with the help of the Rust compiler, can avoid such bugs, the concept of borrowing must be introduced.

Borrowing

After the second statement, the _ref_to_n variable represents a reference, and it references the same number represented by the n variable. Can we say that the object represented by the _ref_to_n variable owns the object whose value is 12? Is this relation an ownership or not?

It cannot be an ownership, because that number is already represented by the n variable, so it will be deallocated when the control flow exits the block in which that variable is declared. If it was owned by this reference, then it would be deallocated twice. So, normal references (excluding Box objects and inner references in vectors, strings, and other collections) never own an object.

The concept of a reference that does not own the referenced object is named borrowing. In the Rust documentation, this word is usually shortened to borrow, as a noun. We already saw this word in some compiler error messages.

It will print: 12 12 23. The first statement declares a variable that represents a stack-allocated number, whose value is 12. In the second statement, the b variable borrows that number.

The b variable receives another value in the fifth statement, so the previous borrow terminates before this statement. The last use of b before this assignment is in the fourth statement. Therefore, the first borrow terminates there.

The third statement begins a borrow in which the c variable borrows the same object borrowed also by b. Such borrow terminates at the last use of c, in the fourth statement.

The fifth statement begins a borrow in which the b variable borrows a stack-allocated object, whose value is 23, and that is not represented by any variable. Such borrow terminates at the last use of b, in the sixth statement.

In this program, the second statement begins and ends a mutable borrow , while the third statement begins and ends an immutable borrow. As known, you can take a mutable borrow only from a mutable variable.

Object Lifetimes

We have seen several times the concept of scope, which is the block in which an identifier is defined. That concept is related to syntax, and so to compile time, not to run time.

The relative concept related to runtime objects is named lifetime. The lifetime of an object is the sequence of instruction executions that starts when a value is assigned to that object, and that ends when that object is destroyed, that is, when it becomes invalid.

The start of the lifetime of an object can be an initialization or any successive assignment to a mutable variable.

Actually, we can talk about lifetimes even regarding references. The lifetime of a reference is defined to be just what has been already named borrow.

To determine when a lifetime ends, it is required to make a distinction between references and the other kinds of objects.

For an object that is not a reference, the lifetime ends when that object ceases to have a value, and so it is dropped. This happens when the object receives another value, or when the scope of the variable representing it ends.

Instead, the lifetime of a reference ends just after the last statement that uses that object. References do not own objects, and they do not implement the Drop trait, so nothing happens when their lifetime ends.

A lifetime that corresponds to the scope of a variable is named lexical lifetime . So, references have a nonlexical lifetime, and the other kinds of object have a lexical lifetime.

During its lifetime, an object is said to live or to be alive.

How to Prevent Use After Drop Errors

The technique used by Rust to prevent use after drop errors is simple.

Any such error is caused by a reference that accesses its referenced object in a statement in which that object is no longer defined. So in that statement, the borrow is still alive but the referenced object is no longer alive.

To avoid that a borrow references an invalid object in any statement, it is required that the borrow begins when the object has already been created, and that the borrow ends when the object has not yet been destroyed.

Put in other words, the borrow must be fully contained in the lifetime of the referenced object.

Here the borrow begins and ends in the second statement, but the lifetime of the object whose value is 12 begins in the third statement, and so the borrow is not contained in it. The compiler emits the message: borrow of possibly-uninitialized variable: `_n`. The compiler output also contains the indication that, at the second line, there is a use of possibly-uninitialized `_n`. It means that a borrow of an object, that is a kind of use of an object, cannot be performed if the compiler is not sure that such object has been initialized already.

Here, the borrow begins at the fourth line and ends at the sixth, but the lifetime of the object whose value is 12 begins at the third line and ends at the fifth line, so the borrow is not contained in it. The compiler emits the message: `_n` does not live long enough. This message means that the lifetime of the object represented by _n should be extended to include the last line. Though, this situation could also be interpreted as that the borrow should be shortened to end before the closed brace.

How to Prevent Use After Change by an Alias Errors

The rules to avoid using an object that is changed through another variable are somewhat more complex.

We already know that getting a reference starts a borrow. The third line starts an immutable borrow of _n. Such borrow ends two lines later, at the last use of that reference.

But also the fourth line accesses _n, so we must say that even it starts a borrow, though such access lasts only for the duration of the statement. After that assignment, the object represented by _n and the object represented by _m are not dependent on each other. A borrow that lasts no more than a single statement is named temporary borrow. So, at the fourth line we have a temporary borrow of the object represented by _n.

At the seventh line, the _n variable is the destination of an assignment. So, it is accessed, and therefore we must say that also this statement starts a borrow of _n. This statement does not involve other objects, so even this borrow ends in the statements itself. Therefore also in this statement there is a temporary borrow.

This last borrow is different from the previous one, though. In the fourth line, the borrowed object was accessed just for reading it, so it was an immutable borrow even if the _n variable itself was mutable. Instead, this last borrow, at the seventh line, accessed the borrowed object for writing it, so it was a mutable borrow.

The last line of the example increments _n. This requires both reading and writing the object. When both reading and writing can happen, the writing capability prevails, so we say that even this one is a mutable borrow.

Summarizing, any statement that reads an object, and does not write to it, must be considered as a temporary immutable borrow of that object; and any statement that changes an object must be considered as a temporary mutable borrow of that object. Such borrows start and end within that statement.

Given that, consider that we have this kind of error when an object is simultaneously referenced by several variables, and its value is changed. To be possible to change the value, at least one variable must have mutable access to such object.

So, the rule to avoid use after change by an alias errors is simply this: any object, in any point of the code, cannot have at the same time a mutable borrow and some other borrows (mutable or immutable).

So, for each object, Rust computes the lifetime of each borrow of that object, that is the time from a reference initialization or assignment to the last use of that reference value, and computes the intersection of these lifetimes. If a mutable borrow lifetime intersects another borrow lifetime, mutable or immutable, that’s a programming error.

These rules were adopted by Rust to ensure automatic deterministic memory deallocation, and to avoid invalid references; though, curiously, these rules were already well known in computer science for other reasons. Allow only one writer or several readers is the rule to avoid the so-called data races in concurrent programming. So, this rule allows also Rust to have data-race-free concurrent programs.

In addition, avoiding data races also has a good impact on the performance of single-threaded programs, because it eases CPU cache coherence.

The Need for Lifetime Specifiers in Returned References

It will print: 11.

There are the declarations of the variables n1 and n2 representing two objects, whose values are numbers. Then, each of these objects is borrowed by a reference; such references are represented by the variables _m1 and _m2. So, after the seventh line, _m1 is borrowing the object represented by n1, and _m2 is borrowing the object represented by n2. This is allowed, because _m1 is declared after n1, and _m2 is declared after n2, so these references begin to live after the objects they borrow. Such references die at the first closed brace, while the object represented by n1 dies at the end of the program, and the object represented by n2 dies at the second closed brace. So the references die before the points where the referenced objects die.

At the eighth line, there is the simple expression _m1. As it is the last expression of a block, the value of that expression becomes the value of the block itself, so the value is used to initialize the result variable. That value is a reference to the number represented by n1, so the result variable also borrows that number. This is Also allowed, because result is declared in the same block of n1, but after it, so it can borrow the object represented by n1.

This code will generate the compilation error: `n2` does not live long enough. This happens because now result gets its value from the _m2 expression, and as _m2 borrows the object represented by n2, result also borrows that object. But n2 is declared inside the outer block, so it will die at the last-but-one statement, before result dies, and therefore result cannot borrow its object.

All this reasoning is just a review of what we have already seen about borrowing, but it shows how borrowing can get complicated in just few lines. By the way, the portion of the Rust compiler dedicated to such reasoning is named borrow checker . We just saw that the borrow checker has a rather hard job to do.

Declaring and initializing variables is like passing arguments to a function. Assigning the value of a block to a variable is like assigning the return value of a function to a variable.

The only difference between them is the body of the func function.

According to our rules so far, the first program should be valid, and the second one should be illegal. Instead, the compiler refuses to compile both programs. Actually, both versions of the func function are valid per se. It’s just the borrow checker that would find them incompatible with their specific usage.

As we already saw about generic function parameter bounds using traits, it is a bad thing to consider valid or invalid a function invocation according to the contents of the body of such function; or, put in other words, to consider valid or invalid a function declaration according to the context in which such function is invoked.

The main reason for this is that an error message for a bad use of a function would be understandable only by those who know the code inside the body of the function.

One other reason is that, if the body of any invoked function can make valid or invalid the code where the function is invoked, to be sure that the main function is valid, the borrow checker should analyze all the functions of the program. Such a whole-program analysis would be overwhelmingly complex.

So, similarly to generic functions, functions returning a reference also must isolate the borrow checking at the function signature threshold. There is the need to borrow check any function considering only its signature, its body, and the signatures of any invoked functions, without considering the bodies of the invoked functions. Of course, such bodies will be checked when these functions are separately borrow checked.

For both the previous programs, the compiler emits the message: missing lifetime specifier. Lifetime specifiers will be explained in the next section. They are decorations of function signatures that allow the borrow checker to separately check any function.

Usage and Meaning of Lifetime Specifiers

When a function returns a reference, the reference cannot refer to an object represented by an argument of that function or owned by it (case 1), or represented by a local variable of that function or owned by it (case 2), or to a temporary object or an object owned by it (case 3), because, before that function returns, every one of such objects is destroyed. So, the reference would be dangling, that is, referencing an already destroyed object.

The borrow checker is interested only in the references contained in the return value, and such references can only be of two kinds: those that borrow a static object and those that borrow a function argument. To accomplish its job of analyzing a function without analyzing the body of the functions invoked by it, the borrow checker needs to know which returned references borrow static objects and which borrow a function argument. And for every returned reference that borrows a function argument, the borrow checker needs to know which function argument it borrows, in case that function receives several references as arguments.

This function signature has two references among its arguments, and also two references inside its return value type. Each of the last two references could borrow a static object, or the object already borrowed by the b argument, or the object already borrowed by the second field of the c argument.

Just after the name of the function, a parameter list has been added, like the one used for generic functions. But instead of a type parameter, there is a lifetime specifier.

The <'a> clause is just a declaration. It means: In this function signature, a lifetime specifier is used; its name is "a". The name a is arbitrary. It simply means that in all the occurrences it appears, such occurrences match. It is similar to the type parameters of generic functions. To distinguish lifetime specifiers from type parameters, the former ones are prefixed by a single quote. In addition, while by convention type parameters begin with an uppercase letter, lifetime specifiers are single lowercase letters, like a, b, or c.

Then, this signature contains three other occurrences of the 'a lifetime specifier: in the type of the b argument, in the second field of the type of the c argument, and in the first field of the return value type. But the third field of the return value type is annotated by the 'static lifetime specifier.

The use of the a lifetime specifier means: the first field of the return value borrows the same object already borrowed by the b argument and by the second field of the c argument, so it must live for less time than such objects.

The use of the static lifetime specifier means: the third field of the return value refers to a static object, so it can live for any time, even as long as the whole process.

In this case, the first field of the return value has a static lifetime, meaning that it is not constrained to live less than some other object. Instead, the third field has the same lifetime specifier as the b argument, meaning that it should live less than it, as it borrows the same object. The reference in the type of the c argument is not annotated, as the object it refers is not borrowed by any reference in the return value.

This generic function has two lifetime parameters, and also two type parameters. The lifetime parameter a specifies that the third field of the return value borrows the object already borrowed by the b argument, while the lifetime parameter b specifies that the first field of the return value borrows the object already borrowed by the second field of the c argument. Moreover, the function has two type parameters, T1 and T2, used as usual.

Checking the Validity of Lifetime Specifiers

In this section, we’ll see the first one of such jobs.

If in the function return value there are no references, the borrow checker has nothing to check.

Otherwise, for every reference contained in the return value type, it must check that it has a proper lifetime specifier.

It will print: true 4 Hello 3.14. This is valid because all the three returned references are actually static objects.

will generate two compilation errors. The error for the body of the f function is: explicit lifetime required in the type of `n`. And the error for the body of the g function is: lifetime of reference outlives lifetime of borrowed content....

Both functions are illegal, because those signatures require that the bodies return a static reference. Instead, in the f function the n reference returned has no lifetime specified; and in the g function the m reference returned has a nonstatic lifetime specified. Actually, both function bodies return the same value received as argument, so such return value borrows the same object as the one referenced by the function argument, which is not necessarily static.

It will print: 13 true 12. This code is valid because the reference returned as the first field of the tuple is the value of the y expression, and the y argument has the same lifetime specifier as the first field of the return value; it is b for both of them. The same correspondence holds for the third field of the return value and the x argument; they both have the a lifetime specifier.

will generate two compilation errors, both with the error message: lifetime mismatch. Actually, both the first and the third fields of the return value have a lifetime specified in the argument list that is different from the one specified in the return value type.

It will print: 12 true 13. However, this solution has a different meaning than the previous one. In the previous valid solution, the two references contained in the argument list had independent lifetimes; instead, in this last solution, they share the same lifetime. In this example, it means that both the references in the tuple returned must live longer than both the function arguments.

This function is valid. In its body, the value of the function may be returned in three possible points. In all of them, the returned value borrows the same object borrowed by the x argument. Such argument has the same lifetime of the return value, so the borrow checker is satisfied.

one of the possible return values, the value of the expression &y[2], borrows the object borrowed by y; that argument has no lifetime specifier, so this code is illegal.

When performing data-flow analysis on this function, the compiler could detect that y is never borrowed by the return value of this function; but the borrow checker insists that &y[0] is a possible return value, so it spots this code as invalid.

Using the Lifetime Specifiers of Invoked Functions

As we said at the beginning of the previous section, one of the two jobs of the borrow checker is to check, when compiling a function, that the body of that function is valid, taking into account the signatures of any function invoked in the body.

As an example of this, get back to the last two programs of the section “The Need for Lifetime Specifiers in Returned References.” We said that, according to our rules of borrowing, the first program should be valid and the second one illegal; though, we got the missing lifetime specifier error for both programs. The following ones are those two programs, with the addition of the appropriate lifetime specifiers.

The first program is valid, and it will print 11, while for the second program the compiler will print the message: `n2` does not live long enough. Both have exactly the same semantics of the original programs, which didn’t use functions.

The check of the lifetime specification inside the two func functions has been explained in the previous section. Now let’s see how the main function in the first of the two programs works, regarding lifetimes.

When func is invoked, the alive variables are n1, result, and n2, declared in that order, with n1 and n2 already initialized. The signature of func says that the result value has the same lifetime specifier as the first argument, and that means that the value assigned to result must live no longer than n1. This actually holds, because result has been declared after n1, so it will be destroyed before it.

Now, let’s see why the main function in the second program is illegal. Here, the signature of func says that the result value has the same lifetime specifier of the second argument, and that means that the value assigned to result must live no longer than n2. But this doesn’t hold, actually, because result has been declared before n2, so it will be destroyed after it.

Advantage of Using Several Lifetime Specifiers for a Function

We said before that a function that returns a value containing several references can specify a single lifetime specifier for all of them, or a distinct lifetime specifier for each of them. Now let’s explain why using only one lifetime specifier may be not as good as using several lifetime specifiers.

It will print: 13 12.

the resulting program is illegal, and its compilation generates the error: `j1` does not live long enough.

In both versions, the f function receives references to the numbers i1 and j1. The returned tuple is stored first in the r variable and then its first and third values are used to initialize the i2 and j2 variables, respectively.

In the first version of the program, the first argument and the first field of return value have the same lifetime specifier, and that means that i1 must live longer than i2. Similarly, j1 must live longer than j2. Actually, the order of the declaration of such variables satisfies such requirements.

Instead, in the second version of the program, there is just one lifetime specifier. In accordance with that specifier, both i1 and j1 must live longer than i2 and j2. Yet, j1 lives less than i2, because j1 is declared in the inner block, so it does not live outside of it, while i2 is used in the last line of the program, and so it must live up to there. This does not satisfy the lifetime requirements.

So, it is better to use a distinct lifetime specifier for any logically independent lifetime.