Automatic Memory Management

Every program stores instances of datatypes in memory. This per-instance memory can be allocated either in the data segment, the stack, or the heap. Typically, static variables in a program are stored in the data segment and local variables to a method are stored on the stack. The compiler inserts appropriate logic in the output code to deal with memory management of data-segment-bound and stack-based types. For heap-based types, a programmer has to explicitly deal with memory management issues. Here are the typical steps a programmer goes through to deal with heap-based types:

Humans make mistakes. While programming, it is quite possible that someone will forget to call Step 4. C++ has a neat solution for this. When delete is called on an object, the compiler ensures that the class's destructor is invoked first before the memory is freed. If the disposing logic is moved in the destructor, Step 4 is automatically taken care of when Step 5 is called.

If you forget to call Step 5, not only will it result in a memory leak, but it might also cause the resources not to be disposed off.

This problem has plagued programmers forever. A plethora of tools came into existence to detect such memory leaks in the program. Many “smart pointer” [Ede-92] techniques were designed to ensure that Step 5 is automatically performed. A typical smart pointer implementation keeps count of the number of outstanding references on the object. When the last reference goes away (implying that the object is no longer in use), the object is deleted.

.NET addresses this problem in a different way. In Chapter 2, I touched on the noticeable absence of a delete operator in C#. There is no need to delete an object. The runtime automatically detects whether the object is no longer in use and deletes it. This mechanism is called garbage collection (GC). Let's see how it works.

It is worth mentioning that GC is not a new technique. There are many GC algorithms in use today, but our focus is on the GC algorithm used under .NET.

Also note that what I cover here are some important aspects of memory management. For more intricate details, please read Richter's articles [Ric-00a] and [Ric-00b] in MSDN Magazine.

Garbage Collection

When the common language runtime is initialized in a process, it reserves a contiguous region of memory space on the process's heap. This memory region is called the managed heap.

When an application creates an object using the new operator, the run-time assigns the required size on the managed heap to the object. As each new object is created, a contiguous memory location is assigned to the object.

This simple mechanism makes allocating an object on the managed heap very efficient. In fact, the operation is as fast as allocating memory on the stack.

Note that even though a big chunk of memory is reserved on the heap initially, it is the virtual memory of the process, not the actual storage. The storage is committed as necessary when objects are created at runtime. The implication is that the working set size of the process is relatively small during initialization but may grow as objects are created.

As objects are assigned memory on the managed heap, the heap starts filling up. What happens when a new object needs to be created but there is not enough space available in the managed heap?

This is where the GC kicks in. Here is what it does:

It is interesting to know how the list of nongarbage objects is created. Each application has a set of root objects, or objects that the GC can use as a starting point to detect other objects being used. For example, all the global and static objects in an application are considered root objects. Local variables and method arguments on a thread's stack also constitute as root objects. Finally, any CPU register containing pointers to objects in the managed heap is also considered part of the application's roots. The set of root objects may change as execution proceeds. With some help from the JIT compiler, the runtime maintains the list of active roots.

When the garbage collector starts running, it walks the active roots and builds a graph of all objects reachable from the root. These are the objects that are in use. The rest of the objects are garbage and hence can be collected.

Note that an object is a candidate for collection as soon as it is no longer used. For example, variables that go out of scope are all candidates for collection. Even within the scope, if the execution has passed the last use of a variable, such a variable is also a candidate for collection.

Debugger Changes the Behavior

When an application is being debugged, the runtime extends the lifetime of a variable even after it no longer is used within the scope. This gives you a chance to inspect the objects within the debugger.

Performance Considerations

Because of its nature, GC is a costly operation. Depending on the gaps left in the managed heap, shifting memory could be quite expensive. An even bigger impact is for multithreaded applications. As memory is being shifted around, the runtime has to suspend all other threads to ensure that objects do not point to invalid memory locations.

On the positive side, though, the GC algorithm uses a few different mechanisms to keep threads running as long as possible and to reduce overhead. These mechanisms include fully interrupting the code (on each thread), hijacking a thread, and letting the JIT compiler introduce additional GC-related code at some safe points within a method. The algorithm also employs some other techniques to take advantage of multiple processors, if available. A detailed description of these techniques can be found in [Ric-00b].

Note that all these mechanisms for performance improvement are transparent to your application.

GC Performance Counters

The runtime provides many performance counters that show the status of managed memory for a specific process. These counters are grouped under the .NET CLR memory performance object.

By default, the GC takes place when an object is being created and there is not enough space left in the managed heap. However, it is also possible to programmatically force a GC within the application. The framework provides a class, GC (namespace System), to deal with GC. The following line of code forces a GC:

GC.Collect();

Generally, it is best to let the garbage collector run on its own accord. However, as your application knows more about its behavior than the run-time, you could call this method at some strategic places in your code. For example, it would be a good time to force GC when your application is sitting idle, perhaps waiting for the user's input.

By default, the runtime creates a separate thread to run the GC concurrently. However, it is possible to specify the runtime to run the GC on the same thread as the application. This is done by means of configuration setting gcConcurrent, as shown here:

<configuration>
   <runtime>
       <gcConcurrent enabled="false"/>
   </runtime>
</configuration>

Running GC concurrently reduces performance. For applications based on a user interface, it makes sense to run GC concurrently so the application does not appear to pause. However, for a background application not dependent on the user interface, it is advisable to turn off concurrent GC.

Generations

The GC algorithm has many features to improve collection performance. One such feature is called generations, which based on the following assumptions:

• The newer an object is, the shorter its lifetime will be. For example, variables local to a method are created once the method is entered and are of no use once the method is exited.

• Newer objects tend to have a strong relationship with each other and are frequently accessed around the same time.

• Compacting a portion of the heap is faster than compacting the whole heap.

Many studies have demonstrated that these assumptions are valid for a large set of existing applications.

Under .NET, the managed heap is logically (not physically) grouped into zones called generations. The first release of .NET runtime contains three generations (numbered 0–2, inclusive) and most likely will stay the same for future releases.

When an object is created, it is stored in generation 0. Simply stated, objects in generation 0 are young objects and have not been touched by the garbage collector.

As more objects are created, they are placed in generation 0. When generation 0 fills up, a GC is performed. Those objects that survive the collection are considered older and are moved to generation 1. After the collection, generation 0 is empty.

When the next GC occurs, survivors from generation 1 move to generation 2 and survivors from generation 0 move to generation 1.

As generation 2 is currently the highest generation, when the next GC occurs, survivors from generation 2 simply stay there.

The GC class provides a GetGeneration method that can be used to examine the generation of an object. The following code excerpt demonstrates how an object moves up the generation ladder:

// Project GCGenerations

public static void Main() {
     Object o = new Object();
     Console.WriteLine(GC.GetGeneration(o)); // displays 0

     GC.Collect();
     Console.WriteLine(GC.GetGeneration(o)); // displays 1

     GC.Collect();
     Console.WriteLine(GC.GetGeneration(o)); // displays 2
}

How does generational GC improve performance? When the GC occurs, the garbage collector may choose to examine only the objects in generation 0 and ignore the objects in higher generations. After all, the newer an object is, the shorter its lifetime is expected to be. Collecting and compacting just generation 0 objects is likely to reclaim significant amount of space from the heap and be faster than examining all the objects in all generations.

Of course, if collecting generation 0 doesn't provide the necessary amount of storage, then objects in generation 1 can be collected. Failing this, objects in generation 2 can be collected.

Another performance benefit comes from the statistical likelihood that newer objects have a strong relationship with each other and are frequently accessed around the same time. As new objects are allocated contiguously in memory, you gain performance from locality of reference. It is highly likely that all the new objects can reside in the CPU's cache. Accessing the CPU's cache is much faster than accessing RAM. The application will be able to perform most of its manipulation without having cache misses (which forces RAM access).

It is also possible to programmatically force collection at a higher level. Class GC provides an overloaded version of Collect that takes the generation number as a parameter. The following line of code, for example, forces a GC at generation 2.

GC.Collect(2);

A collection at generation 2 automatically implies a collection at generations 1 and 0.

View Generation Status

.NET provides performance counters to view the number of times objects are collected at various generation levels.

Finalization

GC manages memory, but not any other resources. Read that line again. It's important to understand that the GC can release any unused memory but it cannot deal with any other resources. If you have resources such as file handles that are open, it is your responsibility to release such resources.

Consider the following code excerpt for example:

// Project Finalization

class MyFile {

    public MyFile(String fileName) {

      // Call native Win32 API to open a file
      m_handle = CreateFile(fileName,...);
    }
    public String ReadLine() {
      ...
    }

    private IntPtr m_handle; // native Win32 file handle
}

class MyApp {
    public static void Main() {
      MyFile f = new MyFile("Readme.Txt");
      Console.WriteLine(f.ReadLine());
    }
}

Class MyFile opens a file handle in its constructor using the native Windows API called CreateFile. We will look at invoking platform specific calls in a later chapter on interoperability.

When a MyFile object is collected by the garbage collector, the file handle that was opened never gets closed. If you use such an object a few times, soon you start running short of file handles.

Fortunately, .NET provides a feature called finalization that allows an object to clean itself up when it is being collected. The root system object, System.Object, defines a method called Finalize that a derived class can override and provide its cleanup functionality. The following is its prototype:

protected virtual void Finalize();

So, all we need is to override this method in MyFile class and close the file handle, right? Well, not exactly! C# doesn't let you override Finalize. The compiler flags this as an error. Instead, you need to use the destructor semantics to implement your cleanup, as shown here:

class MyFile {
    ...
    ~MyFile() {
      CloseHandle(m_handle);
    }
}

Under the hood, the compiler converts the destructor code to the Finalize method. Here is how the generated code would look:

protected override void Finalize() {
    try {
      CloseHandle(m_handle);
    }finally {
      base.Finalize();
    }
}

Note that in the generated code, the compiler also inserts logic to invoke the base (parent class) type's Finalize method. The compiler does not let you call base type's Finalize method directly. Implementing a destructor is the only way to ensure that base type's Finalize is called.

The finally clause in C# ensures that irrespective of the outcome of the try block, the code in the finally block is always executed.

How does finalization work? When an object is being collected, the garbage collector sees that the type has a Finalize method and calls the method.

C# Destructor versus C++ Destructor

At first glance, a C# destructor looks very similar to a C++ destructor. However, they are two very different beasts. The invocation of C++ destructors is deterministic. They are called when an object gets deleted or when a variable goes out of scope. Under C#, there is no such deterministic finalization. You have no idea at what point in your code the finalizer will be invoked. Under C++, the thread that invokes the destructor is deterministic. Therefore, C++ destructors can use thread-specific features such as thread local storage. Under .NET, the finalizer is invoked by a special runtime thread. Therefore, C# destructors should never use thread-specific features. Under C++, if you do not define a destructor, the compiler generates one for you. Under C#, the compiler does not generate the destructor for you. As we will see later, finalization under .NET is an expensive process and should be avoided if possible. Under C++, the order of destructors is deterministic. For example, if a class contains member fields, the destructor for the class (the outer object) is invoked before the destructor of member fields (inner objects). In C#, the order of Finalize is not guaranteed. The inner objects might get finalized even before the outer object does. Therefore, the Finalize method must never try to access any inner objects.

Although finalization seems pretty simple on the surface, the internal workings of finalization are not that simple. Essentially, an object that has a Finalize method is placed in a separate data structure (called the Finalization queue) within the managed heap. When a GC occurs, the garbage objects from the Finalization queue are moved to a different queue called the Freachable queue. A special runtime thread is dedicated to calling Finalize on all the objects in the Freachable queue.

Finalization Gotchas

Because of a separate dedicated thread handling the finalization, there are some things that you should be aware of: You should not use any thread-specific feature in your finalizer. It is not your thread. Your finalization code should be as quick to execute as possible. There are other objects waiting in the queue behind you. You should avoid all actions that would block the Finalize method, including any thread-synchronization operations.

Richter's article [Ric-00b] is a good reference to understanding the internals of finalization. For now, it is important to know that invoking finalization on an object is performance intensive and therefore should be avoided when possible.

So now you completely understand that implementing Finalize (i.e., the C# destructor) should be avoided as much as possible, however, your original problem is still not solved. You cannot leave the file handles open. What possible choice do you have to force closing such opened resources?

Disposing Resources

A simple way to clean up the object is to provide an additional method that the consumer of your type can explicitly call. For types that “open” resources, a method name such as Close makes sense for closing the opened resources. The following code excerpt illustrates this:

// Project Dispose

    public void Close() {
      if (INVALID_HANDLE_VALUE != m_handle) {
        CloseHandle(m_handle);
        m_handle = INVALID_HANDLE_VALUE;
      }
    }

Although you can define a method such as Close in your class, and it is quite appropriate, the framework formalizes this notion of explicit cleanup of an object. A type that wishes to expose such functionality must implement a framework-defined interface, IDisposable. Here is its prototype:

interface IDisposable {
     void Dispose();
}

The following code excerpt illustrates how our class MyFile can be modified to support this interface:

// Project Dispose

class MyFile : IDisposable{
    ...

    public void Dispose() {
      if (INVALID_HANDLE_VALUE != m_handle) {
        CloseHandle(m_handle);
        m_handle = INVALID_HANDLE_VALUE;
      }
    }
}

The client code can now explicitly call Dispose to dispose of the resources.

Let's back up a little. The reason we are going through these extra steps is to avoid finalization. However, the finalizer code is still there. We haven't really fixed the actual problem. Let's take out the destructor that we defined. That should fix the problem, right?

Our problem is not really that we added a finalizer on the class. Our problem is that finalization is still being done on our object. If the client calls Dispose, it makes sense that the finalization be suppressed. This is why class GC defines a method, SuppressFinalize, to suppress finalization on an object. Using this method, we can modify our Dispose method as follows:

// Project Dispose

    public void Dispose() {
      if (INVALID_HANDLE_VALUE != m_handle) {
        CloseHandle(m_handle);
        m_handle = INVALID_HANDLE_VALUE;
      }
      GC.SuppressFinalize(this);
    }

Now we have covered all the bases. If the client calls Dispose, finalization is suppressed. If the client forgets to call Dispose, the GC invokes the finalizer as usual.

There is now a new issue we need to consider. We have three methods, Close, Dispose, and Finalize, that are doing pretty much the same thing. From a software engineering point of view, it makes sense that they all call the same method internally. This gives us a single point of maintenance. More lines of code also imply more chances of introducing bugs. For instance, look back at our implementation of Close. Even this method must call SuppressFinalize, which we didn't.

Before we do that, there is a subtle difference between Dispose and Finalize that you need to be aware of. Recall that when Finalize is invoked, its inner objects may have already been collected. The lifetime of these objects is managed by the garbage collector. Therefore, Finalize cannot touch these objects. However, the garbage collector has no control over the resources that it doesn't manage, such as the native file handle in MyFile. Therefore, Finalize should clean up only its unmanaged resources. As a matter of fact, if your class does not have any unmanaged resources, there is no need to implement Finalize.

Note that an unmanaged resource can be wrapped in a managed class, saving others from implementing finalizers. For example, for a class that either inherits from MyFile or uses the MyFile type as a member field, there is no need for the class to implement Finalize on account of MyFile.

Avoid Finalize if Possible

If your class does not have any unmanaged resources, do not implement Finalize on the class. Implementing Finalize for this case does not serve any real purpose. Moreover, you pay a performance penalty for doing so.

Dispose, on the other hand, can and should clean its inner objects and should call its base type's Dispose (or equivalent method), if available.

Based on this subtle difference, we can rewrite our methods as follows:

// Project ABetterDispose

class MyFile : IDisposable {
    ...
    // Common resource clean-up implementation
    protected virtual void Dispose(bool disposing) {
      if (disposing) {
        // ... dispose managed resources
        }
        // ... dispose unmanaged resources
        if (INVALID_HANDLE_VALUE != m_handle) {
          CloseHandle(m_handle);
          m_handle = INVALID_HANDLE_VALUE;
        }
    }

    public void Close() {
      Dispose();
    }

    public void Dispose() {
      Dispose(true);
      GC.SuppressFinalize(this);
    }

    ~MyFile() {
      Dispose(false);
    }
}

This pattern of using a helper Dispose method that can be used from IDisposable.Dispose implementation as well as from Finalize and any other method is generally referred to as the Dispose pattern.

You can use this code as a template for designing a class that needs the dispose and finalize semantics.

Implement Dispose Along with Finalize

Make it a habit to always implement Dispose on a class if you implement Finalize on the class. Do remember to call SuppressFinalize within your Dispose implementation. There is a subtle problem if you implement Finalize without implementing Dispose. A class derived from such a class cannot dispose of the parent class from its Dispose method. There are only two ways to dispose of a class—either call Dispose or call Finalize. However, the parent class doesn't implement Dispose and the compiler won't let you call parent class's Finalize directly. The implication of this is that the child class should not call SuppressFinalize from its Dispose. Otherwise, the Finalize for the parent class will get suppressed, resulting in a resource leak.

Using IDisposable Objects

Once a class implements IDisposable, the users of the class can call Dispose when done with the object. This is illustrated in the following code excerpt:

public static void Main() {
  MyFile f = new MyFile("Readme.Txt");
  Console.WriteLine(f.ReadLine());
  f.Dispose();
}

Although this code works, there is a slight programming issue here. Programmers have to remember to call Dispose on an object, even in the face of an exception. For a method with many return paths, it is easy to forget calling Dispose on one of the return paths.

Now you have a new problem. You are relying on your clients to call Dispose on you. What if they forget to do so?

C# offers a better syntactic flavor for dealing with objects that implement IDisposable. An object can be created within the using scope as follows:

// Project ABetterDispose

public static void Main() {
  using (MyFile f = new MyFile("Readme.Txt")) {
    Console.WriteLine(f.ReadLine());
  }
}

The compiler expands this to something like:

public static void Main() {
  {
    MyFile f = new MyFile("Readme.Txt");
    try {
      Console.WriteLine(f.ReadLine());
    }finally (
      if (null != f) {
        ((IDisposable) f).Dispose();
      }
    }
  }
}

All the code within the using scope is moved into the try block. Once this code is executed, the code in the finally block is executed, ensuring that Dispose is called on an IDisposable object that was successfully constructed.

Note that the using clause can be nested, making it possible to create multiple IDisposable objects.

Verifying If Dispose() Is Called

If you have implemented Dispose as well as Finalize on your class, but you are expecting that users of the class call Dispose on the object, you can add a Debug.Write in your Finalize method. If you see a trace output during testing, you can check the user code to see why Dispose was not called.

A final note on calling Dispose: It is important to understand that calling Dispose on an object implies strong ownership of the object. If an object is being referenced by many other objects, and the ownership of the object is not clear, then Dispose should not be called on the object. Otherwise, some other object may end up using an already disposed object, which would result in an unpredictable behavior. By the same logic, Dispose should not be called multiple times. These are general considerations. If your specific needs require that Dispose is callable multiple times or from multiple threads, you must add appropriate safety to your implementation of Dispose.