Understanding the Problem

As you saw in Chapter 1, the Compiler API is vast, and it’s easy to get lost. Knowing exactly what the problem is and all the potential variants that can crop up in code can be overwhelming, depending on the nature of the issue you’re trying to find. No matter how well-designed the Compiler API is, we’re still dealing with tokens, parsing and semantics, so we need to have a good understanding of what we’re trying to find in code.

Let’s go through an example of enforcing a particular coding standard. In my experience I’ve sometimes seen frameworks expose classes that defined virtual methods that had to be invoked if the method was overriden. For example, a base class may look something like this:

public class MyBaseClass
{
  protected virtual void OnInitialize() { /* ... */ }
}

If you inherited from this class, you needed to do this:

public class MySubClass
  : MyBaseClass
{
  protected override void OnInitialize()
  {
    base.OnInitialize();
    /* ... */  
  }
}

In other words, you had to call the base class’s implementation, and then you could add in whatever implementation you needed to do.

The problem with this approach is that there’s no way to enforce it with the C# compiler. Overridden methods are not required to call the “base” implementation, yet with some designs this is a requirement. Unfortunately, you could easily override OnInitialize() and forget to call the base class’s implementation and the C# compiler will happily produce an assembly based on your errant code. Adding XML documentation to the method helps, but it doesn’t enforce that requirement, and that’s what we really need.

So, let’s write a diagnostic that checks if the base class’s implementation is invoked somewhere within our overridden implementation. We’ll only concern ourselves with virtual methods that are marked with a MustInvokeAttribute. If a subclass overrides that method, it must call the base class’s implementation. That sounds like a good plan, but what are we going to need to look for in a syntax tree to handle this analysis correctly? In the next section, we’ll use the Syntax Visualizer to get a clearer picture of the possibilities.

Using the Syntax Visualizer

Getting a vision for what an analyzer will need to do from a design perspective is a good thing, but we’ll need more information. Specifically, we have to dig into a syntax tree to find the nodes to see what it is we’re really looking for—for example, we’ll need to find the nodes related to method calls and whether or not they’re virtual. Figure 2-1 shows the tree for the code you saw in the “Understanding the Problem” section.

Figure 2-1. Discovering which nodes are in play for method invocations

To get this screenshot, I highlighted the “OnInitialize” base method call in the overriden method. That’s why the IdentifierNameSyntax node is highlighted. It’s a child of an InvocationExpressionSyntax node, which in turn is a child of a MethodDeclaration.

That tells us about the definition of a method and its structure, but we want to find out that if a method is an override, and the method it’s overriding has [MustInvoke], then we have to find at least one invocation of that base class method in the method’s definition.

We’ll attack the problem from two angles. If the user is adding a method invocation in code, we’ll keep looking at the parent nodes until we either get a null or a MethodDeclarationSyntax reference . If the user is declaring or changing a method declaration, we want to examine that “root” node for child invocations. Either way, once we get a MethodDeclarationSyntax reference, we’ll check to see if it’s overriding a method that is marked with [MustInvoke]. If it is, then we’ll look through all of its descendants for InvocationExpressionSyntax nodes and determine if at least one of those invocations is the base method. If we don’t find any, then we’ll report that as a diagnostic issue. To implement this approach, we’ll need to work with both syntax nodes and objects that come from a semantic model, as you’ll see in the next section.

By the way, where is this MustInvokeAttribute class defined? I put it into a separate assembly called MustInvokeBaseMethod. It’s a good idea to separate the analyzer from the rest of your main code that you’ll be analyzing. Even though in this case we have an assembly with only one attribute, we still want that diagnostic code out of what would normally be an assembly that has all of our logic and structure in it.

In the next section, we’ll finally get into the details of a diagnostic.

Using the Template

In the “Visualizing Trees” section in Chapter 1, we walked through a couple of installation steps to get the Syntax Visualizer in place. This also installed a couple of templates to make it easier to create diagnostics and refactorings. Let’s use them to get our projects in place.

Create a solution in Visual Studio, and add a new project to that solution (File ➤ New ➤ Project, or Ctrl+Shift+N). You should see an option under the Extensibility node called Analyzer with Code Fix (NuGet + VSIX) as shown in Figure 2-2.

Figure 2-2. Creating a new diagnostic project

The resulting solution will have three new projects in it, as Figure 2-3 shows.

Figure 2-3. Creating a diagnostic project creates three new projects

Here’s a brief description of these new projects:

• MustInvokeBaseMethod.Analyzers—this Portable Class Library-based project is where you’ll define your analyzer and any code fixes.

• MustInvokeBaseMethod.Test—this .NET MSTest-based project references your analyzer project so you can write tests against your diagnostic. We’ll talk about this project later in the “Unit Testing” section.

• MustInvokeBaseMethod.Vsix—this VS Package-based project references your analyzer and allows you to quickly test your analyzer code in a new Visual Studio instance.

When you create a diagnostic project, the template will generate a diagnostic that looks for types with lowercase letters and provides a code fix to make the name all uppercase. It’s kind of a silly rule, but it does illustrate the machinery you need in place to ensure your diagnostic is implemented correctly. In the next section, we’ll build the diagnostic and code fix for our base virtual method scenario, but if you’ve never built one before, I encourage you to take a look at that code just to see how it’s laid out.

One other project that we’ll add is the MustInvokeBaseMethod project. All this will contain is the MustInvokeAttribute—its definition looks like this:

[AttributeUsage(AttributeTargets.Method,
  AllowMultiple = false, Inherited = true)]
public sealed class MustInvokeAttribute
  : Attribute
{ }

This may seem like a bit of overkill, having a project to contain just one type, but it’s best to separate your analyzer code from the code you’re going to analyze. Remember, the analyzer project is a Portable Class Library (PCL) , and it can’t reference assemblies that are not PCLs, so if you put all your code into the same assembly, this would force your main code to follow the same PCL restrictions. PCLs have a limited set of APIs that they can use from the .NET Framework, and, depending on what your code needs to do, you may end up not being able to implement certain features. Also, keep in mind that when you keep code separated in a non-PCL project, the analyzer project can’t reference that MustInvokeBaseMethod project, but that’s okay. Keeping a strict boundary between the analyzer and the code to analyze also removes issues like versioning, and most of the time you’ll discover nodes via their names, not by a specific type.

Building the Diagnostic

We’ve covered the basics of the diagnostic design and how to use the analyzer project template. Let’s get into the details of how you set up and implement a diagnostic.

[DiagnosticAnalyzer(LanguageNames.CSharp)]
public sealed class MustInvokeBaseMethodAnalyzer
  : DiagnosticAnalyzer
{
  /* ... */
}

private static DiagnosticDescriptor rule = new DiagnosticDescriptor(
  "MUST0001", "Find Overridden Methods That do Not Call the Base Class's Method",
  "Virtual methods with [MustInvoke] must be invoked in overrides",
  "Usage", DiagnosticSeverity.Error, true);

public override ImmutableArray<DiagnosticDescriptor> SupportedDiagnostics
{
  get
  {
    return ImmutableArray.Create(
      MustInvokeBaseMethodAnalyzer.rule);
  }
}

public override void Initialize(AnalysisContext context)
{
  context.RegisterSyntaxNodeAction<SyntaxKind>(
    this.AnalyzeMethodDeclaration, SyntaxKind.MethodDeclaration);
}

private void AnalyzeMethodDeclaration(SyntaxNodeAnalysisContext context)
{
  var method = context.Node as MethodDeclarationSyntax;
  var model = context.SemanticModel;
  var methodSymbol = model.GetDeclaredSymbol(method) as IMethodSymbol;

  context.CancellationToken.ThrowIfCancellationRequested();  

  if(methodSymbol.IsOverride)
  {
    var overriddenMethod = methodSymbol.OverriddenMethod;

    var hasAttribute = false;
    foreach (var attribute in overriddenMethod.GetAttributes())
    {
      if(attribute.AttributeClass.Name == "MustInvokeAttribute")
      {
        hasAttribute = true;
        break;
      }
    }

    context.CancellationToken.ThrowIfCancellationRequested();

    if(hasAttribute)
    {
      var invocations = method.DescendantNodes(_ => true)
        .OfType<InvocationExpressionSyntax>();


      foreach (var invocation in invocations)
      {
        var invocationSymbol = model.GetSymbolInfo(
          invocation.Expression).Symbol as IMethodSymbol;

        if (invocationSymbol == overriddenMethod)
        {
          return;
        }
      }

      context.ReportDiagnostic(Diagnostic.Create(
        MustInvokeBaseMethodAnalyzer.rule,
        method.Identifier.GetLocation()));
    }
  }
}

Designing the Fix

Let’s start by looking at a couple of coding conditions that can happen in a method. If our base method looks like this:

protected virtual void OnInitialize()

It’s pretty easy to create a fix—all we need to do is call it like this:

base.OnInitilize();

Therefore, we just need to generate an invocation. But there’s far more you can do with a method. What if the method is defined like this?

unsafe protected virtual int[] OnInitialize(string a, Guid b, char* c, ref long d);

Now we have a return value, arguments that take pointer types and a ref argument as well. Fixing it would take code that looks like this:

var onInitializeResult = base.OnInitialize(a, b, c, ref d);

Fortunately, we can just pass in the parameters from the overriden method to this method invocation. The developer may want to change that, but a code fix can do this as a simplistic heuristic and let the developer choose to alter it as needed. The developer can also ignore the code fix entirely and correct the issue manually. Providing a code fix does not require the developer to use it; it’s just another way to assist the developer.

Of course, we’re assuming that there isn’t a variable called onInitializeResult. We’ll need to make sure that we check all local variables and see if there are any that have that name. If so, we’ll have to come up with some heuristic to ensure the name is unique without coming up with an ugly name with random characters. Writing a good code fix means you try to generate code that a developer would accept as their own.

So, here’s what we need to do with this code fix:

• Determine if there are any arguments for the method invocation. If so, use the current method definition’s parameters.

• Determine if there’s a return value. If so, we’ll capture that with a simple “var” statement, generating a variable name that is unique.

• We’ll also make sure that our added invocation is the first statement that occurs in the overridden method.

Now that we have a plan in place, let’s implement the code fix.

Implementing the Fix

First, here’s the definition of the code fix class:

[ExportCodeFixProvider(LanguageNames.CSharp)]
[Shared]
public sealed class MustInvokeBaseMethodCallMethodCodeFix
  : CodeFixProvider
{
  /* ... */
}

You need to add the ExportCodeFixProvider and Shared attributes as well as inherit from the CodeFixProvider class for your code fix to work correctly. When you inherit from CodeFixProvider, there are two members that you must provide implementations for: the FixableDiagnosticIds property and the RegisterCodeFixesAsync method . The property is easy to implement:

public override ImmutableArray<string> FixableDiagnosticIds
{
  get
  {
    return ImmutableArray.Create("MUST0001");
  }
}

The identifier we pass into the array is the same one that the diagnostic uses for the rule it reports when a code violation is detected. This is used to tie the rule and fix together so Visual Studio can provide the fix for the issue.

Before we get to RegisterCodeFixesAsync()(in the next section), there’s another virtual member that you don’t have to override, but you should: GetFixAllProvider():

public override FixAllProvider GetFixAllProvider()
{
  return WellKnownFixAllProviders.BatchFixer;
}

This tells Visual Studio that if there are other occurrences of this code issue within a selected scope (document, project, or solution), Visual Studio will automatically apply the fix within that scope. The default implementation for this method in CodeFixProvider is to return null, so if you want Visual Studio to do fixes for you solution-wide, you should have the method return BatchFixer.

Now that the boilerplate is out of the way, let’s add this fix. We’re going to do it two ways: one that generates trees and the other that parses a statement in a string. Let’s do the tree approach first.

Using Syntax Trees

Here’s the implementation for creating a base method invocation. We’ll start with the general implementation of RegisterCodeFixesAsync(), which is shown in Listing 2-4.

Listing 2-4. Implementing RegisterCodeFixesAsync() in a Code Fix class.

public override async Task RegisterCodeFixesAsync(CodeFixContext context)
{
  var root = await context.Document.GetSyntaxRootAsync(
    context.CancellationToken).ConfigureAwait(false);

  context.CancellationToken.ThrowIfCancellationRequested();

  var diagnostic = context.Diagnostics[0];
  var methodNode = root.FindNode(diagnostic.Location.SourceSpan) as MethodDeclarationSyntax;

  var model = await context.Document.GetSemanticModelAsync(  context.CancellationToken);
  var methodSymbol = model.GetDeclaredSymbol(methodNode) as IMethodSymbol;

  var invocation = MustInvokeBaseMethodCallMethodCodeFix.CreateInvocation(
    methodSymbol);
  invocation = MustInvokeBaseMethodCallMethodCodeFix.AddArguments(
    context, methodSymbol, invocation);
  var statement = MustInvokeBaseMethodCallMethodCodeFix.CreateStatement(
    context, methodNode, methodSymbol, invocation);
  var newRoot = MustInvokeBaseMethodCallMethodCodeFix.CreateNewRoot(
    root, methodNode, statement);

  const string codeFixDescription = "Add base invocation";
  context.RegisterCodeFix(
    CodeAction.Create(codeFixDescription,
     _ => Task.FromResult(context.Document.WithSyntaxRoot(newRoot)),
     codeFixDescription), diagnostic);
}

The first thing we do is get the MethodDeclarationSyntax node for the method that had the issue in the first place. We get its location from the Diagnostics array property and then find it from the SyntaxNode tree node. Then we pull its related IMethodSymbolfrom the semantic model, which will make our lives a little easier later on.

Now we build our InvocationExpressionSyntax node in CreateInvocation():

private static InvocationExpressionSyntax CreateInvocation(
  IMethodSymbol methodSymbol)                                                                      
{
  return SyntaxFactory.InvocationExpression(
    SyntaxFactory.MemberAccessExpression(
          SyntaxKind.SimpleMemberAccessExpression,
      SyntaxFactory.BaseExpression().WithToken(
        SyntaxFactory.Token(SyntaxKind.BaseKeyword)),
        SyntaxFactory.IdentifierName(
          methodSymbol.Name))
    .WithOperatorToken(
        SyntaxFactory.Token(
          SyntaxKind.DotToken)));
}                                                  

This code generates an expression like this: “base.BaseMethod() ”. Calling BaseExpression() provides the “base” keyword, and IdentifierName() creates the name of the method to invoke.

Next, we have to add any arguments to the invocation if needed—this is shown in Listing 2-5.

Listing 2-5. Adding arguments for the base method invocation.

private static InvocationExpressionSyntax AddArguments(  CodeFixContext context, IMethodSymbol methodSymbol,
  InvocationExpressionSyntax invocation)
{
  context.CancellationToken.ThrowIfCancellationRequested();

  var argumentCount = methodSymbol.Parameters.Length;
  if (argumentCount > 0)
  {
    // Define an argument list.
    var arguments = new SyntaxNodeOrToken[(2 * argumentCount) - 1];

    for (var i = 0; i < argumentCount; i++)
    {
      var parameter = methodSymbol.Parameters[i];
      var argument = SyntaxFactory.Argument(
        SyntaxFactory.IdentifierName(parameter.Name));                                                                              

      if (parameter.RefKind.HasFlag(RefKind.Ref))
      {
        argument = argument.WithRefOrOutKeyword(
          SyntaxFactory.Token(SyntaxKind.RefKeyword));
      }
      else if (parameter.RefKind.HasFlag(RefKind.Out))
      {
        argument = argument.WithRefOrOutKeyword(
          SyntaxFactory.Token(SyntaxKind.OutKeyword));
      }

      arguments[2 * i] = argument;

      if (i < argumentCount - 1)
      {
        arguments[(2 * i) + 1] = SyntaxFactory.Token(SyntaxKind.CommaToken);
      }                                                        
    }

    invocation = invocation.WithArgumentList(
      SyntaxFactory.ArgumentList(
        SyntaxFactory.SeparatedList<ArgumentSyntax>(arguments))
      .WithOpenParenToken(SyntaxFactory.Token(SyntaxKind.OpenParenToken))
      .WithCloseParenToken(SyntaxFactory.Token(SyntaxKind.CloseParenToken)));
  }

  return invocation;
}

If arguments are necessary, we build up the argument list via SyntaxFactory.Argument(). If the parameter is a ref or an out, we add those keywords with WithRefOrOutKeyword. We also have to separate each argument with a SyntaxToken of kind SyntaxKind.CommaToken. That’s why the arguments array’s size looks a little awkward at first. For example, if we have four arguments, we need to generate four ArgumentSyntax objects plus three SyntaxToken representing the commas between the arguments, which means the array will have seven elements. Once we have all of the arguments, we add them to invocation via WithArgumentList(), remembering to reassign invocation to the return value.

Now we need to create a StatementSyntax node that will contain the invocation. This is where we’ll handle the call if it returns a value or not, which is showing in Listing 2-6.

Listing 2-6. Creating a statement for the method invocation

private static StatementSyntax CreateStatement(CodeFixContext context,
  MethodDeclarationSyntax methodNode, IMethodSymbol methodSymbol,
  InvocationExpressionSyntax invocation)
{
  context.CancellationToken.ThrowIfCancellationRequested();

  StatementSyntax statement = null;                                                                              

  if (!methodSymbol.ReturnsVoid)
  {
    var returnValueSafeName = CreateSafeLocalVariableName(
      methodNode, methodSymbol);

    statement = SyntaxFactory.LocalDeclarationStatement(
      SyntaxFactory.VariableDeclaration(
        SyntaxFactory.IdentifierName("var"))
      .WithVariables(SyntaxFactory.SingletonSeparatedList  <VariableDeclaratorSyntax>(
        SyntaxFactory.VariableDeclarator(
          SyntaxFactory.Identifier(returnValueSafeName))
      .WithInitializer(SyntaxFactory.EqualsValueClause(invocation)))));
  }
  else
  {
    statement = SyntaxFactory.ExpressionStatement(invocation);
  }

  return statement.WithAdditionalAnnotations(Formatter.Annotation);
}

If we have to handle a return value, we create a local variable and assign the return value to that. We get the name of this local variable from CreateSafeLocalVariableName()– Listing 2-7 shows what it looks like.

Listing 2-7. Creating a safe variable name for a return value

private static string CreateSafeLocalVariableName(
  MethodDeclarationSyntax methodNode, IMethodSymbol methodSymbol)
{
  var localDeclarations = methodNode.DescendantNodes(
    _ => true).OfType<VariableDeclaratorSyntax>();
  var returnValueName =
    $"{methodSymbol.Name.Substring(0, 1).ToLower()}{methodSymbol.Name.Substring(1)}Result";
  var returnValueSafeName = returnValueName;
  var returnValueCount = 0;                                                                              

  while (localDeclarations.Any(_ =>   _.Identifier.Text == returnValueSafeName))
  {
    returnValueSafeName = $"{returnValueName}{returnValueCount}";
    returnValueCount++;
  }

  return returnValueSafeName;
}

We use the name of the method as a base for the variable name (making it camel-cased in the process), and keep adding a numeric value to the end until we find a unique name. The chances of us even getting into the while loop once is extremely small, but with this code in place we don’t ever have to worry about running into a collision.

Finally, once the statement is generated, we register a code fix on the context via RegisterCodeFix(), which uses a new root created from CreateNewRoot():

private static SyntaxNode CreateNewRoot(
  SyntaxNode root, MethodDeclarationSyntax methodNode,
  StatementSyntax statement)
{
  var body = methodNode.Body;
  var firstNode = body.ChildNodes().FirstOrDefault();

  var newBody = firstNode != null ?
    body.InsertNodesBefore(body.ChildNodes().First(),
      new[] { statement }) :
    SyntaxFactory.Block(statement);

  var newRoot = root.ReplaceNode(body, newBody);
  return newRoot;
}

Notice that we have to be a bit careful in the case in which the method doesn’t have any code in it. In that case, the body won’t have any child nodes, so we can just create a new BlockSyntax node. Otherwise, we insert our new StatementSyntax node as the first child node in the method’s body.

Parsing Statements

Although creating trees like the ones you saw in the previous section aren’t too hard, it’s also not an easy endeavor. I had to use a combination of the Syntax Visualizer and RoslynQuoter tools to ensure I was generating the right nodes. This is kind of tedious. Fortunately, there’s an alternative approach that you can use in some scenarios that results in a lot less code than creating trees and nodes manually. You can generate the code you want as a string, and then use SyntaxFactory’s ParseStatement()to give you a StatementSyntax node directly. Listing 2-8 shows how that’s done.

Listing 2-8. Using ParseStatement() to generate a tree

private static StatementSyntax CreateStatement(
  MethodDeclarationSyntax methodNode, IMethodSymbol methodSymbol)
{
  var methodParameters = methodSymbol.Parameters;
  var arguments = new string[methodParameters.Length];

  for(var i = 0; i < methodParameters.Length; i++)
  {
    var parameter = methodParameters[i];
    var argument = parameter.Name;

    if (parameter.RefKind.HasFlag(RefKind.Ref))
    {
      argument = $"ref {argument}";
    }
    else if (parameter.RefKind.HasFlag(RefKind.Out))
    {
      argument = $"out {argument}";
    }

    arguments[i] = argument;
  }

  var methodCall =
    $"base.{methodSymbol.Name}({string.Join(", ", arguments)});{Environment.NewLine}";

  if(!methodSymbol.ReturnsVoid)
  {
    var variableName = MustInvokeBaseMethodCallMethodCodeFix.CreateSafeLocalVariableName(
      methodNode, methodSymbol);
    methodCall = $"var {variableName} = {methodCall}";
  }

  return SyntaxFactory.ParseStatement(methodCall)
    .WithAdditionalAnnotations(Formatter.Annotation);                                                        
}

Logically, this code ends up at the same spots as the other approach but with a lot less code. Essentially, a statement is created, like “var onInitializeResult = base.OnInitialize(a, b);”, and then this string is passed into ParseStatement(). That’s it! Note that there are other “Parse” methods on SyntaxFactory, like ParseExpression()and ParseArgumentList(). Depending on the kind of code fix you need to make, it may be easier to generate the code in a string and let SyntaxFactory do the heavy lifting.

Executing the Diagnostic and Code Fix

After all that work, it’s finally time to see our code run in Visual Studio. The easiest way to do this is to make the VSIX project the startup solution and run that project. It’ll create a new instance of Visual Studio with the analyzer installed as an extension (we’ll talk more about deployments in the “Deploying and Installing Diagnostics” section). Create a new solution, and create a base class with a virtual method that has the [MustInvoke] on it.

Create a class that inherits from the base class, override the virtual method, but don’t call it. You should see a red squiggle under the method definition, as Figure 2-4 shows.

Figure 2-4. Getting the diagnostic to display in Visual Studio

Now, move the cursor so it’s on the method definition, and press Ctrl + “ . ” . You should see the code fix window pop up with a code diff view, as shown in Figure 2-5.

Figure 2-5. Getting the code fix to display in Visual Studio

Notice the options at the bottom of the popup. If there were other base method invocation issues like this in the document, project, or solution, we could choose to fix them all at once. Applying the fix makes the error go away!

We can also make the method take some arguments and return a value—the code fix handles it as expected. If we had this code in place:

protected override int OnInitialize(int a, string b)                                                                      
{
  return 43;
}

Figure 2-6 shows what the code fix would do.

Figure 2-6. Calling the base method with a return value

If we had a variable that collided with our initial choice, Figure 2-7 shows that we won’t collide with it.

Figure 2-7. Generating a local variable with a unique name

Diagnostics and code fixes are formidable tools to have, and they’re not as complex as they may seem at first glance. The core of this diagnostic is only 40 lines of code, while the code fix (using the parsing technique) is under 50 lines of code. The key is to figure out what you’re targeting in your code and the possible cases you can run into.

Having the ability to write diagnostics is powerful because you’re able to enforce coding standards and desired idioms along with providing automatic code fixes, but you’re still dealing with parsing, trees, and so on. It’s crucial to have good testing in place to ensure that your code works as expected. That’s what we’ll cover in the next section.

Unit Testing

When I am consulting at a client, one factor that I use to determine the health of a project is this: are there a large suite of unit tests that developers can run quickly? This isn’t a guarantee that the project is stable, but it’s definitely a key aspect. Given how complex compiler code can get, it’s important to have those tests in place.

When you create the diagnostic project, the template creates an MSTest-based class library with some helper code to assist you with writing tests that target analyzers and code fixes. Over time I’ve created my own set of helper methods, which is included in the code for this book. As you get more familiar with the Compiler API you’ll probably add your own as well. You may choose to delete the project and create your own test project using a different testing framework like NUnit, xUnit or Fixie. Whatever you choose to do, I strongly encourage you to write tests, especially if you end up changing your implementation. Even if you don’t change your code after you’ve written it once, write the tests!

Let’s begin by looking at some tests you could write for the analyzer. In the first scenario, the test will check to ensure the analyzer doesn’t fire if a base method has [MustInvoke] but the overriden method calls the base method. In the second scenario, the text should fire the analyzer as the overriden method does not invoke the base method. There are other cases to cover (and they’re in the source code) but we'll just cover these two in the book. Here’s the first scenario:

[TestMethod]
[Case("Case2")]
public async Task Case2Test()
{
  await TestHelpers.RunAnalysisAsync<MustInvokeBaseMethodAnalyzer>(
    $@"Targets{nameof(MustInvokeBaseMethodAnalyzerTests)}Case2.cs",
    new string[0]);
}

The TestHelpers class contains a number of methods that I use to make analyzer and code fix tests easier to write. I’ll show you what RunAnalysisAsync() does in a moment, but first let’s address some other aspects of the test.

The actual name of the test method is longer in the code, so I shortened it for the book. The CaseAttribute is just a way to mark which case is being tested. As you can see in the partial file path, that name is used to put test code in separate folders so they can be loaded in the right test. Here’s what Case2.cs looks like:

using MustInvokeBaseMethod;

namespace
  MustInvokeBaseMethod.Analyzers.Test.Targets.MustInvokeBaseMethodAnalyzerTests
{
  public class Case2Base
  {
    [MustInvoke]
    public virtual void Method() { }
  }

  public class Case2Sub
    : Case3Base
  {
    public override void Method()
    {
      base.Method();
    }
  }
}

The Build Action property for this file in Visual Studio is set to “None”, and the Copy to Output Directory value is set to “Copy always”. This way, you can use the Syntax Visualizer in Visual Studio to see the structure of your code and find the location of the span where a diagnostic will report an error without having the code compiled as part of your test assembly. Plus, by copying it to the output directory you can load its textual context and give it to an analyzer. Let’s now take a look at RunAnalysisAsync()to see how all of the test code ties together—this is in Listing 2-9.

Listing 2-9. Testing diagnostics against source code.

internal static async Task RunAnalysisAsync<T>(string path,
  string[] diagnosticIds,
  Action<ImmutableArray<Diagnostic>> diagnosticInspector = null)
  where T : DiagnosticAnalyzer, new()
{
  var code = File.ReadAllText(path);
  var diagnostics = await TestHelpers.GetDiagnosticsAsync(
    code, new T());
  Assert.AreEqual(diagnosticIds.Length, diagnostics.Count(),
    nameof(Enumerable.Count));

  foreach (var diagnosticId in diagnosticIds)
  {
    Assert.IsTrue(diagnostics.Any(_ => _.Id == diagnosticId),
         diagnosticId);
  }                                                                              

  diagnosticInspector?.Invoke(diagnostics);
}

RunAnalysisAsync() loads the code in the given file with ReadAllText(). Then it passes that code into another helper method, GetDiagnosticsAsync(), along with a new instance of the diagnostic whose type was specified with the T generic parameter. The following code shows you what that method does:

internal static async Task<ImmutableArray<Diagnostic>> GetDiagnosticsAsync(
  string code, DiagnosticAnalyzer analyzer)
{
  var document = TestHelpers.Create(code);
  var compilation = (await document.Project.GetCompilationAsync())
    .WithAnalyzers(ImmutableArray.Create(analyzer));
  return await compilation.GetAnalyzerDiagnosticsAsync();
}

Via Create() we’re able to get a Document instance. From that, we can compile the project associated with the document, giving it the analyzer we created in RunAnalysisAsync(). Once that’s done, we return the set of diagnostics back to the call. Listing 2-10 shows what Create() does.

Listing 2-10. Creating a document for test code

internal static Document Create(string code)
{
  var projectName = "Test";
  var projectId = ProjectId.CreateNewId(projectName);

  var solution = new AdhocWorkspace()
    .CurrentSolution
    .AddProject(projectId, projectName, projectName,
          LanguageNames.CSharp)
    .WithProjectCompilationOptions(projectId,
      new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary))
    .AddMetadataReference(projectId,
      MetadataReference.CreateFromFile(
            typeof(object).Assembly.Location))
    .AddMetadataReference(projectId,
      MetadataReference.CreateFromFile(
            typeof(Enumerable).Assembly.Location))
    .AddMetadataReference(projectId,
      MetadataReference.CreateFromFile(
            typeof(CSharpCompilation).Assembly.Location))
    .AddMetadataReference(projectId,
      MetadataReference.CreateFromFile(
            typeof(Compilation).Assembly.Location))
    .AddMetadataReference(projectId,                                                                              
      MetadataReference.CreateFromFile(
            typeof(MustInvokeAttribute).Assembly.Location));

  var documentId = DocumentId.CreateNewId(projectId);
  solution = solution.AddDocument(documentId, "Test.cs",
    SourceText.From(code));

  return solution.GetProject(projectId).Documents.First();
}

We need to get a Document object, and the simplest way is to do that through an AdHocWorkspace (I’ll discuss workspaces in Chapter 3). We add a project to the workspace’s CurrentSolution with the appropriate metadata references. Then, we add a C# document to the project based on the given text, and return that document to the caller.

If we need to test for the existence of a diagnostic, Listing 2-11 shows what that looks like.

Listing 2-11. Testing for the presence of a diagnostic

[TestMethod]
[Case("Case3")]
public async Task Case3()
{
  await TestHelpers.RunAnalysisAsync<MustInvokeBaseMethodAnalyzer>(
    $@"Targets{nameof(MustInvokeBaseMethodAnalyzerTests)}Case3.cs",
    new[] { "MUST0001" }, diagnostics =>
    {
      Assert.AreEqual(1, diagnostics.Count(), nameof(Enumerable.Count));
      var diagnostic = diagnostics[0];
      var span = diagnostic.Location.SourceSpan;
      Assert.AreEqual(276, span.Start, nameof(span.Start));
      Assert.AreEqual(282, span.End, nameof(span.End));
    });                                                                      
}

In this test method, we’re testing the case in which an overriden method isn’t invoking the base method like it should. Now we get a diagnostic coming back from the compilation, and we can test that we only get one diagnostic back with its location being the identifier of the method in the subclass. Because we have the code for the test in a C# file in the test project, we can easily use the Syntax Visualizer to find the Start and End values for the span of the diagnostic, as Figure 2-8 shows.

Figure 2-8. Using the Syntax Visualizer to get span values

We also need tests for the code fix. Unlike the analyzer, we can assume that the code fix will only be invoked if there was an issue. However, we have a couple of cases to ensure we handle arguments and return values correctly. Again, the source code has all of the tests cases, but we’ll just cover one here. Listing 2-12 has the test that covers if a base method invocation with no arguments and no return value was added correctly.

Listing 2-12. Verifying that a code fix works correctly

[TestMethod]
[Case("Case0")]
public async Task VerifyGetFixes()
{
  var code = File.ReadAllText(
    $@"Targets{nameof(MustInvokeBaseMethodCallMethodCodeFixTests)}Case0.cs");
  var document = TestHelpers.Create(code);
  var tree = await document.GetSyntaxTreeAsync();
  var diagnostics = await TestHelpers.GetDiagnosticsAsync(
    code, new MustInvokeBaseMethodAnalyzer());
  var sourceSpan = diagnostics[0].Location.SourceSpan;

  var actions = new List<CodeAction>();
  var codeActionRegistration =
    new Action<CodeAction, ImmutableArray<Diagnostic>>(
      (a, _) => { actions.Add(a); });

  var fix = new MustInvokeBaseMethodCallMethodCodeFix();
  var codeFixContext = new CodeFixContext(document, diagnostics[0],
    codeActionRegistration, new CancellationToken(false));
  await fix.RegisterCodeFixesAsync(codeFixContext);                                                                              

  Assert.AreEqual(1, actions.Count, nameof(actions.Count));

  await TestHelpers.VerifyActionAsync(actions,
    "Add base invocation", document,
    tree, new[]
        {
          "
        {
            base.Method();
        }
    
        "});
}

As with the analyzer test, we create a diagnostic via GetDiagnosticsAsync(). This time, we use that diagnostic to create a CodeFixContext. This is passed into the code fix’s RegisterCodeFixesAsync(). The Action<CodeAction, ImmutableArray<Diagnostic>> instance that we pass into the context is used to capture any registered actions with the code fix, which we can use to verify what we expect to happen in this test. The text that is passed into VerifyActionAsync()is the changed text that we expect to see in the new syntax tree. Here’s what VerifyActionAsync() does.

internal static async Task VerifyActionAsync(List<CodeAction> actions,
  string title, Document document,
  SyntaxTree tree, string[] expectedNewTexts)
{
  var action = actions.Where(_ => _.Title == title).First();

  var operation = (await action.GetOperationsAsync(
    new CancellationToken(false))).ToArray()[0] as ApplyChangesOperation;
  var newDoc = operation.ChangedSolution.GetDocument(document.Id);
  var newTree = await newDoc.GetSyntaxTreeAsync();
  var changes = newTree.GetChanges(tree);

  Assert.AreEqual(expectedNewTexts.Length, changes.Count,
    nameof(changes.Count));

  foreach (var expectedNewText in expectedNewTexts)
  {
    Assert.IsTrue(changes.Any(_ => _.NewText == expectedNewText),
      string.Join($"{Environment.NewLine}{Environment.NewLine}",
      changes.Select(_ => $"Change text: {_.NewText}")));
  }
}                                                                              

With these tests in place, we have a high level of confidence in our code and how it should behave. The next section discusses testing our diagnostic code within Visual Studio.

VSIX Installation

Unit tests are great, but you still need to run integration and end-to-end testing to make sure that the code works in the environment where it will really run. As you saw in the “Executing the Diagnostic and Code Fix” section, you can use the VSIX project to automatically install the analyzers and code fixes in a separate instance of Visual Studio. You can also run the code under the debugger so you can set breakpoints and see what your code is doing when Visual Studio invokes it.

Keep in mind that your code may stop at certain points if you use the CancellationTokento exit a method if cancellation was requested. Also, Visual Studio may end up calling your code multiple times from different threads, so the debugger may jump around a bit as you step through code. If you use the VSIX project for debugging, try to keep your sample code that you use for testing small, or comment out most of your sample code if you’re narrowing your focus to one specific case. It’ll make the debugging experience easier.

One other issue I’ve seen with the VSIX project is that, sometimes, your extension won’t be updated, even if you update your code. There’s no hard and fast rule as to when or why this happens, but if you notice that your analyzer or code fix code isn’t firing when you thought it should, you may want to uninstall your extension. To do this, go to Tools ➤ Extensions and Updates. You should see a screen like the one in Figure 2-9.

Figure 2-9. Finding your extension in Visual Studio

Select your extension and click Uninstall. Close that instance of Visual Studio, and run your VSIX project again. That usually takes care of most update issues that I’ve seen when I use the extension.

Visual Studio Logging

Even with thorough testing, it’s still possible that a developer can do something in their code that you didn’t expect (for example, using a new feature in a new version of C#), and cause your code fix to crash because the expected syntax tree format is no longer valid. If it does, you’ll get the “yellow bar of death” in Visual Studio that looks like what you see in Figure 2-10.

Figure 2-10. Getting an error notification when a code fix fails

The error that you see in Figure 2-10 was generated by a code fix in the ThrowsException sample code for this book. Its sole purpose is to report a diagnostic error whenever it runs into a method definition that returns an int:

[DiagnosticAnalyzer(LanguageNames.CSharp)]
public sealed class ThrowExceptionAnalyzer
  : DiagnosticAnalyzer                                                
{
  private static DiagnosticDescriptor rule = new DiagnosticDescriptor(
    "THROW0001", "Returning Ints From Methods",
    "Returning ints is a really bad idea.",
    "Usage", DiagnosticSeverity.Error, true);

  public override ImmutableArray<DiagnosticDescriptor> SupportedDiagnostics
  {
    get
    {
      return ImmutableArray.Create(
        ThrowExceptionAnalyzer.rule);
    }
  }

  public override void Initialize(AnalysisContext context)
  {
    context.RegisterSyntaxNodeAction<SyntaxKind>(
      this.AnalyzeMethodDeclaration, SyntaxKind.MethodDeclaration);
  }

  private void AnalyzeMethodDeclaration(SyntaxNodeAnalysisContext context)
  {
    var method = context.Node as MethodDeclarationSyntax;
    var model = context.SemanticModel;
    var methodSymbol = model.GetDeclaredSymbol(method) as IMethodSymbol;
    var returnType = methodSymbol.ReturnType;
    var intType = typeof(int).GetTypeInfo();

    if (returnType.Name == intType.Name &&
      returnType.ContainingAssembly.Name == intType.Assembly.GetName().Name)
    {
      context.ReportDiagnostic(Diagnostic.Create(
        ThrowExceptionAnalyzer.rule,
        method.Identifier.GetLocation()));                                                                              
    }
  }
}

The code fix, in turn, will immediately throw an exception:

[ExportCodeFixProvider(LanguageNames.CSharp)]
[Shared]
public sealed class ThrowsExceptionCodeFix
  : CodeFixProvider
{
  public override ImmutableArray<string> FixableDiagnosticIds
  {
    get
    {
      return ImmutableArray.Create("THROW0001");
    }
  }

  public override FixAllProvider GetFixAllProvider()
  {
    return WellKnownFixAllProviders.BatchFixer;
  }

  public override Task RegisterCodeFixesAsync(CodeFixContext context)
  {
    throw new NotSupportedException("I can't fix this!");
  }
}

As Figure 2-10 shows, Visual Studio immediately disables the analyzer—the developer has to choose to turn it back on via the Enable button. Note that Visual Studio does not exhibit the same behavior if a diagnostic throws an exception during its analysis—this only occurs for the code fix side of the equation.

This is nice in the sense that Visual Studio won’t completely crash if a code fix keeps throwing exceptions due to bugs in the code base. However, the problem for the developer of the fix is, how do you get any diagnostic information about the exception? There’s a command line switch that you can pass into Visual Studio when it launches called /log, which will write logging information to an XML file. Figure 2-11 shows what it looks like when you pass this switch into the command line.

Figure 2-11. Launching Visual Studio with logging enabled

Notice that if you use the Developer Command Prompt that’s installed with Visual Studio 2015, you don’t have to enter the path where Visual Studio’s executable is located.

When you activate logging, the file is located at %APPDATA%MicrosoftVisualStudioVersionActivityLog.xml. On my machine, I found it at C:UsersjasonbAppDataRoamingMicrosoftVisualStudio14.0. Here’s a small sample of what you’ll find in the file:

<?xml version="1.0" encoding="utf-16"?>
<?xml-stylesheet type="text/xsl" href="ActivityLog.xsl"?>
<activity>
  <entry>
    <record>1</record>
    <time>2016/01/17 15:39:29.523</time>
    <type>Information</type>
    <source>VisualStudio</source>
    <description>Microsoft Visual Studio 2015 version: 14.0.24720.0</description>
  </entry>
  <entry>
    <record>2</record>
    <time>2016/01/17 15:39:29.523</time>
    <type>Information</type>
    <source>VisualStudio</source>
    <description>Creating PkgDefCacheNonVolatile</description>
  </entry>

If a developer is experiencing problems with your code fix, they can invoke logging and hopefully obtain some exception information for you. Here’s what I see when ThrowsExceptionCodeFix fails :

<entry>
  <record>544</record>
  <time>2016/01/17 15:45:09.078</time>
  <type>Error</type>
  <source>ThrowsExceptionCodeFix</source>
  <description>I can&apos;t fix this! at MustInvokeBaseMethod.Analyzers.ThrowsExceptionCodeFix.RegisterCodeFixesAsync(CodeFixContext context) in M:JasonBockPersonalCode ProjectsCompilerAPIBookChapter 2ThrowsExceptionThrowsExceptionThrowsExceptionCodeFix.cs:line 30 at Microsoft.CodeAnalysis.CodeFixes.CodeFixService.&lt;&gt;c__DisplayClass19_1.&lt;ContainsAnyFix&gt;b__1() at Microsoft.CodeAnalysis.Extensions.IExtensionManagerExtensions.&lt;PerformActionAsync&gt;d__2.MoveNext()</description>
</entry>                                                                      

You can see in the call stack that the exception is at line 30 of the ThrowsExceptionCodeFix.cs file. It would be helpful if Visual Studio also logged the file and location in the developer’s code where the code fix was executed as that would help figure out why the code fix is failing. But at least this location gives you a starting point to track down the problem.

One last point about debugging. If you want to run Visual Studio with logging enabled while you’re debugging your code fix, you need to change one property of the VSIX project. Go to the project’s properties, then go to the Debug tab. In the Start Options section, change the Command Line Arguments to what you see in Figure 2-12.

Figure 2-12. Changing VSIX command line options to enable debugging

By the way, the /rootsuffix option launches Visual Studio in an “experimental” mode. That means that your VSIX isn’t installed into your normal space. You can verify this by looking at installed extensions in Visual Studio when you launch it normally in Windows—you won’t see the VSIX package in that list. But as you saw in Figure 2-9, you will see it in the experimental instance. This is beneficial because it keeps potentially buggy code isolated from other Visual Studio modes. However, it does change where the log file is stored. In my case, it looks like this: C:UsersjasonbAppDataRoamingMicrosoftVisualStudio14.0Roslyn. Note the change in the version part of the path. It includes “Roslyn” in the path because that name was included in the command line argument for /rootsuffix. You can choose to make your own experimental modes if you’d like, just make sure you keep track of what got installed in it.

The last part of the diagnostic puzzle is deployment and installation, so you know how developers can use your diagnostics and code fixes. That’s what we’ll explore in the final section of this chapter.

VSIX Packaging

You’ve already seen how the VSIX project helps you debug your analyzers and code fixes by launching a separate instance of Visual Studio. You can also take the generated .vsix file in the debug folder and publish it in numerous ways, such as e-mail attachments, file servers, or even the Visual Studio Gallery. If you have the .vsix file, all that you need to do is double-click on the file, and it’ll automatically kick in an installation process like the one in Figure 2-13.

Figure 2-13. Installing an extension from the file

Keep in mind when you’re using a diagnostic that’s installed with a VSIX that any errors reported by a diagnostic will not be included in the build such that the build fails. Figure 2-14 shows you the Error window when the MustInvokeBaseMethodAnalzyer reports an issue.

Figure 2-14. Successful build even with diagnostic errors

Even though there’s an error in the list, the build says it’s successful. Personally, I don’t like this behavior. If a diagnostic is reporting a problem, but the build is successful, I probably won’t check the Error list.

Also, when you install a diagnostic via a VSIX, that diagnostic will run for every project that you load in Visual Studio. For some diagnostics, this may be too coarse-grained, especially if it’s tied to a specific framework that you only use in specific projects. However, if the diagnostic is one that enforces an idiom with the .NET Framework that you want all developers on your team to adhere to, this may be the right approach.

The other installation option is a NuGet package.

NuGet Packaging

NuGet has become the standard way of sharing and installing assemblies in .NET. You can find a myriad of packages that cover all sorts of aspects in software development: logging, business rules, dependency injection containers, and so on. With diagnostics, you can also publish your analyzers as a NuGet package. The analyzer project template will create the necessary PowerShell scripts and the .nuspec file to create the package when you build the project. You’ll find a .nupkg file in the appropriate subfolder of the in folder. Publish that file to either Nuget.org or a local NuGet repository, and you’re done.

To contrast this NuGet package installation option with the VSIX approach, errors reported from a diagnostic will cause a build to fail. Figure 2-15 shows how this works when you reference the MustInvokeBaseMethod.Analyzer package in a project.

Figure 2-15. Analyzers failing from a NuGet installation will fail the build

With a NuGet installation, however, remember that it’s per-project. You can install a NuGet package across an entire solution, but if you add projects to the solution after that, you’ll still need to remember to install the NuGet package into these new projects.