Appendix B. Extra tools and frameworks

This book would not be complete without an overview of some tools and basic techniques you can use while writing code. From database testing to UI testing and web testing, this appendix lists tools you should consider. Some of them are used for integration testing, and some allow unit testing. I’ll also mention some that I think are good for beginners.

The tools and techniques listed below are arranged in the following categories:

Tip

An updated version of the following list can be found on the book’s wiki site: ArtOfUnitTesting.com.

Let’s begin.

B.1. Isolation frameworks

Mock or isolation frameworks are the bread and butter of advanced unittesting scenarios. There are many to choose from, and that’s a great thing:

Here is a short description of each framework.

B.1.1. Moq

Moq is an open source newcomer to the world of mocking and has adopted an API that tries to be both simple to learn and easy to use. The API also follows the arrange-act-assert style (as opposed to the record-and-replay model) and relies heavily on .NET 3.5 features, such as lambdas and extension methods. If you’re planning on working with .NET 3.5 exclusively, this is a relatively pain-free learning curve, but you need to feel comfortable with using lambdas.

In terms of features, it has fewer than most other mock frameworks, which also means it’s simpler to learn.

You can get Moq at http://code.google.com/p/moq/.

B.1.2. Rhino Mocks

Rhino Mocks is a widely used open source framework for mocks and stubs. It’s also the framework used throughout this book for examples, and it’s discussed more in chapter 5.

Rhino Mocks is loaded with features and has recently moved to using the new arrange-act-assert syntax.

You can get Rhino Mocks at http://ayende.com/projects/rhino-mocks.aspx.

B.1.3. Typemock Isolator

Typemock Isolator is a powerful commercial isolation framework that tries to remove the terms “mocks” and “stubs” from its vocabulary in favor of a more simple and terse API.

Isolator differs from the other frameworks by allowing you to isolate components from their dependencies regardless of how the system is designed (although it supports all the features the other frameworks have). This makes it ideal for people who are getting into unit testing and want an incremental approach to learning. Because it doesn’t force you to design for testability, you can learn to write tests correctly and then move on to learning better design, without having to mix the two together.

Note

Full disclosure: While writing this book, I’ve also been working at Typemock.

You can get Typemock Isolator at www.typemock.com.

B.1.4. NMock

NMock is an open source mocking framework that started out as a direct port of jMock. It used to be the de facto mocking framework until Rhino Mocks took its place in the open source world. The main reason it was dethroned is that it did not offer strong typing of method names. (You had to use strings to define expectations on methods.) Lately, it has been getting back into development, and the new 2.0 release marks a nice change in the API. It remains to be seen how well it will do against the current competition.

You can get NMock at www.NMock.org.

B.1.5. NUnit.Mocks

NUnit.Mocks is an ultra-small open source mocking framework that comes built into NUnit as a separate DLL. It initially started life as an aid for testing NUnit internally, and NUnit’s authors still discourage people from using it in their own projects, because it may “disappear” in the future.

NUnit.Mocks is one of the simplest frameworks to learn with, but I don’t recommend using it for anything other than a learning tool.

You can get NUnit.Mocks as part of the installation of NUnit at http://nunit.com.

B.2. Test frameworks

The test frameworks are the bases from which we start writing our tests. Like mock frameworks, there are many to choose from, and this competition has brought lots of innovation with it. Here are some of the available frameworks:

Let’s look at each in turn.

B.2.1. Microsoft’s Unit Testing Framework

Microsoft’s Unit Testing Framework (also known as MSTest) comes bundled with any version of Visual Studio .NET 2008 professional or above. It includes basic features that are similar to NUnit, but it runs a little slower. The upcoming versions of Visual Studio (2010) will add a lot of power to this framework, but you can use it today as easily as NUnit.

One big problem with this framework is that it’s not as easily extensible as the other testing frameworks. To see how cumbersome it is to add a simple attribute, see the discussion of YUnit on MSDN at http://code.msdn.microsoft.com/yunit.

One big plus for this framework is that it’s integrated into the Visual Studio Team System tool suite and provides good reporting, coverage, and build automation out of the box. If your company uses Team System, I highly suggest using MSTest as your test framework because of the good integration possibilities.

You can get MSTest with Visual Studio.

B.2.2. NUnit

NUnit is currently the de facto test framework for unit test developers in .NET. It’s open source and is in almost ubiquitous use among those who do unit testing. I cover NUnit deeply in chapter 2. NUnit is easily extensible and has a large user base and forums. I’d recommend it to anyone starting out with unit testing in .NET. I still use it today.

You can get NUnit at http://nunit.com.

B.2.3. MbUnit

MbUnit is fully open source, and the “mb” stands for “model-based” testing. It started out as a competitor to NUnit but soon zoomed past NUnit in terms of features and abilities.

MbUnit is easily extensible and supports lots of interesting test attributes, such as Repeat and Timeout. MbUnit has its own UI and console runners that also support running tests written in NUnit. If you’re looking for something more in your current test framework, MbUnit is a good step up from NUnit. I almost never have to use such features myself, but if you’re mixing integration testing and unit testing with the same framework, MbUnit is a good fit.

You can get MbUnit at www.mbunit.com.

B.2.4. Gallio

Gallio is an open source platform for running tests written in most (if not all) unit test frameworks in .NET, from NUnit to MSTest. Gallio is also extensible, and you can create your own custom test runner using it. It has plugins for Visual Studio .NET, which can highlight coding errors relating to tests and other things. It’s still not in very popular use, but it’s built by the same people who maintain MbUnit, and it’s a fully working, robust product that seems to have been in an endless beta cycle for the past year or so.

You can get Gallio at www.gallio.org.

B.2.5. xUnit

xUnit is an open source test framework, developed in cooperation with one of the original authors of NUnit, Jim Newkirk. It’s a minimalist and elegant test framework that tries to get back to basics by having fewer features, not more, than the other frameworks, and by supporting different names on its attributes.

What is so radically different about it? It has no setup or teardown methods, for one. You have to use the constructor and a dispose method on the test class. Another big difference is in how easy it is to extend.

Because xUnit reads so differently from the other frameworks, it takes a while to get used to it if you’re coming from a framework like NUnit or MbUnit. If you’ve never used any test framework before, xUnit is easy to grasp and use, and it’s robust enough to be used in a real project.

For more information and download see www.codeplex.com/xunit.

B.2.6. Pex

Pex (short for program exploration) is an intelligent assistant to the programmer. From a parameterized unit test, it automatically produces a traditional unit test suite with high code coverage. In addition, it suggests to the programmer how to fix the bugs.

With Pex, you can create special tests that have parameters in them, and put special attributes on those tests. The Pex engine will generate new tests that you can later run as part of your test suite. It’s great for finding corner cases and edge conditions that aren’t handled properly in your code. You should use Pex in addition to a regular test framework, such as NUnit or MbUnit.

You can get Pex at http://research.microsoft.com/projects/pex/.

B.3. IoC containers

IoC (Inversion Of Control) containers can be used to improve the architectural qualities of an object-oriented system by reducing the mechanical costs of good design techniques (such as using constructor parameters, managing object lifetimes, and so on).

Containers can enable looser coupling between classes and their dependencies, improve the testability of a class structure, and provide generic flexibility mechanisms. Used judiciously, containers can greatly enhance the opportunities for code reuse by minimizing direct coupling between classes and configuration mechanisms (such as by using interfaces).

We’ll look at the following tools:

Let’s look briefly at each of these frameworks.

B.3.1. StructureMap

StructureMap is an open source container framework that has one clear differentiator from the others. Its API is very fluent and tries to mimic natural language and generic constructs as much as possible.

StuctureMap has a relatively small user base, and the current documentation on it is lacking, but it contains some powerful features, such as a built-in automocking container (a container that can create stubs automatically when requested to by the test), powerful lifetime management, XML-free configuration, integration with ASP.NET, and more.

You can get StructureMap at http://structuremap.sourceforge.net.

B.3.2. Microsoft Unity

Unity is a latecomer to the DI container field, but it provides a simple and minimal approach that can be easily learned and used by beginners. Advanced users may find it lacking, but it certainly answers my 80-20 rule: it provides 80 percent of the features you look for most of the time.

Unity is open source by Microsoft, and it has good documentation. I’d recommend it as a starting point for working with containers.

You can get Unity at www.codeplex.com/unity.

B.3.3. Castle Windsor

Castle is a large open source project that covers a lot of areas. Windsor is one of those areas, and it provides a mature and powerful implementation of a DI container.

Castle Windsor contains most of the features you’ll ever want in a container and more, but it has a relatively high learning curve due to all the features.

You can learn about Castle Windsor at www.castleproject.org/container/ and download the Castle project at www.castleproject.org.

B.3.4. Autofac

Autofac is a fresh approach to IoC in .NET that fits well with the C# 3.0 syntax. It takes a rather minimalistic approach in terms of APIs. The API is radically different from the other frameworks, and requires a bit of getting used to. It also requires .NET 3.5 to work, and you’ll need a good knowledge of lambda syntax. Autofac is difficult to explain, so you’ll have to go to the site to see how different it is. I recommend it for people who already have experience with other DI frameworks.

You can get it at http://code.google.com/p/autofac/.

B.3.5. Common Service Locator Library

The Common Service Locator (CSL) was born out of a need to create a common infrastructure in applications for getting instances of things. Using the advice of the leading open source container frameworks, Microsoft created a shared library that can help abstract away the actual container your application might use. The CSL sits in front of the container of your choice, be it Unity, StructureMap, or Castle.

You don’t need to use the CSL, but the concept of abstracting away the choice of container in your application is a useful one. Most application authors were doing this anyway, so using CSL is one step in making it more of a recommended design pattern.

You can get the Common Service Locator library at www.codeplex.com/CommonServiceLocator.

B.3.6. Spring.NET

Spring.NET is an open source container framework. It’s one of the oldest and is a port of the Java Spring Container libraries. It has a lot of abilities, but many consider it to be a sort of dinosaur among the other frameworks, with an API that isn’t as easy to use, and configuration that isn’t as friendly as it could be.

You can get Spring.NET at www.springframework.net.

B.3.7. Microsoft Managed Extensibility Framework

The Managed Extensibility Framework (MEF) isn’t actually a container, but it does fall in the same general category of providing services that instantiate classes in your code. It’s designed to be much more than a container; it’s a full plugin model for small and large applications. MEF includes a lightweight IoC container framework so you can easily inject dependencies into various places in your code by using special attributes.

MEF does require a bit of a learning curve, and I wouldn’t recommend using it strictly as an IoC container. If you do use it for extensibility features in your application, it can also be used as a DI container.

You can get MEF at www.codeplex.com/MEF.

B.3.8. Ninject

Ninject is a latecomer to the DI field, but it has a simple syntax and good usability. There isn’t much else to say about it except that I highly recommend taking a look at it.

You can find out more about Ninject at http://ninject.org/.

B.4. Database testing

How to do database testing is a burning question for those who are starting out. Many questions arise, such as, “Should I stub out the database in my tests?” This section provides some guidelines.

First, let’s talk about doing integration tests against the database.

B.4.1. Use integration tests for your data layer

How should you test your data layer? Should you abstract away the database interfaces? Should you use the real database?

I usually write integration-style tests for the data layer (the part of the app structure that talks directly to the database) in my applications because data logic is almost always divided between the application logic and the database itself (triggers, security rules, referential integrity, and so on). Unless you can test the database logic in complete isolation (and I’ve found no really good framework for this purpose), the only way to make sure it works in tests is to couple testing the datalayer logic to the real database.

Testing the data layer and the database together leaves few surprises for later in the project. But testing against the database has its problems, the main one being that you’re testing against state shared by many tests. If you insert a line into the database in one test, the next test can see that line as well.

What we need is a way to roll back the changes we make to the database, and thankfully there’s good support for this in the current test tools and the .NET framework.

B.4.2. Use rollback attributes

The three major frameworks—MbUnit, NUnit, and xUnit—support a special [Rollback] attribute that you can put on top of your test method. When used, the attribute creates a special database transaction that the test code runs in. When the test is finished, the database transaction is rolled back automatically, and the changes to the database vanish.

To learn more about how this works, see an MSDN article I wrote a while back, called “Simplify Data Layer Unit Testing using Enterprise Services” at http://msdn.microsoft.com/en-us/magazine/cc163772.aspx.

If you aren’t interested in using the [Rollback] attributes the frameworks provide, you can use the simple class introduced in .NET 2.0 called TransactionScope.

B.4.3. Use TransactionScope to roll back

For examples on how to use the TransactionScope class in your setup and teardown code, see a blog article called “Even Simpler Database Unit Testing with TransactionScope” at http://www.bbits.co.uk/blog/archive/2006/07/31/12741.aspx.

Some feel that another good option is to run the tests against an in-memory database. My feelings on that are mixed. On the one hand, it’s closer to reality, in that you also test the database logic. On the other hand, if your application uses a different database engine, with different features, there’s a big chance that some things will pass or fail during tests with the in-memory database, and will work differently in production. I choose to work with whatever is as close to the real thing as possible. Usually that means using the same database engine.

B.5. Web testing

“How do I test my web pages?” is another question that comes up a lot. Here are some tools that can help you in this quest:

The following is a short description of each tool.

B.5.1. Ivonna

Ivonna is a unit-testing framework that abstracts away the need to run ASP.NET-related tests using a real HTTP session and pages. It does some powerful things behind the scenes, such as compiling pages that you want to test and letting you test controls inside them without needing a browser session, and it fakes the full HTTP runtime model.

You write the code in your unit tests just like you’re testing other in-memory objects. There’s no need for a web server and such nonsense.

Ivonna is being developed in partnership with Typemock and runs as an add-on to the Typemock Isolator framework. You can get Ivonna at http://sm-art.biz/Ivonna.aspx.

B.5.2. Team System Web Test

Visual Studio Team Test and Team Suite editions include the powerful ability to record and replay web requests for pages and verify various things during these runs. This is strictly integration testing, but it’s really powerful. The latest versions also support recording Ajax actions on the page, and make things much easier to test in terms of usability.

You can find more info on Team System at http://msdn.microsoft.com/en-us/teamsystem/default.aspx.

B.5.3. NUnitAsp

NUnitAsp is an open source framework that’s no longer being supported but is still used in many places. It allows you to write tests against live HTTP page objects.

Most people end up using NUnitAsp for acceptance testing. In those cases, it would be better to use frameworks such as Watir and WatiN, described next.

You can get NUnitAsp at http://nunitasp.sourceforge.net/.

B.5.4. Watir

Watir (pronounced “water”) stands for “Web application testing in Ruby”. It’s open source, and it allows scripting of browser actions using the Ruby programming language. Many Rubyists swear by it, but it does require that you learn a whole new language.

You can get Watir at http://wtr.rubyforge.org/.

B.5.5. WatiN

WatiN (pronounced “what-in”) is a product inspired by Watir. You don’t need to know Ruby to use WatiN, but it offers much the same scripting abilities as Watir.

You can get WatiN at http://watin.sourceforge.net/.

B.5.6. Selenium

Selenium is a suite of tools designed to automate web app testing across many platforms. It has existed longer than all the other frameworks in this list, and it also has an API wrapper for .NET.

Selenium is an integration testing framework, and it’s in wide use. It’s a good place to start. But beware: it has many features and the learning curve is high.

You can get it at http://selenium.openqa.org/.

B.6. UI testing

UI testing is always a difficult task. I’m not a great believer in writing unit tests or integration tests for UIs because the return on such tests is low compared to the amount of time you invest in writing them. UIs change too much to be testable in a consistent manner, as far as I’m concerned. That’s why I usually try to separate all the logic from the UI into a lower layer that I can test separately with standard unit-testing techniques.

Nevertheless, there are several tools that try to make the UI-testing job easier:

Here is a short rundown of each tool.

B.6.1. NUnitForms

NUnitForms is an open source framework that allows you to instantiate Windows Forms classes and check values of controls inside them. It has specific APIs for getting controls in a form and asserting values against them, but it will only work for simple controls.

More complex controls aren’t supported and the framework doesn’t seem to be in active development anymore, so I recommend not using it. Instead, I recommend Project White (discussed next) for WinForm testing.

You can get it at http://nunitforms.sourceforge.net/.

B.6.2. Project White

Project White is, in a way, a successor to NUnitForms, in that it supports a richer set of application types (WPF, WinForm, Win32, and Java JWT), and sports a newer API with better usability. Unlike NUnitForms, White is more of an integration-test framework, because it allows spawning separate processes that it tests against.

White uses the UIAutomation API (part of Windows) to do its bidding, which gives it much more power. You can think of it as Selenium or WatiN for WinForms.

You can get White at http://www.codeplex.com/white.

B.6.3. Team System UI Tests

The upcoming version of Visual Studio Team System will support a new kind of test—a UI test. You’ll be able to record actions on UI windows and play them back and verify assertions during test runs. As with all Team System tools, its main advantage will be its integration with other Team System tools, reports, source control, and servers.

You can learn more about Team System at http://msdn.microsoft.com/en-us/teamsystem/default.aspx.

B.7. Thread-related testing

Threads have always been the bane of unit testing. They’re simply untestable. That’s why new frameworks are emerging that let you test thread-related logic (deadlocks, race conditions, and so on), such as these:

I’ll give a brief rundown of each tool.

B.7.1. Typemock Racer

Typemock Racer is a managed framework for multithreaded code testing that helps visualize, detect, and resolve deadlocks and race conditions in managed code. You use it by putting an attribute on top of an existing test method, and the engine does all the work. It also allows full thread visualization during debugging of a threaded test.

Note

Full disclosure: during the writing of this book, I’ve been part of the developer team at Typemock.

Racer is a commercial product that works with all flavors of Visual Studio (including Express) and all test frameworks. You can get it at www.Typemock.com.

B.7.2. Microsoft CHESS

CHESS is an upcoming tool that will be offered by Microsoft with Visual Studio 2010. (It’s currently offered only as part of MS Research.) Much like Typemock Racer, CHESS attempts to find thread-related problems (deadlocks, hangs, livelocks, and more) in your code by running all relevant permutations of threads on existing code. These tests are written as simple unit tests.

CHESS currently supports native code, but a managed .NET version should be coming out soon as part of Visual Studio Team System (a commercial product).

You can get CHESS at http://research.microsoft.com/projects/CHESS/.

B.7.3. Osherove.ThreadTester

This is a little open source framework I developed a while back. It allows you to run multiple threads during one test to see if anything weird happens to your code (deadlocks, for example). It isn’t feature-complete, but it’s a good attempt at a multithreaded test (rather than a test for multithreaded code).

You can get it from my blog, at http://weblogs.asp.net/rosherove/archive/2007/06/22/multi-threaded-unit-tests-with-osherove-threadtester.aspx.

B.8. Acceptance testing

Acceptance tests enhance collaboration between customers and developers in software development. They enable customers, testers, and programmers to learn what the software should do, and they automatically compare that to what it actually does. They compare customers’ expectations to actual results. It’s a great way to collaborate on complicated problems (and get them right) early in development.

Unfortunately, there are few frameworks for automated acceptance testing, and just one that works these days! I’m hoping this will change soon. Here are the tools we’ll look at:

Let’s take a closer look.

B.8.1. FitNesse

FitNesse is a lightweight, open source framework that makes it easy for software teams to define acceptance tests—web pages containing simple tables of inputs and expected outputs—and to run those tests and see the results.

FitNesse is quite buggy, but it has been in use in many places with varying degrees of success.

You can learn more about FitNesse at www.fitnesse.org.

B.8.2. StoryTeller

StoryTeller is a response to FitNesse, which has existed for a long time but presented many usability and stability problems for users. It’s an open source framework, still in development, that attempts to create a more compelling UI and usability story than FitNesse. There aren’t many details on it yet, but hopefully it will be out soon. It will support running tests written for FitNesse.

The project page is at http://storyteller.tigris.org/, but most of the real details are found on the author’s blog: http://codebetter.com/blogs/jeremy.miller/archive/tags/StoryTeller/default.aspx.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.141.25.41