Chapter 7. New Components for Your Modernized .NET Applications
Back in Chapter 1, we talked about those home improvement shows on HGTV. Whenever one of the smiling participants hires an interior decorator to fix up their home, a lot of “stuff” leaves the house, and lots of new “stuff” finds its way in. The house isn’t gutted; key pieces of furniture stay, and the structure isn’t dramatically altered. But a fresh design warrants new pieces. The same goes for your modernized .NET applications. When refactoring your software, it’s an opportune time to freshen up your codebase with components that reflect your new priorities. In this chapter, let’s look at what you should consider adding to your modernized apps.
Open Source Data and Messaging Software
Changing your data platforms is scary. I get it. Your databases store years of records and many applications have coalesced around them. Whatever you use for integrating data—be it classic Enterprise Service Bus or commercial Extract, Transform, and Load (ETL) tools, or both—likely forms an integral part of your enterprise architecture. You better have a good reason to swap out these technologies.
In many cases, you do have such a reason. For our modern .NET apps, we’re prioritizing flexiblity, portability, and performance, not to mention compatibility with the modern patterns we’re implementing. Let’s begin with database engines. Classic commercial database platforms are still a good bet. They’re powerful and feature-rich. They’re also expensive. As you consider patterns where you have one database per microservice, that cost adds up. Also, you want database engines that you can provision and manage via APIs, thus making developer self-service and platform-managed software a reality.
Consider a few alternatives. For relational workloads, you have a variety of open source options. MySQL is one. Also, take a long look at PostgreSQL. It’s a multiplatform, ACID-compliant engine that’s proven at scale and quite extensible. PostgreSQL has all the familiar relational database functionality you’d expect, including synchronous and asynchronous replication, indexes, schemas, stored procedures, and triggers. Besides drivers for .NET apps, PostsgreSQL offers connectors for all other major languages.
As part of modernizing your .NET apps, you might be considering some schemaless database engines. MongoDB is a document-oriented database that you should consider. It has field indexing, a rich query syntax, built-in data aggregation functionality, multidocument ACID transactions, and high availablity through replica sets. There’s solid .NET support via an official driver.
Redis is another one to look at. It’s a remarkably popular in-memory key/value store that’s ideal for your caching needs. It has a scalable replication strategy you can employ to scale reads or achieve data redundancy. Like the other open source databases, Redis has a rich set of supported and community-contributed language bindings. Yes, including .NET.
Now, about your messaging systems. Your classic Enterprise Service Bus (ESB) probably doesn’t have an expansive API for creating or managing instances. Nor does it lend itself to “citizen integrators” who can quickly deploy their own integrations. Odds are, you have a team of specialists who keep these platforms healthy. Remember one of our key objectives when modernizing our .NET apps: speed. This means eliminating (or automating) anything that slows down the delivery of valuable features to production. If you have to wait days, weeks, or months to get integrations prioritized and shipped, you’ll never achieve continuous delivery.
One way to wean yourself off the monolithic integration products is by starting small. Maybe you deploy a lightweight messaging broker for the microservices that power your software. You can introduce something like NATS for lightning-fast, fire-and-forget routing between services. Or, use the battle-tested RabbitMQ for reliable delivery of business data between your services or systems. And consider event-processing engines like Apache Flink or Apache Kafka when you want a durable, append-only log for your event stream. Each of these technologies work well with .NET applications and offers straightforward interfaces that every developer can use.
Cloud-Based Data and Messaging Services
I’d be remiss if I didn’t address the fact that many application modernization efforts were sparked by the rapid embrace of the public cloud. You just signed up to use a public cloud and want to avoid just lifting and shifting software over to it. If you want all the goodness of on-demand, scalable infrastructure, you need to make some changes! Fortunately, each public cloud offers some pretty special data services for your modernized .NET apps.
It’s easy to find managed relational databases in the public cloud. Amazon Relational Database Service (Amazon RDS0 delivers managed instances of Microsoft SQL Server, MySQL, Oracle databases, and PostgreSQL. Google Cloud SQL supports MySQL and PostgreSQL. Microsoft Azure gives you easy access to Microsoft SQL Server, PostgreSQL, and MySQL. In each case, the cloud provider handles provisioning, configuration, backups, and more. And your .NET code “just works” with any of these because each database service supports its standard interface.
The standardization fades a bit when we look at NoSQL options in the public cloud. This is an area where each provider is innovating on its own, which means your code must be refactored to use it. Amazon DynamoDB is a blazing-fast nonrelational database that offers synchronous replication across regions and automates everything, including scaling. Azure Cosmos DB has intelligent global replication and easy scaling built in. It also offers multiple types of data models: SQL, MongoDB, Cassandra, Gremlin, and key/value. Both Amazon DynamoDB and Azure Cosmos DB have standard support for .NET apps via dedicated drivers.
If you want to do away with any care-and-feeding of messaging systems, the public cloud has you covered. Amazon Web Services (AWS) offers its SQS platform for basic, durable messaging. Google Cloud has Pub/Sub, and Azure sells its Service Bus. In each case, the engines have extremely high performance limits and simple .NET interfaces to work with. For event stream processing, you have options like Amazon Kinesis and Azure Event Hubs. One easy way to get started with these is to make them the default choice when you’re integrating cloud-hosted services.
Modern .NET Packages
How about your code itself? What should you change there when refreshing your .NET software? There are many exceptional NuGet packages to add, but I’ll focus on two that make a big impact: xUnit and Steeltoe—xUnit because unit tests are the foundation of a successful delivery pipeline, and Steeltoe because it accelerates the adoption of cloud-native patterns.
xUnit
To have confidence in your deployments, you must have confidence in your code. To have confidence in your code, you must have confidence in your tests. xUnit is a unit testing framework for .NET Framework and .NET Core. It was created by the gang behind NUnit v2 and is part of the .NET Foundation.
Doing test-driven development can feel like a chore. “Let me just write the code,” you might say. But if we’re trying to constantly ship value to production, we can’t just hand off untested code to a testing team. By thoughtfully designing declarative tests, we improve quality and make it possible to continuously integrate, and even continuously deploy, software.
When writing tests with xUnit, you have two types of unit tests: facts and theories. The xUnit authors say that “facts are tests which are always true” and “theories are tests which are only true for a particular set of data.” So for theories, your tests might pass or fail based on the input data. With xUnit, you can run tests using the .NET Core CLI (“dotnet test”) or even Visual Studio. And you can test your code against multiple target platforms—say, .NET Framework 4.7 and .NET Core 2.1—on each run. There’s no doubt that it’s a real investment to add tests to your code, but it’s one that’s honestly worth the cost.
Steeltoe
Even though microservices are an exciting way to decompose systems, they add complexity to your architecture. Where once there was a static, predictable landscape, you now have a dynamic environment with more moving parts. Assuming that your app warrants a microservices architecture, you’re going to want some help to simplify things. Enter Steeltoe.
Steeltoe is a set of libraries created by Pivotal to bring microservices patterns to your .NET Framework and .NET Core applications. It’s inspired by the vigorous microservices support in the ubiquitous Spring Framework, also maintained by Pivotal. .NET apps powered by Steeltoe can run on Windows or Linux, and on Pivotal Cloud Foundry or any application host.
We take a closer look at Steeltoe in Chapter 9, when we apply it to a few modernization recipes, but let’s first outline its core capabilities.
Configuration services
.NET Core introduced a new configuration provider model, and Steeltoe takes advantage of that. .NET Core supports configuration sources like command-line arguments, JSON files, and environment variables. Steeltoe adds two more that work with .NET Framework and .NET Core apps: Cloud Foundry and Spring Cloud Config Server. The Cloud Foundry provider parses standard Cloud Foundry environment variables and makes them available to your .NET app. Spring Cloud Config Server makes it easy to serve up configurations stored in Git repos, filesystems, or HashiCorp Vault. The Steeltoe Config Server provider fetches those configurations and makes it easy to access them in your .NET code.
Service discovery
So…where are my microservices? As you scale application instances in and out, and have some healthy ones and some not, it’s critically important to have fresh information about where to route a request. Netflix Eureka offers an in-memory database of service locations and responds only with healthy instances. The Steeltoe Discovery client registers your service with the registry and sends occasional heartbeats to the Eureka server to indicate healthiness. The Steeltoe Discovery client also connects your app to the registry, caches the information, and periodically updates its local cache. Your code refers to a service’s friendly name, and relies on the Steeltoe library to exchange that friendly name for a route at runtime.
Circuit breaker
Using the circuit breaker pattern, you prevent hiccups in key services from cascading failure to the rest of the system. You do this by shutting off traffic to the failing service, and providing fallback behavior until that service returns to a healthy state. Steeltoe uses Netflix Hystrix as its implementation. Calls to downstream services are wrapped in a HystrixCommand, which works with a fixed thread pool. If the pool is exhausted or too many downstream failures occur, it trips the circuit and triggers a fallback operation. That operation might return cached or static results until the Hystrix component determines that the offending service is back online.
Management endpoints
Observability is a key demand for your modern .NET apps. When something goes wrong, you need to be able to quickly diagnose it. Steeltoe transparently (and optionally) adds a set of powerful management endpoints to your .NET Framework or .NET Core application:
/healthReturns UP or DOWN information based on built-in health contributors, or any custom ones you write.
/infoReturns Git information as well as any app configuration values under the “info” key.
/loggersLets you view and change the log level for your .NET applications.
/traceReturns the last handful of requests to your app, with metadata about the requestor.
/refreshTriggers a reload of configuration values from configuration sources.
/envReturns configuration values and keys that your app is using.
/mappingsReturns all the routes exposed by the application.
/metricsReturns a wide range of CLR, HTTP client, and HTTP server metrics for your app.
/dumpThis is for Windows-only environments, and returns information about all the threads used by your application.
/heapdumpThis is also Windows-only, and generates a mini-dump of your application for later analysis.
/cloudfoundryenables integration with the Pivotal Applications Manager UI in Pivotal Cloud Foundry.
Service connectors
One of the 12-factor app criteria refers to bound services. To help you discover and use bound services in a Cloud Foundry environment, Steeltoe offers a handful of connectors. These connectors parse the list of bound services for your .NET app, and provide those connection details to your code. Steeltoe offers connectors for MySQL, PostgreSQL, Microsoft SQL Server, RabbitMQ, Redis, and OAuth.
Security
Steeltoe makes it simple to use Cloud Foundry’s OAuth2 security services in your apps. The OAuth2 SSO provider lets you use the credentials in a User Access and Authentication (UAA) server or Pivotal single sign-on (SSO) service for authentication and authorization purposes. For accessing RESTful services, the Steeltoe JSON Web Token (JWT) provider lets you secure access to endpoints.
You can use one or all of the previous capabilities in your modernized .NET apps. Each capability is represented as a NuGet package and typically works with ASP.NET Core, ASP.NET (MVC, Web Forms, WebAPI, WCF), and console applications.
Continuous Integration and Continuous Delivery Tools
What’s the most effective way to develop a sustainable path to production? Put your .NET apps on pipelines. This means automating the key steps of integrating, packaging, and deploying software to its target destination. If you do nothing else I’ve recommended in this chapter, at least do this.
There’s no shortage of products in this space. There are continuous integration tools like Jenkins, CircleCI, TeamCity, AWS CodePipeline, and Visual Studio Team Services (VSTS) CI. I’m personally partial to Concourse, which offers declarative pipelines, stateless execution environments (so no messy cleanup!), and an intuitive dashboard. When doing continuous delivery, review products like GoCD and Spinnaker, but also consider the CI tools just listed because they’re capable of also deploying integrated packages.
Summary
When tasked with modernizing your apps, don’t miss this amazing opportunity to actually improve your software. Consider introducing new data services, code libraries, and deployment tools that stand to deliver more resilient, scalable, change-friendly apps that are sustainable for the next decade. Chapter 8takes a look at where to run all this modernized .NET software.