Elasticsearch is such a widely useful technology that it's worth looking at in a little detail. It's a Java application, but running in Docker, you can treat it as a black box and manage it in the same way as all other Docker workloads—you don't need to install Java or configure the JDK. Elasticsearch exposes a REST API for writing, reading, and searching data, and there are client wrappers for the API available in all major languages.

Data in Elasticsearch is stored as JSON documents, and every document can be fully indexed so that you can search for any value in any field. It's a clustered technology that can run across many nodes for scale and resilience. In Docker, you can run each node in a separate container and distribute them across your server estate to gain scale and resilience, but with the ease of deployment and management you get with Docker.

The same storage considerations apply to Elasticsearch as they do to any stateful workload—in development, you can save data inside the container so that when the container is replaced, you start with a fresh database. In test environments, you can use a Docker volume mounted to a drive folder on the host to keep persistent storage outside the container. In production, you can use a volume with a driver for an on-premises storage array or a cloud-storage service.

There's an official Elasticsearch image on Docker Hub, but it currently has only Linux variants. I have my own image on Docker Hub which packages Elasticsearch into a Windows Server 2019 Docker image. Running Elasticsearch in Docker is the same as starting any container. This command exposes port 9200, which is the default port for the REST API:

 docker container run -d -p 9200 `
 --name elasticsearch `
 --env ES_JAVA_OPTS='-Xms512m -Xmx512m' `
 sixeyed/elasticsearch:5.6.11-windowsservercore-ltsc2019

Elasticsearch is a memory-hungry application, and by default it allocates 2 GB of system memory when it starts. In a development environment, I don't need that much memory for the database. I can configure this by setting the ES_JAVA_OPTS environment variable. In this command, I limit Elasticsearch to 512 MB of memory.

There is a NuGet package for Elasticsearch called NEST, which is an API client for reading and writing data, and is targeted for the .NET Framework and .NET Core. I use this package in a new .NET Core console project, NerdDinner.MessageHandlers.IndexDinner. The new console app listens for the dinner-created event message from NATS and writes the dinner details as a document in Elasticsearch.

The code to connect to the message queue and subscribe to messages is the same as the existing message handler. I have a new Dinner class, which represents the Elasticsearch document, so that the message handler code maps from the Dinner entity to the dinner document and saves it in Elasticsearch:

var eventMessage = MessageHelper.FromData<DinnerCreatedEvent>(e.Message.Data);
var dinner = Mapper.Map<documents.Dinner>(eventMessage.Dinner);
var node = new Uri(Config.Current["Elasticsearch:Url"]);
var client = new ElasticClient(node);
client.Index(dinner, idx => idx.Index("dinners"));

Elasticsearch will run in a container, and the new document message handler will run in a container, all in the same Docker network as the rest of the NerdDinner solution. I can start the new containers while the existing solution is running, as there are no changes to the web application or the SQL Server message handler. Adding this new feature with Docker is a zero-downtime deployment.

The Elasticsearch message handler has no dependency on EF or any of the legacy code, just like the new REST API. I've taken advantage of this to write those apps in .NET Core, which gives me the freedom to run them in a Docker container on Linux or Windows hosts. My Visual Studio solution now has .NET Framework, .NET Standard, and .NET Core projects. Parts of the codebase are shared between the .NET Framework and .NET Core application projects. I can use multistage builds for each application Dockerfile, but that could cause problems in larger projects.

Large .NET codebases tend to have a multi-solution approach, with a master solution containing all the projects used in the CI server, and different .sln files for each area of the application, which each have a subset of projects. This lets different teams work on their part of the codebase without every developer having to load millions of lines of code into Visual Studio. It saves a lot of developer time, but it does introduce the risk that changes to a shared component can break another team's build.

If you move to multi-stage builds for all your components, you could still have this problem when you move to Docker. In that case, you can use an alternative approach where you build all the code in a single Dockerfile, much like the old master solution for Visual Studio.