As laid out in Chapters 4 and 5, applications in a service-oriented architecture can profit greatly from using common protocols and data formats (such as JSON over HTTP), and from choosing a single IDL for interapplication communication.
In this chapter, we lay out why applying uniformity of programming languages, data stores, or other technology components across all the services in your microservices application might not be the best choice.
After looking at the various components in which the services in your overall system can change, this chapter discusses the benefits and pitfalls of using differing technologies throughout your service implementations. Next we provide some guidelines for deciding when to become more polyglot in your architecture. Finally, we’ll offer some practical tips for the process of introducing technology options to your microservices application.
Dimensions of Change
The majority of services in a microservices environment expose many areas that together make up a multidimensional vector for potential improvements.
Many service applications can be rewritten using different programming languages. Even if they choose to keep the language the same across services (or different versions of the same service), the service owners can use a different set of libraries to rely on. Some companies we know, while keeping the core programming language the same, vary application frameworks between different deployment units.
Another popular change is to examine the use of a different type of data store, such as by switching some portion of the service's persistence layer to use a NoSQL solution in place of an RDBMS.
Introducing a caching layer , for example by adding CDN usage to a publicly exposed service, or by adding Memcached or in-memory caching between the service application code and the data persistence layer, is another adjustment that is often introduced after the initial launch of a microservices application, and it can be applied selectively to a subset of the services an organization has deployed.
Advantages of Using Different Languages and Technologies
Considering the fact that software engineering as a discipline is less than seven decades old, the number of options for technological choices any software developer faces in today's industry landscape is simply staggering.
Let's look at some of the resulting advantages of such advancements in technology diversity, and see why a business would want to operate in a polyglot technology environment.
Specialization
When looking at the various layers of systems that make up the entirety of a modern online application, you will find a large number of different languages and technologies in use.
Looking at the mobile devices landscape, Apple's iOS and Google's Android deliver very specialized operating systems, programming languages, and API frameworks, as well as surrounding tools to build native applications. The main goal of this specialization is to enable developers to use the limited resource of the devices optimally.
Specializing in simple use cases of key-based access, key-value stores can be optimized for much more efficient access to persisted data than any SQL-based database system.
The lingua franca of dynamic web-based clients is JavaScript, mostly because all major browsers support it. But still, even within the JavaScript ecosystem, several different frameworks have been able to establish themselves, further specializing for particular use cases. Examples are:
Google-sponsored Angular.js ( https://angularjs.org ), which is particularly popular for building single-page web applications because it offers synchronized two-way data binding, and alleviates much of the hassle of writing code for DOM manipulation.
Facebook's React.js ( https://facebook.github.io/react/ ), which shines when it comes to highly dynamic web applications, where the concept of working on a virtual DOM results in client-side performance gains.
Yehuda Katz' MVC framework Ember.js ( http://emberjs.com ), which aims to combine the two-way data-binding and server-side DOM rendering advantages of the previous two frameworks.
The main theme in all of these specializations is optimizing for the requirements of new use cases. The fact that all these technologies exist and continue to thrive is a testament to the fact that they serve a purpose, and any business that wants to build a successful online application is effectively forced into being polyglot already.
By the same token , and as an extension to this point, applying diversity to the languages and frameworks with which the service applications are being built in the back-end can also help with optimization at this layer.
Experimentation
Progress in the computer software and hardware space is still very fast-paced. And even if skeptics are correct that the applicability of Moore's law about transistor density in integrated circuits is about to cease, software system will still see orders of magnitude improvements. Google's company culture asks their employees to apply "10x thinking" when thinking about innovation. Their rationale is:
[T]rue innovation happens when you try to improve something by 10 times rather than by 10%. 1
Both the development and the use of such quantum leap advances require a great deal of willingness to experiment. Therefore, curiosity remains a key competitive advantage for any software developer, as well as for the businesses that employ software engineers; in our experience, it is usually the most talented people who will be motivated by opportunities to learn new technologies.
The good news is that implementing a microservices architecture provides a business with many benefits for experimentation. Because the units of deployment are small applications with very limited concerns, experimentation can happen with a very low impact on the overall business.
Evolution
Businesses can only achieve longevity when they learn how to adapt to an ever-changing environment, and staying agile by quickly adjusting or reversing course is key. Often the result of such environmental changes is that the business needs shift toward lesser, or sometimes even entirely new, use cases.
As an example, we have been part of companies where the initial system was not designed to cater to sophisticated, end user triggered product searches, often including search criteria that are dependent on geolocation data gathered about the customer. This can often be best addressed by researching and introducing specialized technology, such as Apache Lucene ( https://lucene.apache.org ) and Solr ( https://lucene.apache.org/solr ), or Elasticsearch ( https://www.elastic.co/products/elasticsearch ) for the search use cases.
It is also not uncommon that companies become the victim of their own success, having to serve a much larger number of online visitors than initially envisioned. In such a case, evolving the core services of your infrastructure to a language or framework that makes better use of CPU and memory resources, or that can more easily deal with a higher-level concurrency, can often help in addressing the bottleneck before it turns into a case of subpar user experience or bad press.
Navigating the Pitfalls of Being Polyglot
Going from an environment where all your services are using the same technology stack to embracing a more diverse set of tools surely also has some inherent risks that need addressing.
First, you will need to think about the development cycle. Your software engineers will need some time to familiarize themselves with the new language or any other technology that is being investigated. This is not simply restricted to learning the syntax of an unfamiliar language, but also extends to decisions about libraries, frameworks for automated testing or web request handling, and how to automatically install the development environment on an engineer's work computer.
Usually, these bootstrapping decisions are most efficiently made by a very small group of engineers that are passionate to learn about the technology in question. Their findings can then be spread among the rest of the service-owning team in short “brown bag lunch” tech talk sessions.
Similar issues can be encountered in the area of deploying new technologies, especially when you have to work in an environment where host machines are shared between services, and the new technology could potentially impact deployment strategies of existing services.
The cleanest option to address such concerns is generally to avoid co-deploying more than one system, for example, using application containers.2 Otherwise, we recommend pairing up an application developer with an operations engineer to investigate the cleanest way to deploy the new tools alongside existing technology. That way, initial deployment guidelines for the new language stack, or the like, benefit from input of a larger set of engineering functions in the business.
Once deployment issues are addressed, it is paramount not to forget the requirements for operational support for running new machinery in a production environment. As mentioned for the deployment aspects, having dedicated servers or containers per service will make this task easier, as the newcomer will not be competing for resources with existing services.
Another thing that could help during the deployment and operational support research steps is choosing a technology that is significantly similar to other standards you have already chosen. If, for example, your company already supports JVM applications via running JRuby, and you are trying to introduce a new language that is also JVM-based (such as Clojure), then you might find that you will have to address fewer open issues when introducing this new technology.
It is very important to involve all parties that have a stake in supporting the new technology in production in the process of deciding how to initially launch the new tech component in production. If your organization is following the “you built it, you run it” setup where application developer teams entirely support the software they own in all production aspects, then fewer departments need to be part of this decision process. But if you are in the fortunate position to work with a technical operations support team, make sure you involve them in these discussions.
Considering the tail end of a technology's usage life cycle, you should also think about what it will take to retire any system you introduce. For one, this applies on the one hand to technologies you experiment with and find that they do not add enough value to justify the effort of maintaining them. But more generally this also applies to the technology that is superseded by its successfully tested replacement. Make sure to have follow-through on such technology retirements and replacements, or you will run the risk of having too many technologies to support.3
A Process for Adopting New Technologies
When adding new technologies to their stack, companies are well advised to come up with a written policy concerning the selection process. The advantage of this lies in the fact that decision making can be based on a published, step-by-step process with clear milestones and measurable criteria for moving forward.
This section lists a few questions to answer during the selection, implementation, and production launch steps.
Document All Findings
We have seen all too often that decisions about adopting or rejecting a certain language or technology tool are made without documenting the results on which the rationale for those decisions is based. The drawbacks of such lack of documentation are at least twofold.
For one, there is a potential for wasted time and effort of a second internal team, which might have started its own investigation process for the same technology, although they could have taken advantage of the results of the previous investigations that were unknown to them.
Second, the business or technological environment in which the company operates might change over time, and those circumstances might invalidate, or at least significantly affect, the findings of a prior investigation of a given technology. If there is no detailed documentation about past technology trials, then the company might miss out on the competitive advantage a previously rejected tool might bring them under the new conditions.
Statement of Motivation and Goals
When documenting your technology trials, we recommend starting out with a section that describes the motivation for the new language or toolset.
Often such sections will list areas where the current tech stack is known to be lacking, or particular aspects that need to be considered with regard to a new technology (for example, maturity, safety, community and tool support, or performance).
At other times the goal of the experiment might not necessarily be directed at a well-established and recognized problem. The motivation might simply be that the general tech community is praising a new tool or language that can solve problems for which there was no prior awareness inside your company, or where improvements can help the company to be more successful.
A reason for investigating a new language might be that the same language could be used in several layers of your stack, thereby reducing the general skill set required from your developers. We have seen this argument used when companies decide to introduce Node.js ( https://nodejs.org/en/ ) to write their back-end services in JavaScript when their browser-based logic is already coded in this language. Similarly, we often hear the argument for adopting ClojureScript ( http://clojurescript.org/ ) in the front-end from companies that are invested in Clojure as their service-side language.
Another very prominent reason for testing a new technology is the attempt to speed up the performance of business-critical areas. We know of companies that have discovered that full-text search in their product database became an increasingly important feature for their growing customer base, and investigations needed to be made to replace the initially MySQL-backed search functionality because of performance issues. Similar arguments sometimes lead companies to experiment with migrating a service implementation from one programming language to another one (for example, moving from Ruby to Rust, Go, Elixir, or the like).
In attempts to avoid being a victim of their own success, many companies realize later on in their existence that the technology they chose does not scale well under load, so they will need to research switching to technologies that behave more resource-efficiently under high load.
Do you see the costs of maintaining a particular technology you use skyrocket? Then it might be time to research more stable or more easily extensible alternatives.
If your company has a hard time attracting talented engineers or keeping your most valuable tech personnel engaged and excited, then experimenting with new and exciting technologies to counteract attrition or to increase the recruiting success is surely also a reasonable motivation for experimentation.
Whatever turns out to be the reason behind your research, make sure to explicitly state the areas you will cover in your experimentation, and list the expected outcomes and metrics associated with judging the success of the experiment.
Implementation Planning
Once the groundwork is laid with the description of the goals of the research, the first milestone to pass is to get an official go-ahead from the company's management that the experiment can proceed, and that the metrics and acceptance criteria are sound. We highly recommend getting such buy-in, and officially setting aside time for team members to work on such "new tech" projects. In our experience, these projects will not happen unless they appear on a formal roadmap for company work.
Now it is time for planning the execution. It is always a good idea to publicize the plan to test a new technology internally to the broadest possible group of people.
This can not only help solicit feedback on the strategy and goals, but also aid in finding support and like-minded co-workers who are interested in pairing on the initial implementation.
Make sure to measure and record all data that you laid out in your statement of goals as you go about implementing the new technology in your organization. This documentation will help you and a potential gatekeeper decide if the experiment results meet the acceptance criteria to take things to the next step. And even if the outcome of that analysis is such that a go-ahead cannot be recommended, it is very useful to document just this fact and preserve this knowledge within your organization.
In an ideal scenario, most of this implementation can be performed in a sandbox of staging environment, although we realize that true results—especially where performance is concerned—will only be achieved once the new technology is exposed to production traffic.
Go-Live Criteria
The most important point before adding any new technology to production is to clearly understand the potential impact and failures modes. Hopefully you have had a chance to discover some of these scenarios during a pre-production test in a staging environment.
If you are replacing existing functionality like for like, such that the new technology has to support all the current use cases, then a great help in verifying that the new service behaves as expected will be a suite of black-box tests4 that you run against both the old and the new system.
To help mitigate the risk involved in such projects you should seriously consider adopting deployment strategies that allow for a fast and easy rollback mechanism (such as blue-green deployments or canary releases, which were explained in the previous chapter).
Aspects that you can never neglect when going to production are logging, monitoring, alarming, and data collection. If you cannot analyze a technology's status messaging, then you will never know if it is in failure mode or what particular problems it might encounter in case of an outage. And if your business prides itself on being data driven, then integrating the newcomer into the data warehouse layer must be a first priority.
It is generally a good guideline to keep the number of languages and other technology options in production as small as possible, but as large as necessary. This reduces maintenance costs and cognitive load for the tech personnel while at the same time providing an environment for staying ahead of the curve and agile.
It is never a good idea to use a technology that only one or two technical people in the company are sufficiently familiar with to support it in a production setting. This means that you should neither adopt a technology where only a small number of developers understand the system, nor migrate away from a technology after there is too little know-how left to grasp what it takes to deprecate it.
One way to avoid this is to focus on hiring polyglot and well-rounded engineers who make a habit of learning the company's infrastructure. Make sure that a significant amount of time during the on-boarding process for new team members is spent learning the ins and outs of all technologies that are in use by your team.
A Case Study: Moving books-service from Rails to Rust
When looking at our example book store business that is running in a microservices environment, we discovered that our most trafficked service is the books-service, which contains all the core product data about the books for sale on the site.
The service has been implemented using the rapid application development framework Ruby on Rails , mostly because the developers were most familiar with this technology, and because it is very convenient and easy to implement a JSON API service in it.
During day-to-day operations the team owning books-service notices that response times start to become affected during peak business times when the largest number of customers browse the product catalog. While there is no immediate urgency to address the issue—that is, no decrease in purchase rates can be correlated to these times of heavy load—the development team is starting to investigate the root cause of the scaling issue.
Analysis of the logs and New Relic APM5 data for books-service shows that much time is spent in garbage collection of Ruby objects, and—even though the service was implemented as a Rails API-only6 project—lots of framework code is logged as being busy handling the request and response objects outside the actual application logic.
Alice, one of the senior engineers on the books-service team, suggests starting to investigate alternatives, and the team agrees to take over part of her current responsibilities so that she has time to dedicate to this effort.
Thus tasked, she starts out by writing down a problem statement, including all data gathered about the nature of books-service's perceived scaling issues.
After doing some research about more resource-efficient languages and less impactful web frameworks, she comes across the Mozilla Research-sponsored language Rust ( https://www.rust-lang.org/ ), which seems very promising for addressing the perceived performance challenges. Additionally, the language seems to raise a lot of interest in the community, and many of the great engineers in her professional network have started to gravitate toward Rust.
Alice formulates a goal statement of the experimentation to include concrete and measurable gains in performance and reduction in computing resource usage, compared to the current production-deployed version of books-service. She also includes specifics about the expected advantages of using the Rust language for the experiment, quoting some very rudimentary benchmarks she has gathered from credible sources, as well as her own small experiments. As the company is hiring and therefore in competition with many other employers for experienced, high-quality engineers, she also lists the increase in attractiveness to potential engineering candidates caused by using a cutting-edge technology like Rust. Her metric to test this hypothesis is to compare feedback from the engineering team's combined professional network, so she plans to ask all teams to survey their contacts about an increase in interest in a company that uses Rust in its technology stack.
The books-service team meets to discuss Alice's proposed experiment based on the statement of motivation and goals, and unanimous agreement is given to go ahead with the next phase of the experiment. This means that time is officially dedicated in the next two team sprints to work on the first stage of the implementation plan. Alice will pair with Peter, another engineer on her team, on this project, simply to make sure that knowledge is not restricted to a single person, and they agree to schedule regular “brownbag lunch” sessions where they share updates with the rest of the team.
Alice and Peter are also asked to work to identify a contact on the company's technical operations (TechOps) team, in order to share knowledge about potential deployment or operational concerns, as well as solicit suggestions they might have for running Rust in the existing infrastructure.
After learning the basics of the Rust language and identifying potential API frameworks in the language, Alice and Peter decide to use Iron ( http://ironframework.io/ )—a very lightweight and extensible web framework for Rust. They experimented with several other Rust frameworks, but found that they introduced more latency than Iron, and as performance is one of the main objectives of the experiment, they decide to stick with Iron.
While reimplementing all the features of the existing Ruby version of books-service in Rust, they can rely on an existing, out-of-process black-box test suite, written in Ruby using the Cucumber DSL. Once they get to the point of feature equivalence between the Ruby and Rust versions, they can finally measure response times and resource usage for the Rust service under a simulated large load in the company's staging network environment.
Initial results are very promising, and Alice and Peter write up a page on their internal company Wiki which summarizes the results. They present the results first to their own team, which is impressed with the observed 10x response time improvements in the staging environment.
An official vote is taken and recorded to go ahead and spend time investigating and addressing all the necessary steps that are involved with a “go-live” decision in production. These steps include more training on the technologies in use for developers and TechOps engineers, as well as making sure that the team has visibility into the new service's production behavior by integrating it with the logging and alarming infrastructure. Data warehouse integration of the metrics gathered for business intelligence queries also needs to be addressed before the Rust service can go live.
Once integrating Rust into the company's canary deployment process is addressed by the TechOps department, the team can finally observe the new service under actual production load, and compare it side by side with the existing Ruby version of the service.
Summary
This chapter discussed the implications of being technologically polyglot. After explaining the vectors of change in a microservices architecture where technological diversity can and should be tried, we looked at the opportunities in adopting several competing technologies.
We identified the main potential advantages to be the benefits of specialization, progress achieved by experimentation, and remaining agile by setting yourself up to evolve more quickly with your business requirements.
Next we looked at how to navigate around the potential pitfalls of working in a polyglot technology environment, before shifting our focus to describing a viable process for experimentation with, and adoption of, a new technology in a company's tech stack.
We concluded the chapter with a fictitious case study about replacing a Ruby-on-Rails based microservice in our example bookstore with an equivalent service and APIs implemented in Rust.
The next chapter will go into much more detail about the implementation of the microservices architecture that makes up the sample book store application.
Footnotes
1 “Creating a Culture of Innovation: Eight ideas that work at Google”, Google for work, https://apps.google.com/learn-more/creating_a_culture_of_innovation.html , as of September12, 2016
3 See notes on a process for evaluating a new technology later in this chapter.
5 This is an application-performance monitoring tool. See https://newrelic.com/application-monitoring .
6 A variant of a standard Ruby on Rails application that tailors the middleware components used to the minimum need for API services. See the respective Rails guide at http://edgeguides.rubyonrails.org/api_app.html for more details.