Chapter 13: Internationalization and Languages

In this chapter, we are going to talk about the internationalization and multilingual features in Drupal 9 from the point of view of a module developer. Many of the built-in capabilities of this system are oriented toward site builders—enabling languages and translating content and configuration entities, as well as the Drupal interface (for administrators and visitors alike). Our focus will be what we as module developers need to do programmatically to ensure that site builders and editors can use the aforementioned features. To that end, this chapter will be more of a reference guide with various tips, techniques, and even rules we need to follow when writing our code. Notwithstanding, we will also talk a bit about how we can work with languages programmatically.

First, however, we will start with an introduction to the multilingual ecosystem that comes out of the box and the modules responsible for various parts of it.

The main topics we will cover in this short chapter are as follows:

• The multilingual ecosystem
• Internationalization
• Translating the interface and content

Introduction to the multilingual ecosystem

The multilingual and internationalization system is based on four Drupal core modules. Let's quickly go through them and see what they do:

• Language
• Content translation
• Configuration translation
• Interface translation

Language

The Language module is responsible for dealing with the available languages on the site. Site builders can choose to install one or more languages from a wide selection. They can even create their own custom language if necessary. The installed languages can then be added to things such as entities and menu links in order to control their visibility, depending on the current language. Apart from the installed ones, Drupal comes with two extra special languages as well: Not Specified and Not Applicable.

The module also handles the contextual language selection based on various criteria, as well as provides a language switcher to change the current language of the site:

Figure 13.1: Language configuration page

Content translation

The Content translation module is responsible for the functionality that allows users to translate content. Content entities are the principal vehicle for content, and with this module, the data inside can be translated (and granularly configured for it at field level). In other words, users can control which fields and which entity type bundles should be translatable:

Figure 13.2: Enabling content translation

Configuration translation

The Configuration translation module is responsible for providing the interface via which users can translate configuration values. These can be from simple configuration objects or configuration entities. We've already seen how we can ensure that our configuration values can be translated in previous chapters, so we won't dive into that again here.

I recommend you reference the section on configuration schemas from Chapter 6, Data Modeling and Storage:

Figure 13.3: Configuration translation page

Interface translation

The Interface translation module is responsible for providing an interface that allows users to translate any string or text output on the website, in all the languages that are installed. Moreover, it provides a connection to the localize.drupal.org platform from which it can download translations for many languages of the more common interface strings that come with Drupal:

Figure 13.4: Interface translation page

These four modules are not alone in the multilingual system but rely on a cross-application standard of ensuring that all the written code works well with it. In other words, the entire Drupal code base is intertwined with the multilingual system at various levels and is written in such a way that anything that should be translatable or localizable can be. This means that all the code we write needs to respect the same standard.

Internationalization

The idea behind internationalization is to ensure that everything that gets output on the site can be translated into the enabled languages through a common mechanism. This refers to content, visible configuration values, and the strings and text that come out of modules and themes. But there are many different ways this can happen, so let's see how in each of these cases we would ensure that our information can be translated.

A principal rule when writing Drupal modules or themes is to always use English as the code language. This is to ensure consistency and keep open the possibility that other developers will work on the same code base, who may not speak a particular language. This is also the case for text used to be displayed in the UI. It should not be the responsibility of the code to output the translated text, but rather to always keep it consistent, that is, in English.

Of course, this is dependent on it being done right, in order to allow it to be translated via interface translation. There are multiple ways this can be ensured, depending on the circumstances.

The most common scenario we need to be aware of is when we have to print out to the user a PHP string of text. This is where the t() function comes into play, through which these strings are run. This function should be used whenever we are not inside a class context:

return t('The quick brown fox');

However, when we are inside a class, we should check whether any of the parents are using the StringTranslationTrait. If not, we should use it in our class and then we'll be able to do this instead:

return $this->t('The quick brown fox');

Even better still, we should inject the TranslationManager service into our class because the aforementioned trait makes use of it.

None of the examples given before should be new to us as we've been using these throughout the code we've been writing in this book. But what actually happens behind the scenes?

The t() and StringTranslationTrait::t() functions both create and return an instance of TranslatableMarkup (essentially delegating to its constructor), which, upon rendering (being cast to a string), will return the formatted and translated string. The responsibility of the actual translation is delegated to the TranslationManager service. This process has two parts. Static analyzers pick up on these text strings and add them to the database in the list of strings that need to be localized. These can then be translated by users via the user interface. Second, at runtime, the strings get formatted and the translated version is shown, depending on the current language context. And because of the first part, we should never do something like this:

return $this->t($my_text);

The reason is that static analyzers can no longer pick up on the strings that need to be translated. Moreover, if the text is coming from user input, it can lead to XSS attacks if not properly sanitized before.

That being said, we can still have dynamic, that is, formatted, text output using this method, and we've seen this in action as well:

$count = 5;

return $this->t('The quick brown fox jumped @count times', ['@count' => $count]);  

In this case, we have a dynamic variable that will be used to replace the @count placeholder from the text. Drupal takes care of sanitizing the variable before outputting the string to the user. Alternatively, we can also use the % prefix to define a placeholder we want Drupal to wrap with <em class="placeholder">. The cool thing is that, when performing translations, users can shift the placeholder in the sentence to accommodate language specificity.

One of the intended consequences of the static analyzer picking out and storing the strings that need to be translated is that, by default, each individual string is only translated once. This is good in many cases but also poses some problems when the same English string has different meanings (which map to different translations in other languages). To counter this issue, we can specify a context to the string that needs to be translated so that we can identify which meaning we actually want to translate. This is where the third parameter of the t() function (and method) we saw in the previous paragraphs comes into play.

For example, let's consider the word Book, which is translated by default in its meaning as a noun. But we may have a submit button on a form that has the value Book, which clearly has a different meaning as a call to action. So, in the latter case, we could do it like this:

t('Book', [], ['context' => 'The verb "to book"']);  

Now in the interface translation, we will have both versions available:

Figure 13.5: Interface translation page with translation context

Another helpful tip is that we can also account for plurals in the string translations. The StringTranslationTrait::formatPlural() method helps with this by creating a PluralTranslatableMarkup object similar to TranslatableMarkup, but with some extra parameters to account for differences when it comes to plurals. This comes in very handy in our previous example with the brown fox jumping a number of times, because if the fox jumps only once, the resulting string would no longer be grammatically correct. So instead, we can do the following:

$count = 5;

return $this->formatPlural($count, 'The quick brown fox jumped 1 time', 'The quick brown fox jumped @count times')];  

The first parameter is the actual count (the differentiator between singular and plural). The second and third parameters are the singular and plural versions, respectively. You'll also notice that since we specified the count already, we don't have to specify it again in the arguments array. It's important to note that the placeholder name inside the string needs to be @count if we want the renderer to understand its purpose.

The string translation techniques we discussed so far also work in other places—not just in PHP code. For example, in JavaScript we would do something like this:

Drupal.t('The quick brown fox jumped @count times', {'@count': 5});

Drupal.formatPlural(5, 'The quick brown fox jumped 1 time', 'The quick brown fox jumped @count times');  

So, based on this knowledge, I encourage you to go back and fix our incorrect use of the string output in JavaScript in the previous chapter.

In Twig, we'd have something like this (for simple translations):

{{ 'Hello World.'|trans }}

{{ 'Hello World.'|t }}  

Both of the above lines do the same thing. To handle plurals (and placeholders), we can use the {% trans %} block:

{% set count = 5 %}

{% trans %}

  The quick brown fox jumped 1 time.

{% plural count %}

  The quick brown fox jumped {{ count }} times.

{% endtrans %}  

Finally, the string context is also possible like so:

{% trans with {'context': 'The verb "to book"'} %}

  Book

{% endtrans %}

In annotations, we have the @Translation() wrapper, as we've seen already a few times when creating plugins or defining entity types.

Finally, in YAML files, some of the strings are translatable by default (so we don't have to do anything):

• Module names and descriptions in .info.yml files
• The _title (together with the optional _title_context) key values under the defaults section of .routing.yml files
• The title (together with the optional title_context) key values in the .links.action.yml, .links.task.yml, and .links.contextual.yml files

Dates are also potentially problematic when it comes to localization, as different locales show dates differently. Luckily, Drupal provides the DateFormatter service, which handles this for us. For example:

Drupal::service('date.formatter')->format(time(), 'medium');

The first parameter of this formatter is the UNIX timestamp of the date we want to format. The second parameter indicates the format to use (either one of the existing formats or custom). Drupal comes with a few predefined date formats, but site builders can define others as well, which can be used here. However, if the format is custom, the third parameter is a PHP date format string suitable for input to date(). The fourth parameter is a timezone identifier we want to format the date in, and the final parameter can be used to specify the language to localize to directly (regardless of the current language of the site).

Content entities and the Translation API

So far in this chapter, we've mostly talked about how to ensure that our modules output only text that can also be translated. The Drupal best practice is to always use these techniques regardless of whether the site is multilingual. You never know whether you'll ever need to add a new language.

In this section, we are going to talk a bit about how we can interact with the language system programmatically and work with entity translations.

A potentially important thing you'll often want to do is check the current language of the site. Depending on the language negotiation in place, this can either be determined by the browser language, a domain, a URL prefix, or others. The LanguageManager is the service we use to figure this out. We can inject it using the language_manager key or use it via the static shorthand:

$manager = Drupal::languageManager();  

To get the current language, we do this:

$language = $manager->getCurrentLanguage();  

Where $language is an instance of the Language class that holds some information about the given language (such as the language code and name). The language code is probably the most important as it is used everywhere to indicate what language a given thing is.

There are other useful methods with this service that you can use. For example, we can get a list of all the installed languages with getLanguages() or the site default language with getDefaultLanguage(). I encourage you to check out the LanguageManager for all the available API methods.

When it comes to content entities, there is an API we can use to interact with the data inside them in different languages. So, for example, we have figured out the current language with the previous method, so we can now get some field values in that language. The way this works is that we ask for a copy of the entity in the respective language:

$translation = $node->getTranslation($language->getId());  

$translation is now almost the same as $node, but with the default language set to the one we requested. From there, we can access field values normally. However, not all nodes have to have a translation, so it's better to first check whether a translation exists:

if ($node->hasTranslation($language->getId())) {

  $translation = $node->getTranslation($language->getId());

}

Since we can configure entity translatability at the field level (allowing only the fields that make sense to be translated), we can also check which of these fields can have translated values:

$fields = $node->getTranslatableFields();

Finally, we can also check which languages there are translations for:

$languages = $node->getTranslationLanguages();  

Since it's up to the editors to add translations to an entity, we cannot guarantee in code that one exists.

Programmatically, we can also create a translation to an entity really easily. For example, let's imagine we want to translate a Node entity and specify its title to be in French:

$node->addTranslation('fr', ['title' => 'The title fr']);

The second parameter is an array of values that needs to map to the entity fields just like when creating a new entity. Now the respective node has the original language (let's say EN) but also a French translation. It should be noted that the values of all the other fields apart from the title, even in the French translation, remain in the original language because we did not pass any translated values when creating the translation.

And just as we add a translation, we can also remove one:

$node->removeTranslation('fr');  

If we want to persist the addition or removal of a translation, we need to save the entity as we are used to. Otherwise, it's stored only in memory. Content entities implement the DrupalCoreTypedDataTranslationStatusInterface, which allows us to inspect the status of the translations. So, for example, we can do this:

$status = $node->getTranslationStatus('fr');

Where $status is the value of one of three constants from the TranslationStatusInterface class:

• TRANSLATION_REMOVED
• TRANSLATION_EXISTING
• TRANSLATION_CREATED

Summary

In this short chapter, we talked about the Drupal multilingual and internationalization system from a module developer perspective. We started with an introduction to the four main modules responsible for languages and translating content, configuration entities, and interface text.

Then, we focused on the rules and techniques we need to respect in order to ensure that our output text can be translated. We saw how we can do this in PHP code, Twig, and YAML files, and even in JavaScript. Finally, we looked a bit at the language manager and Translation API to see how we can work with content entities that have been translated.

The main takeaway from this chapter should be that languages are important in Drupal even if our site is only in one language. So, in developing modules, especially if we want to contribute them back to the community, we need to ensure that our functionality can be translated as needed.

In the next chapter, we are going to talk about data processing using batches and queues, as well as the cron system that comes with Drupal.