Native plugins allow several aspects of the Elasticsearch server to be extended, but they require a good knowledge of Java.
In this recipe we will see how to set up a working environment to develop native plugins.
You need an up-and-running Elasticsearch installation as we described in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.
A Maven tool, or an IDE that supports Java programming, such as Eclipse or IntelliJ IDEA, is required.
The code to this recipe is available in the chapter17/simple_plugin
directory.
Generally, Elasticsearch plugins are developed in Java using the Maven build tool and deployed as a ZIP file.
To create a simple JAR plugin, we will perform the following steps:
pom.xml
is used to define the build configuration for Maven.es-plugin.properties
defines the namespace of the plugin class that must be loaded.<name>plugin.java
is the main plugin class, which is loaded at startup and also initializes the action's plugin.plugin.xml
assemblies, which defines how to execute the assembly steps with Maven. It is used to build the ZIP file to deliver the plugin.pom.xml
file is used for creating a plugin containing the following code:The Maven pom.xml
header:
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <name>elasticsearch-simple-plugin</name> <modelVersion>4.0.0</modelVersion> <groupId>com.packtpub</groupId> <artifactId>simple-plugin</artifactId> <version>${elasticsearch.version}</version> <packaging>jar</packaging> <description>A simple plugin for Elasticsearch</description> <inceptionYear>2013</inceptionYear> <licenses>... </licenses>
pom.xml
file is used to derive common properties or settings:<parent> <groupId>org.sonatype.oss</groupId> <artifactId>oss-parent</artifactId> <version>7</version> </parent>
<properties> <elasticsearch.version>5.1.1</elasticsearch.version> <maven.compiler.target>1.8</maven.compiler.target> <elasticsearch.assembly.descriptor>${project.basedir} /src/main/assemblies/plugin.xml </elasticsearch.assembly.descriptor> <elasticsearch.plugin.name>simple-plugin </elasticsearch.plugin.name> <elasticsearch.plugin.classname> org.elasticsearch.plugin.simple.SimplePlugin </elasticsearch.plugin.classname> <elasticsearch.plugin.jvm>true</elasticsearch.plugin.jvm> <tests.rest.load_packaged>false</tests.rest.load_packaged> <skip.unit.tests>true</skip.unit.tests> </properties>
<dependencies> <dependency> <groupId>org.elasticsearch</groupId> <artifactId>elasticsearch</artifactId> <version>${elasticsearch.version}</version> <scope>compile</scope> </dependency> <dependency> <groupId>org.apache.httpcomponents</groupId> <artifactId>httpclient</artifactId> <version>4.5.2</version> </dependency> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-api</artifactId> <version>2.3</version> </dependency> <!- Testing dependencies --> <dependency> <groupId>org.hamcrest</groupId> <artifactId>hamcrest-core</artifactId> <version>1.3</version> <scope>test</scope> </dependency> <dependency> <groupId>org.hamcrest</groupId> <artifactId>hamcrest-library</artifactId> <version>1.3</version> <scope>test</scope> </dependency> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.12</version> <scope>test</scope> </dependency> </dependencies>
<build> <plugins> <plugin><!- for compiling --> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>3.1</version> <configuration> <source>1.7</source> <target>1.7</target> </configuration> </plugin> <plugin><!- optional for executing tests --> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.12.3</version> <configuration> <includes> <include>**/*Tests.java</include> </includes> </configuration> </plugin> <plugin><!- optional for publishing the source --> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-source-plugin</artifactId> <version>2.3</version> <executions> <execution> <id>attach-sources</id> <goals> <goal>jar</goal> </goals> </execution> </executions> </plugin> <plugin>><!- for packaging the plugin --> <artifactId>maven-assembly-plugin</artifactId> <version>2.3</version> <configuration> <appendAssemblyId>false</appendAssemblyId> <outputDirectory>${project.build.directory} /releases/</outputDirectory> <descriptors> <descriptor>${basedir}/src/main/assemblies /plugin.xml</descriptor> </descriptors> </configuration> <executions> <execution> <phase>package</phase> <goals><goal>single</goal></goals> </execution> </executions> </plugin> </plugins> </build> </project>
src/main/resources/plugin-descriptor.properties
file, which defines the entry point class that must be loaded during plugin initialization. This file must be embedded in the final JAR; it is usually put in the src/main/resources
directory of the Maven project. It's generally rendered with the data taken from the Maven properties. For example:classname=${elasticsearch.plugin.classname} version=${project.version}
src/main/java/org/elasticsearch/plugin/simple/SimplePlugin.java
class is an example of the basic (the minimum required) code that needs to be compiled for executing a plugin:package org.elasticsearch.plugin.simple; import org.elasticsearch.plugins.Plugin; public class SimplePlugin extends Plugin { }
src/main/assemblies/plugin.xml
file used in the Maven assembly step. This file defines the resources that must be packaged into the final ZIP archive. Let's take a look:<?xml version="1.0"?> <assembly> <id>simple-plugin</id> <formats> <format>zip</format> </formats> <includeBaseDirectory>false</includeBaseDirectory> <dependencySets> <dependencySet> <outputDirectory>/</outputDirectory> <useProjectArtifact>true</useProjectArtifact> <useTransitiveFiltering>true</useTransitiveFiltering> <excludes> <exclude>org.elasticsearch:elasticsearch</exclude> </excludes> </dependencySet> </dependencySets> </assembly>
Several parts make up the development life cycle of a plugin, such as designing, coding, building, and deploying. To speed up the build and deployment steps, which are common to all plugins, we need to create a Maven pom.xml
file.
The preceding pom.xml
file is a standard for developing Elasticsearch plugins. This file is composed of:
elasticsearch-simple-plugin
):<name>elasticsearch-simple-plugin</name>
groupId
and artifactId
used to define the plugin artifact name:<groupId>com.packtpub</groupId> <artifactId>simple-plugin</artifactId>
<version>${elasticsearch.version}</version>
<packaging>jar</packaging>
<description>A simple plugin for Elasticsearch</description> <inceptionYear>2016</inceptionYear>
<licenses> <license> <name>The Apache Software License, Version 2.0</name> <url>http://www.apache.org/licenses/LICENSE-2.0.txt</url> <distribution>repo</distribution> </license> </licenses>
sonatype
base pom.<parent> <groupId>org.sonatype.oss</groupId> <artifactId>oss-parent</artifactId> <version>7</version> </parent>
properties
section, the Elasticsearch version and other library versions are set. The properties are used to modularize the Maven strings (replace the properties variable value in different part of pom.xml
). Here we can also define variables that will be resolved during Maven execution.log4j
library are required for the compile phase. If you need to test it, remember to add the test JAR as dependencies.The compiler section, which requires a source compilation. The Java version is fixed at 1.8:
<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>3.5.1</version> <configuration> <source>${maven.compiler.target}</source> <target>${maven.compiler.target}</target> </configuration> </plugin>
The source section, which enables the creation of source packages that are to be released with the binary (useful for debugging):
<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-source-plugin</artifactId> <version>2.3</version> <executions> <execution> <id>attach-sources</id> <goals> <goal>jar</goal> </goals> </execution> </executions> </plugin>
The assembly section, which builds a ZIP taking a configuration plugin.xml
file and puts the output in the releases directory:
<plugin> <artifactId>maven-assembly-plugin</artifactId> <version>2.3</version> <configuration> <appendAssemblyId>false</appendAssemblyId> <outputDirectory>${project.build.directory} /releases/</outputDirectory> <descriptors> <descriptor>${basedir}/src/main/assemblies/ plugin.xml</descriptor> </descriptors> </configuration> <executions> <execution> <phase>package</phase> <goals><goal>single</goal></goals> </execution> </executions> </plugin>
Related to pom.xml
, we have the plugin.xml
file, which describes how to assemble the final ZIP file. This file is usually contained in the /src/main/assemblies/
directory of the project.
The most important sections of this file are as follows:
formats
: Here the destination format is defined:<formats><format>zip</format></formats>
excludes sets in dependencySet
: Contains the artifacts to exclude from the package. Generally, we exclude the Elasticsearch JAR as it's already provided in the server install:<dependencySet> <outputDirectory>/</outputDirectory> <useProjectArtifact>true</useProjectArtifact> <useTransitiveFiltering>true</useTransitiveFiltering> <excludes> <exclude>org.elasticsearch:elasticsearch</exclude> </excludes> </dependencySet>
includes sets in dependencySet
: Contains the artifacts to include into the package. They are mainly the required JAR files used to run the plugin:<dependencySet> <outputDirectory>/</outputDirectory> <useProjectArtifact>true</useProjectArtifact> <useTransitiveFiltering>true</useTransitiveFiltering> <includes>... truncated ...</includes> </dependencySet>
During plugin packaging, the include
and exclude
rules are verified and only files that are allowed to be distributed are put in the ZIP.
After having configured Maven, we can start to write the main plugin class.
Every plugin class must be derived from Plugin
one and it must be public otherwise it cannot be loaded dynamically from the JAR:
import org.elasticsearch.plugins.Plugin; public class SimplePlugin extends Plugin {
After having defined all files required to generate a ZIP release of our plugin it is enough to invoke the maven package
command. This command will compile the code and create a zip
package in the target/releases
directory of your project: the final ZIP file can be deployed as a plugin on your Elasticsearch cluster.
In this recipe, we configured a working environment to build, deploy and test plugins. In the next recipes, we will reuse this environment to develop several plugin types.
Compiling and packaging a plugin are not enough to define a good lifecycle for your plugin: a test phase for testing your plugin functionalities needs to be provided.
Testing the plugin functionalities with test cases reduces the number of bugs that can affect the plugin when its released.
It is possible to add a test phase in the Maven build pom.xml
.
Firstly, we need to add the package dependencies required for testing Elasticsearch and Lucene. These dependencies must be added for the test:
<dependencies> <dependency> <groupId>org.apache.lucene</groupId> <artifactId>lucene-test-framework</artifactId> <version>${lucene.version}</version> <scope>test</scope> </dependency> <dependency> <groupId>org.elasticsearch.test</groupId> <artifactId>framework</artifactId> <version>${elasticsearch.version}</version> <scope>test</scope> </dependency> </dependencies>
The order is very important, so make sure to put the lucene-test-framework
at the top of your dependencies; otherwise problems with loading and executing tests may occur.
For the unit and integration test, the Elasticsearch community mainly uses the hamcrest
library (https://code.google.com/p/hamcrest/). To use this, you need to add its dependencies in the dependencies
section of the pom.xml
:
<dependency> <groupId>org.hamcrest</groupId> <artifactId>hamcrest-core</artifactId> <version>1.3</version> <scope>test</scope> </dependency> <dependency> <groupId>org.hamcrest</groupId> <artifactId>hamcrest-library</artifactId> <version>1.3 </version> <scope>test</scope> </dependency>
To complete the test part, we need to add a Maven plugin, which executes the tests:
<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.12.3</version> <configuration> <includes><include>**/*Tests.java</include></includes> </configuration> </plugin>
The includes
section lists all the possible classes that contain tests via the glob expression.
3.145.125.51