- Elasticsearch 5.x Cookbook Third Edition
- Credits
- About the Author
- About the Reviewer
- www.PacktPub.com
- Customer Feedback
- Dedication
- Preface
- 1. Getting Started
- 2. Downloading and Setup
- Introduction
- Downloading and installing Elasticsearch
- Setting up networking
- Setting up a node
- Setting up for Linux systems
- Setting up different node types
- Setting up a client node
- Setting up an ingestion node
- Installing plugins in Elasticsearch
- Installing plugins manually
- Removing a plugin
- Changing logging settings
- Setting up a node via Docker
- 3. Managing Mappings
- Introduction
- Using explicit mapping creation
- Mapping base types
- Mapping arrays
- Mapping an object
- Mapping a document
- Using dynamic templates in document mapping
- Managing nested objects
- Managing child document
- Adding a field with multiple mapping
- Mapping a GeoPoint field
- Mapping a GeoShape field
- Mapping an IP field
- Mapping an attachment field
- Adding metadata to a mapping
- Specifying a different analyzer
- Mapping a completion field
- 4. Basic Operations
- Introduction
- Creating an index
- Deleting an index
- Opening/closing an index
- Putting a mapping in an index
- Getting a mapping
- Reindexing an index
- See also
- Refreshing an index
- Flushing an index
- ForceMerge an index
- Shrinking an index
- Checking if an index or type exists
- Managing index settings
- Using index aliases
- Rollover an index
- Indexing a document
- Getting a document
- Deleting a document
- Updating a document
- Speeding up atomic operations (bulk operations)
- Speeding up GET operations (multi GET)
- 5. Search
- Introduction
- Executing a search
- Sorting results
- Highlighting results
- Executing a scrolling query
- Using the search_after functionality
- Returning inner hits in results
- Suggesting a correct query
- Counting matched results
- Explaining a query
- Query profiling
- Deleting by query
- Updating by query
- Matching all the documents
- Using a boolean query
- 6. Text and Numeric Queries
- Introduction
- Using a term query
- Using a terms query
- Using a prefix query
- Using a wildcard query
- Using a regexp query
- Using span queries
- Using a match query
- Using a query string query
- Using a simple query string query
- Using the range query
- The common terms query
- Using IDs query
- Using the function score query
- Using the exists query
- Using the template query
- 7. Relationships and Geo Queries
- 8. Aggregations
- Introduction
- Executing an aggregation
- Executing stats aggregations
- Executing terms aggregation
- Executing significant terms aggregation
- Executing range aggregations
- Executing histogram aggregations
- Executing date histogram aggregations
- Executing filter aggregations
- Executing filters aggregations
- Executing global aggregations
- Executing geo distance aggregations
- Executing children aggregations
- Executing nested aggregations
- Executing top hit aggregations
- Executing a matrix stats aggregation
- Executing geo bounds aggregations
- Executing geo centroid aggregations
- 9. Scripting
- 10. Managing Clusters and Nodes
- 11. Backup and Restore
- 12. User Interfaces
- 13. Ingest
- 14. Java Integration
- Introduction
- Creating a standard Java HTTP client
- Creating an HTTP Elasticsearch client
- Creating a native client
- Managing indices with the native client
- Managing mappings
- Managing documents
- Managing bulk actions
- Building a query
- Executing a standard search
- Executing a search with aggregations
- Executing a scroll search
- 15. Scala Integration
- 16. Python Integration
- 17. Plugin Development
- 18. Big Data Integration
The power of the pipeline definition is the ability for to be updated and created without a node restart (compared to Logstash). The definition is stored in a cluster state via the put pipeline API.
After having defined a pipeline, we need to provide it to the Elasticsearch cluster.
You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.
To execute curl via the command line, you need to install curl for your operative system.
To store or update an ingestion pipeline in Elasticsearch, we will perform the following steps:
- We can store the ingest pipeline via a
PUTcall:curl -XPUT 'http://127.0.0.1:9200/_ingest/pipeline/add-user- john' -d '{ "description" : "Add user john field", "processors" : [ { "set" : { "field": "user", "value": "john" } } ] , "version":1 }' - The result returned by Elasticsearch, if everything is okay, should be as follows:
{"acknowledged":true}
The PUT pipeline method works both for creating a pipeline as well as updating an existing one.
The pipelines are stored in a cluster state, and they are immediately propagated to all ingest nodes. When the ingest nodes receive the new pipeline, they will update their node in-memory pipeline representation: the pipeline changes take effect immediately.
When you store a pipeline in the cluster, pay attention to provide a meaningful name to it (in the example, add-user-john) so as to easily understand what the pipeline does.
The name of the pipeline used in the put call will be the ID of the pipeline in other pipeline flows.
After having stored your pipeline in Elasticsearch, you can index a document providing the pipeline name as query argument.
For example:
curl -XPUT http://localhost:9200/my_index/my_type/my_id?pipeline=add-user-john -d '{}'
The document will be enriched by the pipeline before being indexed.
-
No Comment