The consumption of files is not something that immediately comes to mind when talking about transactions. Camel provides you with a way to do just this, through the use of the File Component.
This recipe will show you how to perform file consumption, and explain how Camel does its best to guarantee an all-or-nothing process, isolated from other threads and optionally processes, that leaves the consumed file in a state consistent with the overall operation.
The Java code for this recipe is located in the org.camelcookbook.transactions.fileconsumption
package. The Spring XML files are located under src/main/resources/META-INF/spring
and prefixed with fileConsumption
.
The File Component in Camel is part of the camel-core
library. To consume files from a specific directory, define a file:
endpoint URI within a consumer endpoint, that is within a from
statement as per the following pattern:
file:///path/to/source
When a file is successfully processed, it is moved into the .camel
subdirectory under the source directory, by default. This can be overridden through the move
attribute, which takes a relative or absolute path:
file:///path/to/source?move=///path/to/completed
It is also possible to instruct the component to delete completed files rather than moving them:
file:///path/to/source?delete=true
The component scans the directory referenced in the URI, and processes each file as an individual exchange through the route. Exceptions raised while processing the file are handled gracefully, by reattempting the consumption of the entire file.
It is possible to move any failed files to another directory, by providing the failure location in the moveFailed
attribute. For example, in the following route, a failed file will be moved to an error directory, where it can be manually inspected; otherwise, on success, its contents will be sent to an output directory:
from("file:" + inputDirectory + "?moveFailed=" + errorDirectory)
.log("Consumed file ${header[CamelFileName]}: ${body}")
.convertBodyTo(String.class)
.choice()
.when(simple("${body} contains 'explode'"))
.throwException(
new IllegalArgumentException("File caused explosion"))
.otherwise()
.to("file:" + outputDirectory)
.endChoice();
In this example, the directories are provided as variables when the route is instantiated. See Reusing routing logic through template routes of Chapter 1, Structuring Routes, for an overview of this approach.
File consumption is performed as kind of best-effort transaction. Underneath the covers, the File Component binds a org.apache.camel.spi.Synchronization
instance to the exchange as described in the Defining completion actions dynamically recipe in Chapter 7, Error Handling and Compensation. This callback will be invoked by the Camel runtime when that exchange has completed processing.
The File Component provides the logic for moving or deleting, the file if its processing was successful, and either leaving it where it was or moving it to another directory if the file failed. This behavior is configurable through the endpoint URI.
The File Component makes use of the technique covered in the Preventing duplicate invocation of routing logic recipe to ensure that the same files are not reprocessed. Idempotent consumption is disabled by default, but can be enabled by providing a reference to an IdempotentRepository
instance in the consumer URI with the idempotentRepository
attribute. It is useful in the following situations:
noop
attribute is set to true
. Here, an in-memory idempotent repository will be used by default, unless overridden by the idempotentRepository
attribute.IdempotentRepository
implementation whose state is visible to both runtimes. This usually means providing a reference to a JDBCIdempotentRepository
that refers to a shared database.There are a large number of other options around consuming files that include the ability to define file name patterns for consumption, ordering, read locking, recursive directory traversals, and others. It is also possible to use the File Component as a producer endpoint in order to write messages into a directory. See the Camel File Component documentation for full details.
3.145.37.126