Every so often you will find yourself developing routes that you need to validate for performance under load, as well as their general correctness. The mocking techniques that have been discussed so far will only help you determine whether a test message correctly exercises the route.
This recipe will show how you can use the DataSet Component to generate a set of test data and play it through your route. It will then demonstrate how this same component can be used as a bulk mocking mechanism to validate your route logic under load. This technique is not a replacement for proper system integration load testing. It provides a means to more easily unit test scenarios involving larger numbers of messages.
Consider the following routing logic, which is defined in a RouteBuilder implementation that includes property setters allowing us to inject start and end URIs:
public class SlowlyTransformingRouteBuilder
extends RouteBuilder {
private String sourceUri; // setters omitted...
private String targetUri;
@Override
public void configure() throws Exception {
from(sourceUri)
.to("seda:transformBody");
from("seda:transformBody?concurrentConsumers=15")
.transform(simple("Modified: ${body}"))
.delay(100) // pretend this is a slow transformation
.to("seda:sendTransformed");
from("seda:sendTransformed") // one thread used here
.resequence().simple("${header.mySequenceId}").stream()
.to(targetUri);
}
}This RouteBuilder implementation uses the SEDA Component to allow multiple threads to execute a slow transformation on a number of incoming messages in parallel. When the messages are successfully transformed, they are handed to another route that uses the Resequencer EIP. The Resequencer is a pattern built into Camel that sorts exchanges flowing through a route. In this instance, exchanges are sorted according to a header that identifies the original order that they were sent in (mySequenceId). This header is set on the incoming messages before they enter the first route.
We would like to verify that the messages are sent out in the order that they came in, and also that we have enough concurrentConsumers set on the second route to deal with 100 messages a second.
The Java code for this recipe is located in the org.camelcookbook.examples.testing.dataset package.
To apply the DataSet Component for bulk testing, perform the following steps:
- To generate a set of inbound test messages, extend the
DataSetSupportclass, overriding the abstractcreateMessageBody()method, as well as theapplyHeaders()template method:public class InputDataSet extends DataSetSupport { @Override protected Object createMessageBody(long messageIndex) { return "message " + messageIndex; } protected void applyHeaders(Exchange exchange, long messageIndex) { exchange.getIn() .setHeader("mySequenceId", messageIndex); } }The
DataSetSupportclass is an abstract implementation of theDataSetinterface, which is used by theDataSetComponent to generate messages. - To generate the messages that will be expected at the end of the route, repeat by extending
DataSetSupportagain, this time skipping the setting of headers:public class ExpectedOutputDataSet extends DataSetSupport { @Override protected Object createMessageBody(long messageIndex) { return "Modified: message " + messageIndex; } } - In your
CamelTestSupporttest class, register the two beans with the Camel context:@Override protected CamelContext createCamelContext() throws Exception { final int testBatchSize = 1000; InputDataSet inputDataSet = new InputDataSet(); inputDataSet.setSize(testBatchSize); ExpectedOutputDataSet expectedOutputDataSet = new ExpectedOutputDataSet(); expectedOutputDataSet.setSize(testBatchSize); SimpleRegistry registry = new SimpleRegistry(); registry.put("input", inputDataSet); registry.put("expectedOutput", expectedOutputDataSet); return new DefaultCamelContext(registry); }If using this style of testing with Spring, the two
DataSetimplementations should be registered as beans in the Spring context. - Instantiate your
RouteBuilderand set twodataset:endpoints: thestartURIendpoint pointing to theInputDataSetbean, and thetargetUriendpoint pointing to theExpectedOutputDataSet:@Override protected RouteBuilder createRouteBuilder() throws Exception { SlowlyTransformingRouteBuilder routeBuilder = new SlowlyTransformingRouteBuilder(); routeBuilder.setSourceUri( "dataset:input?produceDelay=-1"); routeBuilder.setTargetUri("dataset:expectedOutput"); return routeBuilder; }In your test method, fetch the
dataset:expectedOutputmock endpoint, set the maximum time that it should wait to receive its expected 1,000 messages, and assert that it was satisfied:@Test public void testPayloadsTransformedInExpectedTime() throws InterruptedException { MockEndpoint expectedOutput = getMockEndpoint("dataset:expectedOutput"); expectedOutput.setResultWaitTime(10000); // 10 seconds expectedOutput.assertIsSatisfied(); }
The DataSet Component is used here both as a consumer endpoint to generate test messages, and as a producer endpoint to act as a bulk mock testing mechanism.
When the Camel context starts the route, a single thread will be created by the dataset:input endpoint. This thread will perform the following steps in a loop:
- Generate the message body by calling
createMessageBody()on theinputbean - Set the message headers through
applyHeaders() - Process the resulting exchange through the route
Each time a message is constructed, a messageIndex parameter is passed in that allows your code to vary the message.
A produceDelay attribute on the URI allows you to set how long the thread should sleep between having finished the routing of the message and generating the next message; -1 indicates no delay.
When used as a producer in a to(..) block statement, a DataSetEndpoint interface acts as a bulk mock endpoint that automatically sets its own expectations.
At the other end of our routes, dataset:expectedOutput waits to receive 1,000 messages. If the time set in your test by setResultWaitTime()
expires, the assertion will fail. If the messages were received, the endpoint will generate that same number of exchanges through its own implementation class. It does this in order to compare the received messages against the expected ones.
The two sets of Exchange objects are compared to each other in order. The ordering is determined by matching the CamelDataSetIndex attributes that are set by the consumer and producer dataset: endpoints. The two message bodies are then compared for equality. Other headers and attributes are not considered. A mismatch of either the index attributes, or the bodies will result in the assertion failing.
It is possible to generate a default set of message headers by overriding DataSetSupport.populateDefaultHeaders(Map<String, Object>). These will be overwritten on a per-message basis in applyHeaders().
There is a consumeDelay attribute that can also be used as part of the producer URI to simulate a slow consumer.
- DataSet Component: http://camel.apache.org/dataset.html
- Resequencer: http://camel.apache.org/resequencer.html