You created a transacted InVM listener and, when a runtime exception was thrown, the transaction was rolled back resulting in the message being placed back in the InVM message queue. This happened five times before the rollback discarded the message, sending it to the DLQ.

Here is the configuration. Examine the highlighted section very carefully:

<providers>
  <jms-provider connection-factory="ConnectionFactory"
                name="JBossMQ">
    <jms-bus busid="Chapter5EsbChannel1">
      <jms-message-filter dest-name="queue/chapter5_Request_esb1"
                          dest-type="QUEUE" transacted="false"/>
    </jms-bus>
    <jms-bus busid="Chapter5EsbChannel2">
      <jms-message-filter dest-name="queue/chapter5_Request_esb2"
                          dest-type="QUEUE" transacted="true"/>
    </jms-bus>
  </jms-provider>
</providers>
<services>
  <service category="Chapter5Sample"
           description="A template for Chapter5"
           name="Chapter5Service">
    <listeners>
      <jms-listener busidref="Chapter5EsbChannel1"
                    name="Chapter5provider"/>
    </listeners>
    <actions mep="OneWay">
      <action class="org.jboss.soa.esb.actions.SystemPrintln"
              name="printMessage">
        <property name="message" value="Incoming"/>
      </action>
      <action class="org.jboss.soa.esb.actions.StaticRouter"
              name="RouteToB">
        <property name="destinations">
          <route-to service-category="Chapter5Sample"
                    service-name="Chapter5BService"/>
        </property>
      </action>
    </actions>
  </service>
  <service category="Chapter5Sample" description="Chapter5 B Service"
           invmScope="GLOBAL" name="Chapter5BService"
           invmTransacted="true">
    <actions mep="OneWay">
      <action class="org.jboss.soa.esb.samples.chapter5.MyAction"
              name="processTransaction"/>
      <!--  Will throw an Exception. This should trigger the transaction to be rolledback and the message placedback onto the InVM queue.  -->
      <action class="org.jboss.soa.esb.samples.chapter5.MyAction"
              name="throwException" process="causesException"/>
      <action class="org.jboss.soa.esb.actions.SystemPrintln"
              name="printMessage">
        <property name="message" value="Outgoing"/>
      </action>
    </actions>
  </service>
</services>

Go ahead and remove the invmTransacted attribute and see how the application behaves.

One additional optimization available to the InVM transport is the ability to pass messages using pass-by-reference semantics. This removes the necessity to serialize the message in one service scope and deserialize it in the target service scope.

While this does increase the performance of communication over the InVM transport there are caveats which must be understood:

The message parts accessed in the target service must share a ClassLoader scope with the invoking context

The first issue usually arises when a message part is defined using different definitions of a common class or interface, leading to ClassCastExceptions being raised when the message part is accessed. This will normally occur when the two service scopes are each deployed with a JAR containing the common class or interface. If this cannot be rectified then the pass-by-reference semantics can be explicitly disabled, forcing the message to be serialized in the originating context and deserialized in the target context.

The second issue occurs when the message reference has been passed by reference to the target service yet the originating context continues to modify the contents of the message. These modifications will be visible to the target service although, as a result of the concurrent nature of the execution, it will always be uncertain as to when this will occur. This leads to undesirable race conditions within the application, often hard to track down. If it is not possible to modify the behavior of the services then disabling the pass-by-reference semantics will cause a separate instance of the message to be delivered to the target service as a consequence of the serialization/deserialization.

The pass-by-reference semantics are disabled by explicitly enabling pass-by-value on the service, done by setting the inVMPassByValue property on the <service> element to true. An example configuration would look like this:

<service category="Chapter5Sample" name="Chapter5Service"
         invmScope="GLOBAL">
  <property name="inVMPassByValue" value="true"/>
</service>

Modify the custom action to print the body object's address using Integer.toHexString(System.identityHashCode(message.getBody())) and see what gets printed when it is printed both from Chapter5Service and Chapter5BService. You will have to add MyAction to your service like this:

<service category="Chapter5Sample"
         description="A template for Chapter5"
         name="Chapter5Service">
  <listeners>
    <jms-listener busidref="Chapter5EsbChannel1"
                  name="Chapter5provider"/>
  </listeners>
  <actions mep="OneWay">
    <action class="org.jboss.soa.esb.actions.SystemPrintln"
            name="printMessage">
      <property name="message" value="Incoming"/>
    </action>

    <action class="org.jboss.soa.esb.samples.chapter5.MyAction"
            name="printAddress"/>
    <action class="org.jboss.soa.esb.actions.StaticRouter"
            name="RouteToB">
      <property name="destinations">
        <route-to service-category="Chapter5Sample"
                  service-name="Chapter5BService"/>
      </property>
    </action>
  </actions>
</service>
<service category="Chapter5Sample" description="Chapter5 B Service"
         invmScope="GLOBAL" name="Chapter5BService">
  <actions mep="OneWay">
    <action class="org.jboss.soa.esb.samples.chapter5.MyAction"
            name="printAddress"/>   
  </actions>
</service>

Your MyAction.java file would look like this:

public Message process(Message message) throws Exception {
    System.out.println("Body: " + Integer.toHexString(System.identityHashCode(message.getBody())));
    return message;
}

Does it change when inVMPassByValue is set to true?

One of the problems which may surface when using the InVM transport is that the target service may become swamped by the incoming messages, with the message originators being able to send messages to the service faster than the service can process them.

JBoss ESB includes a mechanism which enables the automatic throttling of the message originator in order to prevent this scenario, known as lock-step delivery.

The "lock-step" delivery mechanism blocks the originator until the service is in a position to consume the message or until a specified timeout has occurred, preventing the originator from producing messages at a significantly faster rate than the consumer can process.

It should be noted that this is not a synchronous delivery mechanism, it is the delivery of the message which releases the originator and not a response from the service.

Lock-step delivery is disabled by default but can be explicitly enabled on a service using the inVMLockStep and inVMLockStepTimeout properties.

An example configuration with lock-step delivery enabled would look like this:

<service category="Chapter5Sample" name="Chapter5Service"
         invmScope="GLOBAL">
  <property name="inVMLockStep" value="true"/>
  <property name="inVMLockStepTimeout" value="4000"/>
</service>