In the vast majority of cases, you will want to extract a value of the single attribute of a target object, in which case you can use the built-in ReflectionExtractor class. The ReflectionExtractor accepts a method name as a constructor argument, invokes the specified method on a target object via reflection, and returns the result of that method invocation.

As a matter of fact, the ReflectionExtractor is used so often that you can simply specify a method name as a string in most places where a value extractor is expected and an instance of a ReflectionExtractor will be created automatically for you, which is what we took advantage of in the previous filter definitions. For example, the filter definition:

Filter filter = new BetweenFilter("getTime", from, to);

Is really just a shorter form of:

Filter filter = new BetweenFilter(
new ReflectionExtractor("getTime"),
from, to);

I will have to admit that as useful as ReflectionExtractor is, I have never liked it much. The main reason for this is that it forces you to spell out a full method name for an attribute, when a property name, as defined by the Java Bean specification should've been enough and would've made the code more readable. This is especially bothersome when accessing a Coherence cluster from a .NET client, in which case the'get' prefix in front of the property name truly feels unnatural.

Fortunately, it is easy to fix the problem by implementing a similar value extractor that uses introspection to obtain an attribute value:

public class PropertyExtractor
implements ValueExtractor, Serializable {
private final String m_propertyName;
private transient volatile Method m_readMethod;
public PropertyExtractor(String propertyName) {
m_propertyName = propertyName;
}
public Object extract(Object o) {
if (o == null) {
return null;
}
Class targetClass = o.getClass();
try {
if (readMethod == null || readMethod.getDeclaringClass() != targetClass) {
PropertyDescriptor pd =
new PropertyDescriptor(propertyName, o.getClass());
readMethod = pd.getReadMethod();
}
return readMethod.invoke(o);
}
catch (Exception e) {
throw new RuntimeException(e);
}
}
}

Now we can use PropertyExtractor instead of ReflectionExtractor in our filter definitions:

Filter filter = new BetweenFilter(
new PropertyExtractor("time"),from, to);

In this example the difference is not significant and it could even be argued that the PropertyExtractor makes the code harder to read as we have to specify it explicitly, instead of using a filter constructor that takes string as an argument and creates ReflectionExtractor for us. However, in the next section we will implement a helper class that makes filter creation much simpler, and the PropertyExtractor will allow us to make things as simple as they can be.

While the ReflectionExtractor is definitely the one that is used most often, there are several other value extractors that ship with Coherence.

ValueExtractor ex =
new ChainedExtractor(new ValueExtractor[] {
new ReflectionExtractor("getId"),
new ReflectionExtractor("getAccountId") });

ValueExtractor ex =
new ChainedExtractor("getId.getAccountId");

While the built-in value extractors should be sufficient for most usage scenarios, there might be some situations where implementing a custom one makes sense. We have already implemented one custom value extractor, PropertyExtractor, in order to improve on the built-in ReflectionExtractor and allow ourselves to specify JavaBean property names instead of the full method names, but there are other scenarios when this might be appropriate.

One reason why you might want to implement a custom value extractor is to enable transformation of cache values from their native type to some other type. For example, most applications use UI controls such as drop-downs or list boxes to present a list of possible choices to the user. Let's assume that we need to display a list of countries in a drop-down on the registration screen for new customers.

We already have a cache containing all the countries, so we could easily get all the values from it and send them to the client, which would use them to populate the drop-down. However, the Country class we defined in Chapter 2 has a number of attributes we don't need in order to populate the drop-down list, such as capital, currency symbol, and currency name—the only attributes we do need are the country code and country name, so by sending any other information to the client we would only be wasting network bandwidth.

As a matter of fact, for most, if not all, drop-downs and list boxes in an application we will need only an identifier that will be returned as a selected value, and a description that should be used for display purposes. That means that we can define a class containing only those two attributes:

public class LookupValue implements Serializable {
private Object m_id;
private String m_description;
public LookupValue(Object id, String description) {
m_id = id;
m_description = description;
}
public Object getId() {
return m_id;
}
public String getDescription() {
return m_description;
}
}

Now that we have a holder class that can be used to represent any lookup value, the remaining question is how we can transform instances of the Country class into the instances of the LookupValue class. The answer is simple—we can write a custom value extractor that will do it for us:

public class LookupValueExtractor
extends AbstractExtractor
implements PortableObject, Serializable {
private ValueExtractor m_idExtractor;
private ValueExtractor m_descriptionExtractor;
public LookupValueExtractor(ValueExtractor idExtractor,
ValueExtractor descriptionExtractor) {
m_idExtractor = idExtractor;
m_descriptionExtractor = descriptionExtractor;
}
public Object extractFromEntry(Map.Entry entry) {
Object id = InvocableMapHelper.extractFromEntry(m_idExtractor,
entry);
String description = (String)
InvocableMapHelper.extractFromEntry(m_descriptionExtractor, entry);
return new LookupValue(id, description);
}
// equals and hashCode omitted for brevity
}

The implementation is actually very simple: we allow users to specify two value extractors, an idExtractor and a descriptionExtractor, that we use to extract the values that are used to create a LookupValue instance. However, one thing deserves clarification.

Instead of simply implementing the ValueExtractor interface, we are extending the AbstractExtractor class and implementing the extractFromEntry method. The reason for this is that we want to be able to extract id and description not only from the entry value, but from the entry key as well.

In order to achieve that, we rely on the InvocableMapHelper class, which provides a utility method that can be used to extract a value from any object that implements Map.Entry interface.

Of course, the LookupValueExtractor is only part of the story—we still need a way to execute this extractor against all the objects in the countries cache and get the collection of extracted lookup values back. We will see what the best way to do that is shortly, but for now let's return to Coherence filters and see how we can make complex queries easier to create.

The idea behind the FilterBuilder implementation is that many queries are based on simple attribute comparisons, where multiple attribute comparisons are concatenated using the logical AND, or less often OR operator.

If you review the table of the built-in filter types at the beginning of this chapter, you will see that Coherence already provides all the core facilities we need to implement this: we have all the common comparison operators, as well as some of the less common ones, and there are AllFilter and AnyFilter, which allow us to create logical AND and OR filters for an array of filters respectively. What we don't have is an easy way to create an array of filters, and that's exactly what the FilterBuilder will help us do.

The goal is to be able to create a filter using code similar to this:

Filter filter = new FilterBuilder()
.equals("id.accountId", 123)
.between("time", from, to)
.build();

This will allow us to define complex queries in a much shorter and significantly more readable way. In order to support the previous syntax, we can implement the FilterBuilder class as follows:

public class FilterBuilder {
private Class defaultExtractorType;
private List<Filter> filters = new ArrayList<Filter>();
// constructors
public FilterBuilder() {
this(PropertyExtractor.class);
}
public FilterBuilder(Class defaultExtractorType) {
this.defaultExtractorType = defaultExtractorType;
}
// public members
public FilterBuilder equals(String propertyName,
Object value) {
return equals(createExtractor(propertyName), value);
}
public FilterBuilder equals(ValueExtractor extractor,
Object value) {
filters.add(new EqualsFilter(extractor, value));
return this;
}
public FilterBuilder notEquals(String propertyName,
Object value) {
return notEquals(createExtractor(propertyName), value);
}
public FilterBuilder notEquals(ValueExtractor extractor,
Object value) {
filters.add(new NotEqualsFilter(extractor, value));
return this;
}
public FilterBuilder greater(String propertyName,
Comparable value) {
return greater(createExtractor(propertyName), value);
}
public FilterBuilder greater(ValueExtractor extractor,
Comparable value) {
filters.add(new GreaterFilter(extractor, value));
return this;
}
// and so on...
}

Basically, we are implementing two overloaded methods for each built-in filter: one that accepts a value extractor as the first argument, and one that accepts a string and creates a value extractor for it.

However, unlike the built-in filters, we do not create an instance of a ReflectionExtractor automatically, but delegate the actual creation of an extractor to the createExtractor factory method:

protected ValueExtractor createExtractor(String propertyName) {
if (propertyName.indexOf('.') >= 0) {
return new ChainedExtractor(
createExtractorArray(propertyName.split(".")));
}
if (propertyName.indexOf(',') >= 0) {
return new MultiExtractor(
createExtractorArray(propertyName.split(",")));
}
return createDefaultExtractor(propertyName);
}

As you can see, if the specified property name is a dot-separated string, we will create an instance of a ChainedExtractor. Similarly, we will create an instance of a MultiExtractor for a comma-separated list of property names.

For all other properties, we will delegate extractor creation to the createDefaultExtractor method:

protected ValueExtractor createDefaultExtractor(String propertyName) {
Constructor ctor = getConstructor(defaultExtractorType);
return (ValueExtractor) ctor.newInstance(propertyName);
}

This allows us to control on a case-by-case basis which value extractor should be used within our filter. In most cases, the default PropertyExtractor should work just fine, but you can easily change the behavior by specifying a different extractor class as a constructor argument:

Filter filter = new FilterBuilder(ReflectionExtractor.class)
.equals("getCustomerId", 123)
.greater("getTotal", 1000.0)
.build();

You can even specify your own custom extractor class—the only requirement is that it implements a constructor that accepts a single string argument.

As we discussed in the previous chapter, data affinity can provide a significant performance boost because it allows Coherence to optimize the query for related objects. Instead of executing the query in parallel across all the nodes and aggregating the results, Coherence can simply execute it on a single node, because data affinity guarantees that all the results will be on that particular node. This effectively reduces the number of objects searched to approximately C/N, where C is the total number of objects in the cache query is executed against, and N is the number of partitions in the cluster.

However, this optimization is not automatic—you have to target the partition to search explicitly, using KeyAssociatedFilter:

Filter query = ...;
Filter filter = new KeyAssociatedFilter(query, key);

In the previous example, we create a KeyAssociatedFilter that wraps the query we want to execute. The second argument to its constructor is the cache key that determines the partition to search.

To make all of this more concrete, let's look at the final implementation of the code for our sample application that returns account transactions for a specific period. First, we need to add the getTransactions method to our Account class:

public Collection<Transaction> getTransactions(Date from, Date to) {
return getTransactionRepository().findTransactions(m_id, from, to);
}

Finally, we need to implement the findTransactions method within the CoherenceTransactionRepository:

public Collection<Transaction> findTransactions(
Long accountId, Date from, Date to) {
Filter filter = new FilterBuilder()
.equals("id.accountId", accountId)
.between("time", from, to)
.build();
return queryForValues(
new KeyAssociatedFilter(filter, accountId),
new DefaultTransactionComparator());
}

As you can see, we target the query using the account identifier and ensure that the results are sorted by transaction number by passing DefaultTransactionComparator to the queryForValues helper method we implemented earlier. This ensures that Coherence looks for transactions only within the partition that the account with the specified id belongs to.

One situation where a direct query using the entrySet method might not be appropriate is when you need to query a near cache.

Because there is no way for Coherence to determine if all the results are already in the front cache, it will always execute the query against the back cache and return all the results over the network, even if some or all of them are already present in the front cache. Obviously, this is a waste of network bandwidth.

What you can do in order to optimize the query is to obtain the keys first and then retrieve the entries by calling the CacheMap.getAll method:

Filter filter = ...;
Set keys = cache.keySet(filter);
Map results = cache.getAll(keys);

The getAll method will try to satisfy as many results as possible from the front cache and delegate to the back cache to retrieve only the missing ones. This will ensure that we move the bare minimum of data across the wire when executing queries, which will improve the throughput.

However, keep in mind that this approach might increase latency, as you are making two network roundtrips instead of one, unless all results are already in the front cache. In general, if the expected result set is relatively small, it might make more sense to move all the results over the network using a single entrySet call.

Another potential problem with the idiom used for near cache queries is that it could return invalid results. There is a possibility that some of the entries might change between the calls to keySet and getAll. If that happens, getAll might return entries that do not satisfy the filter anymore, so you should only use this approach if you know that this cannot happen (for example, if objects in the cache you are querying, or at least the attributes that the query is based on, are immutable).

We have already seen that the entrySet method allows you to pass a Comparator as a second argument, which will be used to sort the results. If your objects implement the Comparable interface you can also specify null as a second argument and the results will be sorted based on their natural ordering. For example, if we defined the natural sort order for transactions by implementing Comparable within our Transaction class, we could've simply passed null instead of a DefaultTransactionComparator instance within the findTransactions implementation shown earlier.

On the other hand, if you use near cache query idiom, you will have to sort the results yourself. This is again an opportunity to add utility methods that allow you to query near cache and to optionally sort the results to our base repository class. However, there is a lot more to cover in this chapter, so I will leave this as an exercise for the reader.

The LimitFilter is somewhat special and deserves a separate discussion. Unlike other filters, which are used to compose query criteria, the LimitFilter is used to control how many result items are returned at a time. Basically, it allows you to page through query results n items at a time.

This also implies that unlike other filters, which are constructed, executed, and discarded, an instance of a LimitFilter is something you might need to hold on to for an extended period of time, as it is a mutable object that keeps track of the current page number, top and bottom anchor objects, and other state that is necessary to support paging.

Let's look at a simple example to better demonstrate the proper usage of a LimitFilter:

NamedCache countries = CacheFactory.getCache("countries");
LimitFilter filter = new LimitFilter(
new LikeFilter("getName", "B%"), 5);
Set<Map.Entry> entries = countries.entrySet(filter, null);
// contains countries 1-5 whose name starts with a letter 'B'
filter.nextPage();
entries = countries.entrySet(filter, null);
// contains countries 6-10 whose name starts with a letter 'B'
filter.setPage(4);
entries = countries.entrySet(filter, null);
// contains countries 21-25 whose name starts with 'B'

As you can see, you can page through the result by executing the same query over and over again and modifying the current page of the LimitFilter between the query executions by calling the nextPage, previousPage, or setPage method.

The LimitFilter is extremely powerful as it allows you to execute the main query only once and then obtain the results in chunks of the size you specify. It maps very nicely to a common requirement for results paging within a web application, allowing you to bring the web server only the data it needs to generate the current page, thus reducing network traffic and improving application performance and scalability. You can safely store an instance of a LimitFilter within a user's HTTP session and reuse it later when the user navigates to another page of the results.

One thing to note in the preceding example is that we are using the entrySet method to retrieve the results, contrary to what we have discussed in the previous section. The reason for that is that we want to return countries sorted by name (natural order), and as I mentioned earlier, if we need to support paging over the sorted results we have no other option but to sort them within the cluster using an overload of the entrySet method that accepts a comparator.

However, this is really not an issue, as the amount of data sent over the wire will be naturally limited by the LimitFilter itself and will typically be very small, so we don't need to optimize the query for the near caching scenario.

A Coherence index is an instance of a class that implements the com.tangosol.util.MapIndex interface:

public interface MapIndex {
ValueExtractor getValueExtractor();
boolean isOrdered();
Map getIndexContents();
Object get(Object key);
}

The getValueExtractor method returns the value extractor used to extract the attribute that should be indexed from an object, while the isOrdered method returns whether the index is sorted or not.

The get method allows us to obtain a value of the indexed attribute for the specified key directly from an index, which avoids object deserialization and repeat value extraction.

Finally, the getIndexContents method returns the actual index contents. This is a map that uses the value extracted from the indexed attribute as a key, while the value for each index entry is a set of cache keys corresponding to that attribute value.

Looking at an example should make the previous paragraph much easier to understand.

Let's assume that we have the following entries in the cache:

If we create an index on the lastName attribute, our index contents will look like this:

On the other hand, an index on the firstName attribute will look like this:

The previous example should also make it obvious why indexes have such a profound effect on query performance.

If we wanted to obtain a list of keys for all people in the cache that have last name 'Seovic', without an index Coherence would have to deserialize each cache entry, extract the lastName attribute and perform a comparison, and if the comparison matches then it would add the cache entry key to the resulting list of keys.

With an index, Coherence doesn't need to do any of this—it will simply look up an index entry with the key Seovic and return the set of keys from that index entry.

Now that we know how Coherence indexes are structured and why we should use them, let's look at how we can create them.

At the beginning of this chapter, we showed an incomplete definition of a QueryMap interface. What we omitted from it are the two methods that allow us to create and remove cache indexes:

public interface QueryMap extends Map {
Set keySet(Filter filter);
Set entrySet(Filter filter);
Set entrySet(Filter filter, Comparator comparator);
void addIndex(ValueExtractor extractor,
boolean isOrdered,
Comparator comparator);
void removeIndex(ValueExtractor extractor);
}

As you can see, in order to create an index you need to specify three things:

The first argument is by far the most important, as it determines index contents. It is also used as the index identifier, which is why you need to ensure that all value extractors you create implement the equals and hashCode methods properly.

If you decide to create an ordered index, index entries will be stored within a SortedMap instance, which introduces some overhead on index updates. Because of that, you should only order indexes for attributes that are likely to be used for sorting query results or in range queries, such as greater, less, between, and so on.

The last argument allows you to specify a comparator to use for index ordering, but you can specify null if the attribute you are indexing implements the Comparable interface and you want the index to use natural ordering. Of course, if an index is not ordered, you should always specify null for this argument.

Now that we know all of this, let's see what the code to define indexes on firstName and lastName attributes from the previous example should look like:

NamedCache people = CacheFactory.getCache("people");
people.addIndex(new PropertyExtractor("firstName"), false, null);
people.addIndex(new PropertyExtractor("lastName"), true, null);

As you can see, adding indexes to a cache is very simple. In this case we have created an unordered index on the firstName attribute and an ordered index using natural string ordering on the lastName attribute.

The last thing you should know is that the call to the addIndex method is treated by Coherence as a hint that an index should be created. What this means in practice is that you can safely create the same set of indexes on each Coherence node, even if another node has already created those same indexes. If an index for the specified extractor already exists, Coherence will simply ignore all subsequent requests for its creation.

Just as with filters, Coherence ships with a number of useful built-in aggregators, and an equivalent of the AverageAggregator is one of them. Actually, there are two average aggregators built-in, as you can see in the following table:

The important thing to note about the various average, max, min, and sum aggregators is that they differ from each other in how they treat the numeric values they are aggregating, as well as by the type of the return value.

For example, while you can use the DoubleAverage aggregator to calculate the average for any set of java.lang.Number-derived values, you should be aware that each individual value will be converted to Double first using the Number.doubleValue method, which might lead to rounding errors. What you will typically want to do is use the most appropriate aggregator based on the actual type of the values you are aggregating, and convert the final result to the desired type if necessary.

So far we have learned how to implement an aggregator and which aggregators are shipped with Coherence, but we haven't learned how to use them yet.

In order to execute an aggregator, you need to use one of the methods defined by the com.tangosol.util.InvocableMap interface:

public interface InvocableMap extends Map {
Object aggregate(Collection keys,
InvocableMap.EntryAggregator aggregator);
Object aggregate(Filter filter,
InvocableMap.EntryAggregator aggregator);
}

There are few more methods in the InvocableMap interface that we will cover in more detail in the next chapter, but these two are all we need to execute aggregators against cache entries.

The first overload of the aggregate method accepts an explicit collection of keys for a set of entries to aggregate, while the second one uses a filter to determine the set of entries aggregation should be performed on. Both methods accept an aggregator instance as a second argument, which can be either one of the built-in aggregators or a custom aggregator you have implemented.

Earlier in the chapter, we started the implementation of a generic solution that will allow us to extract lookup values that are suitable for data binding to UI controls such as drop-downs and list boxes. So far, we have implemented a LookupValueExtractor, which allows us to extract a LookupValue instance from any object, in any cache.

In this section we will complete the exercise by implementing a LookupValuesAggregator—a simple aggregator that can be used to aggregate extracted lookup values into a list.

public class LookupValuesAggregator
extends AbstractAggregator {
private transient List<LookupValue> results;
public LookupValuesAggregator(ValueExtractor idExtractor,
ValueExtractor descriptionExtractor){
super(new LookupValueExtractor(idExtractor,
descriptionExtractor));
}
protected void init(boolean isFinal) {
results = new ArrayList<LookupValue>();
}
protected void process(Object value, boolean isFinal) {
if (isFinal) {
results.addAll((Collection<LookupValue>) value);
}
else {
results.add((LookupValue) value);
}
}
protected Object finalizeResult(boolean isFinal) {
return results;
}
}

As you can see, both init and finalizeResult methods are trivial—the first one simply initializes the results list, while the second one returns it. This works both for the main and parallel aggregators, so we can ignore the isFinal flag.

The process method, however, uses the isFinal flag to determine if it should add a single LookupValue instance or the list of LookupValues returned by the parallel aggregator to the results collection.