Apache Solr
is an open-source search platform built on top of the Apache Lucene
search engine library. Spring Roo's Solr add-on
provides support for integrating the Roo-generated domain model with Solr platform. In this recipe, we'll look at how Roo makes use of SolrJ Java client library
to add domain model data into Solr server for indexing and to search domain model data based on user supplied query parameters.
To see Roo's support for Solr in action, you need to download and run the Solr server, as described here:
Solr
website
and unzip the bundle into a directory. Let's call the unzipped directory as SOLR_HOME
.SOLR_HOMEexample
directory and start Solr server:C:...apache-solr-1.4.0example> java –jar start.jar
http://localhost:8983/solr/admin/
Now, create a sub-directory ch06-solr
inside C:
oo-cookbook
directory, copy ch06_web_app.roo
script and start Roo shell from C:
oo-cookbookch06-solr
.
Follow these step to add search capability:
ch06_solr.roo
script, as shown here:roo> script --file ch06_web_app.roo
The script creates a flightapp-web
Roo project consisting of Flight
and FlightDescription
JPA entities.
flightapp-web
project using the solr
setup
command:Updated ROOTpom.xml [Added dependency org.apache.solr:solr-solrj:1.4.0] Created SRC_MAIN_RESOURCESMETA-INFspringsolr.properties Updated SRC_MAIN_RESOURCESMETA-INFspringapplicationContext.xml
.. roo> solr setup
solr
all
command as shown here:.. roo> solr all Updated SRC_MAIN_JAVA...FlightDescription.java Updated SRC_MAIN_JAVA...Flight.java Created SRC_MAIN_JAVA..Flight_Roo_SolrSearch.aj Created SRC_MAIN_JAVA...FlightDescription_Roo_SolrSearch.aj
.. roo> controller class --class ~.web.FlightDescriptionSearchController --preferredMapping /flightdescriptionsearch Created SRC_MAIN_JAVA...FlightDescriptionSearchController .java Created SRC_MAIN_WEBAPPWEB-INFviewsflightdescriptionsearch Created SRC_MAIN_WEBAPPWEB-INFviewsflightdescriptionsearchindex.jspx
Copy MySolrField.java
and FlightDescriptionSearchController.java
files from the source code that accompanies this chapter to sample.roo.flightapp.web
package. Also, replace /WEB-INF/views/flightdescriptionsearch/index.jsp
with the index.jsp
file from the source code that accompanies this chapter.
perform
eclipse
command so that you can import the flightapp-web
project into your Eclipse IDE as shown here:.. roo> perform eclipse
tomcat:run
goal of Maven Tomcat Plugin from ch06-solr
directory to deploy the flightapp-web
project in embedded Tomcat container as shown here:C:
oo-cookbookch06-solr> mvn tomcat:run
http://localhost:8080/flightapp-web
. If you see the following home page of the web application, it means that the flightapp-web
project is successfully deployed on Tomcat:In the given screenshot, Flight Description Search Controller View menu option sends request to FlightDescriptionSearchController
class, which in turn renders the index.jsp
page located in /WEB-INF/views/flightdescriptionsearch
folder.
FlightDescription
entities in database, as shown here:The given screenshot shows that you need to enter information about the following fields: Price, Origin and Destination. Create two FlightDescription
instances with the information shown in the following table:
Instance |
Price |
Origin |
Destination |
Instance-1 |
1200 |
NYC |
DELHI |
Instance-2 |
1400 |
MUMBAI |
ATLANTA |
FlightDescription
entity instance, Roo's support for Solr adds the entity data into Solr for indexing and searching. Select the Flight Description Search Controller View menu option that searches the Solr server for documents that have a field named flightdescription_solrsummary_t
and displays it in a tabular format, as shown here:flightdescription.origin_s
field of the Solr document that contains flightdescription_solrsummary_t
field.We'll come back to these fields and look at how things work behind the scenes in the How it works… section.
The integration between Solr search platform and Roo-generated domain model is achieved by:
Let's now look at how Roo simplifies Solr integration.
The solr
setup
command configures Solr for the Roo project. When solr
setup
command is executed, Roo takes the following actions:
pom.xml
file. SolrJ is used by JPA entities to add entity data to Solr index and for querying the Solr search server.@Async
annotated methods in the project by adding <annotation-driven>
element of Spring's task
namespace in applicationContext.xml
file, as shown here:<task:annotation-driven executor="asyncExecutor" mode="aspectj" />
The executor
attribute refers to an implementation of the java.util.concurrent.Executor
interface, responsible for executing the @Async
annotated method.
ThreadPoolTaskExecutor
in applicationContext.xml
using <executor>
element of Spring's task
namespace, as shown here:<task:executor id="asyncExecutor" pool-size="${executor.poolSize}" />
Spring's ThreadPoolTaskExecutor
configures a java.util.concurrent.ThreadPoolExecutor
instance (an implementation of java.util.concurrent.Executor
) with the thread pool size specified by the pool-size
attribute value. The ${executor.poolSize}
placeholder's value comes from the solr.properties
file.
CommonsHttpSolrServer
instance (a subclass of SolrJ's SolrServer
abstract class) in the applicationContext.xml
file to allow JPA entities to interact with the Solr search server over HTTP protocol:<bean class="org.apache.solr.client.solrj. impl.CommonsHttpSolrServer" id="solrServer"> <constructor-arg value="${solr.serverUrl}"/> </bean>
Behind the scenes, CommonsHttpSolrServer
makes use of Apache Commons HttpClient
to interact with the Solr search server. The constructor of CommonsHttpSolrServer
accepts URL of the Solr search server as an argument. The <constructor-arg>
element specifies the value of the constructor argument as ${solr.serverUrl}
, which refers to the solr.serverUrl
property defined in solr.properties
file.
solr.properties
file in the SRC_MAIN_RESOURCESMETA-INFspring
directory. The properties file defines an executor.poolSize
property, which specifies the thread pool size required by ThreadPoolExecutor
, as shown here:executor.poolSize=10
The solr.properties
file also contains a solr.serverUrl
property, which identifies the URL where the Solr search server is running, as shown here:
solr.serverUrl=http://localhost:8983/solr
If your Solr server is running on a different host or port, then change the URL in the solr.properties
file or use the searchServerUrl
argument of the solr
setup
command to specify the Solr search server URL.
Imagine that you want to search for FlightDescription
instances where the origin
field is NYC
. You can perform this search against the database in which you persist your FlightDescription
entity instances or you can add the FlightDescription
instance data into Solr index and search against it. We'll look at how Roo supports adding entity instance data to Solr index, and in the next section, we'll look at how to query that data in Solr search server.
Though there are multiple ways in which you can push data into Solr, Roo makes use of the SolrJ client library to interact with the Solr search server. When the solr
all
Roo command is executed, it adds certain methods (via AspectJ ITD) to JPA entity classes that are fired when an entity is added, removed, or updated. These methods are responsible for adding, updating, and deleting entity data from Solr index using SolrJ client library.
When solr
all
command is executed, the following actions are performed by Roo:
@RooSolrSearchable
annotation to JPA entity class that triggers creation of the corresponding *_Roo_SolrSearch.aj
AspectJ ITD file.The following code listing shows the FlightDescription
JPA entity of flightapp-web
project after solr
all
command was executed:
@RooEntity(identifierColumn = "FLIGHT_DESC_ID", table = "FLIGHT_DESC_TBL", finders = { "findFlightDescriptionsByDestinationAndOrigin" })
@RooSolrSearchable
public class FlightDescription {
...
}
The given code shows that @RooSolrSearchable
annotation is added to FlightDescription
entity. If you look at the Flight
entity, you'll find that the @RooSolrSearchable
annotation is also added to it.
*_Roo_SolrSearch.aj
AspectJ ITD file (corresponding to each JPA entity in the project. *_Roo_SolrSearch.aj
) that introduces methods into JPA entity class for adding, updating, and removing entity from Solr index. Also, *_Roo_SolrSearch.aj
defines methods for querying the Solr server using SolrJ client library.Let's now look at methods and attributes introduced by the FlightDescription_Roo_SolrSearch.aj
file:
solrServer
attribute that refers to the CommonsHttpSolrServer
bean configured in applicationContext.xml
file is shown as follows:@Autowired transient SolrServer FlightDescription.solrServer;
solrServer()
: A static method that returns the solrServer
attribute introduced by the ITD file is shown as follows:public static final SolrServer FlightDescription.solrServer() { SolrServer _solrServer = new FlightDescription().solrServer; .. return _solrServer; }
indexFlightDescriptions
: A static method that adds a collection of FlightDescription
entity instances to the Solr index is shown as follows:import org.springframework.scheduling.annotation.Async; ... ... @Async public static void FlightDescription.indexFlightDescriptions (Collection<FlightDescription> flightdescriptions) { java.util.List<SolrInputDocument> documents = new java.util.ArrayList<SolrInputDocument>(); for (FlightDescription flightdescription : flightdescriptions) { SolrInputDocument sid = new SolrInputDocument(); sid.addField("id", "flightdescription_" + flightdescription.getId()); sid.addField("flightdescription.id_l", flightdescription.getId()); sid.addField("flightdescription.origin_s", flightdescription.getOrigin()); ... sid.addField("flightdescription.price_f", flightdescription.getPrice()); sid.addField("flightdescription_solrsummary_t", ...); documents.add(sid); } try { SolrServer solrServer = solrServer(); solrServer.add(documents); solrServer.commit(); } catch (Exception e) { e.printStackTrace(); } }
The given code shows that the indexFlightDescriptions
method is annotated with Spring's @Async
annotation, which means that it is invoked asynchronously. The method iterates over all the FlightDescription
instances (passed as method argument) and creates a list of SolrInputDocument
. The SolrJ's SolrInputDocument
class represents a document that you want to feed to Solr server for indexing. The addField
method of SolrInputDocument
identifies the field that you want to add to the document.
The field name that is added by Roo to the SolrInputDocument
has the following naming convention:
<entity-simple-name>.<field-name>_<field-type>
Here, entity-simple-name
is the simple name of JPA entity, field-name
is the name of the field, and field-type
is the type of the field. So, the orgin
field is added to SolrInputDocument
with the name flightdescription.origin_s
and price
field is added with the name flightdescription.price_f
.
If the JPA entity field type isn't Integer
, String
, Long
, Boolean
, Float
, Double
, or Date
, then the field name with which the JPA entity field is added to SolrInputDocument
is shown as follows:
<entity-simple-name>.<field-name>_t
For instance, the Flight
class in flightapp-web
project contains the flightDescription
relationship field of type FlightDescription
, which is added to SolrInputDocument
with name flight.flightdescription_t
(refer to the Flight_Roo_SolrSearch.aj
AspectJ ITD file).
You might be wondering, why Roo doesn't add JPA entity fields with their exact name in the SolrInputDocument
. Here is a short description of how Solr works:
SolrInputDocument
represents a document that you add to Solr search server. The document consists of fields and you need to tell Solr search server, which of these fields should be indexed. It is important to note that if a field is not indexed, then you can't search or sort documents based on that field. You tell the Solr search server, which fields of a document should be indexed by specifying the fields in schema.xml
file located in SOLR_HOMEexamplesolrconf
directory. Solr has the concept of Dynamic Fields, wherein if a field follows a standard naming convention, then it is automatically indexed by Solr search server. The following XML fragment from the schema.xml
file defines the dynamic fields that will be automatically indexed by Solr:
<dynamicField name="*_s" type="string" indexed="true" stored="true"/> <dynamicField name="*_l" type="slong" indexed="true" stored="true"/> <dynamicField name="*_t" type="text" indexed="true" stored="true"/> <dynamicField name="*_f" type="sfloat" indexed="true" stored="true"/>
The given XML fragment instructs Solr to index any field that matches the pattern *_s
, *_l
, *_t
, or *_f
. So, now you can see the link between Roo generated field names and the dynamic fields defined by Solr.
The indexFlightDescriptions
method also adds an id
field name to the SolrInputDocument
. It is mandatory for any SolrInputDocument
to contain a field named id
, which uniquely identifies the document in Solr index. By default, Roo sets the value of id
field to "flightdescription_"
+
flightdescription.getId()
. We'll see later in this section that this id
field value is used for deleting the document from Solr index.
The indexFlightDescriptions
method also adds an extra field, flightdescription_solrsummary_t
, in SolrInputDocument
so that it can be used to search all documents that have been indexed by Solr for FlightDescription
JPA entity. Similarly, the indexFlightDescriptions
method of Flight_Roo_SolrSearch.aj
AspectJ ITD adds flight_solrsummary_t
field in SolrInputDocument
to allow searching for documents indexed by Solr for the Flight
JPA entity.
The following code in the indexFlightDescriptions
method adds the SolrInputDocument
s to Solr index:
SolrServer solrServer = solrServer(); solrServer.add(documents); solrServer.commit();
indexFlightDescription
: A static method, which adds a FlightDescription
entity instance to Solr index, which is shown as follows:public static void FlightDescription.indexFlightDescription(FlightDescription flightdescription) { List<FlightDescription> flightdescriptions = new ArrayList<FlightDescription>(); flightdescriptions.add(flightdescription); indexFlightDescriptions(flightdescriptions); }
As the given code shows, indexFlightDescription
method delegates the responsibility of adding FlightDescription
instance to Solr index to indexFlightDescriptions
method.
deleteIndex
: A static method, which deletes a Solr document corresponding to a FlightDescription
JPA entity instance is shown as follows:@Async public static void FlightDescription.deleteIndex(FlightDescription flightdescription) { SolrServer solrServer = solrServer(); try { solrServer.deleteById("flightdescription_" + flightdescription.getId()); solrServer.commit(); } catch (Exception e) { e.printStackTrace(); } }
In the given code, the deleteById
method of SolrServer
deletes the document (from Solr index), which has the id
attribute value "flightdescription_"
+
flightdescription.getId()
. The Spring's @Async
annotation means that the deleteIndex
method is invoked asynchronously.
postPersistOrUpdate
method, which is invoked when the FlightDescription
JPA entity instance is persisted or updated in the database. This method is responsible for adding or updating the Solr index with the modified JPA entity instance data, as shown here:import javax.persistence.PostPersist; import javax.persistence.PostUpdate; ... ... @PostUpdate @PostPersist private void FlightDescription.postPersistOrUpdate() { indexFlightDescription(this); }
The
@PostUpdate
and @PostPersist
JPA annotations indicate that postPersistOrUpdate
method is invoked when FlightDescription
JPA entity is updated or persisted in the database. The call to indexFlightDescription
method suggests that the entity data is updated or added to the Solr index.
preRemove
method, which removes the entity data from Solr index by calling the deleteIndex
method:import javax.persistence.PreRemove; ... ... @PreRemove private void FlightDescription.preRemove() { deleteIndex(this); }
The @PreRemove
JPA annotation means that the preRemove
method is invoked before the JPA entity instance is removed from the database.
search(SolrQuery
query)
method, which allows searching Solr documents that match the search query:public static QueryResponse FlightDescription.search(SolrQuery query) { try { return solrServer().query(query); } catch (Exception e) { e.printStackTrace(); } return new QueryResponse(); }
SolrQuery
represents a query object, which contains the field information based on which the search has to be performed, the fields to return, and so on. The query
method of SolrServer
sends the search request to Solr search server using Apache Commons HttpClient
and returns a QueryResponse
object from which you can extract the Solr documents that matched the search query.
search(String)
method that only returns Solr document(s) corresponding to FlightDescription
entity in Solr search server:public static QueryResponse FlightDescription.search(String queryString) { String searchString = "FlightDescription_solrsummary_t:" + queryString; return search(new SolrQuery(searchString.toLowerCase())); }
In the given code, the SolrQuery
object is created using the searchString
. The searchString
specifies the Solr query used for finding matching Solr documents. As searchString
already contains the constant value "FlightDescription_solrsummary_t:"
, which means that you can only search for Solr documents that contain "FlightDescription_solrsummary_t"
field. If you remember from the earlier discussion, the "FlightDescription_solrsummary_t"
field is only available in Solr documents which have been added corresponding to the FlightDescription
entity.
Let's now look at how the FlightDescriptionSearchController
controller makes use of search
methods defined in the FlightDescription
JPA entity to search documents indexed by Solr search server.
FlightDescriptionSearchController
defines methods which search for Solr documents corresponding to the FlightDescription
entity. The following code listing shows FlightDescriptionSearchController
class:
@Controller public class FlightDescriptionSearchController { private List<List<MySolrField>> getAllFields() { QueryResponse response = FlightDescription.search("*"); SolrDocumentList documentList = response.getResults(); return getSolrDocumentFieldList(documentList); } private List<List<MySolrField>> getMatchingFields() { SolrQuery solrQuery = new SolrQuery(). setQuery("flightdescription_solrsummary_t:*"). setParam("fl", "flightdescription.origin_s"); QueryResponse response = FlightDescription.search(solrQuery); SolrDocumentList documentList = response.getResults(); return getSolrDocumentFieldList(documentList); } private List<List<MySolrField>> getSolrDocumentFieldList(SolrDocumentList list) { List<List<MySolrField>> matchingDocList = new ArrayList<List<MySolrField>>(); ... return matchingDocList; } }
The getAllFields
method invokes search(String
queryString)
method of FlightDescription
entity and passes *
as the method argument. As we saw earlier, the search(String
queryString)
method of FlightDescription
will create the following query: "FlightDescription_solrsummary_t:*"
, which means search for all Solr documents, which contain "FlightDescription_solrsummary_t"
field. This query will return all the Solr documents corresponding to FlightDescription
entity that we added to Solr index.
The getMatchingFields
method invokes the search(SolrQuery
query)
method passing the SolrQuery
object, which queries for all Solr documents corresponding to FlightDescription
JPA entity but specifies that the query result should only contain the flightdescription.origin_s
field. The setQuery
parameter of SolrQuery specifies the query and setParam
specifies that only flightdescription.origin_s
field should be returned in the result.
The getResults
method of the QueryResponse
object returns SolrDocumentList
representing the list of matching Solr documents returned by the query.
The getSolrDocumentFieldList
method takes SolrDocumentList
as the argument and extracts SolrDocument
instances from it. The method then extracts field names and their values from each SolrDocument
instance to create a List<List<MySolrField>>
. The MySolrField
represents a custom class that we created in flightapp-web
project to represent a single field-value pair in SolrDocument
.
The /WEB-INF/views/flightdescriptionsearch/index.jsp
JSP page displays data returned by getFields
and getMatchingFields
methods. This is the reason why selecting Flight Description Search Controller View menu option shows two different types of tables. One table type shows all the Solr document fields and the other table type only shows the flightdescription.origin_s
field.
Solr index is updated in @PreRemove
, @PostPersist
, and the @PostUpdate
annotated method. So, what if the transaction fails to commit but the entity data is stored as Solr document in Solr search server? You need to take care of maintaining the integrity yourself, because Roo doesn't help you there.
Let's now look at the attributes that @RooSolrSearchable
defines to customize names of Roo-generated methods in *_Roo_SolrSearch.aj
AspectJ ITD.
The following table describes the attributes of @RooSolrSearchable
annotation:
18.118.24.106