Adding search capability to your domain model with Sol

Apache Solr is an open-source search platform built on top of the Apache Lucene search engine library. Spring Roo's Solr add-on provides support for integrating the Roo-generated domain model with Solr platform. In this recipe, we'll look at how Roo makes use of SolrJ Java client library to add domain model data into Solr server for indexing and to search domain model data based on user supplied query parameters.

Getting ready

To see Roo's support for Solr in action, you need to download and run the Solr server, as described here:

  1. Download the Solr server version 1.4.0 ZIP file from Solr website and unzip the bundle into a directory. Let's call the unzipped directory as SOLR_HOME.
  2. Go to the SOLR_HOMEexample directory and start Solr server:
    C:...apache-solr-1.4.0example> java –jar start.jar
    
  3. Open the web browser and verify that Solr server has successfully started by going to the following URL: http://localhost:8983/solr/admin/

Now, create a sub-directory ch06-solr inside C: oo-cookbook directory, copy ch06_web_app.roo script and start Roo shell from C: oo-cookbookch06-solr.

How to do it...

Follow these step to add search capability:

  1. Execute the ch06_solr.roo script, as shown here:
    roo> script --file ch06_web_app.roo
    

    The script creates a flightapp-web Roo project consisting of Flight and FlightDescription JPA entities.

  2. Setup Solr for the flightapp-web project using the solr setup command:
    Updated ROOTpom.xml [Added dependency org.apache.solr:solr-solrj:1.4.0]
    Created SRC_MAIN_RESOURCESMETA-INFspringsolr.properties
    Updated SRC_MAIN_RESOURCESMETA-INFspringapplicationContext.xml
    
    .. roo> solr setup
    
  3. Make all JPA entities in the project searchable by executing the solr all command as shown here:
    .. roo> solr all
    
    Updated SRC_MAIN_JAVA...FlightDescription.java
    Updated SRC_MAIN_JAVA...Flight.java
    Created SRC_MAIN_JAVA..Flight_Roo_SolrSearch.aj
    Created SRC_MAIN_JAVA...FlightDescription_Roo_SolrSearch.aj
    
  4. Create a controller, which is responsible for searching Solr documents, as shown here:
    .. roo> controller class --class ~.web.FlightDescriptionSearchController --preferredMapping /flightdescriptionsearch
    
    Created SRC_MAIN_JAVA...FlightDescriptionSearchController
    .java
    Created SRC_MAIN_WEBAPPWEB-INFviewsflightdescriptionsearch
    Created SRC_MAIN_WEBAPPWEB-INFviewsflightdescriptionsearchindex.jspx
    

    Copy MySolrField.java and FlightDescriptionSearchController.java files from the source code that accompanies this chapter to sample.roo.flightapp.web package. Also, replace /WEB-INF/views/flightdescriptionsearch/index.jsp with the index.jsp file from the source code that accompanies this chapter.

  5. Execute the perform eclipse command so that you can import the flightapp-web project into your Eclipse IDE as shown here:
    .. roo> perform eclipse
    
  6. Exit the Roo shell and execute the tomcat:run goal of Maven Tomcat Plugin from ch06-solr directory to deploy the flightapp-web project in embedded Tomcat container as shown here:
    C:
    oo-cookbookch06-solr> mvn tomcat:run
    
  7. Open your web browser and go to http://localhost:8080/flightapp-web. If you see the following home page of the web application, it means that the flightapp-web project is successfully deployed on Tomcat:
    How to do it...

    In the given screenshot, Flight Description Search Controller View menu option sends request to FlightDescriptionSearchController class, which in turn renders the index.jsp page located in /WEB-INF/views/flightdescriptionsearch folder.

  8. At this time make sure that your Solr server is up and running. Now, select Create new Flight Description menu option to view the form for creating new FlightDescription entities in database, as shown here:
    How to do it...

    The given screenshot shows that you need to enter information about the following fields: Price, Origin and Destination. Create two FlightDescription instances with the information shown in the following table:

    Instance

    Price

    Origin

    Destination

    Instance-1

    1200

    NYC

    DELHI

    Instance-2

    1400

    MUMBAI

    ATLANTA

  9. When you create the FlightDescription entity instance, Roo's support for Solr adds the entity data into Solr for indexing and searching. Select the Flight Description Search Controller View menu option that searches the Solr server for documents that have a field named flightdescription_solrsummary_t and displays it in a tabular format, as shown here:
    How to do it...
  10. The given screenshot shows four tables: two of them are titled Solr Document All Fields and the remaining two are titled Solr Document Matching Fields. Solr Document All Fields titled tables show all the fields of the Solr document that contained the field flightdescription_solrsummary_t and Solr Document Matching Fields titled tables show only the flightdescription.origin_s field of the Solr document that contains flightdescription_solrsummary_t field.

    We'll come back to these fields and look at how things work behind the scenes in the How it works… section.

How it works...

The integration between Solr search platform and Roo-generated domain model is achieved by:

  • Configuring Solr for Roo project
  • Defining methods to add domain model data to Solr index
  • Defining methods for querying Solr search server

Let's now look at how Roo simplifies Solr integration.

Configuring Solr for Roo project

The solr setup command configures Solr for the Roo project. When solr setup command is executed, Roo takes the following actions:

  • Adds dependency of project on SolrJ 1.4.0 in pom.xml file. SolrJ is used by JPA entities to add entity data to Solr index and for querying the Solr search server.
  • Enables support for @Async annotated methods in the project by adding <annotation-driven> element of Spring's task namespace in applicationContext.xml file, as shown here:
    <task:annotation-driven executor="asyncExecutor"
          mode="aspectj" />

    The executor attribute refers to an implementation of the java.util.concurrent.Executor interface, responsible for executing the @Async annotated method.

  • Configures Spring's ThreadPoolTaskExecutor in applicationContext.xml using <executor> element of Spring's task namespace, as shown here:
    <task:executor id="asyncExecutor" 
        pool-size="${executor.poolSize}" />

    Spring's ThreadPoolTaskExecutor configures a java.util.concurrent.ThreadPoolExecutor instance (an implementation of java.util.concurrent.Executor) with the thread pool size specified by the pool-size attribute value. The ${executor.poolSize} placeholder's value comes from the solr.properties file.

  • Configures SolrJ's CommonsHttpSolrServer instance (a subclass of SolrJ's SolrServer abstract class) in the applicationContext.xml file to allow JPA entities to interact with the Solr search server over HTTP protocol:
     <bean class="org.apache.solr.client.solrj.
       impl.CommonsHttpSolrServer" id="solrServer">
        <constructor-arg value="${solr.serverUrl}"/>
     </bean>

    Behind the scenes, CommonsHttpSolrServer makes use of Apache Commons HttpClient to interact with the Solr search server. The constructor of CommonsHttpSolrServer accepts URL of the Solr search server as an argument. The <constructor-arg> element specifies the value of the constructor argument as ${solr.serverUrl}, which refers to the solr.serverUrl property defined in solr.properties file.

  • Creates a solr.properties file in the SRC_MAIN_RESOURCESMETA-INFspring directory. The properties file defines an executor.poolSize property, which specifies the thread pool size required by ThreadPoolExecutor, as shown here:
    executor.poolSize=10

    The solr.properties file also contains a solr.serverUrl property, which identifies the URL where the Solr search server is running, as shown here:

    solr.serverUrl=http://localhost:8983/solr

If your Solr server is running on a different host or port, then change the URL in the solr.properties file or use the searchServerUrl argument of the solr setup command to specify the Solr search server URL.

Adding domain model data to Solr index and searching Solr documents

Imagine that you want to search for FlightDescription instances where the origin field is NYC. You can perform this search against the database in which you persist your FlightDescription entity instances or you can add the FlightDescription instance data into Solr index and search against it. We'll look at how Roo supports adding entity instance data to Solr index, and in the next section, we'll look at how to query that data in Solr search server.

Though there are multiple ways in which you can push data into Solr, Roo makes use of the SolrJ client library to interact with the Solr search server. When the solr all Roo command is executed, it adds certain methods (via AspectJ ITD) to JPA entity classes that are fired when an entity is added, removed, or updated. These methods are responsible for adding, updating, and deleting entity data from Solr index using SolrJ client library.

When solr all command is executed, the following actions are performed by Roo:

  • Adds the @RooSolrSearchable annotation to JPA entity class that triggers creation of the corresponding *_Roo_SolrSearch.aj AspectJ ITD file.

    The following code listing shows the FlightDescription JPA entity of flightapp-web project after solr all command was executed:

    @RooEntity(identifierColumn = "FLIGHT_DESC_ID", table = "FLIGHT_DESC_TBL", finders = { "findFlightDescriptionsByDestinationAndOrigin" })
    @RooSolrSearchable
    public class FlightDescription {
     ...
    }

    The given code shows that @RooSolrSearchable annotation is added to FlightDescription entity. If you look at the Flight entity, you'll find that the @RooSolrSearchable annotation is also added to it.

  • Creates a *_Roo_SolrSearch.aj AspectJ ITD file (corresponding to each JPA entity in the project. *_Roo_SolrSearch.aj) that introduces methods into JPA entity class for adding, updating, and removing entity from Solr index. Also, *_Roo_SolrSearch.aj defines methods for querying the Solr server using SolrJ client library.

Methods and attributes introduced by *_Roo_SolrSearch.aj AspectJ ITD

Let's now look at methods and attributes introduced by the FlightDescription_Roo_SolrSearch.aj file:

  • solrServer attribute that refers to the CommonsHttpSolrServer bean configured in applicationContext.xml file is shown as follows:
    @Autowired
    transient SolrServer FlightDescription.solrServer;
    
  • solrServer(): A static method that returns the solrServer attribute introduced by the ITD file is shown as follows:
    public static final SolrServer FlightDescription.solrServer() {
      SolrServer _solrServer = new FlightDescription().solrServer;
      ..
      return _solrServer;
    }
  • indexFlightDescriptions: A static method that adds a collection of FlightDescription entity instances to the Solr index is shown as follows:
    import org.springframework.scheduling.annotation.Async; 
    ...
    ...
    @Async
    public static void 
        FlightDescription.indexFlightDescriptions
         (Collection<FlightDescription> flightdescriptions) {
    
       java.util.List<SolrInputDocument> documents = 
            new java.util.ArrayList<SolrInputDocument>();
       
       for (FlightDescription flightdescription : 
             flightdescriptions) {
         
         SolrInputDocument sid = new SolrInputDocument();
         sid.addField("id", "flightdescription_" + 
             flightdescription.getId());
         sid.addField("flightdescription.id_l", 
             flightdescription.getId());
         sid.addField("flightdescription.origin_s", 
             flightdescription.getOrigin());
         ...
         sid.addField("flightdescription.price_f", 
             flightdescription.getPrice());
         sid.addField("flightdescription_solrsummary_t", ...);
         documents.add(sid);
       }
       try {
         SolrServer solrServer = solrServer();
         solrServer.add(documents);
         solrServer.commit();
       } catch (Exception e) {
          e.printStackTrace();
       }
    }

    The given code shows that the indexFlightDescriptions method is annotated with Spring's @Async annotation, which means that it is invoked asynchronously. The method iterates over all the FlightDescription instances (passed as method argument) and creates a list of SolrInputDocument. The SolrJ's SolrInputDocument class represents a document that you want to feed to Solr server for indexing. The addField method of SolrInputDocument identifies the field that you want to add to the document.

    The field name that is added by Roo to the SolrInputDocument has the following naming convention:

    <entity-simple-name>.<field-name>_<field-type>
    

    Here, entity-simple-name is the simple name of JPA entity, field-name is the name of the field, and field-type is the type of the field. So, the orgin field is added to SolrInputDocument with the name flightdescription.origin_s and price field is added with the name flightdescription.price_f.

    If the JPA entity field type isn't Integer, String, Long, Boolean, Float, Double, or Date, then the field name with which the JPA entity field is added to SolrInputDocument is shown as follows:

    <entity-simple-name>.<field-name>_t
    

    For instance, the Flight class in flightapp-web project contains the flightDescription relationship field of type FlightDescription, which is added to SolrInputDocument with name flight.flightdescription_t (refer to the Flight_Roo_SolrSearch.aj AspectJ ITD file).

    You might be wondering, why Roo doesn't add JPA entity fields with their exact name in the SolrInputDocument. Here is a short description of how Solr works:

    SolrInputDocument represents a document that you add to Solr search server. The document consists of fields and you need to tell Solr search server, which of these fields should be indexed. It is important to note that if a field is not indexed, then you can't search or sort documents based on that field. You tell the Solr search server, which fields of a document should be indexed by specifying the fields in schema.xml file located in SOLR_HOMEexamplesolrconf directory. Solr has the concept of Dynamic Fields, wherein if a field follows a standard naming convention, then it is automatically indexed by Solr search server. The following XML fragment from the schema.xml file defines the dynamic fields that will be automatically indexed by Solr:

    <dynamicField name="*_s"  type="string"  indexed="true"  
      stored="true"/>
    <dynamicField name="*_l"  type="slong"   indexed="true"  
      stored="true"/>
    <dynamicField name="*_t"  type="text"    indexed="true"  
      stored="true"/>
    <dynamicField name="*_f"  type="sfloat"  indexed="true"  
      stored="true"/>

    The given XML fragment instructs Solr to index any field that matches the pattern *_s, *_l, *_t, or *_f. So, now you can see the link between Roo generated field names and the dynamic fields defined by Solr.

    The indexFlightDescriptions method also adds an id field name to the SolrInputDocument. It is mandatory for any SolrInputDocument to contain a field named id, which uniquely identifies the document in Solr index. By default, Roo sets the value of id field to "flightdescription_" + flightdescription.getId(). We'll see later in this section that this id field value is used for deleting the document from Solr index.

    The indexFlightDescriptions method also adds an extra field, flightdescription_solrsummary_t, in SolrInputDocument so that it can be used to search all documents that have been indexed by Solr for FlightDescription JPA entity. Similarly, the indexFlightDescriptions method of Flight_Roo_SolrSearch.aj AspectJ ITD adds flight_solrsummary_t field in SolrInputDocument to allow searching for documents indexed by Solr for the Flight JPA entity.

    The following code in the indexFlightDescriptions method adds the SolrInputDocuments to Solr index:

    SolrServer solrServer = solrServer();
    solrServer.add(documents);
    solrServer.commit();
  • indexFlightDescription: A static method, which adds a FlightDescription entity instance to Solr index, which is shown as follows:
    public static void 
      FlightDescription.indexFlightDescription(FlightDescription 
        flightdescription) {
        List<FlightDescription> flightdescriptions = 
          new ArrayList<FlightDescription>();
        flightdescriptions.add(flightdescription);
        indexFlightDescriptions(flightdescriptions);
    }

    As the given code shows, indexFlightDescription method delegates the responsibility of adding FlightDescription instance to Solr index to indexFlightDescriptions method.

  • deleteIndex: A static method, which deletes a Solr document corresponding to a FlightDescription JPA entity instance is shown as follows:
    @Async
    public static void 
       FlightDescription.deleteIndex(FlightDescription 
         flightdescription) {
       SolrServer solrServer = solrServer();
       try {
         solrServer.deleteById("flightdescription_" + 
           flightdescription.getId());
         solrServer.commit();
       } catch (Exception e) {
         e.printStackTrace();
       }
    }

    In the given code, the deleteById method of SolrServer deletes the document (from Solr index), which has the id attribute value "flightdescription_" + flightdescription.getId(). The Spring's @Async annotation means that the deleteIndex method is invoked asynchronously.

  • postPersistOrUpdate method, which is invoked when the FlightDescription JPA entity instance is persisted or updated in the database. This method is responsible for adding or updating the Solr index with the modified JPA entity instance data, as shown here:
    import javax.persistence.PostPersist;
    import javax.persistence.PostUpdate;
    ...
    ...
    @PostUpdate
    @PostPersist
    private void FlightDescription.postPersistOrUpdate() {
        indexFlightDescription(this);
    }

    The @PostUpdate and @PostPersist JPA annotations indicate that postPersistOrUpdate method is invoked when FlightDescription JPA entity is updated or persisted in the database. The call to indexFlightDescription method suggests that the entity data is updated or added to the Solr index.

  • preRemove method, which removes the entity data from Solr index by calling the deleteIndex method:
    import javax.persistence.PreRemove;
    ...
    ...    
    @PreRemove
    private void FlightDescription.preRemove() {
      deleteIndex(this);
    }

    The @PreRemove JPA annotation means that the preRemove method is invoked before the JPA entity instance is removed from the database.

  • search(SolrQuery query) method, which allows searching Solr documents that match the search query:
    public static QueryResponse FlightDescription.search(SolrQuery query) {
        try {
            return solrServer().query(query);
        } catch (Exception e) {
            e.printStackTrace();
        }
        return new QueryResponse();
    }

    SolrQuery represents a query object, which contains the field information based on which the search has to be performed, the fields to return, and so on. The query method of SolrServer sends the search request to Solr search server using Apache Commons HttpClient and returns a QueryResponse object from which you can extract the Solr documents that matched the search query.

  • search(String) method that only returns Solr document(s) corresponding to FlightDescription entity in Solr search server:
    public static QueryResponse 
       FlightDescription.search(String queryString) {
        String searchString = 
          "FlightDescription_solrsummary_t:" + queryString;
        return search(new SolrQuery(searchString.toLowerCase()));
    }

    In the given code, the SolrQuery object is created using the searchString. The searchString specifies the Solr query used for finding matching Solr documents. As searchString already contains the constant value "FlightDescription_solrsummary_t:", which means that you can only search for Solr documents that contain "FlightDescription_solrsummary_t" field. If you remember from the earlier discussion, the "FlightDescription_solrsummary_t" field is only available in Solr documents which have been added corresponding to the FlightDescription entity.

Let's now look at how the FlightDescriptionSearchController controller makes use of search methods defined in the FlightDescription JPA entity to search documents indexed by Solr search server.

Searching Solr documents

FlightDescriptionSearchController defines methods which search for Solr documents corresponding to the FlightDescription entity. The following code listing shows FlightDescriptionSearchController class:

@Controller
public class FlightDescriptionSearchController {
    
  private List<List<MySolrField>> getAllFields() {
    QueryResponse response = FlightDescription.search("*");
    SolrDocumentList documentList = response.getResults();
    return getSolrDocumentFieldList(documentList);
  }
    
  private List<List<MySolrField>> getMatchingFields() {
    SolrQuery solrQuery = new SolrQuery().
      setQuery("flightdescription_solrsummary_t:*").
      setParam("fl", "flightdescription.origin_s");
    QueryResponse response = 
      FlightDescription.search(solrQuery);
      SolrDocumentList documentList = response.getResults();
      return getSolrDocumentFieldList(documentList);
  }

  private List<List<MySolrField>> 
    getSolrDocumentFieldList(SolrDocumentList list) {
      List<List<MySolrField>> matchingDocList 
        = new  ArrayList<List<MySolrField>>();
      ...    	
      return matchingDocList;
    }
}

The getAllFields method invokes search(String queryString) method of FlightDescription entity and passes * as the method argument. As we saw earlier, the search(String queryString) method of FlightDescription will create the following query: "FlightDescription_solrsummary_t:*", which means search for all Solr documents, which contain "FlightDescription_solrsummary_t" field. This query will return all the Solr documents corresponding to FlightDescription entity that we added to Solr index.

The getMatchingFields method invokes the search(SolrQuery query) method passing the SolrQuery object, which queries for all Solr documents corresponding to FlightDescription JPA entity but specifies that the query result should only contain the flightdescription.origin_s field. The setQuery parameter of SolrQuery specifies the query and setParam specifies that only flightdescription.origin_s field should be returned in the result.

The getResults method of the QueryResponse object returns SolrDocumentList representing the list of matching Solr documents returned by the query.

The getSolrDocumentFieldList method takes SolrDocumentList as the argument and extracts SolrDocument instances from it. The method then extracts field names and their values from each SolrDocument instance to create a List<List<MySolrField>>. The MySolrField represents a custom class that we created in flightapp-web project to represent a single field-value pair in SolrDocument.

The /WEB-INF/views/flightdescriptionsearch/index.jsp JSP page displays data returned by getFields and getMatchingFields methods. This is the reason why selecting Flight Description Search Controller View menu option shows two different types of tables. One table type shows all the Solr document fields and the other table type only shows the flightdescription.origin_s field.

There's more...

Solr index is updated in @PreRemove, @PostPersist, and the @PostUpdate annotated method. So, what if the transaction fails to commit but the entity data is stored as Solr document in Solr search server? You need to take care of maintaining the integrity yourself, because Roo doesn't help you there.

Let's now look at the attributes that @RooSolrSearchable defines to customize names of Roo-generated methods in *_Roo_SolrSearch.aj AspectJ ITD.

Customizing Roo-generated *_Roo_SolrSearch.aj AspectJ ITD

The following table describes the attributes of @RooSolrSearchable annotation:

Attribute

Description

deleteIndexMethod

Specifies a custom name for the deleteIndex method. Value blank " " instructs Roo not to generate deleteIndex method.

indexMethod

Specifies a custom name for the index methods. Value blank " " instructs Roo not to generate index methods.

postPersistOrUpdateMethod

Specifies a custom name for the postPersistOrUpdate methods. Value blank " " instructs Roo not to generate postPersistOrUpdate methods.

preRemoveMethod

Specifies a custom name for the preRemove method. Value blank " " instructs Roo not to generate preRemove method.

searchMethod

Specifies a custom name for the search method, which accepts SolrQuery as argument. Value blank " " instructs Roo not to generate search method, which accepts SolrQuery argument.

simpleSearchMethod

Specifies a custom name for the search method, which accepts String as argument. Value blank " " instructs Roo not to generate search method, which accepts String argument.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.118.24.106