FQL can be used with the Query Object Model and the Query Web Service to execute complex search queries. This may be desirable when KQL does not offer sufficient flexibility. A good example is when you request result sets that have custom ranking sequences. These sequences can be set by boosting term or property values in the query. These boost values are usually context-aware and hidden from the end user. This way, users do not realize their specific search would rank differently; they simply experience a more relevant result.

Although SharePoint search Web Parts do not support FQL, they can be modified to do so, and you can create new Web Parts that can use FQL. FQL is the native language for querying the FAST index and has several capabilities beyond KQL:

FQL takes the query and splits it into operators and query strings. It isolates query strings rather than individual terms. These strings, whether text or some other value, are tokenized and special characters are removed. This is because not all characters are eligible to be tokenized and searched for.

The identified operator takes a set of values enclosed in parentheses and can be limited by a scope specification. For example, the query title:string(“Swedish meatballs”, mode=“and”) passes a Boolean AND for Swedish meatballs in the title of the items. The specification title: limits the scope of the query to the title of the items in the search index; the string operator takes the values in parentheses and passes them with the limitation. In this case, Swedish AND meatballs is passed by the string operator. Simpler queries can be passed (such as title:“Swedish meatballs”), but you should be as specific as possible to avoid any reserved terms in the query language.

FQL can support nested sub-expressions and queries that consist of search tokens (keywords), properties, and operators. As in KQL, queries are limited to 2,048 characters, and queries are not case sensitive.

xrank(xrank("meatballs","Swedish",boost=200),"Norwegian",boost=500, boostall=yes)

Table 8-4 lists the available operators in FQL and gives a brief explanation of their usage and syntax, and provides some examples. More information and examples are available from Microsoft at http://msdn.microsoft.com/en-us/library/ff394462.aspx.

The search Web Parts included in SharePoint use the SharedQueryManager class to get a reference to the current page’s QueryManager. All search Web Parts displaying information from the same result set share this QueryManager.

When you use QueryManager to search, results are returned as XML. The returned XML is formatted to work with the XSLT transformation included in the search Web Parts that turn the XML into the HTML displayed to the end user.

An example of two Web Parts sharing a QueryManager is the core results Web Part and the refinement Web Part. During search execution, they each register which parts of the search result they want returned and only one query is sent over to FS4SP. This returns both refiner results and the search hits at the same time. The refiners are retrieved via the RefinementManager class, which is the refiner equivalent to the SharedQueryManager. RefinementManager in turn uses the QueryManager internally.

The QueryManager is implemented as a List<LocationList>, as shown in Figure 8-3, and has very few options that you can set. These options are described in Table 8-8.

The LocationList class contains a list of federated locations and is implemented as a List<Location>. As you move down the class hierarchy, you can see that the class options get more specific toward the search being executed, as shown in Table 8-9.

The Location class represents a federated location. A federated location is any search location listed under Federated Locations on your FAST Query SSA in SharePoint Central Administration. For FS4SP, the Location is named Local FAST Search Results, or FASTSearch, which is the internal name you have to use when referring to it, as shown in Table 8-10.

The Location class has 21 properties that work with FS4SP; these classes are listed in Table 8-11. The other properties available are SharePoint Search specific.

The FASTSearchRuntime is what ties together the Location definition with the KeywordQuery class and runs the query via the FAST Query SSA over to the FS4SP farm, as shown in Figure 8-3. Together with the KeywordQuery class, the FASTSearchRuntime class is the only class that you can use to enable FQR for a search query and specify how you want the results sorted. In addition, several properties control how the result should be performed and formatted, and there are properties for controlling the refiners. All available properties are listed in Table 8-12.

When using the QueryManager class to execute a search query, you have to use both LocationList and Location. The following sample illustrates the use of the SharedQueryManager to execute two different queries against FS4SP inside of a Web Part on a SharePoint page.

// Get a reference to the Query SSA Proxy
var searchProxy = (SearchServiceApplicationProxy) SearchServiceApplicationProxy.
GetProxy(SPServiceContext.Current);

// Get a reference to the SharePoint page's query manager
var qm = SharedQueryManager.GetInstance(Page).QueryManager;

// Create a new LocationList object
var locationList = new LocationList();

// Create a Location object for query #1 and add to the list
var fastLocationOne = new Location("FASTSearch", searchProxy);
fastLocationOne.UserQuery = "FAST Search for SharePoint";
fastLocationOne.ItemsPerPage = 2;
locationList.Add(fastLocationOne);

// Create a Location object for query #2 and add to the list
var fastLocationTwo = new Location("FASTSearch", searchProxy);
fastLocationTwo.UserQuery = "Microsoft";
fastLocationTwo.ItemsPerPage = 2;
locationList.Add(fastLocationTwo);

// Add and trigger the federated locations
qm.Add(locationList);
qm.IsTriggered(locationList);
// Execute the query and get merged XML
XmlDocument resultXml = qm.GetResults();
// Number of results returned - 4 in total (2+2)
int returnedResults = locationList.ReturnedResults;
// Total number of results aggregated for both queries
int totalResults = locationList.TotalResults;

The following example shows the XML output returned from the previous code. The sample has been shortened for readability. The interesting part is that the XML for each location is merely appended after each other. First are two results for the query FAST Search for SharePoint with id 1 and 2, followed by the number of results and total results for that query. Next are the two results from the query Microsoft, also with id 1 and 2, and the number of results and total results. Appending result sets in this manner makes parsing the XML more complicated. The use of GetResults() from the QueryManager might not be the best way to get the result XML when you query more than one federated location at a time.

<All_Results>
  <Result>
    <id>1</id>
    <rank>18890</rank>
    <title>FS4SP Book Project</title>
    <size>3793511</size>
    <path>http://test/Shared Documents/FS4SP book Project.pptx</path>
    <write>5/16/2011</write>
    <contentclass>STS_ListItem_DocumentLibrary</contentclass>
    <isdocument>True</isdocument>
    <url>http://test/Shared Documents/FS4SP book Project.pptx</url>
    <fileextension>PPTX</fileextension>
  </Result>
  <Result>
    <id>2</id>
    <rank>18862</rank>
    <title>FS4SP Workshops</title>
    <size>2092916</size>
    <path>http://test/Shared Documents/FS4SP presentation.pptx</path>
    <write>5/27/2011</write>
    <contentclass>STS_ListItem_DocumentLibrary</contentclass>
    <isdocument>True</isdocument>
    <url>http://test/Shared Documents/FS4SP presentation.pptx</url>
    <fileextension>PPTX</fileextension>
  </Result>
  <TotalResults>3</TotalResults>
  <NumberOfResults>2</NumberOfResults>
  <Result>
    <id>1</id>
    <rank>1921</rank>
    <title>Microsoft Partner Program</title>
    <size>281600</size>
    <path>http://test/Shared Documents/Platform-Test-Framework-1.0.doc</path>
    <write>11/30/2004</write>
    <contentclass>STS_ListItem_DocumentLibrary</contentclass>
    <isdocument>True</isdocument>
    <url>http://test/Shared Documents/Platform-Test-Framework-1.0.doc</url>
    <fileextension>DOC</fileextension>
  </Result>
  <Result>
    <id>2</id>
    <rank>1582</rank>
    <title>Processing components.docx</title>
    <size>239493</size>
    <path>http://test/Shared Documents/Processing components.docx</path>
    <write>6/23/2011</write>
    <contentclass>STS_ListItem_DocumentLibrary</contentclass>
    <isdocument>True</isdocument>
    <url>http://test/Shared Documents/Processing components.docx</url>
    <fileextension>DOCX</fileextension>
  </Result>
  <TotalResults>18</TotalResults>
  <NumberOfResults>2</NumberOfResults>
</All_Results>

In order to avoid the appending of XML, you can remove the following line.

XmlDocument resultXml = qm.GetResults(locationList);

Replace it with the following two lines.

XmlDocument locationOneXml = fastLocationOne.GetResults(qm);
XmlDocument locationTwoXml = fastLocationTwo.GetResults(qm);

This way, you get one XML document per result set, which is easier to work with.

The previous code sample has duplicate trimming turned on by default, but both TotalResults and TotalResultsIncludingDuplicates return the same total even if there are duplicates being removed from the result set.

In order to get the exact number of results when TrimDuplicates is enabled, you have to enable refiners for your search query. Because duplicate trimming is applied only to search results and not to refiner results, the total count of a refiner can be higher than that of the search result if there are duplicates within the result set.

Internally, the TotalResultsIncludingDuplicates property is assigned from the refiner values if there is a refiner with a higher value than that of the TotalResults property. The refiner property has to cover all returned items in order to get the correct count. Examples of managed properties that you can use are write and size, which both contain values for all items. When using the default configuration of the RefinementManager, write is included.

If you modify the code sample shown in the “Executing a Query by Using QueryManager, LocationList, and Location” section earlier in this chapter with refiner settings added, you get the correct count in TotalResultsIncludingDuplicates. Look at the code that follows.

var searchProxy = (SearchServiceApplicationProxy)SearchServiceApplicationProxy.
GetProxy(SPServiceContext.Current);
var qm = SharedQueryManager.GetInstance(Page).QueryManager;
var locationList = new LocationList();
var fastLocationOne = new Location("FASTSearch", searchProxy);
fastLocationOne.UserQuery = "FAST Search for SharePoint";
fastLocationOne.ItemsPerPage = 2;
locationList.Add(fastLocationOne);

qm.Add(locationList);
qm.IsTriggered(locationList);

// Get a reference to the SharePoint page's refinement manager
RefinementManager refinementManager = RefinementManager.GetInstance(this.Page);
// Use the default refiner configuration for FAST
refinementManager.UseDefaultConfiguration = true;

// Initialize the use of refiners on the query
refinementManager.Initialize(fastLocationOne);

XmlDocument resultXml = qm.GetResults();
int returnedResults = locationList.ReturnedResults;
int totalResults = locationList.TotalResults;
int totalResultsIncludingDuplicates = locationList.TotalResultsIncludingDuplicates;

RefinementManager refinementManager = RefinementManager.GetInstance(this.Page);
XmlDocument refinementXml = refinementManager.GetRefinementXml();

Querying with FASTSearchRuntime requires an instance of a Location object that points to FASTSearch and a reference to the Query SSA Proxy. The following code sample executes a search for microsoft by using FQL and sorts the results with the newest items on top using FASTSearchRuntime.

// Get a reference to the Query SSA Proxy
var searchProxy = (SearchServiceApplicationProxy)SearchServiceApplicationProxy.
GetProxy(SPServiceContext.Current);
FASTSearchRuntime runtime = new FASTSearchRuntime(searchProxy);
// Create a Location object for FASTSearch
runtime.Location = new Location("FASTSearch", searchProxy);
// Enable queries written in FQL
runtime.EnableFQL = true;
// Set sort order
runtime.SortList.Add("write", SortDirection.Descending);
// Execute query
XmlDocument resultXml = runtime.SendRequest("string("microsoft")");

When developing custom search applications that are not based around the search Web Parts included in SharePoint, the KeywordQuery class offers the most flexibility compared to the APIs in the Federated Search Object Model in terms of the parameters available when specifying a search query, as listed in Table 8-13. When using KeywordQuery, the search results are returned as a ResultTableCollection and not as XML like the Federated Search Object Model APIs.

Many properties are available with KeywordQuery, and it takes time to get to know them all. The following sample queries for FAST Search for SharePoint sorts the results with the newest items on top, and, if the dates are equal, sorts on rank. The sample includes best bets and visual best bets and includes refiners for last modified date and the format of the items. The sample also shows a second query, refining on Microsoft PowerPoint files, where the refiner is picked from the first query.

For the third query, you execute a Find Similar query by using the first result hit as the source for the query.

// Get a reference to the Query SSA Proxy
var searchProxy = (SearchServiceApplicationProxy)SearchServiceApplicationProxy.
GetProxy(SPServiceContext.Current);
KeywordQuery kq = new KeywordQuery(searchProxy);
kq.QueryText = "FAST Search for SharePoint";
kq.ResultsProvider = SearchProvider.FASTSearch;

// Return search results, refiners and best bets
kq.ResultTypes = ResultType.RelevantResults | ResultType.RefinementResults | ResultType.
SpecialTermResults;

// Add refiner for last modified date and file format
// "write" is needed for exact TotalRowsIncludingDuplicates count
kq.Refiners = "write,format";
// Set sorting to newest items on top
kq.SortList.Add("write", SortDirection.Descending);
// If dates are the same, sort on rank
kq.SortList.Add("default", SortDirection.Descending);

// Set the user context group in order to get Synonyms, Promotions,
// Best Bets and Visual Best Bets
kq.UserContextGroupID = SPContext.Current.Site.ID.ToString();

// Execute query
ResultTableCollection resultTableCollection = kq.Execute();

// Get a reference to the search results
ResultTable resultTable = resultTableCollection[ResultType.RelevantResults];

// Get the total count with and without duplicates from the search result table
int totalRows = resultTable.TotalRows;
int totalRowsIncludingDuplicates = resultTable.TotalRowsIncludingDuplicates;

// Get the visual best bets
ResultTable visualBestBetsTable = resultTableCollection[ResultType.VisualBestBetsResults];

// Pulls out and discards the values to show the retrieval process
foreach (DataRow row in visualBestBetsTable.Table.Rows)
{
   string name = (string)row["Name"];
   string uri = (string)row["Uri"];
   string description = (string)row["Description"];
   string keyword = (string)row["Keyword"];
   string teaser = (string)row["Teaser"];
   string contentType = (string)row["TeaserContentType"];
}

// Get a reference to the refiners
ResultTable refinementTable = resultTableCollection[ResultType.RefinementResults];

// Find the PowerPoint refiner
string powerPointRefinerToken = null;
foreach (DataRow row in refinementTable.Table.Rows)
{
   if ((string)row["RefinerName"] == "format"
                 && (string)row["RefinementName"] == "Microsoft PowerPoint")
   {
      // Save the token for the PowerPoint refiner
      powerPointRefinerToken = (string)row["RefinementToken"];
      break;
   }
}

// Add the PowerPoint refinement token to the query object
if (!string.IsNullOrEmpty(powerPointRefinerToken))
   kq.RefinementFilters.Add(powerPointRefinerToken);

// Add docvector to be used in the subsequent find similar query
kq.SelectProperties.Add("docvector");

// Execute the search one more time with refinement on "PowerPoint" files
resultTableCollection = kq.Execute();

// Prepare a find similar query for the first hit
resultTable = resultTableCollection[ResultType.RelevantResults];
string docvector = (string)resultTable.Table.Rows[0]["docvector"];

kq.SimilarType = SimilarType.Find;
kq.SimilarTo = docvector;

// Remove refinement for "PowerPoint" files
kq.RefinementFilters.Clear();

// Execute the search one more time using find similar
resultTableCollection = kq.Execute();

The following sample code searches for the term FAST Search for SharePoint and gives 1,000 extra rank points to items created by the author Mikael. If the items also contain the names Marcus and Robert, an additional 2,000 rank points are given.

FQL formatted for readability

xrank(
    xrank(
        string(
            "FAST Search for SharePoint",
             mode="simpleall"
        ),
         author:string(
            "mikael"
        ),
        boost=1000
    ),
     or(
        string(
            "marcus"
        ),
         string(
            "robert"
        )
    ),
     boost=2000
)

KeywordQuery class executing an FQL query

// Get a reference to the Query SSA Proxy
var searchProxy =
    (SearchServiceApplicationProxy)SearchServiceApplicationProxy.
GetProxy(SPServiceContext.Current);
KeywordQuery kq = new KeywordQuery(searchProxy);

// Enable FQL
kq.EnableFQL = true;
// Add the FQL query
kq.QueryText = "xrank(xrank(string("FAST Search for SharePoint", mode="simpleall"), author:s
tring("mikael"),boost=1000), or(string("marcus"), string("robert")), boost=2000)";
kq.ResultsProvider = SearchProvider.FASTSearch;

// Return search results
kq.ResultTypes = ResultType.RelevantResults;

// Execute search
ResultTableCollection resultTableCollection = kq.Execute();

The following sample builds a console application that uses the executing user’s domain credentials for authentication. The Query Web Service is added as a Web Service reference created with wsdl.exe (Microsoft Web Services Description Language Utility).

The sample queries for FAST Search for SharePoint sort the results with the newest items on top, includes best bets and visual best bets, and includes refiners for last modified date and the format of the items. The sample also shows a second query, refining on PowerPoint files, where the refiner is picked from the first query.

using System;
using System.Data;
using System.Net;

namespace FS4SP.ConsoleSamples
{
   internal class Program
   {
      private static void Main(string[] args)
      {
         // The Query Packet XML template for our query
         string templateXml =@"
<QueryPacket xmlns=""urn:Microsoft.Search.Query"">
   <Query>
      <Context>
         <QueryText language=""en-US"" type=""STRING"">FAST Search for SharePoint</QueryText>
      </Context>
      <SupportedFormats>
         <Format revision=""1"">urn:Microsoft.Search.Response.Document:Document</Format>
      </SupportedFormats>
      <ResultProvider>FASTSearch</ResultProvider>
      <Range>
         <StartAt>1</StartAt>
         <Count>10</Count>
      </Range>
      <EnableStemming>true</EnableStemming>
      <EnableSpellCheck>Off</EnableSpellCheck>
      <IncludeSpecialTermResults>true</IncludeSpecialTermResults>
      <IncludeRelevantResults>true</IncludeRelevantResults>
      <ImplicitAndBehavior>true</ImplicitAndBehavior>
      <TrimDuplicates>true</TrimDuplicates>
      <SortByProperties>
         <SortByProperty name=""write"" direction=""Descending"" />
      </SortByProperties>
      <IncludeRefinementResults>
         <Refiners>
            <Refiner>format</Refiner>
            <Refiner>write</Refiner>
         </Refiners>
      </IncludeRefinementResults>
      {0}
      <Properties>
         <Property name=""Rank"" />
         <Property name=""Title"" />
         <Property name=""Author"" />
         <Property name=""Size"" />
         <Property name=""Path"" />
         <Property name=""Write"" />
         <Property name=""SiteName"" />
         <Property name=""ContentClass"" />
         <Property name=""IsDocument"" />
         <Property name=""Url"" />
         <Property name=""FileExtension"" />
         <Property name=""SpSiteUrl"" />
         <Property name=""docvector"" />
      </Properties>
   </Query>
</QueryPacket>";

         // Accept all certificates - should only be used against sites you trust!
         ServicePointManager.ServerCertificateValidationCallback +=
            (sender, certificate, chain, sslPolicyErrors) => true;

         // Get an instance to the Web Service proxy
         // Generated with "wsdl.exe http://<server>/_vti_bin/search.asmx
         QueryService service = new QueryService();

         // Set the SharePoint server URL
         service.Url = "http://test/_vti_bin/search.asmx";

         // Set credentials for the executing user
         service.PreAuthenticate = true;
         service.UseDefaultCredentials = false;
         service.Credentials = CredentialCache.DefaultNetworkCredentials;

         // Create the Query Packet and execute the query
         string queryXml = string.Format(templateXml, string.Empty);
         DataSet resultSet = service.QueryEx(queryXml);

         // Get a reference to visual best bets and display the results
         DataTable visualBestBetsTable = resultSet.Tables["VisualBestBetsResults"];
         if (visualBestBetsTable != null)
         {
            foreach (DataRow row in visualBestBetsTable.Rows)
            {
               Console.WriteLine(row["Name"]);
               Console.WriteLine(row["Uri"]);
               Console.WriteLine(row["Description"]);
               Console.WriteLine(row["Keyword"]);
               Console.WriteLine(row["Teaser"]);
               Console.WriteLine(row["TeaserContentType"]);
            }
         }

         // Get a reference to results and print properties and values
         DataTable resultTable = resultSet.Tables["RelevantResults"];
         for (int i = 0; i < resultTable.Rows.Count; i++)
         {
            Console.WriteLine("RESULT #" + (i + 1));
            DataRow row = resultTable.Rows[i];
            for (int colIdx = 0; colIdx < row.ItemArray.Length; colIdx++)
            {
               Console.WriteLine(resultTable.Columns[colIdx].ColumnName
                  + ":" + row.ItemArray[colIdx]);
            }
            Console.WriteLine("-------------------------");
         }

         // Get a reference to the refiners
         DataTable refinementTable = resultSet.Tables["RefinementResults"];

         // Find the PowerPoint refiner
         string powerPointRefinerToken = null;
         foreach (DataRow row in refinementTable.Rows)
         {
            if ((string)row["RefinerName"] == "format"
               && (string)row["RefinementName"] == "Microsoft PowerPoint")
            {
               // Save the token for the PowerPoint refiner
               powerPointRefinerToken = (string)row["RefinementToken"];
               break;
            }
         }

         // Create the refinement filter for the query and add it to the Query Packet XML
         string refinementXml = "<RefinementFilters><RefinementFilter>" +
                                 powerPointRefinerToken +
                                "</RefinementFilter></RefinementFilters>";
         queryXml = string.Format(templateXml, refinementXml);

         // Execute the search one more time with refinement on "PowerPoint" files
         resultSet = service.QueryEx(queryXml);
      }
   }
}

If you want to execute the query by using FQL instead of Keyword Query Syntax, replace the following line.

<QueryText language="en-US" type="STRING">FAST Search for SharePoint</QueryText>

Use the following line as the replacement.

<QueryText language="en-US" type="FQL">string("fast for sharepoint", Mode="AND")</QueryText>