Since the dictionary words are stored in a JSON file, we need to parse the data before saving it to a local database.
First, we'll add the class that will represent our data. In the base package (com.packt.quickstart.dictionary), we first add a new package called data. There, we put the DictionaryEntry class, which will be our data model representation:
data class DictionaryEntry(val term: String, val explanation: String)
The class has two properties: the searchable term and the term definition. It'd be also nice to have a readable string representation of this class, so we let the Kotlin compiler generate a toString method for us with its data class.
Since the file of words is around 20 MB in size, to avoid loading that much data in the memory at once and deserializing the whole file in one go, we'll deserialize it manually, entry by entry.
For this, we'll build the EntriesIterator class, which can be iterated over, that is, it can supply dictionary entries one by one. That's why it implements the generic Iterator interface:
class EntriesIterator : Iterator<DictionaryEntry> {
private val jsonParser: JsonParser = MappingJsonFactory()
.createParser(javaClass.classLoader.getResourceAsStream("dictionary.json"))
private var currentToken: JsonToken?
JsonParser is the Jackson library class that provides an API for reading JSON content. We can get an instance of this class from the MappingJsonFactory type by providing an input stream. We've put the dictionary.json file in the project's resources folder. The easiest way to get an input stream of it is from the class loader that our classes are using.
In the class initializer (constructor), we obtain the current token of the JSON parser. The token represents the current location of a parser. Then, with a while loop, we position the token to the start of the first entry:
init {
currentToken = jsonParser.currentToken
while (currentToken != JsonToken.FIELD_NAME) {
currentToken = jsonParser.nextToken()
}
}
Our JSON file is structured like this:
{
"exportable" : "Suitable for exportation; as, exportable products.",
"irrisible" : "Not risible. [R.],
...
}
So, the first token will be START_OBJECT and the last one will be LAST_OBJECT. The dictionary term will be under FIELD_NAME and the definition will under the VALUE_STRING token.
With this information, we can implement the two required methods of the Iterator interface.
hasNext returns true until we reach the end object token:
override fun hasNext(): Boolean {
return currentToken != JsonToken.END_OBJECT
}
And the next method has to return a DictionaryEntry instance. It does that by reading the JSON key-value pair from the JSON parser:
override fun next(): DictionaryEntry {
var term = ""
var definition = ""
if (currentToken == JsonToken.FIELD_NAME) {
term = jsonParser.valueAsString
currentToken = jsonParser.nextToken()
}
if (currentToken == JsonToken.VALUE_STRING) {
definition = jsonParser.valueAsString
currentToken = jsonParser.nextToken()
}
if (currentToken == JsonToken.END_OBJECT) {
jsonParser.close()
}
return DictionaryEntry(term, definition)
}
We also check whether we have reached the end of the JSON file so we can properly close the parser, which will close the input stream for us.
Now that we have a type that can parse the JSON file and create dictionary entries, we need to save them to a database.
The JDBC driver will need to establish a connection to our database and give us a Connection object if we wish to query or manipulate the database. We'll create a separate type that takes care of this. In the same package, let's add the DbConnectionProvider interface:
interface DbConnectionProvider {
fun getConnection(): Connection
companion object {
val defaultProvider: DbConnectionProvider = LocalDbConnectionProvider()
}
}
private class LocalDbConnectionProvider : DbConnectionProvider {
override fun getConnection(): Connection = DriverManager.getConnection("jdbc:sqlite:dictionary.db")
}
Notice how Kotlin allows companion objects inside interfaces also. Our companion object holds the default implementation of the interface inside a property. The default implementation is defined in the same file and is hidden from outside, as the class is marked as private.
The database file will be in the same folder as our app, so construction of the connection URL is pretty simple; we just need the database filename after the jdbc:sqlite URL.
Next is the DictionaryLoader class, which will iterate over all entries and save them to a database. This is a long-running operation and it will be done asynchronously. Writing async code got a lot easier in Java 8, thanks to the CompletableFuture class. It implements the Future interface, which has been present in Java since version 1.5, but the problem with previous implementations of this interface was that calling its get function to obtain a result would block the calling thread until a result became available. Working with CompletableFuture is a lot nicer, since it offers triggering actions and invoking functions upon its completion. It even allows executing these functions on a different executor/thread. This makes it perfect for our use case; we can offload the long-running database loading to a background thread and then schedule a completion task on the main thread:
class DictionaryLoader(private val iterator: Iterator<DictionaryEntry>,
private val dbConnectionProvider: DbConnectionProvider = DbConnectionProvider.defaultProvider) {
fun loadDictionary(): CompletableFuture<Void> {
return CompletableFuture.runAsync {
dbConnectionProvider.getConnection().use { conn ->
conn.prepareStatement("INSERT INTO entries(entry, explanation) VALUES (?, ?)").use { stmnt ->
iterator.forEach { e ->
stmnt.setString(1, e.term)
stmnt.setString(2, e.explanation)
stmnt.executeUpdate()
}
}
}
}
}
}
The easiest way to create CompletableFuture is to call the static runAsync function and provide a lambda that will be executed asynchronously. We don’t have to specify an Executor that will be used for this task; by default, a ForkJoinPool is used, which is more than enough for our case.
The loadDictionary function returns a generic CompletableFuture. The Void generic parameter says that there is no result in this Future. We could have also used Kotlin's Unit, but then the lambda should explicitly return a Unit. That is because when a CompletableFuture is built with a Runnable, runAsync returns CompletableFuture<Void>:
public static CompletableFuture<Void> runAsync(Runnable runnable)
Inside the lambda, we get a Connection object from our DbConnectionProvider. The Connection interface extends the Closable interface since database connections should be closed after we're done with them. We’ll close them with the use function from the Kotlin Standard Library. Since we’ll be inserting a lot of rows into the database, we can make the initial population faster if we precompile the SQL Insert statement. The Connection object has a method for it; prepareStatement takes an SQL command and returns a PreparedStatement object. Notice how we used two question marks in the SQL command; these are command parameters and we bind them to our model’s term and explanation properties when we iterate the whole word list.
PreparedStatement is also closable, and again Kotlin's use function makes sure that it gets closed properly.
The last thing our data package needs is a repository type, which will talk to the database and offer a search function. Let's add the Repository class in the same package:
class Repository(private val dbConnectionProvider: DbConnectionProvider = DbConnectionProvider.defaultProvider)
The Repository class also needs a connection, and it will also get it from DbConnectionProvider. We'll again be working with Statements and PreparedStatements. To make the code a bit nicer and shorter, and to also show how receiver functions can be used, first we'll write two helper functions. They will obtain a connection, create a statement, and then execute the receiver function on that statement. prepareStatement works with the PreparedStatement object:
private inline fun <T> prepareStatement(sql: String, block: PreparedStatement.() -> T): T {
dbConnectionProvider.getConnection().use { conn ->
conn.prepareStatement(sql).use { stmt ->
return stmt.block()
}
}
}
The function is generic as we want to have the option of returning different types from this helper function. Although this is not needed in our small app, the purpose is to show how Kotlin can be used. Also, we marked it inline, so the compiler will inline the receiver lambda directly to the call site.
The second one is similar but works with Statement objects:
private inline fun <T> executeStatement(block: Statement.() -> T): T {
dbConnectionProvider.getConnection().use { conn ->
conn.createStatement().use { stmt ->
return stmt.block()
}
}
}
For searching dictionary entries, we'll need to extract data from a ResultSet object. ResultSet internally holds a cursor that points to a row in a database and is moved forward with its next function. We can create an extension function on ResultSet that knows how to extract all dictionary entries from it:
private fun ResultSet.getEntries(): List<DictionaryEntry> {
val entries = ArrayList<DictionaryEntry>()
while (this.next()) {
val term = this.getString(1)
val explanation = this.getString(2)
entries.add(DictionaryEntry(term, explanation))
}
return entries
}
With these helper functions in place, our search function is only two lines of code:
fun search(searchTerm: String, exactMatch: Boolean = true): List<DictionaryEntry> {
return prepareStatement("SELECT * FROM entries WHERE entry MATCH ?") {
setString(1, if (exactMatch) searchTerm else "$searchTerm*")
executeQuery().use { resultSet -> resultSet.getEntries() }
}
}
The prepareStatement function takes an SQL command and a receiver lambda. Inside the receiver lambda, we work with the PreparedStatement object of this SQL command, set the string parameter, execute it, and return the dictionary entries with our extension function.
We also need a method for creating our full-text search table. This method will be called each time our app starts, so we create the table only if it doesn't exist:
fun onStart() {
executeStatement { executeUpdate("CREATE VIRTUAL TABLE IF NOT EXISTS entries USING FTS4 (entry, explanation)") }
}
And finally, we need a function that will tell us if the data from the JSON file has been loaded. If the database is empty, our app will show a loading screen while we're parsing and inserting the data:
fun isDataLoaded(): Boolean {
return executeStatement {
val result = executeQuery("SELECT COUNT(*) FROM entries")
result.next() && result.getInt(1) > 0
}
}