There are several freely available datasets online. One platform I came across was Kaggle. It supports analytics competitions based on user-mined datasets. After a bit of digging, I uncovered a CSV file of user reported UFO Sightings. This turned out to be interesting data. For instance, which countries had the most reported UFO sightings? Is there a particular day of the week where sightings occur the most? I had so many questions! My first step however, was to expose the data as a GraphQL API.
Dependencies Used
I wanted to experiment with various libraries written entirely in Kotlin. These are the ones that I chose in order to create my GraphQL Server:
* Ktor – Server Framework
* Koin – Dependency Injection
* KGraphQL – GraphQL Support
* Squash – Database Access
Here’s the dependencies block of the build.gradle file:
dependencies {
// Kotlin
compile "org.jetbrains.kotlin:kotlin-stdlib-jdk8:$kotlin_version"
// Ktor Server
compile "io.ktor:ktor-server-core:$ktor_version"
compile "io.ktor:ktor-server-netty:$ktor_version"
compile "io.ktor:ktor-gson:$ktor_version"
compile "io.ktor:ktor-locations:$ktor_version"
// Logging
compile "ch.qos.logback:logback-classic:$logback_version"
// Networking
compile "com.github.kittinunf.fuel:fuel:$fuel_version"
// GraphQL
compile "com.github.pgutkowski:kgraphql:$kgraphql_version"
// Dependency Injection
compile "org.koin:koin-ktor:$koin_version"
// Database
compile "org.jetbrains.squash:squash-h2:$squash_version"
// Testing
testCompile group: 'junit', name: 'junit', version: '4.12'
}
Project Structure
The code can be found in the src/main/kotlin folder. From there it’s broken down into the following packages:
info/adavis– contains the Ktor application class, routes and the KGraphQL schemainfo/adavis/model– contains the UFOSighting data classinfo/adavis/di– contains the Koin dependency moduleinfo/adavis/dao– contains the CSV parser for the UFO Sighting data and the DAO for accessing the H2 database
Finally the resources folder holds the Ktor server configuration and the UFO Sightings CSV file.
Ktor Application Setup
Ktor is a framework that aims to be a simple and powerful way to create connected applications. This made it ideal for my use-case. I just wanted to expose one endpoint /graphql and grab the data from an H2 database. Ktor operates based on what it calls “Pipelines“. This allows you to add on to the Server only what you need. Here’s a standard setup in the main() function:
install(DefaultHeaders)
install(CallLogging)
install(Locations)
install(ContentNegotiation) {
gson {
setPrettyPrinting()
}
}
importData()
What this gives you is some headers with application info when a response is sent, logging support, and then automatic content conversion that uses Gson. And believe it or not you can now run your application. The final piece for us in the main() function however, is to parse the CSV file with the UFO sighting data and store it in the H2 database. For that we create a simple method importData and we call it after we’ve set up our pipelines.
Database Setup
In order to store and query our data, we’re using the H2 database. It’s small, fast and offers robust SQL querying options. However, in the spirit of keeping things minimal I used the Squash Kotlin library to serve as a data access layer for H2.
First, you define an object to represent your database tables. I decided to call the table, UFOSightings. Here’s it’s definition:
object UFOSightings : TableDefinition() {
val id = integer("id").autoIncrement().primaryKey()
val date = date("date")
val city = varchar("city", 128)
val state = varchar("state", 4)
val country = varchar("country", 4)
val shape = varchar("shape", 28)
val duration = decimal("duration", 10, 2)
val comments = varchar("comments", 1024)
val latitude = decimal("latitude", 12, 8)
val longitude = decimal("longitude", 12, 8)
}
Key things to note are first, you need to extend the TableDefinition class. And then second, each column is represented by a property in your class. It’s a simple mapping between the fields of your model object and their database representation.
Next, we’ll define an Interface that we’ll use to represent our data storage class, UFOSightingStorage. It contains the basic methods that we will call from our GraphQL resolver functions. Notice that although we used the UFOSightings class for our table definition, we actually use the UFOSighting data class for our business logic.
interface UFOSightingStorage : Closeable {
fun createSighting(sighting: UFOSighting): Int
fun getSighting(id: Int): UFOSighting?
fun getAll(size: Long): List
}
Finally, we implement this interface as the UFOSightingDatabase class. Let’s take a look at one of the methods, getAll():
override fun getAll(size: Long): List<UFOSighting> = db.transaction {
from(UFOSightings)
.select()
.orderBy(UFOSightings.date, false)
.limit(size)
.execute()
.map { it.toUFOSighting() }
.toList()
}
Here, we want to retrieve a set number of UFOSighting objects from the UFOSightings table. We use a Squash DB transaction to execute a select statement and then map the result to our list of desired objects. A lot of power in a small amount of code!
GraphQL Schema Definition
With our database in place, we can now set up our GraphQL schema, using KGraphQL. We’ll start by specifying our GraphQL Types:
val schema = KGraphQL.schema {
type<UFOSighting> {
description = "A UFO sighting"
}
}
We use the type DSL and supply our UFOSighting data class. Then we can also add a description for our type. (Note: At the time of this writing, the description was not being returned in introspection queries.)
Next, we want to prepare the queries that we’ll support on our GraphQL Server. We define those with the query DSL:
val schema = KGraphQL.schema {
query("sightings") {
resolver { size: Long -> storage.getAll(size) }.withArgs {
arg { name = "size"; defaultValue = 10 }
}
}
type<UFOSighting> {
description = "A UFO sighting"
}
}
This means that when we receive the sightings query, we will check the size argument and attempt to return that number of UFO sightings. If the size is not provided we’ll default to the number 10. But notice that we’re using a function on the storage object to retrieve the records. Where did that come from?
Our schema is contained inside of a wrapper class, AppSchema. This class uses dependency injection, via Koin, to inject an instance of the UFOSightingStorage class.
class AppSchema(private val storage: UFOSightingStorage) {
val schema = KGraphQL.schema {
...
}
}
Dependency Injection
Getting dependencies with Koin is very simple. You create a module and by means of a succinct DSL, you specify the dependencies you need. If you’ve worked with more complex frameworks in the past, this is a breeze to understand and implement. I think it’s ideal for small applications, like our sample app here.
val mainModule = applicationContext {
provide { AppSchema(get()) }
provide { UFOSightingDatabase() as UFOSightingStorage }
}
API Endpoint
The final piece to make this work is to expose our /graphql endpoint so that we can make our GraphQL query requests. For this we’re going to use Ktor Routes and Locations.
@Location("/graphql")
data class GraphQLRequest(val query: String = "")
fun Route.graphql(log: Logger, schema: Schema) {
post<GraphQLRequest> {
val request = call.receive<GraphQLRequest>()
val query = request.query
log.info("the graphql query: $query")
call.respondText(schema.execute(query), ContentType.Application.Json)
}
}
To specify an endpoint, you pass a string to the @Location annotation. Then you create a data class to represent the post/get request data. This allows for type safety and code completion when working with your HTTP requests.
Next, inside of our post function, we can grab the query value and then execute it against our GraphQL schema. Finally, we return the result as JSON. This will allow us to run queries on our GraphQL Server.
I had fun working on this project, it’s nice to explore different kinds of datasets. You can find the complete project here on GitHub.
