Kicking off Spiro

Categories Database, Elixir

As has served me well every time I try to pick up a new technology or language, I need an interesting and challenging project to build once I’ve acquired as much book knowledge as I can endure.  The same goes for learning Elixir.  In that vein, I’m embarking on building a new package for the Elixir ecosystem named Spiro.

What is Spiro going to do?  The hopes are that it will define an interface and ecosystem to support graph databases in Elixir.  At present, there are pretty much four types of databases in the marketplace:

  • Relational.  These are table-based systems like MySQL, Oracle, MSSQLServer, etc.
  • Key-Value.  These store a key-value pair where the key is often a string and the value is often a blob (there are obviously some exceptions).  Common systems are REDIS and Mnesia.
  • Document.  Think MongoDB, and to a lesser extent some RDBMS solutions that are capable of storing and querying structured document blobs like Postgres.
  • Graph.  These store vertices with edges representing relationships between vertices.  Often both vertices and edges can have additional properties.  Think Neo4j, OrientDB, digraph.

The current go-to datastore interface for Elixir is Ecto, which does an incredible job of providing an abstract interface into relational databases.  It’s leveraged by Phoenix, and has a solid architecture.  However, it’s geared heavily towards supporting relational databases, and attempts to add NoSQL-based adapters have struggled, especially of the graph variety.  Hence the idea that maybe we need a different abstraction for different database types, and the idea for Spiro was born.

The plan is to borrow the best elements and architectural concepts behind Ecto, but adapt them to play nicely with the DSL required for graph traversals.  In searching for a good base for the Spiro DSL, I stumbled across Apache TinkerPop and the Gremlin Structure interface.  It provides a chainable traversal interface that has been proven to work for many graph databases.  Additionally, the API is designed in such a way that it has been ported to several programming languages, including some functional ones.  Bonus points for the fact it’s in its third iteration AND is backed by Apache.

The end goal?  To eventually have a library that provides an abstract interface into various graph databases as Ecto does for relational systems, without compromising the DSL.  With the growing market share for graph databases and the fact Elixir scales so well to meet evolving project needs, it seems like a natural fit.