Loggerhead: Because Reinventing The Wheel Is Fun

A few months ago, I started building a flight tracker (just for fun) using freely available plane location data. Most of this data is available in real-time or near real-time.

I started looking for the right database to store and query this data. Although I have used many databases in the past for geolocation data (PostGis on Postgres, Redis). I picked Redis because I don't need to store historical data (and it will quickly become a Terabyte of data, and Alkawn, my homeland, doesn't yet have enough storage for that).

One interesting thing for me is that there are no specialized Geolocation databases, like TigerBeetle is a specialized financial transaction database.

So I asked myself, what would a pure geolocation database look like? Then I asked myself a better question: What if you built the first version of it?

And that's how I was caught in this downright fun endeavor of reinventing the geolocation wheel.

So, let me introduce Loggerhead, an in-memory geolocation database built in Golang.

Loggerhead is a simple database that allows you to solve two problems very well:

1. Ingest a lot of data quickly (easily reach 5+ million RPM writes).
2. Query locations over a large area like Africa up to 30 times per millisecond (you read correctly).

It has the concept of Namespaces, which allows you to logically group location entities. This is useful if you want to track different kinds of entities separately (e.g., Bikes and planes), and you will never search for both simultaneously.

Loggerhead is not supposed to be your primary storage; It is your fast-read storage that helps you power your dashboard or mobile app map.

It currently listens to 5 commands:

• GET to get one location back using its ID and namespace.
• POLY to get all locations in a rectangle using two opposite corners and their namespace.
• SAVE to create or update a location.
• DELETE, self-explanatory.

Loggerhead is open source, so you can dive deep into the code, critique it, or improve it. I also made the roadmap visible to you, which can help me move faster.

GitHub - fabricekabongo/loggerhead: Geolocation in-memory database, focused, simple, fast 1million write per sec on 1 core

Geolocation in-memory database, focused, simple, fast 1million write per sec on 1 core - fabricekabongo/loggerhead

GitHubfabricekabongo

Have fun! Now and then, reinvent the wheel to learn and push your imagination.