Formulatel#

formulatel is a hobby project of mine that collects sim-racing telemetry from multiple different sources (different games) and provides both live and historic dashboards to analyze vehicle telemetry.

---
config:
  theme: redux
  look: neo
title: formulatel
---
flowchart LR
  subgraph Sources [Sim Racing Games]
    T1[Title A]
    T2[Title B]
    TN[Title C]
  end
  Ingest[ingest<br/>Receive, Transform, Publish]
  subgraph PubSub [Pub/Sub Topics by Type]
    TypeA[Topic: Vehicle]
    TypeB[Topic: Motion]
    TypeN[Topic: Laps]
  end
  G[Grafana<br/>Visualization]
  T1 --> Ingest
  T2 --> Ingest
  TN --> Ingest
  Ingest --> TypeA
  Ingest --> TypeB
  Ingest --> TypeN
  Persist[persist<br/>Persistence Layer]
  DB[(Datastore)]
  Persist --> DB
  PubSub --> Persist
  PubSub <--Live Visualization--> G
  G <--Analysis--> DB

Goals#

The major goal of the project is to be able to capture telemetry from 2+ racing games and visualize that telemetry using a single dashboard, the idea being that the driver will want to see the same information regardless of the title they’re playing, and that they might like to share those dashboards with other drivers. We capture and transform telemetry per-title, provide interesting dashboards/visualizations of the telemetry, and give the driver the power to create their own dashboards.

The next goal is to package formulatel for other less tech-savvy drivers to use. Current users have to do a fair amount of setup that we expect most users won’t be comfortable with, and I expect most users to be running Windows or using a console. The original plan was to try to package formulatel along with mosquitto and grafana for users to run in a single executable, but I didn’t like the idea of distributing a massive binary like that and felt like packaging a Windows installer was outside of the scope of this project. I did consider creating a desktop app with something like Wails, but I would have to recreate all of the charting with javascript and drop Grafana, and that was never something I wanted to do.

In the end, I decided to go with a cloud deployment for a number of reasons, but the largest motivator was realizing I could do it for free. Oracle Cloud Infrastructure has a generous free tier for a specific class of ARM processors that was perfect for this, allowing us up to 4 vCPUs and 24GiB of memory. Cloud deployments and kubernetes is what I’ve spent my career doing, so I preferred this route over learning how to write a nice desktop app and trying to center divs. It wasn’t totally free because I did spend a few dollars on the domain name, but for the amount of use I imagine it getting, OCI gives me plenty of room to run postgres, kubernetes, grafana, mosquitto, persist, and anything else I need to run.

Design Decisions#

The core design decisions of formulatel were:

  • separate telemetry ingestion and telemetry persistence: persistence is where the service needs to scale
  • encode telemetry using protocol buffers: regardless of the transport, data should be encoded using protocol buffers.
  • decouple native data ingestion, data transformation, and data transport: we want to be able to write ingest clients for any title and support multiple methods of data transport between ingest and persist
  • decouple persistence from the datastore: formulatel must be able to support multiple datastores (e.g. postgres and ClickHouse)
  • Grafana will visualize everything: Grafana is the only supported visualization frontend

While these are all still important design facets, the practical reality of using Grafana Live to visualize the live telemetry means that we’re somewhat locked in to MQTT, and that locks us in to JSON rather than the binary encoding of protobufs. This is one of the tradeoffs that I have written a lot about while creating and deploying formulatel on the GitHub Wiki. formulatel is still developing and changing every week, and the wiki is where all of that detail will be documented.

Motivation#

This project is one I started for fun and, originally, to learn more about OpenTelemetry. I’m a hands-on learner and am a staunch supporter of system observability, so I thought it would be best to come up with a project idea when I was learning about OpenTelemetry. When I began this project, I had been spending some of my free time playing racing simulation games, many of which have a feature to broadcast telemetry packets over UDP, and I thought that would make a fun project that combines my hobbies/interests and OpenTelemetry.

The original plan was to use OT as the method of transportation. I toyed with creating instruments for different vehicle sensors, creating traces for laps with spans for sectors or turns, and looked into creating a collector distribution for vehicle telemetry, but I had to concede to MQTT upon the realized that the live visualization relied on it. That doesn’t mean I don’t still use the project as a way to learn about OpenTelemetry: tracing was one of the first features we added before our cloud deployment.

Usage#

See the repository README for information on installing and running formulatel.

Backward civilization sim generic handler Forward