You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
Go to file
William Perron 2b3ff1aa04
improve instrumentation of internal notifier
7 months ago
.github/workflows bump system deps 10 months ago
cmd/themis-server add some emojis 7 months ago
correlation fix lints and update logger 10 months ago
data update provinces table with latest version 11 months ago
migrations use table instead of view for better perf 11 months ago
.gitignore Add absences feature set 1 year ago
Dockerfile bump system deps 10 months ago
LICENSE add golang-migrate 1 year ago
README.md bump system deps 10 months ago
absences.go fix lints and update logger 10 months ago
absences_test.go fix lints and update logger 10 months ago
audit_log.go fix lints and update logger 10 months ago
claim_type.go Add `claimables` view to database 11 months ago
conflicts.go finish instrumenting with opentelemetry 10 months ago
conflicts_test.go fix lints and update logger 10 months ago
errors.go add last flush to /info command 12 months ago
fly.toml mark some responses as ephemeral and enable cpu profiling 10 months ago
fmt.go finish instrumenting with opentelemetry 10 months ago
fmt_test.go fix lints and update logger 10 months ago
go.mod add fly detector to otel setup 9 months ago
go.sum add fly detector to otel setup 9 months ago
notify.go improve instrumentation of internal notifier 7 months ago
start.sh set up litestream replication 2 years ago
store.go mark some responses as ephemeral and enable cpu profiling 10 months ago
store_test.go fix failing test 10 months ago
time.go Add absences feature set 1 year ago
time_test.go Add absences feature set 1 year ago
uptime_darwin.go Add `claimables` view to database 11 months ago
uptime_linux.go add correlation IDs and propagate ctx 11 months ago
uptime_test.go add correlation IDs and propagate ctx 11 months ago

README.md

themis

Discord App to allow EU4 players to take claims on regions and provinces.

Setup

Requirements

To develop:

  • Go version 1.21 or higher installed locally
  • sqlite3 installed locally (already ships by default on most OSes)

To deploy:

Steps

To work with the core modules that simply interact with the database, or make sure that everything compiles correctly, you can run the following commands:

go test ./...
go build -o ./bin/ ./cmd/...

Operations

This application is deployed via Fly on a single virtual machine instance. It uses an embed SQLite database as its database engine, and uses Litestream to replicate the database to an S3 bucket.

Application Configuration

The deployment configurations can be found in the fly.toml file and you can find more information on the configuration options in the official Fly documentation.

The virtual image is based off of the Dockerfile which is a multi-stage build that builds the main application binary, downloads Litestream and packages everything on top of an Ubuntu 22.04 base.

Entrypoint

The application is started using a custom entrypoint shell script that is in charge of first restoring the database file through Litestream and then starting the main application as a child process of Litestream's replication process.

It's a very simple script but is necessary since the application doesn't have a persistent volume to rely on and must rehydrate its database file after every deployment.

Environment Variables

Env Var Defined At
DISCORD_TOKEN fly secret
DISCORD_APP_ID fly.toml
DISCORD_GUILD_ID fly.toml
AWS_ACCESS_KEY_ID fly secret
AWS_SECRET_ACCESS_KEY fly secret

Local Development

Application Entrypoint

  • ./cmd/themis-server This is the main application entrypoint
  • ./cmd/themis-repl coming soon

Core Functions

The core database functions can be developed and tested locally easily, just run go test ./... to test your changes locally.

You can also load a test database easily by using the init.sql script. You can use the cli sqlite3 client or connect using whatever SQL editor you prefer that can open sqlite3 connections.

sqlite3 local.db < migrations/init.sql
sqlite3 local.db
# interactive SQLite session

Discord Integration

This is a work in progress, but I am currently using a dedicated Discord server with this application already signed into it. You can contact me directly to be invited into that server. From there you only need to set the following environment variables before launching the ./cmd/themis-server entrypoint:

export DISCORD_APP_ID="1014881815921705030"
export DISCORD_GUILD_ID="[test server id goes here]"

Litestream replication

Litestream is not a necessary component to run the ./cmd/themis-server entrypoint and can be safely ignored when developing locally. Still, if you wish to try it out, you can find the Litestream commands used in production in the start.sh script. As per the Litestream docs, it should work fine with Minio but I have not tested it yet nor are there any scripts provided to run it (yet).

SQLite

Importing From CSV

This is a neat feature built-in to SQLite. Using the source file at data/eu4-provinces.csv you can import the data directly into a SQLite database using the following command:

$ sqlite3
# ...
sqlite> .mode csv
sqlite> .import data/eu4-provinces.csv provinces
sqlite> .schema provinces
CREATE TABLE provinces(
  "ID" TEXT,
  "Name" TEXT,
  "Development" TEXT,
  "BT" TEXT,
  "BP" TEXT,
  "BM" TEXT,
  "Trade good" TEXT,
  "Trade node" TEXT,
  "Modifiers" TEXT,
  "Type" TEXT,
  "Continent" TEXT,
  "Superregion" TEXT,
  "Region" TEXT,
  "Area" TEXT
);
sqlite> select count(1) from provinces;
3925

Creating an SQL Dump From Imported Data

The init script at migrations/init.sql was initially created after importing the CSV data and running the following commands:

# In the same SQlite session as last section
sqlite> .output dump.sql
sqlite> .dump

Note: The column names were edited manually to remove capital letters and spaces to make it easier to work with.

Claims Schema

CREATE TABLE claim_types (
    claim_type TEXT PRIMARY KEY
);

CREATE TABLE claims (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    player TEXT,
    claim_type TEXT,
    val TEXT,
    FOREIGN KEY(claim_type) REFERENCES claim_types(claim_type)
);