Guide

Where should a bot keep its data?

Prefixes, levels, warnings, balances: most bots end up remembering something. There are three sensible places to put it, and the right one depends on how much there is and what else needs to read it.

← All guides3 min readUpdated

A JSON file

For a handful of settings that are read at startup and rarely change, a JSON file is fine. It needs no library and you can open it to see what is in it.

It stops being fine as it grows. Every change rewrites the whole file, and a crash or restart halfway through a write leaves half a file that will not parse, which loses everything in it rather than the one change. Write to a temporary file and rename it over the old one: a rename replaces the file in one step, so a reader sees the old version or the new one and never a broken mix.

Node.js

const fs = require('node:fs');

function save(path, data) {
    const tmp = `${path}.tmp`;
    fs.writeFileSync(tmp, JSON.stringify(data, null, 2));
    fs.renameSync(tmp, path); // replaces the old file in one step
}

SQLite

SQLite is a full SQL database stored in a single file, with no server to run. It writes only what changed, survives a crash mid-write, and handles far more data than a bot usually has. For most bots it is the right answer.

Python includes it as the sqlite3 module, and Bun as bun:sqlite. On Node.js, better-sqlite3 is the usual package; it is a native add-on, compiled when it installs. Recent Node.js releases also include an experimental node:sqlite module.

Always pass values as parameters, like the ? below, rather than building SQL out of strings. A user's name or message pasted into a query string is how a bot gets its database emptied.

better-sqlite3

const Database = require('better-sqlite3');
const db = new Database('bot.db');

db.exec('CREATE TABLE IF NOT EXISTS balances (user_id TEXT PRIMARY KEY, coins INTEGER NOT NULL DEFAULT 0)');

const addCoins = db.prepare(
    'INSERT INTO balances (user_id, coins) VALUES (?, ?) ' +
        'ON CONFLICT(user_id) DO UPDATE SET coins = coins + excluded.coins'
);

addCoins.run(interaction.user.id, 10);

Python

import sqlite3

db = sqlite3.connect("bot.db")
db.execute("CREATE TABLE IF NOT EXISTS balances (user_id TEXT PRIMARY KEY, coins INTEGER NOT NULL DEFAULT 0)")

db.execute(
    "INSERT INTO balances (user_id, coins) VALUES (?, ?) "
    "ON CONFLICT(user_id) DO UPDATE SET coins = coins + excluded.coins",
    (str(interaction.user.id), 10),
)
db.commit()

A database server

MariaDB and PostgreSQL run as a separate service your bot connects to over the network. That is more to set up than a file, and worth it when more than one program needs the same data: shards running as separate processes, a web dashboard that shows levels or changes settings, or a second bot.

Create one connection pool when the bot starts and use it everywhere. Opening a new connection for every command is slow, and a busy bot can run into the database's connection limit.

Node.js, with pg

const { Pool } = require('pg');

// One pool for the whole bot, created at startup.
const pool = new Pool({ connectionString: process.env.DATABASE_URL, max: 5 });

// Then, inside a command:
const { rows } = await pool.query('SELECT coins FROM balances WHERE user_id = $1', [userId]);

Python, with asyncpg

# Once, in setup_hook or your startup code:
pool = await asyncpg.create_pool(os.environ["DATABASE_URL"], max_size=5)

# Then, inside a command:
row = await pool.fetchrow("SELECT coins FROM balances WHERE user_id = $1", user_id)

Which to choose

A few settings that hardly change: a JSON file, written safely. Anything that grows with the number of users or servers, such as levels, warnings or balances: SQLite. Data that more than one program reads or writes, or that has to live somewhere other than the bot's own disk: a database server.

Moving up later is not hard if the bot's code goes through a few functions of your own, such as getBalance and addCoins, rather than scattering queries everywhere. Only those functions change.

On our hosting

A JSON or SQLite file lives on your server's disk beside your code, stays there across restarts, and counts towards the 512 MB of disk a free server has, which is room for a great deal of bot data. Download a copy from the file manager now and then: it is the only copy there is.

If you need a database server, every account also gets free MariaDB and PostgreSQL databases, created from Databases in the panel. They do not need a server here and accept connections from anywhere, so the bot, a dashboard and your own machine can all reach the same data.

FAQ

Questions

Will uploading my code overwrite the database file?

It can, if the upload includes an old copy of it. Keep the data file out of the folder you upload from, and out of git, so a new version of the code never replaces live data with old data.

Can several shards share one SQLite file?

Within one process, yes. Several processes can open the same file, but only one can write at a time, and the others wait. Past a couple of shards in separate processes, a database server is the cleaner fit.

Is sqlite3 in Python a problem for an async bot?

Each query blocks the bot while it runs, which for small queries on a small bot is too brief to notice. If it ever is, aiosqlite offers the same module with an async interface.

Read next