ArPack

Binary serialization code generator for Go, C#, TypeScript, Lua, C11, and GameMaker (2.3+).

Define wire messages as Go structs, then generate compact little-endian serializers for every target. ArPack is intentionally narrow. It targets protocols you own, where predictable layout, cross-language compatibility, and low runtime overhead matter more than schema evolution.

Install

go install github.com/edmand46/arpack/cmd/arpack@latest

Generate

arpack \
  -in messages/messages.go \
  -out-go messages \
  -out-cs client/Messages \
  -out-ts web/src/messages \
  -out-lua defold/scripts/messages \
  -out-gml GameMaker/scripts/src_arpack \
  -out-c native/messages

Flag	Purpose
-in	Input Go schema file, repeatable
-name	Base name for generated output files; defaults to the first -in file's name
-out-go	Generated Go methods, co-located with the schema package
-out-cs	Generated C# files
-out-ts	Generated TypeScript files
-out-lua	Generated Lua files
-out-gml	Generated GameMaker Language (GML) files
-out-c	Generated C11 files
-cs-namespace	C# namespace, default Arpack.Messages

Output names:

Target	File
Go	{name}_gen.go
C#	{Name}.gen.cs
TypeScript	{Name}.gen.ts
Lua	{name}_gen.lua
GML	{folder}/{folder}.gml
C	{name}_gen.h, {name}_gen.c

Schema

Schemas are Go files. Message structs, nested message structs, enum types, and enum constants must be defined in the schema files.

Multiple -in files are merged into one output file per target. Files sharing a package name are type-checked together (cross-file references work within a package); different packages are parsed independently and merged. Type names must be unique across all inputs, for example system messages from a framework plus game messages from your project:

arpack \
  -in vendor/framework/messages.go \
  -in game/messages.go \
  -name messages \
  -out-cs client/Messages

Cross-package type references (framework.Vec3 in a game struct) are not supported; shared structs must live in one of the inputs.

package messages

type Opcode uint16

const (
    OpcodeUnknown Opcode = iota
    OpcodeMove
)

type Vector3 struct {
    X float32 `pack:"min=-500,max=500,bits=16"`
    Y float32 `pack:"min=-500,max=500,bits=16"`
    Z float32 `pack:"min=-500,max=500,bits=16"`
}

type MoveMessage struct {
    Op       Opcode
    PlayerID uint32
    Position Vector3
    Velocity [3]float32
    Trail    []Vector3
    Active   bool
    Visible  bool
    Name     string
}

Supported field shapes:

Kind	Notes
bool	Consecutive bool fields are bit-packed
int8/uint8 through int64/uint64	Explicit-width integers only; Lua rejects 64-bit integers
float32, float64	Optional `pack:"min=...,max=...,bits=8
string	UTF-8 bytes with uint16 length prefix
[N]T, []T	Fixed arrays and uint16-length slices
map[K]V	uint16 entry count, then (key, value) pairs in ascending key order. Keys: string, explicit-width integers, enums. Values follow slice-element rules
named structs and enums	Must be declared in the schema file

Unsupported: external package types, pointers, embedded fields, nested collections (including maps of collections and collections of maps), bool/float/struct map keys, and platform-dependent int, uint, or uintptr.

Runtime Contract

• Wire format is little-endian.
• Fields are encoded in declaration order.
• Strings, slices, and maps use uint16 length prefixes.
• Map entries are written in ascending key order: numeric for integers and enums, UTF-8 byte order for strings. Deserializers reject unsorted or duplicate keys.
• Enum fields use their declared underlying integer type.
• Quantized floats fail fast on NaN or out-of-range values.
• Deserializers reject malformed or truncated input.
• Multi-target generation is staged: a failing target does not partially update earlier outputs.

Unchanged schemas keep the same wire layout. Field order, field type, enum underlying type, length encoding, bool grouping, and quantization changes are wire-breaking.

Generated APIs

Go:

func (m *MoveMessage) Marshal(buf []byte) []byte
func (m *MoveMessage) Unmarshal(data []byte) (int, error)

C#:

public int Serialize(Span<byte> buffer)
public static int Deserialize(ReadOnlySpan<byte> data, out MoveMessage msg)

public unsafe int Serialize(byte* buffer, int length)
public static unsafe int Deserialize(byte* buffer, int length, out MoveMessage msg)

TypeScript:

serialize(buffer: Uint8Array): number
static deserialize(data: Uint8Array): [MoveMessage, number]

serialize(view: DataView, offset: number): number
static deserialize(view: DataView, offset: number): [MoveMessage, number]

Lua:

local messages = require("messages.messages_gen")
local data = messages.serialize_move_message(msg)
local decoded, bytes_read = messages.deserialize_move_message(data)

GML:

var _msg = new MoveMessage();
var _buf = _msg.serialize();                    // creates a buffer, seeked to start
// _msg.serialize(_buf);                        // or append into an existing buffer

buffer_seek(_buf, buffer_seek_start, 0);
var _result = MoveMessage.deserialize(_buf);
var _decoded = _result[0];                      // MoveMessage
var _bytes = _result[1];                        // bytes consumed

Map fields are ds_maps (created by the constructor and filled by deserialize); the caller owns them and must ds_map_destroy when done.

C:

#include "messages_gen.h"

MoveMessage msg = {0};
msg.player_id = 777;
msg.name = (ArpackString){ "TestPlayer", 10 };

uint8_t buf[256];
int n = MoveMessage_serialize(&msg, buf, sizeof buf);      // bytes written, or ARPACK_ERR_*

MoveMessage decoded;
int used = MoveMessage_deserialize(&decoded, buf, (size_t)n); // bytes read, or ARPACK_ERR_*
if (used < 0) puts(arpack_strerror(used));
MoveMessage_free(&decoded);                                   // frees strings, slices, map entries

Field names are snake_case. Strings are ArpackString { char *data; uint16_t len; }; deserialize allocates them and NUL-terminates. Slices are T *x; uint16_t x_count;. Maps are { key, value } entry arrays with x_count; serialize sorts them in place and rejects duplicate keys. Deserialize allocates through ARPACK_MALLOC/ARPACK_CALLOC (override with -D flags); on error it frees what it allocated and returns a negative ARPACK_ERR_* code. X_free releases owned memory and zeroes the struct.

Map representation per language: Go map[K]V, C# Dictionary<K, V>, TypeScript Map<K, V>, Lua table, GML ds_map, C sorted { key, value } array.

Benchmarks

Recent Go results on Apple M3 Max:

BenchmarkArPack_Marshal-16                         8.95-9.04 ns/op     0 B/op    0 allocs/op
BenchmarkArPack_Unmarshal-16                      24.73-25.22 ns/op   40 B/op    2 allocs/op
BenchmarkProto_Marshal-16                         179.9-182.7 ns/op    0 B/op    0 allocs/op
BenchmarkProto_Unmarshal-16                       236.3-245.0 ns/op  248 B/op    7 allocs/op
BenchmarkFlatBuffers_Marshal-16                   143.4-144.0 ns/op    0 B/op    0 allocs/op
BenchmarkFlatBuffers_Unmarshal-16                 35.64-35.87 ns/op   24 B/op    1 allocs/op
BenchmarkArPack_MapMarshal-16                     332.1-349.4 ns/op  128 B/op    1 allocs/op
BenchmarkArPack_MapUnmarshal-16                   272.8-278.2 ns/op   64 B/op    8 allocs/op
BenchmarkProto_MapMarshal-16                      2469-2520 ns/op    960 B/op   56 allocs/op
BenchmarkProto_MapUnmarshal-16                    1923-1966 ns/op   1376 B/op   57 allocs/op

Map benchmarks serialize ScoreboardMessage (map[string]int32 + map[uint16]Vector3, 8 entries each); protobuf runs with Deterministic: true so both sort keys, and the ArPack map unmarshal reuses the decoded message (clear + refill). FlatBuffers has no native map type.

These results use a full-float sample schema for all formats. Marshal benchmarks reuse caller-owned buffers/builders where the library supports it. Unmarshal benchmarks decode into a fresh message on each iteration. The FlatBuffers baseline is a hand-written encoding using the FlatBuffers Go runtime with inline struct Vec3 values.

Format	Encoded size
ArPack	66 bytes
Protobuf	68 bytes
FlatBuffers	120 bytes

Run locally:

make bench-stats

Development

make test
make bench
make size

Regenerate checked-in benchmark code:

make generate