6. Deep learning and trained models

You can use deep learning, including deep reinforcement learning. The constraint is where each part happens:

Step by step

1. Train and save the weights.

torch.save(model.state_dict(), "model.pt")        # or: np.savez("model.npz", **arrays)

2. Write your agent with a WEIGHTS line. Start from examples/nn_agent.py, which has everything wired up:

import numpy as np
from papersseum import nn, unpack_weights

WEIGHTS = ""

class Agent:
    def reset(self, config):
        self.W = unpack_weights(WEIGHTS)           # {"conv1.weight": array, ...} as float32

    def act(self, obs):
        h = nn.relu(nn.conv2d(obs["local"], self.W["conv1.weight"], self.W["conv1.bias"], padding=1))
        h = nn.relu(nn.conv2d(h, self.W["conv2.weight"], self.W["conv2.bias"], stride=2, padding=1))
        logits = nn.linear(nn.global_avg_pool(h), self.W["fc.weight"], self.W["fc.bias"])
        return int(np.argmax(logits))

3. Embed the weights (this rewrites the WEIGHTS = "..." line and tells you the size):

papersseum pack model.pt --into my_agent.py                 # float16: half the size, almost no accuracy loss
papersseum pack model.pt --into my_agent.py --dtype int8    # smallest, check your bot still plays well

It accepts .pt, .pth, .npz and .npy. Reading .pt files needs PyTorch on your machine, not in the sandbox. If the file would exceed 1 MB, pack refuses and writes nothing.

4. Check it.

papersseum validate my_agent.py        # the platform's upload check
papersseum eval my_agent.py --games 30

What fits

About 350,000 weights at float16 or 700,000 at int8 inside the 1 MB limit, leaving room for your code. That comfortably fits convolutional and fully connected policies over the local view (the example network has about 7,000 weights and runs a forward pass in well under a millisecond). Large networks (ResNets, transformers) do not fit. Prefer small, efficient architectures.

papersseum.nn: the building blocks

All functions work on one sample in PyTorch layouts, so a state dict maps straight across: conv2d(x, w, b, stride, padding) with x: (C, H, W) and w: (out, in, kh, kw); linear(x, w, b) with w: (out, in); relu, leaky_relu, tanh, sigmoid, softmax; max_pool2d, avg_pool2d, global_avg_pool, global_max_pool, flatten; and inference-mode batch_norm(x, running_mean, running_var, weight, bias). Anything else (dilated or grouped convolutions, recurrent layers, attention) you write in NumPy yourself.

Tips

• Check parity first. Run the same observation through your PyTorch model and your NumPy forward pass and compare the outputs before you embed anything.
• Bake in preprocessing. Fold batch normalisation into the preceding convolution during export if you can, and keep any input normalisation inside act.
• Measure. papersseum validate reports your slowest decision. Keep it well under 50 ms, and assume a modest single CPU: your bot gets one core, shared with nothing else but not a fast one.
• Use reset() to unpack weights, precompute tables and allocate buffers, and run one dummy forward pass there to warm up: the first numpy call is slower than the rest, and you do not want it to be your first act(). Each bot also gets a single math thread, as on the platform (papersseum validate sets the same limit locally).
• Exploration noise. Your bot's random stream is seeded by config["seed"] and config["pid"].