01 — Live agent

Tell it where to go.

Name a place on de_dust2, and the agent sets off like an assistant answering a prompt. It tells you when it arrives, and if you change your mind halfway, your next message stops it and sends it somewhere else.

Live view of the agent on the map
Waiting for the arena…
Idle
1 2 3 switch camera M callout map Scroll over the view to zoom The minimap marks the goal and the region to go through
Agent
Mesocort Language Action Model
Idle
Try “rush B site” or “go to A site through short”. Send a new message any time to change course.
Moving
02 — Inspiration

An agent you can talk to, inside a 3D world.

Inspired by SIMA 2 from Google DeepMind.

01

Talk, don't script

Commands are plain English: “rush B”, “go to A site through short”, “take long to A”. A small parser picks out the place for scoring and the minimap, but the policy reads the words themselves.

02

Words in, keys out

The agent acts through the same controls a player has: forward, strafe, walk, duck, jump, and turning the camera. Ten times a second it decides from the command and what it senses. The goal is never given to it; it has to come from the words.

03

Interrupt any time

Like a chat assistant, it reports back when it's done, and you can cut in. A new message stops the run where the agent stands and starts the new one from there.

64 HzSource 2 movement simulation
10 Hzdecisions, each one keyboard and camera action
256agents training in parallel, 2 × 128
~2,000training images rendered a second
03 — Architecture

From words to footsteps.

A compact language-conditioned policy in two variants. A frozen T5 reads the instruction; the agent senses the world either through the rendered frame or through its own state. A small transformer combines the two, and a recurrent memory turns them into one keyboard-and-camera action at a time.

Input

Words + senses

The instruction, and either the agent's view at 112 px or its state: position, heading, velocity and range rays.

Encoders

T5 + DINOv2 or state

Frozen T5-small encodes the text. Frozen DINOv2-small turns a frame into 65 patch tokens, or each state field becomes a token of its own.

Fusion

Transformer

Two layers at 256 dimensions let the words attend to what the agent senses.

Memory

GRU · 512

Carries what the agent has seen and done since the command began.

Output · 10 Hz

Action heads

Move, strafe, walk, duck, jump, and yaw and pitch rates, sampled on every decision.

↺
Source 2 arena — a C# simulation of CS2 movement on de_dust2, with the game's own navigation mesh and collision. It runs each action for its ticks at 64 Hz and returns what the agent senses next.
01

Practice

PPO from scratch in the arena with 2 × 128 agents, rewarded for every unit of walking distance it closes and for arriving. The critic sees the route; the actor only its own inputs and the words.

02

Grow the map

A curriculum starts with neighbouring regions and adds a region hop each time success passes 80 %, out to the farthest pairs on de_dust2. Routes through a region on the way come next.

03

Talk

This page. Every message becomes an episode, and a new message interrupts the one under way. The live view is drawn by the Source 2 Viewer renderer with a CS2 player model standing in for the agent.