Abstract
Kaggriculture is a two-player farming economy hosted on Kaggle. A full game consists of about 719 decisions in which each player buys land, plants, processes, transports and sells in a market shared with the opponent. This project trains a recurrent policy by behaviour cloning on 140 complete games (100,660 decisions) played by two scripted agents, Pipe7 and V47.
The learned policy is not yet competitive. The main contribution so far is methodological. The game engine reports a match as completed even when an agent raised an error during play, so the first evaluation overstated performance. A stricter audit of 96 games found 30 invalid games, all caused by a single input-encoding limit, and no wins against either scripted agent. My best leaderboard score, 2,562.9, comes from a scripted agent and not from the learned policy.
01Problem statement
Rewards in this game are delayed and depend on chains of actions. Buying land is useful only if labour, seed, processing, transport and demand all become available before day 29. The opponent changes prices and the set of reachable options. An apparently idle move can be correct because it preserves cash for a more valuable chain two days later.
A scripted agent can execute one coherent plan very well. It cannot recover when the opponent, or its own earlier error, moves the game into a state the plan did not anticipate. The aim of this project is a learned agent that can.
02Data
Games were generated by running Pipe7 and V47 across seeds, both seats and several opponents. Each game is stored as one ordered sequence, together with the engine version, the opponent, the final cash and a hash that allows exact replay.
| Corpus | Value |
|---|---|
| Complete games | 140 |
| Expert decisions | 100,660 |
| Decisions per game | 719 |
| Seeds shared between training and test splits | 0 |
03Approach
The policy is a recurrent network trained to predict the expert's action at each decision. When training on a later segment of a game, the earlier segment is first replayed through the network so that its hidden state matches the state it would have at that point in a real game. Without this step, the model is asked to predict day 20 from a memory that has not observed days 0 to 19, and it learns an inconsistent mapping.
04Evaluation audit
One checkpoint was evaluated on 16 unseen seeds, in both seats, against three opponents, for a total of 96 games. The first summary appeared acceptable. The engine, however, marks a match as DONE even if an agent raised an error partway through. A second evaluator was written that checks every status and confirms that the agent was called at every decision.
| Opponent | Games | Valid wins | Valid losses | Invalid (agent error) |
|---|---|---|---|---|
| Wheat baseline | 32 | 28 | 0 | 4 |
Pipe7 | 32 | 0 | 18 | 14 |
V47 | 32 | 0 | 20 | 12 |
| Total | 96 | 28 | 38 | 30 |
These figures describe one behaviour-cloning checkpoint and are not my best competition result. For reference, V47 beat Pipe7 in 30 of 40 recorded games.
05Findings
An encoding limit appeared as poor strategy
All 30 invalid games failed immediately after the farm grew beyond 16 active units. The policy input has room for 16 units and the game permits more. Clipping the input would have hidden the defect, so the input format must be extended and the 30 games remain marked as invalid.
The policy generalised only to the opponent seen in training
The checkpoint was trained on games against the wheat baseline. It wins most valid games against that baseline and none against the two scripted agents. In one traced game, its first deviation from Pipe7 occurred at decision 34, while both farms were still identical. States produced by a strong opponent were absent from the training data.
Action matching does not teach strategy
Rewarding the policy for selecting the expert's action is the imitation loss expressed as a reward. It cannot reward a different action that ends the game with more cash. Querying the script for its action after the learner has left the expert's trajectory is also invalid, because the script carries hidden state (its route, pending actions and selling mode) that no longer corresponds to the game. A script is deterministic only while its internal state and the game state agree.
06Planned work: a hand-over curriculum for reinforcement learning
Applying PPO to the current policy would mainly optimise around these defects. The planned curriculum keeps the game state valid and exposes the learner to real outcomes.
- Select a seed, seat, opponent, script and hand-over day.
- The script plays from turn zero to the hand-over day.
- Control passes to the learner permanently.
- The game is played to the end, and the final cash is compared with a paired game in which the script played throughout.
The hand-over day moves earlier in stages: the last three days, late investment, operation of an established farm, the opening, and finally all 719 decisions. Each stage must outperform its paired control before the next stage begins.