About the bot
At the “Neural” level you play a neural network with 0.84 million weights, a card transformer. Nobody showed it how to play well. It only knew the rules and played 33 million hands against itself.
- 33M
- hands of self-play in training
- 3.6 h
- training on one graphics card (RTX 4090)
- ≈ 20 ms
- thinking time per move, in the browser
- 56.3%
- of hands won against “Strong”
Three bots with rules, one without
The Easy and Medium levels follow rules of thumb: lead cheap cards, keep pairs together, give up high cards only for tricks worth it. Strong adds a search on top. For each move it samples hundreds of possible distributions of the unseen cards, plays each one out and picks the move with the best average. Once no more cards are drawn it calculates exactly.
Neural does not search. The network looks at the position once and gives a probability for every legal move. Its judgement comes from training, not from rules a person wrote down.
What the network sees
A real example from self-play. Diamonds are trumps; the hand holds ace, ten, king and queen of trumps, plus the ten of clubs and the king of spades. The trump nine lies face up, and two tricks are already won. The network gets no pictures, only numbers: for each of the 24 cards, whether it is in the own hand, already played, face up or still unknown.
Your hand
Face-up trump
Stock: 8 cards
Points 10 : 0, tricks 2 : 0
The card inputs: six rows of 24 cells
Each cell is one card. Filled means 1, empty means 0. Trump is always on the left.
| ♦︎ | ♣︎ | ♠︎ | ♥︎ | |||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| A | 10 | K | Q | J | 9 | A | 10 | K | Q | J | 9 | A | 10 | K | Q | J | 9 | A | 10 | K | Q | J | 9 | |
| Own hand | ||||||||||||||||||||||||
| Already played | ||||||||||||||||||||||||
| Shown by opponent | ||||||||||||||||||||||||
| Face-up trump | ||||||||||||||||||||||||
| Card led | ||||||||||||||||||||||||
| Still unknown | ||||||||||||||||||||||||
On top come four more card rows (what the opponent may still hold, what they played, the last trick), 47 numbers for stock size, score, tricks, closing, the meld obligation and pairs in hand, and the whole trick sequence: which card fell in which trick, who led and who won.
The network’s answer
Probability for each legal move. Value of the position: +3.0 game points.
Two tricks make learning easier. The cards are rotated so that trump is always the first suit: whether hearts or spades are trumps changes nothing about the game, so the network learns each situation once instead of four times. And the network is not told which of the other three suits is which, only rank and whether it is trump. Every situation is therefore the same for all three suits.
Architecture
The network is a small transformer, built around the cards. Each of the 24 cards gets its own slot with 31 features: where it is, whether the opponent may still hold it, and its history, meaning in which trick it fell, who played it and who took the trick. A 25th slot holds the game state: stock, points, tricks, closing.
In 4 blocks the slots look at each other (attention). Each card knows which cards share its suit and which card it met in the same trick, and learns by itself how much that matters. At the end every card scores its own move, king and queen together score the meld, and the game-state slot decides on exchanging and closing and estimates the game points the hand will end with. That number is what the evaluation chart in the game shows.
In the browser the network runs in plain JavaScript, with no library and no server. The weights are stored as 8-bit numbers inside the game file (1.2 MB). The rounded weights choose the same move as the original in 99.0% of positions.
How much accuracy does rounding cost?
| Bits per weight | same move as the original |
|---|---|
| 8 | 99.0% |
| 6 | 97.1% |
| 5 | 93.5% |
| 4 | 87.5% |
24,361 positions from self-play.
Training by self-play
Training used PPO (Proximal Policy Optimization), a standard reinforcement learning method. In each of the 4,000 rounds the graphics card plays 8,192 hands at once. Half of them against the current network, the other half against up to 12 older versions, so it does not develop a weakness that only it fails to exploit.
The only feedback is the result: game points won or lost, 1 to 3 per hand. Which moves led there, the network has to work out itself. The learning rate falls from 0.0003 to 0.00001 over the run, so the network fine-tunes at the end instead of jumping around.
Win rate against the three rule-based bots during training
Measured every 200 rounds: 250 hands against Easy and Medium, 1,000 against Strong.
Values as a table
| Round | vs Easy | vs Medium | vs Strong |
|---|---|---|---|
| 200 | 77.6% | 59.0% | 50.0% |
| 400 | 78.1% | 62.9% | 53.4% |
| 600 | 75.8% | 65.1% | 55.0% |
| 800 | 74.9% | 64.1% | 54.6% |
| 1,000 | 74.4% | 64.4% | 55.5% |
| 1,200 | 78.0% | 64.3% | 55.6% |
| 1,400 | 75.5% | 63.3% | 53.7% |
| 1,600 | 76.8% | 64.1% | 53.1% |
| 1,800 | 75.6% | 64.3% | 55.1% |
| 2,000 | 76.4% | 66.5% | 55.9% |
| 2,200 | 77.8% | 62.6% | 54.6% |
| 2,400 | 76.0% | 63.5% | 54.8% |
| 2,600 | 75.7% | 63.9% | 55.1% |
| 2,800 | 78.4% | 65.3% | 56.2% |
| 3,000 | 76.6% | 65.8% | 55.3% |
| 3,200 | 76.6% | 63.3% | 55.7% |
| 3,400 | 75.8% | 61.3% | 55.3% |
| 3,600 | 80.5% | 63.4% | 56.0% |
| 3,800 | 76.2% | 64.8% | 56.0% |
| 4,000 | 75.8% | 64.2% | 56.2% |
What the network picks up: actions per hand in self-play
Average over both seats, measured every 10 rounds.
How strong is it?
Everyone against everyone, 1,000 hands per pair. Each deal is played twice with seats swapped, so luck of the deal cancels out. Read a row: how often this bot wins against the opponent in the column.
| row vs column, wins in % | Random | Easy | Medium | Strong | Net at 400 | Net at 1,600 | Net final (Neural) |
|---|---|---|---|---|---|---|---|
| Random | 19 | 8 | 7 | 7 | 5 | 6 | |
| Easy | 81 | 30 | 21 | 22 | 24 | 23 | |
| Medium | 92 | 70 | 36 | 35 | 35 | 38 | |
| Strong | 92 | 78 | 64 | 46 | 45 | 44 | |
| Net at 400 | 93 | 78 | 65 | 53 | 47 | 47 | |
| Net at 1,600 | 95 | 76 | 64 | 54 | 53 | 51 | |
| Net final (Neural) | 94 | 77 | 62 | 55 | 52 | 49 |
Does it play differently from the search bot?
Both win about equally often against the weak levels, but not in the same way. Measured over 13,000 hands per bot:
The bot as a coach
The network helps you too. The Hint button shows its best move, how sure it is and what the second choice would be. When a rule of thumb agrees with the network, its reasoning is shown as well.
The evaluation chart is on by default (you can turn it off under Training aids in the settings). Like in chess, it shows after each of your moves how the network rates the position, and grades your move: best, good, inaccurate or mistake. To do so the network plays through every legal move in several possible card distributions and measures how many game points your choice costs compared with the best move. Hover over the chart to see what you played and what would have been better. At the end of the hand a report lists every trick with its cards, points, the change in position and what your move cost.