ptcg-2ply-bounded-search

Pokémon TCG — 2-ply Bounded Search on BC Agent

Why search on top of BC?

Pure BC is reaction-based: it picks the highest-probability action at the current state. It cannot look ahead. The 1st-place LB 1204 model uses "RL + bounded search" — search is the structural advantage.

A BC model already has a value head (trained alongside the policy head). Re-encoding a hypothetical future state and reading the value gives a leaf evaluation — usable for any-tree, expectimax, or simple best-of-K scoring.

Algorithm (validated pipeline, v8)

def agent(obs_dict, deck, model):
    obs = to_observation_class(obs_dict)
    if obs.select is None: return deck
    options = obs.select.option
    if not options: return []

    # 1. Root forward (one shared pass)
    enc_idx, enc_val, enc_off, opt_t = encode(obs, deck, options)
    with torch.no_grad():
        root_logits, _ = model(enc_idx, enc_val, enc_off, opt_t)
    root_probs = sigmoid(root_logits)  # [N_opt]

    # 2. For each candidate option i:
    scores = []
    for i in range(N_opt):
        sample_scores = []
        for _ in range(N_SAMPLES):              # = 5
            opp = sample_opponent_hidden(obs, deck)  # SNORLAX pad
            ss = search_begin(obs, ..., opp)         # root branch
            # 2a: my move i
            s1 = search_step(ss.search_id, [i])
            # 2b: opponent's BC-greedy response
            opp_choice = bc_greedy(model, s1.observation, deck)
            # 2c: opponent's move
            s2 = search_step(s1.search_id, opp_choice)
            # 2d: leaf value
            leaf_v = encode_and_value(model, s2.observation, deck)
            sample_scores.append(root_probs[i] + GAMMA * leaf_v)
            search_release(ss.search_id)
        scores[i] = mean(sample_scores)

    # 3. top-k by score
    return topk_indices(scores, n_select)

Parameters (validated defaults)