Sparse Coupled Placement Transformer with Reinforcement Learning — a two-phase
policy-gradient system for automated PCB component placement.
Overview
SCPT-RL trains a cross-attention transformer policy to place PCB components
sequentially on a discrete grid. Training has two phases:
BC warm-start — behavioural cloning from expert placements embedded in
existing KiCad .kicad_pcb files.
PPO-EAL fine-tuning — on-policy PPO with an augmented-Lagrangian penalty
that enforces hard constraints (clearance, HPWL, partition-cut budget) without
discarding feasible policy updates.
The Rust crate (pcb_parser) provides high-speed DRC, HPWL, and net-topology
primitives via a PyO3 wheel.
Reports: mean reward, BC eval loss, constraint costs, per-board breakdown, and
the Lagrangian multiplier values frozen in the checkpoint.
Note on Metrics: BC eval loss measures distance from expert demonstrations
under teacher-forcing supervision (cross-entropy of the policy's logit for the
expert cell), whereas rollout reward measures autonomous policy placement quality
under greedy execution. These two numbers are deliberately decoupled: a
policy can improve its greedy reward while BC loss increases (it has found a
better strategy than the expert), or vice versa. Do not use them as proxies
for each other.
lambda_curve.png — Lagrangian multipliers per constraint
phi_c_curve.png — constraint violations with feasibility boundary
policy_loss.png — total PPO-EAL loss
bc_loss_curve.png — BC pretraining loss
placement_heatmap.png — where the policy places components on the grid
Architecture
SCPTPolicy
Cross-attention transformer: grid cells (queries) attend to placed components
(keys/values). The asymmetry matches the problem structure — "where should
component c* go" is a query-to-context lookup, not a self-attention problem.
Illegal grid cells receive −∞ logits before the Categorical is constructed.
PPO-EAL
Standard clipped PPO objective augmented with an augmented-Lagrangian penalty
for constraint satisfaction:
Dual variables λᵢ are updated with momentum-smoothed ascent after each PPO
epoch. Action masks are stored in the rollout buffer at collection time and
never recomputed during the loss pass (prevents mask-staleness bugs).
Rust bridge
pcb_parser (PyO3) provides:
load_kicad_pcb(path) → JSON string (SCPT IR)
clearance_cost(json, min_spacing) → float
hpwl_incremental(json, ref_des) → float
The bridge is a boundary, not a layer — no logic, no caching.
No comments yet. Be the first to ask about this board.