Small Stories Visualizer

Each ball groups the model’s token pieces into a word. Green arrows show attention, averaged over heads and the receiving word’s pieces. Positions are a force layout, not model embedding coordinates; dragging changes only the display. In 3D, a gentle Huber penalty favors a shallow plane while allowing outliers in depth.

Orange links show positive effects on the chosen next token: the change in log probability when a prior word’s input embeddings are replaced with zeros. The affected suffix is recomputed through every layer. These effects depend on that baseline, are not additive, and do not uniquely explain a prediction. New words begin near positive support; continuation pieces stay in their existing word.

Sampling uses temperature 0.8 and top-p 0.9 with a fixed seed. Start/end/unknown tokens are excluded so playback continues until paused. At 512 tokens, inference restarts on roughly the latest 384 tokens; earlier text stays in the editor. Text is processed locally and is not sent to a server.