A 211M-param transformer acts as a game engine. It draws every next frame from the last 2.4 seconds and my inputs.

Built a simulator of 18 Backrooms levels in PyTorch to generate the data.

Recorded 440 hours of scripted play in it.

Trained the model from scratch on that, using flow matching in a pretrained autoencoder's latent space.

The model was trained on its own imperfect frames as context so it doesn't drift right away. It recalls a few older frames by position, so a quick look back usually works.

Inspired by Oasis Minecraft.

To try it you need an NVIDIA GPU with ~4 GB.

Weights: https://huggingface.co/yelarys/noclip

Video with sound: https://x.com/yelaryss


Comments URL: https://news.ycombinator.com/item?id=50594070

Points: 1

# Comments: 0