Sony Presents AI for Predicting Game Frames

Sony has filed a patent application for an artificial intelligence technology capable of predicting the next frame in a video game and displaying it before the graphics processor does.

Unlike traditional frame generation methods that create intermediate images between two real frames and increase latency, the new approach does not require waiting for full rendering.

The model analyzes the already displayed frame and the latest commands from the user's gamepad to instantly generate a prediction of the next image. This allows reducing system response time.

The process involves several stages: an encoder compresses the current frame into a compact data set, and a decoder reconstructs the next image from it for display output.

• Decoding is performed based on compressed data from the current frame
• The system can generate multiple future frames simultaneously
• Model training occurs automatically on video streams without manual labeling

An alternative to this technology involves using pixel blocks that move along motion vectors without neural networks. Developers promise stable frame rates in resource-intensive games and no freezes.

Currently, the application lacks precise data on performance gains and image quality. In demonstration examples, predicted frames may lose some details compared to the original.

The technology complements Sony's strategy of implementing AI upscaling on its consoles, indicating potential future developments in games where part of the graphics are generated by neural networks in real-time.