In a revolutionary move, startup Melty Gym is leveraging the popular video game Grand Theft Auto V (GTA V) to train robots for various tasks. Founded by David Miller in 2026, Melty initially focused on modifying and combining video games. Now, they have applied this technology to robotics, creating Melty Gym - a platform that enables researchers to integrate their robot models into existing games and customize environments, objectives, and interactions for training and evaluation. During its November 13 launch demonstration, humanoid robots were observed attempting to open cars, drive through virtual cities, and work in data centers inside GTA V. This approach allows AI models to learn through trial and error without the need for physical robots for every attempt.
However, it is crucial to note that a robot successfully completing a task within a video game does not guarantee its ability to perform the same task in reality. Physical objects, friction, balance, sensors, and unpredictable surroundings pose challenges not replicable in virtual environments. Melty Gym is currently in research preview, with no published results demonstrating how reliably skills learned in these virtual environments translate to physical robots. Despite this limitation, Melty Gym presents an innovative approach to robot training, raising questions about the potential for virtual gaming skills to translate into real-world abilities.
David Miller, founder of Melty, a Y Combinator Fall 2026 startup, has taken an innovative approach to robot training by utilizing video games as virtual labs. Melty Gym, their latest project, allows researchers to insert their own robot models into existing games and customize environments, objectives, and interactions for training and evaluation. In its November 13 demonstration, humanoid robots were seen navigating through GTA V's virtual city, opening cars and driving, as well as working on server racks in a custom-built data center environment. This method enables AI models to learn through trial and error without the need for physical robots for every attempt.
It is important to note that a robot's success in a video game does not guarantee its ability to perform the same task in reality due to physical objects, friction, balance, sensors, and unpredictable surroundings. Melty Gym is currently under research preview, with no published results demonstrating how consistently skills learned in these virtual environments apply to physical robots. Nonetheless, this approach offers an intriguing new method for robot training, raising questions about the potential for virtual gaming skills to translate into real-world abilities.
David Miller, founder of Melty, a Y Combinator Fall 2026 startup, has introduced a groundbreaking platform for robot training - Melty Gym. This innovative project transforms video games into virtual labs, enabling researchers to integrate their robot models and customize environments, objectives, and interactions for training and evaluation. During its November 13 demonstration, humanoid robots were observed attempting to open cars, drive through cities, and work on server racks inside GTA V. This approach allows AI models to learn through trial and error without the need for physical robots for every attempt.
It is essential to acknowledge that a robot's success in a video game does not necessarily mean it can perform the same task in reality due to physical objects, friction, balance, sensors, and unpredictable surroundings. Melty Gym is currently in research preview, with no published results demonstrating how reliably skills learned in these virtual environments translate to physical robots. Despite this limitation, Melty Gym presents an exciting new approach to robot training, raising questions about the potential for virtual gaming skills to translate into real-world abilities.
