Cartwheel generates 3D animations from scratch to power up creators

You can also check out live 3D examples on their site, like this boxing one or a little dude doing a solo waltz.

The model they built is totally original, said co-founder Andrew Carr, also the company’s chief scientist.

“We have multiple data sources, all ethically sourced, and our own labelers labeling these motions,” he explained. “Motion is represented as a matrix — this is well-known in the literature — a matrix with poses, time, velocity and so on. So you associate the motion matrix with a text description of the motion, and do pretty standard training on motion-language pairs, the same way you’d generate images or video.”

The animations you get are “on average about 80% of the way there,” Carr estimated. Jarvis said it can produce impressive, professional-level results, “and sometimes it’s a miss.” But it’s immensely faster and simpler than a traditional animation workflow, especially in environments where you have multiple artists working on one process and even small tweaks have to go up and down the line.

The models they use aren’t even that big, meaning they’re cheaper to run and could conceivably be locally hosted.

“Actually, this is really cool,” Carr said. “For a video model, you’re predicting 2,000×2,000 pixels, every frame for 60 frames per second … that’s just such a massive thing it has to wrangle. What we’re predicting is an order of magnitude or more smaller; it can run on the CPU, or older GPUs, and we can train models faster.”

Jarvis even suggested they might eventually be able to render new or modified animations on the fly, a holy grail for interactivity in games, where characters are generally limited to a set of canned movements and dialogue lines. Things like camera movements and angles can also be intuitively described, and non-human characters are in the pipe as well — though the startup is first focused on human animations, since they’re by far the most commonly required.

It’s reminiscent in a way of how Wonder Dynamics greatly simplified the process of 3D character insertion, with a focus not on replacing animators or artists but on skipping the repetitive drudgery of elementary animation work. The industry as a whole has embraced AI-adjacent tools as time savers that let their creatives focus more on creation; certainly Autodesk has, at any rate, as shown by its acquisition of Wonder Dynamics two weeks ago.

It may be a little unseemly to speculate about the ultimate fate of a startup right as it’s starting up, but Cartwheel could conceivably go the same way: snapped up as a promising and powerful feature that could prove an advantage over competitors. But it could also build itself up as an platform-agnostic tool and offer a growing suite of services to professional animators.

Either way, it has secured its first round of funding, a $5.6 million seed led by by Accel, with participation by Khosla Ventures, Human Ventures, Heretic VC, MVP Ventures, Correlation Ventures, and Pelion VC — and a handful of angels, as is tradition.

For now Cartwheel is happy to act as one of the many prosumer type apps that animators rely on to do their work. You can test it yourself if you sign up for the beta.

“There’s this notion of AI replacing creative work, and as someone who does creative work, it’s like, no! This leads to more animation, more motion, one person doing more,” Jarvis said. “And that will eventually go up the stack to the big studios that will be doing incredible stuff we haven’t even thought of. But between the Pixars and the people playing on their phones, there’s a whole bunch of levels, and that’s where a lot of the creative work really gets done.”