Want to jump on the latest TikTok dance trend, but don’t have time to learn the moves? Now, an artificial intelligence (AI) can create a video of you dancing from a single still image.
Tan Wang at Nanyang Technological University in Singapore and colleagues, including researchers at Microsoft, developed a model they call Disentangled Control for Referring Human Dance Generation in Real World (DisCo), which splits an image into three parts: the background, the foreground and the pose of the person in the shot.
Their AI can then morph the person into a series of poses to create individual frames that, when compiled back into a video, produce realistic footage of that person dancing.
“With these things, you can try to compose anything you want,” says Wang. “If you want Elon Musk to dance, you can just use our [code].”
Read more:
An AI dedicated to drawing hands could help all the other AIs improve
The team trained DisCo on around 700,000 generic images of people taken from TikTok, so that it could learn about poses and how to separate foregrounds from backgrounds. The researchers then trained it further on a small data set of about 350 dance videos, each 10 to 15 seconds long, to give the AI a deeper knowledge of how people move while dancing.
Sign up to our The Daily newsletter
The latest science news delivered to your inbox, every day.
Sign up to newsletter
Wang says that DisCo scores higher on measures of realism than other models, such as the Google and Nvidia-backed DreamPose. He suggests it could be added to platforms such as TikTok to let people take part in dance trends, even if they can’t dance.
Reference:
arXiv DOI: 10.48550/arXiv.2307.00040
