One of the more pleasant surprises from last week’s Microsoft Build conference in Seattle was the reveal of Story Remix — a new Windows 10 app coming with the free Fall Creators Update, which drops later this year.
Windows Story Remix basically takes the photos and videos that you shoot on your iPhone or Android smartphone and automatically cuts them together into a movie, complete with cinematic-style film transitions and a soundtrack. It’s like Google Photos’ artificial-intelligence powered automatic slideshows, kicked up a notch.
And if you don’t like what it came up with automatically, you can either have it try another edit, adjust the algorithm so it, say, makes it look more like an action movie, or just edit in manually. Then, in editing, you can add all kinds of special effects, like Snapchat’s famed camera filters, but over existing footage.
It’s hard to get across in text, so here’s Microsoft’s demonstration of Story Remix:
Chris Pratley, the Microsoft exec behind the app, demonstrated the app for Business Insider last week. In just a few moments, he took footage of a child in a wizard robe waving a stick around, added a dragon, and then a lightning effect on the wand. As the kid waved his wand, the lightning effect tracked with it, making the effect more convincing.
“Now, is it perfect like Hollywood? No,” says Pratley. “But for something that takes four seconds…”
The genesis of Story Remix was “a confluence of several things at once,” explains Pratley. First, people take lots of pictures on their phone. Second, PCs are getting more powerful. Third, artificial intelligence is rapidly maturing and making automatic photo and video recognition possible.
With those factors in place, Pratley says, his team thought “I guess we should do something revolutionary.”
And the effort speaks to Microsoft’s new focus on making Windows 10 a real friend, not foe, to Apple and Android smartphones.
The idea, Pratley says, is that while people take lots of pictures on the phones they always have on them, they’re not really ideal for doing heavy-duty photo and video editing — while apps like Snapchat and Facebook let you add pre-made video effects to short video clips on the fly, processing special effects and doing more in-depth video editing takes a ton of power. Even assuming your phone has the horsepower, it’d still be a huge drain on your battery.
Meanwhile, he says, PCs have only gotten more powerful, with even cheaper models getting beefier processors and graphics hardware over the years. And yet, much of that processing power goes largely unusued, “except in gaming,” Pratley says. The idea with Windows Stories Remix is to really show off what a PC can do, Pratley says.
Finally, artificial intelligence has come so far, so fast, that it really makes it possible not only to do that automatic movie editing and cutting, but also to do the special effects — a feat of artificial intelligence that requires the system to recognize, say, a soccer ball, and apply a “fireball” effect as it soars into a goal.
With that AI, “I don’t know if [the possibilities] are endless, but there’s certainly a lot of them,” says Pratley.
Combine all of those, and you have something that takes photos from the smartphone and uses the PC to automatically do something with them that would be much more difficult than otherwise. With Microsoft boasting how “Windows 10 loves your devices,” this is an important point of proof.