Posted November 20, 2015

A 3D touchscreen made of drones

Here's a crazy idea. Instead of putting on virtual reality goggles and moving around in a virtual world, you could rebuild that world in real space. Oh yes, and use flying building blocks. An early version of this has been demonstrated by Queens University of Canada.

Let's first look at the video demo, so you know it's no joke. You'll notice the demo is very crude but of course it's a proof of principle. It's easy to imagine more building blocks, smaller ones, and other advancements that make it more lifelike and practical.

Missing flash plugin. Please download the latest Adobe Flash Player:
Get Adobe Flash Player

So to sum up: you can have drones represent pixels or volume elements. You can position them anywhere in space to create some model. You can adjust the position of drones manually, rotate your model and even pinch-to-zoom in or out, just like you do on a multitouch screen. Then you can click on a drone to select it, create a group and select the group, and the perform some kind of operation like changing the color. There's even a flying 2D touchscreen that can be used to do more traditional things like present menus or have a video call. It works thanks to a camera system that monitors the location of all the parts of the system, including the user's hands.

It's an idea of prof. Roel Vertegaal, leading researcher at the Human Media Lab at Queen's University in Kingston, Ontario. He calls the system BitDrones and he showed it at the ACM Symposium on User Interface Software and Technology in Charlotte, North Carolina earlier this month. 'Flying interactive programmable matter,' 'real reality interface' and 'physical self-levitating building blocks' were verbal innovations used to describe what was there. Vertegaal, who was born in the Netherlands, sees possible applications in 3D modeling, gaming, molecular modeling, medical imaging, robotics and online information visualization.

A fun feature is that during a video call the drone representing the other person can reproduce the other person's head movements. This enables this person to (virtually) move around in the model his display drone is part of.

The Human Media Lab is now working to upgrade the system to incorporate thousands of drones. Will these make a deafening noise together? Vertegaal doesn't think so: 'The smaller they get, the less noise they make.' Aso, as soon as you have plenty of drones, flight time will no longer be a problem as you can allow a certain number of drones some time off to recharge while other drones replace them.

Vertegaal's long term goal is to have an interface that makes VR goggles and things like Microsoft's Hololens obsolete. 'The inventor of virtual reality, Ivan Sutherland, used to say the ultimate display is one in which, if you would recreate a bullet there, it would be fatal.'

About Herbert Blankesteijn

Herbert Blankesteijn is a technology journalist from the Netherlands who has written for many prominent Dutch newspapers. He presented and directed television and radio programmes and has 10 books to his name. Herbert is interested in nascent fields such as 3D printing, drones, robotics and the private space business.