Recently I ran across this post on a Microsoft blog talking about “natural user interface (NUI) technology”. The idea is fairly simple — instead of having to interact with computers through not-so-intuitive and sometimes even counter-intuitive devices like a computer mouse or trackpad, soon users will be able to control their computers doing natural human movements such as gestures, whole-body movement, eye movement, and perhaps even such things as facial expressions.
This is not pie in the sky, it is basically here now in the form of Xbox 360 Kinect. Unlike Nintendo’s Wii, the Kinect senses your movements without the use of a controller. It is, in other words, watching what you are doing instead of you telling it what to do by waving or manipulating something. But this, as Microsoft puts it on their blog, is just the beginning:
Even with all the excitement, the reality is that our industry – Microsoft included – has only scratched the surface of what is possible. Most people’s view of natural technology is sensory related – touch or speech – but Microsoft’s exploration into NUI is multi-dimensional. By combining sensory inputs with the knowledge of what you’re trying to do (contextual awareness), where you are and what is around you (environmental awareness), 3D simulation and anticipatory learning, we can foresee a future where technology becomes almost invisible. Imagine a world where interacting with technology becomes as easy as having a conversation with a friend.
This news, along with MIT’s Recompose, seems to presage a new age of entirely new ways of interacting with computers. Is this a good thing? Yes, it most definitely is. Believe me on this.
No related posts.
[...] learned, from Roy Tennant, about wok that Microsoft and others are doing with natural user interfaces (NUIs). What’s an [...]
I recently met a gentleman that runs this company: http://www.affectiva.com/
Essentially the technology was developed to monitor changes in the body in order to help determine the emotions of people with autism. The technology is now being used in many different ways.
At the time he had just finished up a series of meetings with Sony (i think). They were working on figuring out if they could use the technology to change video games. So basically if you’re feeling anxious the video game would adjust accordingly.
Pretty awesome and its good to know that multiple companies are looking into the technology
I want to confess my sympathy for the commandline, that is the old interface. You cannot beat it in terms of clarity. Also a command is clearly reproducible. Gestures, thoughts, are they going to be clear, and will the computer do the same thing when I think about something (being distracted by children, traffic, iphone…).
As much as I like GUIs, but it makes me itchy not seeing what actually goes on in the background. Speech recognition is still comparatively poor.