“In the next five to ten years you should expect that whatever your average computing device is; your phone, your game console, your television, your personal computer, it could be fifty to a hundred times more powerful” says Craig Mundie, Chief Research and Strategy Officer for Microsoft. As the executive in charge of Microsoft’s long-term plans, Mundie oversees one of the world’s largest computer-science labs in Redmond, Washington. Engineers there try to imagine what the future will look like, creating mock-ups of touchscreen computers the size of living room walls and kitchen counters that can recognize pills just by their colour and shape alone. After a brief tour of their labs, I sat down with Mundie for an hour-long discussion where he outlined what he sees as the main triggers behind the next evolution in computing.

Craig Mundie, Chief Research and Strategy Officer, Microsoft

Computers A Hundred Times More Powerful

The internet as it exists today is a source of content that you download and run on your desktop, laptop, or smartphone. You may grab a website, a video, or an animation from the web, but it’s your computer that “runs” it. If you have an older computer that isn’t very fast or powerful, you know that your web browser won’t work as well and online videos and games may stutter as your computer tries to keep up.

That’s something that will change with a new arrangement that many are calling “Cloud Computing” where the internet itself will be able to take on the capability and much of the role of a computer. It won’t just be a source of content, but will be able to “run” that content too.

Microsoft sees this scenario creating a special opportunity for them to create software and applications that can draw on the power of both computers, the one in your laptop and the one that will appear on the internet through cloud computing. These programs will be able to make use of both sources of computing power in a smart and intelligent way, efficiently multi-tasking between the two to make the best of their combined use. It’s a technology referred to as “Parallel Computing” and while it has reached that level yet, today’s computer processors have a similar arrangement with dual and quad cores, taking processing power from two or more sources and working them together for a more efficient, combined whole.

“Assuming we crack the code on how to write parallel programs, and I think we will do that,” predicts Mundie, “then these machines don’t just get percentage–wise faster, they get a factor of ten or more times faster.”

The Incredible Shrinking Computer

Since their first introduction as room-sized behemoth boxes, computers have been dramatically shrinking in size with each passing year, something Microsoft predicts will only continue. The “guts” of today’s computers are made up of a number of components. You have a CPU chip to act as its brain, another chip to control graphics, and another still to handle audio, etc. Over the course of the next few years Microsoft expects we’ll begin to see these different components, one-by-one, become assimilated into a single chip.

“The whole thing collapses out into the SOCs (Systems On a Chip)” explains Mundie, “and in a black hole-like fashion it will suck in other of the computing sub-systems that are not part of a traditional computer. And they’ll become very powerful.”

As is always the case with technology, the smaller size will be compensated by the way these single chips can more efficiently multi-task and manage the different systems or cores within, allowing computers to still grow and do new things that current computer generations cannot.

“And so you have the incredible shrinking computer, in physical size as well as of consumption requirements, but you have a continued expansion of its computing capability, largely by inclusion of parallelism as the way you get speed instead of just faster clock rates”

It’s The Displays That Will Grow Bigger

Perhaps Microsoft’s most interesting prediction is that future televisions and computer displays will become ridiculously inexpensive and thin. We might think that today’s LCD and Plasma televisions are pretty narrow, the width of pencils in some cases, but Microsoft is currently playing with displays that are as thin and flexible as paper or fabric.

For many years now the industry has discussed a new technology called organic light emitting diodes (OLED) and how it could be used to create flexible displays that can be folded or even rolled up like paper. Microsoft is confident that the time for this technology is very near.

In one very dramatic example of this, John Cluts, who runs the department in charge of creating prototypes of future devices, took me into a mock-up of a teenage girl’s bedroom that had been literally wallpapered, from ceiling to floor, with computer screens. It was decorated with rock band posters and animations that had been downloaded from the internet and displayed photos, videos, and incoming messages from all of her social networks. With a simple command, the whole room could be swapped out and changed, just like the wallpaper or “theme” on your current computer.

Animated wallpaper, the teenage bedroom of the future. 

Cluts predicts we’ll be able to get away with this kind of use simply because OLEDs are very energy-efficient and because the manufacturing process will be so much easier and more dependable than LCDs or Plasmas, the cost will drop significantly as a result.

“All OLEDs have several advantages” explains Cluts. “They’re capable of being printed as a thin line, so you’re able to create big sheets of it. If you know anything about LCDs or Plasmas you’ll know that they actually create big sheets of that material and then they cut around the imperfections, because it’s very difficult to make a perfect LCD panel. In the case of OLEDs, they think they can print them and once they print that line they can pretty much print them forever with a very high level of consistency.”This has motivated Microsoft’s researchers to use screens like fabric, wrapping them as a display surface for coffee tables and kitchen counters and like paper for smaller uses, imagining screens that can be folded and tucked under the arm like a newspaper or handed out like business cards.

Inexpensive displays of the future make computers of business cards.... 

and newspapers. 

With the advent of touchscreen and gesture controls, it becomes possible to use a screen of any size and to explore the idea of computing that leaves the traditional work of desks and typing and instead explore all the simple tasks that make up the rest of our day.

Human Sensory-Like Capabilities

Our world today is quickly being populated by sensors. Accelerometers, GPS, microphone arrays, camera sensors, motion trackers, proximity detectors, light sensors, each an electronic component so inexpensive to produce you can now find them in most kinds of gadgets and devices. With an accelerometer, Motorola’s latest hands-free unit can detect when you open a car door. The latest cameras use GPS to record where you took a photograph, and the Nintendo Wii’s sensor bar allows it to follow the movements of players acting out game motions.

Today’s devices use these sensors for very simple tasks, but it’s Microsoft’s intent to allow the more powerful computers of our near future to tap into these sensory networks, to use these listening, watching, detecting sensors to collect information about the world around them and to interact with humans in a more natural fashion, in much the same way we use our own biological senses. The difference will be, that these computers will also be using these sensors to make note of subtle differences in us, the users, their primary relationship with the world around them.

“The combination of all these special accelerometers, low-cost sensors, new types of input and output mechanisms, we’re now going to be able to integrate them into one holistic way of the computer interacting with people, more like people interacting with people” explains Mundie. “With that will come an opportunity to employ computing in a wide array of tasks and frankly for a wide array of people who today just don’t benefit from computers. “

Machine Learning

If the idea of your computer having its own “senses” seems chilling, Microsoft assures us that computers are still a long way off from being able to develop human-like perceptions or minds of their own, that as sophisticated as the new sensor systems can become, and as powerful as the new computers will be, they will remain as input/output devices in use.

“People have talked for many years about Artificial Intelligence and I don’t think we’re anywhere near that” Mundie explains. “The way I think of this is that the computer is more and more near to having expert systems in it.”

Today’s computers function under a set of commands or rules that we give them. Microsoft is working to transition that to a point where a computer can create its own rules for itself. A computer would simply record lots and lots of data about a specific task its user performs often and then analyze that information for patterns or statistical behaviour that would adjust the rules it operates under to customize that task in a way the user likes it to be done.

“The analogy I give is to think about someone like me who has, in business, a great personal assistant” explains Mundie. “I mean, I couldn’t do my job without actually multiple great assistants, one who handles all the travel, one who handles all the appointments, etc. Anyone who has had one and worked with one, you realize that the longer they work for you the more valuable they are because they have a lot of learned context.”

Microsoft wants to create a computer that can uses its sensors to follow the way we perform tasks and to “notice” the subtle ways we do things differently from everyone else, and then adjust its use to match.

Today’s version of speech recognition is an early example of this where the longer the software has to listen to your voice, the better it can adjust its recognition abilities to adapt to your personal speech patterns.

Microsoft can use a number of different machine learning methods to achieve this effect; pattern-matching, Bayesian inference systems, but in each case they would be limited in use to one specific task. A computer could learn to sort your e-mail the way you like it, but could not apply that same learning behaviour to selecting recipes for dinner. A separate learning system would have to be created for each task. When you include a number of these systems within a computer, then it may seem like your computer is becoming more and more “intelligent”, but Mundie promises it won’t.

“I think it’s the brittleness of the rule-based systems that have been so unsatisfying to people in the past,” adds Mundie, “but when you get into these statistical systems, they’re complex enough to distinguish fairly subtle patterns of difference in your behaviours and to emulate them because it’s all statistical and people find that a bit more natural.”

While computers may be limited in the number of tasks it can learn, those specific tasks can themselves grow sophisticated in use. When John Cluts took me into his department’s Home of the Future, a mock-up of an actual house they use to play with futuristic systems and devices, he was greeted by “Grace”, a female voice emanating from the very walls themselves. By tracking John’s movements Grace knew it was him that had walked through the front door and promptly read off a summary of notifications and alerts it knew he’d want to review.

When Grandma stays over, the change their daughter's bedroom to this "theme"

Amongst them was a report on his grandmother. The computer in her apartment had been monitoring her movements and used pattern-matching to compare them against her movements of past days. Having suffered some health issues, John had the computer in his house connect with the one in her apartment in order to check on her. “Grandma is having a normal day”, Grace reported, meaning that she was up and about and going through a variation of her daily routine, no changes of concern. Although this being a simulation, Cluts admitted he wasn’t sure what those kind of worried changes might be yet.