Remember When We Used to Look At Screens?

Images everywhere could get a tad confusing and screens, for whatever their other shortcomings, do provide an organizational function. But who will need a screen when your devices do your thinking for you? JL

James Wilson and Paul Dougherty report in the Wall Street Journal:

"Smart" objects simply don't lend themselves to screens. The result will be new screen-free interfaces, driven by such things as voice commands and gesture controls, or simply automated responses to sensor data. So far, people seem to prefer these next-generation interfaces. In fact, screens came in a distant third in terms of popularity,
Say goodbye to the screen. For decades, screens have been the center of our information world -- first with computers and more recently with smartphones and tablets. But as computing power and Internet connectivity become embedded in more objects, the way we interact with devices will change dramatically. The "smart" objects belonging to the Internet of Things simply don't lend themselves to screens. The result will be a host of new screen-free interfaces, driven by such things as voice commands and gesture controls, or simply automated responses to sensor data. So far, people seem to prefer these next-generation interfaces. In fact, screens came in a distant third in terms of popularity, behind interfaces that operate invisibly and those based on natural human movements, in a large-scale study we conducted. Here is a closer look at some screen-free interfaces that could revolutionize the way we work, as well as some of the challenges companies may face as they become more widespread. Ambient Notifications. The ORBneXt, a screenless cube-like device sold by Advanced Lumonics LLC, continuously tracks any data stream you choose and changes color to notify you to take action when, say, an important email arrives or product inventory drops below a threshold. You could, for instance, program the cube to glow green when you get an email from your boss or an important collaborator. These kinds of screen-free, background notifications are essentially a way to curb the digital itches we tend to continuously scratch -- such as checking our inboxes or stock prices -- guiding us back to more productive activities. Of course, ambient notifications are limited in terms of the amount of data that is conveyed, so they can't operate independently of screens and more traditional input devices. There could also be a learning curve in remembering the color or sound or pattern that corresponds to each of your chosen notifications. Gesture Controls. Imagine wearing an armband that senses the movement of your fingers and hand so that your body itself acts as a remote control to open doors, operate machinery or nudge a robot to move a certain way. Or, think about doctors manipulating a 3-D scan of a patient's heart by gesturing as if they were holding it in their hands. Gesture controls let us interact with the world in a way that feels more natural and mimic physical interactions, if not actually allowing us to engage directly with real objects. Researchers are developing a kind of "connected paper" that wraps around packages and transmits information, such as a parcel's delivery location or time, to the cloud through a simple touch, eliminating the need for scanners and screens, and improving speed and process flow. The challenges for designers will be to make sure the user feels comfortable and in control of the input. For instance, how large or small must the motion be to register? It might make sense for these interfaces to include additional modes, such as audio feedback, to confirm that a movement registered. Wearables. As sensors and computing hardware shrink, they can be embedded in objects ranging from eyeglasses to necklace pendants to T-shirts. Mechanics and other industrial workers can wear displays that resemble eyeglasses and get real-time assistance in fixing machines. In military and industrial settings, smart textiles can monitor heart rate, body temperature, blood pressure, hydration and location. Smart jewelry and wristbands that track physical activity are already popular, and more businesses are deploying them in their workforce to potentially reduce health-care costs. A cross-industry collaboration between textile and technology companies could bring fabrics that activate phone calls so you don't have to look at the screen. Researchers at Rice University have developed a vest embedded with sensors and tiny motors that captures outside information, such as human language or critical alerts, and converts it into vibrations wearers can learn to recognize. As computing gets closer to our bodies, however, questions of privacy, autonomy and security arise. Who has access to a worker's location? How effective is it to penalize employees for not being more active? How will sensitive information such as health data collected by a smart hospital gown be kept secure? Automated Responses. This interface is invisible, working behind the scenes of more-traditional interfaces such as screens. As sensors proliferate in our environment, they monitor the output from factories and help automate lighting, heating and cooling in buildings. They track the weather, our whereabouts and our proximity to objects and other people. Thanks to machine-learning algorithms, we can now make sense of all of this data, and design systems that perform some sort of action, hewing to general protocols and preferences, but effectively acting autonomously. Smart inventory systems can suggest when and how stock needs to be reallocated between stores before levels get too low. In the office, such a system might monitor employees' use of websites and email, establishing baseline behaviors. If an employee's behavior suddenly changes, it could indicate fraud or another problem. In industrial settings, security of automated response systems is paramount. For instance, an automated railway braking system should communicate via private, not public, IP addresses. It's also critical to understand the best way to deploy systems such as those that monitor email and Internet use so as to not alienate employees. There will certainly be a learning curve, but if these new interfaces catch on, the flat screen display could become a blip in the gadget timeline.