A visual narrative of Head-Mounted Display Augmented Reality (HMD-AR) for Dismounted Soldiers

These images have graced pages of NYTimes.com, Gizmodo.com, and a few gazillian other web sites and yet we never posted them on our own blog for you to see.

Ooops!

Without further ado, we would like to present you with a visual narrative (storyboards) for how we envision HMD-AR as used by dismounted military personnel.

Opening Scene

Gear Overview

Split Perspective

Team View

Environmental Documentation

Markup Demonstration

Weapon and Point Selection

Passive Object Selection (Eyetracking)

Target Assignment Alert

Cultural Dilemma

Team Alert

Telepresent Assistance

Launch the Drone

Birds-eye View

Historical Record

Display Map Gesture

Moving In

Positions Taken

Assess Building Layout

Entering the Building

Thank you.

Defining Augmented Reality (HMD-AR) and why it will be a performance enhancing technology

Augmented Reality
With our recent media coverage and the fine work of everyone aggressively exploring the field of Augmented Reality  we get a ton of questions about what “Augmented Reality” actually means and why should we care. So here goes…

The definition is tricky because there are a few interstitial technologies out there garnering the lion’s share of attention and purporting to be AR (i.e. Mobile Phone AR). To differentiate slightly, I’ve been calling our work Head Mounted Display Augmented Reality (or HMD-AR).

HMD-AR requires some form of head gear, like glasses (or a SCBA mask) to display computer information over the world you see. It embeds persistent, contextually aware data and command services that are needed and available without the need for a fixed terminal or other interaction system as the arbiter. You never have to look down or away. HMD-AR is always visible, floating in or attached to your world. The data it presents is relevant to you and your surroundings at any given moment, and is the bare-minimum only. Why minimal? Well, our world is cluttered enough, right? If you happen to be working in a life-or-death field, it’s pretty important that you see everything. Besides, the term “augment” means “to improve” right? ;-)

To embrace HMD-AR, conventions need to be revised. We must discard the traditional, legacy control systems like QWERTY keyboards, mice, and monitors and re-envision display metaphors like folders, buttons, and pages to become objects that are much more natural and intuitive when placed in our physical environment. It is the enhancement of our environment with cues and controls that are digitally connected. It is about enhancing our performance through the use of connected digital systems.

Point
Here’s an example of a powerful use for HMD-AR. Consider the power of pointing. When you want to tell someone the location of an object, the simplest, most effective, and most intuitive way to do it is to point. Writing an email or verbally describing the location takes longer and introduces all sorts of interpretation errors. HMD-AR allows us to fall back to the common-sense approach to things.

point

In our first responder work, we developed “point” technology that allows first responders to point at locations, people, etc as a way to communicate interest. When the first responder points at something the system understands the position and direction of the pointing arm as well as the first responder’s physical location and highlights the first physical object that intersects with the “point vector.” This information is instantly conveyed to others’ HMD-AR displays (shown as “point highlights” on the visible world) and even to the site commander on a command map. Pretty useful, right? That’s the (errr…) point.

While the emphasis industry wide has been on visualization approaches, Augmented Reality also includes aural, and haptic modes. Sound and feeling are also key communication systems available to developers as we advance this technology. The secret to the success of HMD-AR; for it to become widely accepted it must be much more than game or entertainment medium (although those will be cool too). It, first and foremost, must be about performance enhancement.

Design Engineering and embedded computer prototyping – for hire!

Oh ya! We build hardware prototypes.

Cool ones, things like sensors that transmit data wirelessly to a computer or smartphone. Or synthetic vision robots that guide autonomous air vehicles. Even our very popular DIY Thermal Scanner. Most of our work is based on the vast repository of knowledge available through the open-source hardware movement and it seems to be a very popular service with our clients. That makes sense, if you think about it. Everything has a computer in it these days, even my watch has an IP address…

The service is called Design Engineering. It’s not new to the world or us (except we are now pimping it to our customers).

From Wikipedia:

The design engineer usually works with a team of engineers and designers to develop the conceptual, preliminary and detail design and the most critical parts. He/she may work with industrial designers and marketing to develop the product concept and specifications, and he/she may direct the design effort from that point. Products are usually designed with input from a number of sources such as marketing, manufacturing, purchasing, tool making and packaging engineering. In addition design engineers deal with much more complex technological and scientific systems (aircraft, spacecraft, rockets, trains, ships, dams, bridges, building structures, urban infrastructure, machinery, production systems, propulsion systems, oil, gas, and mining exploration systems, manufacturing processes, military systems, cars, electronics, computers, power generation systems – nuclear, fossil, wind, ocean, and power distribution systems).

(They don’t mention “Shark-Mounted LASERS” but we digress)

Here’s a better way to say it: You come to us with a cool idea to build a little electronic widget that connects with or expands your product offering. Perhaps a pedometer for squirrels, for example. We then research the components and build you a prototype device (sometimes in as little as a week!) and work with you to miniaturize it and put it into production. That means we collaborate with Industrial Designers, User Experience Architects, Graphic Designers, Electrical Engineers, Product Managers, Software Developers, and on the rare, yet exciting occasion the FCC to bring together your new product.

We are continually impressed by the amazing ideas people bring us and we always have fun making them real.

Do you have an idea for a cool new product? Drop us a line and let’s get started!

CNN features Tanagram’s Vision for the Future of Firefighting

For those of you who missed our CNN special segment, here it IS!!!

A month back, I was honored by a visit by the folks at CNN to discuss the work we had done and some that is currently underway. While this video only showcases our past concept work it gives another unique understanding into the need for the gear we are building. Our vision for specializing in Augmented Reality solutions for high risk areas is no accident. Saving lives is THE BEST way to socialize augmented reality and there is a ton of opportunity. Currently, we have applications for first responders, and military uses and are looking for a partner to expand into public safety.

I’d like to express my sincere gratitude to Reynolds Wolf, Ryan Bergeron, and all the fantastic folks at CNN for sharing my work with the world!

Transhumanism & the eyeborg documentary

Rob Spence is a cyborg. He lost his eye in a shotgun accident and a group of engineers made him a camera replacement. Rob, a cyborg consultant for Eidos Montreal, made this documentary featuring the state-of-the-art in comparison with the future vision presented in the recently released Deus Ex – Human Revolution.

A compelling piece of marketing, this video will blow your mind with the tech Rob features. The future is bright for those of us with physical injuries or disabilities. Will we ever get to the point where we buy cybernetic augmentations because they perform better than our existing faculties? I think yes. Begin your debate below!

Oh! and YES Tanagram’s Firefighter Augmented Reality Mask is featured in this video. I hope you enjoy it.

Return top