Augmented or Virtual Reality?

While Augmented Reality (AR) has been buzzing in the tech industry for a few years, Virtual Reality (VR) has made a huge splash and comeback with the recent announcement of Facebook acquiring Oculus VR for about $2B.

I've heard some conversation lately about whether VR just knocked AR out of the picture, or if one is better than the other. The recent Facebook acquisition at such an enormous buying price is likely going to spawn a bit of a gold rush as people suddenly try to get on the bandwagon, and this might make it harder for AR startups to find funding (already a hard thing to do).

I think first, we need to understand the differences between AR and VR as well as how these technologies are going to be used. Both have the potential to be massive agents of change, but not in the way that most people think.

Augmented Reality

Augmented Reality (AR) is best described as a digital layer of information superimposed on what you are looking at. Most people are familiar with the term "heads up display" or HUD (as you would see in an airplane cockpit) or the digital yellow first-down line in football. That line isn't actually there, but it is added to the video feed as a layer of information.

Wikitude.jpg

Many implementations of mixed reality are referred to as Augmented Reality. Unless you are really picky like me, or an academic, its probably fine to refer to both as Augmented Reality or AR. 

I wanted to make the distinction here for a couple of reasons, but if you are new to all this, just think Augmented Reality when I say Mixed Reality and you will be fine.

Mixed Reality

Mixed Reality (MR) is essentially the same as Augmented Reality (and almost always referred to as AR) except instead of an overlay with information (for example, imagine people's names floating over their heads), the experience would have digital 3D objects placed in the environment around you, which in some cases you could interact with. Examples of this would be 3D Zombies shambling around a college campus, or replacing your head with Darth Vader's helmet while you are on a video chat with your friends. 

Mixed Reality, or the full blending of the real world and the virtual, is where I think the greatest potential for amazing things is at, and certainly (in my opinion) the coolest stuff to work on.

Virtual Reality

Virtual Reality (VR) is where you are fully immersed in a virtual environment or virtual world (VW), generally through a wearable display, like those that Oculus VR makes, much larger and complicated contraptions (see image to the right, lol), or through something like a bunch of giant screens around you or a dome that you would stand inside.

VR really captured the minds of the public in the late 80s and through the mid-90s with a bunch of very popular and high profile movies. Unfortunately, these movies set the bar of expectations as to what the current day technology could do, and VR was ultimately a huge disappointment. It eventually faded away, with barely a handful of VR evangelists still proclaiming the world-changing implications of VR.

Still though, with the right interface and experience, VR is going to be pretty mind-blowing.

Double Rainbows! What does it all mean?

Augmented Reality (both types) and Virtual Reality are pretty damn awesome and compelling when you think about it. 

AR is ultimately going to have an effect on every day life much in the way the Internet and the Web has so far. You can disagree with this all you want, but it is coming. It might to envision right now while things are still so early, but at some point critical mass is going to occur, and some tremendously amazing application in an area like education or entertainment (early on in any case) will blow the industry up and we will see another information age evolution and rapid revolution occur before our eyes. Some people today are worried about competitors in this industry...honestly, I think they are missing the point. AR is going to spawn entirely new professions and industries, and ultimately change the way we think about and interact with the environment around us and other people. Pick an industry and I'll tell you how it will be reinvented with AR.

VR on the other hand, is, at first glance, much more compelling and engaging. While AR is all about who and what is around you, VR has the potential to take you to an entirely new place (real or imaginary) where you can be anyone and experience anything. Even the rules of physics can go right out the door. Imagine experiencing the world from the perspective of a fly, or exploring something while you are no bigger than half a dozen atoms. On the other end of the scale, flying through space and seeing galaxies in the palm of your hand. What about traveling through time and seeing an event unfold, first-hand, from the point of view of someone that was actually there? Anything and everything is possible with VR, and I haven't even started talking about things like new ways to learn or interact with pure data and information. Of course, I'll be first in line to see the digital frontier like TRON did. 

Remember, AR is about the world around you, blended with digital objects, whereas VR is about an entirely different world, and all immersive. Both of these have tremendous potential and great promise, provided that we don't give up on getting there, and we can get away from horribly designed and implemented gimmicks, just for the sake of a quick buck, or trying to get on the bandwagon.

Having said that, the really smart people out there understand that there are very similar underlying technologies in both AR and VR. It may make sense to differentiate the two for marketing of some product or another, or in some academic discussion, but the real power of both is going to happen when the technologies mature to some level, and we start seeing some convergence and coadunation of both, along with other elements like the internet of things and sensors, new control devices, and a reinvention of information display (are windows and pages really the right metaphor for AR and VR? I don't think so).

Not so fast Buckaroo Banzai!

The best way I can describe the state of the industry compared to what people expect from all the marketing is to use a time machine as the example.

What everyone expects because of all the marketing hype: Dr. Who's T.A.R.D.I.S

Step right up folks!

Step right up folks!

What is actually available on the market: A cardboard box with some blinking lights, a fan blowing some air, and a questionable looking guy outside shaking the box and making sounds like  "EERrwWwoooooSHHHh" "EERrwWwoooooSHHHhEERrw" as loud as he can.

Joking aside, we have seen glimpses of what AR and VR could be like in the future mostly thanks to Hollywood. This sort of stuff is coming, it may just take a while.

Having said that, the question of whether or not AR and/or VR are just technological fads still remains for many people. It is fun for a couple of minutes and then...meh? What's the point?

Virtual Reality still has a few things to work out with the head mounted displays, the big thing being the irritating sense of dizziness and motion sickness that a lot of people experience after only a few brief minutes wearing the gear. I expect that will be conquered very soon. The other issue is that of user interface...how do we interact with these amazing virtual worlds? Right now, we have to stumble blindly using a keyboard we can't see, or some other type of hardware like a joystick, motion controller, or maybe a glove studded with sensors. This too will probably be sorted out in the near future, but for all of the glorious experiences we see with VR on TV and in the movies, we will have to wait until we can jack into our brains directly, or something else that can record impulses from our spinal cord and interpret those to control a 3D avatar in the virtual world. This by the way, is quite different from the traditional approach of trying to "read" the mind by using generic EEG sensors to identify certain brainwave patterns. This is commonly referred to as reading the mind by a lot of companies and a few recent tech toys, but nothing could be further from the truth. 

Augmented Reality has its own issues. It is one thing using printed "markers" (basically black and white patterns) or simple image recognition to tell the software where to display the content associated with those markers, but it is an entirely different beast to do full-on AR like in the videos under Mixed Reality at the beginning of this post. To do that, you have to understand the environment around you (I call this spatial intelligence) and that is a bit tricky. Even trickier trying to do that for an experience that multiple people are engaging in at the same time. More on this later in a different blog post. I have a lot to say about this : )

Needs more cowbell

needs-more-cowbell.jpg

In any case, at the end of the day, both AR and VR need "more cowbell". Even if the technology is getting where it needs to go (albeit slowly, mostly due to lack of risk capital available for innovation and R&D, but also lack of enough visionaries really trying to push things forward with ambition), there still remains the quality of the experience itself, and what the application is. 

For now, VR, in all of its glory and recent industry excitement, still lacks killer experiences. Most of what I've seen out there, is essentially just replacing the monitor on your desktop with a wearable display, and then calling it a day. VR is unique in many respects, and I don't think it will take off as much as it should until there are experiences and content designed specifically for VR instead of just being tacked on.

AR is probably in a worse position right now...the industry has been flooded with a bunch of novelty gimmicks meant to attract attention, but without giving any real substance or added value to the experience. There is plenty of hype coming from people releasing awesome concept videos that look good, but have little to do with what their tech or app actually does, and in some cases, there is little thought given to the realities of what is necessary to pull off some of the capabilities shown in these videos. I'll save my comments for what AR needs and how the tech needs to evolve to get where it needs to be for the magic to happen for another post soon.

A little critical...

I might be a little bit critical with a dose of cynicism in my posts here and elsewhere, but this is more about the state of the industry and not so much about the technology itself. Most of my frustration comes from over-marketing, get-rich-quick people, and venture capital pouring into dumb ideas and companies instead of seeking out real innovators that have the potential to make a difference. So, whatever you think, don't let me discourage you about the tech or the industry. Some amazing things are going to happen, and relatively soon.

A few companies I like to keep an eye on:

Paracosm

Occipital

OculusVR

These guys are all doing it.  Really smart people, solid technology, and thoughtful roadmaps. One step at a time, but that is the way to go. 

You won't get a bunch of razzle dazzle from these three companies or concept videos that strain the sense of credulity. There are too many of those already in the industry (and they shall remain unnamed here). It is safe to say (in my opinion) that at least half the industry is either full of it, a waste of time, or just trying to ride the hype and repacking something else and making it seem great. You know who you are!

In the Movies

In case you still don't know what AR or VR is, here are a couple of movies you can watch easily enough to get a better idea. Have fun.

AR in Movies: Terminator, Minority Report, Iron Man, Avatar, Robocop

VR in Movies: The Matrix, Lawnmower Man, Johnny Mnemonic, Virtuosity, and many more.

You should also check out Dennou Coil (anime), and read stuff like Neuromancer (one of my all time favorites), Cryptonomicon , Diamond Age, Rainbow's End, and Dream Park. There is a ton of fantastic material out there that will blow your mind. Not hard to find. Speaking of which, feel free to list your favorite movies or books that have anything to do with AR or VR in the comments. Other readers will appreciate it.

See you in the future.

AR Fiction: Forensics and Crime Scene Analysis

This was originally posted in my personal blog (Dec 17, 2009), and I thought it would be better placed here in this section. I couldn't just move it without losing the comments or the original URL, so I'm just going to copy it here. Enjoy.

- Robert

In a recent post, I mentioned that I was a contributing author to Working Through Synthetic Worlds

Anyway, the format of the book is unique. As the description says:

The editors use a distinctive format for the book, consisting of a set of chapters composed of three parts: a story or vignette that describes work conducted within a synthetic world based loosely on the question, ‘what will work be like in the year 2025?’, founded on the expert authors’ expectations of plausible future technologies; a scholarly review of the technologies described by the stories and the current theories related to those technologies; and, a prescription for future research required to bridge the current state-of-the-art with the notional worlds described in the stories.

What I wanted to do here was share part of my chapter with you, at least just the fiction part. The rest of the chapter gets into the technical analysis, scholarly review, etc. This is the unedited version (the final is much cleaner), so please ignore any glaring writing errors. I hope you enjoy the short story…I might continue it at a later date depending on reader response (I had a limited amount of space to work with in the chapter).

******

The call had arrived at a little past two-thirty in the morning and Kendra had drawn the short straw to venture out in the freezing rain that had been cascading from an inky black sky for the last several hours. She waited at the stoplight, listening to the soft hum of her electric car and the turn signal’s metronome clicking away little segments of time. She was tempted to let her mind wander while staring at the glistening raindrops snaking their way down her windows and brightly lit by the stoplight. Kendra had always felt something mysterious about inclement weather in the middle of the night, especially when there were only few people on the roads heading to some unknown destination.  The only thing that could make the atmosphere a little more surreal would be snow materializing out of eternity and blanketing the city in silence, with only the reflections of street lights free to echo through the night.

The light finally changed to green and Kendra turned her mind back to the task at hand and drove the rest of the way to the crime scene, preparing herself for her work.

Kendra arrived at the scene and shouldered her way through the overly eager press reporters that had already gathered like hungry vultures desperate for a meal. She ignored their barrage of questions and ducked under the police cordon, showing her badge to the officers attempting to control the growing crowd. This was the fourth murder in as many days and the public was beginning to demand answers. She made her way across the driveway and up the brick stairs leading to the front door of the house, dodging the growing pools of rainwater that would soak her feet if she wasn’t careful.

She entered the house and waited while most of the other officers cleared out to give her some room. The medical examiner had already been on the scene and the bodies were taken away. Kendra could sync up her databases with the examiner’s later, it was time for her to do her own work now. She set down her bag and reached up to her glasses, tapping a barely noticeable touch sensitive key at the corner of the frames. A softly glowing heads up display pulsed into existence in front of her, immediately sensing her location in the real world and on the meta-net.

“Initialize.” she commanded. “Voice authentication: Alpha-Zulu-Romeo-Bravo-Zero-Niner-Seven.”

A nondescript male voice spoke in her ear. “Authenticated, Lieutenant Kendra Jackson, Enhanced Sensory Perception, ESPer Team 5”

“Begin scanning, crime scene, domestic, murder.” She began to look at every area of the room, making a few mental notes of things she would pay special attention to when she returned for her detail and evidence collection sweep. The voice spoke in her ear again. “Location confirmed. Date and time confirmed. Scanning in process. Comparing visual point cloud to public architectural records and builder plans. Synchronizing. Process complete.”

“Display wireframe” she said.

“Wireframe active. No anomalies detected.”

Faint blue lines appeared, following the course of the walls, the floor, the ceiling, the doorframes, and every other part of the house that she could see. It was like seeing a 3D blueprint of the house overlaid on the real thing. This served multiple purposes, but for now she needed it to act as a reference for the scale, positioning, and orientation of everything in the house, particularly any evidence she would come across. She remembered hearing stories of a SWAT team that had used augmented glasses and the wireframe to navigate through a pitch black office building where terrorists had cut the power and had set off multiple smoke bombs. They never knew what hit them. After that particular incident, Fire departments all over the country began requesting the same gear. Not being able to see where you were going in a smoke filled building was no longer a problem. You simply needed to follow the blue lines or the floating translucent red arrows that pointed towards the nearest exit. Life was so much easier with augmented reality and the glasses.

“Activate WATSON” she requested, almost without thinking. She had grown accustomed to working with the artificial life intelligent agent. Originally he was not much more than a tool that helped gather information and run intelligent semantic database searches, all wrapped up in a life sized human representation. Kendra had spent a lot of time and a fair amount of her own money to customize him to her liking and upgrade his program with a more realistic personality and some law enforcement grade functionality that was not available to the public.

“WATSON is now online.” The voice stated. She wondered if she should change her settings and give it a personality or if she preferred it to be completely devoid of any emotion. Having WATSON around was more than enough she guessed.

Kendra turned her head slightly and regarded WATSON, who was now standing in the middle of the room wearing his usual Victorian garb and peering at his surroundings while muttering to himself. He was faintly translucent, which was required by law for any virtual person, human controlled avatar or AI-driven persona. There were some public safety reasons for this and people generally preferred to keep it that way, mostly for the same reasons that humanoid synthetic androids were restricted from being too lifelike.

“WATSON, access the domestic security system and copy the logs from the last 72 hours” she said to her virtual companion while she began to start walking around each of the rooms on the first floor, observing everything.

WATSON nodded and pulled a worn leather-bound journal from thin air and started to make notes in it. He stopped muttering to himself and looked quite intent on his recording task. When he was finished, he walked out of the room, waddling slightly (he was a tad overweight, and it affected his walk cycle) and caught up with her.

When Kendra completed her initial walkthrough, she returned to where she began and started collecting evidence. Every bullet and casing were recovered, lasers used to determine angle of the bullet holes in the wall, fingerprints lifted, bloodstains sampled, and so on. By the time she was done, a highly detailed 3D mirror of the entire house had been created on the servers back at the station. Every photograph she took and corresponding video clips of everything she saw were also stored, but each was tagged with appropriate time and location data. She had taken the time to attach some notes and observations to a few key items and places of interest in the house. Anyone looking at the 3D version of the crime scene would have ready access to all of the data and her notes.

As Kendra was packing up her gear and preparing to leave, she had a sudden hunch. “WATSON, display all augmented reality content and channels associated with this residence please.”

“Ah, yes yes, good idea Ma’am. Complying.” WATSON looked like he was staring off into the distance at something vaguely interesting, while a wide variety of virtual objects began popping into existence. Normally, this would only be visible to the residents and friends that have access to their channels. Some abstract art materialized on one wall, shifting patterns and undulating waves rippling across the largest piece, their smooth motion was mesmerizing. A few virtual pets popped into existence next and started scampering around playing at some learning game intended for young children. A wonderfully detailed plan for a new house slowly appeared on the dining room table complemented by a soft woman’s voice reciting a sales pitch and mentioning a number of optional additions that could be purchased.

WATSON frowned and coughed politely. “I’m sorry, but there seems to be some additional content here that is restricted. It is not listed on the normal public and private channels. It may be an illegal hack. Shall I attempt to access?”

Kendra considered for a moment. Normally, law enforcement personnel have access to all AR content, public and private, within the domain of a crime scene, but there were some levels of protection and privilege that required a court order or security clearance to access. In cases like these, discretion was usually prudent and it was safe to obtain a warrant first. Then again, there was a strong possibility it was an illegal hack and this was a murder scene, not a regular crime scene. Kendra’s curiosity and intuition got the better of her.

“Force access WATSON, on my authorization. All means necessary.”

“Yes Ma’am. I shall do my best”. WATSON’s eyebrows furrowed and little beads of sweat began to form on his virtual brow. A few long minutes passed and Kendra started wondering if WATSON had locked up…he hadn’t moved in a bit and even flickered a few times, which was extremely unusual. She was about to give up and restart his program when he lurched with a gasp and a pained look on his face.

“Ahem, that was extraordinarily difficult, but I seem to have succeeded. A spot of tea would be really nice methinks.”

“Thank you WATSON, what is the location of the hidden content?”

“In the living room I believe. Shall we investigate?” queried WATSON.

“Of course; this should be interesting.” Kendra grinned and headed towards the living room. She stopped short at the doorway and stared at the center of the room in disbelief. A massive floating black skull hovered there, blood-red fire silently blazing in its hollow eye sockets, and a nasty black ichor dripping out of its mouth on to the carpet. The pools of liquid that formed stretched out and oozed across the rug to form intricate letters spelling something out in a bizarre language she did not recognize.

“Oh, my, this cannot be good.” WATSON said, stating the obvious.

“No kidding. Begin scanning the usual databases for gang signs, military unit insignia, secret societies, tattoos, and ancient symbology. I think we may have stumbled onto something that we weren’t meant to discover” said Kendra. “Let’s head back to the office and start putting the pieces together.”

The next several days were a blur for Kendra. Most of her time was spent processing evidence, populating the database with all of the information and media, and focusing every remaining waking moment trying to research the origin and meaning of the skull. It didn’t take too long to establish a timeline of the crime and how the murders occurred, but her research efforts resulted in a lot of dead-ends and few leads.

It was Thursday afternoon and Kendra headed over to the simulations room to tweaking her presentation before the staff briefing the next day. The higher-ups were demanding some progress and wanted to look at her work so far. She went to Room Three, which she had reserved earlier, and looked around. Room Three was entirely empty and slightly larger than a racquetball court. The walls and floor were painted a light grey color with thin black registration marks at every corner. She thought it was pretty depressing and wanted to get started.

She touched the corner of her glasses and went through the usual initialization and authentication process. “Load SIM four-zero-three, set location to living room.” Kendra paused and waited while her command was processed. After a few second a photorealistic 3D mirror of the crime scene faded into existence. Every detail was reproduced exactly and to scale, all based on the visual data she had recorded during her initial walkthrough and subsequent evidence collection at the scene.

Kendra continued giving the main computer additional commands, and after half an hour or so, the victim’s bodies were also visible. Faint red beams crossed the room, originating at each of the areas where the bullets were recovered, giving her a good idea where the killer or killers had stood and fired from. All of the fingerprints that were taken were also referenced, faintly glowing various shades of green. She touched a pair that was on the coffee table and a small window materialized above the prints with a zoomed in version and information about who the prints belonged to. In this case, the prints were marked as unknown and not in any of the usual fingerprint databases.

She moved on, making notations and observations to each area of evidence. Everything was checked and rechecked. Kendra reviewed all of the collected data and media, making sure everything was properly linked to the relevant objects. If there were any questions at the meeting, she literally wanted everything at her fingertips. All it would take was a brief touch on anything (or anyone) and all of the reports, photographs, video clips, and lab data would be instantly viewable to anyone in the room.

Once she was satisfied with everything, Kendra started tweaking the reconstruction of the crime based on the evidence they had so far. She started by tracking backwards from what the Police discovered when they responded to the crime, continually adjusting the position of each of the victims as she worked through the timeline. When she finished, she directed the system to fill in the blanks based on the forensic data already in the database.

After the processing was complete, she would be able to view a realistic recreation of the crime from any angle and perspective. The combination of the 3D objects, advanced physics modeling, and actual forensics data made these simulations incredibly lifelike. She had heard stories of the occasional jury member being shocked after witnessing these simulations in court, particularly if the crime was unusually brutal and violent. It felt like being an unseen ghost right in the middle of the crime as it occurred, and was much more compelling than the old way of relying on photographs, charts, and awkward animations on a projector screen. This put you THERE and left little doubt as to what happened. Kendra wondered how long it would be before the engineers would figure out how to implement smell and tactile feedback.

She ran through the simulation a few times and checked all of the datapoints once again to make sure she hadn’t missed anything. It was good enough she supposed. Maybe she should take another hour or two and check in with some of her contacts that were helping her with the skull research and then call it a night. Her boss didn’t take kindly to oversleeping or flubbing a presentation with the brass due to lack of sleep. Tomorrow should be interesting. She hoped the press didn’t show up, that always made her nervous.

“Save all files, backup, and copy to private storage. End simulation.” Kendra looked around at the empty room once more and left for her office.

(To be continued…?)

******

-Excerpt from Working Through Synthetic Worlds, Chapter 11, Augmented Reality Tools for Enhanced Forensics Simulations and Crime Scene Analysis by Robert Rice

Facebook to Acquire Oculus Rift for $2B...

oculus-rift-slider.jpg

So, yeah, Facebook just announced it is acquiring VR headset company Oculus Rift for $2B. Wow and holy cow, what does this mean for the industry?

Take a look at Mark Zuckerberg's Status Update announcement. Mark discusses the amazing potential of virtual reality as a new platform and some pretty amazing experiences, but doesn't really mention anything about how this relates to Facebook, other than their mission to make the world more open and connected. 

This acquisition is certainly going to make the founders and investors at Oculus very happy, and if they are allowed to continue operating independently, this huge cash influx should hopefully accelerate everything that Oculus is working on, but there are still a lot of remaining questions about age-old problems with VR headsets, the main one being the eye-strain and motion-sickness that tends to occur after using the headsets for several minutes. We will probably see a bunch of VR-enabled games hitting Facebook, but honestly, creating good experiences for VR is COMPLETELY different from just making a 3D game or mobile app. We will probably see some conversions or whatever just to build up a portfolio, which will either be great and really amazing, or it will fizzle out very quickly as gimmicky and novelty as everyone tries to get on the bandwagon. 

Having said that, I am generally optimistic. There are some great people at Oculus and they are making tremendous strides in an industry that went *poof* in the mid-90s. The hardware isn't quite there yet, but it is getting pretty damn close. Once the hardware is right, all of the amazing experiences we have been promised for many years about virtual reality (VR) and its potential will start coming true. However! Who are the VR developers? Where are they? Where are the investment dollars going into VR startups? Who is pushing the edge of innovation for VR in terms of user interface, user experiences, immersion, and so forth? What happened to all of the VR pioneers and innovators from the 80s and 90s? I'd say that they are out there, but they are disillusioned or disinterested... try talking to a venture capitalist with an idea for changing the world or technology now if you are over 25. Good luck with that. I'd argue there is a lost generation of expertise and vision for virtual reality out there that needs to be recovered. 

Virtual Reality isn't going anywhere any time soon until sources of capital (angel investors, seed funds, venture capitalists, etc.) decide to embrace the vision and potential of the technology (and its inherent risks as "new" technology) and start pumping capital into new startups. These startups must be driven by the pioneers and visionaries that already understand this tech and how to apply it...young guys with an idea for a facebook VR game just aren't going to cut it. Much more is at stake, and the potential for really changing the world is huge. 

Now, I'm not knocking the young guys, I've been there myself. But with VR in particular, there is already a wealth of knowledge, expertise, and vision (pardon the pun) that can be tapped into, and I'm just not seeing it happen. I really don't want to see Farmville VR, Flappy Bird VR, or Snapchat VR. 

I would guess though that Facebook's move here might just open the floodgates for VR companies as everyone starts thinking that hell, if Facebook thinks this is the future, we better get on the bandwagon or we are going to miss out. This might be a good thing, as capital will flow, but on the other hand, in the past when this sort of thing has happened, it has created a bubble and tons of really stupid ideas and teams get crazy funding, which just goes up in smoke as they are busy burning the cash doing "cool" things instead of building something really awesome

Getting back to Facebook though, I'm still not clear on why Facebook decided to acquire Oculus...I sincerely hope we don't see a Facebook VR application any time soon. VR is about immersion and experience, not 2D windows full of status updates, "likes", or selfies. There is nothing about VR (now at least) that will contribute to Facebook's mission of making the world more open or connected. Dean Putney has an interesting post on boingboing that is worth reading. I think that it would have been better for Oculus (and the world) if Google or Apple had acquired them instead. It would have made much more sense, and would align with their efforts in other areas, like augmented reality, wearables, and so forth. 

All in all, I think that Facebook's acquisition will ultimately prove out to be a good thing, but in the short term I think there is a big risk of novelty gimmicks running on the buzz of VR for a money grab, and very little actual innovation. Time will tell I guess.

I do believe that there is insane potential for virtual reality, augmented reality, virtual worlds, wearable displays, wearable devices, and the internet of things, particularly in areas where they intersect or are complimentary. Anyone interested in changing the world and building the future is welcome to contact me. 

I have to hand it to Zuckerberg though, this is a really interesting development in the industry and I'm looking forward to seeing how it plays out and what Facebook is ultimately going to do with Oculus. 

Just remember, there is no spoon. 

Really, there is no spoon.

Update: And now, there is no Minecraft VR for Oculus Rift.

My first tech job was creating virtual reality

Oculus Rift Virtual Reality Wearable Display

Oculus Rift Virtual Reality Wearable Display

I am sure you have heard about Oculus VR by now. They have an awesome head mounted display for virtual reality. Great company, great team, and they just scored another $75M in funding. They even managed to get John Carmack to leave Id, which is something like getting Steve Jobs to leave Apple. While Carmack is known for his immense contributions to 3D graphics and the game industry, not many know that he had a connection to Virtual Reality a good twenty years ago. I'll explain in a minute, but I need to provide some background. 

AWT's Reality Rocket

AWT's Reality Rocket

After working for a few years in various interesting jobs, I started my first business, a comic book store in the early 90's. This lasted for a little over a year (maybe a year and a half) when I closed the store down and took a job as the first employee for the first virtual reality game company based in the US, called Alternate Worlds Technology (or AWT). This was around 1993 or 1994 I think, I don't remember off hand. I was their first full time employee and had the lofty title of "Senior Multimedia Engineer" which meant that I knew how to use a computer, I could teach myself graphics, 3D modeling, animation, and sound effect design. Plus I could handle other design, user experience, and whatever else the CEO wanted me to do. 

Somehow or another, the CEO of AWT managed to convince Id and John Carmack to agree to some sort of license to Wolfenstein 3D (at the time, Doom was still in early development). AWT was going to do a virtual reality conversion of the game for arcades. We were developing stand-up and seated arcade game cabinets (the seated one had the unfortunate name "Reality Rocket").

Clip from the AWT Marketing Brochure

Clip from the AWT Marketing Brochure

Not only did we end up finishing and releasing Wolfenstein VR (which was a huge amount of fun), but built a whole new game on the engine called Cybertag VR...the first multi-player VR arcade game if I remember correctly. Cybetag is my first real game design credit...I did the overall design, gameplay, character design, graphics, animation, and sound effects. 

Those were fun times. I had to teach myself to do a lot of this from scratch and then figure out how to train other people. Deluxe Paint ,Animator Pro, Pre-release version of 3D Studio, Wave for Windows, the list goes on. I had to stumble through trial and experimentation with some of this software...didn't have manuals and certainly didn't have Google to go search Youtube for tutorials haha. 

Anyway, I remember John Carmack being very interested in virtual reality, even back then. I'm really excited to see what Oculus is going to do in the future. We live in interesting times!

 

Your Augmented Reality concept video is awesome, but...

 I love watching new augmented reality concept videos. It seems like all of the cool kids have one these days. While these are fantastic at communicating the potential of augmented reality (including perceptual computing, mixed reality, virtual reality, sensor fusion, and much more), these videos are a little problematic.

First, they are setting unrealistic expectations from the consumers that are watching them, and I'm willing to bet fooling potential investors and venture capitalists that are looking at funding these companies.   

lots of windows.jpg

These videos are sometimes used as marketing pieces, and for the common man who doesn't know better, it looks like this is what the company is developing, when the reality couldn't be farther.  

It seems to me, and I could be wrong, that very little thought is being given to the technical elements required for these experiences or any consideration at all for design challenges, particularly around areas like user interface, cognitive overload, information visualization, and much more. Have you noticed that there is a good amount of mind-reading in half of these videos, not to mention context driven experiences that would probably require a quantum computer or two to actually pull off?

At least make it clear that your concept video is a work of fiction and be sure to pair it with actual videos of what your product or tech actually does, or what you expect it to do within a reasonable amount of time, maybe not more than a year or so.  

Here are a few examples, good and bad. 

There is sooo much that is misleading about some of these, it is almost embarrassing.  

 

2010, Year One: Decade of Ubiquity

I’ve blogged in the past about Future Vision and the coming Decade of Ubiquity and my predictions for what might occur between now and 2012, which is a bit beyond the current crop of 2010 predictions by some really smart people as aggregated by Games Alfresco. I’ve always had a knack for thinking ahead, and more often than not, I’ve been too early. I started a company in 1995 to build the first real-time 3D MMORPG (during the days of VGA and 2D sprite “3D” graphics) with a strong emphasis on social gameplay, and in 1999 I was evangelizing the digital nation as a virtual world community platform, and in 2000 I shifted to 3D interfaces to the Internet along with virtual goods and microtransactions, and I made a scathing indictment of online worlds and MMORPGs back in 2006 about the decline of that industry’s craft and lore which many people are finally beginning to see and agree with. Of course, back then many people attacked my point of view (notice the low rating of the book and comments on Amazon.com).

2005-2006 was around the time I was designing Immortal Destiny, which was meant to be a true next-generation virtual world and MMORPG. The whole world was designed to be AI-driven and a fully adaptive and evolving ecology that would change based on what players did (or did not) do. We even found some really interesting genetic computation algorithms that we were going to leverage as sort of an artificial life intelligence to control many of the game systems and mechanics. The full scope of the world was to give players the chance to finally be important, and the drivers of the story, on both micro and macro levels, instead of just churning through static canned content. There are a lot of other problems with MMORPGs and Virtual Worlds right now (which I addressed in my book, and are still relevant). Sure, some games like World of Warcraft are successful financially, but they could be so much MORE successful, the market could be bigger, and games could be more engaging and interesting.

Anyway, I tried finding funding for Immortal Destiny, but at the time, I just couldn’t do it. Much of the interest in the industry had moved on to casual and social games and worlds, large MMO projects were getting shut down left and right (remember Sigil and Perpetual Studios?), and it seemed that the only way to find funding was if you were a baseball star or a former employee of blizzard (regardless of what you actually did there). So, I made the call and suspended development. Sometimes, if you aren’t getting any traction, it is best to stop and move on. I still plan on creating Immortal Destiny and shaking up the game industry, but unless one of my blog readers has $20M to drop (and no, you do not need a $500M budget to blow the industry out of the water), I’ll be self funding this in the future.

So, back to the topic. In mid 2006, probably around August when I was at the beachhouse on our annual trip to Topsail Island, and was making the decision to close the doors on the MMO, I started thinking about technology. What the obvious trends were, what trends were developing in the underlying currents of various industries, what was happening on the internet, in virtual worlds, in games, in social media, in mobile, in hardware, software, telecom, etc. etc. This is about the time where I discovered QR codes, Datamatrix, and found a handful of videos about augmented reality on youtube.

I admit that this was a huge surprise to me. The beginning of my career in interactive media was in the very early 90s at the first virtual reality arcade game company in the US (Alternate Worlds Technology), so I was quite familiar with all things virtual reality, which is not a huge leap from augmented reality. I didn’t think that the state of things was as far advanced as it seemed to be, and certainly not accessible. After a bit more research, I discovered ARTag, ARToolkit, DART, and a few other things. I immediately saw the potential here, and a lot of old ideas came flooding back.

To me, the full potential of augmented reality can only be realized when we can break away from the desktop, making it mobile and ubiquitous, while moving beyond the handheld “lens” (i.e. hold up your iPhone and look through it) with wearable displays. Even then though, the wearables must be in an eyeglass form factor, and the lenses must be transparent. This combination is still a few years off (sooner if I had my way), and is the absolute basic requirement for the impending media evolution.

Now, don’t get me wrong. I firmly believe that the state-of-the-art of augmented reality has an insane amount of potential on its own, but to be honest, most of what came out in 2009 was desktop marker based (stable, looks great, lots of uses, but ultimately deployed in ways that were pure gimmick and schlock) or directory services that are pseudo AR. Almost all of the so-called AR Browsers out there fall into this category (and some don’t qualify as AR to begin with). I think what we are seeing right now, and definitely through 2010 is more like the emergence of location based content and mobile experiences, wrapped and marketed as augmented reality. This is ok though. The industry is still barely born, and we have a long way to go. A few more years of technology advancement and industry maturity is required before we start seeing real things that will have a lasting effect on our daily lives.

The point though, is that all of these things calling themselves augmented reality now are just the start. Everyone is getting their feet wet, experimenting, exploring, and beginning to innovate. We can argue about what is or isn’t augmented reality, but it doesn’t really matter. What does matter is the continual push for advancing the technology, the industry, and getting people to start using it. My own company, Neogence Enterprises, has been working quietly in the background on our own stuff with an eye to the future, but I think our goals have been too ambitious for the short term. I’m not satisfied with the current state-of-the-art, and I want more. However, the longer we take in development, the less ability we have to build our own brand and compete for marketshare and eyeballs. So, we are shifting gears a bit and will be releasing our own AR Browser and a few other nifty things in the near future to stake our claim, while we continue developing the other stuff, and solving the really hard problems that others haven’t even begun to consider yet. Remember, I like to think very far ahead, and work backwards…developing a roadmap that lays out a plan to execute. That is what we are doing.

But what does all of this have to do with the title of this post? Yes, I agree that augmented reality effectively exploded on the blogosphere in 2009 (even though its been around for years) and it will really start taking off in 2010 (expect AR startups coming out of the woodwork, venture capital starting to flow, a couple of failures and closings, some mergers and acquisitions, and some really interesting applications (but not until later in the year at the earlist)). But what is really going on here? If you set aside all of the glitz of augmented reality and consider what is happening on a very subtle level, you begin to see the beginnings of some other trends. Augmented reality just happens to be the umbrella that all of this is getting lumped under and is the easy buzzword to throw around.

* Mobile Paradigm Shift

I’m not going to go into much detail here about this beyond saying that mobile devices aren’t just for making phone calls anymore. The mobile device is becoming the replacement for laptops, and for most casual computing. Even as dramatic as this shift is here in North America, we are still half a decade or so behind what is going on in Asia or some third world countries where they skipped the whole “copper wires in the ground” phase that we are still dealing with as legacy. You might not believe it, but some countries are moving towards a cashless system and the mobile device is replacing the wallet. Think about that for a minute.

The rapid development of smart mobiles (the explosion starting with the iPhone) is nearing fever pitch. The new devices we are going to see over the next year or two are going to be amazing. The things we will be taking for granted by the time 2012 rolls around would stun us today to even consider, yet it is coming.

* Location, Location, Location

I mentioned directory AR earlier as very early implementations of location based content. If the buzz in 2009 was around AR (at least in some circles), I’m fairly confident it will be location based content and services in 2010. As I have said dozens and dozens of times in the past year, who you are, where you are, and what is around you will be important. In the past we have gone to places on the internet to get information, now we will start seeing information served to us on a silver platter that is relevant to where we are. This too will take a couple of years to really get cooking, and we have already started seeing early efforts here (have you heard the rumors of Google considering an acquisition of Yelp for $500m? (Update: More rumors report that Yelp has spurned this offer)). My favorite app for location based anything right now is probably Foursquare. I checked into a local pizza place yesterday (Sauced Pizza) and Foursquare gave me a $5 off coupon on a large pizza. Holy cow. How awesome is that?

* Ubiquitous and Pervasive

Ubiquitous: existing or being everywhere at the same time : constantly encountered

Pervasive: to become diffused throughout every part of

When I talk about the decade of ubiquity, I mean to say that during the next ten years (sooner, really, but it is such a great line, I’m sticking with it), what I define as augmented reality (in broad terms) or “the blend between the real and the virtual” will definitely, absolutely, and unavoidably occur. Computing will become smaller and almost unnoticable, and be part of nearly every aspect of our lives. The various implementations and modes of this will change and evolve to be sure. For now, we are holding up our mobile devices and peering at the tiny screens. In the future, you will simply walk into a room and it will know you are there. You will buy things by swiping your phone over a sensor. Your car will start when you get close to it. You will never have to punch a time clock at the office. You will always have directions to get where you need to be, without having to look it up. Intelligent agents (running on a mobile device) will recognize your voice and order pizza for you, make calls, book appointments, and arrange airfare. Interactive 3D virtual goods and characters (appearing like holograms) will be all over the place along with dynamic data overlays…all designed to your tastes, preferences, and habits.

Every industry and way of life will feel the effects of mobile, ubiquitous computing, augmented reality, smart devices, embedded sensors, and automation. It used to be fun talking about this, reading science fiction, and watching movies, but we are finally at the point where we can see light at the end of the tunnel, and the future we (well at least the older folks) have been dreaming of is rushing towards, gaining speed every year.

Of course, there are obstacles along the way…the economy, world politics, the strangulation of commerce and innovation funding, apathy, bad business models, greedy people, and misdirection of talent and resources, but we will overcome. The golden technology utopia of the future that we all desire is too bright and the siren call is too strong. Yes, it might take longer than we would like, and it might not turn out like we hope (try reading 1984 and Brave New World over the same weekend), but as long as we strive, and refuse to capitulate to failure or weak minded individuals who swear the sky is falling and every ambition is a waste of time, we will get there.

At least, that is what I aim for. As Tesla once said “The present is theirs ; the future, for which I really worked, is mine.” Don’t be satisfied with the status quo, and don’t be discouraged when you see someone else building (and profiting) from things you have imagined or had the idea for on your own. Do it anyway, do it better, and always strive to reach higher and farther than anyone else. Success will find you sooner or later.

So, here is an early welcome to you to year one of the decade of ubiquity. How will it change your life? What are you going to do? Are you going to jump in and make it happen? Are you going to sit back and watch? Are you going to slow down the visionaries and workers making it happen by complaining about things and marginalizing their efforts? Are you content? Or are you driven? The future is yours to create and invent, or you can fade into the past.

As for me, I’m going for gold. I’m never going to quit, I’m never going to be satisfied, and I will never settle. I may have to walk in smaller steps at times, but every one of those steps is leading to a leap.

I can’t wait.

Robert

 

"Working Through Synthetic Worlds" now published...

on Amazon. Check it out!

I am a contributing author to the book and I wrote Chapter 11. My title for the chapter is: Augmented Reality Tools for Enhanced Forensics Simulations and Crime Scene Analysis. Long, yes, but to the point.

Amazon’s Product Description:

Virtual environments (VE) are human-computer interfaces in which the computer creates a sensory-immersing environment that interactively responds to and is controlled by the behaviour of the user. Since these technologies will continue to become more reliable, more resolute and more affordable, it’s important to consider the advantages that VEs may offer to support business processes.

The term ‘synthetic world’ refers to a subset of VEs, having a large virtual landscape and a set of rules that govern the interactions among participants. Currently, the primary motivators for participation in these synthetic worlds appear to be fun and novelty. As the novelty wears off, synthetic worlds will need to demonstrate a favourable value proposition if they are to survive. In particular, non-game-oriented worlds will need to facilitate business processes to a degree that exceeds their substantial costs for development and maintenance.

“Working Through Synthetic Worlds” explores a variety of different tasks that might benefit by being performed within a synthetic world. The editors use a distinctive format for the book, consisting of a set of chapters composed of three parts: a story or vignette that describes work conducted within a synthetic world based loosely on the question, ‘what will work be like in the year 2025?’, founded on the expert authors’ expectations of plausible future technologies; a scholarly review of the technologies described by the stories and the current theories related to those technologies; and, a prescription for future research required to bridge the current state-of-the-art with the notional worlds described in the stories.

The book will appeal to undergraduate and graduate students, professors, scientists and engineers, managers in high-tech industries and software developers.

Augmented Reality: Not exciting to "normal" users...

Chris Dannen, over at Fast Company just put out a post called Put Your Phone Down: Augmented Reality is Overblown where he dismisses augmented reality offhandedly and doesn’t appear to really understand what it is or could be. The opinion in his blog post is myopic at best and he makes a few subtle stabs, like the following:

…few of the “normal” smartphone users I talk to can get excited about augmented reality. Sure, they’re not thinking five years ahead, as we’re told tech visionaries do.”

Is he suggesting that those people getting excited about augmented reality aren’t normal, being nothing more than the techno-fringe? The next bit feels slightly sardonic as well…presenting tech visionaries as people who say they are thinking five years ahead, and being dubious about that.

In the next breath, he gets a little enthusiastic about AR being right around the corner and not five years in the future, and that it is in its “last trimester”, but then saying that few people seem to care. The recent issue of Esquire, featuring marker based AR, is highlighted as the example proving his point.

Then, shockingly, he quotes Jonathan Wegener, a developer of an AR app as saying that he thinks that consumers will get tired of AR and that it won’t catch on. Wegener’s app (NYC Exit Strategy Subway Map) sells for $4.99 in the Apple app store. Why bother if you don’t think consumers are interested in it?

Dannen ends his blog post by shifting positions again and saying that:

AR isn’t bound to fail—that’s not what I’m arguing. Iteration after iteration will be pushed on consumers until the technology finds its niche.

See what he did there? iterations pushed on consumers and niche. He seems almost annoyed and irritated that augmented reality apps are being shoved down his throat on a daily basis, each one slightly different than the one before it, and that maybe, after as much trial and error as Thomas Edison went through to perfect the light bulb, the industry will finally find a niche for the technology. Anyway, what exactly is he arguing? Nearly the entire post is a little sarcastic and almost mocking of both augmented reality, the people developing it, and the consumers that are interested in it.

The last line:

But if you’re expecting apps like Layar to drastically change the way you live your everyday life, well, that’s just not reality.

I’m not aware of anyone out there suggesting that apps like Layar, Wikitude, Robotvision, Yelp, Bionic Eye, Accelair, etc. are going to change the way everyday life is being led. I am aware of some people challenging whether or not these apps should even qualify as augmented reality, and a vast number of people realizing that these apps are barely the equivalent of outputting “hello world” in BASIC. So much more is to come, and is already in development. We haven’t even scratched the surface.

But Chris ignores almost everything else in the emerging industry and the phenemonal technologies beginning to creep out of the Universities, and focuses on very basic implementations of AR. His opinion is validated by his “industrial designer” friend who didn’t think the Esquire magazine implementation was all that. What kind of industrial designer is his friend? If the guy designs next-generation toilet seats and urinals, I doubt he can make any kind of authoritative judgment on user interface, multimedia, or a unique technology like augmented reality. The point isn’t that you need a PC and a webcam to experience the AR in Esquire, the point is that you can experience the magazine in an entirely different way.

I’ve spoken before about the dangers of industry hype, setting the right expectations, avoiding the wrong judgment of the technology before it matures, and of course, the need to strive for innovation. Chris’ post is short sighted, does not add anything of value to the discussion of augmented reality (negative or positive), and is unfairly dismissive in a condescending manner.

It is opinions like this one in forums like Fast Company that are making it harder for the guys on the ground to gain traction with their efforts and is demoralizing. Is it any wonder that the spirit of innovation and entrepreneuring in North America has been beaten down?

Take a look at “augmented reality” in google trends and judge for yourself.

Is augmented reality just a fad? Are only the out of touch tech-elite and basement nerds excited about it? Is it true that “normal” and intelligent people are merely tolerating augmented reality, giving each other knowing nods like wise adults humoring elementary students excited about Santa Claus?

I think not. If Chris had done his homework and spent any real time researching instead of making an off-the-cuff blog post characterizing an entire industry and technology on the opinions of a few friends, he might have come to a different conclusion.

A few more things for you to consider:

US Post Office, GE, Best Buy, Proctor & Gamble, Molson Beer, Lego, ING, Wimbledon, Mattel, Topps, Facebook, Adidas, Nickelodeon, Doritos, Eminem, John Mayer, Papa Johns, Canon, Philips, Ray Ban, Sony, HP, Disney, Esquire, Popular Science, Wired, Burger King Toyota, Volkswagen, Fiat, Ford, BMW, Citroen, Nissan and many others…

have ALL used augmented reality technology in some fashion or another in the last twelve months, mostly for marketing or advertising.

Companies like Microsoft, Intel, IBM, Cisco, Nokia, NVIDIA and others that include multiple branches of the US Military are using augmented reality applications in real and practical manners. If you take some time and do your own research, you will find that augmented reality has potential applications in nearly every industry and sector…manufacturing, education, entertainment, industry, military, libraries, simulations, chemistry, telepresence, business to business, visualization, biology, forestry, farming, design, etc.

Yes, augmented reality will change the way you live your every day life. Just not tomorrow or through the very early stage implementations we are seeing now. Normal people are indeed excited about it. The ones that aren’t excited, or that are quick to dismiss it, fundamentally don’t really understand it, or are incapable of making that intellectual leap to anticipate where things are going.

This particular blog post by Chris reminds me of some other famous quotes:

 

“I think there is a world market for maybe five computers.”

- Thomas Watson, chairman of IBM, 1943

 

“I have travelled the length and breadth of this country and talked with the best people, and I can assure you that data processings is a fad that won’t last out the year.”

- The editor in charge of business books for Prentice-Hall, 1957

 

“But what…is it good for?”

- Engineer at the Advanced Computing Systems Division of IBM, 1968, commenting on the microchip

 

“There is no reason anyone would want a computer in their home.”

-Ken Olson, president, chairman and founder of DEC

 

ISMAR 09 Observations and Comments

ISMAR, the International Symposium on Mixed and Augmented Reality was held this past week in Orlando Florida. It was pretty awesome and my expectations for the symposium were exceeded in many ways. I had thought that this year was going to be the break-out year, but I’m beginning to think it was only a precursor to the one next year in Seoul Korea. There is so much “on deck” right now that is going to explode out of the box in the next twelve months, that 2010 is going to be freakishly awesome.

ISMAR 09 was a huge success for me, and very exciting. I have been pretty enthused about augmented reality already, but now I am close to vibrating with energy and optimism about the future of the industry, and I absolutely cannot wait until ISMAR 10 next year. Now that I am home (and dead tired) I wanted to put out some observations, comments, and ideas while things are still fresh on my mind, and after I have had a chance to think about it on the plane home. Grab some coffee and have a seat, this is going to be a long post.

And here we go…

It was my first time attending ISMAR, but definitely not my first time attending an industry conference that was inwardly focused (in contrast to something like E3, where it is designed to highlight commercial releases, media frenzy, press releases, and heavily marketing/sales). I mention this to give some perspective to the rest of my comments in this post.

First and foremost, I want to make it clear that the conference was, overall, damn fantastic. I would almost rate it as the best conference I have attended in the last decade, in terms of people I met, the relationships I was able to grow, the things I learned about the state-of-the-art from academic, research, and commercial sources, and the general quality of the attendees and speakers.

I am relatively new to the Augmented Reality industry. While I have been dabbling in different areas of technology that are complementary to AR, or core elements (like 3D graphics, interactive media, etc.) for most of my career, it was only about two years ago that I jumped in head-first and 100%. Even so, I’ve felt like an outsider observing from a distance, with my voice swirling away in the wind. Part of this is due to the fact that my startup, Neogence Enterprises, has largely been under the radar or very quiet about what exactly we are doing to any degree of detail (purposefully), and the other part is because augmented reality is not exactly a new technology, although I would argue that it is a extremely new industry.

By this I mean that augmented reality has been around in some form or another for the better part of the last three or four decades (longer by some counts) and generally limited to research in Universities, limited military and defense applications, and basement R&D at some large companies. There hasn’t been an industry to “break in to” and there are only a handful of Universities where you can find any real educational tracks and research departments doing this stuff. The ISMAR conference itself is old by some standards, with ISMAR 09 being the tenth annual conference.

During 2009, pandora’s box was opened and there were suddenly startups everywhere looking at doing AR applications, content, tools, and so forth. Even a lot of media campaigns started using low level marker based AR to do their marketing stunts. We all started running around with our dreams, plans, ambitions, and in some cases (I’m looking at you mass market media) completely misunderstanding what exactly AR is and babbling on about things that just served to misinform people. From the academic perspective, they were shoved aside for all the brash new folks, who were suddenly getting a lot of attention, accolades, and credit for “inventing” things that others had been working on and researching for years and years.

And ISMAR, which has been a reputable conference for academics, peer-reviewed papers, research, and heavy on the “science and technology”, kindly expanded their program this year in an experiment to embrace “arts and humanities” (which sort of included everything else, like business, design, and whatever).

This is good, and the timing is perfect. Actually, I think this is a must. There is currently no other real augmented reality conference (as the main point of the conference…ones that have an AR track don’t count), and what has been an area of research has suddenly been propelled, abruptly, into the spotlight as an industry (when it is barely more than a lot of academics and a handful of startups). This new “industry” still doesn’t have standard practices and methodologies, a lexicon, standard business models, a professional association, a well formed and active community, or any of the other things that a industry normally has.

This is a bit problematic. Our industry is unique in many respects, and it can’t always be considered in the same way as some other technology related industry. To complicate things, there is a growing demand for products, applications, content, and new innovations…I can’t tell you how many companies, agencies, and whatever have contacted me in the last six months looking for quotes on producing some sort of application or whatever, to be surprised that what they want has to be built from scratch. The mass market doesn’t realize that the vast majority of Augmented Reality technology right now is barely more than a research project at a university. With few exceptions, there is little that is ready for the market. Heck, we can’t all even agree on what the definition of Augmented Reality actually is or is not right now.

This is an opportunity for us, the industry, and ISMAR. Expanding and extending the scope of ISMAR can provide a fertile ground for the continuing birth of the AR industry, and act as a guiding force to mature the technology into a real industry. By doing so, it will grow in relevance, prestige, and legitimacy. If it does not, some other body or organization will fill in the rapidly growing void and eventually marginalize the Symposium, leaving it to the academics to publish their papers and wonder why the industry that could very well be bigger than the Web is today, has left them behind.

While most of my experiences at ISMAR were unbelievably awesome, I did not get the chance to see or experience as much of it as I liked. This is mostly because I was either speaking at one of the sessions myself, or I was engaged in meetings. I think I only caught about half of the sessions I was really interested in. So, I should note now that my opinion, and some of my gripes, are from this limited perspective. I didn’t see everything, so keep that in mind.

Ok, moving on. First, I picked up on a vibe (at the conference, and for a few months leading up to it) that some of the “old guard” was of the opinion that entrepreneurs and other efforts to commercialize AR were nothing more than riff raff, or uneducated blowhards jumping on the bandwagon for a quick buck, without really understanding anything, and probably not capable of doing anything of real value or innovation. I also picked up on some tangible disdain for “arts and humanities” as something that was incapable of measuring up to scientific standards, or even that including it would diminish the prestige and legitimacy of the conference. While I can understand where this is coming from, these points of view are archaic, close-minded, and disillusioned in the face of reality or what goes on in the world in other industries. The “science and technology” of augmented reality is only one element of several that are critical and necessary for the advancement of technology in general, as well as the emergence of a whole industry.

I should note that this seemed to be extremely limited and is not representative of the symposium at large. In fact, quite the opposite is true. The overall atmosphere was very welcoming, open, engaging, appreciative, and willing to grow and expand. Looking over the proceedings and comments from the Chairs, there is a clear desire to grow the Symposium and embrace the industry. This is part of why I think this year was so important, and why I think that ISMAR is going to be an absolutely critical cornerstone for the growth of the industry at large. I only point out that I picked up on the negative vibe (however minor or marginalized) because I think it is important to recognize that these feelings and perceptions do exist, and they should be addressed, lest they cause problems in the future. For example, I heard some rumors about some “disagreements” occuring behind the scenes on the Wikipedia entry for Augmented Reality. Yes, I realize this has absolutely nothing to do with ISMAR, but it is indicitive of a problem that is percolating amongst academics and the commercial sector. This needs to be dealt with immediately. This is the sort of thing that greatly diminishes the credibility and legitimacy of the whole industry at large, and if we let some rampant egos, holier than thou attitudes, or sniping about who is doing (or has done) what, and whatever else, it will have far reaching negative effects. (Note: I have not confirmed this, nor do I know the context of what the problem is between the editors on the wikipedia entry for augmented reality).

I would also like to mention that the startups and commercial people need to be a little more aware about what they are doing and telling the press. Recognition must be given (and indeed, is quite deserved) to those academics, researchers, and innovators that have invested years of their lives to create the base technologies that we are all beginning to capitalize on, and advance in our own right. There would be no AR industry without the pioneers that have gone before us. Even as they embrace us and welcome us into what has been their domain, we must embrace them as well and collaborate with them to advance the technology and the industry as a whole. Doing otherwise would be folly and disrespectful.

Moving on to another topic: The printed proceedings of the conference (totally worth the cost by the way) weighs in at about 300 pages. Only 61 were related to the “arts and humanities” track. While the papers were quite good, and lived up to the expectations of the symposium, I would have liked to see much more in terms of cognition, psychology, sociology, perception, user interface, iconography, filtering, multi-senses, and so forth. Each of these disciplines have much to offer and should be sought after in the future. Maybe “arts and humanities” needs to be renamed or split into other segments, both for papers and for broader topics covered in panels, workshops, and speaker sessions. Maybe science and technology, arts and entertainment, business and media, and design. I had a number of people ask me about business related topics, wishing there was a source for them to learn about starting a tech company, navigating tech transfer and licensing, the patent process, or even collaborating with the commercial sector to support research programs and projects.

Ok, on to my gripes:

1) I got screwed by the person at Marriott who took my reservations and assured me, several times, that I had a room at the Marriott Downtown. Instead they booked me at the Marriott Courtyard. To add insult to injury, I had to pay for parking, when it would have been free if I was staying at the courtyard. The “manager” of the valet parking guys gave me a hard time about it. Never mind the fact that I was a speaker at the conference, or that I was staying AT A MARRIOTT, he wouldn’t have anything to do with it. Very annoying.

2) I had to pay for the boxed lunch (which was weak) and for the awards banquet. I’m annoyed I had to pay for the boxed lunch, but the banquet dinner was mediocre. They served everyone some sort of fish, with mashed potatoes and vegatables, a small salad, one roll, a thin slice of cheesecake, and a glass of water. If you wanted anything else, like a coke, beer, or wine, you had to buy it yourself. They did offer coffee (after the dinner).

3) The wireless was ludicrous. It was free in the center of the lobby downstairs, but something like $10 a day for wireless in the rest of the hotel. The first day after I paid, I had to wait HALF AN HOUR to get a special access code, and then I got a new one every morning. It was also very slow, with frequent disconnects (for me anyway). Extremely annoying.

4) There was plenty of coffee available, but not much water or soda (without going to the gift shop). I was thirsty almost non-stop.

5) Some of the rooms were hot with little air circulation. I almost dozed off in one of my own presentations because of it.

6) Signage and directions on the first day were inadequate. I was waiting for someone in the lobby on Monday morning, and I directed not less than 27 people upstairs to the registration area, after they had wandered off down the wrong corridor because of a sign pointing out where one of the rooms was.

7) While I get the upside down-reverse printing of the proceedings and the schedule to contrast science/technology and arts/humanities, it was annoying to have this occur several times in the schedule. This made it harder to find anything at a glance and was really irritating.

For the price of the conference, none of these annoyances should have occured. As a speaker, I should not have had to worry about any of it, especially since I paid for my own travel and hotel.

I think there could have been a much better venue, and a better arrangement with the hotel. It would have been nice to be closer to a variety of restaurants as well (walking distance).

Pretty lame gripes, I know. But I’m tired and cranky, and they have all been bugging me for days. I needed to blog about it and get it off my chest.

So, that’s it. Overall the conference was fantastic, and the organizers/chairs were amazing. I felt like I was welcomed into the community with open arms where before I had felt like some guy on the periphary. I met a lot of amazing people with a real and sincere passion for augmented reality, and I witnessed a clear vision for the future of ISMAR. My negative comments are limited to very few people and this was based mostly on some casual observations salted by a couple of rumors. Its almost not worth mentioning.

For next year, I highly recommend you start saving and planning now. It will absolutely be worth going to Seoul Korea to attend, and I very highly suggest that you start thinking now about submitting a paper or two. If you are inexperienced at this or aren’t sure how to create one that will bear up to high academic standards, then put a hand out to friends working at Universities for some help. Get off your butt and learn how to do it. Sure, it might take some extra work, but it will pay off in the long run. Quality breeds quality, and mediocrity breeds mediocrity.

I want to thank ISMAR for providing me with the opportunity to participate as a speaker, presenter, panelist, and reviewer (Arts and Humanities). It was a wonderful experience, and I am in awe of many of the people I had the good fortune to meet and spend time with.

See you next year…

Robert Rice

Augmented Reality: Blue Sky, Green Earth

When I was in Rotterdam last week for emerce’s eDay (wonderful conference by the way), I had a lot of people talking to me afterwards, usually with questions about the technology, how I thought it would impact one industry or another, what the future might hold, or what the business models might be. One thing I noticed though, was that most of these people could be divided into two different camps.

Blue Sky

These people had their eyes opened during my presentation. They had no idea what augmented reality was, or if they did, they did not realize what the state-of-the-art is, or how close we are getting to the really cool stuff in the very near future. One woman gushed about how excited she was and that now she could see the implications of AR for her company. She was quite thankful to have had the chance to be exposed to something she considered off in the distant future. For her, the important thing was getting exposed to the “blue sky” potential of augmented reality and a glimpse of what was coming.

Green Earth

These were the practical people. They are up to speed on everything going on out there, and they are almost tired of hearing the phrase augmented reality. They are still enthused about the potential, but they don’t really care to hear someone like me talk about the cool stuff, what is coming, or how it will change the world. They want to know, right now, when is it going to be here, how is it going to be practical, and how are they going to be able to make money on it. What is the business model? Who are the early adopters? Which industries will glom on to AR first and run with it? One of these people even complained about the first part of my presentation where I explained what AR was. “We get it already, let’s talk about the practical stuff”.

In the middle…

I think that for now, at this stage of the industry and the technology, we have to continue evangelizing the blue sky. Many people are aware of AR now, but there are still loads more that haven’t heard of it, or are just now discovering the GE Smart Grid AR campaign from the beginning of the year, or they stumble across an old demo from an AR company two or three years ago. Heck, even the really savvy people are just now cluing in to Evan Sutherland’s Ultimate Display essay from 1965 that outlines some AR concepts. Yeah. 1965. And we still don’t have flying cars or a moonbase either, to my eternal annoyance.

Anyway, this is important because the AR industry is just getting started, and for the immediate future, the tech isn’t going to be all that fantastic. We have some time to build all the neat stuff and eventually get to the contact lens displays (which are not coming soon, press to the contrary notwithstanding). We need to keep people focused on what is coming, how it will be useful, and how we will get there. This will all directly relate to early user adoption, funding, R&D, etc. We have to keep people engaged, interested, and eager for the future. I’m tired of boring listless people with no sense of hope or childlike amazement and excitement. I want another golden age of hope, invention, and innovation. To get that, we have to inspire.

On the other hand though, the risk of over hyping, unrealistic expectations, marketing saturation, and everything else along those lines is still a big risk factor. Already I’m seeing people on twitter complaining about seeing too much about augmented reality. I’ve certainly been critical of a number of things that I thought were over hyped, or make claims that are easily misunderstood and representated (I’m looking at you, contact lens display guys!). However, instead of trying to shove AR back into Pandora’s Box (and AR is definitely something big enough to suit that phrase), we should instead focus on the here and now. How do we plan on building “it”? How do we execute? How are we going to frame the standards and protocols? How will it make money? What are the business models? What are the technology obstacles or challenges? When can I start using this for my business? How exactly will it benefit me and my business?

We can’t focus all of our attention on the blue sky anymore. We had that last year and during the spring of this year. Now, we have to split between the pretty clouds and butterflies with the trench digging in the ground…pulling out rocks and building walls and buildings.

So, when you talk to people about AR, start with the blue sky, get them caught up to speed, and watch for the twinkle in their eye when the proverbial light bulb goes off, and then take them right to the green earth. Talk about the particulars, the plans, the risks, and the models. Keep it real, don’t overblow it.

There will be 1,000 AR companies out there in the next couple of years. This is a race too early to call, and an industry too young to peg into a niche or particular framework. Keep your minds open, head in the sky, and feet on the ground.

Then get out there and DO it.

Robert