Taylor Davidson · The Augmented Future (of Photography)

My interest in the imaging industry stemmed from my love for photography. I rolled, shot, processed and printed black and white film in high school, I played around in darkrooms, I carried a camera and took pictures of people and places to share moments about me and what I saw. As I got older, I continued to carry a camera, often walking around with a point and shoot (first film, then digital) in my hands all the time, and I attempted to do more than just share what I saw but say something about my life. Being a photographer, and being good at photography, was a part of my identity. ^[1]

When I began writing about the photography industry nearly a decade ago (archives), my perspective and interest in the industry originated from this love for photography and blended in my professional experience in technology. Even though DSLRs had become more commonplace and Flickr had been popular for a couple of years ^[2], photography on the web was nothing like it is today. Photos had become digital, but they had not yet become social or mobile; photos were not air, messaging, or data. Phones had not become cameras, and lenses had not been networked. ^[3]

The photography industry today has been eclipsed by the imaging industry. Photos today are more than just visual artifacts, they are rich with the context and data around them, and it’s the “everything around them” that is driving the growth in the broader imaging industry today. Cameras don’t take photos, they capture what’s happening in the world today into digital files that are then shared, processed, and spun out into the network to be used to communicate far more than what’s happening visually in a photo. ^[4] Image recognition - powered by machine learning - is reshaping tech as companies work to structure images the same way they structured text. Snap rebranded themselves as a camera company as competitive positioning, noting in their SEC filing that “images created by smartphone cameras contain more context and richer information than other forms of input like text entered on a keyboard.” Pinterest rolled out visual discovery, allowing you to point a camera at a product and use it to find similar ideas on Pinterest. Unsurprisingly, Facebook is attempting to build a camera platform. If everyone is taking photos, then everyone wants them, a simple commercial reality.

I still believe all the tech helping us make better images can help us take better photos ^[5], but to be honest, that’s not where the mass market is trending, and it doesn’t seem to me to be what the next generations care about. The exponential growth in photos changed our relationship with photos and images ^[6], but more so, it’s about to be upended by another technological shift. Meet the new reality - realities - of augmented reality, mixed reality, and virtual reality.

Last year I moderated two panels about image recognition, the first at the LDV Summit about Where will Computer Vision Be in 5, 10 & 20 years? and the second at the DMLA Annual Meeting about Advances In Visual Recognition. In both, we discussed what computer vision and image recognition is and how it’s being applied in technology today, and how we expect the technology to impact business and culture.

The last few years have been exciting times for image recognition. Better tech, new algorithms, better access to image recognition tools (both open-source and commercial), more applications of the technology that have a real impact for millions of people. ^[7] New companies have come up, major companies have made acquisitions and investment into building the technologies and capabilities into their products, and people use these tools directly or indirectly on a daily basis.

That said, the topic of image recognition has been dwarfed by the attention paid to the new realities of virtual (VR), augmented (AR) and mixed (MR) realities. Magic Leap, Google, Apple, Facebook (Oculus), Samsung, and more have all been in the news for their work in creating virtual reality hardware, and with Snapchat Lenses and Pokemon Go, millions of people have been exposed to mixed reality and augmented reality, respectively. ^[8] Each of these new “realities” has the potential to be a new medium and change how people visually capture and experience the world, and thus it’s not surprising to see the investment and commercial interest by companies looking to gain an early foothold into a technology that many see could be the next platform, i.e. the next Internet.

But it’s important to note that the value chains differ. The economics of creating and distributing virtual reality hardware and content likely means that the value chain will be dominated by a few large companies in content creation. Augmented reality stands likely to see a wide variety of approaches in content and tools, but harder to lead to sustainable commercial wins, since the tools to create and distribute are far more open and inexpensive to use. Mixed reality is interesting since the camera has to have the technology to recognize what one sees - faces, objects, and more, using image recognition technologies- and layer on digital content appropriately. This is what Snapchat does with Lenses, and it’s what Facebook is attempting to empower with their Camera Effects platform. Facebook’s announcement of Frame Studio and AR Studio on the camera effects platform is notable because in providing the tools to create and the distribution channel, it’s possible for anyone to reach everyone with a new product ^[9] I created a basic frame yesterday in less than 10 minutes, for example, and it’s in Facebook’s interest to build the ability for anyone to make digital effects. It’s not a stretch to see publishing an augmented reality or mixed reality product as easily as publishing digital content, with a brand- and advertising-centric business model following shortly behind if people use the effects. ^[10] Silly to think that companies would not attempt to monetize a screen by putting interruptive ads - no, interactive, value-added experiences, they say - on them.

I always wanted to build an app that helped people be a better photographer. The editing tools all focus on helping you edit and adjust photos after you take them, but I wanted to build an app that helped people recognize the scene and the subject and make better choices at the time of composition and capture. Like an expert photographer watching you shoot, the app would see what you saw and provide directions and hints to help you frame your shot, move closer or farther, move your key subject around in the image, find the right composition angle, adjust for light and shadows, helping you make a better photo. Using scene, object and aesthetic image recognition ^[11], the app would identify the major components of the photo and provide real-time assessment and directions on what could improve the photo. Imagine it as a far more powerful rule of thirds composition guide.

Perhaps it’s a hard problem, but with the increasing capabilities of image recognition and the computer power of the devices, it doesn’t seem fanciful. But given what people care about in photos and where the focus in technology and imagery is going, it wouldn’t appear to be the most valuable application of the technology. Technology aimed at making a photo is losing to technology aimed at sharing photos; people don’t use Snap because it takes the best photos, but because it’s the most fun way to share memories through photos. ^[12] Photos - and photographers - that win Instagram don’t succeed because they make great images, but because they are able to connect to audiences through a flow of photos that shares something more around the photo itself, be it a lifestyle, the location, or more. ^[13]

Augmented, mixed and virtual reality are easy to dismiss because the current examples - funny selfies, mustaches, hats and cartoons - look like toys, but it’s not hard to see how the fun scales. The adoption of the first apps in the space creates the experiences that helps the image recognition technology improve, and also the rationale for the operating systems and device manufacturers to build better technology to maintain or steal market share, creating better capabilities for app developers to utilize. Apple might already be doing that with the dual-lens system in the 7 Plus, with portrait mode in their camera app just a basic application of depth mapping technology.

The toys help train people to be comfortable with using digital content layered on the real world, creating a cultural awareness and practice that will ready us for future digital overlays that could be far more powerful and culturally explosive. Imagine mixed reality applications that changed the race, skin tone, figure or age of people in our photos? Or added in realistic props or backgrounds that fundamentally changed the photos and memories we share? How would we react to that?

It’s not hard to see how AR, MR, and VR could change popular photography. The smartphone made photography available to anyone, and changed the notion of who is a photographer. The networked camera changed how we use photography to communicate. Even though the usage of Instagram filters has declined off a bit, filters changed photography by making it easier for people to create a photo they felt comfortable sharing. Taking a skill understood by few and making it available to anyone using technology changes one’s appreciation for photography. Snapchat crystallized the usage of photos as messaging, using photos to express ideas easier, faster, and better than just text, and at the same time making the photo itself far less important, more disposable, less permanent. Even though we may not see AR, MR, and VR as “photography”, if they change how we create, use, and value visual imagery, they will have an impact on photography.

So, back to the beginning: what should we do?

If one believes that augmented and virtual reality technologies are a new visual medium that could change our relationship with visual imagery, then it’s important to view them through the same lens as to how digital, social and mobile technologies changed photography.

As a technology professional, the answer is obvious. New technologies create new opportunities and new competitive positions to serve what people want to do, and the opportunity to provide something new to power how people use augmented, mixed and virtual reality is obvious.

As a photographer, it’s a bit different. If the notion of what’s popular or powerful in photography changes, then it puts one in an interesting position. There will still be a place for pure photography - digital or film - but the opportunities will exist for people that want to evolve and take advantage of the new technologies. When I first started writing my nearly-decade-old series about how photographers can evolve their business models, it was aimed to photographers looking to adapt to the new technologies that were impacting their profession. Digital, social and mobile changed what is popular in photography and it changed the businesses in the entire value chain in the industry, from device manufacturers to photographers to publishers and customers. AR, MR, and VR could do the same. If popular visual imagery could be changed by these technologies - and I believe that will happen - then it will change the business models for photographers. Visual creative skill, storytelling, and creating great images will still matter, but the process and skills required to create, share, and win customers and fans could be tactically very different than today.

I wrote about Lytro a few years ago, and my primary message was about how the Lytro was not just a new camera, but a new medium:

The key is that the creator of a great Lytro image will have to think differently than the creator of a great static image. Creative expression will be uniquely different with a Lytro camera, just as all new forms of artistic technology grant a new range of ways to create. The Lytro is different enough to warrant a different scope of creative imagination and interpretation.

I was completely wrong about the Lytro, as it didn’t pick up the widespread adoption and impact required to establish itself as a new medium. But the key message applies the same today for augmented, mixed and virtual reality, and calling it a new medium is far less of a stretch than the Lytro. While the technology behind AR, MR, and VR today is far more aimed at developers, technologists, and commercial applications looking to layer digital content in simple ways, it’s not hard to see how they could be applied far deeper, used far more widespread, and powering a wide range of commercial and artistic endeavors.

Just to be clear, the practical applications of AR, MR, and VR outside of the photography industry are tremendous, and I’m only trying to look at it through the lens of photography. Applying a digital layer to the real world, layering in information and content to add to our experiences, helping us to understand what we are seeing and make decisions, that’s outside the scope of this thinking but terribly exciting also.