How vivo and ZEISS are building smartphone cameras for the TikTok generation

Smartphones singlehandedly wiped out the entire category of pocket cameras and have had a significant impact on the professional imaging market too. Purists might still want to stick to traditional cameras — I know I still use one — but by and large, smartphones are what everyone reaches out for. Not just because they’re with us all the time, but also because of their excellent capabilities and one-click post-processing.

Our picks: The best camera phones to buy

The confluence of computational photography and more powerful/diverse lenses in smartphones has enabled incredible photography right from the palm of our hands.

Enabling those experiences requires technical know-how, but truly crossing the diminishing gap between phones and standalone cameras requires a helping hand. You see, photography is an art that needs an equal amount of ‘feel’ and technical accuracy, and smartphones could stand to gain a lot from a decades-long experience of crafting cinematic experiences.

Android Authority recently got a chance to chat about cameras and, more importantly, about how a fast-paced smartphone brand like vivo comes together with an old-school photography major like ZEISS to advance the field of smartphone imaging.

Crafting the next generation of imaging

Computational photography has, by and large, solved the challenge of everyday photography. Preferences may vary, but it is rare to find a smartphone that shoots bad pictures. Video is, however, the next big frontier. It shouldn’t come as a surprise at a time when Instagram and TikTok are the predominant use cases for imaging-first users. The average user spends upwards of 30 minutes on the platforms every day, and there’s an increasing proclivity towards content creation rather than just consumption.

Speaking to Vikas Tagra, Head of Product Marketing at vivo, it becomes apparent that the company isn’t necessarily looking years into the future for its research pipeline. Instead, the decision to pivot towards making headway in videography comes from a place of hard data.

Read more: The best TikTok alternatives and TikTok apps on Android

Tagra says, “The vision, is coming from our customers and the data backs that the future is video. So that’s the direction we decide to take. Consumers are downloading and editing. So we decided to make it easy and build it into the cinematography workflow.”

To that end, the vivo and ZEISS collaboration is perhaps one of the more complete examples of integration between a smartphone and imaging brand. Video, in particular, has historically been highly influenced by the glass in front of the camera. There’s a good chance that at least a portion of your favorite movie was shot on a classic ZEISS Jena lens, or perhaps an anamorphic lens. Focussing on the stylistic aspect of cinematography is just one of many different routes that are bound to influence the future of smartphone videography.

How should a cinematic look feel? There isn’t a single anamorphic effect. It is a type of lens with various experessions. To pick out which of these properties of anamorphic lenses should be implemented in the software generated bokeh, that takes a lot of experience to get to that point. — Oliver Schinderbeck, ZEISS

One of the ways that the collaboration between ZEISS and vivo has manifested itself is in the form of lens simulations. Be it the famous Biotar lens or the newly imagined anamorphic lens on the vivo X80 Pro. ZEISS built complex computer simulations of existing lenses to recreate the characteristics of the lens in software. These simulations are then used as the base to recreate the lens characteristics you see on vivo’s phones.

While the technical challenge of recording high-resolution, high-refresh-rate content is a problem that is being solved continually through more modern camera sensors, it isn’t the be-all-end-all of imaging. By vivo’s own admission, the company can’t accurately predict what sensors might be able to achieve in the next two to three years, and this makes focusing on the actual art of cinematography all the more important.

For every innovation, we take a consumer-first approach. You have to think: where is it going, where it’s going to go? But I’d say cinematography is going to be the focus for the next few years. This time, we’ve done it differently. It’s all coming from consumers. Half of the time we’re doing consumer research. — Vikas Tagra, vivo

Although capturing videos is just one part of the workflow, it’s still the most important one. It all starts with what you capture. Tagra points out, “Our whole direction in video terms is to give people the best tools possible so that they don’t need to learn how to edit. It’s a point-and-shoot experience that creates output that looks professional.”

This brings us to the science of color. Consistency is a constant worry we have when reviewing smartphone cameras at Android Authority. It’s all too common for multiple camera sensors to have completely different color profiles and characteristics. But we noticed something different on the vivo X70 Pro Plus earlier, and now on the vivo X80 Pro: It is clear that the company has, for the most part, cracked the formula to ensure consistent colors. However, what those colors look like can be a deeply personal, or even biased choice.

In the case of vivo, the company has built a bit of a following around vivid colors and vibrant hues. That is certainly appealing to a certain audience, but where do you go from there? With the point-and-capture market saturated even at the mid-range segment, the raison d’etre of these high-powered imaging beasts is to get some semi-professional imaging in your pocket. That starts with a neutral color profile.

But instead of sacrificing its in-house color profile, vivo has partnered with ZEISS to create a best of both worlds situation. My colleague Hadlee talks about it in-depth in his vivo X80 Pro review, but the objective is straightforward: Give users the option to choose what they want instead of imposing your company’s choice. It’s a sentiment I wholeheartedly agree with.

This [choice to offer dual-color options] comes after extensive consumer research based on the color preferences. You’d also know that most users prefer a more vibrant color profile. We have two directions. Our default is the vivo Vivid Color which is our signature look. Where ZEISS color comes in is a profile that is adored by professionals. — Keshav Chugh, vivo

That neutral color profile is key to enabling color correction, color grading, and true cinematic video production for professionals, but it doesn’t necessarily tie into the theme of enabling on-device, out-of-the-box experiences.

That’s where the extra cinema profiles come in. In the case of the vivo X80 Pro, that’s a forced anamorphic perspective combined with a movie-like 24FPS frame rate. It’s a very specific look that instantly transforms your perspective and footage even though it might not be at the height of technical brilliance.

ZEISS claims that it understands how a cinematic look should feel because of its close working relationship with moviemakers. It wouldn’t be wrong either. In our testing, the anamorphic lens simulation certainly went a bit overzealous with the infamous JJ Abrams lens flare effect, but the end result was very movie-like, to say the least.

According to the company, identifying which characteristics of the lens can and should be implemented in the software-generated bokeh is key to a natural simulation.

Recommended reading: Blueprint for the future of photography: Computational photography

For ZEISS, software is the obvious future

Interestingly, ZEISS’s approach toward smartphone imaging comes from a place of necessary compulsion. Despite having a legacy steeped in optics, the emergence of electronics and digital imaging has inevitably pushed the brand toward software. And there’s no bigger target market for a brand like ZEISS than smartphones.

Oliver Schinderbeck, Senior Smartphone Technology Manager at ZEISS added to the conversation, “Sometimes I like to say that smartphones are a big compromise. There are so many limitations and restrictions. Size, time, materials, and cost, and to bring all of these together, it takes a very good understanding of all components. Not just optics. But also capabilities of sensors, the algorithms, the software. And therefore, our focus has shifted from optics to the entire imaging system.”

A very significant pillar of ZEISS’s future in the smartphone imaging space is certification. The company has come up with a laundry list of parameters that define how a camera should perform. Part of this is the delta E level that defines the difference between how color is reproduced, and how it should be. Scoring a low delta E level is essential to getting certification from the brand.

ZEISS’s partnership with vivo, however, runs deeper. According to Schinderbeck, it’s more exciting to work with smartphone brands right from the R&D stage.

Our partnership with vivo takes a look at the holistic imaging chain starting from scratch, the optical design, setting the specification, deciding what is mandatory for optics and what isn’t, and what can be compensated by software. What features can be implemented and shouldn’t? If you have to do it in a traditional way, it’s not possible so we develop algorithms with vivo. Therefore staying limited to optics just doesn’t make sense anymore [for ZEISS]. — Oliver Schinderbeck, ZEISS

While ZEISS continues to work with other brands like Sony and is happy to lend its T-coating for reduced lens flare, its collaboration with vivo begins earlier and runs through the entire product cycle. Cinematic bokeh, for example, is a result of collaborative development right from the planning stage of the product.

We asked, you said: You don’t think smartphone camera partnerships result in better pics

The future of smartphone imaging is co-dependent

Outputting stellar images or videos requires more than just technical expertise. Everything from sensors to lenses has been commodified across the board, and the difference between good and incredible imaging comes down to just one thing — the software pipeline.

Moreover, the close collaboration between traditional camera companies and smartphone brands isn’t just a happy coincidence. The fundamental business of cameras has evolved significantly since the advent of the camera phone. For a company like ZEISS, not having a foot in the door could very well be a step towards obsolescence.

The combination of the two companies’ expertise is symptomatic of larger changes in the consumer photography landscape. Smartphone photography has reached a saturation point and it is rare to come across a bad camera, even in the mid-range. Sure, there will always be small preference differences, but the basics are to a large degree, a solved problem.

The next step in smartphone photography is bound to come not just from a place of technical brilliance — something that smartphone manufacturers excel at — but also from recreating the emotional aspect of photography. And that is an avenue that traditional photography majors excel at. It’s still early days, and not every collaboration between a smartphone brand and legacy camera or lens company is going to be a success. However, the meaningful and deliberate changes in vivo and ZEISS’s imaging integrations signal a step in the right direction.

Up next: The best cameras for every type of user