How Extended Realities Will Impact Food Processing

The information age—where physical experiences gradually give way to digital interfaces—is in full swing. Immersive technologies merge the physical and the digital or simulated world to provide a sense of being immersed in 360-degree environments.

Immersive technologies, including virtual reality (VR), augmented reality (AR), and mixed reality (MR), offer opportunities to interact with virtual objects and digitally overlaid real objects and places in order to enhance experiences. Although immersive technologies have been available since the mid-20th century, excitement over VR and AR innovation has mounted in the past two decades, which have been marked by the development of third-generation devices. However, the recent disappearance of VR and AR from the Gartner Hype Cycle, which tracks the life cycle of emerging technologies, indicates a new maturation era for these technologies.

Differentiating AR, VR, and MR

In VR, users are fully immersed in simulated environments that are different from their physical environment. Headsets, also called “head-mounted display,” provide a 360-degree view of simulated space, creating a sense of being in an alternative environment. Oculus Quest 2, Oculus Rift, HTC Vive, HP Reverb G2, and Valve Index headsets allow a touch-free input for operating a VR system. Common examples of VR include virtual tours of museums and real estate listings featuring simulated or recorded environments of physical spaces.

Augmented reality, on the other hand, enhances the actual physical environment of the user by overlaying digital elements and does not necessarily require a headset or controlled device. AR can be facilitated through 2-D headsets, holographic handheld mobile devices, or smart goggles such as Google Glass.

Applications of AR include the worldwide gaming phenomenon Pokémon GO, interior decoration apps via mobile devices, and small displays of information that appear before the eyes through goggles such as RealWear and the early version of Google Glass.

MR capitalizes on the benefits of both VR and AR by articulating and anchoring objects in physical space with added spatial understanding of the physical environments (i.e., spatial computing). Microsoft HoloLens, Magic Leap, and Real View Holoscope headsets create a mixed reality by overlaying objects such as annotations and 3-D graphics on a physical environment, thus enabling recognition of the physical components, matching the digital objects with them, and allowing users to interact with the digital and physical objects.

Mobile phones that use LiDAR (light detection and ranging) scanner technology to provide a depth map of surroundings open new possibilities for implementing MR applications.

Extended Reality

Extended Reality (XR), an umbrella term that represents VR, AR, and MR, extends the real world by simulating or adding digital components and unfolding a new set of interactive possibilities. XR creates disruptions not only in gaming and entertainment, but also in healthcare, education, manufacturing, marketing, real estate, retail, and the military.

According to research conducted on immersive technologies in education, training, behavioral sciences, telepresence, human-computer interaction paradigms, storytelling, psychology, ethics, and experiential design, immersive technologies are expected to completely transform how we interact with the digital world in both social and work environments. The global AR and MR market is expected to reach $326.1 billion by 2027, with a compound annual growth rate of 63% between 2020 and 2027, according to a MarketResearch.com report.

Developments in the Internet of Things, headgear, cloud computing, mobile devices, and big data drive demand for XR, particularly in the gaming and medical industries. There are opportunities for the food industry to take advantage of XR technologies in areas including remote work environments, workforce training, inspection, marketing, and customer support. The successful implementation of XR in the food industry has lagged over the past decade, however. With rapid developments in software and hardware, as well as a shifting work and operational culture spurred by changes during the COVID-19 pandemic, it is likely that this will change and XR will be adopted rapidly in the food processing industry during the next decade.

Training and Troubleshooting

Because it allows workers to train at their own pace, XR is valuable for employee training and onboarding and promotes learning retention. In addition, XR can introduce manufacturing lines, construction sites, or unfamiliar procedures—which may be costly and/or have safety risks—to a new workforce in low-risk environments. Lockheed Martin, in collaboration with Azure Mixed Reality Services, reduced costs by 93% and shaved more than 7 hr off an 8-hr activity in its spacecraft manufacturing operations area, the XR Association reports.

Taqtile, an industrial XR solutions provider, develops strategies to train frontline workers through the digital platform Manifest. Kelly Malone, chief customer officer for Taqtile, says remote work setups during the COVID-19 pandemic demonstrated frontline workers’ need for support from remote working specialists. He maintains that “80% of the workforce, corresponding to 2.8 billion people, is deskless” and “this portion of the workforce has not benefited from the ongoing digital transformation advantages.

 “A retiring generation is also taking accumulated years of institutional knowledge on equipment and processes with them, causing added burden on businesses to train the upcoming workforce,” Malone continues. “Immersive technology solutions address a growing need for training a next-generation workforce, which tends to have an affinity towards digital tools and data.”

Malone notes that MR tools have been used to allow employers to compress weeks of training into as little as three days. Case studies from various industries have shown an average 25%–30% increase in productivity with 46% faster task completion rates with the use of MR-assisted training (Deloitte 2019).

Manufacturers can also benefit from MR-assisted routine maintenance and line inspections with remote support options and material handling. Operators working to troubleshoot equipment or process lines can perform tasks by receiving step-by-step instructions through a mobile device or MR headset. This enables a workflow collaboration in a “see what I see” manner.

According to a study conducted by Deloitte to assess live remote support options, 84% of the 200 industry operators surveyed preferred remote live guidance through MR systems versus a standard video call, leading to 32% improvement in problem-solving during downtimes and 50% fewer errors (Deloitte 2019).

The Manifest platform integrates routine maintenance with step-by-step instructions and remote live support. For example, subject matter experts can guide operators to conduct a maintenance or trouble-shooting task with information displayed on a specific part of the equipment through MR headsets. Assistance can also be provided via previously constructed workflows. Operators can perform tasks by interacting with pictures, videos, and spatial indicators, such as overlaying 3-D digital ink or pointers over the physical space and overlaying sensor data from the equipment. In addition, evidence can be created to follow instructions during a routine maintenance activity. Internet connectivity limitations can be overcome by integrating offline operations with prerecorded instructions in the AR platform.

Marketing and Consumer Retention

The most widely implemented examples of XR applications may come from the marketing and sales areas due to the powerful storytelling features of immersive technologies. Food and beverage events, including the IFT annual event, have seen a growing traction in implementation of 360 tours.

Food companies are utilizing VR 360 tours to invite suppliers and customers in for a glimpse of their facilities and operations. For example, Farm Food 360 from Canada provides online virtual tours of Canadian apple orchards, livestock farms, and egg, oat, and cheese processing facilities via compatible web browsers, mobile phones, and VR goggles. Patrón distillery offers AR and VR experiences to consumers, taking them through the production, aging, and bottling of its tequila.

Experiential marketing strategies by food brands include Remy Martin’s AR integration with Microsoft HoloLens in 2017, Coca-Cola’s “Holiday Magic” campaign with AR apps, and Kraft Heinz’s interactive placemats in restaurants for gaming and AR-based recipe books for ketchup in restaurants, among many others. Despite the creative opportunities in marketing and consumer engagement, a clear understanding of the user journey and interaction with the product is essential, and implementation of these strategies integrated into a top-down digitalization ecosystem is needed (Rivera-Pesquera et al. 2021).

Opportunities and Challenges

The use of XR in food manufacturing operations can provide productivity, maintenance, training, and marketing benefits. However, for XR technologies to be fully incorporated in the food industry, several challenges must be overcome. For example, companies must not only target XR applications, but they must also integrate XR into a greater top-down digital transformation scheme. Some operators may resist wearing headsets or using AR-based software. The lack of internet connectivity can be a barrier to online-supported solutions.

Deploying these technologies in an unsecure manner can also be a vector for data breach and hacking activities. “The secure systems that use one-way data transfer as a part of supervisory control and data acquisition systems minimize the risks by reducing the data control,” says Taqtile’s Malone.

Also, even though AR technologies enable interaction with preconstructed interactive elements and facilitate feedback, AR systems cannot currently detect mistakes during operations, Malone points out. He adds that use rates within industry are expected to increase as more digitally savvy individuals enter the workforce.

To overcome technology challenges, processors can start small by implementing less-sophisticated mobile device–based solutions to prevent new technology rejection among employees. Business and investment leaders can also benefit from increased learning and engagement training in XR as well as increased monitoring of operations.

REFERENCES

Deloitte. 2019. Digital Reality changes everything: Step into the future. https://www2.deloitte.com/content/dam/Deloitte/dk/Documents/Grabngo/Digital%20Reality%20GrabNGo_2019_030419.pdf.

Rivera-Pesquera, M., S. Cacho-Elizondo, and R. Duran-Dergal. 2021. “Forget-me-not? Using Immersive Technologies in Brand-building Communication Processes: An Exploratory Study in the Mexican Context.” J. of Creative Communications. https://doi.org/10.1177/09732586211036768.