The past five years have seen groundbreaking production of theatre and opera that have integrated augmented reality (AR) and virtual reality (VR) – or, as the industry term refers to all types of immersive technology, XR – to develop new kinds of live and virtual experiences. The Welsh National Opera, in 2017, combined ‘The Magic Flute’ and ‘Madam Butterfly’ for ‘Magic Butterfly,’ an experience that used Google Daydream VR to demystify opera for audiences, in a new venue. Commonwealth Shakespeare Company produced ‘Hamlet 360: Thy Father’s Spirit’ in the past year, working with Google’s AR/VR Lens team to create five seven-minute acts in VR.

Yet, some of the boldest work in XR for theatre is not surprisingly being produced in New York. An event produced by Chris Pfaff, and sponsored by the VR AR Association New York Chapter and Verizon’s 5G Lab, on Wednesday, November 6, 2019 featured some of the leading practitioners of XR in the theatre world who discussed their work and and their platform and technology choices, as well as how to shape narratives for interactive audiences in real-time experiences. Demonstrations of their work, as well as a guest appearance from award-winning XR production firm Felix & Paul Studios, were held prior to and after the main event, which was held in Verizon’s 5G Lab at the Alley powered by Verizon.

The entire event can be viewed at: https://www.youtube.com/watch?v=wQQhDrp0vtQ&t=31s

Brandon Powers, producer/choreographer of ‘Frankenstein AI,’ a piece that premiered at the Sundance Film Festival in 2018, and producer/choregrapher of ‘Queerskins,’ discussed how he worked with the Intel Studios volumetric studio to use his choreographic and immersive theater expertise to move ‘visitors’ through the virtual space. ‘Frankenstein AI,’ which expressed our AI inhabiting a human body to communicate with the audience, and ‘Queerskins’ demonstrate ways for theatre producers to create new work with volumetric capture. In ‘Frankenstein AI,’ the human dancer performed choreography, which was created using a new choreographic notation that Brandon invented. The choreography was manipulated by an AI in real-time based on how the audience responded to the AI’s prompted questions, prompting the dancer to alter her performance for a unique live experience. Brandon is also developing a new episode in the VR experience ‘Queerskins,’ which involves an interactive choreographed duet utilizing volumetric capture which will give a unique experience for each user.

Kiira Benzing, from Double Eye Studios, the renowned producer of ‘Loveseat,’ a VR theatre piece that was featured at the 2019 Venice Biennale, in which two lonely, ordinary people are drawn into a reality show competition to win the love of a Perfect Partner (who looks an awful lot like an empty chair). Part-story, part-interaction between virtual and real worlds; the performers intersect realities in this comedy. Performed simultaneously to virtual and real world audiences. Kiira also discussed her work on ‘Loveseat’ and her more recent work on ‘Runnin,’ which was shot at Intel Studios in Los Angeles. ‘Runnin’ is currently available through the Samsung Gear app through Oculus Quest. Her work in designing virtual worlds has led her to ask ‘what is a stage?’ and ‘what is an audience?’ when audience members are represented by virtual beings and avatars.

Maciej Wisniewski, 99 Center Opera founder, and producer and composer of ‘Edelweiss VR, Act 1,’ or EVR1, the first original VR opera experience, discussed and demonstrated his work in creating a music-driven narrative that enables viewers to unlock hidden aspects of the story. Maciej’s work includes blockchain for encrypted messages that will be retrieved by the players of the EVR1 game, to unlock content. The story of EVR1 takes viewers through an interactive VR story in five acts, based on a true World War II experience of his great-aunt escaping a prison in Poland and becoming a Resistance fighter. Maciej’s work involves breathing light and texture into scenes based on music.

David Rodriguez, founder of Zanni, the first Audience Experience Design Studio, demonstrated the firm’s XR tools, Ovees™ and Zanni™ AXD, for both immersive audiences and theatre producers. David’s work with the Metropolitan Opera led him to consider how opera, and theatre, could evolve through VR and AR tools for both audience participation and narrative extensions, as well as how to enable immersive workflows for theatre/opera producers. Ovees is an XR take on stereoscopic opera glasses – and Zanni AXD – a mixed reality design tool that creates space to view stage production elements, reducing design process time and enhancing stage experience planning for design teams. Zanni is a new company, and the event marked the public debut of Zanni.

The panel discussion delved into how traditional theatre producers, directors, lighting designers, choreographers, and actors themselves are moving to expand the medium through immersive technologies. The group discussion yielded the overarching understanding of theatre being gamified and moved into a new genre altogether. The panelists discussed how technologies are rapidly evolving to meet demands for creative decisions. Kiira Benzing discussed how difficult it was to mount ‘Love Seat’ in Venice, with a massive point cloud that could, at any time, crash. The shared experience of theatre, in a live setting, pushes the boundaries of technical production for real-time communal energy.

Chris Pfaff welcomes the audience at ‘XR in Theatre: Immersing New Audiences’ at the Verizon 5G Lab, 11.06.19

Chris Pfaff, Mikael Chagnon, from Felix & Paul Studios, and Brandon Powers

Maciej Wisniewski (foreground) and Kris Ramanathan (rear) showcase their EVR1 musical interactive VR piece

Brandon Powers discusses his immersive choreographic work on ‘Queerskins’ and ‘Frankenstein AI’

Kiira Benzing discusses ‘Runnin’ and ‘Love Seat,’ her interactive VR theatre pieces, and her work with Intel Studios

Maciej Wisniewski, from 99 Cent Opera, discusses his use of blockchain to unlock content in the interactive VR musical narrative

David Rodriguez, from Zanni, discusses his use of immersive tools for theatre and opera

The Q&A session for ‘XR in Theatre: Immersing New Audiences’ 

The 100+ audience at ‘XR in Theatre: Immersing Audiences,’ at the Verizon 5G Lab in New York, 11.06.19

David Rodriguez; Brandon Powers; Chris Pfaff; Kiira Benzing, and Maciej Wisniewski, after the ‘XR in Theatre’ panel

VR/AR Association New York chapter members Chris Pfaff, Cindy Mallory, Michael Owen, and chapter president Gordon Meyer

Maciej Wisniewski, Linda Aro, and Kris Ramanathan before the ‘XR in Theatre: Immersing New Audiences’ event

Kiira Benzing, Chris Pfaff, David Rodriguez, and Brandon Powers before the ‘XR in Theatre: Immersing New Audiences’ event

VR/AR Association New York chapter member Chris Pfaff, chapter president Gordon Meyer, and member Gordon Yee before the event

The third anniversary of the VR/AR Association (VRARA) New York chapter was a positive snapshot on the growth of the immersive/XR industry in the greater New York area in the past few years. The event was hosted on Tuesday evening, May 21^st, at RLAB (https://www.rlab.nyc), the massive space that NYU and its academic and City of New York partners opened in late-November, 2018. RLAB hosts several companies, including members of its XR Beta program, and is being built out as a larger facility for immersive innovation, including volumetric capture space.

Dex Yee (left), from VRARA, watches as Jason T. Jaslow signs in the VRARA NY 3rd anniversary

Chris Pfaff, VRARA NY Chaper advisor, welcomes the crowd at RLAB

As VRARA has grown globally, so too has its New York chapter, and this mirrors the broader focus on the XR industry in New York, which embraces enterprise and consumer firms alike.

Banu Ozden in discussion with Janice Brown, manager, education and outreach at RLAB

Robin White Owen and Michael Owen, the first XR couple of Brooklyn, at Rlab

Unseen Media demos its soon-to-be-released narrative AR game

Gur Arie Bittan, from Mantis Vision, demos for Banu Ozden and Jeffrey Ginsberg

More than 50 guests attended the mixer event, which featured demos from XR Beta companies, as well as Mantis Vision (http://mantis-vision.com), and remarks by RLAB’s Alexis Seeley and VRARA New York chapter advisor Chris Pfaff. And, of course, numerous demos of mobile AR experiences were shown by VRARA members as well.

Robin White Owen and Michael Owen listen to Tim Meyer, from IBM’s IoT group

Alexis Seeley, director of education and opportunity programs at Rlab, welcomes the crowd

Mantis Vision’s mo-cap installation, and some of its forthcoming collaboration tools, were a major hit at the event. Unseen Media ((https://www.unseenmedia.io), a narrative AR game developer, demonstrated its soon-to-be-released game, while echoAR (https://www.echoar.xyz), and AR-focused CMS and CDN provider, and SIY (Speak it Yourself – https://www.siyvr.com), a VR-based language instruction firm, demonstrated their solutions as well.

Chris Pfaff and Gordon Meyer, one of the Top 5 AR influencers in the industry

Chris Pfaff and Kate Specter, from Toonpack, at Rlab

Alexis Seeley and Janice Brown, with Rlab, prior to the VRARA event

It is always a pleasure to work with Tracey Swedlow and her team at the TV of Tomorrow Show. I have known Tracy since 1998, and she has indefatigably helped lead the discussion around advanced TV and video for more than two decades. And, yes, we shared the virtual stage in a famous Producers Guild of America (PGA) New Media Council event in February, 2008 (it was a webcast, with me in New York at the New School; she in San Francisco, at the Macromedia building), in which we presented panelists and debated the New York vs. San Francisco tech/new media scene.

At the recent TV of Tomorrow Show New York, held at the SVA Theatre in Manhattan on December 7th, 2018, I moderated a session, ‘Machine Learning and Artificial Intelligence for the OTT Universe,’ that dove into a wide range of issues facing broadcasters; MSOs; user-generated content, and advertisers as machine learning (ML) and artificial intelligence (AI) increase their sophistication in the overall OTT world.

My panelists included:

Romain Eude, CEO/founder, Utelly

Randa Minkarah, COO, Transform

Janne Neuvonen, CEO/co-founder, BCaster

Aman Sareen, CEO, Zypmedia

Have a listen to the audio from this session by clicking the link embedded in this post from the TV of Tomorrow site:

https://thetvoftomorrowshow.com/radio-itvt-machine-learning-and-artificial-intelligence-ott-universe

The orgy of bombastic advertising self-love otherwise known as the Super Bowl, the National Football League’s annual championship game, has become a prisoner of the web and social media, and cannot justify the outrageous cost of broadcast TV spots. This was particularly evident in Super Bowl LIII’s mostly miscued spots and lost opportunities, highlighted even further by the lowest-scoring Super Bowl of all time.

All of this sounds familiar: brands using the bullhorn and the pie in the face to scream over the din. But that isn’t the real issue: it’s a lack of understanding where the audience is. The audience for the Super Bowl is everywhere (even on smart speakers), not just TV, and while TV is still king, it can no longer justify a one-screen-fits-all story.

What mystifies me about Super Bowl LIII’s ads is the lack of innovation, of 2^nd-screen and even 3^rd-screen interactive installations, calls to action, or use of technology that would make a non-football fan interested in a brand’s message. Why merely put up 30 seconds of overstuffed hyperbole and visual mayhem on broadcast television when you can leverage people’s more ubiquitous viewing screens: their mobile devices. This seems to me to be a travesty of capitulation, not mere conservatism.

I say that the Super Bowl ads are a prisoner of the web for a few reasons. One, it seems that brands are so skittish about instantaneous negative or critical reactions to spots that they are not taking any risks. This blandness and message sanitization is the curse of corporate communications in an age of increased polarization and, yes, fake news, but it doesn’t make sense if you are spending $5 million for a spot, on top of production and promotion costs. The “go big or go home” mantra for the world’s biggest advertising moment (never mind that more people watch the World Cup Final than the Super Bowl) seems to have been reduced to “go safe, or go home.” Secondly, the prisoner analogy seems apt in the way that brands now seem to have capitulated to the idea that whatever transmedia implementation they might embark upon is not worth the risk, or is only of interest to a tiny sliver of the audience. In an era of short-form content (even with the death of Vine), there seems to be much more that brands can do around the Super Bowl.

To be fair, the spots that were well-produced (Bud Light, Budweiser, Turkish Airlines, Weather Tech, to name a few) either have longstanding audiences and built-in online and social media audiences, but one would have expected some kind of prompt to an app or web extension, or even Facebook conversation. The mere fact that Ridley Scott returned to the Super Bowl with the cinematic thriller of an ad for Turkish Airlines was exciting, and had a nice call to action. The Burger King spot with Andy Warhol eating a Whopper in 1982 was amazing just for the site of Warhol eating a Whopper. Who knew? This was from a Swedish film, and the Warhol segment has been posted on YouTube. Perhaps #EatLikeAndy will catch on, and become part of a larger campaign, but there is no interactive component. Warhol would have found this a waste. He would have pulled all kinds of tricks with our mobile devices, game consoles, and even smart speakers. It was ironic that he showed up on this Super Bowl ad roster, in this sense.

And, yes, one of the few tech-related ads, from Amazon Alexa, was a massive failure of imagination and use of the technology. No spiffs for users of Alexa, no interesting narrative about the technology, and no tie-in via Amazon’s many sites (or, for that matter, Whole Foods). Just a colossal waste of time and money, with Harrison Ford (yes, get Han Solo and his dog to sell a backward message about what Alexa doesn’t do well) looking old and cranky. This was quite sad to see, on an evening where few tech companies were advertising.

If the Super Bowl really were the kind of event that we would all watch, regardless of our football affinities, the NFL and the broadcaster would make it as participatory as possible, leveraging more than 8K cameras, surround sound or the latest iso camera position. But, under fire for its handling of Colin Kaepernick and the Chronic Traumatic Encephalopathy (CTE) lawsuits that former players have filed, the NFL and its advertisers seemed to feel under pressure from a blitz – from its own would-be audience. Hopefully, Super Bowl LIV will “LIV” up a greater expectation for putting on a show, and actually having fun with its audience, instead of just talking to themselves. And playing it safe.

Take a listen to my conversation with my brother, Fred Pfaff, on the hits and misses of Super Bowl LIII ads, which we conducted on February 4, 2019 over lunch at P.J. Clarke’s at Lincoln Center, New York City. You can listen at: https://soundcloud.com/chris-pfaff-1/chris-and-fred-pfaff-discussing-super-bowl-liii-ads

Fred Pfaff (left) and me, at P.J. Clarke’s at Lincoln Center, NYC

The industry gathering that officially cemented Vancouver as one of the major centers of virtual reality and augmented reality (or, just use extended reality – or XR – as your umbrella designation), the VR/AR Global Summit, provided a view of the technology and creative sectors that are driving new avenues for audiences and investors on a global scale. Held at the gleaming Parq Vancouver hotel and casino, which just recently celebrated its first anniversary, the event was largely sponsored by Vancouver’s events chamber, and had the British Columbia economic development stamp of approval all over it. And, for those of us who have known Vancouver as a powerhouse in the visual effects and videogaming spaces for years, the increased emphasis on Vancouver as a hub for XR development of all kinds is both obvious and welcoming.

Nathan Pettyjohn welcomes the crowd at the VR/AR Global Summit in Vancouver, September 21, 2018

HP Entertainment’s Joanna Popper details HP’s work in the immersive space

While the VR/AR Global Summit, held on Friday, September 21^stand Saturday, September 22^nd, was as much a homecoming and meet-up for members of the three-year-old VR/AR Association, many of whom had never met in person (and, yes, Kris Kolo, the elusive, near-mythical executive director, was there for people to actually meet, and dispel rumors that he is merely a clever avatar), the event summoned an even greater level of introduction to innovation that is happening rapidly in the immersive space, even for those whose revenue depends on some form of XR development.

The brainchild of Anne Marie Ens, executive producer of the event, the VR/AR Global Summit brought leaders from Hollywood, New York, Washington, D.C., the Bay Area, Taiwan, Ukraine, New Zealand, and, of course, all across Canada. What was resident, even from hardware providers, at the event was the level of creative production that is just starting to bubble in a meaningful way from major tech companies, from Intel’s Optane platform, showcased in its Smithsonian Museum Renwick Gallery walk-through, to HP’s impressive work with the likes of VR Studios, which itself got a PR boost the week prior to the event with Cineplex’s announcement of more than 40 full VR arcades to be installed across Canada by 2021.

Intel’s Raj Puran, with YDreams’ Daniel Japiassu; VR Studios’ Chanel Summers; Dark Slope Studios’ Ben Unsworth, and DreamCraft Attractions’ Krystian Guevara at the location-based entertainment panel, September 22, 2018

Perhaps the most famous global IP that was on display at the show was from Taiwan’s Studio2 Animation, which debuted its VR series of shorts for the popular animated character Barkley the cat. The 6-minute block of 25 shorts marks a major move into VR for animated content; the Barkely feature film was a huge hit in mainland China and Taiwan in 2016.

Chris Pfaff, Grace Chuang, and Chiu Li Wei (Studio2 Animation) with Studio2’s Barkley the cat

Producers were quite evident at the show, and highlighted by back-to-back panels on Saturday afternoon, as Chris Pfaff led the ‘Producing in XR: What to Know Before Immersion’ panel, with Silverscreen Cinematics’ Jeff Olm; AWE Company’s Srinivas Krishna, and MediaCombo’s Michael Owen. The panel explored major AR and VR production techniques, as well as post-production considerations. A second panel, ‘Storytelling and Content Creation in VR/AR’ featured Observe Media’s Travis Cloyd; InspireVR’s John Penn; Vuze’s Jim Malcolm, and Cloudhead Games’ Denny Unger.

Michael Owen, from MediaCombo; Jeff Olm, from Silverscreen Cinematics; Chris Pfaff, from Chris Pfaff Tech Media, and Srinivas Krishna, from AWE Company, prior to, and on the ‘Producing in XR: What to Know Before Immersion’ panel, September 22, 2018

Michael Owen, Srinivas Krishna, Chris Pfaff, Travis Cloyd, and John Penn after their panels, September 22, 2018

Local Vancouver firms were quite visible, from Cognitive3D to Mythical City Games to Stambol Studios. Dark Slope Studios, based in Toronto, held a private event with its principals; the studio is creating location-based immersive entertainment and features an all-star cast of principals, including Raja Khanna, Ben Unsworth, and CJ Hervey.

Raja Khanna, center, executive chairman of Dark Slope Studios, welcomes the audience at a private event, Tap & Barrel, Athlete’s Village, Vancouver, September 21, 2018

Of all the myriad experiences being demonstrated on platforms from Vive to Hololens to, yes, Magic Leap, perhaps the most compelling was YDreams’ ‘The Last Squad,’ produced with ArkaveVR. The 3-person VR shooter is an ideal arcade game, and is addictive in its graphical clarity, screen direction, and sheer entertainment value. As a sign of Vancouver’s attractiveness for the XR community, YDreams recently moved its company to the city. The BC Tech group put the final punctuation on the event by holding an after-party at The Cube, the 6,000-square foot co-working space that hosts a wide range of start-ups in the VR/AR space.

The Cube, scene of the after-party for the VR/AR Global Summit, Vancouver, September 22, 2018

Overall, the VR/AR Global Summit was an affirmation of what the VR/AR Global Association set out to accomplish when Nathan Pettyjohn and a few people started a truly global organization in 2015: bringing the immersive community together on a grand scale. This is still an industry in its early days, and yet even those who have been involved in it for decades see something new every day. So it was in Vancouver; another eye-opener for the immersive crowd.

Amar Dhaliwal, Atheer; Parm Sandhu, Telus, and Kris Kolo, executive director of the VR/AR Association 

Some of New York’s leading AR technology developers attracted a crowd at The Alley, Verizon’s 5G collaborative lab and work space in Chelsea last Tuesday, May 8^th, at an event sponsored by the VR/AR Association’s New York chapter. ‘AR for Producers: How to Engage Audiences on Multiple Platforms’ was well-titled, as most of the audience was comprised of new media and television producers who are looking at AR for a wide range of production elements, including fan engagement, 2^nd-screen extensions, and location-based entertainment.

JR Dawkins welcomes the crowd to The Alley     Michael and Serge Doudy provide an overview of the VR/AR Association

Eric Schwertzel, head of business development for eyecandylab’s US presence, demonstrated the AugmenTV concept, in which TV content triggers experiences on mobile using an AR app. This is a long sought-after play, and should have traction in sports as well as episodic TV.

Eric Schwertzel discusses AugmentTV

Gordon Meyer, head of marketing for Lampix, demonstrated the company’s projector-based AR solution, which is ideal for retail and public venue experiences.

Gordon Meyer presents the Lampix innovation

JR Dawkins, head of business development for Envrmnt by Verizon, showcased the Envrmnt AR Designer tool, the industry’s first true drag-and-drop AR tool. The simplicity of the tool enables anyone – with no coding experience – to create AR triggers on physical objects, and more.

JR Dawkins presents AR Designer      Ryan Hilla assists with AR Designer demo

Far beyond what previous generations of AR solutions have provided, the presenters discussed the value that 5G will bring to AR, enabling seamless, low-latency experiences with greater connectivity levels.

JR Dawkins, Gordon Meyer, and Eric Schwertzel discuss the AR industry

Chris Pfaff, JR Dawkins, Gordon Meyer, and Eric Schwertzel during the panel discussion at ‘AR for Producers’

For many in the audience, it was their first time in The Alley, which opened last spring, and has gained a following among the technorati of the city.

Lampix demo at ‘AR for Producers’

Chris Pfaff and Ally Perez, event coordinator for The Alley, powered by Verizon

Heikki Rotko, founder and chairman of Media Honeypot (right), welcomes the crowd at Media Honeypot in Helsinki, as Janina Salo-Glasemann (center), CEO of Media Honeypot, and Ralph Simon (left), CEO of Mobilium Global, look on

The 3^rd annual Media Honeypot Helsinki, the gathering of the European media start-up tribes, was another remarkable showing of the strength of media innovation in Europe. This year’s event, held on February 15th at the Crowne Plaza Helsinki, did much more than match the leading media companies in Europe with the leading media start-ups – it reinforced what many of us in the media and telecom industry have known for years: that Europe’s broadcasters are the most progressive in the world, and have been natural incubators for some of the most creative applications of media tech. With last year’s opening of Media City Bergen in Norway, there is much more activity on generating clearly defined start-ups that can address tomorrow’s needs for media companies.

Tomas Franzén, CEO of Bonnier, presents his keynote at Media Honeypot 2018

The Media Honeypot program – expertly curated by Janina Salo-Glasemann and her team – was emceed by the estimable Ralph Simon, whose bon mots in Finnish and Swedish brought more than a few chuckles from the crowd. Tomas Franzén, CEO of Bonnier, discussed how the company has managed to move into the digital space, while also nurturing a venture arm. He reminded the audience that, at its core, Bonnier’s chief job is to promote and protect democracy; a welcome message during these times. Moritz Holzgraefe, with Axel Springer, was one of several major publishers on the lookout for new startups; his company has certainly led the way in transforming a proud old publisher into a digital powerhouse. Laura Avonius, co-founder of Data Refinery, showed how her unit at Aller Media turned the company around by using its own data and spun it off as Data Refinery.

Laura Avonius (left), co-founder and CEO of Data Refinery, presents Data Refinery’s case for publishers

Anthony Herman, now heading up Amazon Web Services in the Nordics, discussed his background as a Silicon Valley venture capitalist, and how that informs his current role. Trygve Refvem, head of MTG Ignite, discussed what a broadcast group looks for in start-up development.

Valerie Vlasenko, from Arctic Startup, introduces the pitching finalists at Media Honeypot

Yet, for all of the prescient case studies and presentations and discussions regarding the next step for European media companies and start-ups, the most impressive aspect of Media Honeypot was the line-up of companies participating in the event. Frankly, one would have been hard put to identify an event that could present the likes of what we saw at the Crowne Plaza Helsinki last Thursday. From Stream Time, developer of a true cross-service EPG to Bibblio, provider of a content relevance and recommendation platform to Adlaunch, creator of an AI-based ad creation platform, there was a surfeit of talent and vision to be matched with the right media company.

And, yes, the pitching competition presented five finalists – Cloudbounce, Utelly, Kieku, Bibblio and Valossa – that virtually advertised the event for anyone paying attention: the leading AI media start-ups are in Europe. Valossa convincingly won the pitching competition, with founder/CEO Mika Rautiainen giving a clear reason for why Valossa delivers value for media companies.

The winner of the Media Honeypot 2018 Pitching Competition was Valossa. Here are, from Valossa: Sami Niski (left), VP of Sales; Mika Rautiainen, founder/CEO (center), and Tommi Karjalainen (right), business development director

Mika Rautiainen, me, and Janina Salo-Glasemann, CEO of Media Honeypot

Heikki Rotko, founder of Media Honeypot, has built something that should only expand in the coming years, as more European broadcasters and publishers seek new revenue streams.

Heikki Rotko (left), with Trygve Refvem, head of MTG Ignite, at Media Honeypot 2018

The CES 2018 that has come and gone will be remembered less for the things that we build and think that we can control – namely, AI systems, robots, and high-resolution displays – than for yet another instance of Mother Nature interfering with our well-laid plans, in the form of more than one inch of rain on Day 1 of the great CES show (Las Vegas receives around 4 inches of rain in an average year, so the Great CES 2018 Deluge was almost one-third of the annual intake. Many Las Vegans were quick to pridefully point out that “we just had 170 consecutive days without rainfall!”). And, oh yes, we will remember CES 2018 for the lights going out on Day 2 of the show (I was giving an executive VIP tour in the Intel booth when the power hit occurred). This produced a 2-hour shutdown of the Central and North Halls of the Las Vegas Convention Center. Somehow, we were given an indication of the things that we will soon be better prepared for – if only humans would listen.

But the humans were listening intently to the messages all over town during CES 2018: more autonomy; more AI; more machine learning. This may have been the year that it was clear how companies with well-developed, simply-focused point solutions could break through and show the way to not just new behaviors, but new industries. The big players – from Samsung to LG to Sony to Intel – were either moving in step to the trend, or were completely in thrall to the tune. CES acknowledged that the IoT and Smart Cities sectors were not just another way to bring in new crowds of well-funded companies, but a recognition of how the CE industry is now part of a larger industrial revolution, in which the device is – as we used to define it in telco speak – the “endpoint” on the greater network.

AR and VR were somewhat subdued at the show, even though HTC introduced its latest Vive headset with Wi-Gig, the Vive Pro, a nod to the emerging short-throw high-speed wireless transfer protocol as a way to untether headset wearers (getting the damn headset off their heads is the next step, but we’ll save that for another chat). But the likes of Vive and Oculus were not on the show floor, leaving more room for 3Drudder and others to show off new gear. CES Unveiled had its share of interesting start-ups, including Sniffy, a French platform for delivering fragrance on demand with touchscreens (see the demo at: https://www.youtube.com/watch?v=5dNPov0nsiA) and Bellus3D showed off its 3D face scanning camera for mobile devices.

Bellud3D shows off some face masks made with its 3D face scanning camera for mobile devices

Autostereoscopic 3D displays made a big move forward with streamTV Networks, which showed off a 65-inch 8 million pixel display that comfortably rendered 3D images from a safe distance.

streamTV Networks demonstrated an impressive 65-inch 3D auto stereoscopic display

When it came to AR and VR at the show, Intel made the most impressive statement, demonstrating its TrueVR to great effect, showcasing 360 real-time processing for sports venues as well as real-time streaming Web VR for the upcoming Winter Olympic Games. Intel announced that it was making yet another foray into Hollywood with the launch of an LA-based studio that will manage the back-end processing of the live streams. Intel also created a full VR replica of its CES booth, available for consumers to download as a full Unity experience for Vive. This was an amazing use of the venue and the technology, as viewers could even interact with Intel experts from Intel technology centers around the globe. Intel also showcased its work in the 5G arena, setting up a 5G network in its booth, courtesy of an Ericsson base station and Nokia repeaters. 5G was not as dominant a theme at the show this year, even though the year started with great optimism from carriers, notably Verizon, regarding their 5G trials in 2018.

Intel showed off more of its drone family, including the new Shooting Star drone, which is optimized for synchronized flights, such as the one that accompanied the Bellagio’s dancing water experience after Intel CEO Brian Krzanich’s keynote address on Monday night. The Intel experience also included one of the coolest demos at the show, a real-time 3D facial effects demo from its Intel China lab, originally used to create effects for a popular Chinese singer’s performance. The demo, using a standard Logitech web camera with algorithmically derived real-time processing, produced vivid, engaging facial effects. Intel was one of many companies highlighting its AI efforts, with an “AI Tower” comprised of code-based images that triggered an AR demo, shown on a Surface Pro tablet, which detailed Intel AI projects, including a “snotbot” for tracking whales. Intel also surely won the “Great CES Blackout Entertainment Award,” when a violinist, Häana, who was part of a synchronized music and images performance strolled out to the stage and played a solo, proving that analog still rules (at least, in some instances).

Intel’s Mobileye demo of smart sensors for vehicles

The Intel Shooting Star drone, optimized for synchronized flying

Häana, violinist who played a solo in the dark during the Great CES Blackout, and her trio mates, at the Intel booth. See video of her solo here: https://www.youtube.com/watch?v=R8JsMfnPOfc

The Intel “AI Tower,” with Aditi demonstrating the AR-driven experience showcasing different Intel AI projects

Intel’s real-time 3D facial effects demo. See video of the demo here: https://www.youtube.com/watch?v=R8JsMfnPOfc

AI at CES was mostly seen, in its most progressive form, in the North Hall – dubbed by some wags as “The Las Vegas International Auto Show” – but there were a number of leading AI companies conducting demos off the show floor, including Valossa (showing in the Sprockit Suite at the Cosmopolitan Hotel), which added new video insight features to what many in the industry consider to be the leading video AI platform.

Samsung’s CES presence was a radical departure from past years, for several reasons. For one, Samsung seems to have given up its annual tit-for-tat with its Korean arch-rival, LG and instead focused less on individual product lines and more on the underlying technology and platforms necessary to effect the smart home, or smart society. Samsung even highlighted some of its work in telematics, featuring an impressive OLED screen as a dashboard that could show video feeds from your Samsung refrigerator to indicate what food you need to buy on the way home. Samsung played up the number of partners now working with its SmartThings platform, and focused on its messaging capabilities for the SmartHub – even sending messages to the front screen on its latest refrigerator. The most impressive demos were saved for its display area, which did not showcase QLED TV products, but rather highlighted concepts, including a 146-inch 4K display, ‘The Wall,’ which is the world’s first modular screen. This was an amazing conceptual demonstration: tiled screens that can be modularized, and even display different resolutions. This could be a revolutionary way to deliver on-demand viewing experiences. Imagine if you wanted to see the news in a lower resolution than your favorite TV drama, or wanted to see a film letter-boxed. Screens could be modularized to accommodate larger viewing audiences, or to satisfy particular content resolutions.

‘The Wall’ at Samsung’s booth – a 146-inch display, the world’s first modular display

Samsung had a Video AI demo that showed some of the concepts on which it is working that hint at just these kinds of future product directions. Samsung, by the way, did not even try to outshine LG in the content display war. LG, which seemingly has won this battle for several years running now, continued to show incredible images in massive displays at the front of their booth.

LG wins the high-res display wall war, again, at CES 2018

One of the other major news stories at CES this year had more to do with something that did not happen than something that did. I am referring to Huawei’s thwarted press conference with AT&T to introduce the Mate 10 smartphone, which AT&T was planning to sell in the US, marking Huawei’s official US market introduction. The day before the scheduled event, AT&T announced that it was pulling out of the deal, no doubt receiving heavy pressure from Washington. This put the brakes on the Huawei keynote address by Richard Yu, CEO of Huawei’s consumer products group, which saw him stumbling and mumbling his way through a lengthy sales pitch for the Mate 10. The phone itself is worth looking at (this was the “AI phone” that Huawei said it was working on last year), and breaks new ground in displaying relevant contextual content for users.

Set-up at the Huawei booth, which featured the ‘WOW WAY’ tagline

The Huawei booth at CES 2018. See demo of the Huawei Mate 10 smartphone here: https://www.youtube.com/watch?v=KGulQdjQwqw

The North Hall was all about enhanced features for autonomous vehicles (AVs), and aside from General Motors (who did not show, given that the Detroit International Auto Show was too close to CES this year), all the major auto manufacturers were on the floor. Delphi was back on the floor this year, after a foolish move at CES 2017 to draw traffic to a parking lot near the convention center. New faces, including Austin Electric Vehicles, were there as well.

Ford did not shine this year, even though their Smart City moving video wall was an impressive work of art. Mercedes-Benz outperformed. Their MBUX (Mercedes-Benz User Experience) platform showcased personalization as the hallmark of every car they will sell going forward. One cool feature was from a London start-up, What Three Words, which delivers a new kind of navigation system based on three-meter blocks being crushed into three-word addresses (like “dog.banana.fish”). This is, according to Mercedes-Benz, a more accurate way to deliver directions and locations than GPS addresses. Mercedes-Benz also showcased the Project 1, a 1000 horsepower engine futuristic vehicle that will sell in limited quantities for $3,000,000.

The Mercedes-Benz Project 1: a 1000-horse power engine and a $3 million price tag

Nissan showcased its b2v (brain-to-vehicle) system, which reads brain waves through headrest detectors to help deliver better predictive moves. Nvidia showed off its new AV processor, which is capable of 320 trillion operations per second.

The biggest splash on the CES car show floor was made by Chinese firm Byton, whose S.I.V. – Smart Intuitive Vehicle – was introduced at the show, marking only the second time that a car has been introduced at the show (two years ago, troubled Chinese firm Le Eco’s subsidiary Faraday Future introduced a car, which may never see a street, but – oh, what a buzz it was!). Byton’s vehicle will sell, according to its press release, in the US in 2020. Priced at $45,000, in the Tesla ballpark, the Byton vehicle has biometric sensors, AI, and a single screen dashboard, along with a tablet in the steering wheel.

Byton, a Chinese car company, introduced its S.I.V. – Smart Intuitive Vehicle, at CES 2018

Kia Motors showed off a personal assistant, and also touted what it calls the first in-vehicle 5G connection, showing a content stream from Seoul to Las Vegas, using the HECAS low-latency mobile video platform. Honda showed off its robotics, and no cars, further emphasizing their focus on CES as an opportunity to sign up OEM partners. Toyota won the cool concept car award this year, with the e-Palette, an autonomous delivery van with smart screens for digital signage. The boxy vehicle was an interesting design play, and was somewhat reminiscent of the Accessible Olli vehicle – the world’s most autonomous accessibility bus – that was on display in the Central Hall concourse of the Las Vegas Convention Center. Accessible Olli is already on the road, and is a joint venture between the CTA Foundation, IBM Watson and Knoxville, Tennessee-based Local Motors, which 3D-printed the vehicle. Accessible Olli fits up to 12 people, has smart screens, can determine if a passenger is blind, and has an ultrahaptic air button for signaling stops.

Accessible Olli – a smart, AI-powered autonomous accessibility bus

The e-Palette autonomous delivery van, a concept from Toyota

Nvidia introduced the first chip designed for AV use, capable of 320 trillion operations per second

Nissan unveiled its brain-to-vehicle (b2v) platform at CES 2018

Eureka Park this year was another strong showing by La French Tech, with about 350 French start-ups, almost half of all the start-ups in the area. There were many more clever IoT and mobile start-ups, as well as a smart musical ring, using MIDI, from Enhancia. My favorite Normandy start-up, Event Bots, was at the show, demonstrating their hospitality robots.

La French Tech was dominant at Eureka Park. Normandy start-up Event Bots was part of the massive French presence

The South Hall featured some new players, including Alibaba, which took over the South Hall Lower front booth. Alibaba’s presence reminded visitors of the strength of China’s companies at CES 2018. From TCL to Huawei to Baidu and Alibaba, Chinese technology was on display in great force. As a further example, Hi-Sense, which took over the Microsoft booth in the Central Hall, opposite Intel, a few years ago, showed a stunning laser TV product, which uses a short-throw laser projection system for high-resolution, large-screen displays. This is an astonishing change from just 8 years ago, when the only Chinese companies at CES were tucked away in a lackluster “China Pavilion” in the then-Las Vegas Hilton ballroom area. Now, they are a dominant presence.

Alibaba took over the South Hall lower level front booth this year

Other South Hall stops of note included gaming PC company Razer, who had their gaming tournament large keyboard on display. Speaking of gaming, Atari had a pong exhibit on the floor, creating great pangs of nostalgia for all.

Retro time meets Big Time: Atari Pong (left) and the Razer tournament gaming keyboard for e-sports

Last year’s CES was the “Alexa Show,” with Amazon’s intelligent assistant being shown in dozens of booths. This year, Google tried to step up to the plate, branding the Las Vegas Monorail with “Hey Google” and setting up numerous “Hey Google” signs in partner booths. A large Google Assistant pavilion in the Central Hall parking lot made its presence felt. They need it, of course; Amazon has about 70% of the smart speaker market to Google’s 23%, but many people would have argued that Amazon actually won this year’s battle, given that Alexa was still so prevalent. Cortana, Microsoft’s assistant, was virtually nowhere to be heard.

The Google Assistant pavilion in the Central Hall parking lot

What we have learned from CES 2018 is that more consumer choices will be made based on AI-powered devices or platforms and more products will highlight their AI or on-demand capabilities more than ever. It will be interesting to see how this trend translates into next year’s show, but it augurs a shift into the cloud in greater measures for the CE industry at large.

On the first morning of IBC 2017, Chris Pfaff Tech Media and Integer 1 hosted the IBC Video Content Innovation Summit at the Rockstart Accelerator in Amsterdam, which featured some of the leading innovators in the broadcast technology space today, including Vimond; Verizon envrmnt; HCL, and Valossa.

Emceed by Chris Pfaff, the breakfast event, held in the main event room at Rockstart, featured a technology keynote from Sanjay Macwan, focused on the shifting landscape for content distribution and consumption, and presented the audience of 55 producers, distributors, and vendors a view into where global content behaviors are headed. Macwan moderated a panel with Helge Hoibraaten, CEO of Vimond, Mika Rautiainen, CEO of Valossa, and Raheel Khalid, CTO of Verizon envrmnt (Kanwaldeep Kalsi, VP of Media for HCL, could not make the event, due to a delayed flight). The event literally presented some of the leading figures in OTT (Vimond), VR and AR (Verizon envrmnt), AI (Valossa), and cloud infrastructure (HCL).

Chris Pfaff emcees the IBC Video Content Innovation Summit

Sanjay Macwan, CEO of Integer 1, presents his keynote 

Sanjay gets into discussing the Muybridge experiment

Poster on front door of Rockstart Accelerator, Amsterdam

Raheel Khalid, CTO of Verizon envrmnt

Helge Hoibraaten, CEO of Vimond

Sanjay Macwan, left, and Mika Rautiainen, right

Long shot of the event room at Rockstart Accelerator

Sanjay poses a question to the panel

Raheel Khalid, Verizon envrmnt           Helge Hoibraaten, Vimond                   Mika Rautiainen, Valossa                     AI, OTT, and VR/AR all-stars

Highlights from the panel discussion included Helge’s discussion regarding content consumption (“we now have all the content we would ever want – in some ways, we don’t have to produce any more”) and content behaviors (“we interviewed teenagers about TV and they said ‘I have everything that I need in my phone'”). Raheel discussed the issue of every-cinreasing on -demand behaviors, in that people will “not wait for large files to download.” he said that Verizon envrmnt can now stream large-file content by taking game logic and running it on the fly. In this sense, VR and AR content is seamlessly distributed. He discussed Verizon envrmnt’s mobile edge compute strategy – which takes 20 milliseconds for a round trip. With 5G coming in, 1 gigabit bandwidth will enable VR content to be flawlessly streamed. Mika talked about the need to understand how video can be read and how machine learning can train to enable smarter questions around content itself.

See the promo highlight video at: https://youtu.be/lfZPjMLZa-w

The event was a swan song for Rockstart, as it was the last event in the main room at their 182 Herengracht location (they have since moved slightly north to a larger facility). But, it was a kickstart for IBC 2017, and in addition to a good meal, the crowd feasted on some hearty discussion.

The brainchild of VentureFuel and the Experiential Advertising Group, the Advertising Week New York 2017 TechX experience in Times Square, in a pop-up space between H&M and the NASDAQ MarketSite, curated some of the hippest tech for the ad world, from September 25-28. From digital art projects to AR apps to VR tools and holographic displays, the TechX arena offered a wide palette of commercially available experiences for attendees – creatives and media buyers – to play with and understand.

Ryan Hilla demonstrates Verizon envrmnt’s AR activation for Time Inc. to Guy Story and Vinne Grasso

The value of TechX cannot be understated: for years, Advertising Week has shunted ad-tech demos into the lower level of the Times Center on 8th and 41st, mainly as an afterthought to the main programming upstairs. TechX was both long overdue and also groundbreaking in its depth, bringing start-up companies in the immersive and AI worlds into a forum for major brand conversations on the state of the ad industry.

Mika Rautiainen demonstrates Valossa AI

TechX saw the launch of immersive analytics company QuantumXPR, with Scott Susskind and Kris Matheney camped out in the lower level of the venue, near two rows of digital art projects. Verizon envrmnt, the end-to-end VR/AR studio from Verizon Labs, was in full force, showcasing its AR projects for the likes of Time Inc., and Conde Nast, as well as its Virtual Sportsbar experience. The Valossa AI demo presented attendees with a new way to understand video, and VNTANA

showcased the best holographic display technology we have yet seen. Retinad’s VR in-console ad analytics and Tunity’s out of home display audio solution presented measurable technology for audiences underserved, or not yet measured. DreamSail Games showcased an expansive VR game environment, and Holosonics presented directional audio solutions.

VNTANA’s holographic display demonstrated the full-size immersive side of out of home displays

Sean Brown, with Turner Courageous, gets a demo of the Verizon envrmnt AR activation for a Verizon Store drone campaign, from JR Dawkins

Brian Roth, from Immersv, gets a VR demo from Christian Egeler, director of VR/AR at Verizon envrmnt

Nestled just steps away from the Times Square bustle, TechX was a rare opportunity to glimpse some of the more inventive ways to deliver and measure ads – and no PowerPoint involved.

Yes – there were sessions at TechX as well. Here’s a lively VC discussion in play