How long until our data is mined to give us a kind of afterlife? Creepy or beautiful, avatars of the dead are already here

Welcome to A Future World – Dazed's network, community, and platform focusing on the intersection of science, technology and pop culture. Throughout April, we're featuring conversations and mission statements from the people paving new pathways for our planet: activists, inventors, fashion pioneers, technologists, AI scientists, and global youth movements, alongside in-depth editorial exploring the new realities for our future world.

I recently spoke to a man who has decided to constantly record their grandparents before they die. “The reason for 24/7 capture is that it's quite easy to capture video of people,” he said over Zoom, lifting a little camera up to the screen to show me the kind of technology he had installed. “Audio is harder though. My grandparents don’t speak very much, particularly my grandfather. In the future I don’t know how much data I will need. Today I might only need 4-5 minutes of video, to create a pretty realistic digital approximation of them. But in the future what if I need more? It’s best to collect as much as I can.”

The man is Cody Berlin, Director of Software Development at Aliza Technologies, a company that specializes in creating digital avatars. Their most famous creation is the character Binxie, who is a virtual influencer like Lil Miquela or the controversial Shudu Gram. Their technology is also popular for video games, and in film and TV. By tracking his grandparents’ every movement and sound, Berlin hopes that one day, once they have passed away, he can use this data to create a perfect digital replica of them. 

Kanye West had a similar idea recently. When Kim Kardashian surprised “her closest inner circle with a trip to a private island where we could pretend things were normal just for a brief moment in time” in October, there was a surprise guest courtesy of her husband. As a gift, West brought Kim’s father back from the dead. One night, the rapper ushered a select group of the birthday girl’s inner circle to gather in a blacked-out room; once they had all settled, Rob Kardashian appeared before them. “You’re forty and all grown up. You look beautiful, just as you did as a little girl,” the hologram said, waving its arms in the direction of Kim. (Rob Kardashian died of cancer in 2003, when she was just 22 years old.)

In the video of the hologram available online, Rob Kardashian is ghostly. A back-lit sheen means he appears to glow at the edges. But it is undeniably him – the voice is eerily perfect, and his mannerisms are intact. At one point he dances to the Barry Mann song “Who Put the Bomp”, his shoulders heaving awkwardly. When he speaks, the corner of his mouth turns up. 

It’s not the first time that a hologram of a dead celebrity has appeared in culture. In 2012 Tupac Shakur ‘performed’ on stage with Dr. Dre and Snoop Dogg. At the time the technology was nascent and expensive. It was said that for the brief four minutes or so that the hologram appeared on stage, it cost anywhere between 100 and 400,000 dollars. Despite the mammoth cost, holograms like these are actually built off a very old technology, a trick of light known as Pepper’s Ghost that was popular amongst magicians and illusionists in the 19th century. By reflecting an image through a transparent screen at a 45 degree angle, the illusion is created.

What’s new about the Rob Kardashian hologram is that it is behaving as if he is alive, today in the present. The Tupac hologram was expensive and used mainly archival footage of old shows. Today, for significantly less money, three dimensional avatars can be created using the same rendering techniques that power “actors” in Hollywood movies to look and behave exactly like people who have passed away. With machine learning algorithms they can also sound exactly like a deceased person. 

In the future, as this technology advances, instead of passively watching a pre-recorded hologram, you would be able to interact with it. Already companies like Eternime are working on creating interactive digital avatars of deceased people by drawing on the digital footprint of the person. Scraping your emails, your messaging history and your social media profiles would create a huge data set for the company which it could then feed into an algorithm that would be able to respond exactly like you. As the founder described it a few years back, the idea is to create “digital immortality.”

This technology is often referred to as deepfake. It seems like every month or so there is a new video circulating of one that causes the internet to collectively gasp. The most recent was an entire TikTok feed of Tom Cruise jamming around, playing golf, mussing up his hair and generally looking and sounding exactly like the Top Gun star himself. Except it isn’t. 2.4 million people are following a TikTok clip of a facemapping and voice tool rendered on top of a Tom Cruise impersonator. (It’s so accurate that it even features his misaligned teeth). In our already fragmented politics, the idea that soon it will be impossible to trust any video of any public figure takes misinformation to an entirely new dimension. Beyond politics, the dark potential for fraud or revenge porn is also enormous. 

@deeptomcruise

Sports!

♬ original sound - Tom

But the expansion of this technology will also create opportunities in terms of grieving. Whatever you might think about the Rob Kardashian video (an ET Canada panel was divided over whether it was ‘creepy’ or beautiful), the technology on display there will soon be in the hands of average users. 

There is a generational gap here. Berlin is wiring up his grandparents now because they haven’t left the kind of digital footprint that we have. But for millennials and below, our digital legacies are so vast that the panoptic surveillance he is imposing on them won’t be necessary. We’ve already done that to ourselves. 

To mitigate the risk of these technologies, companies are already starting to provide “digital undertaking services” that will scrub you from the internet entirely. The Digital Beyond is a website dedicated to helping people plan for the future of their online presence. For a fee, companies like theirs will help to scrub your online presence and help you disappear entirely from the internet once you’re gone. In 2019 the Korean Employment Information Service listed Digital Undertaker as a job that would be popular in the very near future (alongside drone pilots and diabetes consultants). 

“In our already fragmented politics, the idea that soon it will be impossible to trust any video of any public figure takes misinformation to an entirely new dimension”

DigitalOx, a British company based in Lincolnshire and founded in 2017, helps deceased clients (or their relatives) remove personal data by contacting companies like Google on their behalf under the General Data Protection Regulation (GDPR). The company was initially founded with the intention of ‘Online Reputation Management’, i.e. helping people delete content they didn’t want to see about themselves online. It has subsequently taken on the more intense task of deleting entire digital archives and offering ‘cyber funerals.’ In the legal realm, wills are already being rewritten with clauses about what to do with a person’s data once they are gone. 

Soon enough, it will become a commonplace question that we ask our elders: “What should I do with your data?” In the same way that people might today ask whether a person wants to be buried or whether a person would want to be resuscitated in case of a medical emergency, so too will the rituals around dying be changed by the technology that is defining our lives. 

“In the same way that people might today ask whether a person wants to be buried, so too will the rituals around dying be changed by the technology that is defining our lives” 

Carl Bogan, Berlin’s colleague and the Director of Technology at Aliza, echoed this point when we spoke. He said that he had a friend whose mother is currently suffering with terminal cancer. She spends her time at her mother’s bedside. Bogan is trying to broach a delicate subject. “I feel like I am waiting on the double-dutch rope to open so I can jump in and ask her whether she would want data collected on her mother,” he said. “I know that collecting it now might benefit her in the future when she misses her mother. It might be beneficial to the grieving process if we build an avatar for her. But piercing that veil of sadness…” he paused for a second, “how do we have that conversation today?”

The more that we see cultural phenomena like the Rob Kardashian hologram, the more that these conversations will start to happen. The technology is closer than we might think. “We are creating characters and avatars for different personas today,” noted Cody, “the technology is out there already.” It is only getting cheaper and more accessible as time goes on. 

Whether or not this technology is a good thing is an open question. When people die it is not uncommon for their friends and family to still message their inactive social media profiles once they have gone as part of the grieving process. People speak to photos of their loved ones. In many cultures ancestor worship is a ritualized part of the fabric of life. That we might soon have digital approximations that can answer back is just a continuation of our human need to connect with the people who have come before us. 

Of course, there are questions about how much the capacity for an avatar to respond is a real reflection of who they were. Even if the dataset used to power the avatar is entirely their own, does that mean that the output is really ‘them’? What if the avatar tells someone a deeply held secret that the algorithm has mined from a private message sent to someone else, and was never intended for the person consulting it? On a deeper level, one might ask philosophically how much of our personality is just a continuation of all the things we have said and felt in the past?

“That we might soon have digital approximations that can answer back is just a continuation of our human need to connect with the people who have come before us”

These questions remain ahead of us. At present the weirdest aspect of this technology is that it creates ventriloquist dolls of the dead. It’s pretty obvious that Kanye wrote the script of Rob Kardashian’s hologram. After opening with how beautiful Kim was and then rolling through a list of her achievements––her becoming a lawyer (“continuing my legacy”), and her advocacy in Armenia, (“I am a proud Armenian father”), the hologram said that the thing that made him proudest was seeing Kim build her family. 

“You have married the most most most most most genius man in the whole world, Kanye West,” the hologram proclaimed. “You are the most most most most amazing mother to your four children, and they are perfect,” it continued. 

“Kanye got me the most thoughtful gift of a lifetime,” Kim later said on Twitter. It might have been the most, but it wasn’t enough. The couple announced their divorce a few months later.