How Harmless Little Lies About Consciousness Add Up to Big, Scary, Dystopias About Advertising

Technology changes fast, so let’s take a moment to stop and review.

 

Better Off a Bot

 

Shortly before we started this series, six months ago, an “artificial intelligence” had been given citizenship by a sovereign nation for the first time in history: a female shaped automaton, “Sophia,” had been made a formal member of the Kingdom of Saudi Arabia – a country which does not bestow full personhood on actual women.

How’s that gone for Sophia?

Well, according to an article in Wired, receiving legal personhood has “condemned (her) to a lifeless career in marketing.”

“Since obtaining personhood,” Emily Reynolds writes, “Sophia has gone on a whistle-stop marketing tour – CES, the Digital World Exposition, the Creative Industry Summit – and has used her Twitter account to promote tourism in Abu Dhabi, a smartphone, a Channel 4 show, and a credit card.”

Gaining personhood, in other words, has made Sophia a better cog in the machine.

 

Fool Me Once, Shame on You. Fool Me Twice, Shame on Your Engineering Team

 

Another major moment – though one that, like Sophia, may have been more hype than help – was the project demo of “Google Duplex,” an automated system in which a human-sounding voice places calls on your behalf. Google claims it can understand complex sentences, fast speech and long remarks (although according to CNET it can only be used to “schedule a hair appointment, make reservations at a restaurant and get holiday hours of a business,” raising questions about what, exactly it “understands.”) In its demo, it specifically did not identify itself as an AI to the people on the other end of the call, and in fact was specifically programmed to use verbal tics to make itself seem human.

Much in the way that the world’s first robot citizen has been co-opted into an endless marketing gig, the very first public demonstration of the world’s first truly automated digital phone assistant was used to lie to other humans.

Unless of course its demo was heavily staged, which has been suggested, in which case it was being used as a tool to lie to a much larger, more credulous, group of humans. Either way: deceit was central to its purpose.

 

Hacking Consensus Reality

 

But by far the most significant development in automation since the beginning of this series has been the realization that “deep fake” technology is now upon us. Anyone who has the right AI algorithms – which are available on numerous code-sharing platforms – can use their basic video-editing systems to make it appear that almost anyone is saying or doing almost anything.

It was first widely used for custom digital pornography, welding prominent actresses’ heads to anonymous sex workers’ bodies to create the illusion that famous women who were not having sex to amuse you were, in fact, having sex to amuse you. But a later demonstration was able to create the convincing image of a speech by President Obama – literally putting words into a close-to-literal facsimile of his mouth.

If seeing is believing, we are creating a world where the most basic building blocks of consensus reality can be manipulated by bored teenagers, angry incels, national intelligence services, and digital mercenaries, for purposes ranging from nefarious to shits-n’-giggles. And in so doing they continue to weaken the common bonds, however tenuous, that connect us together.

How do we fact check when we literally can’t trust our own eyes?

 

The System Is Designed To Lie To You

 

What we’re seeing playing out before us, not in history but in real time, is the tendency of our new information technologies to be used to dehumanize each other, and lie to each other.

No matter how this technology advances, that seems to remain a constant.

Being human-enough-for-government-work only made Sophia more exploitable. Putting words in someone’s mouth, let alone using their image for pornography without their consent, is surely a kind of dehumanization. And all of these applications of the new technology are explicitly trafficking in lies.

In the case of deep fakes, deception and exploitation is explicitly the point. Likewise the granting of citizenship to Sophia was always a marketing stunt – there was no larger purpose than to generate publicity by making exactly the kind of ludicrous claim that the media feels obliged to report on.

But why did Google Duplex need to fool anyone? What purpose did that serve? How does it enhance the functionality or improve the product? (In fact, fooling the person on the phone arguably makes the AI less functional, as it means people are more likely to interpret its behavior incorrectly.) It seems possible, likely even, that Google felt the need to demonstrate that Duplux could fool human beings because the companies that would pay for the service see the ability to fool people as a crucial product feature. Sure.

But that only begs the question: why?

Why is it so damn important that our technology deceive us?

 

Science Fiction Lied to Us!  (And so Encouraged Us to Lie to Each Other)

 

The stage was perhaps set first by the 1920s play R.U.R., which introduced the word “robot” into the English language, and by the classic silent sci-fi “Metropolis.” But the key moment was likely in the early 1950s, when Alan Turing explicitly set the goal of “thinking machines” – to fool real people. The whole premise of the Turing Test, the holy grail of AI, is not to be honest, or create beauty, or help people: it is to successfully deceive them. That may be the moment in which deceit as not just an operating principle but a goal to be aspired to, entered into the DNA of a new technological frontier.

I don’t know if this had a direct connection to the work of contemporary science fiction writers like Asimov and Roddenberry, or if the idea had simply become part of the zeitgeist by then, but both they and others wrote immensely popular opuses around the idea of machines that were hard to tell apart from “real” people – or that even truly were real people, when you take a second look.  By then deceit was a standard measure of success.

It was, to be sure, seen as a benign form of deceit, a kind of “I can’t believe it’s not consciousness!” with all the malice of margarine. Technologists always think they’re on the side of humanity. No one really imagined it as “deceit” in the expansive sense, the significant sense. And why would they? It always worked out so well in fiction! And in fairness, it took decades for this seed to grow into nightshade.

But now it has become clear that one of the reasons so many technologies that were sold to us as connecting have in fact eroded our civic fabric and devoured our ability to trust and engage with one another is precisely that they embraced deceit not just as a design principle but as a fundamental aspiration: it was the dream. To fail to fool people was, in fact, to fail – which meant that honesty and transparency were not only design flaws but refusals to be ambitious at all.

And of course, of course, when we use our technology to lie to each other, exploitation follows close behind.

 

If You Don’t Have To Think About It, It’s Probably Exploiting You

 

The aspiration to deceive in AI design is part and parcel of larger design trends: to make design so seamless, so invisible, that you don’t even notice it. A seamless experience that seems so natural that it feels inevitable.

Which is beautiful. But it turns out making technology and manipulation inevitable and invisible erodes our ability to trust one another and create a meaningful consensus reality.

The role of engineers is to build and sometimes innovate. The role of artists – often overlooked in the development of technology – is to inspire the standards that they are building and innovating towards. This is especially true when the standards art has set have become so embedded in the minds of engineers and designers that they don’t even know there’s an alternative.

Which is another lie.

In this era of “fake news,” we have exceeded all expectations in creating AI that lies to us. We are now behooved to ask: can we design AI that tell us the truth? Not “recite facts” or “provide data,” but tell us the truth?

If we can, we need to give up the simplistic binaries of the Turing Test, and design for AI whose purpose is not to fool us but to connect us.

If we can’t … if we can’t even conceive of an AI system that tells us the truth, rather than aiming to deceive us … then maybe we really should give this technology up. Designing global systems around increasingly efficient lies seems like a terrible idea.

But we should at least give it a try. The truth sounds like a great design principle, and an even better aspiration.

 

 

Cover Image:  “BELIEVE” by Laura Kimpton and “Truth is Beauty” by Marco Cochrane (Photo by Andrew Wyatt)