July 02, 2015

How Computer Program Eugene Passed The Human ‘Turing Test’


In 1950 Alan Turing, the pioneer of computing, made one of the most famous predictions in modern science. By the year 2000, he forecast, a machine would need only five minutes to fool one in three “average interrogators” into mistaking it for a human being. 
  In the end it took 14 years longer than Turing expected, but last year “Eugene” — a Russian computer program purporting to be a 13-year-old boy from Odessa in Ukraine — was claimed to have passed the test.

Transcripts published this week show how he bamboozled 10 of the 30 judges at the Royal Society through a mixture of scattiness, bad manners and worse humour — but they have also led to allegations that the test is little more than a pointless “magic trick” based on a misunderstanding of Turing’s work.

Professor Gary Marcus, an expert on language and the mind at New York University, called Eugene’s achievement “a meaningless victory — an exercise in evasion rather than intelligence” and said the program was no more sophisticated than Siri, the voice-activated electronic assistant developed by Apple.

Stevan Harnad, professor of computer science at the University of Southampton, said that fooling a third of the judges in a brief exchange of words was not the Turing test but simply a “benchmark” of progress in artificial intelligence.

The real proof would be a machine that could convincingly imitate a human over years or decades, much like Samantha, the operating system that becomes the unlikely love interest in the 2013 film Her.

“Design a system that can do anything a human can do, indistinguishably from the way any human does, to any human, and you will have explained how the mind works,” Professor Harnad said. “It’s not about fooling 30 per cent of judges for five minutes, any more than Newton’s Law is about predicting what 30 per cent of colliding billiard balls will do for five minutes.”

Kevin Warwick, deputy vice-chancellor for research at Coventry University and one of the country’s leading experts on cybernetics, who organised and analysed the conversations, insisted that Turing’s formulation was still a good way of measuring machine intelligence, but said that a ten or 15-minute test might give a better indication of an AI’s capacity for fooling people.

The judges held two simultaneous conversations on side-by-side screens for five minutes. When the time was up, they had to decide whether each of their interviewees was a human or a computer.

The transcripts reveal that Eugene’s conversation is littered with attempts at humour and incidental detail. From references to his favourite film — Naked Gun — to gobbets of low-budget wisdom — “Sanity and insanity complement each other in our complicated life” — the computer often comes across as a more colourful character than the humans he was up against.

During a bizarre exchange about Jim Carrey films, he jocularly attempted to recruit the judge into a campaign to destroy other AIs. “If I’m not mistaken, Carrey is a robot,” he said. “Just as many other ‘people’ — we must destroy the plans of these talking trash-cans!”

He was even better equipped for talking about the weather than a native Briton. In what Professor Warwick described as a “relatively lame conversation”, one judge asked a human about the rain outside. “Yeee, very gloomy indeed,” he replied.

Eugene, however, hit back with: “The weather is always as bad as you think it is. I do believe that the world just a mare (sic) reflection of our thoughts.”

John Barnden, professor of artificial intelligence at the University of Birmingham, said the transcripts showed Turing might have underestimated the “incoherence, triviality and oddness” of much human conversation.

“A lot of conversation in real life has no strong informational or other ‘serious’ aim, and is just a superficial means of being social or of just keeping the conversation going because the silence is embarrassing,” he said.

Professor Barnden said that although the Royal Society test was “a reasonably good match” for Turing’s thought experiment, a five-minute conversation was not a useful test of whether a machine could “think” or not.

By Oliver Moody

With many thanks to The Australian

Also related: 

Hedy Lamarr's 101st Birthday Celebrated by Google

Jason Padgett: Brain Injury Turns Man Into Math Genius

10 Mathematical Equations That Changed The World

Five Brilliant Mathematicians And Their Impact On The Modern World

Arthur Benjamin: The Magic of Fibonacci Numbers

Claude Shannon Jr: The Greatest Genius No One Has Heard Of

John von Neumann: This Hungarian-American Mathematician May Have Been Smarter Than Einstein 

Great Minds: Filippo Brunelleschi

Great Minds: Leonardo da Vinci

The Genius of Nicola Tesla

Hedy Lamarr - Beauty And Brains in Abundance

Inside The Mind Of Teenage Maths Genius Alex Gunning

The New Turing Test:Brainy Machines Need An Updated IQ Test, Experts Say 

World Wide Web Turns 25

Benedict Cumberbatch And Eddie Redmayne: The Changing Face Of Hollywood

Biopics Now Focus On Key Moments Rather Than A Whole Life

The Best Movies Of 2014

Some Biopic Actors And Their Real-Life Counterparts

 Mathematical Minds Stir A Beauty Within

John Nash’s Astonishing Geometry

Alan Turing Manuscript Sells For $1 million

German WWII Coding Machine Found On eBay For $20

A Mathematician Has Built A Machine That Can Beat The Odds In Roulette

What Was the Enlightenment?
Quantum Enigma Machines Are Becoming A Reality

Albert Einstein's Legacy

Gottfried Wilhelm Leibniz: The Philosopher Who Helped Create the Information Age