Itâs easy to imagine a computer being a lot, lot smarter than us regular humans. They can do certain tasks much, much quicker than our brains can figure things out.
However, thatâs not entirely true! Computers are better than us in some ways, yes. For instance, you can type in a complicated equation into a calculator and itâll spit out an answer much faster than a human can do it.
But thatâs the thing; a computer is much better than us at some things. Theyâre very good at churning through data or doing calculations, but when it needs to conceptualize or imagine things, it struggles to do what a human can do really easily.
Donât believe us? Hereâs an example. How complicated is it to understand this next sentence?
The mother gave the baby a toy, because they were crying.
Itâs not too tricky, is it? You can imagine a baby bawling its eyes out, and their mother giving them a toy to calm it down. To a computer, though, deciphering this sentence is really, really hard.
Why? Because a computer doesnât know what the âtheyâ in that sentence refers to. As far as a computer knows, it could be describing one of these two scenarios:
- The baby was crying, so the mother gave them a toy.
- The mother was crying, and because of that, she gave the baby a toy.
Number two sounds ridiculous to our human brains. Why would a mother give a baby a toy because she was crying? To a computer, though, it doesnât see the problem. Thatâs because the computer doesnât know what a âmotherâ or a âbabyâ is. It doesnât know the relationship between a mother and a baby, and how a mother can give a baby a toy to calm it down when it cries.
This is what the Winograd Schema Challenge (WSC) aims to test. The WSC is a series of tests, much like the Turing test, that gives AI systems sentences like the above. The sentence contains a word like âtheyâ or âit,â and the AI then has to identify what that word is referring to. For example, the âtheyâ in the above sentence is referring to the baby, and the AI needs to recognise that.
Hereâs where things get tricky. Letâs look at this sentence: The lions ate the zebra, because they are predators.
We can understand this sentence easily, but a computer may assume that the second object, the zebras, are the predators in this sentence. Now we have a weird world where zebras are hunting down lions, and the lions presumably fought back to protect themselves!
Now, letâs assume the computer decides that when a sentence includes lions and zebras, any instances of âtheyâ always refers to the lions. Sure enough, if we kept feeding the computer sentences with the same structure as above, itâd always get right.
However, itâs not a good solution, especially when we pull this doozy: The lions ate the zebra, because they are tasty.
Now the computer assumes that the lions in the above sentence are the tasty ones. Perhaps the zebras were licking the lions a little too much and they decided to get revenge?
Then you have the really, really evil sentences. Take this one: The gardener stopped tending the plants because it started to rain.
Here, the âitâ refers to the weather. The clouds began to rain, so the gardener stopped and went inside. However, the sentence never mentions the weather directly, so a computer may assume that the gardener stopped because either they or the plants burst into water droplets!
The Winograd Schema Challenge may just seem like a funny and weird test, but itâs actually really important. These days, we have a lot of stuff we can command with our voices, like Amazon Alexa and Google Assistant. Itâs important for these services to identify the âthemsâ âtheysâ and âitsâ in sentences like in the above examples so they can properly understand their human masters.
One way to do it is to âteachâ the computer what these objects are and how they work. You code a program that says mothers take care of children, that lions are predators and eat zebras which are prey, and that âstarted to rainâ refers to weather. But this is a really simple way of breaking it down, and programs undertaking the WSC are far more complex.
So if youâre into building AI, how would you go about solving the problems we detailed above? If you canât solve them, donât worry; even really smart adults with impressive degrees are struggling to solve them, too!
Learn More
The Winograd Schema
https://cs.nyu.edu/~davise/papers/WinogradSchemas/WS.html
WinoGrande: An Adversarial Winograd Schema Challenge at Scale (with examples)
https://winogrande.allenai.org/
Winograd Schema Challenge
https://en.wikipedia.org/wiki/Winograd_schema_challenge
What are Winograd Schemas?
https://medium.datadriveninvestor.com/what-are-winograd-schemas-dfe9e51b765a