Stupid Artificial Intelligence

You can relax. It doesn't look like the robots are gonna take over any time soon.

Exhibit 1: 20 Questions Harry Potter

20Q has a Harry Potter module. It is very good at guessing relatively common entities from the HP universe in 20 questions or less. It also has a very deep database -- even when trying to be obscure, I've only managed to come up with a item it's never heard of twice (one was the sign of the Deathly Hallows, the other I've forgotten), and it has a lot of items that I've never heard of, which is saying something.

But a mind reader it's not. I tracked 20 rounds, and I stumped it exactly half the time. The artificial intelligence protocol seems to be limited by human ignorance on two fronts: the input side and the design side.

Some of the times that I've won the game, it's at least partly because other players have "taught" 20Q wrong answers to questions -- and/or the answers are really matters of interpretation, and the majority of players have chosen a different answer than I did. I've always been way too much an analytical stickler for exactitude to get along well with true/false questions. But 20Q "knows" a lot of things about the HP universe that do not fit the classic philosophical definition of knowledge as "justified true belief."

But more of the time, it seems to be because of weaknesses in the program design.

Part of this is stupid questions. Some are either too broad or two narrow to be valuable. "Do you have dreams?" doesn't seem likely to ever yield useful information, since the HP canon doesn't indicate that there are any characters who definitely don't dream. And then there are questions like "Are you useful?" "Are you helpful?" and "Do you help accomplish tasks?" -- awfully redundant. And that's not to mention a bunch of questions that defy consistent answers, perhaps because they are matters of opinion or depend upon perspective.

A bigger problem seems to be a lack of sophistication in the program's process of elimination. To some degree, this might just reflect a difference in how the computer "thinks" from how human beings think. If I were playing 20 questions, I would start with broad categories and try to progressively narrow down the possibilities. But 20Q seems to be working with a field of possible questions that it throws out more or less randomly. Maybe there is some method in the madness, but it's not apparent.

Now, I recognize that asking some redundant questions is good policy, because the answerer may answer some questions incorrectly or inconsistently. But really. If we've already established that the item in question is a thing, an object, and a tool, does it really make any sense to ask whether it teaches Transfiguration? If we've just established that the person in question is in Ravenclaw, why waste the next question asking whether s/he is in Gryffindor?

The game also seems to treat the possible answers (Yes, No, Unknown, Irrelevant, Sometimes, Probably, Doubtful) as mutually exclusive categories, rather than points on a spectrum. It seems like it would "learn" with more subtlety if it recognized, for example, that "doubtful" is a shade of "no," or that a critical mass of "irrelevant" votes should soften the weight given to a "yes" or "no" answer, even if it is the majority position.

It's a free internet game, not the cutting edge of AI research. But it seems to me it could be fine-tuned quite a bit.

Exhibit 2: Recommendations

This advert from hit my inbox this weekend:

Now available: "Silicon Carbide and Related Materials 2006 (Materials Science Forum)" ...

I've been tempted to turn off the feature in which Amazon e-mails me to recommend titles it thinks I might like, but I'm curious about and often amused by what it comes up with, so I leave it on. Usually the titles it recommends have at least some distant relation to something I might find interesting, but this one really stumped me. That is, until I clicked on the e-mail and saw the part of the subject line that was hidden in my browser:

... by N. Wright

Aha. Amazon "knows" that I own, and regard highly, several books by the Bishop of Durham, Nicholas Thomas Wright, aka "N.T."

What Amazon can't seem to figure out is that the "N. Wright" who wrote The Resurrection of the Son of God is not the same "N. Wright" who wrote Silicone Carbide and Related Materials 2006. Which sort of undermines any hope the bean-counters at Amazon might have that my willingness to swallow hard and shell out $39 for the former might indicate that I would jump at the chance to spend $398 (!!!) on the latter.

This simply illustrates one of my biggest pet peeves about Amazon: there is no mechanism for disambiguating authors with similar names. I find this annoying when I'm trying to find books by one author, and the results are cluttered with listings from completely different people who just happen to have similar names. And it's not like there aren't ways to do this. The Library of Congress already successfully distinguishes between multiple authors with the same name -- all Amazon would have to would be to apply the LoC tags to its inventory. The information exists, but Amazon does not make use of it. And the result is laughable misfirings of a system that is supposed to provide targeted advertising that will be welcomed by the recipient because it is tailored to her interests.

I don't think I've ever bought a book for which Amazon has sent me a recommendation e-mail.


Robin said...

"disambiguating!" You made my day.


Sarah said...

You made my day too: "all Amazon would have to would be to apply the LoC tags to its inventory." Someone out there appreciates proper cataloging!

And this in the same week that I found a beautiful lesson on information literacy on Telford's website. I'm giddy.

Have you read Thomas Mann (author of the Oxford Guide to Library Research)? Or did you appreciate LoC just because it's wonderful?

Interesting Stuff

Books! Books! Books!

  • Bookfinder
  • BestBookBuys
  • Bookcrossing
  • Book Sale Finder
  • Library Thing
  • Good Reads
  • Disclosure: links from this page to commercial sites -- particularly -- may or may not be affiliate links that remunerate the blogger for sales made through said links. In no case does affiliate status affect the opinions offered on this site.

    HTML 101: How to add a link

    <a href="http://exact-url- of-site-to-which-you-wish-">WORDS TO APPEAR AS LINK</a>


    Blog Archive