Thoughts on Jeopardy and IBM’s Watson

Posting to NGC4LIB

Concerning the article in the NY Times: “Computer Wins on ‘Jeopardy!’: Trivial, It’s Not” about the IBM computer “Watson” defeating the greatest champions on Jeopardy.

Although the competition was only semi-fair (at least in my opinion) since it could buzz within 10 milliseconds, the competitive aspects are beside the point. The computer actually could understand human-language questions, and it appears that you could—potentially—get into a type of dialogue with it (or in other words, a “reference interview”?). To me, this is much more significant than when Deep Blue beat world’s chess champion Gary Kasparov in the match, since it was not that Deep Blue played better than Kasparov, it was just that Kasparov cracked under the pressure, and made some of the worst mistakes in his career. Besides, humans behind the scenes were “tweaking” the computer during the match.

It seems as if something like Watson could be of tremendous importance to the library community, whether we happen to like it or not. Something tells me that the number of librarians is not going to grow tremendously. See the Bureau of Labor Statistics where they predict a respectable 8% growth by 2018, but in their analysis:

“Employment of librarians is expected to grow by 8 percent between 2008 and 2018, which is as fast as the average for all occupations. Growth in the number of librarians will be limited by government budget constraints and the increasing use of electronic resources. Both will result in the hiring of fewer librarians and the replacement of librarians with less costly library technicians and assistants. As electronic resources become more common and patrons and support staff become more familiar with their use, fewer librarians are needed to maintain and assist users with these resources. In addition, many libraries are equipped for users to access library resources directly from their homes or offices through library Web sites. Some users bypass librarians altogether and conduct research on their own. However, librarians continue to be in demand to manage staff, help users develop database-searching techniques, address complicated reference requests, choose materials, and help users to define their needs.

Jobs for librarians outside traditional settings will grow the fastest over the decade. Nontraditional librarian jobs include working as information brokers and working for private corporations, nonprofit organizations, and consulting firms. Many companies are turning to librarians because of their research and organizational skills and their knowledge of computer databases and library automation systems. Librarians can review vast amounts of information and analyze, evaluate, and organize it according to a company’s specific needs. Librarians also are hired by organizations to set up information on the Internet. Librarians working in these settings may be classified as systems analysts, database specialists and trainers, webmasters or Web developers, or local area network (LAN) coordinators.”

While 8% is normal overall, compare this to their related occupations: 23% for curators, 13% for primary/secondary teachers, 15% for postsecondary teachers, 24% for computer scientists, 20% for computer systems analysts. Also, the 8% presumes the higher retirement of current librarians.

Whether we like any of this or not, librarians are going to need help, and perhaps something on the order of this computerized Watson can provide a first-line of reference help. Such a tool may be perfect for a virtual community such as Second Life. People are often very hesitant to ask reference questions and my own experience bears this out. I get relatively few face-to-face reference questions, but I have created a number of what I call Two-Minute Tutorials on the web that my users can access anywhere at anytime, and these tutorials get used hundreds and hundreds of times each semester, so many times that it completely dwarfs what I do in person. Seeing those statistics was eye-opening for me.

I would personally love to have something like Watson to help people, so long as it would be very clear about when it is unsure and in those cases, transfer the question(s) to a human expert.