Posting to NGC4LIB
This is essentially an advertisement for Google Search, but still shows a few rather peculiar mindsets, at least from the librarian point of view. The first thing I noticed is where one of their specialists said that they are interested in getting the best results *for each user*, which actually reveals quite a bit I think.
Someone notices that a search "is not performing as well as they would like" and it goes to "ranking engineers" who work with it. Then it goes to "raters", i.e. external people who have been trained to judge whether one ranking is more relevant or higher quality than another. Then it all goes to a "search analyst" and a related committee, where the ultimate goal is "to provide an informed, data-driven decision and to present an unbiased view." Then comes a real example. Also, they claim they make over 500 updates a year to the search results.
So, from this short film there is an overall impression of search results that are changing constantly, i.e. updates of almost 2 a day; plus a great deal of subjectivity, i.e. the best results *for each user*, mixed with supposed objectivity (reliance on "experts" and data-driven analysis to get an unbiased view) that, when you consider it, doesn't really explain anything at all. For instance, who are these "raters" and who trains them and how? They seem to be the focal point.
I just discovered this (via a Google search!): Google General Guidelines for Remote Quality Raters (2007)
http://www.seobook.com/full-text-googles-general-guidelines-remote-quality-raters-april-2007 It was taken down along with some videos at Google's request! (That is revealing!) But I found a summary of it:
In spite of all this, it seems that Google remains completely a black box that takes in information and spits it out and no one, at least no one outside of the company, really knows why it ranks sites the way it does. What does it mean when they say a search "does not perform as well as they would like"? Still, reading those guidelines for raters makes me wonder if this is going to be what the Subject Heading Manuals will become someday.
I hope not!