Document (#32471)

Author

Bar-Ilan, J.
Keenoy, K.
Yaari, E.
Levene, M.

Title

User rankings of search engine results

Source

Journal of the American Society for Information Science and Technology. 58(2007) no.9, S.1254-1266

Year

2007

Abstract

In this study, we investigate the similarities and differences between rankings of search results by users and search engines. Sixty-seven students took part in a 3-week-long experiment, during which they were asked to identify and rank the top 10 documents from the set of URLs that were retrieved by three major search engines (Google, MSN Search, and Yahoo!) for 12 selected queries. The URLs and accompanying snippets were displayed in random order, without disclosing which search engine(s) retrieved any specific URL for the query. We computed the similarity of the rankings of the users and search engines using four nonparametric correlation measures in [0,1] that complement each other. The findings show that the similarities between the users' choices and the rankings of the search engines are low. We examined the effects of the presentation order of the results, and of the thinking styles of the participants. Presentation order influences the rankings, but overall the results indicate that there is no "average user," and even if the users have the same basic knowledge of a topic, they evaluate information in their own context, which is influenced by cognitive, affective, and physical factors. This is the first large-scale experiment in which users were asked to rank the results of identical queries. The analysis of the experimental results demonstrates the potential for personalized search.

Theme

Suchmaschinen
Benutzerstudien

Object

Google
MSN Search
Yahoo

Similar documents (author)

1. Bar-Ilan, J.; Levene, M.: ¬The hw-rank : an h-index variant for ranking web pages (2015) 5.41

   5.40834 = sum of:
     5.40834 = sum of:
       2.1058996 = weight(author_txt:ilan in 1694) [ClassicSimilarity], result of:
         2.1058996 = score(doc=1694,freq=1.0), product of:
           0.5952885 = queryWeight, product of:
             8.085969 = idf(docFreq=36, maxDocs=44218)
             0.07361993 = queryNorm
           3.5376115 = fieldWeight in 1694, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.085969 = idf(docFreq=36, maxDocs=44218)
             0.4375 = fieldNorm(doc=1694)
       3.3024404 = weight(author_txt:levene in 1694) [ClassicSimilarity], result of:
         3.3024404 = score(doc=1694,freq=1.0), product of:
           0.80351204 = queryWeight, product of:
             1.1618029 = boost
             9.394302 = idf(docFreq=9, maxDocs=44218)
             0.07361993 = queryNorm
           4.1100073 = fieldWeight in 1694, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.394302 = idf(docFreq=9, maxDocs=44218)
             0.4375 = fieldNorm(doc=1694)
   

2. Bar-Ilan, J.; Levene, M.; Mat-Hassan, M.: Methods for evaluating dynamic changes in search engine rankings : a case study (2006) 3.86

   3.8630998 = sum of:
     3.8630998 = sum of:
       1.5042138 = weight(author_txt:ilan in 616) [ClassicSimilarity], result of:
         1.5042138 = score(doc=616,freq=1.0), product of:
           0.5952885 = queryWeight, product of:
             8.085969 = idf(docFreq=36, maxDocs=44218)
             0.07361993 = queryNorm
           2.5268652 = fieldWeight in 616, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.085969 = idf(docFreq=36, maxDocs=44218)
             0.3125 = fieldNorm(doc=616)
       2.358886 = weight(author_txt:levene in 616) [ClassicSimilarity], result of:
         2.358886 = score(doc=616,freq=1.0), product of:
           0.80351204 = queryWeight, product of:
             1.1618029 = boost
             9.394302 = idf(docFreq=9, maxDocs=44218)
             0.07361993 = queryNorm
           2.9357195 = fieldWeight in 616, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.394302 = idf(docFreq=9, maxDocs=44218)
             0.3125 = fieldNorm(doc=616)
   

3. Bar-Ilan, J.; Keenoy, K.; Levene, M.; Yaari, E.: Presentation bias is significant in determining user preference for search results : a user study (2009) 3.86

   3.8630998 = sum of:
     3.8630998 = sum of:
       1.5042138 = weight(author_txt:ilan in 2703) [ClassicSimilarity], result of:
         1.5042138 = score(doc=2703,freq=1.0), product of:
           0.5952885 = queryWeight, product of:
             8.085969 = idf(docFreq=36, maxDocs=44218)
             0.07361993 = queryNorm
           2.5268652 = fieldWeight in 2703, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.085969 = idf(docFreq=36, maxDocs=44218)
             0.3125 = fieldNorm(doc=2703)
       2.358886 = weight(author_txt:levene in 2703) [ClassicSimilarity], result of:
         2.358886 = score(doc=2703,freq=1.0), product of:
           0.80351204 = queryWeight, product of:
             1.1618029 = boost
             9.394302 = idf(docFreq=9, maxDocs=44218)
             0.07361993 = queryNorm
           2.9357195 = fieldWeight in 2703, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.394302 = idf(docFreq=9, maxDocs=44218)
             0.3125 = fieldNorm(doc=2703)
   

4. Zhitomirsky-Geffet, M.; Bar-Ilan, J.; Levene, M.: Testing the stability of "wisdom of crowds" judgments of search results over time and their similarity with the search engine rankings (2016) 3.86

   3.8630998 = sum of:
     3.8630998 = sum of:
       1.5042138 = weight(author_txt:ilan in 3071) [ClassicSimilarity], result of:
         1.5042138 = score(doc=3071,freq=1.0), product of:
           0.5952885 = queryWeight, product of:
             8.085969 = idf(docFreq=36, maxDocs=44218)
             0.07361993 = queryNorm
           2.5268652 = fieldWeight in 3071, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.085969 = idf(docFreq=36, maxDocs=44218)
             0.3125 = fieldNorm(doc=3071)
       2.358886 = weight(author_txt:levene in 3071) [ClassicSimilarity], result of:
         2.358886 = score(doc=3071,freq=1.0), product of:
           0.80351204 = queryWeight, product of:
             1.1618029 = boost
             9.394302 = idf(docFreq=9, maxDocs=44218)
             0.07361993 = queryNorm
           2.9357195 = fieldWeight in 3071, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.394302 = idf(docFreq=9, maxDocs=44218)
             0.3125 = fieldNorm(doc=3071)
   

5. Zhitomirsky-Geffet, M.; Bar-Ilan, J.; Levene, M.: Analysis of change in users' assessment of search results over time (2017) 3.86

   3.8630998 = sum of:
     3.8630998 = sum of:
       1.5042138 = weight(author_txt:ilan in 3593) [ClassicSimilarity], result of:
         1.5042138 = score(doc=3593,freq=1.0), product of:
           0.5952885 = queryWeight, product of:
             8.085969 = idf(docFreq=36, maxDocs=44218)
             0.07361993 = queryNorm
           2.5268652 = fieldWeight in 3593, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.085969 = idf(docFreq=36, maxDocs=44218)
             0.3125 = fieldNorm(doc=3593)
       2.358886 = weight(author_txt:levene in 3593) [ClassicSimilarity], result of:
         2.358886 = score(doc=3593,freq=1.0), product of:
           0.80351204 = queryWeight, product of:
             1.1618029 = boost
             9.394302 = idf(docFreq=9, maxDocs=44218)
             0.07361993 = queryNorm
           2.9357195 = fieldWeight in 3593, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.394302 = idf(docFreq=9, maxDocs=44218)
             0.3125 = fieldNorm(doc=3593)
   

Similar documents (content)

1. Bar-Ilan, J.; Levene, M.; Mat-Hassan, M.: Methods for evaluating dynamic changes in search engine rankings : a case study (2006) 0.65

   0.64897174 = sum of:
     0.64897174 = product of:
       1.3520244 = sum of:
         0.05497407 = weight(abstract_txt:computed in 616) [ClassicSimilarity], result of:
           0.05497407 = score(doc=616,freq=1.0), product of:
             0.13193254 = queryWeight, product of:
               7.61935 = idf(docFreq=58, maxDocs=44218)
               0.017315459 = queryNorm
             0.4166832 = fieldWeight in 616, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.61935 = idf(docFreq=58, maxDocs=44218)
               0.0546875 = fieldNorm(doc=616)
         0.011454634 = weight(abstract_txt:that in 616) [ClassicSimilarity], result of:
           0.011454634 = score(doc=616,freq=3.0), product of:
             0.051036444 = queryWeight, product of:
               1.2439251 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.017315459 = queryNorm
             0.22444029 = fieldWeight in 616, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.0546875 = fieldNorm(doc=616)
         0.046810266 = weight(abstract_txt:queries in 616) [ClassicSimilarity], result of:
           0.046810266 = score(doc=616,freq=2.0), product of:
             0.11852429 = queryWeight, product of:
               1.3404256 = boost
               5.106586 = idf(docFreq=727, maxDocs=44218)
               0.017315459 = queryNorm
             0.39494237 = fieldWeight in 616, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.106586 = idf(docFreq=727, maxDocs=44218)
               0.0546875 = fieldNorm(doc=616)
         0.05935847 = weight(abstract_txt:engine in 616) [ClassicSimilarity], result of:
           0.05935847 = score(doc=616,freq=2.0), product of:
             0.13885722 = queryWeight, product of:
               1.4508525 = boost
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.017315459 = queryNorm
             0.42747846 = fieldWeight in 616, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.0546875 = fieldNorm(doc=616)
         0.04510316 = weight(abstract_txt:experiment in 616) [ClassicSimilarity], result of:
           0.04510316 = score(doc=616,freq=1.0), product of:
             0.14567822 = queryWeight, product of:
               1.48606 = boost
               5.6614056 = idf(docFreq=417, maxDocs=44218)
               0.017315459 = queryNorm
             0.3096081 = fieldWeight in 616, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.6614056 = idf(docFreq=417, maxDocs=44218)
               0.0546875 = fieldNorm(doc=616)
         0.046369303 = weight(abstract_txt:order in 616) [ClassicSimilarity], result of:
           0.046369303 = score(doc=616,freq=2.0), product of:
             0.13482304 = queryWeight, product of:
               1.7509217 = boost
               4.446962 = idf(docFreq=1407, maxDocs=44218)
               0.017315459 = queryNorm
             0.34392715 = fieldWeight in 616, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.446962 = idf(docFreq=1407, maxDocs=44218)
               0.0546875 = fieldNorm(doc=616)
         0.04256431 = weight(abstract_txt:were in 616) [ClassicSimilarity], result of:
           0.04256431 = score(doc=616,freq=3.0), product of:
             0.12243999 = queryWeight, product of:
               1.926707 = boost
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.017315459 = queryNorm
             0.34763405 = fieldWeight in 616, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.0546875 = fieldNorm(doc=616)
         0.028268175 = weight(abstract_txt:users in 616) [ClassicSimilarity], result of:
           0.028268175 = score(doc=616,freq=1.0), product of:
             0.14479998 = queryWeight, product of:
               2.342574 = boost
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.017315459 = queryNorm
             0.19522223 = fieldWeight in 616, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.0546875 = fieldNorm(doc=616)
         0.08331988 = weight(abstract_txt:results in 616) [ClassicSimilarity], result of:
           0.08331988 = score(doc=616,freq=7.0), product of:
             0.16535987 = queryWeight, product of:
               2.7422986 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.017315459 = queryNorm
             0.50387 = fieldWeight in 616, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.0546875 = fieldNorm(doc=616)
         0.19599293 = weight(abstract_txt:engines in 616) [ClassicSimilarity], result of:
           0.19599293 = score(doc=616,freq=7.0), product of:
             0.25550196 = queryWeight, product of:
               2.7832432 = boost
               5.3016257 = idf(docFreq=598, maxDocs=44218)
               0.017315459 = queryNorm
             0.7670897 = fieldWeight in 616, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               5.3016257 = idf(docFreq=598, maxDocs=44218)
               0.0546875 = fieldNorm(doc=616)
         0.16425465 = weight(abstract_txt:search in 616) [ClassicSimilarity], result of:
           0.16425465 = score(doc=616,freq=9.0), product of:
             0.2736897 = queryWeight, product of:
               4.320903 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.017315459 = queryNorm
             0.6001492 = fieldWeight in 616, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.0546875 = fieldNorm(doc=616)
         0.5735546 = weight(abstract_txt:rankings in 616) [ClassicSimilarity], result of:
           0.5735546 = score(doc=616,freq=8.0), product of:
             0.5385878 = queryWeight, product of:
               4.517907 = boost
               6.8847027 = idf(docFreq=122, maxDocs=44218)
               0.017315459 = queryNorm
             1.064923 = fieldWeight in 616, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               6.8847027 = idf(docFreq=122, maxDocs=44218)
               0.0546875 = fieldNorm(doc=616)
       0.48 = coord(12/25)
   

2. Bar-Ilan, J.: Comparing rankings of search results on the Web (2005) 0.56

   0.5564544 = sum of:
     0.5564544 = product of:
       1.2646692 = sum of:
         0.08885152 = weight(abstract_txt:computed in 1068) [ClassicSimilarity], result of:
           0.08885152 = score(doc=1068,freq=2.0), product of:
             0.13193254 = queryWeight, product of:
               7.61935 = idf(docFreq=58, maxDocs=44218)
               0.017315459 = queryNorm
             0.67346174 = fieldWeight in 1068, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.61935 = idf(docFreq=58, maxDocs=44218)
               0.0625 = fieldNorm(doc=1068)
         0.015116198 = weight(abstract_txt:that in 1068) [ClassicSimilarity], result of:
           0.015116198 = score(doc=1068,freq=4.0), product of:
             0.051036444 = queryWeight, product of:
               1.2439251 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.017315459 = queryNorm
             0.2961844 = fieldWeight in 1068, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.0625 = fieldNorm(doc=1068)
         0.053497445 = weight(abstract_txt:queries in 1068) [ClassicSimilarity], result of:
           0.053497445 = score(doc=1068,freq=2.0), product of:
             0.11852429 = queryWeight, product of:
               1.3404256 = boost
               5.106586 = idf(docFreq=727, maxDocs=44218)
               0.017315459 = queryNorm
             0.4513627 = fieldWeight in 1068, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.106586 = idf(docFreq=727, maxDocs=44218)
               0.0625 = fieldNorm(doc=1068)
         0.04796889 = weight(abstract_txt:engine in 1068) [ClassicSimilarity], result of:
           0.04796889 = score(doc=1068,freq=1.0), product of:
             0.13885722 = queryWeight, product of:
               1.4508525 = boost
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.017315459 = queryNorm
             0.34545478 = fieldWeight in 1068, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.0625 = fieldNorm(doc=1068)
         0.052827623 = weight(abstract_txt:retrieved in 1068) [ClassicSimilarity], result of:
           0.052827623 = score(doc=1068,freq=1.0), product of:
             0.14808215 = queryWeight, product of:
               1.4982711 = boost
               5.707926 = idf(docFreq=398, maxDocs=44218)
               0.017315459 = queryNorm
             0.35674536 = fieldWeight in 1068, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.707926 = idf(docFreq=398, maxDocs=44218)
               0.0625 = fieldNorm(doc=1068)
         0.1068948 = weight(abstract_txt:urls in 1068) [ClassicSimilarity], result of:
           0.1068948 = score(doc=1068,freq=1.0), product of:
             0.23690073 = queryWeight, product of:
               1.8950568 = boost
               7.2195506 = idf(docFreq=87, maxDocs=44218)
               0.017315459 = queryNorm
             0.4512219 = fieldWeight in 1068, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.2195506 = idf(docFreq=87, maxDocs=44218)
               0.0625 = fieldNorm(doc=1068)
         0.02808516 = weight(abstract_txt:were in 1068) [ClassicSimilarity], result of:
           0.02808516 = score(doc=1068,freq=1.0), product of:
             0.12243999 = queryWeight, product of:
               1.926707 = boost
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.017315459 = queryNorm
             0.22937898 = fieldWeight in 1068, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.0625 = fieldNorm(doc=1068)
         0.080477886 = weight(abstract_txt:results in 1068) [ClassicSimilarity], result of:
           0.080477886 = score(doc=1068,freq=5.0), product of:
             0.16535987 = queryWeight, product of:
               2.7422986 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.017315459 = queryNorm
             0.48668328 = fieldWeight in 1068, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.0625 = fieldNorm(doc=1068)
         0.2239919 = weight(abstract_txt:engines in 1068) [ClassicSimilarity], result of:
           0.2239919 = score(doc=1068,freq=7.0), product of:
             0.25550196 = queryWeight, product of:
               2.7832432 = boost
               5.3016257 = idf(docFreq=598, maxDocs=44218)
               0.017315459 = queryNorm
             0.87667394 = fieldWeight in 1068, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               5.3016257 = idf(docFreq=598, maxDocs=44218)
               0.0625 = fieldNorm(doc=1068)
         0.16555314 = weight(abstract_txt:search in 1068) [ClassicSimilarity], result of:
           0.16555314 = score(doc=1068,freq=7.0), product of:
             0.2736897 = queryWeight, product of:
               4.320903 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.017315459 = queryNorm
             0.60489357 = fieldWeight in 1068, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.0625 = fieldNorm(doc=1068)
         0.40140462 = weight(abstract_txt:rankings in 1068) [ClassicSimilarity], result of:
           0.40140462 = score(doc=1068,freq=3.0), product of:
             0.5385878 = queryWeight, product of:
               4.517907 = boost
               6.8847027 = idf(docFreq=122, maxDocs=44218)
               0.017315459 = queryNorm
             0.74529094 = fieldWeight in 1068, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.8847027 = idf(docFreq=122, maxDocs=44218)
               0.0625 = fieldNorm(doc=1068)
       0.44 = coord(11/25)
   

3. Bar-Ilan, J.; Keenoy, K.; Levene, M.; Yaari, E.: Presentation bias is significant in determining user preference for search results : a user study (2009) 0.40

   0.40320495 = sum of:
     0.40320495 = product of:
       0.84001034 = sum of:
         0.103158884 = weight(abstract_txt:snippets in 2703) [ClassicSimilarity], result of:
           0.103158884 = score(doc=2703,freq=1.0), product of:
             0.18362132 = queryWeight, product of:
               1.179738 = boost
               8.988837 = idf(docFreq=14, maxDocs=44218)
               0.017315459 = queryNorm
             0.5618023 = fieldWeight in 2703, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.988837 = idf(docFreq=14, maxDocs=44218)
               0.0625 = fieldNorm(doc=2703)
         0.010688766 = weight(abstract_txt:that in 2703) [ClassicSimilarity], result of:
           0.010688766 = score(doc=2703,freq=2.0), product of:
             0.051036444 = queryWeight, product of:
               1.2439251 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.017315459 = queryNorm
             0.20943399 = fieldWeight in 2703, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.0625 = fieldNorm(doc=2703)
         0.037828404 = weight(abstract_txt:queries in 2703) [ClassicSimilarity], result of:
           0.037828404 = score(doc=2703,freq=1.0), product of:
             0.11852429 = queryWeight, product of:
               1.3404256 = boost
               5.106586 = idf(docFreq=727, maxDocs=44218)
               0.017315459 = queryNorm
             0.31916162 = fieldWeight in 2703, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.106586 = idf(docFreq=727, maxDocs=44218)
               0.0625 = fieldNorm(doc=2703)
         0.04796889 = weight(abstract_txt:engine in 2703) [ClassicSimilarity], result of:
           0.04796889 = score(doc=2703,freq=1.0), product of:
             0.13885722 = queryWeight, product of:
               1.4508525 = boost
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.017315459 = queryNorm
             0.34545478 = fieldWeight in 2703, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.0625 = fieldNorm(doc=2703)
         0.07289771 = weight(abstract_txt:experiment in 2703) [ClassicSimilarity], result of:
           0.07289771 = score(doc=2703,freq=2.0), product of:
             0.14567822 = queryWeight, product of:
               1.48606 = boost
               5.6614056 = idf(docFreq=417, maxDocs=44218)
               0.017315459 = queryNorm
             0.5004023 = fieldWeight in 2703, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.6614056 = idf(docFreq=417, maxDocs=44218)
               0.0625 = fieldNorm(doc=2703)
         0.052827623 = weight(abstract_txt:retrieved in 2703) [ClassicSimilarity], result of:
           0.052827623 = score(doc=2703,freq=1.0), product of:
             0.14808215 = queryWeight, product of:
               1.4982711 = boost
               5.707926 = idf(docFreq=398, maxDocs=44218)
               0.017315459 = queryNorm
             0.35674536 = fieldWeight in 2703, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.707926 = idf(docFreq=398, maxDocs=44218)
               0.0625 = fieldNorm(doc=2703)
         0.06742569 = weight(abstract_txt:asked in 2703) [ClassicSimilarity], result of:
           0.06742569 = score(doc=2703,freq=1.0), product of:
             0.1742391 = queryWeight, product of:
               1.6252189 = boost
               6.1915555 = idf(docFreq=245, maxDocs=44218)
               0.017315459 = queryNorm
             0.38697222 = fieldWeight in 2703, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.1915555 = idf(docFreq=245, maxDocs=44218)
               0.0625 = fieldNorm(doc=2703)
         0.02808516 = weight(abstract_txt:were in 2703) [ClassicSimilarity], result of:
           0.02808516 = score(doc=2703,freq=1.0), product of:
             0.12243999 = queryWeight, product of:
               1.926707 = boost
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.017315459 = queryNorm
             0.22937898 = fieldWeight in 2703, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.0625 = fieldNorm(doc=2703)
         0.04568827 = weight(abstract_txt:users in 2703) [ClassicSimilarity], result of:
           0.04568827 = score(doc=2703,freq=2.0), product of:
             0.14479998 = queryWeight, product of:
               2.342574 = boost
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.017315459 = queryNorm
             0.31552678 = fieldWeight in 2703, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.0625 = fieldNorm(doc=2703)
         0.08815911 = weight(abstract_txt:results in 2703) [ClassicSimilarity], result of:
           0.08815911 = score(doc=2703,freq=6.0), product of:
             0.16535987 = queryWeight, product of:
               2.7422986 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.017315459 = queryNorm
             0.5331348 = fieldWeight in 2703, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.0625 = fieldNorm(doc=2703)
         0.119728714 = weight(abstract_txt:engines in 2703) [ClassicSimilarity], result of:
           0.119728714 = score(doc=2703,freq=2.0), product of:
             0.25550196 = queryWeight, product of:
               2.7832432 = boost
               5.3016257 = idf(docFreq=598, maxDocs=44218)
               0.017315459 = queryNorm
             0.46860194 = fieldWeight in 2703, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.3016257 = idf(docFreq=598, maxDocs=44218)
               0.0625 = fieldNorm(doc=2703)
         0.16555314 = weight(abstract_txt:search in 2703) [ClassicSimilarity], result of:
           0.16555314 = score(doc=2703,freq=7.0), product of:
             0.2736897 = queryWeight, product of:
               4.320903 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.017315459 = queryNorm
             0.60489357 = fieldWeight in 2703, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.0625 = fieldNorm(doc=2703)
       0.48 = coord(12/25)
   

4. Vaughan, L.: New measurements for search engine evaluation proposed and tested (2004) 0.38

   0.3755079 = sum of:
     0.3755079 = product of:
       1.0430775 = sum of:
         0.011337148 = weight(abstract_txt:that in 2535) [ClassicSimilarity], result of:
           0.011337148 = score(doc=2535,freq=1.0), product of:
             0.051036444 = queryWeight, product of:
               1.2439251 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.017315459 = queryNorm
             0.22213829 = fieldWeight in 2535, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.09375 = fieldNorm(doc=2535)
         0.14390667 = weight(abstract_txt:engine in 2535) [ClassicSimilarity], result of:
           0.14390667 = score(doc=2535,freq=4.0), product of:
             0.13885722 = queryWeight, product of:
               1.4508525 = boost
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.017315459 = queryNorm
             1.0363643 = fieldWeight in 2535, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.09375 = fieldNorm(doc=2535)
         0.077319704 = weight(abstract_txt:experiment in 2535) [ClassicSimilarity], result of:
           0.077319704 = score(doc=2535,freq=1.0), product of:
             0.14567822 = queryWeight, product of:
               1.48606 = boost
               5.6614056 = idf(docFreq=417, maxDocs=44218)
               0.017315459 = queryNorm
             0.5307568 = fieldWeight in 2535, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.6614056 = idf(docFreq=417, maxDocs=44218)
               0.09375 = fieldNorm(doc=2535)
         0.029905029 = weight(abstract_txt:which in 2535) [ClassicSimilarity], result of:
           0.029905029 = score(doc=2535,freq=2.0), product of:
             0.07733285 = queryWeight, product of:
               1.5312145 = boost
               2.9167147 = idf(docFreq=6503, maxDocs=44218)
               0.017315459 = queryNorm
             0.3867054 = fieldWeight in 2535, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.9167147 = idf(docFreq=6503, maxDocs=44218)
               0.09375 = fieldNorm(doc=2535)
         0.04212774 = weight(abstract_txt:were in 2535) [ClassicSimilarity], result of:
           0.04212774 = score(doc=2535,freq=1.0), product of:
             0.12243999 = queryWeight, product of:
               1.926707 = boost
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.017315459 = queryNorm
             0.34406847 = fieldWeight in 2535, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.09375 = fieldNorm(doc=2535)
         0.053986207 = weight(abstract_txt:results in 2535) [ClassicSimilarity], result of:
           0.053986207 = score(doc=2535,freq=1.0), product of:
             0.16535987 = queryWeight, product of:
               2.7422986 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.017315459 = queryNorm
             0.32647708 = fieldWeight in 2535, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.09375 = fieldNorm(doc=2535)
         0.12699147 = weight(abstract_txt:engines in 2535) [ClassicSimilarity], result of:
           0.12699147 = score(doc=2535,freq=1.0), product of:
             0.25550196 = queryWeight, product of:
               2.7832432 = boost
               5.3016257 = idf(docFreq=598, maxDocs=44218)
               0.017315459 = queryNorm
             0.4970274 = fieldWeight in 2535, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.3016257 = idf(docFreq=598, maxDocs=44218)
               0.09375 = fieldNorm(doc=2535)
         0.20987691 = weight(abstract_txt:search in 2535) [ClassicSimilarity], result of:
           0.20987691 = score(doc=2535,freq=5.0), product of:
             0.2736897 = queryWeight, product of:
               4.320903 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.017315459 = queryNorm
             0.7668426 = fieldWeight in 2535, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.09375 = fieldNorm(doc=2535)
         0.3476266 = weight(abstract_txt:rankings in 2535) [ClassicSimilarity], result of:
           0.3476266 = score(doc=2535,freq=1.0), product of:
             0.5385878 = queryWeight, product of:
               4.517907 = boost
               6.8847027 = idf(docFreq=122, maxDocs=44218)
               0.017315459 = queryNorm
             0.6454409 = fieldWeight in 2535, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.8847027 = idf(docFreq=122, maxDocs=44218)
               0.09375 = fieldNorm(doc=2535)
       0.36 = coord(9/25)
   

5. Spink, A.; Jansen, B.J.; Blakely, C.; Koshman, S.: ¬A study of results overlap and uniqueness among major Web search engines (2006) 0.33

   0.33228225 = sum of:
     0.33228225 = product of:
       0.92300624 = sum of:
         0.006613337 = weight(abstract_txt:that in 993) [ClassicSimilarity], result of:
           0.006613337 = score(doc=993,freq=1.0), product of:
             0.051036444 = queryWeight, product of:
               1.2439251 = boost
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.017315459 = queryNorm
             0.12958068 = fieldWeight in 993, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.3694751 = idf(docFreq=11241, maxDocs=44218)
               0.0546875 = fieldNorm(doc=993)
         0.09362053 = weight(abstract_txt:queries in 993) [ClassicSimilarity], result of:
           0.09362053 = score(doc=993,freq=8.0), product of:
             0.11852429 = queryWeight, product of:
               1.3404256 = boost
               5.106586 = idf(docFreq=727, maxDocs=44218)
               0.017315459 = queryNorm
             0.78988475 = fieldWeight in 993, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               5.106586 = idf(docFreq=727, maxDocs=44218)
               0.0546875 = fieldNorm(doc=993)
         0.05935847 = weight(abstract_txt:engine in 993) [ClassicSimilarity], result of:
           0.05935847 = score(doc=993,freq=2.0), product of:
             0.13885722 = queryWeight, product of:
               1.4508525 = boost
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.017315459 = queryNorm
             0.42747846 = fieldWeight in 993, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.5272765 = idf(docFreq=477, maxDocs=44218)
               0.0546875 = fieldNorm(doc=993)
         0.06537084 = weight(abstract_txt:retrieved in 993) [ClassicSimilarity], result of:
           0.06537084 = score(doc=993,freq=2.0), product of:
             0.14808215 = queryWeight, product of:
               1.4982711 = boost
               5.707926 = idf(docFreq=398, maxDocs=44218)
               0.017315459 = queryNorm
             0.44144982 = fieldWeight in 993, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.707926 = idf(docFreq=398, maxDocs=44218)
               0.0546875 = fieldNorm(doc=993)
         0.06589128 = weight(abstract_txt:rank in 993) [ClassicSimilarity], result of:
           0.06589128 = score(doc=993,freq=1.0), product of:
             0.18756075 = queryWeight, product of:
               1.6862036 = boost
               6.4238877 = idf(docFreq=194, maxDocs=44218)
               0.017315459 = queryNorm
             0.35130635 = fieldWeight in 993, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.4238877 = idf(docFreq=194, maxDocs=44218)
               0.0546875 = fieldNorm(doc=993)
         0.03475361 = weight(abstract_txt:were in 993) [ClassicSimilarity], result of:
           0.03475361 = score(doc=993,freq=2.0), product of:
             0.12243999 = queryWeight, product of:
               1.926707 = boost
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.017315459 = queryNorm
             0.283842 = fieldWeight in 993, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.6700637 = idf(docFreq=3061, maxDocs=44218)
               0.0546875 = fieldNorm(doc=993)
         0.094475865 = weight(abstract_txt:results in 993) [ClassicSimilarity], result of:
           0.094475865 = score(doc=993,freq=9.0), product of:
             0.16535987 = queryWeight, product of:
               2.7422986 = boost
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.017315459 = queryNorm
             0.5713349 = fieldWeight in 993, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               3.482422 = idf(docFreq=3693, maxDocs=44218)
               0.0546875 = fieldNorm(doc=993)
         0.27717587 = weight(abstract_txt:engines in 993) [ClassicSimilarity], result of:
           0.27717587 = score(doc=993,freq=14.0), product of:
             0.25550196 = queryWeight, product of:
               2.7832432 = boost
               5.3016257 = idf(docFreq=598, maxDocs=44218)
               0.017315459 = queryNorm
             1.0848287 = fieldWeight in 993, product of:
               3.7416575 = tf(freq=14.0), with freq of:
                 14.0 = termFreq=14.0
               5.3016257 = idf(docFreq=598, maxDocs=44218)
               0.0546875 = fieldNorm(doc=993)
         0.22574644 = weight(abstract_txt:search in 993) [ClassicSimilarity], result of:
           0.22574644 = score(doc=993,freq=17.0), product of:
             0.2736897 = queryWeight, product of:
               4.320903 = boost
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.017315459 = queryNorm
             0.82482624 = fieldWeight in 993, product of:
               4.1231055 = tf(freq=17.0), with freq of:
                 17.0 = termFreq=17.0
               3.6580524 = idf(docFreq=3098, maxDocs=44218)
               0.0546875 = fieldNorm(doc=993)
       0.36 = coord(9/25)