Document (#20947)

Author

Burgin, R.

Title

¬The Monte Carlo method and the evaluation of retrieval system performance

Source

Journal of the American Society for Information Science. 50(1999) no.2, S.181-191

Year

1999

Abstract

The ability to distinguish between acceptable and unacceptable levels of retrieval performance and the ability to distinguish between significant and non-significant differences between retrieval result are important to traditional information retrieval experiments. The Monte Carlo method is shown to represent an attractive alternative to the hypergeometric model for identifying the levels at which random retrieval performance is exceeded in retrieval test collections and for overcoming some of the limitations of the hypergeometric model

Theme

Retrievalstudien

Similar documents (author)

1. Burgin, R.: Variations in relevance judgements and the evaluation of retrieval performance (1992) 6.01

   6.010904 = sum of:
     6.010904 = weight(author_txt:burgin in 3308) [ClassicSimilarity], result of:
       6.010904 = fieldWeight in 3308, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.617446 = idf(docFreq=7, maxDocs=44218)
         0.625 = fieldNorm(doc=3308)
   

2. Burgin, R.: ¬The retrieval effectiveness of 5 clustering algorithms as a function of indexing exhaustivity (1995) 6.01

   6.010904 = sum of:
     6.010904 = weight(author_txt:burgin in 3365) [ClassicSimilarity], result of:
       6.010904 = fieldWeight in 3365, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.617446 = idf(docFreq=7, maxDocs=44218)
         0.625 = fieldNorm(doc=3365)
   

3. Burgin, M.: Terminological and teleological systems (1996) 6.01

   6.010904 = sum of:
     6.010904 = weight(author_txt:burgin in 6323) [ClassicSimilarity], result of:
       6.010904 = fieldWeight in 6323, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.617446 = idf(docFreq=7, maxDocs=44218)
         0.625 = fieldNorm(doc=6323)
   

4. Burgin, R.: ¬The effect of indexing exhaustivity on retrieval performance (1991) 6.01

   6.010904 = sum of:
     6.010904 = weight(author_txt:burgin in 5262) [ClassicSimilarity], result of:
       6.010904 = fieldWeight in 5262, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.617446 = idf(docFreq=7, maxDocs=44218)
         0.625 = fieldNorm(doc=5262)
   

5. Shaw, W.M.; Burgin, R.; Howell, P.: Performance standards and evaluations in IR test collections : cluster-based retrieval models (1997) 3.61

   3.606542 = sum of:
     3.606542 = weight(author_txt:burgin in 7256) [ClassicSimilarity], result of:
       3.606542 = fieldWeight in 7256, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         9.617446 = idf(docFreq=7, maxDocs=44218)
         0.375 = fieldNorm(doc=7256)
   

Similar documents (content)

1. Hopkins, M.E.; Zavalina, O.L.: Evaluating physicians' serendipitous knowledge discovery in online discovery systems : a new approach (2019) 0.18

   0.18292455 = sum of:
     0.18292455 = product of:
       0.76218563 = sum of:
         0.021665014 = weight(abstract_txt:test in 5842) [ClassicSimilarity], result of:
           0.021665014 = score(doc=5842,freq=1.0), product of:
             0.06868777 = queryWeight, product of:
               1.0167469 = boost
               5.046608 = idf(docFreq=772, maxDocs=44218)
               0.013386498 = queryNorm
             0.315413 = fieldWeight in 5842, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.046608 = idf(docFreq=772, maxDocs=44218)
               0.0625 = fieldNorm(doc=5842)
         0.02434157 = weight(abstract_txt:limitations in 5842) [ClassicSimilarity], result of:
           0.02434157 = score(doc=5842,freq=1.0), product of:
             0.0742345 = queryWeight, product of:
               1.0570025 = boost
               5.2464166 = idf(docFreq=632, maxDocs=44218)
               0.013386498 = queryNorm
             0.32790104 = fieldWeight in 5842, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.2464166 = idf(docFreq=632, maxDocs=44218)
               0.0625 = fieldNorm(doc=5842)
         0.03019907 = weight(abstract_txt:model in 5842) [ClassicSimilarity], result of:
           0.03019907 = score(doc=5842,freq=2.0), product of:
             0.085710846 = queryWeight, product of:
               1.6062237 = boost
               3.986234 = idf(docFreq=2231, maxDocs=44218)
               0.013386498 = queryNorm
             0.35233662 = fieldWeight in 5842, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.986234 = idf(docFreq=2231, maxDocs=44218)
               0.0625 = fieldNorm(doc=5842)
         0.030739926 = weight(abstract_txt:method in 5842) [ClassicSimilarity], result of:
           0.030739926 = score(doc=5842,freq=1.0), product of:
             0.10927445 = queryWeight, product of:
               1.8136247 = boost
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.013386498 = queryNorm
             0.28130937 = fieldWeight in 5842, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.0625 = fieldNorm(doc=5842)
         0.32762003 = weight(abstract_txt:monte in 5842) [ClassicSimilarity], result of:
           0.32762003 = score(doc=5842,freq=1.0), product of:
             0.52921283 = queryWeight, product of:
               3.9911985 = boost
               9.905128 = idf(docFreq=5, maxDocs=44218)
               0.013386498 = queryNorm
             0.6190705 = fieldWeight in 5842, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.905128 = idf(docFreq=5, maxDocs=44218)
               0.0625 = fieldNorm(doc=5842)
         0.32762003 = weight(abstract_txt:carlo in 5842) [ClassicSimilarity], result of:
           0.32762003 = score(doc=5842,freq=1.0), product of:
             0.52921283 = queryWeight, product of:
               3.9911985 = boost
               9.905128 = idf(docFreq=5, maxDocs=44218)
               0.013386498 = queryNorm
             0.6190705 = fieldWeight in 5842, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.905128 = idf(docFreq=5, maxDocs=44218)
               0.0625 = fieldNorm(doc=5842)
       0.24 = coord(6/25)
   

2. Almerri, J.; McGregor, D.R.: Codon signatures : a document retrieval method (1996) 0.16

   0.16272777 = sum of:
     0.16272777 = product of:
       0.5811706 = sum of:
         0.07854868 = weight(abstract_txt:acceptable in 6970) [ClassicSimilarity], result of:
           0.07854868 = score(doc=6970,freq=1.0), product of:
             0.13969879 = queryWeight, product of:
               1.4500048 = boost
               7.1970778 = idf(docFreq=89, maxDocs=44218)
               0.013386498 = queryNorm
             0.5622717 = fieldWeight in 6970, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.1970778 = idf(docFreq=89, maxDocs=44218)
               0.078125 = fieldNorm(doc=6970)
         0.06655389 = weight(abstract_txt:method in 6970) [ClassicSimilarity], result of:
           0.06655389 = score(doc=6970,freq=3.0), product of:
             0.10927445 = queryWeight, product of:
               1.8136247 = boost
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.013386498 = queryNorm
             0.60905266 = fieldWeight in 6970, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.078125 = fieldNorm(doc=6970)
         0.037137143 = weight(abstract_txt:between in 6970) [ClassicSimilarity], result of:
           0.037137143 = score(doc=6970,freq=2.0), product of:
             0.09705162 = queryWeight, product of:
               2.0933177 = boost
               3.4633842 = idf(docFreq=3764, maxDocs=44218)
               0.013386498 = queryNorm
             0.3826535 = fieldWeight in 6970, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4633842 = idf(docFreq=3764, maxDocs=44218)
               0.078125 = fieldNorm(doc=6970)
         0.073081315 = weight(abstract_txt:ability in 6970) [ClassicSimilarity], result of:
           0.073081315 = score(doc=6970,freq=1.0), product of:
             0.16774425 = queryWeight, product of:
               2.247046 = boost
               5.5765896 = idf(docFreq=454, maxDocs=44218)
               0.013386498 = queryNorm
             0.43567106 = fieldWeight in 6970, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5765896 = idf(docFreq=454, maxDocs=44218)
               0.078125 = fieldNorm(doc=6970)
         0.10869594 = weight(abstract_txt:performance in 6970) [ClassicSimilarity], result of:
           0.10869594 = score(doc=6970,freq=3.0), product of:
             0.17347716 = queryWeight, product of:
               2.7986908 = boost
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.013386498 = queryNorm
             0.6265721 = fieldWeight in 6970, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.078125 = fieldNorm(doc=6970)
         0.14212056 = weight(abstract_txt:distinguish in 6970) [ClassicSimilarity], result of:
           0.14212056 = score(doc=6970,freq=1.0), product of:
             0.26134527 = queryWeight, product of:
               2.8047585 = boost
               6.9606886 = idf(docFreq=113, maxDocs=44218)
               0.013386498 = queryNorm
             0.5438038 = fieldWeight in 6970, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9606886 = idf(docFreq=113, maxDocs=44218)
               0.078125 = fieldNorm(doc=6970)
         0.07503314 = weight(abstract_txt:retrieval in 6970) [ClassicSimilarity], result of:
           0.07503314 = score(doc=6970,freq=2.0), product of:
             0.19542311 = queryWeight, product of:
               4.2008457 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.013386498 = queryNorm
             0.38395226 = fieldWeight in 6970, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.078125 = fieldNorm(doc=6970)
       0.28 = coord(7/25)
   

3. Schaalje, G.B.; Blades, N.J.; Funai, T.: ¬An open-set size-adjusted Bayesian classifier for authorship attribution (2013) 0.15

   0.1513414 = sum of:
     0.1513414 = product of:
       0.75670695 = sum of:
         0.036986157 = weight(abstract_txt:model in 1041) [ClassicSimilarity], result of:
           0.036986157 = score(doc=1041,freq=3.0), product of:
             0.085710846 = queryWeight, product of:
               1.6062237 = boost
               3.986234 = idf(docFreq=2231, maxDocs=44218)
               0.013386498 = queryNorm
             0.4315225 = fieldWeight in 1041, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.986234 = idf(docFreq=2231, maxDocs=44218)
               0.0625 = fieldNorm(doc=1041)
         0.043472815 = weight(abstract_txt:method in 1041) [ClassicSimilarity], result of:
           0.043472815 = score(doc=1041,freq=2.0), product of:
             0.10927445 = queryWeight, product of:
               1.8136247 = boost
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.013386498 = queryNorm
             0.3978315 = fieldWeight in 1041, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.0625 = fieldNorm(doc=1041)
         0.02100794 = weight(abstract_txt:between in 1041) [ClassicSimilarity], result of:
           0.02100794 = score(doc=1041,freq=1.0), product of:
             0.09705162 = queryWeight, product of:
               2.0933177 = boost
               3.4633842 = idf(docFreq=3764, maxDocs=44218)
               0.013386498 = queryNorm
             0.21646151 = fieldWeight in 1041, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4633842 = idf(docFreq=3764, maxDocs=44218)
               0.0625 = fieldNorm(doc=1041)
         0.32762003 = weight(abstract_txt:monte in 1041) [ClassicSimilarity], result of:
           0.32762003 = score(doc=1041,freq=1.0), product of:
             0.52921283 = queryWeight, product of:
               3.9911985 = boost
               9.905128 = idf(docFreq=5, maxDocs=44218)
               0.013386498 = queryNorm
             0.6190705 = fieldWeight in 1041, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.905128 = idf(docFreq=5, maxDocs=44218)
               0.0625 = fieldNorm(doc=1041)
         0.32762003 = weight(abstract_txt:carlo in 1041) [ClassicSimilarity], result of:
           0.32762003 = score(doc=1041,freq=1.0), product of:
             0.52921283 = queryWeight, product of:
               3.9911985 = boost
               9.905128 = idf(docFreq=5, maxDocs=44218)
               0.013386498 = queryNorm
             0.6190705 = fieldWeight in 1041, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.905128 = idf(docFreq=5, maxDocs=44218)
               0.0625 = fieldNorm(doc=1041)
       0.2 = coord(5/25)
   

4. Rubin, V.L.; Lukoianova, T.: Truth and deception at the rhetorical structure level (2015) 0.14

   0.13851015 = sum of:
     0.13851015 = product of:
       0.38475043 = sum of:
         0.020612016 = weight(abstract_txt:differences in 1816) [ClassicSimilarity], result of:
           0.020612016 = score(doc=1816,freq=1.0), product of:
             0.06644369 = queryWeight, product of:
               4.9634852 = idf(docFreq=839, maxDocs=44218)
               0.013386498 = queryNorm
             0.31021783 = fieldWeight in 1816, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9634852 = idf(docFreq=839, maxDocs=44218)
               0.0625 = fieldNorm(doc=1816)
         0.021665014 = weight(abstract_txt:test in 1816) [ClassicSimilarity], result of:
           0.021665014 = score(doc=1816,freq=1.0), product of:
             0.06868777 = queryWeight, product of:
               1.0167469 = boost
               5.046608 = idf(docFreq=772, maxDocs=44218)
               0.013386498 = queryNorm
             0.315413 = fieldWeight in 1816, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.046608 = idf(docFreq=772, maxDocs=44218)
               0.0625 = fieldNorm(doc=1816)
         0.021353967 = weight(abstract_txt:model in 1816) [ClassicSimilarity], result of:
           0.021353967 = score(doc=1816,freq=1.0), product of:
             0.085710846 = queryWeight, product of:
               1.6062237 = boost
               3.986234 = idf(docFreq=2231, maxDocs=44218)
               0.013386498 = queryNorm
             0.24913962 = fieldWeight in 1816, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.986234 = idf(docFreq=2231, maxDocs=44218)
               0.0625 = fieldNorm(doc=1816)
         0.043472815 = weight(abstract_txt:method in 1816) [ClassicSimilarity], result of:
           0.043472815 = score(doc=1816,freq=2.0), product of:
             0.10927445 = queryWeight, product of:
               1.8136247 = boost
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.013386498 = queryNorm
             0.3978315 = fieldWeight in 1816, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.0625 = fieldNorm(doc=1816)
         0.036528084 = weight(abstract_txt:significant in 1816) [ClassicSimilarity], result of:
           0.036528084 = score(doc=1816,freq=1.0), product of:
             0.122593656 = queryWeight, product of:
               1.9209769 = boost
               4.76737 = idf(docFreq=1021, maxDocs=44218)
               0.013386498 = queryNorm
             0.29796064 = fieldWeight in 1816, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.76737 = idf(docFreq=1021, maxDocs=44218)
               0.0625 = fieldNorm(doc=1816)
         0.02100794 = weight(abstract_txt:between in 1816) [ClassicSimilarity], result of:
           0.02100794 = score(doc=1816,freq=1.0), product of:
             0.09705162 = queryWeight, product of:
               2.0933177 = boost
               3.4633842 = idf(docFreq=3764, maxDocs=44218)
               0.013386498 = queryNorm
             0.21646151 = fieldWeight in 1816, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.4633842 = idf(docFreq=3764, maxDocs=44218)
               0.0625 = fieldNorm(doc=1816)
         0.04794911 = weight(abstract_txt:levels in 1816) [ClassicSimilarity], result of:
           0.04794911 = score(doc=1816,freq=1.0), product of:
             0.14697285 = queryWeight, product of:
               2.1033266 = boost
               5.219915 = idf(docFreq=649, maxDocs=44218)
               0.013386498 = queryNorm
             0.32624468 = fieldWeight in 1816, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.219915 = idf(docFreq=649, maxDocs=44218)
               0.0625 = fieldNorm(doc=1816)
         0.058465052 = weight(abstract_txt:ability in 1816) [ClassicSimilarity], result of:
           0.058465052 = score(doc=1816,freq=1.0), product of:
             0.16774425 = queryWeight, product of:
               2.247046 = boost
               5.5765896 = idf(docFreq=454, maxDocs=44218)
               0.013386498 = queryNorm
             0.34853685 = fieldWeight in 1816, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.5765896 = idf(docFreq=454, maxDocs=44218)
               0.0625 = fieldNorm(doc=1816)
         0.11369644 = weight(abstract_txt:distinguish in 1816) [ClassicSimilarity], result of:
           0.11369644 = score(doc=1816,freq=1.0), product of:
             0.26134527 = queryWeight, product of:
               2.8047585 = boost
               6.9606886 = idf(docFreq=113, maxDocs=44218)
               0.013386498 = queryNorm
             0.43504304 = fieldWeight in 1816, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.9606886 = idf(docFreq=113, maxDocs=44218)
               0.0625 = fieldNorm(doc=1816)
       0.36 = coord(9/25)
   

5. Shaw, W.M.; Burgin, R.; Howell, P.: Performance standards and evaluations in IR test collections : cluster-based retrieval models (1997) 0.13

   0.13117947 = sum of:
     0.13117947 = product of:
       0.54658115 = sum of:
         0.045958437 = weight(abstract_txt:test in 7256) [ClassicSimilarity], result of:
           0.045958437 = score(doc=7256,freq=2.0), product of:
             0.06868777 = queryWeight, product of:
               1.0167469 = boost
               5.046608 = idf(docFreq=772, maxDocs=44218)
               0.013386498 = queryNorm
             0.669092 = fieldWeight in 7256, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.046608 = idf(docFreq=772, maxDocs=44218)
               0.09375 = fieldNorm(doc=7256)
         0.04255097 = weight(abstract_txt:represent in 7256) [ClassicSimilarity], result of:
           0.04255097 = score(doc=7256,freq=1.0), product of:
             0.0822089 = queryWeight, product of:
               1.1123272 = boost
               5.52102 = idf(docFreq=480, maxDocs=44218)
               0.013386498 = queryNorm
             0.51759565 = fieldWeight in 7256, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.52102 = idf(docFreq=480, maxDocs=44218)
               0.09375 = fieldNorm(doc=7256)
         0.09966099 = weight(abstract_txt:random in 7256) [ClassicSimilarity], result of:
           0.09966099 = score(doc=7256,freq=2.0), product of:
             0.11507633 = queryWeight, product of:
               1.3160311 = boost
               6.532101 = idf(docFreq=174, maxDocs=44218)
               0.013386498 = queryNorm
             0.8660425 = fieldWeight in 7256, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.532101 = idf(docFreq=174, maxDocs=44218)
               0.09375 = fieldNorm(doc=7256)
         0.12457544 = weight(abstract_txt:levels in 7256) [ClassicSimilarity], result of:
           0.12457544 = score(doc=7256,freq=3.0), product of:
             0.14697285 = queryWeight, product of:
               2.1033266 = boost
               5.219915 = idf(docFreq=649, maxDocs=44218)
               0.013386498 = queryNorm
             0.84760857 = fieldWeight in 7256, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.219915 = idf(docFreq=649, maxDocs=44218)
               0.09375 = fieldNorm(doc=7256)
         0.10649984 = weight(abstract_txt:performance in 7256) [ClassicSimilarity], result of:
           0.10649984 = score(doc=7256,freq=2.0), product of:
             0.17347716 = queryWeight, product of:
               2.7986908 = boost
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.013386498 = queryNorm
             0.61391276 = fieldWeight in 7256, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.09375 = fieldNorm(doc=7256)
         0.12733547 = weight(abstract_txt:retrieval in 7256) [ClassicSimilarity], result of:
           0.12733547 = score(doc=7256,freq=4.0), product of:
             0.19542311 = queryWeight, product of:
               4.2008457 = boost
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.013386498 = queryNorm
             0.6515886 = fieldWeight in 7256, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.4751394 = idf(docFreq=3720, maxDocs=44218)
               0.09375 = fieldNorm(doc=7256)
       0.24 = coord(6/25)