Document (#39271)

Author

Mutz, R.
Bornmann, L.
Daniel, H.-D.

Title

Testing for the fairness and predictive validity of research funding decisions : a multilevel multiple imputation for missing data approach using ex-ante and ex-post peer evaluation data from the Austrian science fund

Source

Journal of the Association for Information Science and Technology. 66(2015) no.11, S.2321-2339

Year

2015

Abstract

It is essential for research funding organizations to ensure both the validity and fairness of the grant approval procedure. The ex-ante peer evaluation (EXANTE) of N?=?8,496 grant applications submitted to the Austrian Science Fund from 1999 to 2009 was statistically analyzed. For 1,689 funded research projects an ex-post peer evaluation (EXPOST) was also available; for the rest of the grant applications a multilevel missing data imputation approach was used to consider verification bias for the first time in peer-review research. Without imputation, the predictive validity of EXANTE was low (r?=?.26) but underestimated due to verification bias, and with imputation it was r?=?.49. That is, the decision-making procedure is capable of selecting the best research proposals for funding. In the EXANTE there were several potential biases (e.g., gender). With respect to the EXPOST there was only one real bias (discipline-specific and year-specific differential prediction). The novelty of this contribution is, first, the combining of theoretical concepts of validity and fairness with a missing data imputation approach to correct for verification bias and, second, multilevel modeling to test peer review-based funding decisions for both validity and fairness in terms of potential and real biases.

Content

Vgl.: http://onlinelibrary.wiley.com/doi/10.1002/asi.23315/abstract.

Similar documents (author)

1. Bornmann, L.; Mutz, R.; Daniel, H.-D.: Are there better indices for evaluation purposes than the h index? : a comparison of nine different variants of the h index using data from biomedicine (2008) 5.43

   5.427659 = sum of:
     5.427659 = sum of:
       1.3157566 = weight(author_txt:bornmann in 1608) [ClassicSimilarity], result of:
         1.3157566 = score(doc=1608,freq=1.0), product of:
           0.4635137 = queryWeight, product of:
             7.5697527 = idf(docFreq=61, maxDocs=44218)
             0.061232343 = queryNorm
           2.8386574 = fieldWeight in 1608, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             7.5697527 = idf(docFreq=61, maxDocs=44218)
             0.375 = fieldNorm(doc=1608)
       1.6727548 = weight(author_txt:daniel in 1608) [ClassicSimilarity], result of:
         1.6727548 = score(doc=1608,freq=1.0), product of:
           0.5439601 = queryWeight, product of:
             1.0833087 = boost
             8.200379 = idf(docFreq=32, maxDocs=44218)
             0.061232343 = queryNorm
           3.0751424 = fieldWeight in 1608, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.200379 = idf(docFreq=32, maxDocs=44218)
             0.375 = fieldNorm(doc=1608)
       2.4391477 = weight(author_txt:mutz in 1608) [ClassicSimilarity], result of:
         2.4391477 = score(doc=1608,freq=1.0), product of:
           0.69947296 = queryWeight, product of:
             1.2284406 = boost
             9.298992 = idf(docFreq=10, maxDocs=44218)
             0.061232343 = queryNorm
           3.487122 = fieldWeight in 1608, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.298992 = idf(docFreq=10, maxDocs=44218)
             0.375 = fieldNorm(doc=1608)
   

2. Bornmann, L.; Mutz, R.; Daniel, H.D.: Do we need the h index and its variants in addition to standard bibliometric measures? (2009) 5.43

   5.427659 = sum of:
     5.427659 = sum of:
       1.3157566 = weight(author_txt:bornmann in 2861) [ClassicSimilarity], result of:
         1.3157566 = score(doc=2861,freq=1.0), product of:
           0.4635137 = queryWeight, product of:
             7.5697527 = idf(docFreq=61, maxDocs=44218)
             0.061232343 = queryNorm
           2.8386574 = fieldWeight in 2861, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             7.5697527 = idf(docFreq=61, maxDocs=44218)
             0.375 = fieldNorm(doc=2861)
       1.6727548 = weight(author_txt:daniel in 2861) [ClassicSimilarity], result of:
         1.6727548 = score(doc=2861,freq=1.0), product of:
           0.5439601 = queryWeight, product of:
             1.0833087 = boost
             8.200379 = idf(docFreq=32, maxDocs=44218)
             0.061232343 = queryNorm
           3.0751424 = fieldWeight in 2861, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.200379 = idf(docFreq=32, maxDocs=44218)
             0.375 = fieldNorm(doc=2861)
       2.4391477 = weight(author_txt:mutz in 2861) [ClassicSimilarity], result of:
         2.4391477 = score(doc=2861,freq=1.0), product of:
           0.69947296 = queryWeight, product of:
             1.2284406 = boost
             9.298992 = idf(docFreq=10, maxDocs=44218)
             0.061232343 = queryNorm
           3.487122 = fieldWeight in 2861, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.298992 = idf(docFreq=10, maxDocs=44218)
             0.375 = fieldNorm(doc=2861)
   

3. Bornmann, L.; Mutz, R.; Daniel, H.-D.: Multilevel-statistical reformulation of citation-based university rankings : the Leiden ranking 2011/2012 (2013) 5.43

   5.427659 = sum of:
     5.427659 = sum of:
       1.3157566 = weight(author_txt:bornmann in 1007) [ClassicSimilarity], result of:
         1.3157566 = score(doc=1007,freq=1.0), product of:
           0.4635137 = queryWeight, product of:
             7.5697527 = idf(docFreq=61, maxDocs=44218)
             0.061232343 = queryNorm
           2.8386574 = fieldWeight in 1007, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             7.5697527 = idf(docFreq=61, maxDocs=44218)
             0.375 = fieldNorm(doc=1007)
       1.6727548 = weight(author_txt:daniel in 1007) [ClassicSimilarity], result of:
         1.6727548 = score(doc=1007,freq=1.0), product of:
           0.5439601 = queryWeight, product of:
             1.0833087 = boost
             8.200379 = idf(docFreq=32, maxDocs=44218)
             0.061232343 = queryNorm
           3.0751424 = fieldWeight in 1007, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             8.200379 = idf(docFreq=32, maxDocs=44218)
             0.375 = fieldNorm(doc=1007)
       2.4391477 = weight(author_txt:mutz in 1007) [ClassicSimilarity], result of:
         2.4391477 = score(doc=1007,freq=1.0), product of:
           0.69947296 = queryWeight, product of:
             1.2284406 = boost
             9.298992 = idf(docFreq=10, maxDocs=44218)
             0.061232343 = queryNorm
           3.487122 = fieldWeight in 1007, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.298992 = idf(docFreq=10, maxDocs=44218)
             0.375 = fieldNorm(doc=1007)
   

4. Bornmann, L.; Mutz, R.: From P100 to P100' : a new citation-rank approach (2014) 3.34

   3.3376927 = sum of:
     3.3376927 = product of:
       5.006539 = sum of:
         1.7543421 = weight(author_txt:bornmann in 1431) [ClassicSimilarity], result of:
           1.7543421 = score(doc=1431,freq=1.0), product of:
             0.4635137 = queryWeight, product of:
               7.5697527 = idf(docFreq=61, maxDocs=44218)
               0.061232343 = queryNorm
             3.7848763 = fieldWeight in 1431, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.5697527 = idf(docFreq=61, maxDocs=44218)
               0.5 = fieldNorm(doc=1431)
         3.2521968 = weight(author_txt:mutz in 1431) [ClassicSimilarity], result of:
           3.2521968 = score(doc=1431,freq=1.0), product of:
             0.69947296 = queryWeight, product of:
               1.2284406 = boost
               9.298992 = idf(docFreq=10, maxDocs=44218)
               0.061232343 = queryNorm
             4.649496 = fieldWeight in 1431, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.298992 = idf(docFreq=10, maxDocs=44218)
               0.5 = fieldNorm(doc=1431)
       0.6666667 = coord(2/3)
   

5. Bornmann, L.; Mutz, R.: Growth rates of modern science : a bibliometric analysis based on the number of publications and cited references (2015) 3.34

   3.3376927 = sum of:
     3.3376927 = product of:
       5.006539 = sum of:
         1.7543421 = weight(author_txt:bornmann in 2261) [ClassicSimilarity], result of:
           1.7543421 = score(doc=2261,freq=1.0), product of:
             0.4635137 = queryWeight, product of:
               7.5697527 = idf(docFreq=61, maxDocs=44218)
               0.061232343 = queryNorm
             3.7848763 = fieldWeight in 2261, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.5697527 = idf(docFreq=61, maxDocs=44218)
               0.5 = fieldNorm(doc=2261)
         3.2521968 = weight(author_txt:mutz in 2261) [ClassicSimilarity], result of:
           3.2521968 = score(doc=2261,freq=1.0), product of:
             0.69947296 = queryWeight, product of:
               1.2284406 = boost
               9.298992 = idf(docFreq=10, maxDocs=44218)
               0.061232343 = queryNorm
             4.649496 = fieldWeight in 2261, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.298992 = idf(docFreq=10, maxDocs=44218)
               0.5 = fieldNorm(doc=2261)
       0.6666667 = coord(2/3)
   

Similar documents (content)

1. Daniel, H.-D.: Guardians of science : fairness and reliabiblity of peer review (1993) 0.22

   0.21596153 = sum of:
     0.21596153 = product of:
       0.89983976 = sum of:
         0.04955545 = weight(abstract_txt:review in 1713) [ClassicSimilarity], result of:
           0.04955545 = score(doc=1713,freq=4.0), product of:
             0.0652786 = queryWeight, product of:
               1.0486724 = boost
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.0128123965 = queryNorm
             0.7591378 = fieldWeight in 1713, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.078125 = fieldNorm(doc=1713)
         0.042088874 = weight(abstract_txt:decisions in 1713) [ClassicSimilarity], result of:
           0.042088874 = score(doc=1713,freq=1.0), product of:
             0.09293402 = queryWeight, product of:
               1.2512435 = boost
               5.79699 = idf(docFreq=364, maxDocs=44218)
               0.0128123965 = queryNorm
             0.45288983 = fieldWeight in 1713, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.79699 = idf(docFreq=364, maxDocs=44218)
               0.078125 = fieldNorm(doc=1713)
         0.029258274 = weight(abstract_txt:evaluation in 1713) [ClassicSimilarity], result of:
           0.029258274 = score(doc=1713,freq=1.0), product of:
             0.08348198 = queryWeight, product of:
               1.4524342 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0128123965 = queryNorm
             0.35047412 = fieldWeight in 1713, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.078125 = fieldNorm(doc=1713)
         0.29644698 = weight(abstract_txt:peer in 1713) [ClassicSimilarity], result of:
           0.29644698 = score(doc=1713,freq=4.0), product of:
             0.29195857 = queryWeight, product of:
               3.5065906 = boost
               6.49839 = idf(docFreq=180, maxDocs=44218)
               0.0128123965 = queryNorm
             1.0153735 = fieldWeight in 1713, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.49839 = idf(docFreq=180, maxDocs=44218)
               0.078125 = fieldNorm(doc=1713)
         0.168656 = weight(abstract_txt:validity in 1713) [ClassicSimilarity], result of:
           0.168656 = score(doc=1713,freq=1.0), product of:
             0.31820798 = queryWeight, product of:
               3.6608336 = boost
               6.784232 = idf(docFreq=135, maxDocs=44218)
               0.0128123965 = queryNorm
             0.53001815 = fieldWeight in 1713, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.784232 = idf(docFreq=135, maxDocs=44218)
               0.078125 = fieldNorm(doc=1713)
         0.31383416 = weight(abstract_txt:fairness in 1713) [ClassicSimilarity], result of:
           0.31383416 = score(doc=1713,freq=1.0), product of:
             0.4468962 = queryWeight, product of:
               3.8803668 = boost
               8.988837 = idf(docFreq=14, maxDocs=44218)
               0.0128123965 = queryNorm
             0.7022529 = fieldWeight in 1713, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.988837 = idf(docFreq=14, maxDocs=44218)
               0.078125 = fieldNorm(doc=1713)
       0.24 = coord(6/25)
   

2. Lee, C.J.; Sugimoto, C.R.; Zhang, G.; Cronin, B.: Bias in peer review (2013) 0.16

   0.16469122 = sum of:
     0.16469122 = product of:
       0.8234561 = sum of:
         0.05244456 = weight(abstract_txt:review in 525) [ClassicSimilarity], result of:
           0.05244456 = score(doc=525,freq=7.0), product of:
             0.0652786 = queryWeight, product of:
               1.0486724 = boost
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.0128123965 = queryNorm
             0.8033959 = fieldWeight in 525, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.0625 = fieldNorm(doc=525)
         0.030788697 = weight(abstract_txt:research in 525) [ClassicSimilarity], result of:
           0.030788697 = score(doc=525,freq=5.0), product of:
             0.06948975 = queryWeight, product of:
               1.7107432 = boost
               3.170338 = idf(docFreq=5046, maxDocs=44218)
               0.0128123965 = queryNorm
             0.4430682 = fieldWeight in 525, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.170338 = idf(docFreq=5046, maxDocs=44218)
               0.0625 = fieldNorm(doc=525)
         0.09321097 = weight(abstract_txt:funding in 525) [ClassicSimilarity], result of:
           0.09321097 = score(doc=525,freq=1.0), product of:
             0.23084718 = queryWeight, product of:
               2.7888923 = boost
               6.4604454 = idf(docFreq=187, maxDocs=44218)
               0.0128123965 = queryNorm
             0.40377784 = fieldWeight in 525, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.4604454 = idf(docFreq=187, maxDocs=44218)
               0.0625 = fieldNorm(doc=525)
         0.3565543 = weight(abstract_txt:bias in 525) [ClassicSimilarity], result of:
           0.3565543 = score(doc=525,freq=9.0), product of:
             0.2714456 = queryWeight, product of:
               3.0242023 = boost
               7.0055394 = idf(docFreq=108, maxDocs=44218)
               0.0128123965 = queryNorm
             1.3135387 = fieldWeight in 525, product of:
               3.0 = tf(freq=9.0), with freq of:
                 9.0 = termFreq=9.0
               7.0055394 = idf(docFreq=108, maxDocs=44218)
               0.0625 = fieldNorm(doc=525)
         0.29045755 = weight(abstract_txt:peer in 525) [ClassicSimilarity], result of:
           0.29045755 = score(doc=525,freq=6.0), product of:
             0.29195857 = queryWeight, product of:
               3.5065906 = boost
               6.49839 = idf(docFreq=180, maxDocs=44218)
               0.0128123965 = queryNorm
             0.9948588 = fieldWeight in 525, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               6.49839 = idf(docFreq=180, maxDocs=44218)
               0.0625 = fieldNorm(doc=525)
       0.2 = coord(5/25)
   

3. Zhao, Y.W.; Chi. C.-H.; Heuvel, W.J. van den: Imperfect referees : reducing the impact of multiple biases in peer review (2015) 0.16

   0.15936017 = sum of:
     0.15936017 = product of:
       0.66400075 = sum of:
         0.03964436 = weight(abstract_txt:review in 2271) [ClassicSimilarity], result of:
           0.03964436 = score(doc=2271,freq=4.0), product of:
             0.0652786 = queryWeight, product of:
               1.0486724 = boost
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.0128123965 = queryNorm
             0.60731024 = fieldWeight in 2271, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.0625 = fieldNorm(doc=2271)
         0.025853679 = weight(abstract_txt:real in 2271) [ClassicSimilarity], result of:
           0.025853679 = score(doc=2271,freq=1.0), product of:
             0.07792643 = queryWeight, product of:
               1.1457684 = boost
               5.308326 = idf(docFreq=594, maxDocs=44218)
               0.0128123965 = queryNorm
             0.33177036 = fieldWeight in 2271, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.308326 = idf(docFreq=594, maxDocs=44218)
               0.0625 = fieldNorm(doc=2271)
         0.113589235 = weight(abstract_txt:biases in 2271) [ClassicSimilarity], result of:
           0.113589235 = score(doc=2271,freq=2.0), product of:
             0.16591462 = queryWeight, product of:
               1.6718479 = boost
               7.7456436 = idf(docFreq=51, maxDocs=44218)
               0.0128123965 = queryNorm
             0.6846246 = fieldWeight in 2271, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.7456436 = idf(docFreq=51, maxDocs=44218)
               0.0625 = fieldNorm(doc=2271)
         0.013769124 = weight(abstract_txt:research in 2271) [ClassicSimilarity], result of:
           0.013769124 = score(doc=2271,freq=1.0), product of:
             0.06948975 = queryWeight, product of:
               1.7107432 = boost
               3.170338 = idf(docFreq=5046, maxDocs=44218)
               0.0128123965 = queryNorm
             0.19814612 = fieldWeight in 2271, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.170338 = idf(docFreq=5046, maxDocs=44218)
               0.0625 = fieldNorm(doc=2271)
         0.2657599 = weight(abstract_txt:bias in 2271) [ClassicSimilarity], result of:
           0.2657599 = score(doc=2271,freq=5.0), product of:
             0.2714456 = queryWeight, product of:
               3.0242023 = boost
               7.0055394 = idf(docFreq=108, maxDocs=44218)
               0.0128123965 = queryNorm
             0.9790539 = fieldWeight in 2271, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               7.0055394 = idf(docFreq=108, maxDocs=44218)
               0.0625 = fieldNorm(doc=2271)
         0.2053845 = weight(abstract_txt:peer in 2271) [ClassicSimilarity], result of:
           0.2053845 = score(doc=2271,freq=3.0), product of:
             0.29195857 = queryWeight, product of:
               3.5065906 = boost
               6.49839 = idf(docFreq=180, maxDocs=44218)
               0.0128123965 = queryNorm
             0.70347136 = fieldWeight in 2271, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.49839 = idf(docFreq=180, maxDocs=44218)
               0.0625 = fieldNorm(doc=2271)
       0.24 = coord(6/25)
   

4. Abramo, G.; D'Angelo, C.A.: ¬A decision support system for public research organizations participating in national research assessment exercises (2009) 0.15

   0.15342568 = sum of:
     0.15342568 = product of:
       0.47945526 = sum of:
         0.028032795 = weight(abstract_txt:review in 3123) [ClassicSimilarity], result of:
           0.028032795 = score(doc=3123,freq=2.0), product of:
             0.0652786 = queryWeight, product of:
               1.0486724 = boost
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.0128123965 = queryNorm
             0.42943317 = fieldWeight in 3123, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.0625 = fieldNorm(doc=3123)
         0.03656262 = weight(abstract_txt:real in 3123) [ClassicSimilarity], result of:
           0.03656262 = score(doc=3123,freq=2.0), product of:
             0.07792643 = queryWeight, product of:
               1.1457684 = boost
               5.308326 = idf(docFreq=594, maxDocs=44218)
               0.0128123965 = queryNorm
             0.46919414 = fieldWeight in 3123, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.308326 = idf(docFreq=594, maxDocs=44218)
               0.0625 = fieldNorm(doc=3123)
         0.019263066 = weight(abstract_txt:approach in 3123) [ClassicSimilarity], result of:
           0.019263066 = score(doc=3123,freq=2.0), product of:
             0.05818896 = queryWeight, product of:
               1.2126079 = boost
               3.745328 = idf(docFreq=2839, maxDocs=44218)
               0.0128123965 = queryNorm
             0.33104333 = fieldWeight in 3123, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.745328 = idf(docFreq=2839, maxDocs=44218)
               0.0625 = fieldNorm(doc=3123)
         0.012837889 = weight(abstract_txt:data in 3123) [ClassicSimilarity], result of:
           0.012837889 = score(doc=3123,freq=1.0), product of:
             0.06156619 = queryWeight, product of:
               1.440259 = boost
               3.3363478 = idf(docFreq=4274, maxDocs=44218)
               0.0128123965 = queryNorm
             0.20852174 = fieldWeight in 3123, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3363478 = idf(docFreq=4274, maxDocs=44218)
               0.0625 = fieldNorm(doc=3123)
         0.04681324 = weight(abstract_txt:evaluation in 3123) [ClassicSimilarity], result of:
           0.04681324 = score(doc=3123,freq=4.0), product of:
             0.08348198 = queryWeight, product of:
               1.4524342 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0128123965 = queryNorm
             0.5607586 = fieldWeight in 3123, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=3123)
         0.036429673 = weight(abstract_txt:research in 3123) [ClassicSimilarity], result of:
           0.036429673 = score(doc=3123,freq=7.0), product of:
             0.06948975 = queryWeight, product of:
               1.7107432 = boost
               3.170338 = idf(docFreq=5046, maxDocs=44218)
               0.0128123965 = queryNorm
             0.5242453 = fieldWeight in 3123, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               3.170338 = idf(docFreq=5046, maxDocs=44218)
               0.0625 = fieldNorm(doc=3123)
         0.13182023 = weight(abstract_txt:funding in 3123) [ClassicSimilarity], result of:
           0.13182023 = score(doc=3123,freq=2.0), product of:
             0.23084718 = queryWeight, product of:
               2.7888923 = boost
               6.4604454 = idf(docFreq=187, maxDocs=44218)
               0.0128123965 = queryNorm
             0.5710281 = fieldWeight in 3123, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.4604454 = idf(docFreq=187, maxDocs=44218)
               0.0625 = fieldNorm(doc=3123)
         0.16769573 = weight(abstract_txt:peer in 3123) [ClassicSimilarity], result of:
           0.16769573 = score(doc=3123,freq=2.0), product of:
             0.29195857 = queryWeight, product of:
               3.5065906 = boost
               6.49839 = idf(docFreq=180, maxDocs=44218)
               0.0128123965 = queryNorm
             0.57438195 = fieldWeight in 3123, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.49839 = idf(docFreq=180, maxDocs=44218)
               0.0625 = fieldNorm(doc=3123)
       0.32 = coord(8/25)
   

5. Bornmann, L.; Schier, H.; Marx, W.; Daniel, H.-D.: Is interactive open access publishing able to identify high-impact submissions? : a study on the predictive validity of Atmospheric Chemistry and Physics by using percentile rank classes (2011) 0.14

   0.14230248 = sum of:
     0.14230248 = product of:
       0.59292704 = sum of:
         0.01982218 = weight(abstract_txt:review in 4132) [ClassicSimilarity], result of:
           0.01982218 = score(doc=4132,freq=1.0), product of:
             0.0652786 = queryWeight, product of:
               1.0486724 = boost
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.0128123965 = queryNorm
             0.30365512 = fieldWeight in 4132, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.858482 = idf(docFreq=932, maxDocs=44218)
               0.0625 = fieldNorm(doc=4132)
         0.04761812 = weight(abstract_txt:decisions in 4132) [ClassicSimilarity], result of:
           0.04761812 = score(doc=4132,freq=2.0), product of:
             0.09293402 = queryWeight, product of:
               1.2512435 = boost
               5.79699 = idf(docFreq=364, maxDocs=44218)
               0.0128123965 = queryNorm
             0.5123863 = fieldWeight in 4132, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.79699 = idf(docFreq=364, maxDocs=44218)
               0.0625 = fieldNorm(doc=4132)
         0.019472482 = weight(abstract_txt:research in 4132) [ClassicSimilarity], result of:
           0.019472482 = score(doc=4132,freq=2.0), product of:
             0.06948975 = queryWeight, product of:
               1.7107432 = boost
               3.170338 = idf(docFreq=5046, maxDocs=44218)
               0.0128123965 = queryNorm
             0.28022093 = fieldWeight in 4132, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               3.170338 = idf(docFreq=5046, maxDocs=44218)
               0.0625 = fieldNorm(doc=4132)
         0.1537389 = weight(abstract_txt:predictive in 4132) [ClassicSimilarity], result of:
           0.1537389 = score(doc=4132,freq=3.0), product of:
             0.17734487 = queryWeight, product of:
               1.7284777 = boost
               8.008008 = idf(docFreq=39, maxDocs=44218)
               0.0128123965 = queryNorm
             0.8668923 = fieldWeight in 4132, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               8.008008 = idf(docFreq=39, maxDocs=44218)
               0.0625 = fieldNorm(doc=4132)
         0.11857879 = weight(abstract_txt:peer in 4132) [ClassicSimilarity], result of:
           0.11857879 = score(doc=4132,freq=1.0), product of:
             0.29195857 = queryWeight, product of:
               3.5065906 = boost
               6.49839 = idf(docFreq=180, maxDocs=44218)
               0.0128123965 = queryNorm
             0.4061494 = fieldWeight in 4132, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.49839 = idf(docFreq=180, maxDocs=44218)
               0.0625 = fieldNorm(doc=4132)
         0.2336966 = weight(abstract_txt:validity in 4132) [ClassicSimilarity], result of:
           0.2336966 = score(doc=4132,freq=3.0), product of:
             0.31820798 = queryWeight, product of:
               3.6608336 = boost
               6.784232 = idf(docFreq=135, maxDocs=44218)
               0.0128123965 = queryNorm
             0.73441464 = fieldWeight in 4132, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.784232 = idf(docFreq=135, maxDocs=44218)
               0.0625 = fieldNorm(doc=4132)
       0.24 = coord(6/25)