Document (#30049)

Author

Sun, Y.
Kantor, P.B.

Title

Cross-evaluation : a new model for information system evaluation

Source

Journal of the American Society for Information Science and Technology. 57(2006) no.5, S.614-628

Year

2006

Abstract

In this article, we introduce a new information system evaluation method and report on its application to a collaborative information seeking system, AntWorld. The key innovation of the new method is to use precisely the same group of users who work with the system as judges, a system we call Cross-Evaluation. In the new method, we also propose to assess the system at the level of task completion. The obvious potential limitation of this method is that individuals may be inclined to think more highly of the materials that they themselves have found and are almost certain to think more highly of their own work product than they do of the products built by others. The keys to neutralizing this problem are careful design and a corresponding analytical model based on analysis of variance. We model the several measures of task completion with a linear model of five effects, describing the users who interact with the system, the system used to finish the task, the task itself, the behavior of individuals as judges, and the selfjudgment bias. Our analytical method successfully isolates the effect of each variable. This approach provides a successful model to make concrete the "threerealities" paradigm, which calls for "real tasks," "real users," and "real systems."

Theme

Retrievalstudien

Similar documents (author)

1. Kantor, P.B.: ¬The Adaptive Network Library Interface : a historical overview and interim report (1993) 5.11

   5.106579 = sum of:
     5.106579 = weight(author_txt:kantor in 6976) [ClassicSimilarity], result of:
       5.106579 = fieldWeight in 6976, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.1705265 = idf(docFreq=33, maxDocs=44218)
         0.625 = fieldNorm(doc=6976)
   

2. Kantor, P.B.: ¬The logic of weighted queries (1981) 5.11

   5.106579 = sum of:
     5.106579 = weight(author_txt:kantor in 7622) [ClassicSimilarity], result of:
       5.106579 = fieldWeight in 7622, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.1705265 = idf(docFreq=33, maxDocs=44218)
         0.625 = fieldNorm(doc=7622)
   

3. Kantor, A.: Baby steps (1994) 5.11

   5.106579 = sum of:
     5.106579 = weight(author_txt:kantor in 7985) [ClassicSimilarity], result of:
       5.106579 = fieldWeight in 7985, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.1705265 = idf(docFreq=33, maxDocs=44218)
         0.625 = fieldNorm(doc=7985)
   

4. Kantor, P.B.: Information retrieval techniques (1994) 5.11

   5.106579 = sum of:
     5.106579 = weight(author_txt:kantor in 1056) [ClassicSimilarity], result of:
       5.106579 = fieldWeight in 1056, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.1705265 = idf(docFreq=33, maxDocs=44218)
         0.625 = fieldNorm(doc=1056)
   

5. Kantor, P.B.: ¬A model for stopping behavior of the users of on-line systems (1987) 5.11

   5.106579 = sum of:
     5.106579 = weight(author_txt:kantor in 3945) [ClassicSimilarity], result of:
       5.106579 = fieldWeight in 3945, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         8.1705265 = idf(docFreq=33, maxDocs=44218)
         0.625 = fieldNorm(doc=3945)
   

Similar documents (content)

1. Shah, C.; Hendahewa, C.; González-Ibáñez, R.: Rain or shine? : forecasting search process performance in exploratory search tasks (2016) 0.17

   0.17334385 = sum of:
     0.17334385 = product of:
       0.48151067 = sum of:
         0.017626531 = weight(abstract_txt:work in 3012) [ClassicSimilarity], result of:
           0.017626531 = score(doc=3012,freq=1.0), product of:
             0.074326515 = queryWeight, product of:
               1.0112896 = boost
               3.7943997 = idf(docFreq=2703, maxDocs=44218)
               0.019369802 = queryNorm
             0.23714998 = fieldWeight in 3012, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.7943997 = idf(docFreq=2703, maxDocs=44218)
               0.0625 = fieldNorm(doc=3012)
         0.07858531 = weight(abstract_txt:limitation in 3012) [ClassicSimilarity], result of:
           0.07858531 = score(doc=3012,freq=1.0), product of:
             0.15980245 = queryWeight, product of:
               1.0485288 = boost
               7.8682456 = idf(docFreq=45, maxDocs=44218)
               0.019369802 = queryNorm
             0.49176535 = fieldWeight in 3012, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.8682456 = idf(docFreq=45, maxDocs=44218)
               0.0625 = fieldNorm(doc=3012)
         0.012822244 = weight(abstract_txt:this in 3012) [ClassicSimilarity], result of:
           0.012822244 = score(doc=3012,freq=2.0), product of:
             0.06011856 = queryWeight, product of:
               1.2862427 = boost
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.019369802 = queryNorm
             0.21328263 = fieldWeight in 3012, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.0625 = fieldNorm(doc=3012)
         0.022016715 = weight(abstract_txt:users in 3012) [ClassicSimilarity], result of:
           0.022016715 = score(doc=3012,freq=1.0), product of:
             0.09868049 = queryWeight, product of:
               1.4271345 = boost
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.019369802 = queryNorm
             0.22311112 = fieldWeight in 3012, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.0625 = fieldNorm(doc=3012)
         0.06223988 = weight(abstract_txt:individuals in 3012) [ClassicSimilarity], result of:
           0.06223988 = score(doc=3012,freq=1.0), product of:
             0.17235017 = queryWeight, product of:
               1.5399601 = boost
               5.777993 = idf(docFreq=371, maxDocs=44218)
               0.019369802 = queryNorm
             0.36112458 = fieldWeight in 3012, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.777993 = idf(docFreq=371, maxDocs=44218)
               0.0625 = fieldNorm(doc=3012)
         0.05825932 = weight(abstract_txt:evaluation in 3012) [ClassicSimilarity], result of:
           0.05825932 = score(doc=3012,freq=1.0), product of:
             0.20778753 = queryWeight, product of:
               2.3912687 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.019369802 = queryNorm
             0.2803793 = fieldWeight in 3012, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=3012)
         0.076446384 = weight(abstract_txt:task in 3012) [ClassicSimilarity], result of:
           0.076446384 = score(doc=3012,freq=1.0), product of:
             0.24904652 = queryWeight, product of:
               2.6179352 = boost
               4.9112997 = idf(docFreq=884, maxDocs=44218)
               0.019369802 = queryNorm
             0.30695623 = fieldWeight in 3012, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9112997 = idf(docFreq=884, maxDocs=44218)
               0.0625 = fieldNorm(doc=3012)
         0.10401719 = weight(abstract_txt:method in 3012) [ClassicSimilarity], result of:
           0.10401719 = score(doc=3012,freq=2.0), product of:
             0.26146042 = queryWeight, product of:
               2.9990013 = boost
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.019369802 = queryNorm
             0.3978315 = fieldWeight in 3012, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.0625 = fieldNorm(doc=3012)
         0.049497075 = weight(abstract_txt:system in 3012) [ClassicSimilarity], result of:
           0.049497075 = score(doc=3012,freq=1.0), product of:
             0.23484008 = queryWeight, product of:
               3.5951726 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.019369802 = queryNorm
             0.21076928 = fieldWeight in 3012, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.0625 = fieldNorm(doc=3012)
       0.36 = coord(9/25)
   

2. Ferro, N.; Silvello, G.: Toward an anatomy of IR system component performances (2018) 0.15

   0.15222053 = sum of:
     0.15222053 = product of:
       0.47568917 = sum of:
         0.017042771 = weight(abstract_txt:they in 4035) [ClassicSimilarity], result of:
           0.017042771 = score(doc=4035,freq=1.0), product of:
             0.07267628 = queryWeight, product of:
               3.7520406 = idf(docFreq=2820, maxDocs=44218)
               0.019369802 = queryNorm
             0.23450254 = fieldWeight in 4035, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.7520406 = idf(docFreq=2820, maxDocs=44218)
               0.0625 = fieldNorm(doc=4035)
         0.017626531 = weight(abstract_txt:work in 4035) [ClassicSimilarity], result of:
           0.017626531 = score(doc=4035,freq=1.0), product of:
             0.074326515 = queryWeight, product of:
               1.0112896 = boost
               3.7943997 = idf(docFreq=2703, maxDocs=44218)
               0.019369802 = queryNorm
             0.23714998 = fieldWeight in 4035, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.7943997 = idf(docFreq=2703, maxDocs=44218)
               0.0625 = fieldNorm(doc=4035)
         0.11113641 = weight(abstract_txt:variance in 4035) [ClassicSimilarity], result of:
           0.11113641 = score(doc=4035,freq=2.0), product of:
             0.15980245 = queryWeight, product of:
               1.0485288 = boost
               7.8682456 = idf(docFreq=45, maxDocs=44218)
               0.019369802 = queryNorm
             0.6954612 = fieldWeight in 4035, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.8682456 = idf(docFreq=45, maxDocs=44218)
               0.0625 = fieldNorm(doc=4035)
         0.015703978 = weight(abstract_txt:this in 4035) [ClassicSimilarity], result of:
           0.015703978 = score(doc=4035,freq=3.0), product of:
             0.06011856 = queryWeight, product of:
               1.2862427 = boost
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.019369802 = queryNorm
             0.2612168 = fieldWeight in 4035, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.0625 = fieldNorm(doc=4035)
         0.1009081 = weight(abstract_txt:evaluation in 4035) [ClassicSimilarity], result of:
           0.1009081 = score(doc=4035,freq=3.0), product of:
             0.20778753 = queryWeight, product of:
               2.3912687 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.019369802 = queryNorm
             0.48563117 = fieldWeight in 4035, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=4035)
         0.076446384 = weight(abstract_txt:task in 4035) [ClassicSimilarity], result of:
           0.076446384 = score(doc=4035,freq=1.0), product of:
             0.24904652 = queryWeight, product of:
               2.6179352 = boost
               4.9112997 = idf(docFreq=884, maxDocs=44218)
               0.019369802 = queryNorm
             0.30695623 = fieldWeight in 4035, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9112997 = idf(docFreq=884, maxDocs=44218)
               0.0625 = fieldNorm(doc=4035)
         0.051093545 = weight(abstract_txt:model in 4035) [ClassicSimilarity], result of:
           0.051093545 = score(doc=4035,freq=1.0), product of:
             0.20507996 = queryWeight, product of:
               2.656044 = boost
               3.986234 = idf(docFreq=2231, maxDocs=44218)
               0.019369802 = queryNorm
             0.24913962 = fieldWeight in 4035, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.986234 = idf(docFreq=2231, maxDocs=44218)
               0.0625 = fieldNorm(doc=4035)
         0.08573145 = weight(abstract_txt:system in 4035) [ClassicSimilarity], result of:
           0.08573145 = score(doc=4035,freq=3.0), product of:
             0.23484008 = queryWeight, product of:
               3.5951726 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.019369802 = queryNorm
             0.3650631 = fieldWeight in 4035, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.0625 = fieldNorm(doc=4035)
       0.32 = coord(8/25)
   

3. Sun, Y.; Kantor, P.B.; Morse, E.L.: Using cross-evaluation to evaluate interactive QA systems (2011) 0.15

   0.15055087 = sum of:
     0.15055087 = product of:
       0.5376817 = sum of:
         0.01962997 = weight(abstract_txt:this in 4744) [ClassicSimilarity], result of:
           0.01962997 = score(doc=4744,freq=3.0), product of:
             0.06011856 = queryWeight, product of:
               1.2862427 = boost
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.019369802 = queryNorm
             0.32652098 = fieldWeight in 4744, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.078125 = fieldNorm(doc=4744)
         0.027520893 = weight(abstract_txt:users in 4744) [ClassicSimilarity], result of:
           0.027520893 = score(doc=4744,freq=1.0), product of:
             0.09868049 = queryWeight, product of:
               1.4271345 = boost
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.019369802 = queryNorm
             0.2788889 = fieldWeight in 4744, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.078125 = fieldNorm(doc=4744)
         0.07086944 = weight(abstract_txt:cross in 4744) [ClassicSimilarity], result of:
           0.07086944 = score(doc=4744,freq=1.0), product of:
             0.16195655 = queryWeight, product of:
               1.4928044 = boost
               5.601063 = idf(docFreq=443, maxDocs=44218)
               0.019369802 = queryNorm
             0.43758303 = fieldWeight in 4744, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.601063 = idf(docFreq=443, maxDocs=44218)
               0.078125 = fieldNorm(doc=4744)
         0.102988906 = weight(abstract_txt:evaluation in 4744) [ClassicSimilarity], result of:
           0.102988906 = score(doc=4744,freq=2.0), product of:
             0.20778753 = queryWeight, product of:
               2.3912687 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.019369802 = queryNorm
             0.49564525 = fieldWeight in 4744, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.078125 = fieldNorm(doc=4744)
         0.09555798 = weight(abstract_txt:task in 4744) [ClassicSimilarity], result of:
           0.09555798 = score(doc=4744,freq=1.0), product of:
             0.24904652 = queryWeight, product of:
               2.6179352 = boost
               4.9112997 = idf(docFreq=884, maxDocs=44218)
               0.019369802 = queryNorm
             0.3836953 = fieldWeight in 4744, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9112997 = idf(docFreq=884, maxDocs=44218)
               0.078125 = fieldNorm(doc=4744)
         0.15924317 = weight(abstract_txt:method in 4744) [ClassicSimilarity], result of:
           0.15924317 = score(doc=4744,freq=3.0), product of:
             0.26146042 = queryWeight, product of:
               2.9990013 = boost
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.019369802 = queryNorm
             0.60905266 = fieldWeight in 4744, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.078125 = fieldNorm(doc=4744)
         0.061871342 = weight(abstract_txt:system in 4744) [ClassicSimilarity], result of:
           0.061871342 = score(doc=4744,freq=1.0), product of:
             0.23484008 = queryWeight, product of:
               3.5951726 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.019369802 = queryNorm
             0.2634616 = fieldWeight in 4744, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.078125 = fieldNorm(doc=4744)
       0.28 = coord(7/25)
   

4. Díaz, A.; García, A.; Gervás, P.: User-centred versus system-centred evaluation of a personalization system (2008) 0.15

   0.14996119 = sum of:
     0.14996119 = product of:
       0.4686287 = sum of:
         0.017042771 = weight(abstract_txt:they in 2094) [ClassicSimilarity], result of:
           0.017042771 = score(doc=2094,freq=1.0), product of:
             0.07267628 = queryWeight, product of:
               3.7520406 = idf(docFreq=2820, maxDocs=44218)
               0.019369802 = queryNorm
             0.23450254 = fieldWeight in 2094, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.7520406 = idf(docFreq=2820, maxDocs=44218)
               0.0625 = fieldNorm(doc=2094)
         0.017626531 = weight(abstract_txt:work in 2094) [ClassicSimilarity], result of:
           0.017626531 = score(doc=2094,freq=1.0), product of:
             0.074326515 = queryWeight, product of:
               1.0112896 = boost
               3.7943997 = idf(docFreq=2703, maxDocs=44218)
               0.019369802 = queryNorm
             0.23714998 = fieldWeight in 2094, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.7943997 = idf(docFreq=2703, maxDocs=44218)
               0.0625 = fieldNorm(doc=2094)
         0.009066696 = weight(abstract_txt:this in 2094) [ClassicSimilarity], result of:
           0.009066696 = score(doc=2094,freq=1.0), product of:
             0.06011856 = queryWeight, product of:
               1.2862427 = boost
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.019369802 = queryNorm
             0.1508136 = fieldWeight in 2094, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4130175 = idf(docFreq=10762, maxDocs=44218)
               0.0625 = fieldNorm(doc=2094)
         0.05392972 = weight(abstract_txt:users in 2094) [ClassicSimilarity], result of:
           0.05392972 = score(doc=2094,freq=6.0), product of:
             0.09868049 = queryWeight, product of:
               1.4271345 = boost
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.019369802 = queryNorm
             0.54650843 = fieldWeight in 2094, product of:
               2.4494898 = tf(freq=6.0), with freq of:
                 6.0 = termFreq=6.0
               3.569778 = idf(docFreq=3384, maxDocs=44218)
               0.0625 = fieldNorm(doc=2094)
         0.07239384 = weight(abstract_txt:real in 2094) [ClassicSimilarity], result of:
           0.07239384 = score(doc=2094,freq=1.0), product of:
             0.21820466 = queryWeight, product of:
               2.1221755 = boost
               5.308326 = idf(docFreq=594, maxDocs=44218)
               0.019369802 = queryNorm
             0.33177036 = fieldWeight in 2094, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.308326 = idf(docFreq=594, maxDocs=44218)
               0.0625 = fieldNorm(doc=2094)
         0.11651864 = weight(abstract_txt:evaluation in 2094) [ClassicSimilarity], result of:
           0.11651864 = score(doc=2094,freq=4.0), product of:
             0.20778753 = queryWeight, product of:
               2.3912687 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.019369802 = queryNorm
             0.5607586 = fieldWeight in 2094, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=2094)
         0.051093545 = weight(abstract_txt:model in 2094) [ClassicSimilarity], result of:
           0.051093545 = score(doc=2094,freq=1.0), product of:
             0.20507996 = queryWeight, product of:
               2.656044 = boost
               3.986234 = idf(docFreq=2231, maxDocs=44218)
               0.019369802 = queryNorm
             0.24913962 = fieldWeight in 2094, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.986234 = idf(docFreq=2231, maxDocs=44218)
               0.0625 = fieldNorm(doc=2094)
         0.13095693 = weight(abstract_txt:system in 2094) [ClassicSimilarity], result of:
           0.13095693 = score(doc=2094,freq=7.0), product of:
             0.23484008 = queryWeight, product of:
               3.5951726 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.019369802 = queryNorm
             0.55764306 = fieldWeight in 2094, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.0625 = fieldNorm(doc=2094)
       0.32 = coord(8/25)
   

5. Qiu, L.: Analytical searching vs. browsing in hypertext information retrieval systems (1993) 0.15

   0.14837629 = sum of:
     0.14837629 = product of:
       0.74188143 = sum of:
         0.11787796 = weight(abstract_txt:variance in 7416) [ClassicSimilarity], result of:
           0.11787796 = score(doc=7416,freq=1.0), product of:
             0.15980245 = queryWeight, product of:
               1.0485288 = boost
               7.8682456 = idf(docFreq=45, maxDocs=44218)
               0.019369802 = queryNorm
             0.737648 = fieldWeight in 7416, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.8682456 = idf(docFreq=45, maxDocs=44218)
               0.09375 = fieldNorm(doc=7416)
         0.23302971 = weight(abstract_txt:analytical in 7416) [ClassicSimilarity], result of:
           0.23302971 = score(doc=7416,freq=3.0), product of:
             0.21989 = queryWeight, product of:
               1.7394277 = boost
               6.5264034 = idf(docFreq=175, maxDocs=44218)
               0.019369802 = queryNorm
             1.0597558 = fieldWeight in 7416, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.5264034 = idf(docFreq=175, maxDocs=44218)
               0.09375 = fieldNorm(doc=7416)
         0.087388985 = weight(abstract_txt:evaluation in 7416) [ClassicSimilarity], result of:
           0.087388985 = score(doc=7416,freq=1.0), product of:
             0.20778753 = queryWeight, product of:
               2.3912687 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.019369802 = queryNorm
             0.42056894 = fieldWeight in 7416, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.09375 = fieldNorm(doc=7416)
         0.22933914 = weight(abstract_txt:task in 7416) [ClassicSimilarity], result of:
           0.22933914 = score(doc=7416,freq=4.0), product of:
             0.24904652 = queryWeight, product of:
               2.6179352 = boost
               4.9112997 = idf(docFreq=884, maxDocs=44218)
               0.019369802 = queryNorm
             0.9208687 = fieldWeight in 7416, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.9112997 = idf(docFreq=884, maxDocs=44218)
               0.09375 = fieldNorm(doc=7416)
         0.07424561 = weight(abstract_txt:system in 7416) [ClassicSimilarity], result of:
           0.07424561 = score(doc=7416,freq=1.0), product of:
             0.23484008 = queryWeight, product of:
               3.5951726 = boost
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.019369802 = queryNorm
             0.3161539 = fieldWeight in 7416, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3723085 = idf(docFreq=4123, maxDocs=44218)
               0.09375 = fieldNorm(doc=7416)
       0.2 = coord(5/25)