Document (#33883)

Author

Kim, Y.H.
Kim, H.H.

Title

Development and validation of evaluation indicators for a consortium of institutional repositories : a case study of dcollection

Source

Journal of the American Society for Information Science and Technology. 59(2008) no.8, S.1282-1294

Year

2008

Abstract

This study develops evaluation indicators for a consortium of Korean institutional repositories called dCollection and validates the indicators against actual data from the participants of this consortium. The literature review reveals a conceptual framework for institutional repository (IR) evaluation with four categories and 19 items. In developing the initial framework, equal amounts of emphasis are put on the assessments of procedural achievement and actual performance to pinpoint the procedural weaknesses of each IR and to determine its customized solution. A Delphi method of three rounds with the help of IR librarians reveals a converging tendency pertaining to the measures of importance ascribed to the categories and items. Through a focus-group interview with middle- to top-level managers, 39 indicators derived from 19 items are identified as possessing relevancy, measurability, data availability, and differentiability. Validation of evaluation indicators employs actual evaluation data from 32 university IRs. Factor analysis shows a simpler structural pattern containing 12 factors than that of the structural pattern of the conceptual framework that contains 19 items. Correlation analysis using the factor scores identifies six key factors: Registration Rate, Archiving, Resource Allocation, System Performance, Multifunctionality, and Use Rate. The results from regression analyses suggest that two different approaches can be employed to promote the Use Rate factor. In the content-oriented approach, the Registration Rate factor is crucial while in the policy-oriented approach the Archiving factor assumes this role; however, the System Performance factor plays a mediating role for the key factors, thus forming a contingency for either approach.

Similar documents (content)

1. Wang, J.; Guan, J.: ¬The analysis and evaluation of knowledge efficiency in research groups (2005) 0.17

   0.16706516 = sum of:
     0.16706516 = product of:
       0.59666127 = sum of:
         0.03472123 = weight(abstract_txt:conceptual in 4238) [ClassicSimilarity], result of:
           0.03472123 = score(doc=4238,freq=1.0), product of:
             0.0908642 = queryWeight, product of:
               4.891165 = idf(docFreq=902, maxDocs=44218)
               0.01857721 = queryNorm
             0.38212225 = fieldWeight in 4238, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.891165 = idf(docFreq=902, maxDocs=44218)
               0.078125 = fieldNorm(doc=4238)
         0.016529642 = weight(abstract_txt:data in 4238) [ClassicSimilarity], result of:
           0.016529642 = score(doc=4238,freq=1.0), product of:
             0.06341647 = queryWeight, product of:
               1.0231758 = boost
               3.3363478 = idf(docFreq=4274, maxDocs=44218)
               0.01857721 = queryNorm
             0.26065218 = fieldWeight in 4238, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3363478 = idf(docFreq=4274, maxDocs=44218)
               0.078125 = fieldNorm(doc=4238)
         0.023384012 = weight(abstract_txt:approach in 4238) [ClassicSimilarity], result of:
           0.023384012 = score(doc=4238,freq=1.0), product of:
             0.079917 = queryWeight, product of:
               1.1486 = boost
               3.745328 = idf(docFreq=2839, maxDocs=44218)
               0.01857721 = queryNorm
             0.29260373 = fieldWeight in 4238, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.745328 = idf(docFreq=2839, maxDocs=44218)
               0.078125 = fieldNorm(doc=4238)
         0.05932765 = weight(abstract_txt:framework in 4238) [ClassicSimilarity], result of:
           0.05932765 = score(doc=4238,freq=2.0), product of:
             0.11799255 = queryWeight, product of:
               1.39565 = boost
               4.550903 = idf(docFreq=1268, maxDocs=44218)
               0.01857721 = queryNorm
             0.50280845 = fieldWeight in 4238, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.550903 = idf(docFreq=1268, maxDocs=44218)
               0.078125 = fieldNorm(doc=4238)
         0.054348223 = weight(abstract_txt:factors in 4238) [ClassicSimilarity], result of:
           0.054348223 = score(doc=4238,freq=1.0), product of:
             0.14022218 = queryWeight, product of:
               1.5214497 = boost
               4.9611073 = idf(docFreq=841, maxDocs=44218)
               0.01857721 = queryNorm
             0.3875865 = fieldWeight in 4238, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9611073 = idf(docFreq=841, maxDocs=44218)
               0.078125 = fieldNorm(doc=4238)
         0.16240641 = weight(abstract_txt:indicators in 4238) [ClassicSimilarity], result of:
           0.16240641 = score(doc=4238,freq=1.0), product of:
             0.34491456 = queryWeight, product of:
               3.0805578 = boost
               6.027006 = idf(docFreq=289, maxDocs=44218)
               0.01857721 = queryNorm
             0.47085986 = fieldWeight in 4238, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.027006 = idf(docFreq=289, maxDocs=44218)
               0.078125 = fieldNorm(doc=4238)
         0.2459441 = weight(abstract_txt:factor in 4238) [ClassicSimilarity], result of:
           0.2459441 = score(doc=4238,freq=2.0), product of:
             0.3836343 = queryWeight, product of:
               3.5589588 = boost
               5.8024845 = idf(docFreq=362, maxDocs=44218)
               0.01857721 = queryNorm
             0.64109 = fieldWeight in 4238, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.8024845 = idf(docFreq=362, maxDocs=44218)
               0.078125 = fieldNorm(doc=4238)
       0.28 = coord(7/25)
   

2. Pharo, N.; Järvelin, K.: ¬The SST method : a tool for analysing Web information search processes (2004) 0.15

   0.14877321 = sum of:
     0.14877321 = product of:
       0.4132589 = sum of:
         0.027776984 = weight(abstract_txt:conceptual in 2533) [ClassicSimilarity], result of:
           0.027776984 = score(doc=2533,freq=1.0), product of:
             0.0908642 = queryWeight, product of:
               4.891165 = idf(docFreq=902, maxDocs=44218)
               0.01857721 = queryNorm
             0.3056978 = fieldWeight in 2533, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.891165 = idf(docFreq=902, maxDocs=44218)
               0.0625 = fieldNorm(doc=2533)
         0.013223713 = weight(abstract_txt:data in 2533) [ClassicSimilarity], result of:
           0.013223713 = score(doc=2533,freq=1.0), product of:
             0.06341647 = queryWeight, product of:
               1.0231758 = boost
               3.3363478 = idf(docFreq=4274, maxDocs=44218)
               0.01857721 = queryNorm
             0.20852174 = fieldWeight in 2533, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3363478 = idf(docFreq=4274, maxDocs=44218)
               0.0625 = fieldNorm(doc=2533)
         0.04659979 = weight(abstract_txt:categories in 2533) [ClassicSimilarity], result of:
           0.04659979 = score(doc=2533,freq=2.0), product of:
             0.10182366 = queryWeight, product of:
               1.0585904 = boost
               5.17774 = idf(docFreq=677, maxDocs=44218)
               0.01857721 = queryNorm
             0.45765188 = fieldWeight in 2533, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.17774 = idf(docFreq=677, maxDocs=44218)
               0.0625 = fieldNorm(doc=2533)
         0.010023907 = weight(abstract_txt:from in 2533) [ClassicSimilarity], result of:
           0.010023907 = score(doc=2533,freq=1.0), product of:
             0.058027975 = queryWeight, product of:
               1.1301532 = boost
               2.7638826 = idf(docFreq=7577, maxDocs=44218)
               0.01857721 = queryNorm
             0.17274266 = fieldWeight in 2533, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.7638826 = idf(docFreq=7577, maxDocs=44218)
               0.0625 = fieldNorm(doc=2533)
         0.01870721 = weight(abstract_txt:approach in 2533) [ClassicSimilarity], result of:
           0.01870721 = score(doc=2533,freq=1.0), product of:
             0.079917 = queryWeight, product of:
               1.1486 = boost
               3.745328 = idf(docFreq=2839, maxDocs=44218)
               0.01857721 = queryNorm
             0.234083 = fieldWeight in 2533, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.745328 = idf(docFreq=2839, maxDocs=44218)
               0.0625 = fieldNorm(doc=2533)
         0.03356079 = weight(abstract_txt:framework in 2533) [ClassicSimilarity], result of:
           0.03356079 = score(doc=2533,freq=1.0), product of:
             0.11799255 = queryWeight, product of:
               1.39565 = boost
               4.550903 = idf(docFreq=1268, maxDocs=44218)
               0.01857721 = queryNorm
             0.28443143 = fieldWeight in 2533, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.550903 = idf(docFreq=1268, maxDocs=44218)
               0.0625 = fieldNorm(doc=2533)
         0.04347858 = weight(abstract_txt:factors in 2533) [ClassicSimilarity], result of:
           0.04347858 = score(doc=2533,freq=1.0), product of:
             0.14022218 = queryWeight, product of:
               1.5214497 = boost
               4.9611073 = idf(docFreq=841, maxDocs=44218)
               0.01857721 = queryNorm
             0.3100692 = fieldWeight in 2533, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9611073 = idf(docFreq=841, maxDocs=44218)
               0.0625 = fieldNorm(doc=2533)
         0.08076091 = weight(abstract_txt:actual in 2533) [ClassicSimilarity], result of:
           0.08076091 = score(doc=2533,freq=1.0), product of:
             0.2118852 = queryWeight, product of:
               1.8702495 = boost
               6.0984654 = idf(docFreq=269, maxDocs=44218)
               0.01857721 = queryNorm
             0.3811541 = fieldWeight in 2533, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0984654 = idf(docFreq=269, maxDocs=44218)
               0.0625 = fieldNorm(doc=2533)
         0.139127 = weight(abstract_txt:factor in 2533) [ClassicSimilarity], result of:
           0.139127 = score(doc=2533,freq=1.0), product of:
             0.3836343 = queryWeight, product of:
               3.5589588 = boost
               5.8024845 = idf(docFreq=362, maxDocs=44218)
               0.01857721 = queryNorm
             0.36265528 = fieldWeight in 2533, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.8024845 = idf(docFreq=362, maxDocs=44218)
               0.0625 = fieldNorm(doc=2533)
       0.36 = coord(9/25)
   

3. Tsagkias, M.; Larson, M.; Rijke, M. de: Predicting podcast preference : an analysis framework and its application (2010) 0.13

   0.13301773 = sum of:
     0.13301773 = product of:
       0.47506335 = sum of:
         0.011570749 = weight(abstract_txt:data in 3339) [ClassicSimilarity], result of:
           0.011570749 = score(doc=3339,freq=1.0), product of:
             0.06341647 = queryWeight, product of:
               1.0231758 = boost
               3.3363478 = idf(docFreq=4274, maxDocs=44218)
               0.01857721 = queryNorm
             0.18245652 = fieldWeight in 3339, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.3363478 = idf(docFreq=4274, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3339)
         0.028832147 = weight(abstract_txt:categories in 3339) [ClassicSimilarity], result of:
           0.028832147 = score(doc=3339,freq=1.0), product of:
             0.10182366 = queryWeight, product of:
               1.0585904 = boost
               5.17774 = idf(docFreq=677, maxDocs=44218)
               0.01857721 = queryNorm
             0.28315765 = fieldWeight in 3339, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.17774 = idf(docFreq=677, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3339)
         0.008770919 = weight(abstract_txt:from in 3339) [ClassicSimilarity], result of:
           0.008770919 = score(doc=3339,freq=1.0), product of:
             0.058027975 = queryWeight, product of:
               1.1301532 = boost
               2.7638826 = idf(docFreq=7577, maxDocs=44218)
               0.01857721 = queryNorm
             0.15114984 = fieldWeight in 3339, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.7638826 = idf(docFreq=7577, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3339)
         0.07769431 = weight(abstract_txt:framework in 3339) [ClassicSimilarity], result of:
           0.07769431 = score(doc=3339,freq=7.0), product of:
             0.11799255 = queryWeight, product of:
               1.39565 = boost
               4.550903 = idf(docFreq=1268, maxDocs=44218)
               0.01857721 = queryNorm
             0.65846795 = fieldWeight in 3339, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.550903 = idf(docFreq=1268, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3339)
         0.113244146 = weight(abstract_txt:validation in 3339) [ClassicSimilarity], result of:
           0.113244146 = score(doc=3339,freq=2.0), product of:
             0.20118605 = queryWeight, product of:
               1.4879986 = boost
               7.2780466 = idf(docFreq=82, maxDocs=44218)
               0.01857721 = queryNorm
             0.5628827 = fieldWeight in 3339, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.2780466 = idf(docFreq=82, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3339)
         0.03804376 = weight(abstract_txt:factors in 3339) [ClassicSimilarity], result of:
           0.03804376 = score(doc=3339,freq=1.0), product of:
             0.14022218 = queryWeight, product of:
               1.5214497 = boost
               4.9611073 = idf(docFreq=841, maxDocs=44218)
               0.01857721 = queryNorm
             0.27131057 = fieldWeight in 3339, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9611073 = idf(docFreq=841, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3339)
         0.19690731 = weight(abstract_txt:indicators in 3339) [ClassicSimilarity], result of:
           0.19690731 = score(doc=3339,freq=3.0), product of:
             0.34491456 = queryWeight, product of:
               3.0805578 = boost
               6.027006 = idf(docFreq=289, maxDocs=44218)
               0.01857721 = queryNorm
             0.5708872 = fieldWeight in 3339, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.027006 = idf(docFreq=289, maxDocs=44218)
               0.0546875 = fieldNorm(doc=3339)
       0.28 = coord(7/25)
   

4. Yeh, J.-F.; Wu, C.-H.; Chen, M.-J.: Ontology-based speech act identification in a bilingual dialog system using partial pattern trees (2008) 0.13

   0.13133511 = sum of:
     0.13133511 = product of:
       0.54722965 = sum of:
         0.010023907 = weight(abstract_txt:from in 1607) [ClassicSimilarity], result of:
           0.010023907 = score(doc=1607,freq=1.0), product of:
             0.058027975 = queryWeight, product of:
               1.1301532 = boost
               2.7638826 = idf(docFreq=7577, maxDocs=44218)
               0.01857721 = queryNorm
             0.17274266 = fieldWeight in 1607, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.7638826 = idf(docFreq=7577, maxDocs=44218)
               0.0625 = fieldNorm(doc=1607)
         0.08179704 = weight(abstract_txt:pattern in 1607) [ClassicSimilarity], result of:
           0.08179704 = score(doc=1607,freq=2.0), product of:
             0.14816688 = queryWeight, product of:
               1.2769655 = boost
               6.2458487 = idf(docFreq=232, maxDocs=44218)
               0.01857721 = queryNorm
             0.55206025 = fieldWeight in 1607, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               6.2458487 = idf(docFreq=232, maxDocs=44218)
               0.0625 = fieldNorm(doc=1607)
         0.061229575 = weight(abstract_txt:performance in 1607) [ClassicSimilarity], result of:
           0.061229575 = score(doc=1607,freq=3.0), product of:
             0.122151904 = queryWeight, product of:
               1.420036 = boost
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.01857721 = queryNorm
             0.50125766 = fieldWeight in 1607, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.63042 = idf(docFreq=1171, maxDocs=44218)
               0.0625 = fieldNorm(doc=1607)
         0.12947871 = weight(abstract_txt:registration in 1607) [ClassicSimilarity], result of:
           0.12947871 = score(doc=1607,freq=1.0), product of:
             0.25355273 = queryWeight, product of:
               1.6704664 = boost
               8.1705265 = idf(docFreq=33, maxDocs=44218)
               0.01857721 = queryNorm
             0.5106579 = fieldWeight in 1607, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.1705265 = idf(docFreq=33, maxDocs=44218)
               0.0625 = fieldNorm(doc=1607)
         0.07577065 = weight(abstract_txt:evaluation in 1607) [ClassicSimilarity], result of:
           0.07577065 = score(doc=1607,freq=2.0), product of:
             0.19109091 = queryWeight, product of:
               2.292945 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.01857721 = queryNorm
             0.3965162 = fieldWeight in 1607, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=1607)
         0.18892974 = weight(abstract_txt:rate in 1607) [ClassicSimilarity], result of:
           0.18892974 = score(doc=1607,freq=3.0), product of:
             0.28495252 = queryWeight, product of:
               2.5044067 = boost
               6.124733 = idf(docFreq=262, maxDocs=44218)
               0.01857721 = queryNorm
             0.6630218 = fieldWeight in 1607, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.124733 = idf(docFreq=262, maxDocs=44218)
               0.0625 = fieldNorm(doc=1607)
       0.24 = coord(6/25)
   

5. Coleman, A.: Assessing the value of a journal beyond the impact factor (2007) 0.12

   0.11798402 = sum of:
     0.11798402 = product of:
       0.4916001 = sum of:
         0.010023907 = weight(abstract_txt:from in 447) [ClassicSimilarity], result of:
           0.010023907 = score(doc=447,freq=1.0), product of:
             0.058027975 = queryWeight, product of:
               1.1301532 = boost
               2.7638826 = idf(docFreq=7577, maxDocs=44218)
               0.01857721 = queryNorm
             0.17274266 = fieldWeight in 447, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.7638826 = idf(docFreq=7577, maxDocs=44218)
               0.0625 = fieldNorm(doc=447)
         0.057839245 = weight(abstract_txt:pattern in 447) [ClassicSimilarity], result of:
           0.057839245 = score(doc=447,freq=1.0), product of:
             0.14816688 = queryWeight, product of:
               1.2769655 = boost
               6.2458487 = idf(docFreq=232, maxDocs=44218)
               0.01857721 = queryNorm
             0.39036554 = fieldWeight in 447, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.2458487 = idf(docFreq=232, maxDocs=44218)
               0.0625 = fieldNorm(doc=447)
         0.04347858 = weight(abstract_txt:factors in 447) [ClassicSimilarity], result of:
           0.04347858 = score(doc=447,freq=1.0), product of:
             0.14022218 = queryWeight, product of:
               1.5214497 = boost
               4.9611073 = idf(docFreq=841, maxDocs=44218)
               0.01857721 = queryNorm
             0.3100692 = fieldWeight in 447, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.9611073 = idf(docFreq=841, maxDocs=44218)
               0.0625 = fieldNorm(doc=447)
         0.053577933 = weight(abstract_txt:evaluation in 447) [ClassicSimilarity], result of:
           0.053577933 = score(doc=447,freq=1.0), product of:
             0.19109091 = queryWeight, product of:
               2.292945 = boost
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.01857721 = queryNorm
             0.2803793 = fieldWeight in 447, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.4860687 = idf(docFreq=1353, maxDocs=44218)
               0.0625 = fieldNorm(doc=447)
         0.12992513 = weight(abstract_txt:indicators in 447) [ClassicSimilarity], result of:
           0.12992513 = score(doc=447,freq=1.0), product of:
             0.34491456 = queryWeight, product of:
               3.0805578 = boost
               6.027006 = idf(docFreq=289, maxDocs=44218)
               0.01857721 = queryNorm
             0.37668788 = fieldWeight in 447, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.027006 = idf(docFreq=289, maxDocs=44218)
               0.0625 = fieldNorm(doc=447)
         0.19675529 = weight(abstract_txt:factor in 447) [ClassicSimilarity], result of:
           0.19675529 = score(doc=447,freq=2.0), product of:
             0.3836343 = queryWeight, product of:
               3.5589588 = boost
               5.8024845 = idf(docFreq=362, maxDocs=44218)
               0.01857721 = queryNorm
             0.512872 = fieldWeight in 447, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.8024845 = idf(docFreq=362, maxDocs=44218)
               0.0625 = fieldNorm(doc=447)
       0.24 = coord(6/25)