Document (#40921)

Author

Kholghi, M.
Vine, L.D.
Sitbon, L.
Zuccon, G.
Nguyen, A.

Title

Clinical information extraction using small data : an active learning approach based on sequence representations and word embeddings

Source

Journal of the Association for Information Science and Technology. 68(2017) no.11, S.2543-2556

Year

2017

Abstract

This article demonstrates the benefits of using sequence representations based on word embeddings to inform the seed selection and sample selection processes in an active learning pipeline for clinical information extraction. Seed selection refers to choosing an initial sample set to label to form an initial learning model. Sample selection refers to selecting informative samples to update the model at each iteration of the active learning process. Compared to supervised machine learning approaches, active learning offers the opportunity to build statistical classifiers with a reduced amount of training samples that require manual annotation. Reducing the manual annotation effort can support automating the clinical information extraction process. This is particularly beneficial in the clinical domain, where manual annotation is a time-consuming and costly task, as it requires extensive labor from clinical experts. Our empirical findings demonstrate that (a) using sequence representations along with the length of sequence for seed selection shows potential towards more effective initial models, and (b) using sequence representations for sample selection leads to significantly lower manual annotation efforts, with up to 3% and 6% fewer tokens and concepts requiring annotation, respectively, compared to state-of-the-art query strategies.

Content

Vgl.: http://onlinelibrary.wiley.com/doi/10.1002/asi.23936/full.

Footnote

Beitrag in einem Special issue on biomedical information retrieval.

Field

Medizin

Similar documents (author)

1. Koopman, B.; Zuccon, G.; Bruza, P.; Nguyen, A.: What makes an effective clinical query and querier? (2017) 4.22

   4.2243533 = sum of:
     4.2243533 = sum of:
       1.8832294 = weight(author_txt:nguyen in 3922) [ClassicSimilarity], result of:
         1.8832294 = score(doc=3922,freq=1.0), product of:
           0.65418434 = queryWeight, product of:
             9.211981 = idf(docFreq=11, maxDocs=44218)
             0.071014516 = queryNorm
           2.8787441 = fieldWeight in 3922, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.211981 = idf(docFreq=11, maxDocs=44218)
             0.3125 = fieldNorm(doc=3922)
       2.3411238 = weight(author_txt:zuccon in 3922) [ClassicSimilarity], result of:
         2.3411238 = score(doc=3922,freq=1.0), product of:
           0.75633514 = queryWeight, product of:
             1.0752441 = boost
             9.905128 = idf(docFreq=5, maxDocs=44218)
             0.071014516 = queryNorm
           3.0953524 = fieldWeight in 3922, product of:
             1.0 = tf(freq=1.0), with freq of:
               1.0 = termFreq=1.0
             9.905128 = idf(docFreq=5, maxDocs=44218)
             0.3125 = fieldNorm(doc=3922)
   

2. Nguyen, P.H.P.; Kaneiwa, K.; Nguyen, M.-Q.: Ontology inferencing rules and operations in conceptual structure theory (2010) 1.60

   1.5979729 = sum of:
     1.5979729 = product of:
       3.1959457 = sum of:
         3.1959457 = weight(author_txt:nguyen in 4421) [ClassicSimilarity], result of:
           3.1959457 = score(doc=4421,freq=2.0), product of:
             0.65418434 = queryWeight, product of:
               9.211981 = idf(docFreq=11, maxDocs=44218)
               0.071014516 = queryNorm
             4.8853903 = fieldWeight in 4421, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               9.211981 = idf(docFreq=11, maxDocs=44218)
               0.375 = fieldNorm(doc=4421)
       0.5 = coord(1/2)
   

3. Zhang, J.; Nguyen, T.: WebStar: a visualization model for hyperlink structures (2005) 1.51

   1.5065835 = sum of:
     1.5065835 = product of:
       3.013167 = sum of:
         3.013167 = weight(author_txt:nguyen in 1056) [ClassicSimilarity], result of:
           3.013167 = score(doc=1056,freq=1.0), product of:
             0.65418434 = queryWeight, product of:
               9.211981 = idf(docFreq=11, maxDocs=44218)
               0.071014516 = queryNorm
             4.6059904 = fieldWeight in 1056, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.211981 = idf(docFreq=11, maxDocs=44218)
               0.5 = fieldNorm(doc=1056)
       0.5 = coord(1/2)
   

4. Nguyen-Kim, M.T.: ¬Die kleinste gemeinsame Wirklichkeit : wahr, falsch, plausibel? : die größten Streitfragen wissenschaftlich geprüft (2021) 1.51

   1.5065835 = sum of:
     1.5065835 = product of:
       3.013167 = sum of:
         3.013167 = weight(author_txt:nguyen in 235) [ClassicSimilarity], result of:
           3.013167 = score(doc=235,freq=1.0), product of:
             0.65418434 = queryWeight, product of:
               9.211981 = idf(docFreq=11, maxDocs=44218)
               0.071014516 = queryNorm
             4.6059904 = fieldWeight in 235, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.211981 = idf(docFreq=11, maxDocs=44218)
               0.5 = fieldNorm(doc=235)
       0.5 = coord(1/2)
   

5. Nguyen, S.-H.; Chowdhury, G.: Interpreting the knowledge map of digital library research (1990-2010) (2013) 1.32

   1.3182604 = sum of:
     1.3182604 = product of:
       2.6365209 = sum of:
         2.6365209 = weight(author_txt:nguyen in 958) [ClassicSimilarity], result of:
           2.6365209 = score(doc=958,freq=1.0), product of:
             0.65418434 = queryWeight, product of:
               9.211981 = idf(docFreq=11, maxDocs=44218)
               0.071014516 = queryNorm
             4.0302415 = fieldWeight in 958, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               9.211981 = idf(docFreq=11, maxDocs=44218)
               0.4375 = fieldNorm(doc=958)
       0.5 = coord(1/2)
   

Similar documents (content)

1. Li, J.; Zhang, Z.; Li, X.; Chen, H.: Kernel-based learning for biomedical relation extraction (2008) 0.14

   0.13645442 = sum of:
     0.13645442 = product of:
       0.56856006 = sum of:
         0.015036537 = weight(abstract_txt:process in 1611) [ClassicSimilarity], result of:
           0.015036537 = score(doc=1611,freq=1.0), product of:
             0.047511015 = queryWeight, product of:
               1.0460104 = boost
               4.0510116 = idf(docFreq=2091, maxDocs=44218)
               0.011212303 = queryNorm
             0.3164853 = fieldWeight in 1611, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.0510116 = idf(docFreq=2091, maxDocs=44218)
               0.078125 = fieldNorm(doc=1611)
         0.03632042 = weight(abstract_txt:word in 1611) [ClassicSimilarity], result of:
           0.03632042 = score(doc=1611,freq=1.0), product of:
             0.0855322 = queryWeight, product of:
               1.4034717 = boost
               5.4353957 = idf(docFreq=523, maxDocs=44218)
               0.011212303 = queryNorm
             0.4246403 = fieldWeight in 1611, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4353957 = idf(docFreq=523, maxDocs=44218)
               0.078125 = fieldNorm(doc=1611)
         0.1394789 = weight(abstract_txt:extraction in 1611) [ClassicSimilarity], result of:
           0.1394789 = score(doc=1611,freq=3.0), product of:
             0.16647848 = queryWeight, product of:
               2.3980792 = boost
               6.1915555 = idf(docFreq=245, maxDocs=44218)
               0.011212303 = queryNorm
             0.83781946 = fieldWeight in 1611, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.1915555 = idf(docFreq=245, maxDocs=44218)
               0.078125 = fieldNorm(doc=1611)
         0.09532087 = weight(abstract_txt:manual in 1611) [ClassicSimilarity], result of:
           0.09532087 = score(doc=1611,freq=1.0), product of:
             0.20503646 = queryWeight, product of:
               3.073048 = boost
               5.950684 = idf(docFreq=312, maxDocs=44218)
               0.011212303 = queryNorm
             0.4648972 = fieldWeight in 1611, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.950684 = idf(docFreq=312, maxDocs=44218)
               0.078125 = fieldNorm(doc=1611)
         0.10289965 = weight(abstract_txt:learning in 1611) [ClassicSimilarity], result of:
           0.10289965 = score(doc=1611,freq=2.0), product of:
             0.19603582 = queryWeight, product of:
               3.6801639 = boost
               4.750873 = idf(docFreq=1038, maxDocs=44218)
               0.011212303 = queryNorm
             0.5249023 = fieldWeight in 1611, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.750873 = idf(docFreq=1038, maxDocs=44218)
               0.078125 = fieldNorm(doc=1611)
         0.1795037 = weight(abstract_txt:sequence in 1611) [ClassicSimilarity], result of:
           0.1795037 = score(doc=1611,freq=1.0), product of:
             0.33681494 = queryWeight, product of:
               4.403566 = boost
               6.82169 = idf(docFreq=130, maxDocs=44218)
               0.011212303 = queryNorm
             0.53294456 = fieldWeight in 1611, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.82169 = idf(docFreq=130, maxDocs=44218)
               0.078125 = fieldNorm(doc=1611)
       0.24 = coord(6/25)
   

2. Du, C.; Cohoon, J.; Lopez, P.; Howison, J.: Softcite dataset : a dataset of software mentions in biomedical and economic research publications (2021) 0.12

   0.117538236 = sum of:
     0.117538236 = product of:
       0.5876912 = sum of:
         0.018043844 = weight(abstract_txt:process in 262) [ClassicSimilarity], result of:
           0.018043844 = score(doc=262,freq=1.0), product of:
             0.047511015 = queryWeight, product of:
               1.0460104 = boost
               4.0510116 = idf(docFreq=2091, maxDocs=44218)
               0.011212303 = queryNorm
             0.37978232 = fieldWeight in 262, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.0510116 = idf(docFreq=2091, maxDocs=44218)
               0.09375 = fieldNorm(doc=262)
         0.09663383 = weight(abstract_txt:extraction in 262) [ClassicSimilarity], result of:
           0.09663383 = score(doc=262,freq=1.0), product of:
             0.16647848 = queryWeight, product of:
               2.3980792 = boost
               6.1915555 = idf(docFreq=245, maxDocs=44218)
               0.011212303 = queryNorm
             0.58045834 = fieldWeight in 262, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.1915555 = idf(docFreq=245, maxDocs=44218)
               0.09375 = fieldNorm(doc=262)
         0.11438505 = weight(abstract_txt:manual in 262) [ClassicSimilarity], result of:
           0.11438505 = score(doc=262,freq=1.0), product of:
             0.20503646 = queryWeight, product of:
               3.073048 = boost
               5.950684 = idf(docFreq=312, maxDocs=44218)
               0.011212303 = queryNorm
             0.55787665 = fieldWeight in 262, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.950684 = idf(docFreq=312, maxDocs=44218)
               0.09375 = fieldNorm(doc=262)
         0.12347957 = weight(abstract_txt:learning in 262) [ClassicSimilarity], result of:
           0.12347957 = score(doc=262,freq=2.0), product of:
             0.19603582 = queryWeight, product of:
               3.6801639 = boost
               4.750873 = idf(docFreq=1038, maxDocs=44218)
               0.011212303 = queryNorm
             0.6298827 = fieldWeight in 262, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.750873 = idf(docFreq=1038, maxDocs=44218)
               0.09375 = fieldNorm(doc=262)
         0.23514888 = weight(abstract_txt:annotation in 262) [ClassicSimilarity], result of:
           0.23514888 = score(doc=262,freq=1.0), product of:
             0.35709482 = queryWeight, product of:
               4.534199 = boost
               7.0240583 = idf(docFreq=106, maxDocs=44218)
               0.011212303 = queryNorm
             0.65850544 = fieldWeight in 262, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.0240583 = idf(docFreq=106, maxDocs=44218)
               0.09375 = fieldNorm(doc=262)
       0.2 = coord(5/25)
   

3. Billal, B.; Fonseca, A.; Sadat, F.; Lounis, H.: Semi-supervised learning and social media text analysis towards multi-labeling categorization (2017) 0.11

   0.11029769 = sum of:
     0.11029769 = product of:
       0.3939203 = sum of:
         0.010028679 = weight(abstract_txt:model in 4095) [ClassicSimilarity], result of:
           0.010028679 = score(doc=4095,freq=1.0), product of:
             0.046003714 = queryWeight, product of:
               1.0292841 = boost
               3.986234 = idf(docFreq=2231, maxDocs=44218)
               0.011212303 = queryNorm
             0.21799716 = fieldWeight in 4095, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.986234 = idf(docFreq=2231, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4095)
         0.02825865 = weight(abstract_txt:compared in 4095) [ClassicSimilarity], result of:
           0.02825865 = score(doc=4095,freq=2.0), product of:
             0.07284303 = queryWeight, product of:
               1.2951881 = boost
               5.0160327 = idf(docFreq=796, maxDocs=44218)
               0.011212303 = queryNorm
             0.38793898 = fieldWeight in 4095, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.0160327 = idf(docFreq=796, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4095)
         0.025424294 = weight(abstract_txt:word in 4095) [ClassicSimilarity], result of:
           0.025424294 = score(doc=4095,freq=1.0), product of:
             0.0855322 = queryWeight, product of:
               1.4034717 = boost
               5.4353957 = idf(docFreq=523, maxDocs=44218)
               0.011212303 = queryNorm
             0.2972482 = fieldWeight in 4095, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.4353957 = idf(docFreq=523, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4095)
         0.022779703 = weight(abstract_txt:using in 4095) [ClassicSimilarity], result of:
           0.022779703 = score(doc=4095,freq=3.0), product of:
             0.06944357 = queryWeight, product of:
               1.7884213 = boost
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.011212303 = queryNorm
             0.32803187 = fieldWeight in 4095, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.4631186 = idf(docFreq=3765, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4095)
         0.05636973 = weight(abstract_txt:extraction in 4095) [ClassicSimilarity], result of:
           0.05636973 = score(doc=4095,freq=1.0), product of:
             0.16647848 = queryWeight, product of:
               2.3980792 = boost
               6.1915555 = idf(docFreq=245, maxDocs=44218)
               0.011212303 = queryNorm
             0.3386007 = fieldWeight in 4095, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.1915555 = idf(docFreq=245, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4095)
         0.113889046 = weight(abstract_txt:learning in 4095) [ClassicSimilarity], result of:
           0.113889046 = score(doc=4095,freq=5.0), product of:
             0.19603582 = queryWeight, product of:
               3.6801639 = boost
               4.750873 = idf(docFreq=1038, maxDocs=44218)
               0.011212303 = queryNorm
             0.5809604 = fieldWeight in 4095, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               4.750873 = idf(docFreq=1038, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4095)
         0.13717018 = weight(abstract_txt:annotation in 4095) [ClassicSimilarity], result of:
           0.13717018 = score(doc=4095,freq=1.0), product of:
             0.35709482 = queryWeight, product of:
               4.534199 = boost
               7.0240583 = idf(docFreq=106, maxDocs=44218)
               0.011212303 = queryNorm
             0.38412818 = fieldWeight in 4095, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.0240583 = idf(docFreq=106, maxDocs=44218)
               0.0546875 = fieldNorm(doc=4095)
       0.28 = coord(7/25)
   

4. Wu, T.; Pottenger, W.M.: ¬A semi-supervised active learning algorithm for information extraction from textual data (2005) 0.10

   0.10359413 = sum of:
     0.10359413 = product of:
       0.6474633 = sum of:
         0.1288451 = weight(abstract_txt:extraction in 3237) [ClassicSimilarity], result of:
           0.1288451 = score(doc=3237,freq=4.0), product of:
             0.16647848 = queryWeight, product of:
               2.3980792 = boost
               6.1915555 = idf(docFreq=245, maxDocs=44218)
               0.011212303 = queryNorm
             0.77394444 = fieldWeight in 3237, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.1915555 = idf(docFreq=245, maxDocs=44218)
               0.0625 = fieldNorm(doc=3237)
         0.18365638 = weight(abstract_txt:active in 3237) [ClassicSimilarity], result of:
           0.18365638 = score(doc=3237,freq=4.0), product of:
             0.23207577 = queryWeight, product of:
               3.269405 = boost
               6.330911 = idf(docFreq=213, maxDocs=44218)
               0.011212303 = queryNorm
             0.7913639 = fieldWeight in 3237, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               6.330911 = idf(docFreq=213, maxDocs=44218)
               0.0625 = fieldNorm(doc=3237)
         0.17032243 = weight(abstract_txt:seed in 3237) [ClassicSimilarity], result of:
           0.17032243 = score(doc=3237,freq=1.0), product of:
             0.3183079 = queryWeight, product of:
               3.3159513 = boost
               8.561393 = idf(docFreq=22, maxDocs=44218)
               0.011212303 = queryNorm
             0.53508705 = fieldWeight in 3237, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.561393 = idf(docFreq=22, maxDocs=44218)
               0.0625 = fieldNorm(doc=3237)
         0.16463943 = weight(abstract_txt:learning in 3237) [ClassicSimilarity], result of:
           0.16463943 = score(doc=3237,freq=8.0), product of:
             0.19603582 = queryWeight, product of:
               3.6801639 = boost
               4.750873 = idf(docFreq=1038, maxDocs=44218)
               0.011212303 = queryNorm
             0.83984363 = fieldWeight in 3237, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               4.750873 = idf(docFreq=1038, maxDocs=44218)
               0.0625 = fieldNorm(doc=3237)
       0.16 = coord(4/25)
   

5. Jung, H.; Yi, E.; Kim, D.; Lee, G.G.: Information extraction with automatic knowledge expansion (2005) 0.10

   0.10185431 = sum of:
     0.10185431 = product of:
       0.63658947 = sum of:
         0.18221448 = weight(abstract_txt:extraction in 1008) [ClassicSimilarity], result of:
           0.18221448 = score(doc=1008,freq=8.0), product of:
             0.16647848 = queryWeight, product of:
               2.3980792 = boost
               6.1915555 = idf(docFreq=245, maxDocs=44218)
               0.011212303 = queryNorm
             1.0945227 = fieldWeight in 1008, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               6.1915555 = idf(docFreq=245, maxDocs=44218)
               0.0625 = fieldNorm(doc=1008)
         0.1540061 = weight(abstract_txt:learning in 1008) [ClassicSimilarity], result of:
           0.1540061 = score(doc=1008,freq=7.0), product of:
             0.19603582 = queryWeight, product of:
               3.6801639 = boost
               4.750873 = idf(docFreq=1038, maxDocs=44218)
               0.011212303 = queryNorm
             0.7856018 = fieldWeight in 1008, product of:
               2.6457512 = tf(freq=7.0), with freq of:
                 7.0 = termFreq=7.0
               4.750873 = idf(docFreq=1038, maxDocs=44218)
               0.0625 = fieldNorm(doc=1008)
         0.14360295 = weight(abstract_txt:sequence in 1008) [ClassicSimilarity], result of:
           0.14360295 = score(doc=1008,freq=1.0), product of:
             0.33681494 = queryWeight, product of:
               4.403566 = boost
               6.82169 = idf(docFreq=130, maxDocs=44218)
               0.011212303 = queryNorm
             0.42635563 = fieldWeight in 1008, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.82169 = idf(docFreq=130, maxDocs=44218)
               0.0625 = fieldNorm(doc=1008)
         0.15676592 = weight(abstract_txt:annotation in 1008) [ClassicSimilarity], result of:
           0.15676592 = score(doc=1008,freq=1.0), product of:
             0.35709482 = queryWeight, product of:
               4.534199 = boost
               7.0240583 = idf(docFreq=106, maxDocs=44218)
               0.011212303 = queryNorm
             0.43900365 = fieldWeight in 1008, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.0240583 = idf(docFreq=106, maxDocs=44218)
               0.0625 = fieldNorm(doc=1008)
       0.16 = coord(4/25)