Document (#34083)

Author

Zhang, Y.
Xu, W.

Title

Fast exact maximum likelihood estimation for mixture of language model

Source

Information processing and management. 44(2008) no.3, S.1076-1085

Year

2008

Abstract

Language modeling is an effective and theoretically attractive probabilistic framework for text information retrieval. The basic idea of this approach is to estimate a language model of a given document (or document set), and then do retrieval or classification based on this model. A common language modeling approach assumes the data D is generated from a mixture of several language models. The core problem is to find the maximum likelihood estimation of one language model mixture, given the fixed mixture weights and the other language model mixture. The EM algorithm is usually used to find the solution. In this paper, we proof that an exact maximum likelihood estimation of the unknown mixture component exists and can be calculated using the new algorithm we proposed. We further improve the algorithm and provide an efficient algorithm of O(k) complexity to find the exact solution, where k is the number of words occurring at least once in data D. Furthermore, we proof the probabilities of many words are exactly zeros, and the MLE estimation is implemented as a feature selection technique explicitly.

Similar documents (author)

1. Zhang, M.; Zhang, Y.: Professional organizations in Twittersphere : an empirical study of U.S. library and information science professional organizations-related Tweets (2020) 4.54

   4.5423746 = sum of:
     4.5423746 = weight(author_txt:zhang in 5775) [ClassicSimilarity], result of:
       4.5423746 = fieldWeight in 5775, product of:
         1.4142135 = tf(freq=2.0), with freq of:
           2.0 = termFreq=2.0
         6.4238877 = idf(docFreq=194, maxDocs=44218)
         0.5 = fieldNorm(doc=5775)
   

2. Zhang, Y.; Zhang, C.: Enhancing keyphrase extraction from microblogs using human reading time (2021) 4.54

   4.5423746 = sum of:
     4.5423746 = weight(author_txt:zhang in 237) [ClassicSimilarity], result of:
       4.5423746 = fieldWeight in 237, product of:
         1.4142135 = tf(freq=2.0), with freq of:
           2.0 = termFreq=2.0
         6.4238877 = idf(docFreq=194, maxDocs=44218)
         0.5 = fieldNorm(doc=237)
   

3. Zhang, J.: TOFIR: A tool of facilitating information retrieval : introduce a visual retrieval model (2001) 4.01

   4.01493 = sum of:
     4.01493 = weight(author_txt:zhang in 7711) [ClassicSimilarity], result of:
       4.01493 = fieldWeight in 7711, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         6.4238877 = idf(docFreq=194, maxDocs=44218)
         0.625 = fieldNorm(doc=7711)
   

4. Zhang, A.: Multimedia file formats on the Internet : a beginner's guide for PC users (1995) 4.01

   4.01493 = sum of:
     4.01493 = weight(author_txt:zhang in 3212) [ClassicSimilarity], result of:
       4.01493 = fieldWeight in 3212, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         6.4238877 = idf(docFreq=194, maxDocs=44218)
         0.625 = fieldNorm(doc=3212)
   

5. Zhang, J.: ¬A representational analysis of relational information displays (1996) 4.01

   4.01493 = sum of:
     4.01493 = weight(author_txt:zhang in 6403) [ClassicSimilarity], result of:
       4.01493 = fieldWeight in 6403, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         6.4238877 = idf(docFreq=194, maxDocs=44218)
         0.625 = fieldNorm(doc=6403)
   

Similar documents (content)

1. Ponte, J.M.: Language models for relevance feedback (2000) 0.17

   0.17273457 = sum of:
     0.17273457 = product of:
       0.6169092 = sum of:
         0.028871376 = weight(abstract_txt:approach in 35) [ClassicSimilarity], result of:
           0.028871376 = score(doc=35,freq=5.0), product of:
             0.044126812 = queryWeight, product of:
               1.0342581 = boost
               3.745328 = idf(docFreq=2839, maxDocs=44218)
               0.0113915745 = queryNorm
             0.6542819 = fieldWeight in 35, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               3.745328 = idf(docFreq=2839, maxDocs=44218)
               0.078125 = fieldNorm(doc=35)
         0.02749107 = weight(abstract_txt:document in 35) [ClassicSimilarity], result of:
           0.02749107 = score(doc=35,freq=2.0), product of:
             0.057964936 = queryWeight, product of:
               1.1853875 = boost
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.0113915745 = queryNorm
             0.4742707 = fieldWeight in 35, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.078125 = fieldNorm(doc=35)
         0.09255795 = weight(abstract_txt:modeling in 35) [ClassicSimilarity], result of:
           0.09255795 = score(doc=35,freq=3.0), product of:
             0.11374955 = queryWeight, product of:
               1.6605521 = boost
               6.0133076 = idf(docFreq=293, maxDocs=44218)
               0.0113915745 = queryNorm
             0.8136995 = fieldWeight in 35, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               6.0133076 = idf(docFreq=293, maxDocs=44218)
               0.078125 = fieldNorm(doc=35)
         0.11685379 = weight(abstract_txt:proof in 35) [ClassicSimilarity], result of:
           0.11685379 = score(doc=35,freq=1.0), product of:
             0.19163556 = queryWeight, product of:
               2.1553397 = boost
               7.805067 = idf(docFreq=48, maxDocs=44218)
               0.0113915745 = queryNorm
             0.6097709 = fieldWeight in 35, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.805067 = idf(docFreq=48, maxDocs=44218)
               0.078125 = fieldNorm(doc=35)
         0.06740651 = weight(abstract_txt:model in 35) [ClassicSimilarity], result of:
           0.06740651 = score(doc=35,freq=3.0), product of:
             0.12496501 = queryWeight, product of:
               2.7519586 = boost
               3.986234 = idf(docFreq=2231, maxDocs=44218)
               0.0113915745 = queryNorm
             0.5394031 = fieldWeight in 35, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               3.986234 = idf(docFreq=2231, maxDocs=44218)
               0.078125 = fieldNorm(doc=35)
         0.15789676 = weight(abstract_txt:likelihood in 35) [ClassicSimilarity], result of:
           0.15789676 = score(doc=35,freq=1.0), product of:
             0.26811853 = queryWeight, product of:
               3.122387 = boost
               7.538004 = idf(docFreq=63, maxDocs=44218)
               0.0113915745 = queryNorm
             0.5889065 = fieldWeight in 35, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.538004 = idf(docFreq=63, maxDocs=44218)
               0.078125 = fieldNorm(doc=35)
         0.12583175 = weight(abstract_txt:language in 35) [ClassicSimilarity], result of:
           0.12583175 = score(doc=35,freq=4.0), product of:
             0.19256489 = queryWeight, product of:
               4.0420365 = boost
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.0113915745 = queryNorm
             0.65345114 = fieldWeight in 35, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.078125 = fieldNorm(doc=35)
       0.28 = coord(7/25)
   

2. Lassalle, E.; Lassalle, E.: Semantic models in information retrieval (2012) 0.17

   0.16998392 = sum of:
     0.16998392 = product of:
       0.7082664 = sum of:
         0.030157564 = weight(abstract_txt:words in 97) [ClassicSimilarity], result of:
           0.030157564 = score(doc=97,freq=1.0), product of:
             0.090140186 = queryWeight, product of:
               1.4782126 = boost
               5.353007 = idf(docFreq=568, maxDocs=44218)
               0.0113915745 = queryNorm
             0.33456293 = fieldWeight in 97, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.353007 = idf(docFreq=568, maxDocs=44218)
               0.0625 = fieldNorm(doc=97)
         0.04275069 = weight(abstract_txt:modeling in 97) [ClassicSimilarity], result of:
           0.04275069 = score(doc=97,freq=1.0), product of:
             0.11374955 = queryWeight, product of:
               1.6605521 = boost
               6.0133076 = idf(docFreq=293, maxDocs=44218)
               0.0113915745 = queryNorm
             0.37583172 = fieldWeight in 97, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               6.0133076 = idf(docFreq=293, maxDocs=44218)
               0.0625 = fieldNorm(doc=97)
         0.06226747 = weight(abstract_txt:model in 97) [ClassicSimilarity], result of:
           0.06226747 = score(doc=97,freq=4.0), product of:
             0.12496501 = queryWeight, product of:
               2.7519586 = boost
               3.986234 = idf(docFreq=2231, maxDocs=44218)
               0.0113915745 = queryNorm
             0.49827924 = fieldWeight in 97, product of:
               2.0 = tf(freq=4.0), with freq of:
                 4.0 = termFreq=4.0
               3.986234 = idf(docFreq=2231, maxDocs=44218)
               0.0625 = fieldNorm(doc=97)
         0.071181186 = weight(abstract_txt:language in 97) [ClassicSimilarity], result of:
           0.071181186 = score(doc=97,freq=2.0), product of:
             0.19256489 = queryWeight, product of:
               4.0420365 = boost
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.0113915745 = queryNorm
             0.3696478 = fieldWeight in 97, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.0625 = fieldNorm(doc=97)
         0.19315283 = weight(abstract_txt:estimation in 97) [ClassicSimilarity], result of:
           0.19315283 = score(doc=97,freq=1.0), product of:
             0.39168024 = queryWeight, product of:
               4.357712 = boost
               7.890225 = idf(docFreq=44, maxDocs=44218)
               0.0113915745 = queryNorm
             0.49313906 = fieldWeight in 97, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.890225 = idf(docFreq=44, maxDocs=44218)
               0.0625 = fieldNorm(doc=97)
         0.30875665 = weight(abstract_txt:mixture in 97) [ClassicSimilarity], result of:
           0.30875665 = score(doc=97,freq=1.0), product of:
             0.6129696 = queryWeight, product of:
               6.6766367 = boost
               8.059301 = idf(docFreq=37, maxDocs=44218)
               0.0113915745 = queryNorm
             0.50370634 = fieldWeight in 97, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.059301 = idf(docFreq=37, maxDocs=44218)
               0.0625 = fieldNorm(doc=97)
       0.24 = coord(6/25)
   

3. Dang, E.K.F.; Luk, R.W.P.; Allan, J.: Beyond bag-of-words : bigram-enhanced context-dependent term weights (2014) 0.15

   0.15389787 = sum of:
     0.15389787 = product of:
       0.64124113 = sum of:
         0.073069915 = weight(abstract_txt:weights in 1283) [ClassicSimilarity], result of:
           0.073069915 = score(doc=1283,freq=5.0), product of:
             0.08250393 = queryWeight, product of:
               7.24254 = idf(docFreq=85, maxDocs=44218)
               0.0113915745 = queryNorm
             0.88565373 = fieldWeight in 1283, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               7.24254 = idf(docFreq=85, maxDocs=44218)
               0.0546875 = fieldNorm(doc=1283)
         0.00903817 = weight(abstract_txt:approach in 1283) [ClassicSimilarity], result of:
           0.00903817 = score(doc=1283,freq=1.0), product of:
             0.044126812 = queryWeight, product of:
               1.0342581 = boost
               3.745328 = idf(docFreq=2839, maxDocs=44218)
               0.0113915745 = queryNorm
             0.20482263 = fieldWeight in 1283, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.745328 = idf(docFreq=2839, maxDocs=44218)
               0.0546875 = fieldNorm(doc=1283)
         0.023568682 = weight(abstract_txt:document in 1283) [ClassicSimilarity], result of:
           0.023568682 = score(doc=1283,freq=3.0), product of:
             0.057964936 = queryWeight, product of:
               1.1853875 = boost
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.0113915745 = queryNorm
             0.4066024 = fieldWeight in 1283, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.0546875 = fieldNorm(doc=1283)
         0.026387868 = weight(abstract_txt:words in 1283) [ClassicSimilarity], result of:
           0.026387868 = score(doc=1283,freq=1.0), product of:
             0.090140186 = queryWeight, product of:
               1.4782126 = boost
               5.353007 = idf(docFreq=568, maxDocs=44218)
               0.0113915745 = queryNorm
             0.29274255 = fieldWeight in 1283, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.353007 = idf(docFreq=568, maxDocs=44218)
               0.0546875 = fieldNorm(doc=1283)
         0.23901442 = weight(abstract_txt:estimation in 1283) [ClassicSimilarity], result of:
           0.23901442 = score(doc=1283,freq=2.0), product of:
             0.39168024 = queryWeight, product of:
               4.357712 = boost
               7.890225 = idf(docFreq=44, maxDocs=44218)
               0.0113915745 = queryNorm
             0.6102284 = fieldWeight in 1283, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.890225 = idf(docFreq=44, maxDocs=44218)
               0.0546875 = fieldNorm(doc=1283)
         0.27016208 = weight(abstract_txt:mixture in 1283) [ClassicSimilarity], result of:
           0.27016208 = score(doc=1283,freq=1.0), product of:
             0.6129696 = queryWeight, product of:
               6.6766367 = boost
               8.059301 = idf(docFreq=37, maxDocs=44218)
               0.0113915745 = queryNorm
             0.44074303 = fieldWeight in 1283, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.059301 = idf(docFreq=37, maxDocs=44218)
               0.0546875 = fieldNorm(doc=1283)
       0.24 = coord(6/25)
   

4. Bodoff, D.; Wu, B.; Wong, K.Y.M.: Relevance data for language models using maximum likelihood (2003) 0.13

   0.12592226 = sum of:
     0.12592226 = product of:
       0.6296113 = sum of:
         0.015494005 = weight(abstract_txt:approach in 1822) [ClassicSimilarity], result of:
           0.015494005 = score(doc=1822,freq=1.0), product of:
             0.044126812 = queryWeight, product of:
               1.0342581 = boost
               3.745328 = idf(docFreq=2839, maxDocs=44218)
               0.0113915745 = queryNorm
             0.3511245 = fieldWeight in 1822, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               3.745328 = idf(docFreq=2839, maxDocs=44218)
               0.09375 = fieldNorm(doc=1822)
         0.032989282 = weight(abstract_txt:document in 1822) [ClassicSimilarity], result of:
           0.032989282 = score(doc=1822,freq=2.0), product of:
             0.057964936 = queryWeight, product of:
               1.1853875 = boost
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.0113915745 = queryNorm
             0.5691248 = fieldWeight in 1822, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.09375 = fieldNorm(doc=1822)
         0.23766929 = weight(abstract_txt:maximum in 1822) [ClassicSimilarity], result of:
           0.23766929 = score(doc=1822,freq=2.0), product of:
             0.24751179 = queryWeight, product of:
               3.0 = boost
               7.24254 = idf(docFreq=85, maxDocs=44218)
               0.0113915745 = queryNorm
             0.9602342 = fieldWeight in 1822, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.24254 = idf(docFreq=85, maxDocs=44218)
               0.09375 = fieldNorm(doc=1822)
         0.26795968 = weight(abstract_txt:likelihood in 1822) [ClassicSimilarity], result of:
           0.26795968 = score(doc=1822,freq=2.0), product of:
             0.26811853 = queryWeight, product of:
               3.122387 = boost
               7.538004 = idf(docFreq=63, maxDocs=44218)
               0.0113915745 = queryNorm
             0.99940753 = fieldWeight in 1822, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               7.538004 = idf(docFreq=63, maxDocs=44218)
               0.09375 = fieldNorm(doc=1822)
         0.07549905 = weight(abstract_txt:language in 1822) [ClassicSimilarity], result of:
           0.07549905 = score(doc=1822,freq=1.0), product of:
             0.19256489 = queryWeight, product of:
               4.0420365 = boost
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.0113915745 = queryNorm
             0.3920707 = fieldWeight in 1822, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.09375 = fieldNorm(doc=1822)
       0.2 = coord(5/25)
   

5. Meij, E.; Rijke, M. de: Thesaurus-based feedback to support mixed search and browsing environments (2007) 0.12

   0.121700905 = sum of:
     0.121700905 = product of:
       0.5070871 = sum of:
         0.015551298 = weight(abstract_txt:document in 2432) [ClassicSimilarity], result of:
           0.015551298 = score(doc=2432,freq=1.0), product of:
             0.057964936 = queryWeight, product of:
               1.1853875 = boost
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.0113915745 = queryNorm
             0.26828802 = fieldWeight in 2432, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.2926083 = idf(docFreq=1642, maxDocs=44218)
               0.0625 = fieldNorm(doc=2432)
         0.048703723 = weight(abstract_txt:find in 2432) [ClassicSimilarity], result of:
           0.048703723 = score(doc=2432,freq=2.0), product of:
             0.11273258 = queryWeight, product of:
               2.024641 = boost
               4.887848 = idf(docFreq=905, maxDocs=44218)
               0.0113915745 = queryNorm
             0.4320288 = fieldWeight in 2432, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.887848 = idf(docFreq=905, maxDocs=44218)
               0.0625 = fieldNorm(doc=2432)
         0.12631741 = weight(abstract_txt:likelihood in 2432) [ClassicSimilarity], result of:
           0.12631741 = score(doc=2432,freq=1.0), product of:
             0.26811853 = queryWeight, product of:
               3.122387 = boost
               7.538004 = idf(docFreq=63, maxDocs=44218)
               0.0113915745 = queryNorm
             0.47112525 = fieldWeight in 2432, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.538004 = idf(docFreq=63, maxDocs=44218)
               0.0625 = fieldNorm(doc=2432)
         0.073029175 = weight(abstract_txt:algorithm in 2432) [ClassicSimilarity], result of:
           0.073029175 = score(doc=2432,freq=1.0), product of:
             0.20479934 = queryWeight, product of:
               3.151062 = boost
               5.705423 = idf(docFreq=399, maxDocs=44218)
               0.0113915745 = queryNorm
             0.35658893 = fieldWeight in 2432, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.705423 = idf(docFreq=399, maxDocs=44218)
               0.0625 = fieldNorm(doc=2432)
         0.0503327 = weight(abstract_txt:language in 2432) [ClassicSimilarity], result of:
           0.0503327 = score(doc=2432,freq=1.0), product of:
             0.19256489 = queryWeight, product of:
               4.0420365 = boost
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.0113915745 = queryNorm
             0.26138046 = fieldWeight in 2432, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.1820874 = idf(docFreq=1834, maxDocs=44218)
               0.0625 = fieldNorm(doc=2432)
         0.19315283 = weight(abstract_txt:estimation in 2432) [ClassicSimilarity], result of:
           0.19315283 = score(doc=2432,freq=1.0), product of:
             0.39168024 = queryWeight, product of:
               4.357712 = boost
               7.890225 = idf(docFreq=44, maxDocs=44218)
               0.0113915745 = queryNorm
             0.49313906 = fieldWeight in 2432, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.890225 = idf(docFreq=44, maxDocs=44218)
               0.0625 = fieldNorm(doc=2432)
       0.24 = coord(6/25)