Document (#27564)

Author

Wu, K.J.
Chen, M.-C.
Sun, Y.

Title

Automatic topics discovery from hyperlinked documents

Source

Information processing and management. 40(2004) no.2, S.239-255

Year

2004

Abstract

Topic discovery is an important means for marketing, e-Business and social science studies. As well, it can be applied to various purposes, such as identifying a group with certain properties and observing the emergence and diminishment of a certain cyber community. Previous topic discovery work (J.M. Kleinberg, Proceedings of the 9th Annual ACM-SIAM Symposium on Discrete Algorithms, San Francisco, California, p. 668) requires manual judgment of usefulness of outcomes and is thus incapable of handling the explosive growth of the Internet. In this paper, we propose the Automatic Topic Discovery (ATD) method, which combines a method of base set construction, a clustering algorithm and an iterative principal eigenvector computation method to discover the topics relevant to a given query without using manual examination. Given a query, ATD returns with topics associated with the query and top representative pages for each topic. Our experiments show that the ATD method performs better than the traditional eigenvector method in terms of computation time and topic discovery quality.

Theme

Data Mining
Automatisches Klassifizieren

Similar documents (author)

1. Chen, Y.N.; Chen, S.J.: ¬A metadata practice of the OFLA FRBR model : a case study for the National Palace Museum in Taipai (2004) 4.35

   4.3499155 = sum of:
     4.3499155 = weight(author_txt:chen in 3384) [ClassicSimilarity], result of:
       4.3499155 = fieldWeight in 3384, product of:
         1.4142135 = tf(freq=2.0), with freq of:
           2.0 = termFreq=2.0
         6.1517096 = idf(docFreq=255, maxDocs=44218)
         0.5 = fieldNorm(doc=3384)
   

2. Chen, C.C.; Chen, H.H.; Chen, K.H.: ¬The design of the XML/Metadata management system (2000) 4.00

   3.9956524 = sum of:
     3.9956524 = weight(author_txt:chen in 4633) [ClassicSimilarity], result of:
       3.9956524 = fieldWeight in 4633, product of:
         1.7320508 = tf(freq=3.0), with freq of:
           3.0 = termFreq=3.0
         6.1517096 = idf(docFreq=255, maxDocs=44218)
         0.375 = fieldNorm(doc=4633)
   

3. Chen, W.Y.: Observations on cataloguing and classification (1991) 3.84

   3.8448186 = sum of:
     3.8448186 = weight(author_txt:chen in 4184) [ClassicSimilarity], result of:
       3.8448186 = fieldWeight in 4184, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         6.1517096 = idf(docFreq=255, maxDocs=44218)
         0.625 = fieldNorm(doc=4184)
   

4. Chen, H.: Knowledge-based document retrieval : framework and design (1992) 3.84

   3.8448186 = sum of:
     3.8448186 = weight(author_txt:chen in 5283) [ClassicSimilarity], result of:
       3.8448186 = fieldWeight in 5283, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         6.1517096 = idf(docFreq=255, maxDocs=44218)
         0.625 = fieldNorm(doc=5283)
   

5. Chen, P.S.: On inference rules of logic-based information retrieval systems (1994) 3.84

   3.8448186 = sum of:
     3.8448186 = weight(author_txt:chen in 6731) [ClassicSimilarity], result of:
       3.8448186 = fieldWeight in 6731, product of:
         1.0 = tf(freq=1.0), with freq of:
           1.0 = termFreq=1.0
         6.1517096 = idf(docFreq=255, maxDocs=44218)
         0.625 = fieldNorm(doc=6731)
   

Similar documents (content)

1. Potha, N.; Stamatatos, E.: Improving author verification based on topic modeling (2019) 0.14

   0.13752389 = sum of:
     0.13752389 = product of:
       0.49115676 = sum of:
         0.010385565 = weight(abstract_txt:with in 5385) [ClassicSimilarity], result of:
           0.010385565 = score(doc=5385,freq=2.0), product of:
             0.047004715 = queryWeight, product of:
               1.0647846 = boost
               2.4997334 = idf(docFreq=9868, maxDocs=44218)
               0.017659806 = queryNorm
             0.22094731 = fieldWeight in 5385, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4997334 = idf(docFreq=9868, maxDocs=44218)
               0.0625 = fieldNorm(doc=5385)
         0.08547968 = weight(abstract_txt:cyber in 5385) [ClassicSimilarity], result of:
           0.08547968 = score(doc=5385,freq=1.0), product of:
             0.16739127 = queryWeight, product of:
               1.1601038 = boost
               8.1705265 = idf(docFreq=33, maxDocs=44218)
               0.017659806 = queryNorm
             0.5106579 = fieldWeight in 5385, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               8.1705265 = idf(docFreq=33, maxDocs=44218)
               0.0625 = fieldNorm(doc=5385)
         0.032564696 = weight(abstract_txt:given in 5385) [ClassicSimilarity], result of:
           0.032564696 = score(doc=5385,freq=1.0), product of:
             0.11083211 = queryWeight, product of:
               1.3349907 = boost
               4.701121 = idf(docFreq=1091, maxDocs=44218)
               0.017659806 = queryNorm
             0.29382005 = fieldWeight in 5385, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.701121 = idf(docFreq=1091, maxDocs=44218)
               0.0625 = fieldNorm(doc=5385)
         0.05335181 = weight(abstract_txt:certain in 5385) [ClassicSimilarity], result of:
           0.05335181 = score(doc=5385,freq=1.0), product of:
             0.15402824 = queryWeight, product of:
               1.5737858 = boost
               5.542029 = idf(docFreq=470, maxDocs=44218)
               0.017659806 = queryNorm
             0.3463768 = fieldWeight in 5385, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.542029 = idf(docFreq=470, maxDocs=44218)
               0.0625 = fieldNorm(doc=5385)
         0.061860308 = weight(abstract_txt:topics in 5385) [ClassicSimilarity], result of:
           0.061860308 = score(doc=5385,freq=1.0), product of:
             0.19459845 = queryWeight, product of:
               2.1665103 = boost
               5.086191 = idf(docFreq=742, maxDocs=44218)
               0.017659806 = queryNorm
             0.31788695 = fieldWeight in 5385, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.086191 = idf(docFreq=742, maxDocs=44218)
               0.0625 = fieldNorm(doc=5385)
         0.07144886 = weight(abstract_txt:method in 5385) [ClassicSimilarity], result of:
           0.07144886 = score(doc=5385,freq=1.0), product of:
             0.25398678 = queryWeight, product of:
               3.195369 = boost
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.017659806 = queryNorm
             0.28130937 = fieldWeight in 5385, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.0625 = fieldNorm(doc=5385)
         0.17606588 = weight(abstract_txt:topic in 5385) [ClassicSimilarity], result of:
           0.17606588 = score(doc=5385,freq=3.0), product of:
             0.32128513 = queryWeight, product of:
               3.5938568 = boost
               5.062254 = idf(docFreq=760, maxDocs=44218)
               0.017659806 = queryNorm
             0.54800504 = fieldWeight in 5385, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               5.062254 = idf(docFreq=760, maxDocs=44218)
               0.0625 = fieldNorm(doc=5385)
       0.28 = coord(7/25)
   

2. Alkhodair, S.A.; Fung, B.C.M.; Patrick, O.R.; Hung, C.K.: Improving interpretations of topic modeling in microblogs (2018) 0.12

   0.123626165 = sum of:
     0.123626165 = product of:
       0.6181308 = sum of:
         0.058153972 = weight(abstract_txt:performs in 4181) [ClassicSimilarity], result of:
           0.058153972 = score(doc=4181,freq=1.0), product of:
             0.12948233 = queryWeight, product of:
               1.0203184 = boost
               7.1860275 = idf(docFreq=90, maxDocs=44218)
               0.017659806 = queryNorm
             0.44912672 = fieldWeight in 4181, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.1860275 = idf(docFreq=90, maxDocs=44218)
               0.0625 = fieldNorm(doc=4181)
         0.010385565 = weight(abstract_txt:with in 4181) [ClassicSimilarity], result of:
           0.010385565 = score(doc=4181,freq=2.0), product of:
             0.047004715 = queryWeight, product of:
               1.0647846 = boost
               2.4997334 = idf(docFreq=9868, maxDocs=44218)
               0.017659806 = queryNorm
             0.22094731 = fieldWeight in 4181, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               2.4997334 = idf(docFreq=9868, maxDocs=44218)
               0.0625 = fieldNorm(doc=4181)
         0.13832386 = weight(abstract_txt:topics in 4181) [ClassicSimilarity], result of:
           0.13832386 = score(doc=4181,freq=5.0), product of:
             0.19459845 = queryWeight, product of:
               2.1665103 = boost
               5.086191 = idf(docFreq=742, maxDocs=44218)
               0.017659806 = queryNorm
             0.71081686 = fieldWeight in 4181, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               5.086191 = idf(docFreq=742, maxDocs=44218)
               0.0625 = fieldNorm(doc=4181)
         0.12375305 = weight(abstract_txt:method in 4181) [ClassicSimilarity], result of:
           0.12375305 = score(doc=4181,freq=3.0), product of:
             0.25398678 = queryWeight, product of:
               3.195369 = boost
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.017659806 = queryNorm
             0.4872421 = fieldWeight in 4181, product of:
               1.7320508 = tf(freq=3.0), with freq of:
                 3.0 = termFreq=3.0
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.0625 = fieldNorm(doc=4181)
         0.28751436 = weight(abstract_txt:topic in 4181) [ClassicSimilarity], result of:
           0.28751436 = score(doc=4181,freq=8.0), product of:
             0.32128513 = queryWeight, product of:
               3.5938568 = boost
               5.062254 = idf(docFreq=760, maxDocs=44218)
               0.017659806 = queryNorm
             0.8948885 = fieldWeight in 4181, product of:
               2.828427 = tf(freq=8.0), with freq of:
                 8.0 = termFreq=8.0
               5.062254 = idf(docFreq=760, maxDocs=44218)
               0.0625 = fieldNorm(doc=4181)
       0.2 = coord(5/25)
   

3. Finn, A.; Kushmerick, N.: Learning to classify documents according to genre (2006) 0.11

   0.10745999 = sum of:
     0.10745999 = product of:
       0.44774997 = sum of:
         0.009179629 = weight(abstract_txt:with in 6010) [ClassicSimilarity], result of:
           0.009179629 = score(doc=6010,freq=1.0), product of:
             0.047004715 = queryWeight, product of:
               1.0647846 = boost
               2.4997334 = idf(docFreq=9868, maxDocs=44218)
               0.017659806 = queryNorm
             0.19529167 = fieldWeight in 6010, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               2.4997334 = idf(docFreq=9868, maxDocs=44218)
               0.078125 = fieldNorm(doc=6010)
         0.04070587 = weight(abstract_txt:given in 6010) [ClassicSimilarity], result of:
           0.04070587 = score(doc=6010,freq=1.0), product of:
             0.11083211 = queryWeight, product of:
               1.3349907 = boost
               4.701121 = idf(docFreq=1091, maxDocs=44218)
               0.017659806 = queryNorm
             0.36727506 = fieldWeight in 6010, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.701121 = idf(docFreq=1091, maxDocs=44218)
               0.078125 = fieldNorm(doc=6010)
         0.07770953 = weight(abstract_txt:automatic in 6010) [ClassicSimilarity], result of:
           0.07770953 = score(doc=6010,freq=2.0), product of:
             0.13537353 = queryWeight, product of:
               1.4754087 = boost
               5.1955976 = idf(docFreq=665, maxDocs=44218)
               0.017659806 = queryNorm
             0.57403785 = fieldWeight in 6010, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.1955976 = idf(docFreq=665, maxDocs=44218)
               0.078125 = fieldNorm(doc=6010)
         0.06313308 = weight(abstract_txt:query in 6010) [ClassicSimilarity], result of:
           0.06313308 = score(doc=6010,freq=1.0), product of:
             0.16999231 = queryWeight, product of:
               2.02491 = boost
               4.7537646 = idf(docFreq=1035, maxDocs=44218)
               0.017659806 = queryNorm
             0.37138787 = fieldWeight in 6010, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.7537646 = idf(docFreq=1035, maxDocs=44218)
               0.078125 = fieldNorm(doc=6010)
         0.07732538 = weight(abstract_txt:topics in 6010) [ClassicSimilarity], result of:
           0.07732538 = score(doc=6010,freq=1.0), product of:
             0.19459845 = queryWeight, product of:
               2.1665103 = boost
               5.086191 = idf(docFreq=742, maxDocs=44218)
               0.017659806 = queryNorm
             0.3973587 = fieldWeight in 6010, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.086191 = idf(docFreq=742, maxDocs=44218)
               0.078125 = fieldNorm(doc=6010)
         0.17969649 = weight(abstract_txt:topic in 6010) [ClassicSimilarity], result of:
           0.17969649 = score(doc=6010,freq=2.0), product of:
             0.32128513 = queryWeight, product of:
               3.5938568 = boost
               5.062254 = idf(docFreq=760, maxDocs=44218)
               0.017659806 = queryNorm
             0.5593053 = fieldWeight in 6010, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               5.062254 = idf(docFreq=760, maxDocs=44218)
               0.078125 = fieldNorm(doc=6010)
       0.24 = coord(6/25)
   

4. Lempel, R.; Moran, S.: SALSA: the stochastic approach for link-structure analysis (2001) 0.10

   0.09999313 = sum of:
     0.09999313 = product of:
       0.49996567 = sum of:
         0.058153972 = weight(abstract_txt:performs in 10) [ClassicSimilarity], result of:
           0.058153972 = score(doc=10,freq=1.0), product of:
             0.12948233 = queryWeight, product of:
               1.0203184 = boost
               7.1860275 = idf(docFreq=90, maxDocs=44218)
               0.017659806 = queryNorm
             0.44912672 = fieldWeight in 10, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               7.1860275 = idf(docFreq=90, maxDocs=44218)
               0.0625 = fieldNorm(doc=10)
         0.2153813 = weight(abstract_txt:kleinberg in 10) [ClassicSimilarity], result of:
           0.2153813 = score(doc=10,freq=2.0), product of:
             0.24601008 = queryWeight, product of:
               1.4063936 = boost
               9.905128 = idf(docFreq=5, maxDocs=44218)
               0.017659806 = queryNorm
             0.8754978 = fieldWeight in 10, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               9.905128 = idf(docFreq=5, maxDocs=44218)
               0.0625 = fieldNorm(doc=10)
         0.05335181 = weight(abstract_txt:certain in 10) [ClassicSimilarity], result of:
           0.05335181 = score(doc=10,freq=1.0), product of:
             0.15402824 = queryWeight, product of:
               1.5737858 = boost
               5.542029 = idf(docFreq=470, maxDocs=44218)
               0.017659806 = queryNorm
             0.3463768 = fieldWeight in 10, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.542029 = idf(docFreq=470, maxDocs=44218)
               0.0625 = fieldNorm(doc=10)
         0.07142693 = weight(abstract_txt:query in 10) [ClassicSimilarity], result of:
           0.07142693 = score(doc=10,freq=2.0), product of:
             0.16999231 = queryWeight, product of:
               2.02491 = boost
               4.7537646 = idf(docFreq=1035, maxDocs=44218)
               0.017659806 = queryNorm
             0.4201774 = fieldWeight in 10, product of:
               1.4142135 = tf(freq=2.0), with freq of:
                 2.0 = termFreq=2.0
               4.7537646 = idf(docFreq=1035, maxDocs=44218)
               0.0625 = fieldNorm(doc=10)
         0.10165168 = weight(abstract_txt:topic in 10) [ClassicSimilarity], result of:
           0.10165168 = score(doc=10,freq=1.0), product of:
             0.32128513 = queryWeight, product of:
               3.5938568 = boost
               5.062254 = idf(docFreq=760, maxDocs=44218)
               0.017659806 = queryNorm
             0.31639087 = fieldWeight in 10, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.062254 = idf(docFreq=760, maxDocs=44218)
               0.0625 = fieldNorm(doc=10)
       0.2 = coord(5/25)
   

5. Pons-Porrata, A.; Berlanga-Llavori, R.; Ruiz-Shulcloper, J.: Topic discovery based on text mining techniques (2007) 0.10

   0.099928364 = sum of:
     0.099928364 = product of:
       0.6245523 = sum of:
         0.07732538 = weight(abstract_txt:topics in 916) [ClassicSimilarity], result of:
           0.07732538 = score(doc=916,freq=1.0), product of:
             0.19459845 = queryWeight, product of:
               2.1665103 = boost
               5.086191 = idf(docFreq=742, maxDocs=44218)
               0.017659806 = queryNorm
             0.3973587 = fieldWeight in 916, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.086191 = idf(docFreq=742, maxDocs=44218)
               0.078125 = fieldNorm(doc=916)
         0.08931107 = weight(abstract_txt:method in 916) [ClassicSimilarity], result of:
           0.08931107 = score(doc=916,freq=1.0), product of:
             0.25398678 = queryWeight, product of:
               3.195369 = boost
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.017659806 = queryNorm
             0.3516367 = fieldWeight in 916, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               4.50095 = idf(docFreq=1333, maxDocs=44218)
               0.078125 = fieldNorm(doc=916)
         0.2841251 = weight(abstract_txt:topic in 916) [ClassicSimilarity], result of:
           0.2841251 = score(doc=916,freq=5.0), product of:
             0.32128513 = queryWeight, product of:
               3.5938568 = boost
               5.062254 = idf(docFreq=760, maxDocs=44218)
               0.017659806 = queryNorm
             0.88433933 = fieldWeight in 916, product of:
               2.236068 = tf(freq=5.0), with freq of:
                 5.0 = termFreq=5.0
               5.062254 = idf(docFreq=760, maxDocs=44218)
               0.078125 = fieldNorm(doc=916)
         0.17379075 = weight(abstract_txt:discovery in 916) [ClassicSimilarity], result of:
           0.17379075 = score(doc=916,freq=1.0), product of:
             0.39587554 = queryWeight, product of:
               3.9892826 = boost
               5.619245 = idf(docFreq=435, maxDocs=44218)
               0.017659806 = queryNorm
             0.43900353 = fieldWeight in 916, product of:
               1.0 = tf(freq=1.0), with freq of:
                 1.0 = termFreq=1.0
               5.619245 = idf(docFreq=435, maxDocs=44218)
               0.078125 = fieldNorm(doc=916)
       0.16 = coord(4/25)