Re:The Easy Way to Extract Useful Text from Arbitrary HTML
by jonmc12 on Wednesday August 06, 2008 @04:49PM
(#268956)
The patterns could be formed in a more formalized way by applying Shannon's principles of information entropy across the document, lines of text, word patterns, N-Grams, etc. Then Bayesian inference can be applied for probabilistic pattern matching (vs the neural network).
Not that this approach is any easier, its just perhaps more robust for applying to other problem sets.
Nothing new of course, according to their website, Autonomy (Europe's 2nd largest software company) uses these techniques as the basis for their core technology to analyze text, audio, video, etc.
Not that this approach is any easier, its just perhaps more robust for applying to other problem sets.
Nothing new of course, according to their website, Autonomy (Europe's 2nd largest software company) uses these techniques as the basis for their core technology to analyze text, audio, video, etc.