analyze
Description:
Extract terms and or collation keys from some input data (which is normally, but not necessarily, a UTF-8 string).
The terms and corresponding collation keys will be written in order to the provided TermList s.
Implementation notes for subclasses: The analysis process must call tokenize and run the tokens through all term filters added with add_term_filter. Collation keys must be generated with collate_key.
Parameters:
this |
The analyzer to use |
data |
The input data to analyze |
terms_out |
A TermList to place the generated terms in. If |
colkeys_out |
A TermList to place generated collation keys in. If |