You are looking at documentation for an older release. Not what you want? See the current release documentation.
A tokenizer of type
thai that segments Thai text into words. This tokenizer
uses the built-in Thai segmentation algorithm included with Java to divide
up Thai text. Text in other languages in general will be treated the same