flambe.tokenizer.char
¶
Module Contents¶
-
class
flambe.tokenizer.char.
CharTokenizer
[source]¶ Bases:
flambe.tokenizer.Tokenizer
Implement a character level tokenizer.
flambe.tokenizer.char
¶flambe.tokenizer.char.
CharTokenizer
[source]¶Bases: flambe.tokenizer.Tokenizer
Implement a character level tokenizer.