Handle object returned from tokenize.
Properties:
next |
TokenizerHandleNext | Gets the next token and advances ( |
peek |
TokenizerHandlePeek | Peeks for the next token ( |
push |
TokenizerHandlePush | Pushes a token back to the stack |
skip |
TokenizerHandleSkip | Skips a token, returns its presence and advances or, if non-optional and not present, throws |
cmnt |
TokenizerHandleCmnt | Gets the comment on the previous line or the line comment on the specified line, if any |
line |
number | Current line number |