languages

Interface ILineTokens

Expand description

The result of a line tokenization.

interface ILineTokens {
    endState: IState;
    tokens: IToken[];
}

Properties§

§endState: IState

The tokenization end state. A pointer will be held to this and the object should not be modified by the tokenizer after the pointer is returned.

§tokens: IToken[]

The list of tokens on the line.