Interface TokensProvider
Expand description
interface TokensProvider {
getInitialState(): IState;
tokenize(line: string, state: IState): ILineTokens;
}Methods§
§getInitialState(): IState
getInitialState(): IState
The initial state of a language. Will be the state passed in to tokenize the first line.
§tokenize(line: string, state: IState): ILineTokens
tokenize(line: string, state: IState): ILineTokens
Tokenize a line given the state at the beginning of the line.
A "manual" provider of tokens.