Lexer

source: bluebase/lp/lexer.py

---

class LpLexer

Bases: object

A lexer class that converts an input query into tokens.

Variables:

pattern (re.Pattern[str]) – a pattern combining regular expression rules

Rules

Group	Pattern
FLOAT	\d+\.\d+
INT	\d+
BOOL	TRUE|FALSE
STRING	'[^']*'
WORD	[a-zA-Z_*][a-zA-Z0-9_]*
COMP	=|<>|<=|>=|<|>
COMMA	,
LP	\(
RP	\)
DOT	\.
SC	;
WS	[ \t\n]+

assignmenttokenize(query: str) → list[LpToken]

Convert a query into tokens.

Raises:

LpInvalidTokenError – if a token is invalid

Hint

Find matches sequentially using pattern.finditer. If the current position does not match the start position of a match, the token is invalid.

Ignore WS tokens instead of adding them.

Update the current position to the end position of each match. If the final position does not match the length of the query, the query is also invalid.