searchstringparser.lexer

class searchstringparser.lexer.general.GeneralSearchStringLexer(illegal='ignore', **kw_args)

Bases: object

__init__(illegal='ignore', **kw_args)

A composite class of a ply.lex.lex.

This is the setup step necessary before you can iterate over the tokens.

Parameters:

• illegal ({'record', 'ignore', 'error'} (optional)) – Action to be taken when illegal characters are encountered. The default is to record them but continue without regarding them.
• kw_args – Keyword arguments are passed to the ply.lex.lex call.

get_illegal()

Return encountered illegal characters.

Returns:

• None – If no illegal characters occurred.
• Tuple – A pair of lists that contain the illegal characters and the positions where they occurred.

input(data)

Add a new string to the lexer.

This is the setup step necessary before you can iterate over the tokens.

Parameters:data (str) – Any string.

print_tokens(data)

Print all tokens in a string.

First iterates through all tokens found and prints them to sys.stdout. Then prints illegal characters if any occurred.

Parameters:data (str) – Any string.