parse: Handle end of file better
Instead of throwing an error, treat end of files as a specific type of
token that's added to the end of the tokenizer.
This also means we can always return EOF once we reach the end of the
tokens just by sticking to the last token.
1 parent 29dcbe8 commit 1c57b54f70977cd916d041c6abfe98c705b0d07c
@Jookia Jookia authored on 2 Apr 2021
Showing 1 changed file
View
parse.py