tokenize: Add type value for token type
1 parent 4a47c8c commit 3c6d899cfb867dfc438af9e908fe824e06f2b548
@Jookia Jookia authored on 6 Feb 2022
Showing 2 changed files
View
src/tokenize.py
View
tests/test_tokenize.py