tokenize: Always specify a location for symbols

Fork: 0

LuminaSensum / NewLang

Browse code tokenize: Always specify a location for symbols This gets rid of the default symbol location. The 'default' used now is just the result of split_symbols. main
1 parent 57e1e90 commit eadd2aedb78a0c6256ffed7212d4ac758601dc89 Jookia authored on 5 Feb 2022

Patch

Unified Split

Showing 2 changed files

Ignore Space Show notes View src/tokenize.py

Ignore Space Show notes View tests/test_tokenize.py

Show line notes below