tokenize: Move classify_tokens to before locate

Fork: 0

LuminaSensum / NewLang

Browse code tokenize: Move classify_tokens to before locate_tokens main
1 parent 4564cdb commit f41ba0fd2872da61fa2d9172122f04c5f2a19ac6 Jookia authored on 8 Feb 2022

Patch

Unified Split

Showing 1 changed file

Ignore Space Show notes View src/tokenize.py

Show line notes below