History for NewLang / tests / test_tokenize.py
2022-02-06
@Jookia
src: tokenize: Rename symbol to token
Jookia committed on 6 Feb 2022
@Jookia
tests: Generate random locations for symbols ...
Jookia committed on 6 Feb 2022
2022-02-05
@Jookia
tokenize: Add locate_symbols function to add symbol locations ...
Jookia committed on 5 Feb 2022
@Jookia
tests: tokenize: Test symbol structure
Jookia committed on 5 Feb 2022
@Jookia
tokenize: Always specify a location for symbols ...
Jookia committed on 5 Feb 2022
@Jookia
tests: tokenize: Test location structure ...
Jookia committed on 5 Feb 2022
@Jookia
tests: tokenize: Fix test only drawing whitespace
Jookia committed on 5 Feb 2022
2022-02-04
@Jookia
src: tokenize: Use symbol objects for symbols ...
Jookia committed on 4 Feb 2022
@Jookia
src: tokenize: Don't join whitespaces when splitting ...
Jookia committed on 4 Feb 2022
@Jookia
src: tokenizer: Add beginning of new tokenizer
Jookia committed on 4 Feb 2022