History for NewLang / tests / test_tokenize.py
2022-02-27
@Jookia
tests: tokenize: Fix typo
Jookia committed on 27 Feb 2022
2022-02-15
@Jookia
testss: tokenize: Add documentation on the expected behaviours
Jookia committed on 15 Feb 2022
@Jookia
tests: tokenize: Give garbled tokens to the classifier ...
Jookia committed on 15 Feb 2022
2022-02-08
@Jookia
tokenize: Add shebang types
Jookia committed on 8 Feb 2022
@Jookia
tokenize: Classify language keywords
Jookia committed on 8 Feb 2022
@Jookia
tokenize: Add booleans
Jookia committed on 8 Feb 2022
@Jookia
test: tokenize: Move draw_tokens_list to draw_tokens_to_split ...
Jookia committed on 8 Feb 2022
@Jookia
tests: tokenize: Add a draw_token_classified strategy
Jookia committed on 8 Feb 2022
@Jookia
tests: tokenize: Use random tokens again for locations ...
Jookia committed on 8 Feb 2022
@Jookia
tests: tokenize: Test location using types only
Jookia committed on 8 Feb 2022
@Jookia
tokenize: Use type for locating newlines ...
Jookia committed on 8 Feb 2022
@Jookia
tokenize: Add NEWLINE token type ...
Jookia committed on 8 Feb 2022
@Jookia
tests: tokenize: Add more accurate comments
Jookia committed on 8 Feb 2022
@Jookia
tests: tokenize: Have draw_token_* use draw_token_random
Jookia committed on 8 Feb 2022
@Jookia
tests: tokenize: Put generic draws at top of file
Jookia committed on 8 Feb 2022
2022-02-07
@Jookia
tokenize: Add tokenize wrapper function
Jookia committed on 7 Feb 2022
@Jookia
tests: tokenize: Add comments
Jookia committed on 7 Feb 2022
@Jookia
tests: tokenize: Run classification with random input
Jookia committed on 7 Feb 2022
@Jookia
tokenize: Add classify_tokens to set token types
Jookia committed on 7 Feb 2022
@Jookia
tests: tokenize: Rename draw_token to draw_token_random
Jookia committed on 7 Feb 2022
@Jookia
tokenize: Add WHITESPACE TokenType ...
Jookia committed on 7 Feb 2022
2022-02-06
@Jookia
tests: Use random token types for tokens
Jookia committed on 6 Feb 2022
@Jookia
tokenize: Add type value for token type
Jookia committed on 6 Feb 2022
@Jookia
src: tokenize: Rename symbol to token
Jookia committed on 6 Feb 2022
@Jookia
tests: Generate random locations for symbols ...
Jookia committed on 6 Feb 2022
2022-02-05
@Jookia
tokenize: Add locate_symbols function to add symbol locations ...
Jookia committed on 5 Feb 2022
@Jookia
tests: tokenize: Test symbol structure
Jookia committed on 5 Feb 2022
@Jookia
tokenize: Always specify a location for symbols ...
Jookia committed on 5 Feb 2022
@Jookia
tests: tokenize: Test location structure ...
Jookia committed on 5 Feb 2022
@Jookia
tests: tokenize: Fix test only drawing whitespace
Jookia committed on 5 Feb 2022