testing: Add integration tests #16

Currently we only have pytest. This doesn't cover making sure the build targets and final build actually run.

These should be shell scripts that run the make targets, maybe the tests, and somehow the program when run from the zipapp binary and from within the repository. These might just have to be sanity checks.

I've added a Jenkinsfile and similar utilities to build and test the project. So this might actually be done, it depends on whether there's a better CI system. I'll have to investigate:

• https://concourse-ci.org/
• https://kraken.ci/
• https://www.gocd.org/
• https://ovh.github.io/cds/

Last night I scheduled two Jenkins builds to run. It managed to deadlock and hang because one build uses both Linux and Win10, and instead of the first build using both, one build got Linux, one got Win10, and both builds waited for each other to be done.

Somehow this hung. I have no clue why, it would seem to me like once either were finished they could swap builds, but I guess not.

There's also an annoying Jenkins bug where on Windows it won't or can't log Unicode properly. This doesn't amuse me.

I'm going to look at other CI systems just to see if things are better.

I'm starting to need integration tests so it's about time to plan them.

My idea is that we have a client/server architecture or something where one runs programs sent to it and returns the result and the other handles sending tests and checking if the result is correct. This is kind of needed because some targets such as microcontrollers aren't big enough to do all this on their own and might only be able to handle loading and interpreting one small program at a time. Larger systems that run Linux likely won't need this.

So at minimum we'll need:

• Some remote control system
• A client system
• A server system

It kind of makes me wonder what exactly this should be. Starting with something that runs just for tests is a good idea, but such an interface would be useful for debugging later and probably interactive development where you'd like to parse or update code.

So two concrete situations are:

• Linux host that runs both the client and server at once, talking over a pipe
• Linux host and microcontroller target, talking over serial

The host would always have test examples and expected results on it.

Writing something like this seems a bit daunting, so it's probably best I split it like this:

• Target module that just takes commands and runs them
• Remote command handling module
• Host module that sends tests to the module and checks the result
• Launcher that handles setting up command handler and runner

...

Okay, now that I have that grand plan out of system let's try and cut it down to only stuff we need now:

• Code that will parse and interpret a program
• Code that takes program code and output and tests it
• Code that does this for multiple programs and reports errors and output
• Code that will search for program code and matching output
• Code that will glue the searcher to the multiple program tester

I imagine the search code will be the hardest to test since it touches the filesystem, not sure though.

I will need to have MicroPython run the tests on Unix too.