This is a short overview. For a more technical explanation, see the mtest README.
Before any pull request is merged, it is required that it pass all of the automated tests which are set up in the
mtest folder. The tests automatically perform actions which would be performed by a user, and create a new score, which is compared against the expected output.
There are two places you can find mtests: in the
mtest directory (a subdirectory of the musescore root directory), or in the
In the root mtest directory
Tests are arranged into different directories. For example, copy and paste tests are in
mtest/libmscore/copypaste. Each test has a .mscx file without a
-ref, which is the score that tests are performed on, and a counterpart .mscx file with a
-ref file is the reference file: once the non-ref file has had the tests performed on it, it's compared with the reference file. If the files are exactly the same, the tests pass. But you can't run tests in the root
In the build.*/mtest directory
Here, you can run tests, once you've built release, debug etc.
Before you run any tests, make sure that your default style in MuseScore is the same as the MuseScore Default. As of version 3.1 there is a Reset to MuseScore Default button in the Format menu Style... dialog. Click that button prior to running ctest or mtest or any specific test executable. If you have your own styles you wish to preserve, save them in a style file that you can restore later.
To run the entire suite of tests, first
make the tests in the
build.*/mtest directory. To run them all, run the
mtest executable file that is created, or just run
To run just a single suite of tests, enter the relevant directory, e.g.
make the tests, and run the relevant executable file that is created, e.g.
After running tests, depending on which tests you ran, loads of
[testname].mscx files will be created. These are the files that are compared against the reference files in the root mtest directory. If there is a difference between them, the mtest will fail.
If you have made a change that makes the output of a test different, you need to update the reference file(s). You can do this by running
updateReference in the root mtest directory for whichever test you want to update, although this sometimes may not work. Or, you can manually copy accross the changed file(s), for example:
cp build.debug/mtest/libmscore/copypaste/copypaste23.mscx mtest/libmscore/copypaste/copypaste23-ref.mscx
Adding new tests
To add a new test, edit the relevant
tst_*.cpp file in the root mtest directory. It should be easy to figure out how these work.
If you add a new test that diffs files, please create an updateReference script to copy the reference files when any future changes are made.
If you have added a new file to an existing test, and it uses a " -ref" reference file for diff, please update the updateReference script in the test directory, to assist with future maintenance.