Run mtests in QtCreator/MinGW Windows

Just an information log of what I did to make mtest work in QtCreator. This acts as a temporary scrapbook note until (if?) these instructions can be finalized and turned into the real compilation documentation.

Start situation

I have a setup similar to https://musescore.org/en/handbook/developers-handbook/compilation/compi… for which I'll specify the settings here. This was not a clean local copy, but one that has been around for a while and updated several times.

I worked on the 3.x branch at around the 3.5beta tag; I suspect the steps required for master will be quite similar.

  • Kit: Qt 5.12.5 MinGW 64-bit
  • cmake: 3.16.4

Build Settings

CMake

Setting Value
Build directory .\build.qtc
CMAKE_BUILD_TYPE Debug
CMAKE_INSTALL_PREFIX ..\win64install

Build Steps

Build target mscore :
cmake.exe --build . --target mscore -- -j3

Run Settings

Deployment

Build target install :
cmake.exe --build . --target install -- -j3

Run

Custom executable:
%{sourceDir}\win64install\bin\MuseScore3.exe

Enabling mtests

Build settings

Add an additional build step to build the mtests. As building them all can take quite some time, you might consider building just a selection of tests.

For just one test:

  1. Add Build Step, type "Build"
  2. Check the corresponding target (for example tst_repeat)

For all tests:

  1. Add Build Step, type "Custom Process Step"
  2. Command: %{CMake:Executable:FilePath}
  3. Arguments: --build mtest -j3 (the -j is optional and enables parallel builds (here 3 of them))
  4. Working Directory: %{buildDir}

Disable build steps

You can easily disable a certain build step (for example, disable the mtest build during initial development/debugging to save on compilation time). Do this by clicking the "stop sign" for the corresponding build step (in the screenshot the tst_repeat build step will not be executed as it is disabled):
mtest-build-steps.png

Run Settings

All tests need changes to their run settings. As applying the same change to all these tests is quite a laborious task using the QtCreator gui, I've opted for changing it for one run configuration and then used search-and-replace in the project file to update all other targets as well.
This means the non-test targets, such as the "normal" Custom Executable step will also have (some of these) changed settings, but it doesn't negatively affect them.

Change one test run configuration using QtCreator

  1. Select "tst_barline" in the "Run configuration" dropdown
  2. Check the "Add build library search path to PATH"
  3. Under "Environment" expand the Details view
  4. In the list, click on the "Path" variable
  5. Click on "Append Path…"
  6. Browse to a path containing the diff tool, which is required for many test to compare the test result against the reference score. I pointed to C:\Program Files\Git\usr\bin as diff comes along with the Git for Windows install which any windows MuseScore dev should've installed anyway
  7. Change the "Run configuration" back to the "Custom Executable"

Apply this change to all test run configurations

Make sure you close QtCreator for this part!

  1. Open .\CMakeLists.txt.user with a text editor (such as NotePad++)
  2. Find and Replace <value type="bool" key="RunConfiguration.UseLibrarySearchPath">false</value> with <value type="bool" key="RunConfiguration.UseLibrarySearchPath">true</value>
    This will enable that checkbox from step 2 above for all run configurations
  3. Find <valuelist type="QVariantList" key="PE.EnvironmentAspect.Changes"> and copy the entire valuelist element. This is the resulting value of the append path action taken above.
  4. Find <valuelist type="QVariantList" key="PE.EnvironmentAspect.Changes"/> and replace it with the copied valuelist element from step 3
  5. Save :)

Running an mtest (or all of them)

  1. Open QtCreator and the mscore project
  2. Change from project to test view
    mtest-switch-project-to-test-view.png
  3. Scan for tests: Tools → Tests → Rescan Tests
  4. Change the bottom output window to the "Test Results" window
  5. Right-click any test in the list to Run/Debug just that test, or run only selected tests or run All tests

Results

Some test miss reference files in ((un)expected) locations, which can likely be fixed through their makefiles. Those failures are listed below. After manually performing the listed correction steps, this is the end result of running all tests:

1021 passes, 3 fails, 1 expected fails, 1 fatals.

biab

Copy the ./share/styles/ folder to ./mtest

compat114

Copy the ./share/styles/ folder to ./mtest/libmscore
!Do NOT overwrite the CMakeLists.txt file already there
→ Likely it is only required to copy over the chords.xml file

testPalletes

Copy workspace files from ./win64install/workspaces to ./build.qtc/mtest/mscore/workspaces

testWorkspaces

Likely needs the workspace copy as for testPalletes, but there are more issues ongoing here. Currently FAILs

tst_scripting

"missing QtQuick"
See its CMakeLists file which performs {QT_INSTALL_QML} for MSVC, likely the same should happen for MinGW too

runTestScripts

Seems to not complete. It is marked as a PASS, but the cleanup steps doesn't seem to be executed. Perhaps it is handing somewhere? Unsure about what is amiss here.

Do you still have an unanswered question? Please log in first to post your question.