Enable compilation and runnning of mtests on Windows with MSVC

• Apr 11, 2020 - 12:26
Reported version
3.4
Type
Development
Frequency
Once
Severity
S5 - Suggestion
Reproducibility
Always
Status
closed
Regression
No
Workaround
No
Project
MuseScore

This is a placeholder issue to discuss the work needed for enabling the use of mtests under Windows with MSVC.
At the moment mtests can be compiled and run under Linux (see CI Travis example for the source code).
It was possible to compile and run mtests under MacOS and under Windows with MinGW, but I don't know the state of these (i.e. if they still compile fine and successfully run).
Since the main compiler under Windows has become MSVC (since some time) and a lot of (new) developers use this compiler, it would be very useful to be able to locally check the results of mtests also for this compiler.
I tried an initial work to port the mtests to MSVC.
The results at the moment can be seen here:
https://github.com/AntonioBL/MuseScore/tree/mtestwindows

I used MSVC 2017 and command line (i.e. no Visual Studio interface) to compile them.
Here are the steps:
- Compile MuseScore for Windows (I used Debug version, and I used the msvc_build.bat script) and install it (e.g. to its default location MuseScore\msvc.install_x64
- cd MuseScore\msvc.build_x64\mtest
- cmake --build .
- cmake --build . --target INSTALL
- add the binary dir to the PATH, e.g. PATH=%PATH%;C:\MuseScore\msvc.install_x64\bin
- add the folder where diff.exe for Windows is located (for example msys2/usr/bin if msys2 is installed) to the PATH
- run the tests with ctest (or ctest --output-on-failure)

Comments

• Apr 11, 2020 - 12:33

At the moment of writing, two mtests still fail:
tst_workspaces (Exit code 0xc0000409)
tst_scripting (Failed)

The first one needs a "workspaces" folder with the two (Basic and Advanced) workspaces, one level higher than the place where the tst_workspaces.exe is created. Even when manually copying such a folder, the tests still fail (retrieved names for some palette tests different from expected names)

--I think the second one needs to access the plugin interface. I think it is the "qml" folder of the installation, but I don't know where (and how) it should be copied.--

Edit: Indeed, if I manually copy the folder "qml" one level higher than the folder in which tst_scripting.exe is created, then the scripting tests all pass.

Reply

• Apr 11, 2020 - 20:56

Update:
With the current commit in that branch (I updated, rebased and force-pushed), all tests compile and run without runtime errors.

Only tst_workspaces still fails, but I don't understand how the tests contained in this file work.

Reply

• Apr 12, 2020 - 02:27

The only external resources those tests should need are standard workspaces located in their usual place in mscoreGlobalShare (that is, in $mscoreGlobalShare/workspaces) so that WorkspacesManager can find them. So you should probably either ensure that workspaces files are there or modify tests and mscore code to be able to set mscoreGlobalShare to an arbitrary location for test purposes. Currently mscoreGlobalShare location is defined by the path relative to the executable file location, this may be the reason why the test executable may not find workspaces.

If that doesn't help, could you please post an output of those tests?

Reply

• Apr 12, 2020 - 13:20

Ciao Dmitri,
indeed, I managed to make the test run only after installing basic.workspace and advanced.workspace in a workspace folder placed one level higher than the tst_workspaces.exe executable test (see https://github.com/AntonioBL/MuseScore/commit/ba1d9436394ce3e18db23b1ec… ). Sorry that I was not completely clear in my previous post.
It is the test itself which fails. Attached you can find the log. I really do not understand very much of these workspace tests.
I am also attaching the basic and advanced workspaces which I extracted (with 7-zip) from the installed files before running the tests (<*>_install.xml) and the ref files created by the test. As you can see, they are different (I think the _ref file created by the test correct some of the errors present in the installed files). The <*>_install.xml files are the same as those (zipped and) installed in msvc.install_x64 folder.

Thank you for your help.
Ciao,
ABL

Attachment Size
Basic_install.xml 27.04 KB
basic_ref.xml 26.92 KB
Advanced_install.xml 98.02 KB
advanced_ref.xml 97.57 KB
workspace_log.txt 157.91 KB

Reply

• Apr 13, 2020 - 01:36

A part of the issue seems to be in file permissions: currently workspaces define whether they are read-only or not by looking at their file permissions, see, for example here. But making the installed workspaces files read-only doesn't solve the issue fully for me: I get more tests passed in MSVC but not all of them. I don't know what causes them fail, will need to look at this more.

Reply

• Apr 13, 2020 - 17:16

Thank you.
After setting a "read-only" permission to those workspaces, 10 workspace tests pass and 6 still fail.
The failing ones are those requiring a MuseScore restart.
When looking inside a debugger, the workspaces retrieved after a restart (const auto workspaces = WorkspacesManager::visibleWorkspaces();) seem to be the correct ones, but always with the edited worspaces in position [1].
However, WorkspacesManager::currentWorkspace()->translatableName() seems to always get "Basic" name in every case.

Attached a log where I added a couple of qDebug to print workspace names before WorkspacesManager::currentWorkspace() instructions. I kept only the failing cases and their relative counterpart (passing) in which restart is not called.

Attachment Size
workspaceslog2.txt 10.71 KB

Reply

• Apr 15, 2020 - 17:56

If I understood correctly, the problem is that tst_workspaces is not saving the .ini setting file (e.g. MuseScore3Development.ini) and not reading it either, so the "current workspace" string is set to the default "Basic" instead of what should have been written in the .ini file.

If I tried to force writing of settings (e.g. QCloseEvent event = QCloseEvent(); Ms::mscore->closeEvent(&event); at the beginning of TestWorkspaces::cleanupMuseScore()), but it didn't work, because the settings saving path is empty (I checked with qDebug("--------------- settings path : %s", qPrintable(settings.fileName())); at the beginning of MuseScore::writeSettings()).

Settings PATH can be set to a custom path via command line option "-c" https://github.com/musescore/MuseScore/blob/master/mscore/musescore.cpp… but I don't know if this could be helpful in this case...

Reply

• Apr 15, 2020 - 19:48

Under Linux the preferences are written to $HOME/.config/Unknown Organization/tst_workspaces.conf even when not "forced" with Ms::mscore->closeEvent(&event); (i.e. also for the code as it is in master branch). And such .conf file contains the data with the name of the current workspace, e.g:
[application]
playback\playRepeats=true
workspace=Advanced edited
and it is updated at each of the workspace tests.

Reply

• Apr 16, 2020 - 00:00

Good catch about that "Unknown Organization"! Actually for these tests I intended to put workspace files and settings to a temporary folder so these tests could effectively reproduce a situation of running MuseScore from a clean installation and avoid changing settings for an existing MuseScore installation. For this reason I set dataPath variable to a temporary directory path. Apparently I missed to do the same for QSettings. I attached a patch which ensures that workspaces tests initialization and -c option have the same effect, this would probably be a more correct thing to do for these tests. Given what you have described, this may also help to fix these tests with MSVC.

Attachment Size
tst_workspaces_set_config_folder.patch 2.66 KB

Reply

• Apr 16, 2020 - 00:35

Sorry, the previous version actually didn't have the desired effect. Normally MuseScore forces INI settings format for all platforms except MacOS, this needs also to be done in tests, and for test purposes we probably need that on all platforms. I added setting INI format to the patch, now it seems to use the correct temporary location for settings.

Attachment Size
tst_workspaces_set_config_folder1.patch 3.04 KB

Reply

• Apr 16, 2020 - 15:00

@dmitrio95 : THANK YOU for your work. Indeed, that was the last missing piece needed to have mtests working and passing under Windows (with msvc; and I think for this test also when using MinGW).
If I did everything correctly, I managed to insert your patch as a commit, with you as the author, in the PR I am going to submit in a couple of minutes.

Reply

Fix version
3.5.0