Languages

ARMv7: libmscore tests 74% tests passed

ericfontainejazz

• Mar 23, 2016 - 07:53

This is the first time I ran the libmscore tests on my arch linux ARMv7 machine. I'm just reporting here that "74% tests passed, 12 tests failed out of 46", and that two of those (tst_midi & tst_note) were execeptions. I'm betting this is more the the unspecified char stuff. I'm going to look into tomorrow, but just making a note here, and attaching the ctest output:

ctest-output-ArchLinuxARMv7-commit-5a12dff.txt

Reply

Comments

• Mar 23, 2016 - 08:09

The detailed output of the tests would be needed, like the one Travis generates, showing the diff between what the test got and what was expected.

ericfontainejazz

• Mar 23, 2016 - 08:24

Sorry, here is the output with diff by running "ctest --output-on-failure": ctest-output-ArchLinuxARMv7-commit-5a12dff_with-diff.txt

Note: I always get "libEGL warning: DRI2: failed to open rockchip (search paths /usr/lib/xorg/modules/dri)" whenever I run musescore on this ARM...it is only about my ARM's gpu, which shouldn't have any bearing on these tests...

• Mar 23, 2016 - 09:10

Indeed looks like signed vs. unsigned char problems, -1 turning into 255 on transposing etc.

See https://github.com/musescore/MuseScore/blob/master/libmscore/interval.h…
Seems those need to get turned into signed chars.

As you have a testbed ready, I'd rather leave a PR to you ;-)

ericfontainejazz

• Mar 23, 2016 - 16:24

ok...

sure enough I'm noticing transposing notes down does not work, while transposing up does!

• Mar 23, 2016 - 16:28

and if you change those two to signed char?

ericfontainejazz

• Mar 23, 2016 - 17:04

tst_chordsymbol passed

ericfontainejazz

• Mar 23, 2016 - 17:36

and so did a bunch of others...now the only ones left are:

tst_midimappin
tst_importmidi
tst_capella_io
tst_guitarpro

So thanks that was a good eye to help narrow down...

• Mar 23, 2016 - 20:06

I'd bet this baby needs to be signed too: https://github.com/musescore/MuseScore/blob/master/libmscore/durationty…

Edit: I take that back

ericfontainejazz

• Mar 23, 2016 - 20:11

that one didn't seem to fix any of the errors...

jeetee

• Mar 23, 2016 - 16:51

As I remember, a plain char is undefined and left up to the compiler implementer to decide whether to make it signed or unsigned (the implementer doesn't even have to be consistent about it!).
I believe gcc chose to always turn them signed, but I know from my profession that arm-compilers can go either way. Most of them have a compile flag to force chars to be signed or unsigned though. Depending on the number of plain char definitions you might want to try using that compile flag as the easy route to testing what you want to test.

In the long run, plain chars should all be replaced with their correctly signed type imo.

• Mar 23, 2016 - 17:13

Well, yes, you do remember correcly, almost, whether a char is signed or unsigned is implementation defined behavoir (so inconsistency is not allowed, it needs to be defined, one way or the other) and that's what we know since quite a while, at least since the first ARM raleted Problems cropped up some months ago.
And only those chars that matter need to get fixed, those where it does not, don't. The problem is to tell one form the other :-(

ericfontainejazz

• Mar 23, 2016 - 17:22

yes, exactly!

ericfontainejazz

• Mar 23, 2016 - 17:30

Also, does anyone know a good way to tell qt-creator to compile tests in parallel. I'm noticing on my quad-core machine, that appending -j4 to the tests doesn't really help out significantly, because the linking stage is unnecessary serializer, and because for each test, there are only two cpp files which need to be compiled, hence limiting concurrency to a max of 2, e.g.:

mtest/libmscore/timesig/CMakeFiles/tst_tools.dir/tst_tools_automoc.cpp.o

Now I could just open up 2 or 3 terminal windows and split up list of tests, so can compile with maximum parallelism. But I'm wondering if there is something I can specify in the qt-creator to do this...

ericfontainejazz

• Mar 23, 2016 - 19:12

changing this char to signed has fixed tst_midimapping failures: https://github.com/musescore/MuseScore/blob/5a12dff4765bc924273dbd4fb25…

• Mar 23, 2016 - 19:23

This change is definitly the right thing. I don't understand though how that related to midimapping, this it GuitarPro

ericfontainejazz

• Mar 23, 2016 - 19:33

there was one midi test that involved reading a guitar pro file...test file #14 in midimapping.

Anyway, good news after rerunning all tests, that I now only have two failures:

38 - test_importmidi
39 - test_capella_io

ericfontainejazz

• Mar 23, 2016 - 20:57

these are test_importmidi errors...I've been trying to figure out...no luck so far...

I've traced through the key signature assigining...and I haven't spooted anything unusual...

Attachment	Size
TestImportMidi_errors.txt	23.77 KB

ericfontainejazz

• Mar 23, 2016 - 21:55

fyi, this is what the import midi 15-8 looks like on my ARMv7 ...the diff says error is wrong keysignature...

ericfontainejazz

• Mar 23, 2016 - 21:59

while this is what it looks like in x86-64:

And this is what the reference looks like:

ericfontainejazz

• Mar 23, 2016 - 22:13

I actually think the ARM version is more correct...since 5-flats key signature can handle all these notes... Although i suppose 3-flats key signature would actually be better, since would be the minimum number of flats in key signature... I'm going to look into the logic for detecting the key signature a little more. But what my belief is now is that this line:

https://github.com/musescore/MuseScore/blob/e2c12c016bf03b24cf456664861…

is producing machine-dependent code, because I believe that std::sort is sorting a few different equally-valued options, so there can be different orderings that are both correct. The function then returns element [0], which can thus be different. Anyway, I'm thinking that the test function should be changed so is not dependent on a particular marchine's sort ordering.

ericfontainejazz

• Mar 23, 2016 - 22:18

oh...actually I should investiage this suspect:

QDEBUG : TestImportMidi::metertimeSig15_8() ImportMidi: illegal key 254

ericfontainejazz

• Mar 24, 2016 - 03:34

indeed, after carefully tracing through line by line with both x86-64 and armv7 computers side-by-side, I notice that the execution different exactly at that line before the that QDEBUG statement:

https://github.com/musescore/MuseScore/blob/master/mscore/importmidi/im…

That code need to both explicitly cast that uchar *data into a signed char *, and also needs to store it in a signed char (I've checked that just storing into an int doesn't actually handle the sign extension)... PR comming soon...

ericfontainejazz

• Mar 30, 2016 - 16:54

I'm at the point where everything except tst_capella_io passes: https://github.com/musescore/MuseScore/pull/2509

Attachment	Size
tst_cappella_9fails.txt	272.87 KB

• Mar 30, 2016 - 17:03

all about pitch and tpc, esp. pitch may be yet another signed char issue, tcp might be a follow up error.

Hmm, maybe not. both _tpc and _pitch are int and read as int

ericfontainejazz

• Mar 30, 2016 - 18:20

thanks Jojo. Indeed it was pitch. Pitch of CNote: https://github.com/musescore/MuseScore/pull/2509/files#diff-ab0da389d41…

ericfontainejazz

• Mar 30, 2016 - 18:40

ok, I've just double-checked all tests are passing on my arm archlinux labtop:

[e@alarm mtest]$ ctest
Test project /home/e/build-MuseScore-Desktop-Debug/mtest
      Start  1: tst_album
 1/47 Test  #1: tst_album ........................   Passed    3.03 sec
      Start  2: tst_barline
 2/47 Test  #2: tst_barline ......................   Passed    1.92 sec
      Start  3: tst_beam
 3/47 Test  #3: tst_beam .........................   Passed    2.44 sec
      Start  4: tst_breath
 4/47 Test  #4: tst_breath .......................   Passed    1.80 sec
      Start  5: tst_chordsymbol
 5/47 Test  #5: tst_chordsymbol ..................   Passed    2.00 sec
      Start  6: tst_clef
 6/47 Test  #6: tst_clef .........................   Passed    2.47 sec
      Start  7: tst_clef_courtesy
 7/47 Test  #7: tst_clef_courtesy ................   Passed    1.81 sec
      Start  8: tst_compat
 8/47 Test  #8: tst_compat .......................   Passed    3.40 sec
      Start  9: tst_concertpitchbenchmark
 9/47 Test  #9: tst_concertpitchbenchmark ........   Passed    7.32 sec
      Start 10: tst_copypaste
10/47 Test #10: tst_copypaste ....................   Passed    2.72 sec
      Start 11: tst_copypastesymbollist
11/47 Test #11: tst_copypastesymbollist ..........   Passed    2.28 sec
      Start 12: tst_cursor
12/47 Test #12: tst_cursor .......................   Passed    1.33 sec
      Start 13: tst_dynamic
13/47 Test #13: tst_dynamic ......................   Passed    1.35 sec
      Start 14: tst_earlymusic
14/47 Test #14: tst_earlymusic ...................   Passed    1.81 sec
      Start 15: tst_element
15/47 Test #15: tst_element ......................   Passed    1.35 sec
      Start 16: tst_hairpin
16/47 Test #16: tst_hairpin ......................   Passed    1.30 sec
      Start 17: tst_instrumentchange
17/47 Test #17: tst_instrumentchange .............   Passed    2.04 sec
      Start 18: tst_join
18/47 Test #18: tst_join .........................   Passed    1.93 sec
      Start 19: tst_keysig
19/47 Test #19: tst_keysig .......................   Passed    1.93 sec
      Start 20: tst_benchmark
20/47 Test #20: tst_benchmark ....................   Passed   40.93 sec
      Start 21: tst_links
21/47 Test #21: tst_links ........................   Passed    1.78 sec
      Start 22: tst_parts
22/47 Test #22: tst_parts ........................   Passed    5.60 sec
      Start 23: tst_measure
23/47 Test #23: tst_measure ......................   Passed    2.80 sec
      Start 24: tst_midi
24/47 Test #24: tst_midi .........................   Passed    4.21 sec
      Start 25: tst_midimapping
25/47 Test #25: tst_midimapping ..................   Passed    8.12 sec
      Start 26: tst_note
26/47 Test #26: tst_note .........................   Passed    1.89 sec
      Start 27: tst_plugins
27/47 Test #27: tst_plugins ......................   Passed    1.37 sec
      Start 28: tst_repeat
28/47 Test #28: tst_repeat .......................   Passed    2.73 sec
      Start 29: tst_selectionfilter
29/47 Test #29: tst_selectionfilter ..............   Passed    2.67 sec
      Start 30: tst_selectionrangedelete
30/47 Test #30: tst_selectionrangedelete .........   Passed    2.01 sec
      Start 31: tst_spanners
31/47 Test #31: tst_spanners .....................   Passed    2.11 sec
      Start 32: tst_split
32/47 Test #32: tst_split ........................   Passed    1.95 sec
      Start 33: tst_splitstaff
33/47 Test #33: tst_splitstaff ...................   Passed    1.80 sec
      Start 34: tst_timesig
34/47 Test #34: tst_timesig ......................   Passed    2.06 sec
      Start 35: tst_tools
35/47 Test #35: tst_tools ........................   Passed    3.18 sec
      Start 36: tst_transpose
36/47 Test #36: tst_transpose ....................   Passed    1.33 sec
      Start 37: tst_tuplet
37/47 Test #37: tst_tuplet .......................   Passed    1.33 sec
      Start 38: tst_text
38/47 Test #38: tst_text .........................   Passed    2.05 sec
      Start 39: tst_importmidi
39/47 Test #39: tst_importmidi ...................   Passed    4.38 sec
      Start 40: tst_capella_io
40/47 Test #40: tst_capella_io ...................   Passed    2.52 sec
      Start 41: tst_biab
41/47 Test #41: tst_biab .........................   Passed    1.86 sec
      Start 42: tst_mxml_io
42/47 Test #42: tst_mxml_io ......................   Passed   38.31 sec
      Start 43: tst_guitarpro
43/47 Test #43: tst_guitarpro ....................   Passed    9.17 sec
      Start 44: tst_scripting
44/47 Test #44: tst_scripting ....................   Passed    1.84 sec
      Start 45: tst_ove_bdat
45/47 Test #45: tst_ove_bdat .....................   Passed    2.58 sec
      Start 46: tst_ove_ove3
46/47 Test #46: tst_ove_ove3 .....................   Passed    1.89 sec
      Start 47: tst_ove_structure
47/47 Test #47: tst_ove_structure ................   Passed    2.00 sec

100% tests passed, 0 tests failed out of 47

Total Test time (real) = 198.84 sec

Do you still have an unanswered question? Please log in first to post your question.