Saving segment positions with MuseScore 3 leads to elements with sx attributes being mostly 0

• Jun 27, 2020 - 19:07

(I wasn't sure if I should post this here, on in the more general user support forum, but felt like this would probably be the most fitting sub-forum given it's low level / development nature. But feel free to move it somewhere else)

Hi,

I use MuseScore in command line mode to export .mpos and .spos files with measure and segment positions.
For the .mpos files, both MuseScore 2 and 3 do their work correctly (with a small change in positions, but that's probably not a bug, more a small difference in the rendering).

But for the .spos files, MuseScore 3 outputs elements that have mostly an sx="0" attribute. In fact, I think the only elements that don't have an sx of 0 are rests (but not entirely sure about that).
So there is now no way any longer to know the correct bounding box of the segments.
--> I believe this is a bug, isn't it? Can this be fixed somehow?

I did my tests with MuseScore 2.3.2 and MuseScore 3.4.2.9788, on Windows 10 64-bit latest updates.
These are the commands I used:

"C:\Program Files (x86)\MuseScore 2\bin\MuseScore.exe" -o test-ms2.spos test.mscz
"C:\Program Files (x86)\MuseScore 3\bin\MuseScore3.exe" -o test-ms3.spos test.mscz
"C:\Program Files (x86)\MuseScore 2\bin\MuseScore.exe" -o test-ms2.mpos test.mscz
"C:\Program Files (x86)\MuseScore 3\bin\MuseScore3.exe" -o test-ms3.mpos test.mscz

If you look at the test-ms3.spos file, you'll see that sx is 0 for most elements...

I'm attaching the test file + the obtained result files for your convenience and for reproduction purposes
(note that I had to post-fix the names of the .spos and .mpos files with .txt because the forum won't allow me to upload these types of files).

Attachment Size
test.mscz 42.18 KB
test-ms2.spos_.txt 74.7 KB
test-ms3.spos_.txt 71.76 KB

Comments

Frankly, I had no idea such options were supported, and am not surprised at all they don't work - pretty much never do we pay attention to that code when making changes. If you explain you are relying on this for, maybe we can help you find a better way.

In reply to by Marc Sabatella

Hi Marc,

with that functionality, we can generate .png images from a MuseScore file + have a corresponding mapping of wall-clock time to position (note or measure rectangle) on the image. We need this for a (still beta version of) our score follower. You might remember Thomas and Nicolas doing demo's with this technology connected to MuseScore several years ago.

We actually also have C++ code to import scores from a user's MuseScore account (after authorizing with OAuth), but apparently the MuseScore API is now no longer working, so this now also no longer works in our application. It was convenient for a user to import his/her own scores that way, but I can understand that decision...

So, the code was changed to start MuseScore locally in a separate process to perform the conversion and generate the .png, .wav, and .mpos/.spos files locally on the user's own computer from a local MuseScore file. And then I noticed the .mpos file is still OK in MS3, but the .spos file has these widths of 0 (which was not the case in MS2). It looks like the bounding box calculations for notes doesn't work any longer as it used to and leads to zero rectangle widths for the elements (except for rests, I think)?

I had a quick look at the code in MuseScore, and noticed some things are done while iterating over the score elements, but it wasn't clear to me what was going on exactly and why this would now fail in MS3...

Hopefully there is a simple fix?
In any case, thanks for your fast reply.

Kind regards,
Koen

In reply to by KoenTanghe

I do remember that project, very cool! Well, like I said, to my knowledge, no one has maintained the part of code responsible for writing these tags, and much has changed about the layout algorithms, so it doesn't surprise me things don't work anymore. Probably best to have someone on your team figure out what is needed and implement it themselves, we can try to help but you know your requirements between than we do.

In reply to by Marc Sabatella

Sorry for the late reply.

The requirements are simple, really: element tags should have a non-zero widths (sx attributes) when .spos files are generated (just as it was done correctly in MuseScore 2, and as it still is done correctly for the elements when an .mpos file is generated).

It's a bit daunting for an outsider to have to setup an environment that can build MuseScore from source, and then try and figure out what might have changed in the source code compared to previous releases that now leads to these zero widths for elements.

I did check out the repository and had a look which code actually generates these .mpos and .spos files, and from what I could see. it's the methods saveMeasureEvents and savePositions defined in the source file mscore/savePositions.cpp that does this, so probably something changed in there (or in the underlying way element bounding boxes are calculated). These methods get called from doConvert in mscore/musescore.cpp.

So, I think for now, I'll submit a bug report in the issue tracker with this information, and hope one of the current developers can find a brief moment to look at the existing code and knows what might have changed compared to MuseScore 2 (and why it still works for measure elements, but not for note elements).

In reply to by KoenTanghe

Understood that going in directly might be time consuming, but realize also, most of the rest of us have literally no idea what those numbers were supposed to be. Like how they were supposed to calculated, what the units are, etc. So even just a really good precise description of what the numbers are supposed to represent would help.

Hi,

sorry to bother about this again, but, can you perhaps share with us publicly what you shared about your use cases with Emmanuel, so that anyone else wanting to pick this up has the same information?

In reply to by mirabilos

Hi mirabilos,

I believe everything we talked about is already in my original post + my comment above from June 29, 2020 with a bit more use case info.

I also attached the files to my original post + the commands to reproduce the issue showing the difference between MS2 and MS3.

Maybe this phrase I sent to Emmanuel as part of our private conversation helps for you to understand the use case better (but to me, it's essentially the same as above, just in other wordings):

"What our music follower needs is a list of events with a timestamp and a rectangle I can draw over the score image. The rectangle should be an indication of what is currently playing, so it could be a single note or a (partial) chord, ... I would expect that that rectangle encloses the note or the chord.

I think the way it was when I was still using MS2, was that the width of the rectangle was more or less the width of the note or the chord, and the height was fitting the system height, so you would get a rectangle that was always the same height (it doesn't matter if it's a low note or a high note), but the width would depend on what was in the rectangle (note, chord, ...) We also extend the height a little bit so it exceeds the highest and lowest horizontal staff line."

I can also show you the relevant piece of our code that is consuming the .mpos or .spos files (I had already shared the relevant developer note in the other post at https://musescore.org/en/node/311914 but maybe this helps too):

// Add the (place) regions
const double MuseScoreDPI = 360;
const double ExportDPI = m_ImageResolution;
const double xFactor = 1.0 / (pageWidth  * (12 * MuseScoreDPI / ExportDPI));
const double yFactor = 1.0 / (pageHeight * (12 * MuseScoreDPI / ExportDPI));
const double extraFractionAtTopAndBottom = 0.2;
for (XmlElement* elementElement = elementsElement->getChildByName("element"); elementElement != nullptr; elementElement = elementElement->getNextElementWithTagName("element"))
{
    MusicFollowingPlaceMap::Region region;
    region.ID = elementElement->getIntAttribute("id");
    region.Page = elementElement->getIntAttribute("page");
    region.Type = "Rect";

    /* More info on the coordinate conversions for the positions of the elements / measures:
    //     mpos.x = frac.x * PageWidth  * (12 * MuseScoreDPI / ExportDPI)
    //     mpos.y = frac.y * PageHeight * (12 * MuseScoreDPI / ExportDPI)
    // or also:
    //     frac.x = mpos.x / (PageWidth  * (12 * MuseScoreDPI / ExportDPI))
    //     frac.y = mpos.y / (PageHeight * (12 * MuseScoreDPI / ExportDPI))
    // where:
    //     mpos.x / .y are the x and y coordinates in the .mpos file extracted from MuseScore
    //     PageWidth and PageHeight are in pixels
    //     MuseScoreDPI is 300
    //     ExportDPI is 100 by default      */
    double x = elementElement->getDoubleAttribute("x");
    double y = elementElement->getDoubleAttribute("y");
    double sx = elementElement->getDoubleAttribute("sx");
    double sy = elementElement->getDoubleAttribute("sy");
    float X = (float)(x * xFactor);
    float Y = (float)(y * yFactor);
    float W = (float)(sx * xFactor);
    float H = (float)(sy * yFactor);
    region.Rect.Set(X, Y - H * float(extraFractionAtTopAndBottom), W, H * float(1 + 2 * extraFractionAtTopAndBottom));
    display.PlaceMap.Regions.push_back(region);
}

Note as already mentioned in the other post as well:

"But I also see in the code surrounding it that we currently set a const as follows:
const double MuseScoreDPI = 360;
and I found a commit log message (from last June actually) saying this:
"adjusted assumed MuseScore export dpi for PNG files to 360 (default value for MS3; was 300 before)
--> fixes issue with incorrectly placed bounding boxes""

For me, the mpos file still works fine (even though there is that strange factor 12 in there) as we use fractional values anyway. So for me, there is no real need to change the .mpos file (unles you want to make it better/more general for some reason).

This post was about the .spos files which are broken in MS3 and were still working fine in MS2.
I think I gave all the info I can give...

If I understood it correctly from what I read, Emmanuel already did a lot of work to get a fix for this, but due to some disagreements and then a commit of untested code you had made, things are now a bit in a "twilight zone", right?
I don't know the exact details (and I should not interfere in that) but I just hope my input here helps to get to a fix at some point, hopefully with not too many changes.

In reply to by KoenTanghe

Hi Koen,

thanks for the answer!

> I think the way it was when I was still using MS2, was that the width of the
> rectangle was more or less the width of the note or the chord, and the height
> was fitting the system height, so you would get a rectangle that was always
> the same height (it doesn't matter if it's a low note or a high note), but

Is this important to you? The experiment I’m trying for 3.x would not give same-height rectangles every time; I’m trying to make them so to encompass everything (except things that deviate horizontally, like tempo text).

> "adjusted assumed MuseScore export dpi for PNG files to 360 (default value
> for MS3; was 300 before)

The new formats will have values exactly corresponding to pixels in the PNG generated from the same settings the mpx/spx file was generated, so no scaling needed at all.

For when you want to scale, it will also include the maximum page width/height, so then you can scale as 'filevalue * yourpagesize / filepagesize'.

> For me, the mpos file still works fine (even though there is that strange
> factor 12 in there) as we use fractional values anyway. So for me, there is
> no real need to change the .mpos file (unles you want to make it better/more
> general for some reason).

For me, dropping the factor of 12 and fixing the values to whole integers is important. But the old format with 12-scaled values will stay available, as it apparently has consumers; it is, however, also extended to have the page width/height in the header, as requested by a MuseScore core developer.

> If I understood it correctly from what I read, Emmanuel already did a lot of
> work to get a fix for this, but due to some disagreements and then a commit
> of untested code you had made, things are now a bit in a "twilight zone",

Emmanuel did a lot of work trying to fix it, and he got values that “seem” to fit, but the calculations sound “not right” (e.g. width of smallest element plus(!) width of largest element), and they are based on the skyline shape (which excludes some elements).

I asked in IRC whether he’d be willing to try something (as he had a compile and test setup already) and hacked an alternative approach, and rather than understanding that this was an idea expressed in code, he complained about me sending untested code (which I did, because to me it was clear that this is an experiment in the style “hey, look at this, couldn’t something like this work?”). We evidently had multiple communication problems (both of us are not English and differ in natural language), but he was so put off he stopped working on this.

I’m a bit too perfectionist, so I’m personally wary of pushing his solution, even as intermediate. I do intend to come back and fix this completely, because I also wish to eventually implement a score player, but I’m sorry to say it doesn’t have top priority (I only became active because I had thought Emmanuel had fixed it), so if someone else can take it further in the meantime, I’ve published everything.

I don’t have enough spare time right now, but when I get it, I’ll be working on this (the MuseScore part of my TODO currently doesn’t have any items ranked higher than this other than “push the stuff already merged in one branch to the other”, but I also have other TODOs with a small number of more urgent items, plus $dayjob where I direly need to put some billable hours in; I’ve worked more than half of the night on that experiment)… but if you can wait one to two months or so more, I should be able to continue working on this personally. (I cannot promise results yet, but effort.) Now that I actually took some time looking at the code, I’m hooked…

In reply to by mirabilos

>> the height
>> was fitting the system height, so you would get a rectangle that was always
>> the same height (it doesn't matter if it's a low note or a high note), but

> Is this important to you? The experiment I’m trying for 3.x would not give same-height rectangles every time; I’m trying to make them so to encompass everything (except things that deviate horizontally, like tempo text).

Well, that's how it was in MS2, both for the .mpos and .spos files, and for good reason I would think: as explained, the goal of these files (as I understood from Nicolas and Thomas back in the days) is to be able to easily draw a rectangle around or behind the relevant part of the score as a kind of "cursor" moving forward (a rough 1-measure wide cursor for the .mpos file, and a finer note/segment-level cursor for the .spos file).
If the height would not have been constant all the time, then that would mean your rectagle would constantly jump up and down while it's moving forward, which is not desirable... I can understand that some people may also want to get the exact bounding boxes of the individual notes/segements in all directions, but that is not how is was envisioned for the .mpos / .spos files.
So, I would say: yes, that is what I expect (and has always been the case for these files).

For me personally, having the have the page width/height in the header and getting everything in pixel coordinates in the exported image could still be good, even though that changes the format then and it would require some change in my code then, where I would need to split the code after it first checks which type of mpos/spos is being generated by the particular MS version the user has installed on his system. So, more hassle for me, but still manageable I guess. In any case, in the current MS3 version, the spos file is completely broken anyway due to the width (almost) always being 0.

For me, if this gets solved in a cleaner, more general way, it's all fine, as long as it's still easy to get the info I need for driving a moving rectangle of fixed height around the measure or the note/chord/segment from the mpos or spos file, preferably without too many necessary modifications in my code. If the format can remain as it was (and how it worked fine for me), even better.

Koen

In reply to by KoenTanghe

> the goal of these files (as I understood from Nicolas and Thomas back in the days) is to be able to easily draw a rectangle around or behind the relevant part of the score as a kind of "cursor" moving forward

That would be my goal, too.

> If the height would not have been constant all the time, then that would mean your rectagle would constantly jump up and down while it's moving forward, which is not desirable

Ah!

Thanks for this. Yes, I can now see why the extra precision would be unwanted.

> In any case, in the current MS3 version, the spos file is completely broken anyway due to the width (almost) always being 0.

Yes, assume working files are from MS2 ;-)

But the new format will contain information that allows you to detect it (an extra attribute on the top-level XML element).

> For me, if this gets solved in a cleaner, more general way, it's all fine, as long as it's still easy to get the info I need for driving a moving rectangle of fixed height around the measure or the note/chord/segment from the mpos or spos file, preferably without too many necessary modifications in my code. If the format can remain as it was (and how it worked fine for me), even better.

OK, then we’ll keep the old format around, and other than getting the extra page size information, you’ll have the same data. (The page size information makes it so that you won’t need to know the export DPI any more, you can just compare expected and actual image sizes.)

Do you still have an unanswered question? Please log in first to post your question.