Expose Element.posX & Element.posY values to plugins

• Aug 2, 2019 - 17:07
Reported version
3.2
Type
Plugins
Frequency
Once
Severity
S5 - Suggestion
Reproducibility
Always
Status
closed
Regression
No
Workaround
No
Project

A method is needed for plugins to accurate locate laid out positioning for elements like grace notes so that other information like STAFF_TEXT can be visually placed properly.

One example for this need is to position a tin whistle finger hole pattern directly under
a grace note.

Another example is to place note names properly using MuseScore's shipped notename.qml plugin.

Heuristic approaches just don't work well. Here is what the tin-whistle-tab.qml produces when heuristics are used and the score was manually edited for irregular rendering:

HeuristicPlacementWithoutPosExposure-20190802.png

Here is the same score using exposed element.pos information to position the tab images:

PlacementWithPosExposure-20190802.png

Perfect!

-Dale


Comments

Hello,

I've now updated the shipping notename.qml plugin to utilize the Element.posX/Y exposure.

Here is a bit of score listing note names for grace notes using the existing notenames.qml:

NoteNamePluginWithoutPosExposure-20190802.png

Here is the same score using the updated notename-v2.qml (attached to this posting) that uses the exposed element.posX/Y values:

NoteNamePluginUsingPosExposure-20190802.png

Here's a another example of chord stacks... As before here it is with the shipping notenames.qml:

ChordsNoteNamePluginWithoutPosExposure-20190802.png

and here are the results using element.posX/Y exposure:

ChordsNoteNamePluginUsingPosExposure-20190802.png

-Dale

Very cool!

On Telegram, I recently mentioned that if we could get the "physical" size of an element (ie, its bbox, or separate height and width), this could be useful as well. In particular, I could imagine a poor man's word wrap function. You'd have to select a text element first and run it manually of course, but it could still be quite useful I think. Does this sound plausible?

If you're interested in exploring the possibilities I've created commit on top of this issue's work that provides read-only access to this Element values:

  /**
   * Reference position of this element relative to its parent element.
   *
   * This is an offset from the parent object that is determined by the 
   * autoplace feature. It includes any other offsets applied to the 
   * element. You can use this value to accurately position other elements 
   * related to the same parent.
   *
   * This value is in spatium units for compatiblity with Element.offsetX.
   * \since MuseScore 3.3
   */
  Q_PROPERTY(qreal posX READ posX)
  /**
   * Reference position of this element relative to its parent element.
   *
   * This is an offset from the parent object that is determined by the 
   * autoplace feature. It includes any other offsets applied to the 
   * element. You can use this value to accurately position other elements 
   * related to the same parent.
   *
   * This value is in spatium units for compatiblity with Element.offsetY.
   * \since MuseScore 3.3
   */
  Q_PROPERTY(qreal posY READ posY)
  /**
   * Spatium value for this element.
   *
   * \since MuseScore 3.3
   */
  Q_PROPERTY(qreal spatium READ spatium)
  /**
   * pos value for this element.
   *
   * \since MuseScore 3.3
   */
  Q_PROPERTY(QPointF pos READ pos)
  /**
   * Bounding box for this element.
   *
   * \since MuseScore 3.3
   */
  Q_PROPERTY(QRectF boundBox READ boundBox)
  /**
   * Canvas position for this element.
   *
   * \since MuseScore 3.3
   */
  Q_PROPERTY(QPointF canvasPos READ canvasPos)
  /**
   * Page bounding rectangle for this element.
   *
   * \since MuseScore 3.3
   */
  Q_PROPERTY(QRectF pageBoundingRect READ pageBoundingRect)
  /**
   * Canvas bounding rectangle for this element.
   *
   * \since MuseScore 3.3
   */
  Q_PROPERTY(QRectF canvasBoundingRect READ canvasBoundingRect)

Here is the branch you can use to test the new proposed plugin apis.

https://github.com/DLLarson/MuseScore/tree/plugin-expose-element-pos-in…

It builds on the PR for this issue so you would get the posX/Y features as well.

Unfortunately you will have to build this patch on your own as I don't want to send this on to the main MuseScore repository without a feature request issue to hang it on. I don't want to muddy up this particular issue.

If you would like to compose a feature request I would be happy to move this info into that place for discussion. At that point I can create a WIP pull request to the main repo where it can be tracked.

I've also attached a plugin script that just dumps all this data to the console to see what's provided.

-Dale

Yep... I'm in the middle of exposing the tie stuff. Just started testing. It would be good if we had an actual issue to hang the pull request on for it though. A sponsor so-to-speak. ;)

I don't want to muck up this issue with the other.

-Dale

Yeh, keep them separate. The real issue is being able to determine who notes' whole family are unambiguously, it's really that simple. "Expose ability to determine the tie siblings of a note". Ipso facto.

In reply to by BSG

I hope you read my (this morning version) of "adjusting tied notes", whose upshot is that the situation is such a tangle that both the user and any code that deals with tied notes has to be able to ascertain the truth of the situation. Right now, the user (of plugin) knows, but the code doesn't!

Status active fixed

Fixed in branch master, commit 6e46f20491

_Fix #292900: Expose Element.pos X/Y values to plugins.

Expose Element.pos X/Y values so that plugin's can accurately
position things like STAFF_TEXT on the score. One example is
to position a tin whistle finger hole pattern directly under
a grace note. Another is to place note names properly using
MuseScore's shipped notename.qml plugin._

Fixed in branch master, commit 26ab4f5143

_Fix #292900: Modify plugin notenames.qml for element.posXY support

Modify plugin to take advantage of exposure of element.posX/Y which
allows for precise positioning of elements like STAFF_TEXT._

In reply to by BSG

>I hope you read my (this morning version) of "adjusting tied notes", w

Yes I did. I did expose the Tie stuff as well.

I think it needs some exercise to determine usefulness.

I just pushed it up to my fork of the MuseScore repo:

https://github.com/DLLarson/MuseScore/tree/plugin-expose-note-ties

I won't make it a PR until it has a issue for it though.

This issue has now been closed as they merged my changes to master.

-Dale

Oh, beautiful! Just did some flute composing (see my ms page); have to get back to code..... thanks so much!

I'm testing this, and I see a problem that I feared. None of the notes it ever returns have the same address, implying that you wrap notes in some newly-allocated QML wrapper each time, which makes it difficult to make lists of notes, see if you have them or already processed them, or what. Since you are effectively returning list structure in piecemeal form, it is essential to preserve identities in some way; now, I imagine you don't want to maintain a map of objects you've returned, not knowing what's going to happen in the future -- could be large. I suggest a simple solution, note.id, the physical 64-bit address of the real thing it represents, such that a plugin can make such a map if it needs it or otherwise identify two representations of the same object as being that. "EQ vs EQUAL" in Lisp.... How, for that matter, do you know what internal object I'm talking about if I call in with an object created in such a fashion right now? There must be such a map already.

>implying that you wrap notes in some newly-allocated QML wrapper each time,

I intentionally didn't mention it to see if it would be noticed. And....Ding...Ding...Ding!!!!

Yes! You are correct, in fact near as I can tell it's always been true. This is the first time I wanted to make a decision based on comparing notes and found it useless.

I'm looking to see if Qt exposes a unique object id rather than creating something. The obvious need is to compare the addresses of what the wrappers are pointing at. This isn't a unique problem so I'm going to look at what the QML/C++ integration stuff does (or doesn't).

-Dale

In reply to by DLLarson

My career as an operating systems programmer and Lisp wizard was EXTENSIVE -- leave it at that. But you didn't answer my last question -- how, when i call in to set some property of a playEvent or a note, do you know which internal object I'm talking about? The back-pointer must be in there somewhere.

In reply to by BSG

>My career as an operating systems programmer...

Time for a throw down.... ;)

>how, when i call in to set some property of a playEvent or a note, do you know which internal object I'm talking about? The back-pointer must be in there somewhere.<

Of course there is!

We just have the n-wrappers to one object mapping that Javascript isn't aware of. It's a MuseScore contrivance--kinda like onRun. I don't see a language supported method so...

If the underlying Qt objects were directly exported to QML like I think it was in MuseScore 2 you wouldn't have the problem since there is no wrapper.

At this very moment I'm testing exposing 'Element.elementId' on all elements which, essentially gives you the underlying object address. When I'm happy with it I push the change up and drop a note.

-Dale

Sounds good... then js can build a hash-table on it or whatever is necessary... js has to do the garbage-collection -- the C++ side can't know if these objects are in use or not, so it can't efficiently maintain its own hash-table.

I'm back!

I just pushed up a replacement commit for the branch. It add elementId to all elements. I will tell you that 64bit addressing is a PITA when trying to make sure you don't overflow numbers. C++ is VERY picky about even type casts now!

https://github.com/DLLarson/MuseScore/tree/plugin-expose-note-ties

I've attached a new version of the test script at is seems to work very well a discriminating elements.

>then js can build a hash-table on it or whatever is necessary... js has to do the garbage-collection -- the C++ side can't know if these objects are in use or not, so it can't efficiently maintain its own hash-table.<

The plugin framework already handles that (thankfully!) all wrappers are owned by QML engine. The objects they refer to are another matter but also accounted for.

Give it a go!

-Dale

Attachment Size
tie-tests-v2.qml 3.56 KB

Looks good so far -- attaching my enhanced tie-tests-v3.qml . I will try to create something useful. Setting off-time on all the notes in a range is probably buggy now if it encounters tied notes, and that can be fixed with these new exposures ...

Attachment Size
tie-tests-v3.qml 4.02 KB

Massive confusion. How does the current QML support deal with circular structure? Tied notes obviously contain (perfectly valid in C++) circular references, "you're my next-tier, I'm your prev-tier" (which are beastly in German). When, in the course of a plugin's run, does it instantiate the wrapper objects? If I ask for the selected object, and it is a note, I get a Note-wrapper object. Does the tieForward field exist, or get filled in, until I ask for it? And since I ask by property-field (a.b) references, not function calls, how does the JS framework know that my reference to the tieforward field needs a call to C++ to resolve it? If such fields are really hidden function calls, then chasing circular pointers will produce a stream of new wrapper-objects (although elementId can deconfuse them). If not, how is either (a) instantiating infinite objects at once, which it clearly doesn't, or ,(b) keeping the previously-discussed cache on the C++ side to avoid (a), done?

Executive summary -- do the wrapper objects corresponding to the real objects referenced by Note (or any element) exist at any time earlier than my reference to the object properties pointing to them?

In reply to by BSG

Well, I did the experiment and answered my own question, but QML authors have to understand this. Yes, every reference to foo.bar produces a new wrapper object; two successive js references to note.tieForward produce two different js objects; a static-looking javascript reference is actually a call (like Python @property functions). Were this not so, exposing objects with deep or circular structure would not be possible. The management of identity via elementId needs documentation.

Are there any fields of a QML object representing an MS element, other than elementId and anything a user stores in it, which are actually in the js object, and not implemented by calls in?

> do the wrapper objects corresponding to the real objects referenced by Note (or any element) exist at any time earlier than my reference to the object properties pointing to them?<

No. They are created on-demand. They are also cleaned up via garbage collection (gc) or when the QML engine destroyed.

>Are there any fields of a QML object representing an MS element, other than elementId and anything a user stores in it, which are actually in the js object, and not implemented by calls in?<

I'm having a problem parsing this question.

My best stab...

FIrst you can't change the elementId. It's read-only and invariant. It is literally the address of the MuseScore object pointed to by the wrapper object. In addition wrapper objects are always owned by the QML engine so they will be garbage collected when nothing references them and a gc is triggered. The thing they are pointing to may or may not be owned by the QML Engine. If you create an element with newElement it is owned by the QML engine until it is added to the score. Then it switches ownership to the C++ code so the QML engine won't delete it during wrapper gc.

In general the only thing in a wrapper is the pointer to the wrapped object. It's all delegation to the wrapped object after that.

-Dale

In reply to by DLLarson

You are saying that even atomic-object fields (e.g., "pitch") are delegated to C++ to read that field at access time? I think I understand this now. Hopefully, js can store its own values in these wrappers (e.g., "I have processed this one in my way") once they are uniquized, and pointers saved (inhibiting GC) only then...

>You are saying that even atomic-object fields (e.g., "pitch") are delegated to C++ to read that field at access time?

Yes. Exactly. Very thin interface.

>Hopefully, js can store its own values in these wrappers

Why would you want that? I'm missing the utility here.

Are you wishing you could amend the functionality of the wrapper's Javascript object?

You can do stuff like this:

 var note = notes[idx];
 note.joe = "this is a test." // Create new property named 'joe'
 console.log("note.joe=" + note.joe) // It exists here.
 console.log("note.firstTiedNote.joe=" + note.firstTiedNote.joe) // But not here since a different wrapper to same note (in this case anyway).

I don't think you can add functions/properties to the underlying wrapper 'class' (or prototype in Javascript parlance.)

-Dale

In reply to by DLLarson

Note.joe is a property of the wrapper. That's exactly what i was asking, the object, not the prototype. That's fine. That's very useful, that I can hang my own notations and pointers off of it (once uniquized, of course). The behavior shown (non-uniquized) in your example is exactly what I would expect at this point.

Ohhh. I forgot to mention...

I pushed up an updated commit for this work. It moves the elementId up the class hierarchy to ScoreElement so Parts and Score's can be compared as well.

-Dale

I'd like to know more about why the change was made to factor out the plugin api from the base objects. Not being able to tell that two objects represent the same, say, note, is a crippling restriction in the QML, preventing associating any plugin data, during plugin time, with the note, whether by property or hash-table. Always returning the same object would be ideal, but that cannot be done without either a hash table or one little unobtrusive pointer at the bottom of each real object, and a std::set of all of them set during a plugin run so they can be cleared reliably when the plugin exits. What happens to the hash table or pointers when Javascript garbage-collects is tricky; the JS/Qt objects that get destructed have to clear the pointers in their real avatars. This can be designed, ElementId is a kludgy solution with problems, as you point out, but the need for persistent objects on the JS/QML side is real.

In reply to by BSG

>Are we having the discussion here about object identity?

I hope so! The PR was just inspected.

The elementId addition was pulled from the commit due to objections until a discussion takes place. I've directed the GitHub PR to here for the discussion. Let's see what happens.

https://github.com/musescore/MuseScore/pull/5258

Check out dmitrio95's comments there.

We may ultimately want a separate issue for the discussion though. This particular issue is closed.

-Dale

In reply to by DLLarson

My above remarks are in response to his comments; I await his further comments. The world has not ended for want of ties or element-id; as long as the infrastructure supporting my posted plugins continues to work in 3.3, I'll be happy -- for the while.

Fix version
3.3.0