An approach is needed to identify unique MuseScore objects from within the PluginAPI.

• Aug 27, 2019 - 20:03
Reported version
3.2
Type
Plugins
Frequency
Once
Severity
S2 - Critical
Reproducibility
Always
Status
closed
Regression
No
Workaround
No
Project

The MuseScore 3.x plugin QML/Javascript implementation uses wrapper objects to delegate property/method calls to the actual underlying MuseScore C++ objects. The current wrapper strategy creates new wrappers whenever needed which leads to many wrappers possibly pointing to the same internal MuseScore object. This makes it impossible to compare wrappers to see if they point to the same MuseScore object.

This appears to be caused when during the MuseScore 3.x upgrade which created the wrapper based approach.

My initial approach was to export an elementId property for all elements that would provide a unique value for the object. This unique number initially used the object's RAM address--Note that it doesn't have to be that value if a better method to derive a value is provided.

To get started I'm submitting elementId approach as a "straw man" to get some discussion for a better solution.

-Dale


Comments

Comments from "dmitrio95" regarding this issue found in pull request:

https://github.com/musescore/MuseScore/pull/5258

Could you please move this change to a separate pull request? Other changes in this PR are certainly good and can be merged once mostly minor implementation-related issues are resolved, but I would rather avoid using elementId if it is possible at all.

The reason I don't believe this to be a good solution is that elementId value can easily be stored (and seems natural to be stored) without a wrapper object itself. As it is some memory address it can lead to mistakes while trying to indentify objects. For example, if we take elementId of some element which later gets deleted for some reason, its memory address could be (and is likely to be) reused by another element created later. Comparing elementId of this element with the stored one will identify this as the same element as the old one while it is essentially different and can potentially even have a different type.

So I would consider exactly this change debatable and would try to consider other solutions first (like ensuring that the same wrapper is always returned for the same element or exploiting valueOf() for Element type to make it possible to compare different wrappers using == (but not === in this case) operator).

-dmitrio95

What is "valueOf"? I gather you (DLL) looked for it and found naught.... I see -- it's a JS primitive that is delegated to objects to handle however they want. Not clear how that fits into this issue.

In reply to by [DELETED] 1831606

>What is "valueOf"? I gather you (DLL) looked for it and found naught...

It just prints out the class name of the wrapper class along with the address of the wrapper.

I left this note at the old PR regarding valueOf:

"I don't see this implemented in Element in either namespace. It looks like it belongs to Javascript but I don't see how it drills down into the Element held by a wrapper."

It could work if there was a way to get the JavaScript engine to call the wrapper somehow to get valueOf(). But it would still involve addresses in the produced value.

-Dale

Status PR created active

Actually my idea with defining a valueOf() function turns out to be incorrect. The idea was based on the fact that when comparing two variables of different types in JavaScript using == operator the engine tries to convert them to some common type: to Number using valueOf() or to String using toString(). If the same worked for objects then we could define such a conversion in a way that would allow a comparison be performed in the way we need. However when comparing two variables of the same type == is defined by the language standard to be the same as strict comparison (===) which, in case of objects, returns true only if the same object is compared with itself. This makes the entire idea not working.

So far I see the following ways to resolve the issue:

  1. Define ScoreElement.is() function
    We could perform a comparison via a specially defined function:
var score1 = curScore;
var score2 = curScore;
if (score1.is(score2))
    console.log("this is the same score");

This is essentially the same as an elementId approach except that the actual value used for a comparison is never exposed to a plugin. This resolves the main downside of the elementId approach as we can usually safely assume that the object that is referenced in a plugin does actually exist and is not deleted.

  1. Implement extension objects instead of wrappers
    QML allows defining extension objects for arbitrary types in order to define some QML-accessible properties for it without modifying its source code. That would potentially allow both manipulating the original objects from libmscore and leave the API implementation separate from libmscore. However extension objects can define only additional properties but not additional functions. This would force us to move Q_INVOKABLE functions defined in plugins API directly to the types in libmscore which makes this solution not much better then the following one:

  2. Get rid of wrappers and move plugins-related properties and functions back to libmscore
    The main issue here is that plugins API becomes tightly coupled with libmscore structure and this adds a lot of limitations both on libmscore development and on plugins API. Functionality exposed to plugins has some specific requirements: it should always track the object's ownership (plugins vs. C++ code), changes made to a score from plugins should always be undoable, it should always check input values to make it impossible (or at least harder) to crash MuseScore, it sometimes uses different measurement units (like spatium for offset of elements in plugins vs. pixels for the same purpose in libmscore), and generally ease of use has somewhat more priority over a performance there. On the other side, plugins API should stay compatible within the whole 3.X release cycle which would also limit the development of libmscore.
    While it would probably be good to structure libmscore in a way that it starts having a stable API and becomes easier to use, this would currently require a lot of changes. Simply moving Q_INVOKABLE functions from plugins API to the respective types in libmscore would currently cause name conflicts which will be difficult to resolve. For example, the recently added Chord.add() function does not really correspond to Chord::add() function in libmscore: Chord::add() is designed to be a lower-level function which does not take undoability of operations into account and does not try to ensure the correct state of the score after it is executed. Renaming this function in plugins API would complicate the plugins API, and renaming that function in libmscore would require changes throughout the entire MuseScore code.

  3. Return the same wrapper for the same object: mapping approach
    The other possible solution is to ensure that the same wrapper is returned for the same object. As BSG mentioned, one possibility is to store some mapping of elements to their wrappers in the plugin engine. QObject::destroyed signal from wrappers can be used to track when they get destroyed in order to remove them from that mapping.

  4. Return the same wrapper for the same object: member variable approach
    The other way to ensure equality of wrappers could be to cache the pointer to the used wrapper in the ScoreElement object from libmscore itself. This would make searching for a wrapper faster as no search in a mapping would be required, but that would make libmscore use a bit more memory for each element which can be somewhat significant for the entire score. That cache pointer could though be made possible to disable at compile time if no plugins framework is used. It should be noted that Qt itself also uses a similar approach (QObject implementation stores a variable that contains data used by the declarative QML framework).

  5. (didn't check but may be working) Return a non-pointer type to QML plugins
    Currently we typically create a wrapper object and return a pointer to it to QML framework. If we return some non-pointer type instead which has operator== correctly overloaded (and registered to Qt meta-object system) then we could probably make a comparison working properly. That would require though to change the underlying object's lifetime management system: deleting such a wrapper wouldn't mean that the underlying object should be deleted if it doesn't belong to a score. Also it should be checked first whether that approach allows to expose properties to QML framework correctly.

Conclusion

I simply listed the options I see and I am not currently ready to say which approach would be better here. Still, if an immediate solution is needed, the first approach (with is() function) could probably be implemented: while it is also not perfect, it doesn't seem to have large downsides and can easily be adapted without losing a compatibility if we still opt for some solution to ensure uniqueness of wrappers in the future. Still if there are some strong objections against the very model of creating wrapper objects each time on demand then we can certainly reconsider it, for example using the approaches described here.

Wow! Thanks for this thesis-level (or at least article in a technical journal) effort, considering everything that has been said. i will be studying it all morning.

Unlike the case in Clinton saga, it doesn't matter what the meaning of "is" is. Simply comparing two objects for equality of base object is marginally useful. For scores, it's useful. For notes, it's not. If (as I could do with ElementId) I want, in QML, to maintain my own data associated with, say, a Note, I cannot (with this scheme). There is no way to build a (JS) mapping (or "dictionary" in ECMA5) by which I can find it; I would have to keep track of every single note ever returned and do an "is" test against every one to find the object (wrapper or my own) that I am using as a canonical one. This has quadratic inefficiency problems. (More to follow as I think these over).

In reply to by dmitrio95

Hi Dmitri
Thanks for the excellent round-up of various options to solve the problem! For me option 3 (back to libmscore) is a bad idea even though it introduced the comparison problem. It just make the design more brittle and tougher to maintain over a long haul.

The factored approach provides an impedance match between external models and the internal models. It's best just to solve the comparison problem and bank the benefits of the factoring. I will say it took real courage to even suggest it though!

For a short term fix for some folks having short term needs (i.e., 3.3 release) I like number 1 since it hides the implementation a bit more compared to the elementId approach and is benign when a better fix comes along. Just deprecate the api in the docs and point to the preferred approach. No broken code results.

For the longer term I want to explore the others. There are some good ideas in there. Especially if mapping can be avoided.

Focusing on the the immediate time pressure: Are there any other comments about the short term approach (1) or if there should even be a short term approach?

-Dale

>Nor do I see why (1) solves the problem of deletion.

It doesn't but let's focus on one problem at a time.

This issue of dangling deletions already exists in today's code so I think it's fair to separate them and attack them individually. We probably need another MuseScore issue to track that problem.

-Dale

In reply to by DLLarson

See my comment of 6:57. #1 is very flawed, as there is no way to find objects, once gotten, except by linear searching through every single object of that type that you ever received. There must be some kind of identifier that allows dictionaries to be made, if properties cannot be set. What kind of things would (1) enable other than to say "I saw this score before!"? And, AFAICT,it doesn't solve the problem of deletion.

In reply to by DLLarson

The issue I want to solve is being able to, in Javascript, make my own notations or observations about some object, say, a Note, and I don't care whether it's by means of properties, or other objects I construct as long as I can find them from Notes, which with (1) I cannot without keeping track of every Note I have ever seen in an array and searching linearly to test for "==" on each of them. This doesn''t fly.

In reply to by [DELETED] 1831606

I suppose I could create a function that computes some crazy hash of the pitch, tick, duration, voice number, and anything else I can get, and search a hash bucket chain of previously-returned notes of that hash class, requiring a separate algorithm and hash table for each object type I needed. But this is a bit like "teaching the Pope how to pray"... (i.e., subsuming a function that should be provided), a fragile, hard-to-explain, harder-to-justify (to plugin coders) kludge at best. And this only works for objects whose union of POD artifacts is reasonably unique; it would not work, for instance, for NoteEvents.

>There must be some kind of identifier that allows dictionaries to be made

A very good point. Definitely foo.is() can't do that.

In terms of my original question: do you see a short term need for this or can it be kicked down the road some.

There is only a few days until 3.3 features are locked down.

In reply to by DLLarson

Using my nonexistent football (American) skills, kick it down the road some. No pressing need. The proposed solutions are all very disruptive. foo.is() might be an acceptable compromise to faciitate the hash scheme I just proposed, if it can be done without too much disruption. At least for Notes, I don't even think we need that (because the POD set is so uniquizing). But at least it's upward-compatible. Comparing leaf-POD including tick, voice, pitch, tpc, etc could serve as == today.

>because the POD set is so uniquizing

POD?

> Comparing leaf-POD including tick, voice, pitch, tpc, etc could serve as == today.

Sounds kinda nasty to me. Trying to data mangle down to a canonical comparison value is something that elementId should do if such a approach were used. Not the script. It's just a change of implementation.

Option 2 from above seems a non-starter because QML extension objects can only expose additional properties.

Options 3 and 6 encroach on my "do no harm" approach to attacking unfamiliar code bases where I try to have a minimal impact on the base design. So implementation for those items would have to be done by someone else.

For me that leaves options 4 and 5 for deeper investigation.

-Dale

NonEQNotes.png "Plain old data" (i.e., nonstructured, primitive values). The term even appears in some clang (Mac C/++ compiler) error messages. It is so that two notes (almost) cannot occupy the same pitch, voice, tick etc at the same time (they can -- see image - but it's very, very rare). There is no real substitute for pointer comparison. Day 1 of Lisp is that EQUAL is not the same as EQ (and this is true in any language with pointers). If you modify one of two EQUAL things that are not EQ (i.e., two identical notes at the same pitch and tick), the other does not change. A mashed "comparison value" does not substitute for identity, but could be a cue in a hash-based algorithm. It is not an adequate substitute.

How about foo.is() for 3.3 as proposed, with the understanding that it can be made useful, if not perfect (deletion issue there today) with the hash scheme I described (can even handle twin notes when both parts available), and will become meaningless when two non-EQ wrappers for the same underlying object will not, in the future, even be possible? It has the advantage of being upward-compatible.

FWIW...

I'm currently working on option 4 (i.e., return the same wrapper for the same object: member variable approach).

This hit for storage space for the back pointer in Ms::ScoreElement to the wrapper object may be more than compensated by not having so many wrapper objects being created.

-Dale

Needless to say, you're gonna clobber the variable when the JS engine tells you that the avatar has been GC'ed, and when plugin execution terminates....and this approach #5, not #4.

I've just pushed up a patch that partially implements option 5. It needs to be more thoroughly vetted plus needs to deal with the destruction of the wrapper so it can then clear the back pointer in Ms::ScoreElement. I'm making it available to just try out the concept.

I've attached a tie related test score and plugin for trying it out. When I run it I get the following decent looking results:

Running…
Plugin Details:
  Menu Path: Plugins.Test Ties
  Version: 3.0
  Description: This test plugin test note.tieXXX stuff.
  Requires Score
Debug: Hello Tie Walker
Debug: Score name=ttnew
Debug: Score.elementId=161537264
Debug: ###### START STAFF 0 ######
Debug: Element type=CLEF
Debug: e=Ms::PluginAPI::Element(0xcde7570)
Debug: Element type=TIMESIG
Debug: e=Ms::PluginAPI::Element(0xcde77e0)
Debug: Element type=CHORD
Debug: e=Ms::PluginAPI::Chord(0xcde7750)
Debug: Note[0] of ChordNum 0
Debug: ----note=Ms::PluginAPI::Note(0xcde89e0)
Debug: ----note.parent=Ms::PluginAPI::Chord(0xcde7750)
Debug: ----note.firstTiedNote=Ms::PluginAPI::Note(0xcde89e0)
Debug: firstTiedNote is pointing back at ME!
Debug: ----note.lastTiedNote=Ms::PluginAPI::Note(0xcde8230)
Debug: ----note does not have a tieBack.
Debug: ----note has a tieForward:
Debug: --------tieForward.startNote=Ms::PluginAPI::Note(0xcde89e0)
Debug: --------tieForward.endNote=Ms::PluginAPI::Note(0xcde8b00)
Debug: Element type=CHORD
Debug: e=Ms::PluginAPI::Chord(0xcde88c0)
Debug: Note[0] of ChordNum 1
Debug: ----note=Ms::PluginAPI::Note(0xcde8b00)
Debug: ----note.parent=Ms::PluginAPI::Chord(0xcde88c0)
Debug: ----note.firstTiedNote=Ms::PluginAPI::Note(0xcde89e0)
Debug: ----note.lastTiedNote=Ms::PluginAPI::Note(0xcde8230)
Debug: ----note has a tieBack:
Debug: --------tieback.startNote=Ms::PluginAPI::Note(0xcde89e0)
Debug: --------tieback.endNote=Ms::PluginAPI::Note(0xcde8b00)
Debug: ----note has a tieForward:
Debug: --------tieForward.startNote=Ms::PluginAPI::Note(0xcde8b00)
Debug: --------tieForward.endNote=Ms::PluginAPI::Note(0xcde8230)
Debug: Element type=CHORD
Debug: e=Ms::PluginAPI::Chord(0xcde8f80)
Debug: Note[0] of ChordNum 2
Debug: ----note=Ms::PluginAPI::Note(0xcde8230)
Debug: ----note.parent=Ms::PluginAPI::Chord(0xcde8f80)
Debug: ----note.firstTiedNote=Ms::PluginAPI::Note(0xcde89e0)
Debug: ----note.lastTiedNote=Ms::PluginAPI::Note(0xcde8230)
Debug: lastTiedNote is pointing back at ME!
Debug: ----note has a tieBack:
Debug: --------tieback.startNote=Ms::PluginAPI::Note(0xcde8b00)
Debug: --------tieback.endNote=Ms::PluginAPI::Note(0xcde8230)
Debug: ----note does not have a tieForward.
Debug: Element type=REST
Debug: e=Ms::PluginAPI::Element(0xcde9f10)
Debug: Element type=BAR_LINE
Debug: e=Ms::PluginAPI::Element(0xcdea480)
Debug: ^^^^^^ END STAFF 0 ^^^^^^
Attachment Size
ttnew.mscx 5.3 KB
tie-tests-v6.qml 3.79 KB

In reply to by DLLarson

I'm a little afraid to test this because, by your description, it will without doubt crash the second time I invoke the plugin. You can tally up a vector of every ScoreElement into which you've stuffed a pointer and loop through it at the end. the GC case is harder, whereupon use std::unordered_set instead. I know there is a callback for "JS GC terminating object".

Bravo, anyway!

>I'm a little afraid to test this because, by your description, it will without doubt crash the second time I invoke the plugin.

Fair enough. I've fixed that issue now and pushed up the fix.

If you add a call to gc() at the end of the script the back pointers will be properly cleared when the wrappers are destroyed.

Except... I've noticed that when using the plugin creator the cleanup is not run even if I load different scripts into it. When I close the application it does get called. This is not good behavior. I need to investigate. The same may be true when running from the plugin through the menus.

I need to find a way to trigger the cleanup when the plugin is finished.

-Dale

Probably won't be able to get to it this evening, but if you send me a url I can clone from, I'll try. Surely, you are in control when the plugin is finished, and you can rain down the Objektengötterdämmerung without being told to by JS....(and doing it twice should be a no-op).

> This hit for storage space for the back pointer in Ms::ScoreElement to the wrapper object may be more than compensated by not having so many wrapper objects being created.

For those users who do not use plugins this will just add up a memory usage though. It will likely be not very large though, at least for a normal usage on desktop systems.

Still I have a question regarding storing additional element's data in a plugin. Having unique wrappers for objects would not enable using mappings in any way. Am I correct then that your intention is to assign some property to a wrapper object to be used later? I am not sure that it will work that way (although it is likely to work) since it will be some internal QML engine object that will receive this property assignment, not a wrapper itself, and we do not control lifetime of those objects directly. If it turns out to be working we may need also to change ownership of wrapper objects to CppOwnership to make those data persist across plugin invocations or even when receiving a wrapper object for some element two times within the same plugin invocation. But then memory leaks and properties name conflicts between plugins may become possible. We will probably need to deal with these issues somehow.

In reply to by dmitrio95

No, I only hope to store javascript properties in the javascript visibility of the wrapper to be used during the present run of the present plugin, not persisting in any way beyond the execution of the plugin. The latter would be extravagant and outside of the charter of plugins. No mapping of any kind is needed if that much works, and if any mapping were needed, the JS object pointers are perfectly fine map keys. Am I answering your question?

>It will likely be not very large though, at least for a normal usage on desktop systems.

Up to 8 bytes per object. i.e., a single pointer. Looking at Qt source I find that QObjects can be quite heavy when you see all the private data the framework squirrels away behind the scenes.

I've just pushed up my latest work which still needs some vetting (e.g., for collection wrappers) but I think I've got the major areas covered for kicking the tires.

-Dale

I will zipclone, build, and play with this in a few hours. This is amazing work. I was gearing up do it myself, but you've mostly done it already. This is absolutely the right way.

I've never used a dock-type plugin, but what happens if more than one of them are up at once? Can that be? Are there multiple javascript worlds active at once!?!?! That can't be! I must not understand "dock-type plugins". Anybody? One I can try?

Yes, one part of my concerns is exactly about the case when multiple plugins are running simultaneously. Dock-type plugins are one example of that case: they allow to implement a dockable widget which can be used as a part of MuseScore interface. The plugins framework currently lacks a system of events which would notify plugins of some score or application state changes, that could potentially make such plugins much more useful. Also such widgets do not currently persist across MuseScore sessions. Still yes, it would be natural to allow several such plugins to run at a time, and no restrictions are made for that currently. For the unique wrappers problem this implies that several plugins may try to assign properties with the same name for the same object which would potentially create issues in their functionality.

My other concern which I tried to express in my previous comment is about preserving the properties within the same invocation of a single plugin. Consider the following code:

var sel = curScore.selection;
sel.elements[0].someCustomProperty = "someCustomValue";
gc(); // garbage collector may potentially trigger at any time, we'll force it just for demonstration purposes.
console.log(sel.elements[0].someCustomProperty);

It looks like the expected result here would be that someCustomProperty property of the selected element will be preserved after gc() call. But if we allow QML engine manage wrapper objects' lifetime then the corresponding wrapper with any corresponding properties will be deleted, and a new wrapper will be created when we try to access that element again. This means that a plugin developer may safely assume that the information will be preserved only if some reference to every element with such information is explicitly kept in a plugin. This does not seem to be a good limitation from a plugin development perspective since it will be easy to miss that, and, furthermore, this problem will be difficult to discover since not keeping such links will lead to losing information only sometimes, not fully reproducibly.

Concerning dock plugins, a Chord Identifier plugin is an example of it, when launching this plugin a widget appears at bottom-left corner which allows (after resizing it) to define some settings for finding chords. There is an improved version of this plugin but it seems to have switched from dock to dialog type.

If the wrappers are shared between plugins running concurrently, then storing properties in them is ruled out (although careful selection of names could partially address that). Serendipitously, you are now forced to have a javascript map/dictionary (stored in a plugin-local property var) to maintain properties, i.e., mapping javascript objects to "property bags" (i.e., dictionaries). And you know what? That would reference-count the object (used as a key), which would preserve it across garbage collections. So maybe the rule should be, "you can't assign user properties to these wrappers" (maybe could we enforce that somehow), and if you want to do that, you have to make a reference-preserving map/dictionary mapping objects to dictionaries of properties, and just make (or we supply) 2 javascript shim functions to get and set them. How's that?

A dock-type plugin, which I am to understand cedes control to the user like a non-modal Windows dialog, has bigger problems than being undercut by other plugins. Such a plugin may never, ever, store any state or determination about any score, including a pointer to any object, because the user might totally change or delete any or all notes or scores before the next time it is asked to act. If, during response to a click, it were to store properties in objects, it must remove them upon exit from the click -- if all plugins followed that protocol it would be "ok", but not only is that failure-prone, but the map-table way is safer if noncooperating agents are operating in parallel on the same score. Programming considerations for interaction between dock-type and non-dock-type plugins must be a chapter in "the book".

In any of the proposals (let alone today!), if you run your example 4 lines, you will print out two different addresses visible to the eye. But if a program neither stores properties in the object (which we will outlaw) nor retains a pointer to it, the program has no way to observe or notice that the objects values are not the same (EQ), nor can it possibly make a difference to the functioning of the program for that very reason.

Perhaps the Plugin Creator is viewed by the system as a "non-modal dialog/active/dock plugin" itself, and, hence, objects are not destroyed after each "Run"; this is probably wrong, but suggests that the only right time for Objektdämmerung is when the "active plugin count" goes to zero, now that "the plugin" is meaningless in light of dock plugins. But, then again, no dock plugin should be in possession of any object pointers when not actually handling a click.

Can anyone explain to me how to get a score having excerpts in it? Or does anyone have an example I could use for testing.

The score object exposes a list of 'excerpts' but I don't know how to get one.

-Dale

Some observations on QML Plugins and wrapper life times.

For some reason there is only one instance of QmlPluginEngine used by all scripts in the system. This class is a subclass of QJSEngine which is the level that scripts are interpreted and where garbage collection is enforced. As a side effect of this many wrappers may not be out-of-scope when the quit event is triggered. I'm using this event to cause a garbage collection assuming the plugin is finished with its work.

It turns out that quit is caused by the script call Qt.quit() and it runs synchronously within the scope of where it is called. So in the following script excerpt cursor is garbage collected and score is not:

function fooBar() {
    var cursor = curScore.newCursor();
    console.log(cursor);
}
onRun: {
    var score = curScore;
    Qt.quit()   // The wrapper in score is not garbage collected but fooBar's cursor is collected.
}

This makes sense so I need to find some event that really means the plugin is done doing its job. I can't seem to find such an event. This would probably not be a problem if an engine was created and disposed of for each plugin run which would ensure that the wrappers were all cleaned up before another script is run.

I'm not sure how to attack this problem. It seems risky to use the same engine for different plugins since you could get side effects between two concurrently running scripting engines. I've found that if the quit event doesn't call QJSEngine::collectGarbage() the wrappers won't be cleaned up until either gc() is called in a plugin or the MuseScore application is exited. My expectation is that a plugin would run to completion (however that is detected) and then its environment is then fully cleaned up. This is not what I'm seeing.

Am I wrong here?

Having modeless (always active plugin's) would make this problem even worse.

-Dale

The upshot of last night's discussion is that there is no such thing as "the plugin returns": all the active plugins (with docking plugins, there can be more than one) run in the same Javascript engine and space (without which the notion of exposing internal objects as their own avatars, as it was in the previous MS would be meaningless) It's pretty tricky for concurrent plugins to clobber each other inadvertently, because they store into plugin-scoped variables, not "truly global" ones. If they tried to clobber a global resource such as a class prototype, they sure would trouble each other. And if we don't prohibit storing into uniquized wrappers for MS objects, they can misinteract in that way, too. But the most interesting way they can damage each other is implicit in the model, and not a JS issue -- plugin A retains a pointer to a note, and either the user or plugin B deletes that note by explicit action. The user case is more revelatory of the problem than the Plugin B case. Nonmodal/docking plugins are basically tool palettes, which can customize their own UI state, but not dialogue with the user about an ongoing score-modifying action. They cannot, for instance, remember the last note they worked on and move on from there when a button is pushed; the user could delete that note before pushing the button. Garbage collection issues in the next message.

We have to draw a distinction between two operations, garbage collection, through which wrappers and other objects to which no references are held, in any plugin, are destroyed, and what I have been calling Objektdämmerung ("Twilight of the Objects"), in which all wrappers and other objects are destroyed. It could be (it probably is -- you can verify this), that when any plugin returns (no matter how many are active), it relinquishes all references it retains, MS wrappers or other, and not before then. If that is so (I hope it is -- we have to verify), then there should be no MS objects unrelinquished anywhere when "the last plugin returns", and GC should "devolve" to Objektdämmerung, and we are home free. See if this is the case. In one coherent JS space, it should never hurt of one plugin calls (gc) on the JS side; the definition of "care" is that no plugin can care about any object which it has neither modified nor retained a reference to.

Does this make sense?

The "scripting documentation" has to explain the axiom that if you save a reference to an internal object, and ask for the same object later, and see if they are the same (EQ), they WILL BE, even if this plugin or another plugin has provoked or invoked a garbage collection. Corollary to this is (1) today it is simply not so! The same object is NEVER returned! " and (2) if you're just breakpointing or console.log'in successive calls to see if the same object is returned, and you didn't save a reference to it, you didn't read the axiom carefully enough . It has to be explained that could we not differentiate between objects you literally cared about and others, garbage collection of objects would be impossible. Corollary to this is that storing user properties in these objects is not permitted.

I've just pushed up my latest work that adds unique wrapper mapping to selection, PlayEvent's and excerpt's. These particular classes need special treatment since they aren't based on the super class Ms::PluginAPI::ScoreElement.

Excerpts are NOT tested until I can find some way to exercise that code.

I believe this covers everything except the lists of objects that are exposed using Qt's QQmlListProperty class.

I have to study those a bit more if it even makes sense to ensure they are comparable. Opinions are welcome!

For plugin's that have UI's (dialogs and docks) garbage collection will be triggered when their windows are closed.

If a plugin does NOT have a UI, this new code will force a garbage collection when onRun returns. Therefore Qt.quit() is not required for those to do ensure a GC clean up.

-Dale

What are "excerpts"? Does the above strategy provably destroy all wrappers that were created by any plugin, i.e., is the hypothesis of plugin return letting go of all of them hold true?

>What are "excerpts"?

Exactly my question! I'm looking for some guidance there.

>Does the above strategy provably destroy all wrappers that were created by any plugin,

I haven't run a test with multiple concurrent plugins. I have run different plugins in the same application session against same and different scores and I see every wrapper cleaned up. The code really doesn't know what plugin is having wrappers created. Only wrappers that have no more references to them are cleaned up--but this was true before too. The difference is that now you get a single wrapper assigned per object. Before these changes, if gc() wasn't called in a script, I would see all the wrapper cleanup code called at application shutdown when the JS engine is destroyed.

Of course it would be great to have somebody else hammer on this!
Perhaps with a script that previously used elementId's and now could just compare the objects instead.

-Dale

Here's a piece of dismal news from testing. Object references cannot be used as keys in Javascript dictionaries, but can in Maps (or Python or C++) which, unfortunately, are only in ECMAscript 6, which will never appear on the Mac, including mine, because Qt 5.12 doesn't support OsX 10.10/11. That's rather terrible. So if plugins that have to run on the Mac want to keep objects around, they'd better do so in a list or set, and if they want to associate properties with them, there is no choice but linear lookup or store a user property with a "plugin-unique name" (e.g, "chord_plugin__propdict") in addition to saving in a set/list. I recommend restoration of ElementId (with a goofier name, like "IdHash") which, while not ensuring identity (=== will do that), facilitates building a hash table to substitute for what ECMAScript 5 cannot do. Yes, it is a kludge, but no more so than running Qt 5.9 on the Mac and 12 on Windows supporting 2 different JS languages.

>Here's a piece of dismal news from testing.... idHash...

If you ignore the missing ECMAScript 4&5 inability to use objects as keys, can you successfully use the objects for direct comparison?

If this does work would it be a crazy idea to use xxx.valueOf() as the key value?

-Dale

Excerpt is the term used in MuseScore code for extracted individual parts of a score: https://musescore.org/en/handbook/3/parts

Concerning the main topic of this discussion, theoretically it should not be necessary to force garbage collection of unused objects after plugin has ended its execution: QML engine should probably run it itself, although it may happen less often. Also it should be noted that this model of storing pointers to wrappers in ScoreElement objects seems to be able to partially resolve a problem of references to deleted elements in plugins. In ScoreElement destructor we can delete the corresponding wrapper object too. Even though wrappers can be left with JavaScriptOwnership it should be safe to delete them: wrappers are QObjects and have means to notify QML engine of their destruction. Qt documentation seems to imply that trying to access such a deleted QObject from JavaScript code will result in script exception which would terminate a plugin but not crash the entire MuseScore program. Moreover, it may happen that such exceptions can be captured and handled in JavaScript code itself so with some extra accuracy plugins may even be able to deal with such issues gracefully.

Still I would say it resolves only a part of possible issues since there would be still no means to detect whether element actually belongs to a score or is already stored in undo stack, and failing to determine this correctly may result in placing something to undo stack twice which would result in a crash later. For example it seems that nothing stops a plugin from removing the same element from a score twice, and it is not clear currently how to prevent such situations both in plugins and in C++ code.

In reply to by DLLarson

Simply "direct comparison" is not enough. I want to be able to mark objects (say, in a graph algorithm, or assigning musical analyses to them temporarily, etc). There are only 2 ways known to me to do this, store user properties in the object (which has problems with concurrent plugins, but that can be kludged with "sufficiently funny names":) , or storing them in a proxy object/dictionary findable from the wrapper object. This requires making a map of some kind. In ECMA6, there are such maps. In lower ECMA, I cannot use dictionaries in that way, although I can build a table in which I look up every object whose user properties I want to know. Since I cannot make a dictionary keyed by objects, I have to make a linear array of all such objects, which may be hundreds, and look them up linearly by === (which works now). In the absence of Map, I can only build a faster hash table if I have something wide-spreading but not necessarily unique, such as address (say as valueOf?). But in fact, two successive objects cannot occupy the same place in memory as false alias, by the way, unless the first was garbage-collected, which can only happen if no one had any references to it and, thus, "can't tell", i.e., can't make the mistake. So "valueOf" returning the address, would be plenty. If the object were deleted by C++ action, the earlier wrapper has to be marked "invalid to use", something today's strategy can't do. A badly-behaving dock plugin can sequester such pointers today. The documentation will be grim.

I've found a test score with lots of parts so the Excerpt part of my work has now been tested.

All in all, it's working pretty well right now. When things get going a surprising number of "wrap" calls are routed to existing wrappers. I've also tracked (in the debugger) their ultimate destruction. Everything is accounted for. It's fun to watch it churn when I tell the Plugin Manager to reload plugins.

As mentioned before some of the wrappers don't have Element as a super class so those have actual maps from MS objects to wrappers. On those I can see the number of wrappers in the map and watch them grow and ultimately die from there.

As a side note it would also be easy to use internal maps (Option 4) for all wrappers if we wanted to avoid the wrapper pointer in the ScoreElement's. However, the current method is blazing fast.

-Dale

In reply to by [DELETED] 1831606

>How about valueOf

That already exists and is provided by the ECMAScript stuff. It just generates strings along with the object's address (I presume.) Here a simple first level list of all elements from a simple score:

Debug: ###### START STAFF 0 ######
Debug: e.valueOf()=Ms::PluginAPI::Element(0xca7c9f0)
Debug: e.valueOf()=Ms::PluginAPI::Element(0xca7cba0)
Debug: e.valueOf()=Ms::PluginAPI::Element(0xca7c9c0)
Debug: e.valueOf()=Ms::PluginAPI::Chord(0xca7d770)
Debug: e.valueOf()=Ms::PluginAPI::Chord(0xca7db00)
Debug: e.valueOf()=Ms::PluginAPI::Chord(0xca7da70)
Debug: e.valueOf()=Ms::PluginAPI::Chord(0xca7d5c0)
Debug: e.valueOf()=Ms::PluginAPI::Element(0xca7d4a0)
Debug: e.valueOf()=Ms::PluginAPI::Chord(0xca7be50)
Debug: e.valueOf()=Ms::PluginAPI::Element(0xca7c2d0)
Debug: e.valueOf()=Ms::PluginAPI::Chord(0xca7c300)
Debug: e.valueOf()=Ms::PluginAPI::Element(0xca7c540)
Debug: e.valueOf()=Ms::PluginAPI::Element(0xca7c4e0)
Debug: e.valueOf()=Ms::PluginAPI::Chord(0xca7c210)
Debug: e.valueOf()=Ms::PluginAPI::Element(0xca7bf70)
Debug: e.valueOf()=Ms::PluginAPI::Chord(0xca7c660)
Debug: e.valueOf()=Ms::PluginAPI::Element(0xca7c030)
Debug: e.valueOf()=Ms::PluginAPI::Element(0xca7c750)
Debug: e.valueOf()=Ms::PluginAPI::Element(0xca7c360)
Debug: e.valueOf()=Ms::PluginAPI::Element(0xca7c5a0)
Debug: ^^^^^^ END STAFF 0 ^^^^^^

Those should be suitable for dictionary look ups.

-Dale

In the words of the Beatles,, yeah, yeah, yeah! That's terrific. I'll make a better try at playing with it tomorrow now that it's all in place! I'll download it then in case anything changes.

I am testing this. Two observations so far. It seems elementId is still there, and both it and valueOf() work as hash keys. elementId is considerably more efficient, and in view of the ECMA problem, ought stay IMO.

Secondly, the plugin creator is not working properly. Using the name "tie-tests-hashing.qml", the "Reload" button is not reloading, and the "save" button complains "cannot determine file type". Interestingly, the tooltip over the lower pane shows the pathname truncated , i.e., "/Users/bsg/Documents/MuseScore3Development/Plugins/tie-tests-h". 62 characters. Is there a 64 character buffer in its path somewhere?

Yes, renaming to "a.qml" works properly, but not "tie-tests-hashing.qml", although the display is truncated further back ("/Users/bsg/Documents/MuseScore3Development/Plugi"). There is a bad bug here, or a couple.

I ran the following test; "walk one staff" prints out notes in several revealing forms, including as a raw object pointer:

hashes = {}
console.log("First pass, setting hash table")
walkOneStaff(0, computeHashes);
console.log("#hashes= " + obj_count);
console.log("Second pass, report hashes found")
walkOneStaff(0, report_notes);
console.log("Garbage collect, report again -- should all be there");
(gc)
walkOneStaff(0, report_notes);
console.log("Clear dictionary, report again")
hashes = {};
walkOneStaff(0, report_notes);
console.log("Now gc again, report again");
(gc);
walkOneStaff(0, report_notes);

And although all the tests up until "hashes = {}" report the same (EQ) objects every time, as hoped, after clearing the table and even a gc, the addresses printed out are one-for-one identical for eight notes, which, although "theoretically possible" is still suspicious.

Here are more interesting results. My test plugin had been storing properties in the wrappers, and I wondered if that is what was making it fail to collect the objects. So I removed the code that did that. Amazingly, each time I ran the plugin from the creator, the objects were still there, with the properties from preceding runs! I even closed the creator and brought it up again, and the same remained true. The objects persisted beyond the up-ness of any plugin, unless the creator itself is considered a plugin (as I wondered about yesterday). I had to close the score and reopen it to see new objects without the properties. This clearly indicates that there is no eagerness to clean them up. And even in that case, the same object pointers were returned after hash-table clear and gc.

> It seems elementId is still there.

Yes, both approaches are currently in the PR. I'll take care of that once we make a decision.

-Dale

>Yes, renaming to "a.qml" works properly, but not "tie-tests-hashing.qml", although the display
>is truncated further back ("/Users/bsg/Documents/MuseScore3Development/Plugi"). There is a
>bad bug here, or a couple.

I don't think any of that is from my patches. I didn't wander anywhere near that code.

-Dale

>This clearly indicates that there is no eagerness to clean them up.
>And even in that case, the same object pointers were returned after hash-table clear and gc.

Could you try again with the latest code?

In terms of cleaning up, the garbage collection will only clear those that don't have references in any plugin. So is it possible a plugin managed by the plugin manager (not creator) is referencing them? Perhaps the same plugin that is used in the creator?

Anyway... I did clean up some event stuff (which can trigger GC's) in the latest code that you haven't built yet.

-Dale

In reply to by DLLarson

I will d/l again. No, I don't have any dock plugins, nor do the plugins I did use (red note, articulation, just testing) retain pointers, especially since they are not dock plugins! Non-dock plugins should not be ABLE to retain references beyond their run, right, i.e., they have no way to do so. As I said, my test plugin is deliberately retaining and relinquishing references in a controlled fashion ... the collection of purportedly unreferenced objects is exactly what it tests.

>Non-dock plugins should not be ABLE to retain references beyond their run, right, i.e., they have no way to do so.

Agreed... so long as the gc code is called after the run. I don't think my slightly earlier code did that for you hence my asking for a rebuild.

If these results persist I'll need your test score and plugin to investigate on my end. Thanks for testing this!

-Dale

Whooooppsss!

I forgot to push the latest changes up. :(

Sorry about that! You'll have to re-download and build again.

-Dale

In reply to by DLLarson

The "quit" problem is solved, but the objects continue to report as having the same addresses even after a relinquishing of references and explicit (gc). Of course, this is not a "problem", except insofar as it reveals that the life-cycles of the objects are not behaving as we expect. See what you can learn...

There is a difference, though - if I run the plugin again, from the Creator, the objects do not have the same addresses as during the previous run, which is a hopeful sign. But around the relinquishment and GC, they do.

I just fixed up your script because the gc's weren't being called properly.

However when I do fix those a much worse problem pops up. The global curScore is GC'ed! because there isn't active script reference to it at the time of the gc call.

I've attached an updated and renamed version of your script with gc calls fixed and modification that can make or break the script. Comment out the line shown and watch it break!

Bad news for the scoping of the global method or objects. This will take more digging.

-Dale

Attachment Size
a-wrapper-test-v3.qml 3.32 KB

In reply to by DLLarson

That is a true testament about who I am. Do you know why I wrote (gc)? A lifetime of Lisp!!!! When I think of GC, I think of Lisp (where such a call would be so stated!!) A both embarrassing and amusing error! The score getting GC'ed is probably correct, and has to be special-cased. But we are definitely nearing completion... Would cur_score survive gc in today's installed code?

> Would cur_score survive gc in today's installed code?

This problem does NOT break the latest release code!

I have digested things down to a simple QML script that replicates the problem. It's very odd.

Unless @dmitrio95 see's something I screwed up this could be a nail in the coffin for this approach. I haven't given up yet though. I'm running this in an environment that has NO plugin's registered by the plugin manager just to keep things clean as possible.

I've attached the failing test plugin to this post...

-Dale

Attachment Size
gc-curscore-error.qml 590 bytes

In reply to by dmitrio95

>This is very interesting. I don't see right now why that happens, will need to look at it closer.
@dmitrio95

I have a better example of the problem. See the attached updated test qml.

Essentially some active reference to curScore must be made at the scoping level and context of the gc() call to prevent curScore from disappearing. Just checking for existence isn't sufficient to prevent clearing of curScore so this approach breaks curScore:

            if (!curScore)
                  Qt.quit()

but the following approach is sufficient to prevent the error:

            if (curScore == null)
                  Qt.quit()

The weird thing is I tracked the contructions and destructions of the small amount of objects in the test and the curScore wrapper is NOT actually destroyed until it should be, but the curScore property shows up in the script level as null. Once the script exits and internally triggers a call to the gc code then the curScore wrapper is cleaned up. I've attached a screenshot of the breakpoints I set for testing in the latest commit.

BreakpointsToWrapperCreateAndDestroy-20190831.png

FWIW, I'm building against Qt 5.12.2.

-Dale

Some progress on the problem...

I've nailed down the culprit to a call to QQmlData::keepAliveDuringGarbageCollection(qobject); in the GC code. If a bad script is used this function returns false for this call and cleans curScore out:

FailedCallToKeepCurScoreAliveDuringGC-20190831.png

When I run a good version of the script the above function returns true thereby preventing destruction of the curScore wrapper.

-Dale

> and what did you change concerning it?

My post above describes the differences between failing and sucessfull scripts.

-Dale

In reply to by DLLarson

Scratch my comment about the QQmlData::keepAliveDuringGarbageCollection(qobject) call being false on the broken scripts. I stepped through both instances in the disassembler and the code paths are identical. Which is to say the function appears to return false. The code is so highly optimized it's difficult to see.

To @dmitrio95...

Is there a simple way to build MuseScore against the debug versions of Qt so we can have non-optimized code to step through?

-Dale

>Scratch my comment about the QQmlData::keepAliveDuringGarbageCollection(qobject) call being
>false on the broken scripts.

Rescind this!

All calls QQmlData::keepAliveDuringGarbageCollection(qobject) for the Score object return false.
However there is these differences...

When I run a good script the Score object shows up in the loop shown in the screenshot twice. Once when gc() is called and once when Qt.quit() is called (both calls trigger a GC.).

When I run a bad script the Score object only shows up once when gc() is called. It doesn't show up at all when Qt.quit() is called. This implies to me that the runtime interpreter "lost" curScore somewhere. Yet the JS engine does clean it up properly after the script is completely finished.

In both cases the destructor for curScore isn't called until a GC is performed after a return from onRun. So its destruction matches the expected scope. But none-the-less the script believes that it doesn't exist anymore.

-Dale

@dmitrio95, we're looking for guidance on how to debug QT without optimization. As far as I can see this is the only way to determine a resolution to this problem.

The other is to simply not call gc() in the script but that doesn't seem so hot.

-Dale

I managed to get a Windows MuseScore build that will run using the shipped version of Qt's debug DLL's. Working with Qt using the debug DLL's fully exposes what's going on internally in the framework. This is nearly impossible to do with release builds even though you have the symbols. The code optimization will wipe out most interesting things. Important note... I'm running QT 5.12.2 for the following work.

To accomplish this I had to hack up various CMake files to, not so delicately, force a Qt debug build to work. During these struggles I discovered that MuseScore actually triggers a couple asserts because of breaking some rules. I work around those for this hack. The ONLY goal I have is to be able to debug the QML garbage collector and these hacks accomplish that goal.

I will say it's a real treat to romp around in the Qt code and actually watch it do its magic.

I've attached a patch to this posting that can be used to modify the source code of the commit associated with this MuseScore issue. You can apply it using the git am command:

git am Enable-MuseScore-build-using-Qt-s-debug-DLL-s-on-Win.patch

This will create a commit. If you wish to drop the commit and keep the changes use this git command:

git reset HEAD~

This all assumes you will be using Visual Studio 2017 and the MuseScore procedure for it.

You can now build the system as you usually would.

And now a word from the patch's description:

Enable MuseScore build using Qt's debug DLL's on Windows.
 
This is a hack, pure and simple. A desperate move needed to
investigate an issue deep in Qt's QML garbage collector.
 
Important Note: MuseScore doesn't actually run successfully with
Qt's debug bits. Two things will assert and thus need workarounds:
 
1) During app startup in file "workspace.cpp" function "readGlobalMenuBar()"
calls "m->setParent(nullptr)" at line 838. This call will assert
in Qt because calling "setParent()" on a widget is apparently illegal.
See file "qobject.cpp" funtion "setParent()" at line 1996 which has
the test code "Q_ASSERT(!d->isWidget);". To get past this NOP
(intel opcode 0x90) out the Q_ASSERT code by using the disassembler
to locate it's 2 byte 'jne' instruction and set it to NOPs by using the
memory window to edit RAM. This sucks but the alternative is to try to
rebuild Qt without the assert. No thanks!
 
2) The MuseScore Start Center code will also cause a Qt assert
triggered when the "QWebEnginePage" class is being initialized in
file "startcenter.h" line 47. To avoid this "crash" run a retail
version of MuseScore and turn off the start center in the program's
preferences dialog. Then re-run the debug version of MuseScore
built with this patch. You should get past the assert.

For issue (1) you must patch out an opcode in the running application. I've supplied a breakpoint file that can be imported into VS2017's breakpoint pane with a breakpoint set on the Q_ASSERT that must be patched out. When you hit that breakpoint switch to disassembly mode and try to replicate what you see in the following screeshot:

DisableQtAssert-20190907.png

Issue (2) is related to the Start Center which shows up on a freshly installed program. This will crash with a memory fault. To stop this just run a retail build and disable the Start Center.

Also note that the breakpoint file supplied with this post has a number of interesting breakpoints in the Javascript engine garbage collector. That is the focus of this posting.

Finally note that I haven't solved the garbage collection issue yet but I posted this in case that someone else with mad skillz may wish to help out. Hint: @dmitrio95!

-Dale

Option -w should get you past that startcenter issue. Are you running musescore3.exe from the build or the install directory? The former would explain the workspace crash

In reply to by Jojo-Schmitz

>Are you running musescore3.exe from the build or the install directory? The former would explain the workspace crash

I'm running from the install directory.

It's a bona-fide Q_ASSERT as far as I can tell. Perhaps the setParent needs to be called on a cast somewhere down the class hierarchy some. I don't know. I just needed to get past it. The implication is that perhaps some diagnostic info is being left "on the table" because there is no supported method to build MuseScore with debug Qt binaries so it is never done.

-Dale

I have verified that the same garbage collector issue with the gc call in the script causing curScore to be null is in Qt 5.12.4.

I've attached updated Visual Studio breakpoint files for Qt 5.12.2 and Qt 5.12.4. Note that you'll have to update the hard paths to the source code in these files to match your installation and build source locations.

-Dale

I pushed up a commit today that offers a solution to the prematurely GC'ed score objects that are seen in some edge cases.

From the commit message:

Plugin: WIP: Change ScoreElement based wrapper ownership.
 
Change Ms::PluginAPI::ScoreElement based wrapper ownership
to be "QQmlEngine::CppOwnership" rather than
"QQmlEngine::JavaScriptOwnership" so that wrappers won't be
garbage collected before the proxied Ms::ScoreElement is destroyed.
Upon destruction of the proxied element, the wrapper ownership
will be set to "JavaScriptOwnership" and the wrapper's pointer
to the wrapped element is set to nullptr to prevent accessing
freed heap memory.
 
NOTE: This commit should be merged into the main commit when
found to be an acceptable approach.

Basically when the Ms::ScoreElement is destroyed it calls newly created Qt slot "symbiantDestroyed()" destroyed which breaks the wrapper's connection with the proxied object.

This is made especially easy using the Qt Slot system to call the slot without know what kind of class it's being called for. A very nice feature.

-Dale

This new approach is the other extreme of the current (3.2) policy, which latter is "create wrappers whenever an object reference is asked for, garbage collect them as needed, no promise of object identity preservation ever." This new approach is "object identity preserved forever, even between subsequent runs of the same or different plugins (as long as the score exists) and no garbage collection of wrappers, ever." We can't find the right "middle point" because we don't know why curScore, the value of a legitimate variable, is not preserved during a garbage collection when the C++ code is responsive to JS object collection (as it is not in this push). This answer is "conservative" insofar no objects or pointers will get lost. But it does raise the issue of VM consumption if a large score is traversed in QML, instantiating tens of thousands of wrappers that will never go away until the score does. It also says that (in this plan) marking wrappers from Javascript in anyway is a bad idea, for the marks will appear the next time the plugin is run (unless you consider that a feature and code to it, which is an iffy proposition). Maybe "the lifetime of a wrapper is the lifetime of the underlying object (the essence of this proposal)" is a reasonable definition.

In reply to by [DELETED] 1831606

I'll add that an "in-between" approach would be to only apply QQmlEngine::CppOwnership to the wrapper created by the curScore property call. All the others would retain their QQmlEngine::JavaScriptOwnership status so would be collected normally when not referenced anymore.

Note that this has the advantage of still providing notification to the wrappers when the wrapped object is destroyed. This allows the wrapper to clear its pointer to the freed heap object to prevent accessing invalid data.

-Dale

That would indeed be ideal if it can be shown that there are no other "magic objects" (or they can all be found, washed, shaved, and conscripted), and that extant plugins can be proven to work well.

Great! Now we need a QA plan for the whole scheme ... presumably gc now calls the notification "slot" callback and disconnects the realobject in all those cases...

My plugins are not working at all against this push. The articulation plugin comes up without values in its window about 50% of the time, and if I try it in the PluginCreator, it doesn' seem to know what QtQuit() means. It did say this in the creator:
Running…
Plugin Details:
Menu Path: Plugins.Articulation
Version: 3.2
Description: This plugin adjusts the on/off times of a note.
Requires Score
Debug: hello adjust articulation: onRun
Debug: No region selection.
Debug: sel note count 0
Debug: 1 selections
Debug: element.type=19
Debug: on time 0 len 1000 off time 1000
Debug: No note at Apply time.
Debug: No note at Apply time.
Debug: No note at Apply time.

It's having trouble finding the selected note, and maybe keeping it. That's the installed 3.2 version of the plugin.

In reply to by [DELETED] 1831606

>My plugins are not working at all against this push.

Well that's odd. Could I get the score, plugin qml and instructions to replicate?

>it doesn' seem to know what QtQuit() means

This is especially strange as I didn't really touch any of that.

Also....Did you try this with the previous commit before I added my latest stuff?

-Dale

In reply to by DLLarson

I don't remember if I did. How do I back out one with the git command? Happens on any score, but not 100% of the time, but without fail after 10 attempts on different notes. Plugin qml source is in the place pointed to by the plugins page; it is the 3.2 installed version. Just click on notes and see if the plugin comes up empty. If you run it in the creator, you will not be able to quit - that fails 100% of the time.

Here's the score and the plugin. You must assign a keystroke to it. Open this score, and just click on notes and issue the keystroke for each, type ESC. Sooner or later, the plugin window will come up without values in it. This absolutely does not ever happen in 3.3 beta. Try to run it in the creator and the "cancel" button or "ESC" do not work.

I figured out the Qt.quit() thing. It's due to an event disconnect() call before the dialog ended but onRun returned.

I have fix for that now. Not ready for push yet. I want to explore your stuff more.

Am I supposed to be able to select multiple notes?
When I do it says no notes are unornamented. Otherwise I can set and change it at will.

-Dale

Don't worry about selecting multiple notes for now. If you click on a place with no notes, it will issue that message. It should never say that if you really clicked on a note. There are no ornamented notes in that score, btw. It doesn't respect multiple-note selections -- you are supposed to use ranges (I'm not sure that that is undesirable). If you can click on notes anywhere for three minutes straight and see the plugin with real (not greyed-out) numbers in it every time, I either have a mac-specific problem or didn't make clean hard enough.... Also try quitting and restarting musescore and retrying it several times.

I had to be very devious. When run in "the Creator" (omnium pluginum Creator), it doesn't fail -- the Creator affects gc behavior. I remembered that Qt trace output, which includes plugin vocalizations, goes to stdout. So I ran ./mscore from the subdirectory of the app, and watched the terminal window. BINGO.
qml: Score
ScoreFont::draw: invalid sym 0
qml: hello adjust articulation: onRun
qml: No region selection.
qml: sel note count 0
qml: 1 selections
qml: element.type=19
qml: on time 0 len 1000 off time 1000
qml: hello adjust articulation: onRun
qml: No region selection.
qml: sel note count 0
file:///Users/bsg/Documents/MuseScore3Development/Plugins/articulate.qml:124: TypeError: Cannot read property 'elements' of null

123-125:
var selection = curScore.selection;
var elements = selection.elements;
if (elements.length == 1) { // We have a selection list to work with...

Curscore ran off to the dog pound.

In reply to by [DELETED] 1831606

FWIW
[bsg@bsg-mbp-2019 MuseScore (plugin-object-comparison-fix)]$ git show
commit 61a3e784447c41f785bf3ce3d0e35d77ada2a113 (HEAD -> plugin-object-comparison-fix, origin/plugin-object-comparison-fix)
Author: Dale Larson
Date: Wed Sep 11 10:27:21 2019 -0500

Plugin: WIP: Revert prior commit to only affect curScore object.

This restricts changes of wrapper ownership to CppOwnership for objects
returned by the `curScore` scripting property.

>Curscore ran off to the dog pound.

Looks to me like curScore didn't have a problem. Selection had no elements. Which is strange much like curScore is strange. Can you tell me what commit's you're currently using?

-Dale

I just posted what git show said. It is fine to call the plugin when there is no selection. Works every time in 3.3 Beta. curScore.selection returns an empty array in that case (as I can prove by doing it and not seeing that script error). What's worse, there was a selection. Note also that it said it couldn't extract the "selection" property of null- the unambiguous implication is that curScore is null.

To see the recent commit history for our branch:

git log -n 5

Let's step back a commit at a time and try again.

git reset --hard HEAD~

Note that two of these command will take us back to the original commit that relied only on the standard garbage collector.

Basically I'm interested if any level of commit worked for your plugin!

-Dale

Experiment one - failed in two applications:
qml: No region selection.
qml: sel note count 0
file:///Users/bsg/Documents/MuseScore3Development/Plugins/articulate.qml:124: TypeError: Cannot read property 'elements' of null
^C
[bsg@bsg-mbp-2019 MuseScore (plugin-object-comparison-fix)]$ git show
commit 9b231f5cee265096b5c72489888cf7d9b0bec953 (HEAD -> plugin-object-comparison-fix)
Author: Dale Larson
Date: Tue Sep 10 09:37:27 2019 -0500

Plugin: WIP: Change ScoreElement based wrapper ownership.

Change Ms::PluginAPI::ScoreElement based wrapper ownership

>It goes without saying that 3.3Beta works flawlessly with all the plugins (just making sure you know).

Which makes sense since there is no GC going on. Nothing is cleaned up!

It would be interesting to have a plugin that has only a gc() call that is run between other plugin runs to see if everything is stable on the beta.
-Dale

Experiment 2 :

qml: element.type=19
qml: on time 0 len 950 off time 950
qml: hello adjust articulation: onRun
file:///Users/bsg/Documents/MuseScore3Development/Plugins/articulate.qml:64: TypeError: Cannot call method 'createPlayEvents' of null
^C
[bsg@bsg-mbp-2019 MuseScore (plugin-object-comparison-fix)]$ git show
commit 0b3d4a281d581d59f6dd2fd6f7e7698f03bfb395 (HEAD -> plugin-object-comparison-fix)
Author: Dale Larson
Date: Wed Aug 28 12:30:59 2019 -0500

Fix #293837: Plugin: WIP: Add ability to compare MuseScore QML elements

This patch ensures that there is only one QML wrapper object for
each internal object that is exposed to scripting. This enables
simple sameness comparisons between javascript objects.

As expected, no failure here:
c^C
[bsg@bsg-mbp-2019 MuseScore (plugin-object-comparison-fix)]$ git show
commit 46da22f9e4d10d0f5e1f4f483d773ebcdf4635a3 (HEAD -> plugin-object-comparison-fix)
Author: Dale Larson
Date: Tue Aug 27 09:03:52 2019 -0500

Fix #293837: Plugin: WIP: Add ability to compare sameness of MuseScore QML elements.

A property named 'elementId' has been added to all score elements
so that QML scripting can compare two different QML elements to see
if they are accessing the same MuseScore object beneath the plugin

It seems you should be able to duplicate this easily, staying out of the plugin creator and watching output on stdout.

I did lie -- it's the style these days (but I did it unintentionally) -- it said no 'elements' property of 'selection', so the curScore object was there, but its selection property, which should be and is there even if there is no selection, was null (not "undefined", mind you). Is the pointer to "selection" a weak pointer of some kind (I thought you needed ES6 for that). You are using ES5/Qt5.9 as I am, aren't you? Shouldn't the array be created if it's not there when asked for? (note that there actually is a selection in the failing case).

Thanks for all the data! It's a lot to digest.

In terms of Qt I'm running 5.12.4. Pretty much the current version.

I think the GC call the patch is doing internally is causing all this.

To test that idea I'm going to create a new branch using the current master branch (or 3.3beta) and only add the GC operations to the code and see if it's my wrapper changes or just the fact that a GC runs at all. Gotta find out where to split the problem.

Again, Thanks for the help!
-Dale

>selection......Shouldn't the array be created if it's not there when asked for?

Yes, that is definitely broken.

It's as if Qt's QML forgot about the existence of the selection.elements property and never calls the internal code that actually creates the wrapper!

Basically, stuff is busted under the hood. The degree of wreckage may be different between you and me because of the different versions of Qt we're building against. But it's still busted. Time to try to slice down the problem.

-Dale

You must use 5.9 as I am (and other Mac users are forced to).. Mac users will never get 5.12 as long as support for MacOS 10.10 and 10.11 is a business requirement. You haven't duplicated the failure yet, right? 3.3 will create a new wrapper any time an object is asked for. It can never fail.

>3.3 will create a new wrapper any time an object is asked for. It can never fail.

True enough. I can't get a failure in 5.12.4 or 5.9.8 when adding garbage collection at the end of plugin runs and letting it freely create redundant wrappers.

-Dale

This is in 3.3 Beta standard? It should be easy enough to create a plugin that does nothing more than try to access curScore.selection.elements and report curScore.selection being null, which should not happen, even if there is no selection. Again, it can't run in the Creator, because that suppresses failure. If there was a garbage collection, it is reasonable that the selection array should have been garbage collected, and the pointer set to null (C++). But it is not reasonable that when I asked for it, it did not recreate it.,

I've created a new series of commits that separate the internal GC code from the code that ensures only a single wrapper per MS object. They are:

Commit A: Call internal GC on engine after plugin runs.
Commit B: Ensure only a single wrapper per MS wrapped object.

All tests were run against Qt 5.9.8.

With A & B applied. If I call gc() in script, curScore null failures result. If don't call gc() scripts all is well.
With A alone. All runs fine. In this case we have many wrappers per wrapped object.
With B alone. If I call gc() in the script, curScore null failures result. If don't call gc() scripts all is well.

i.e., If gc() is never called in a script things seem just fine.

To get the selection.elements problem I'll need a simpler script to reproduce. In the end though this whole single wrapper thing is looking pretty tough to satisfy in the MuseScore code base.

Making things worse is the complete non-interest in this problem by MuseScore core team members (as I come to know them anyway.)

I'm seriously considering punting on this until the core team sees this as an important enough addition to give some technical attention to. Additional labor feels like it will be a fruitless waste of time.

Note that even if we found a fixable problem in Qt's QML memory manager GC code it would be moot because the Mac version can't evolve its Qt codebase. Working around an internal Qt problem without new Qt code is, if not impossible, highly impractical.

I think the only way to get a short term solution of comparing objects is to go back to the element.elementId idea again.

-Dale

In reply to by DLLarson

I created a minimal plugin (attached) that does, as I suggested, nothing (after "create note events", as articulate does) but check for curScore.selection being absent. Unfortunately, against the current branch, I can invoke it 25 times in a row (watching console log via previous kludge) and not see a failure, but if I try the art. plugin, I get a failure in three tries. I can't really explain that.

I tend to agree that this project is moribund, esp. in the face of the 5.9/5.12 problem, which, as I feared, as you note, has already started to cast a pall over the whole plugin world, and not just for Javascript versions. I tend to concur that, esp. given the lack of interest by others, "Retreat to ElementID" makes the most sense. Note that someone else (i.e., other than me) complained about the removal of object-EQness.

Thanks for your effort on this. Maybe we will learn something in the future, but it doesn't look worth it now (if there were QT bug, it might get QT attention if it weren't 5.9-specific).

"Sad".

Attachment Size
selbomb.qml 1.55 KB
Status fixed PR created
Fix version 3.3.0  

For the time being I pushed a change which implements the approach with is() function, this seems to be the safest and fastest way to resolve the issue for now. Later we can return to the approach with ensuring unique wrappers but for now that would be a large change given the time we have to test it before 3.3 release.

Anyway, @DLLarson, thanks a lot for your research on this, I hope to be able to have a look at the issues you listed once 3.3 version gets prepared.

Fix version
3.3.0