Introducing the new QML based plugin framework

• Lie 10, 2012 - 10:30

Before explaining the new plugin framework, let's define what problems we faced with the plugin subsystem in MuseScore 1.2.

One problem is the binding to the qt library. The code to enable this is very huge and consists of > 1100 files with > 297000 lines of code. This is not maintainable for the MuseScore team. It seems that qt does not maintain this anymore and it will not be available for the next major Qt version.

Another problem is that the MuseScore class hierarchy cannot be exposed easily to the current JavaScript engine. Every MuseScore class must be wrapped by a special proxy class. This also is a big maintenance issue and makes it difficult and expensive to make more elements of MuseScore available for scripting.

New QML based framework

After some hacking and testing i want to come up with a new solution: using the Qt qml system to make MuseScore scriptable.

Qml is a declarative language with the syntax of JavaScript which can contain java script parts which are executed on gui events. Without any further bindings Qml contains all to create modern dynamic guis.

The most important point is that the needed bindings to MuseScore internals are much easier. It is possible to expose the original class hierarchy of MuseScore to the scripting engine.

Caveats are that the new system has some overhead as all score elements now inherit from QObject and incorporate the Qt meta object system.

I hope the new system can offer the same functionality than the old plugin system so that porting of existing plugins is easy. Otherwise its a chance to think again on how the interface can be designed to implement simple things as easy as possible. One of the design goals should be that it should not be possible to crash MuseScore with scripting and it should be not possible to create corrupted scores.

The first version of the new plugin system is available for testing in the nightly builds.

//===============================================================
//      "colornotes" example in qml:
//===============================================================
 
import QtQuick 1.0
import MuseScore 1.0
 
MuseScore {
       menuPath: "Plugins.colornotes"
 
       onRun: {
             console.log("hello colornotes");
 
             var colors = [
                "#e21c48", "#f26622", "#f99d1c",
                "#ffcc33", "#fff32b", "#bcd85f",
                "#62bc47", "#009c95", "#0071bb",
                "#5e50a1", "#8d5ba6", "#cf3e96"
                ];
 
             if (typeof curScore === 'undefined')
                   return;
             var cursor = curScore.newCursor();
             for (var track = 0; track < curScore.ntracks; ++track) {
                   cursor.track = track;
                   cursor.rewind(0);  // set cursor to first chord/rest
 
                   while (cursor.segment) {
                         if (cursor.element && cursor.element.type == MScore.CHORD) {
                               var notes = cursor.element.notes;
                               for (var i = 0; i < notes.length; i++) {
                                     var note = notes[i];
                                     note.color = colors[note.pitch % 12];
                                     }
                               }
                         cursor.next();
                         }
                   }
             Qt.quit()
             }
       }
 
//========================================
// the "old" java script version:
//========================================
 
var colors = [new QColor(226,28,72),new QColor(242,102,34),new
QColor(249,157,28),
new QColor(255,204,51),new QColor(255,243,43),new QColor(188,216,95),
new QColor(98,188,71),new QColor(0,156,149),new QColor(0,113,187),
new QColor(94,80,161),new QColor(141,91,166),new QColor(207,62,150)];
 
function init()
       {
       }
 
function run()
       {
       if (typeof curScore === 'undefined')
             return;
       var cursor = new Cursor(curScore);
       for (var staff = 0; staff < curScore.staves; ++staff) {
             cursor.staff = staff;
             for (var v = 0; v < 4; v++) {
                   cursor.voice = v;
                   cursor.rewind();  // set cursor to first chord/rest
 
                   while (!cursor.eos()) {
                         if (cursor.isChord()) {
                               var chord = cursor.chord();
                               var n     = chord.notes;
                               for (var i = 0; i < n; i++) {
                                     var note   = chord.note(i);
                                     note.color = new QColor(colors[note.pitch % 12]);
                                     }
                               }
                         cursor.next();
                         }
                   }
             }
       }
 
var mscorePlugin = {
       menu: 'Plugins.Color Notes',
       init: init,
       run:  run
       };
 
mscorePlugin;
#----------------------------------------------

Komentarai

• Lie 10, 2012 - 14:06

One good thing about this is that the old .js and new .qml can coexist, MuseScore up to 1.2 (even up 2.0 R5663) will pick the .js one, MuseScore 2.0 (as of c7350b7) will pick the new .qml ones. So the plugins in the repository can just add a .qml to the archive

Reply

• Lie 10, 2012 - 19:24

Thanks for the update. It sounds like a good move.

I am wondering whether it would be possible to support chromatic staff notation systems in MuseScore, by using plugins / scripting. For example, notation systems like these .

You would need to be able to customize the layout of pitches on the staff, in other words, how pitches are mapped to vertical staff positions, (12 chromatic notes on the staff rather than 7 diatonic notes from C major). And also customize the pattern of lines and spaces that make up the staff. (I see that it's already possible to change the note shapes with plugins, as with the shape note notation plugin.)

These kinds of customizations are possible with LilyPond (using scripting), and I wonder if this new plugin system will make them possible in MuseScore as well? Sounds like maybe so?

Thanks,
-Paul

Reply

• Lie 11, 2012 - 17:55

Can we get a list of available elementes, properties and methods?
I do understand that this is a "moving target", i.e. that this list might not only extend but entry could also change and even disappear, but it would be usefull anyway.

Reply

• Lie 12, 2012 - 18:44

It will be a few days before I have a chance to really try this out in earnest, but at first glance, all I can say is "wow". Well, that and "thanks"! OK, those two things plus "latest nightly builds for Windows appear to be corrupt - 7-Zip won't open the file".

Reply

• Lie 13, 2012 - 07:51

tonight's build has this fixed.

What I haven't yet figured is how to work on a selection rather than the entire score. Stuff like goToSelectionStart() and goToSelectionEnd() is not available anymore and I hav't yet spotted a replacement. Same for startUndo() and endUndo().

Edit: ah, I see, from the source code, score.rewind(0) sets to start of score, score.rewind(1) to start of selection and score.rewind(3) to end of selection.

Another question is this: if the onRun needs to end wirh "Qt.quit()", shouldn't a "return" e.g. in the case of "no current score" also be a Qt.quit() instead?

Another one:
The menu "View/Plugin Creator" should be enabled even without a score being open, just like the Plugins menus itself.

Reply

• Lie 13, 2012 - 12:18

If you end a script you have to call Qt.quit(). So its missing in the check for currrentScore.
startUndo() and endUndo() are gone. All operations have to be undoable else the undo/redo stack may get corrupted when referring to an element which has moved or deleted (implementation is not complete on this currently).

Reply

• Lie 13, 2012 - 08:59

Looks a better way for MuseScore developers and plugin developers.
One question: you changed the logic in your example to use tracks. Was this because staffs and voices arent available or to simplify the code?

Reply

• Lie 13, 2012 - 10:03

score.nstaves is avaliable (replaces score.staves), staff is availabe too (but no methods or properties) so you could still use an outer loop on staves and an inner one on voices, but also just use one loop on score.ntracks. At least that's my understanding.

Access to a note's velocity (read and write) seems missing currently.

Reply

• Lie 13, 2012 - 11:25

Only tracks is used by the underlying code. StaffIndex is (track / 4) and voice is (track modulo 4).
Staff and voice are derived values and only there for convenience.
In a simplified picture you can see all musical objects in grid of tracks and segments. Tracks is the y axis and segments are the x axis.

Reply

• Lie 13, 2012 - 21:01

I'd be very grateful if someone could skim over the JavaScript bit of my MIDISightReader plugin and give me some clues as to how to upgrade it to the new QML framework.

Specifically:
- Get the plugin directory.
- Get the directory for temp files.
- Save a score as a MIDI file.
- Pop up a dialog who's text is read from a file, or is generated in the JavaScript.
- Set the stdout of a process to write to a file.
- Trigger a piece of JavaScript code from the stderr of a process run by the plugin.

I had a look at the built-in manual, but many of these methods seem to be missing.

Thanks.

Reply

• Lie 24, 2012 - 18:03

Hi,

From all the above I gather there were three main reasons for switching from the previous QtScript-based plugin framework to the new QML-based framework, but I don't understand them.

1) Bindings for Qt 'built-in' classes were hard (to say the least!) to maintain.

Absolutely true! But I do not understand how QML is better, as it requires the same bindings to be coded or to do without Qt 'built-in' classes in scripts.

2) Access to MuseScore own objects is more complete from QML than from Qt Script.

I really do not understand this. All the stuff used to make MuseScore objects (and their properties / methods) available to QML exists in Qt Script too, via the same techniques (Q_OBJECT, Q_PROPERTY, Q_INVOKABLE, ...) and then through code quite similar to the code currently used in MuseScore (=> classes for script objects (Cursor, Note, etc.) would work with Qt Script with only minimal changes from current code, if any; same for the code auto-generating the plugin doc).

3) Qt Script is being phased out.

It is true that nokia is pushing QML a lot, but I could not find anywhere a statement about Qt Script imminent doom. In the Qt Module Maturity Level page Qt Script is qualified as "Active/Maintained" (the individual QScriptEngineAgent class is deprecated). This means that the module is actively maintained; full documentation for it is present in the current Qt 5 documentation.

Alternatively, the Qt 5 Add-on Modules List qualifies Qt Script with: "Classes for making Qt applications scriptable. Provided for Qt 4.x compatibility, please use the QJS* classes in the QtQml module for new code.". The Qt 5 documentation lists 3 QJS* classes, QJSEngine, QJSValue, QJSValueIterator. QJSEngine (doc page ) seems rather similar to QScriptEngine used by MuseScore 'old' script framework (QJS* classes are not present in Qt 4.x and require Qt 5).

I read all the above as meaning:

*) Qt Script will be present in Qt 5 with all its bell&whistles and all its limitations (maybe a little less of the latter, as addition of new Qt Script functionalities is unlikely but not ruled out: the module is Active, not Done).
*) In the long run (years from now?), QJS* classes will replace QScript* classes, providing more or less the same functionality in more or less the same way (=> Qt Script will continue to exists under another name, with limited differences).
*) None of the above affects significantly points 1) and 2) above.

So, are we sure it does not make any sense to re-asses the Qt Script vs. QML choice?

Thanks,

M.

Reply

• Lie 24, 2012 - 18:55

Yes, we do without the buildin Qt classes. Qml is designed to be the frontend for any qt gui. So nothing more is required from our side to build a GUI. This saves us 219000 lines of code for the gui bindings. This code is generated. It seems that the tools to generate the bindings are not maintained anymore.

Its a bit too offtopic to discuss the pros and cons of traditions widgets vs. qml. But a few buzzwords: qml is hardware accelerated which makes it more powerful for dynamic guis with smooth blending, animations etc. Many things which are hard to do with widgets is easy to do with qml. All this is most beneficial for mobile devices which is the reason Nokia is pushing qml. There is no further development of widgets and it is expected all will be replaced by qml some day.

Reply

• Lie 24, 2012 - 23:21

My post was not intended to be polemic: if it carried across this feeling, I apologize.

"Yes, we do without the buildin Qt classes."

For many plugins, this might be a problem: the need for file I/O has already surfaced. The need for other rather basic resources / services (and maybe some not-so-basic too) may easily pop up, when more plugins will be written. Symmetrically, almost all bindings (except a little minority) could have been taken away also from the old QtScript-based framework, with comparable results (included saving most of those 219000 lines of code).

"So nothing more is required from our side to build a GUI."

True, from one point of view. From another point of view, MuseScore (we?) is reflecting on the plugin writers the burden of creating GUI's with very basic building blocks (but I was not speaking only of GUI's).

"Its a bit too offtopic to discuss the pros and cons of traditions widgets vs. qml"

I was not speaking of GUI's in particular. Widgets (and GUI) are only one aspect of scripting. I was trying to compare (summarily, it's true) Qt Script as a whole to QML as a whole as far as scripting an application as MuseScore is concerned, and to explain why I do not understand the advantages of the latter on the former, at least under the scripting respect (and I may be particularly dumb and aging, but I still do not understand).

"it is expected all will be replaced by qml some day."

By that day, we can hope QML will have matured more that it is today (about GUI and, even more, about anything else); we can even take this for granted. But until that day, I believe of some relevance to assess what "we" are giving the plugin creators to work with.

M.

Reply

• Rug 5, 2012 - 07:26

I'm curious, is there a planned Sunset Date or Version for the JS Plugin architecture? I'm currently working on a plugin thats branched from one in the JS Plugin architecture, so I'm looking for information on the time frame to port it..

Reply

• Rug 5, 2012 - 08:56

JS plugins work with version 1.2 (the current one). The next version, 2.0, will use the QML plugin architecture.

The current goal is to have ver. 2.0 in alpha for the end of the year, beta and release being later (of course!).

It is a bit vague, but this is the current time line, as far as I know.

M.

Reply

• Lap 11, 2012 - 21:23

Hi,

I'm using the latest nightly (2012-11-11) on OS X 10.6.8, and the plugin manual doesn't seem to work. The manual window opens, but is empty. See image.

If this bug is not easy to fix, I'd like to know whether there is some place I could read the manual online or download it. Or how could I genereate it myself from the sources?

NOT FOUND: 1

Prisegtukas Dydis
manual missing.png 50.78 KB

Reply

• Gru 4, 2012 - 09:25

How does internationlization work with the new QML plugins?

It definitly doesn't work the same way as in the old framework.
There it was sufficient a) arm the script with 'qsTr("string to translate")' and b) to place the 'compiled' translation files in a subdirectory named "translations" with names like e.g. "locale_de.qm" (for the German translation).
Somewhere I read the hint that in QML this has to ba a directory called "i18n",n but that doesn't work either.

Any hints? Is it supposed to work at all, currently, or is there possibly some infrastructure missing in MuseScore?

Reply

Do you still have an unanswered question? Please log in first to post your question.