Modularizing MuseScore

• Jan 20, 2019 - 05:34

TLDR

Benefits of modularizing

  • Easier to contribute (consider things in isolation, not hit with everything at once)
  • Easier to maintain (well-defined structure + relationship between modules)
  • Extensible (swap modules in and out like plugins)
  • Portable (remove large dependencies from backend to ease porting to Javascript / compiling to webassembly / mobile apps / Python bindings, etc.)
  • Better storage in Git (see https://bitbucket.org/blog/monorepos-in-git)
  • Chance to tidy up (fix header files, variable names, indentation, etc.)

The long version

The release of MuseScore 3 is a great achievement on the part of the development team and of the wider community. What was once a small hobby for one individual has grown into a global project of epic proportions. MuseScore has more contributors than ever before, more features than ever before, and, unsurprisingly, more code than ever before!

We are no longer at a point where any one individual can claim to understand every aspect of MuseScore’s codebase. With the increased pace of development it becomes ever harder for the development team to keep on top of things and ensure that everything progresses smoothly and in the right direction. It is vital that we look at improving the structure of the code if MuseScore is to sustain this level of growth in the future.

The solution

What we need to do is separate MuseScore’s code into distinct modules. Each module should operate independently of the others, with communication taking place over an API that remains relatively stable. This allows programmers to treat modules as “black boxes”, passing data in and receiving the output without having to worry about the details of how any module works other than the one they are currently working on. It also allows the code within each module to be improved and iterated upon with minimal impact to other modules. On the rare occasion that it is necessary to make a breaking change to the API, the improved structuring would make the change much easier to implement.

Adopting a modular design would enable developers to focus on their individual area of expertise, without worrying about the implementation details of other modules. Each module could have a designated maintainer responsible for reviewing code contributions to that module ensuring that the API exposed to other modules is preserved. This would also enable additional modules to be created and maintained by the community, with the core team able to decide if and when the community modules should become officially supported.

What about extensions?

The extensions mechanism should help to address the problem by separating out new components that are not needed by everybody. However, extensions exist outside of the codebase, so they can only offer additional resources (instrument sounds, templates, palette elements, etc.), not improved performance or functionality; these are challenges that have to be addressed at a deeper level.

It is possible that the extensions mechanism could be used to deliver dynamic libraries (a.k.a. “binary extensions”) that would indeed add new features, such as support for more import and export formats. However, this can only happen after the modularisation has taken place, otherwise there will be no API for the libraries to interact with.

How it would work

The most obvious candidates for modularisation are:

  • Notation dialects (CWMN, Guitar Tablature, Gregorian Chant, Jianpu, Braille, etc.)
  • Import and export formats (MSCX, MusicXML, MNX, MIDI, etc.)

The core code would define abstract (virtual) C++ classes for NotationDialect and NotationFormat, which would then be implemented in the modules by the concrete classes CWMN and MSCX, and possibly in GuitarTab and MusicXML classes too. The community would step in to implement the remaining classes in separate unofficial modules, and the core team could choose to:

  1. Include the community module with MuseScore out of the box.
  2. Provide the module as a binary extension through the extensions mechanism.
  3. Run the module as a service on MuseScore.com (e.g. format conversion).
  4. Add a build option for people to compile it on their own.

Community role

It may be that the core code defines a basic version of a module that will suit the needs of the majority of users, while the community may provide a more advanced version. An example of this would be SVG and PDF export, which can be provided quite easily by Qt but with limited functionality. Here the community would step in to provide better export facilities (in the case of SVG they already have, though like everything else it could benefit from a more modular design).

In the future, the community might even implement modules for PNG, PDF, SVG and MP3 import, which would call upon machine learning techniques to provide automated OMR and audio transcription facilities. While some efforts have been made, a modular design would make it significantly easier to make progress in this area, as it would allow OMR experts to concentrate on OMR rather than on how to get it working with the rest of MuseScore’s code. Progress in this area would further strengthen MuseScore’s university relations, attracting talented students and research funding.

Benefits

Better integration and reduced code duplication

There are many places where the current monolithic design has led to features being duplicated. An example of this is the MIDI import facility and the realtime input mode, both of which implement conversion of MIDI data into music notation, but do so entirely separately.

A modular design would allow these features to share an underlying pianoroll-like internal data structure, which would then be converted to notation using a shared set of algorithms for:

  • quantization (rounding note-on and note-off events to the nearest beat)
  • tuplet detection
  • rhythmic grouping (adding beaming and ties)
  • ornament recognition (detecting staccato, trills, etc.)
  • converting velocities to dynamics (p, mp, mf, etc.) and dynamic changes (cresc., dim. etc)
  • converting MIDI tempos to tempo changes (rit., rall. etc).

Improvements to these algorithms would benefit MIDI import, realtime MIDI input, and a potential transcription-from-audio feature as outlined in the previous section.

Improved portability

Basically we should completely remove Qt from the backend. This would enable:

  • Easier porting to other platforms (e.g. mobile apps, compile to Javascript / WebAssembley)
  • Separating the command line interface from the GUI
    • no need for a GUI for rendering audio and PDF / SVG
    • Don’t need a synthesizer to generate PDFs
    • Don’t need layout to generate audio
    • Simple operations on MSCX files don’t require any of the above

MuseScore as a platform

MuseScore contains many features designed to suit a number of different use cases, but it not possible for a single program to keep everybody happy. The program is large and complicated enough already, but a modular design would allow new features to be developed outside of the existing codebase. This, combined with a binary extensions mechanism, would enable users to pick and choose the features they need.

Wordpress has built an entire ecosystem of open source plugins and themes, and MuseScore could do the same for music notation. It is not hard to imagine a MuseScore “extensions store”, which would operate a bit like an app store, allowing the user to choose extensions that add new features for editing scores, or specialised playback facilities such as a karaoke mode. The desktop software itself would basically just be a client shell with hooks for extensions, a bit like a fresh Wordpress installation. Of course, just like with Wordpress, MuseScore would come with various extensions installed by default, so it would remain a fully functional program out of the box.

MuseScore as a software library

Developing a program for multiple operating systems is a huge challenge that wouldn’t be possible without a framework like Qt. Developing music software that can support multiple notation formats is equally challenging. As the leading open source notation program, MuseScore is already the defacto standard for converting MusicXML to PDF and other formats, so why not expose this ability to other open source programs via an API? A modular design would also make it possible to create bindings to other languages like Python, in addition to C++.

Transition period

Development is a learning process, and it is only by trying things out and making mistakes that you learn the right way to do things. However, the lesson often comes when it already too late to change what you are doing, and you are forever stuck doing things the wrong way. We have seen this with Finale and Sibelius, the two dinosaurs of music notation, and we are already starting to see it in MuseScore (a prime example of this is local time signatures, which is forced to use a very hacky implementation due to the way things are implemented at a lower level in the code).

There comes a time in the development of any software application when you basically need to start all over again and reimplement everything from the ground up, drawing upon the lessons learned the first time around. We have seen from Dorico the transformation that can be achieved when you hire a bunch of developers and tell them to “make the program you wish you had made 10 years ago”. MuseScore clearly beats Dorico in terms of price, but if we are to compete on a feature level then it is essential that we re-examine the core structure of our own codebase.

Fortunately, we don’t literally have to start from scratch. One of the benefits of a modular design is that we can create the modules one-at-a-time, and gradually integrate them into the codebase as they mature. Many of MuseScore’s code files are already partially modular, so in these cases all that is needed is some virtual classes (i.e. the API) and a better directory structure.

Possible modules:

  • Muse: Abstract music module implemented by dialects (pianoroll / MIDI with fractions)
  • Score: a dialect (CWMN, Jianpu, etc) + simple operations like inserting/deleting notes, joining scores etc. (computationally cheap)
  • Layout: operations involving visual rendering of musical symbols (computationally expensive)
  • Synth: operations involving audio rendering for playback (computationally expensive)
  • Model: data construct for GUIs (palettes, templates, etc.)
  • Viewer: GUI that uses the Model to provide playback facilities
  • Editor: extend Viewer to provide editing features

Comments

I was asked to clarify what I meant by "dialect", so I thought I may as well copy my reponse in here:

> It means a different form of music notation. For example, we use Stave Notation (a.k.a. "Common Western Music Notation" or CWMN), but there are other forms of notation like Guitar Tablature, Jianpu (numbered notation), Braille (music notation for blind people), Gregorian Chant, etc.

The point is that MuseScore was originally written for CWMN and other things like guitar tab have been bolted on afterwards. It would be very helpful to have a low level model that is abstract (i.e. it does not make any assumptions about which dialect is being used) as this would make it easier to add more dialects in the future. Of course the focus will always be on CWMN, but an abstract model is helpful even within CWMN for things like MIDI import and rhythmic grouping of notes, beams and rests.

An abstract model is basically a pianoroll with a bit of markup to recognise things like trills as one event rather than as sequence of smaller events like they are in MIDI.

In general, I agree that some kind of modularizing should happen but I believe there are two different aspects here.

The first is the fact that current MuseScore codebase is too monolithic, and things that are logically independent are often coupled too much together in MuseScore. In that regard I totally agree that the situation should be changed, and we should move to some more modularized architecture that reduces components coupling, for those reasons you listed in your post.

However, if I understood you correctly, you propose something bigger than that and propose MuseScore to be actually composed of separate modules that could be used absolutely independently and would have a stable API. This state would probably be ideal but I believe that this would cause a lot of negative effects.

Currently we have some number of features that should maintain some degree of backward compatibility (the most important is probably file format). After moving to a totally modularized architecture we end up having a whole bunch of such features. Maintaining compatibility of each module requires designing their public API very thoroughly, and it is still likely that compatibility requirements will cause some less maintainable solutions to be chosen. Even since the last large file format change we have already at least changed text styles representation and had to add a compatibility code to read old text styles representation, there were probably some more changes that I don't recall precisely right now. I believe that not having to provide a stable library API does a lot of benefit to us saving us from extra work required for keeping extra compatibility points.

Overall, I agree that we should make MuseScore architecture much more modular but I believe that we should be very careful in declaring some parts of MuseScore as separate modules or libraries having stable API and available for usage besides MuseScore.

In reply to by dmitrio95

"Stable" is a relative term ;)

I'm not suggesting we create something that other people can depend on being there forevermore, but with the right planning I think we can come up with a core set of abstract classes that would be relatively unchanging. These should be properly documented (in the code) so that the community can extend them to create concrete classes that add new features, like support for more notation dialects, etc, using our own concrete classes (e.g. for the MSCX format) as a guide.

The new text style representation could have been introduced without any changes to the file format. The old format stored whether text is bold, italic or underlined as three separate boolean properties, but now they are treated as one "style" enum that has a single reset button in the GUI. The GUI could still have this single reset button even if the properties were three booleans in the internal C++ code, and if they were a single enum internally then they could still be three booleans in the file format. Of course there is an advantage to doing things the same way in the file format as they are in the code, and keeping the code in line with the GUI, so the developers were absolutely right to change them all at once in time for the release of MuseScore 3. However, the point is that it is possible to change one without changing the other, so we should not be restricted from making similar changes to the GUI and internal format during the MuseScore 3 release cycle just because we can't update the file format until MuseScore 4.

Modularising doesn't have to stop at the C++ classes. Much of the build system code for packaging and bundling MuseScore for the various platforms could be separated out, and the same is true for resources like fonts, templates and workspaces. This has already been done for the soundfont, and work is underway to do this with the icon assets. This kind of modularisation is relatively straightforward, and has no impact on compatibility.

Of course the bulk of the effort would be in the C++ classes, but everybody would see the benefit of sorting out the header files, and of reducing the learning barrier for newcomers - not to mention for maintainers in reviewing their code! It doesn't have to happen in one go, and it can be small things like making new directories and insisting on proper documentation, rather than shoving everything undocumented in the top-level directory. I think we should start to take positive steps in this direction now to avoid a lot of pain in the future. In this way, we might be able to slowly build our way towards a fully modular system without even noticing it.

In reply to by shoogle

Well, then I misunderstood the original message a bit.

In that regard yes, I agree that the code should be made less monolithic, and it seems to be better to do that with some small steps to make it easier to control consequences.

Do you propose to discuss some more details of how this could happen or is this topic intended just to raise the very question of MuseScore modularizing?

In reply to by dmitrio95

I think you understood the end goal fine, but perhaps I mixed together the end goal and the intermediate steps a bit too much.

I certainly wanted to raise the question of modularizing, and to demonstrate to the project leads that there is awareness and support for this issue among developers. I would ideally like get a response from the lead developers about where they stand on the scale of "we will actively support this effort" to "we'd like it to happen but it will be up to the community", or even "don't bother".

I think I have already proposed some details about how it could happen, so I would welcome feedback on those suggestions. I expect most comments will be along the lines of "while you're at it, can you also do ...", which are welcome, but I would ideally like to get some more suggestions about how the modules should be structured.

If you are ready to start major rewrite of MuseScore, then you should also look at the core data format that was chosen to save notes. It is currently measure based and that makes several things if not impossible at least very challenging to implement, such as an insert mode.
One should make the list of what is felt as much more complex than it should, and select a new core format solving most of these difficulties.
E.g.
-Why is it so difficult to change a tuplet?
-...

I don’t have to much to say around the exact breakdown of MuseScore. However, please oh please write readable code/use comments when needed. And document fully anything touched for future contributors. Too much code suffers from lack of documentation and functions that do 10+ complex slightly related things. Modularizing would be the perfect time to set the new standard for code documentation in MuseScore.

Do you still have an unanswered question? Please log in first to post your question.