Status and GSoC Planning!

Posted 4 years ago

Hi,
some things happened since the last post. So here is a update on what I'm up to. If you don't have so much time please scroll down to the second part (starting with feedback wanted) and leave a comment. I'm very interested in what users thing about my plans for GSoC!

What happened since the last post?

Metric modulation

After Jojo-Schmitz had a look into my very first PR to MuseScore - which is about fixing problems with parts and tempos (see #78416: Tempo text oddities in score with linked parts until save/reopen and the PR) - he proposed to add metric modulation (see #110331: add suport for metric modulaton, incl. playback). This task seemed relative simple and so I did it. Made a PR and it seemed that some things weren't quite good - have a look at the PR. Biggest thing was I thought using 0 (or actually better 0.0 ;)) as default for the relative factor would be a good idea. Using 1.0 as the default makes much more sense. We thought we could get rid of the float comparison - but unfortunately that isn't the case. So I used a Boolean property to check whether it is a relative Tempo Text. There were also some style issues: wrong spaces, not using toTempoText() where it seems appropriate (this is new style for 3.0), fixing up old code using foreach (in 3.0 we like to use C++11 style for (thing : things) instead) and other minor style issues.

So this was a very good opportunity to get a better feeling for MuseScore's coding style and what you should do in a PR! I learned some git related things as well.

I optimized parsing of tempo text and added some common metric modulations to the palette. (Did also some refactoring there!) The next step will be adding some tests.

CC11

I implemented CC11 in collectMeasureEvents() of libmscore/rendermidi.cpp. I did it this time by setting the velocity to 127 and then doing all the volume via CC11. The change currently ends just at the end of the event and adds CC11 with value 127 at the tick after that. It doesn't sound to bad (and I had some errors with decrescendo the last time) but certainly isn't optimal since velocity can change other things than just volume - like which sample actually to play. However it shouldn't make a huge difference for MuseScore's standard soundfont.

I'm currently working on some GUI things. I think I'll add an option to hairpins which will enable CC11 events and have the default setting based on the instrument.

More thoughts down below.

Investigation on ornaments

I found #110771: trills with dynamics play back unevenly and thought it would match my project. Seems #77376: Trills and other ornaments play incorrectly on tied notes is somewhat related to this issue. I think there are a lot other issues as well. But it seems ornaments are actually more complex then I thought - especially with ties! The main problem is: if you imagine you have for example a trill (either a line or just the symbol) on a note of a tie you want that the trill play to the end of all tied notes. That is especially an issue with the symbol because the tied notes following the one with the symbol don't "know" that they need to trill (maybe they should in the future?). So you could just render the trill from the first note until the ties are over and skip the other notes of that tie. But that behavior doesn't make so much sense for other ornaments which you put on a note in a tie which just play on that note for example - so which one should be rendered which one shouldn't?
The current behavior is that every note of a tie gets the ornaments rendered for this note separately until the end of the tie. If you have a trill line for example the generated events add up and you get undesired effects like the ones mentioned above.

Another big problem is that there is no (de)crescendo for any ornament or glissando. That is because currently the events that generated from the ornament/glissando do not contain any velocity information. They are just events like "play a note with this duration and this relative pitch at this time". And then they get the velocity from the note they are part of. If you would just take the velocity from the current position (what is possible since there is a velocity map that gives you a velocity for every tick in the score) you would loose the ability to change the velocity of single note "by hand". So I'm not quite sure right now what would be the best solution for this. I'll give some ideas down below.

QtCreator 4

QtCreator 4 was released - and everybody screams "yay". But I couldn't compile MuseScore anymore. QtC4 handles CMake different than QtC3.6. It is more integrated into QtC4 and "automatically" runs CMake again if it thinks it would be necessary and you also don't provide a command line parameter anymore - instead you see all parameters in the project options now. So my problem was that I needed to change "CMAKE_BUILD_TYPE" to "DEBUG" in that particular window and everything was good. I had also two problems with the debugger - one made it into a bug report (actually had similar problem with QtC3.6 as well) and the other is that I have the feeling that my expressions sometimes don't get updated. But the problem with feelings is that you cannot generate a bug report out of them. I'll need to investigate more (and have a look at the debugger log) to say for sure if it is me or QtC4.

Plan for GSoC - feedback wanted!

I want to make a good plan how to spend my time during GSoC2016 and what is actually important and what users need and want. I think improvements will fall in the three categories Midi, Fluidsynth and Zerberus. Sometimes these are related in one way or another. So would be happy if you can share your thoughts with me!

Midi related

Single note (de)crescendo (aka CC11)

I think I wrote already a bunch of things in this blog. Right now the most important thing is to think of a GUI that might work. Also I'm not entirely happy with setting velocity to 127 and just use CC11 for the rest as I wrote above. Actually there is the even bigger problem that probably every synth out there handles these events somehow different - so maybe it is a good idea to have something that works particularly good with MuseScore's internal synths. But for sfz as it is very open format there is (as far as I know) no standard way of treating CC11. I'm not very happy with the way it is handled in the SF specification. But if we focus on the standard soundfont we might be happy with the solution I already implemented.

Ornaments, glissando, etc.

The current code for ornament generation is very complex. It might be just a good thing to refactor it somehow that it is easier to understand. I think it would be good to render events just for the note they belong to - if we have a tie and something that should be rendered for all notes of that tie (like a trill) this property should propagate to the other notes. This might be a problem that you have to delete these properties if you delete the tie - but I it is hopefully not to complicated.
To have proper crescendo for these events I think it might be a good idea to treat the velocity of the of the actual note as the base and change the velocity according to the change in the velocity map. For example the velocity map says for the tick at base note position value=68 and for the position of the first ornament event value=72, but the note has a user velocity of 50, than the event should be rendered with a velocity of 54. This way you have crescendos and still have user/articulation changed velocities for the note.
The CC11 code should probably end up somewhere next to that code. (Meaning it should move from collectMeasureEvents() to collectNote())
Further more it would be nice to have glissando also rendered as bends - as it would make sense for fretless instruments, voice or trombone.

Legato

The midi part of legato is probably quite simple. Just use CC68 like a sustain pedal. (Maybe even that code just changed for from pedal line to slur) Furthermore it should have an option to add bends - as for voice for example during legato play the pitch of each notes bends to the other one.

Fluidsynth

Fixes from upstream

As already written in my in one of my last posts I really like to have our Fluidsynth as good and shiny as upstream. But I'm still not sure which patches are really necessary. I think this is quite some work of investigation.

Legato

I would like to implement the treatment of the legato CC68 in Fluidsynth. That is not according to SF spec but I think it doesn't matter for our goals. I would "simply" skip the attack and decay portion of the sound and if it has a loop point maybe start there - if not start where attack and decay would have ended. Maybe it makes sense to add a parameter in intruments.xml to fine adjust the sample start (maybe realized via nrpn messages).
The samples should also crossfade so that there is a nice sound.

CC11

I've thought if it makes sense to abandon (as long as we're using Fluidsynth) MIDI and SF spec altogether and use a whole byte for CC11 and make it bipolar. So that you can actually increase the volume and not just decrease it. This way we could just have the default value of 127 do nothing to the volume and from there we can increase and decrease the volume as we like. That means there is no change to any velocity value necessary. I don't know if it is worth it. (Probably if we had more velocity zones in our standard soundfont?) But let me think what you think!

Zerberus

Zerberus shall be the main part of my work. Planned is looping and ADSR support. Don't know what might be important from there. So dear sfz users - let me hear what you want and need! Since I own a copy of "Cakewalk Synthersizers" by Simon Cann I also have the spec at hand!


Comments

Regarding CC#11

There is no reason at all why you shouldn't use an internal bipolar model. When sending to a synth, however, it should be translated into the standard MIDI message otherwise external synthesisers connected via JACK would be unable to process it.

During my days as a backing track programmer, I used to set CC#11 to 64 at the start of a track, representing mf. I then had values of 63 to 0 to go from mf to ppp (although 0 is actually silence) and 65 to 127 to go from mf to fff.

Perhaps you could use this model for your implementation??

It would be better not to fix the velocity at program level, and leave it to the soundfont to process that as velocity can then be used to switch samples for different attacks - the user can then enter the desired velocity for the note into the inspector and control which sample is played.

Once you have a stable version of this working, then I will start work on adapting suitable instruments in the default ssoundfont which will go in a special GM2 bank we define for this purpose.

HTH
Michael

PS The most obvious way of implementing this in SFZ would be to use the gain_onccN opcode with N=11

In reply to by ChurchOrganist

Thank you for your response. Just let me clarify - so you used just CC11 and didn't change the velocity at all?

About SFZ I know that this would be the easiest solution, but I meant that there is not "this is how it is done in sfz". But actually that is a good think, that we have that freedom with sfz!

In reply to by ChurchOrganist

the user can then enter the desired velocity for the note into the inspector and control which sample is played.

This is the type of things I want to avoid as much as possible. The goal should remain to have the best playback by default. Almost nobody will know that one needs to change the velocity to switch samples/attacks and we don't want MuseScore users to become MIDI programmers.

To phrase it differently, I feel the goals are :
1/ Have MuseScore playback better by default (
2/ Do not make MuseScore more complex than it is right now.
3/ Do not bend standard if it causes interoperability problems. Meaning do not change the SF2 implementation in fluid to accommodate a special soundfont, or do not change the MIDI output in a way that it would be incompatible with external synths (except if there is an easy way to switch configuration, as easy as changing instrument for a staff)

In reply to by Nicolas

I would guess that it would only be a small subset of users interested in this level of control.

It would be great to have it work seamlessly, but normally this would be done by the use of a MIDI breath controller such as the Akai EWI or the Yamaha WX7 (unless you were directly programming a DAW), and so you would have to get MuseScore's playback to behave in the same manner.

I would say that you need the soundfont infrastructure in place before you can begin to provide a seamless interface.

Velocity switching is already part of the SF2 (and therefore FluidSynth) specification so it would not involve changing the standard. The idea would be to provide one bank of instruments which would respond in this way, selectable in the mixer by the user should they require it. Otherwise it would work exactly as normal.

Really it comes down to how far we go to satisfy the ever more strident requests for better playback.

Hello
I am very happy with MuseScore 2.0.3 and watching your developments in sound improvement.
Here are some of my ideas and wishes to improve the playback sound:

1
I like to control the legth of all notes in selected ranges. This is very important for all wind instruments (brass and wood) to make the music less stereotype or mechanical.
At the moment I must do this - note by note - with the pianoroll editor, which is very cumbersome.

2
Then it would be great, if the playback could imitate a more human interpretation - with heavy and light beats within a measure.
P.e. play beat 1 and 3 something more accented than beat 2 and 4 - if it deals with classical music - but maybe different in jazz music. So I wish to control the accentuation for the entire time signature.

For me as a musician with few knowledge of any codes or programming I have no ideas, if my wishes to a comfortable notation program are at the right place here.

Greetings - Rudi