Ideas for custom trill creation (Articulation Editor)

This is the follow-up for the Trill Creation Feature Request here [url=https://musescore.org/en/node/277427#comment-881302]. This is an implementation thread, suggesting a possible interface for the request, and discussing how easy it would be to merge into the main development once the desired result is resolved.

The idea is to discuss the feasibility of developing a workable but thoroughly customizable Articulation Editor (of which we'll now call from now on, or AE for short). With the AE dialog, you can customize the PLAYBACK of any articulation, and save them into a custom palette. This includes trill, grace notes, turns, tremolos and rolls, accacciaturas, appogiaturas, arpeggios, and glissandos. It does not affect the notation or its display, only the playing of the note articulation. This dialog will not affect the timing of fermatas, the velocities of dynamic marks, or the volumes of sustained notes.

First let's agree to some terms, which will be used in this article.
NOTE: A single notehead depicting a particular tone, whether part of a chord or not.
NOTE-CHORD: A single segment with one or more notes, all stemmed together if less than a whole note.
ARTICULATION: The symbol indicating that a single note should be played different than just striking the note, usually a sequence of two or more additional notes played over time.
RESULTING (OR ARTICULATED) NOTES: The individual, implicit played notes of an articulation. These are found in playback only, not in the score.
(ARTICULATED) SEQUENCE: The series of resulting notes as a whole unit, such as those of a turn, mordent, etc.
SOURCE NOTE: The original note the articulation is attached to, and the starting pitch (usually) of the resulting notes.

The AE is intended to be an ADVANCED feature, requiring an intermediate knowledge of music notation.

Below is the original envisioned GUI dialog.

Interface
This window should account for many variables involved with the playing of note articulations. It should display the desired result (i.e., the notes that would play, even though they are not displayed in the score), ideally in a simulated staff view (showing its approximation on a staff), as well as the Piano Roll. These resulting notes are for display only and wouldn't be editable from the preview pane.

Preview
You can choose the view mode by selecting either Piano Roll or Staff. As you enter the intervals, the preview is updated to show the notes that will be played according to the parameters set below.
Below the Preview pane are the options for developing the playing of the notes.

Interval List
The first one is important: the Interval List. You will assemble the list of intervals that are played when the articulation is encountered. It is a comma-delimited list of integers, starting with plus or minus, played one after another in the same equal time (the timing of these notes is discussed later) relative to the source note. For example, a simple turn might show the list as "0, -1, 0, +1" and a mordent might be "0, -1, 0, -1, 0". You can choose intervals up to an octave, but this octave might be defined in different ways (see the section on Scale). Negative numbers reduce the MIDI note value, which has the effect of lowering the note in pitch. Positive values raises the pitch by one note of the desired scale.

Timing Options
Below the Interval List are the timing options for how the individual notes will be played, based on dividing the resulting notes, multiplying the sequence by absolute or relative amounts, then either adding a delay or subtracting from the initial playback time.

DIVISION
In this section you can define how you want the resulting notes to be divided, which will directly affect the timing of the notes from the Interval list.
Select "Limited by" to choose the length of each resulting note in absolute terms, in milliseconds. You can then enter a range of times that the resulting notes will play at (if the minimum is the same as the maximum), the resulting notes will all have that same length in milliseconds. If the min/max values are different, their values will be gradated slowly from the first resulting note to the last.
Alternatively, if you want the articulation to be dependent on the BPM (tempo), select "Beat durations" and select the note lengths of the resulting notes. You can also divide the note lengths by a one or more parts of a tuplet.

MULTIPLICATION
You can multiply the number of times you want the Interval List to be played by selecting Repeat and entering the count. Keep in mind this playback will be interrupted at the end of the duration of the source note.
You can also stretch the articulated sequence if you enable Stretch and enter a percentage amount. This will multiply the playback of the resulting notes by that percent. Entering 50 will shorten the resulting sequence by half (playing it twice as quickly), and entering 200 will take it twice as long to play the whole articulation sequence.

ADDITION
Enabling "Delay" will add a percentage of the source note's duration to the initial playback of the articulation (the source note will still be played on time, unless you've unset the check box for Play in the Inspector). If you delay too long before the articulation sequence is played, it will be cut off when the source note ends. Enter 0 to have the Delay happen at Note-On, or 50 to delay playing the resulting notes for half the length of the source note.

SUBTRACTION
The articulation can be opted to play BEFORE the source note by selecting that checkbox. This will cause MuseScore to start playing the articulation before the beat of the source note, interrupting the playback of any notes previously playing in that voice or instrument.
Keep in mind that the Addition and Subtraction settings cancel each other out. So if you set the Delay and then select Before Source Note, the Delay is disabled, and vice versa.

Miscellaneous Playback Options

SCALE: Under Scale, you can select the scale that the resulting notes will adhere to. Chromatic uses the standard Western 12-tone scale, regardless of key; ±12 is a full octave. Choose Diatonic to make sure the articulated notes stick to the key (±8 is an octave). In the future, we'll hopefully be able to add other non-Western or custom scales the resulting notes will follow.

SMOOTHNESS: With Smoothness, you can select Legato, Normal, or Staccato playback. With a legato smoothness, the INDIVIDUAL resulting notes will play back at 100% of their duration. With Normal, 95%. With Staccato, the resulting notes will be shortened to 50% of their normal duration.

RESTRIKE ACTION: This allows you to choose how playback will handle a note that's struck again within the note articulation. When you select Cut Note, the note is stopped, its audio cut off, and the note is replayed from the beginning of its envelope. With Note Fade, the restruck note is stopped, but the Release part of the envelope is played (which is stored in the soundfont file of that instrument) while the note is played again. If Continue is chosen, the note will continue to play in its Sustain loop (or Release node if no Sustain is defined) as the same note is struck again. This last option will eat up CPUs fast if the instrument's sample data does not end the note on its own; there are very few times when this option will be desirable.

SYMBOL: The AE also allows you to customize the look of your articulation. You can opt to choose the standard symbol found in MuseScore (and toggle its visibility); click on the symbol to open a palette to choose another.
Or you can provide a path to an image for your articulation. When selecting a custom image, it must be supported by MuseScore, and a frame is added to the score, attached to the note that's been assigned the articulation, and the image stored in that frame. Wherever the note goes, the image will stay with it. You can then adjust the positioning of the image like any other. (Once placed in the score, you can also save and reuse it in a palette like any other.)

INSTRUMENT: You can assign a different Instrument that the resulting notes will play rather than the instrument assigned to the source note. Normally you'll not want to do this, but there are circumstances where articulations are better sounding when using a different instrument within the soundfont. Keep in mind that the source note always plays the Instrument assigned to it by the staff.

NAME: This name will show up in the tooltip of the symbol if you save it in a palette.

Click Save after you're done to save your changes. Close the window with Escape or clicking the Window X to cancel any changes.

Implementation Notes
Probably the most difficult aspect of this dialog is the Preview pane, which borrows some methods from both the Piano Roll implementation and the Staff drawing implementation, but is not editable. I hope any dev who wishes to take this up doesn't need to reinvent this wheel.

The rest is fairly easy i hope, but i wanted to include some notes on the implementation of the data controls in the window.

1. I would hope that MuseScore can check the Interval List, even as data is being entered by the user, and update the Preview in real time when valid data is found. Perhaps invalid data in the field can be highlighted in red after each character is entered, until MuseScore understands the interval list completely. If this can't be achieved, then a separate "Go" button (or something similar) can be set next to the text field, and the user can press that when he's entered the list of intervals. MuseScore will then check the syntax, and if it's not in right format, alerts the user to correct the data.

2. Speaking of valid syntax, MuseScore would expect to find a series of numbers (preceded by plus or minus, with 0 being the exception), up to the value which denotes an octave (based on the Scale setting). This limit, however, is arbitrary, and is meant to reduce problems that may be encountered. But i don't know of any note articulations that exceed an octave in range. The numbers denoting the intervals should be relative to the source note, not the previous resulting note.

3. In almost all cases, note articulations apply to only one note. If an articulation is allowed to affect all the notes in a chord (creating, for example, mordents of two notes layered over each other), we may need to have the user specify if he wants multiple articulations or to one note only (perhaps the last selected or top note of a chord).

4. I'm hoping that calling note articulations that play notes BEFORE the source note will be easy to implement. Currently, when inserting grace notes, no matter how many you enter, the first grace note stays on the same beat as the source note, which is not often the intended effect. I would like any of the defined "pre-articulations" to be inserted before the beat of the source note, shortening any preceding notes still playing.

5. I don't know if alternate scales can be defined by MuseScore, so that different keys will diminish or augment notes of that key, but if so, it would be easy to add other scales besides Chromatic and Diatonic, such as Arabic, whole-tone, etc. Although it may be entirely unnecessary.

6. The Restrike Action may or may not be workable in MuseScore. Other more sophisticated music players can define what happens when the same note is struck when it's already playing, but i know very little about how soundfonts work. If each note abides by envelopes that can continue on EACH note strike (even if the same note is hit again), then this should be workable. The reason we want to define this is so the user can customize the playback based on the instrument. For example rolling a crash cymbal does not start the cymbal sound again (necessarily), but if the soundfont will allow it, the original strike is processed while the instrument is hit again. I can see it in certain synthetic sounds as well.

7. In the same vein, i don't know if MuseScore will play back a different Instrument without a registered Instrument Change. This might be necessary if an articulation would require a different envelope of the same instrument. For example, playing a trill on a flute only calls the same flute sound over and over, but starting the flute sound introduces a quick but noticeable delay, which becomes a problem when playing legato notes in quick succession. If the Instrument's envelope at the Delay point (bypassing the Attack), can't be called by MuseScore, then maybe then next best workaround is to call a second Instrument, which in this case would be a flute sample without the attack portion of the envelope. This of course would have to be within the same soundfont, if i'm not mistaken. I can't think of any other scenario when playing a different Instrument would be necessary in an articulation, except maybe a drum roll.

8. I'm also hoping that a different image can be applied in a custom articulation. While i understand (i've been told in other posts) that the goal of MuseScore is not to harbor inventions by composers, but rather abide by universally accepted notation standards, i would hope at some point that MS will allow for non-standard notation or playback techniques. In this vein, i thought the option to apply a user-defined image would reduce the number of articulation images not only that MS must have at its fingertips, but also that users must search the Master Palette for.

9. Accessing this dialog could be done when any note is selected. You should be able to press a shortcut key, or access it from the context menu of the selected note. If you want to EDIT a given articulation, the window would pull up its info (and prepare this window) when right-clicking the articulation.

Conclusion
I hope this inspires someone to take the reins of this idea and see if it can be coded. This whole post is simply theory and provides a direction for any possible implementation, but i hope it will begin the discussion of its parameters, requirements, and limitations.