Developing a Plug-In for Real-Time MIDI Inputs (Marching Percussion)

• Nov 16, 2020 - 15:21

Hi Musescore,

I am a Computer Science student at a community college. I am working as part of a team to develop a project building off of Peter Jonas's work with the Real-Time MIDI inputs for Musescore. I have been a very big fan of Musescore and have used it extensively for the past few years during my work as a marching percussion educator.

The project our team is working on is to find a way to create sheet music from real-time MIDI inputs using a USB drum pad. I was wondering, is it possible to speak with the developers for Musescore with some more specific questions? I was finding it difficult to access the contact information through the repository on GitHub and wanted to reach out here where the message may be seen sooner.

Thank you for your time and I hope that I hear back from you soon.

Comments

• Nov 16, 2020 - 15:58

Hi jeetee,

Thank you for the quick response and your help. Can you clarify what GM stands for?

As I understand it, the problem inherent with using the drum pad is that there is no reference to set up the metronome to run while inputting rhythms through the pad. When the plug-in was written for a piano it was able to use the pedal to advance through the score to derive the rhythms, while using a USB pad doesn't offer that compatibility to my knowledge because the only MIDI control is by hitting the pad. For reference, I am using a Bitmi USB drum pad on the most recent version of Musescore.

When I choose the "Real-Time Note Input (automatic)" input option, it just adds a quarter note and advances to the next beat. Furthermore, I don't know whether what is already written can handle the note density/complexity of marching percussion as well as potential latency. I agree, I feel like what is there would just work but so far I have been unsuccessful. The group members and I would like to help and add on what we can to help solve this problem.

Thanks!
-drumzzzT

Reply

• Nov 16, 2020 - 16:13

GM = General Midi, an industry standard definition which includes information for which percussive sound/instrument is linked to which "pitch"

I'd indeed expect you'd resort to the automatic mode for drum input as you don't really have limbs left to feed a timebase. As for that mode, make sure to select one of the smaller durations to be used as a tickbase.
There is no support for tuplets of any kind, nor real quantization that I'm aware of.

Depending on how "natural" you want to be playing whilst inputting notes, you might in the end be better off using a dedicated program to record the MIDI, clean it up a bit and then open it with MuseScore afterwards.

Reply

• Nov 16, 2020 - 17:03

Hi jeetee,

I appreciate the clarification - I should mention the Bitmi is a single input and not a whole drumset; it would be replicating a singular marching snare drum. This would allow for the use of a pedal but I would not have a way to connect it to my computer, so I was hoping for an option or a way to add it to Musescore.

Ideally there would be a way to turn on a metronome, have a 4 count lead-in, and then begin playing along with it until the measures run out or a user inputs a "stop." My thinking is that you could calculate the tuplets by counting ticks against each beat after they are recorded in real-time. I believe we spoke in another thread about how Musescore interprets ticks, with that information of the "tick" and "utick" being used to solve the problem.

However, this may be asking too much of the system and what you suggest may be easier where the MIDI data is recorded, cleaned, and opened into Musescore. Do you have any recommendations on a dedicated program? The team was thinking of putting something together in Projucer but I am not sure how to transfer that data over to Musescore. Any direction you can provide would be greatly appreciated, and hopefully Peter can add his thoughts here as well.

Thanks,
-drumzzzT

Reply

• Nov 16, 2020 - 20:04

Hi jeetee,

I appreciate your candor - the team and I will look into the information you provided and wait for Peter's input as well. Do you have any recommendations for other places to look within the Musescore repo/forum or other people we could contact? Thank you for all the help you have provided.

Thanks!
drumzzzT

Reply

• Nov 17, 2020 - 10:29

Hi drumzzzT

In https://musescore.org/de/node/311656 I wanted to know who is successful to record MIDI sequences with his accordion. Nobody answered. Maybe I should have asked, if somebody successfully records MIDI sequences with his instrument.

Anyway, I programed a lot of tools for recording MIDi sequences in a ".mid" file, quantizing the notes in the sequence, and a lot more. If you are interested in my programs, please let me know.

If you speak German, it would much easier for me.

Reply

• Nov 17, 2020 - 15:18

Hi jmueller5524,

This is great news! What you're sharing is exactly what we're looking for but with a different instrument. Since a marching snare drum is effectively monotonal, it should be much easier to work with for interpreting the rhythms to display in Musescore.

Any information, programs, tools, etc. you can share would be invaluable. Please let me know if you have a GitHub repo or email address I can contact you at, or we can stay in the forum thread as well.

Unfortunately I do not speak German, but I hope that Google Translate will suffice. We can always look elsewhere on the internet for help with translating too (I'm sure there's a subreddit for this).

Thank you so much!
-drumzzzT

Reply

• Nov 18, 2020 - 09:45

Hi drumzzzT

Let's start an interesting experiment step by step. The first step is the MIDI recording. Some programs are in
juerg5524.ch/temp/Recorder.zip
In the first step you only need the MidiRecorder.exe. I will adapt further programs to your needs.

We should do this experiment without MuseScore. When we get to the goal, I'll make it open source. At best, it can be integrated into MuseScore.

Regards
Jürg

Reply

• Nov 17, 2020 - 17:12

Hi drumzzzT. I'm Peter, author of MuseScore's real-time MIDI input.

My code was originally written with a MIDI piano in mind, but it should work with any MIDI instrument that has the capacity to sustain notes indefinitely (e.g. electric piano, wind instruments, and bowed-strings). It might work with other instruments (e.g. percussion and bowed-strings) but it may require a few changes.

If you want to test it and you don't have a MIDI piano available then you can can press "P" to bring up MuseScore's on-screen keyboard.

Automatic Real-time

Click and hold a key on the on-screen piano keyboard. Wait until the note grows to the required duration.

Manual Real-time

Click and hold a key on the on-screen piano keyboard. Press the Enter key on the numeric keypad repeatedly until the note grows to the required duration. (The Return key doesn't work; it has to be the Enter key.)

If your keyboard doesn't have a numeric keypad with Enter key then you can specify a different key in Preferences > Shortcuts. Search for the action called "Move the cursor forwards in real-time input mode" and define a new shortcut for it, such as '\' (backslash).

Using a MIDI drum set

Like @jeetee said, drum input might work already, but there are possible issues.

MIDI note-off events

The obvious problem that I can see with using percussion instruments for real-time input is that they have no method of sustaining notes. There is no way to "hold a note for the required duration", or "hold a note and press the advance button". At least, that is the case with physical percussion instruments.

With a MIDI drum pad it's possible that, rather than striking the pad with the stick, you can simply bring the stick in contact with the pad and keep it there for the required duration. Hopefully the pad will not send the MIDI note-off event until the stick stops touching the pad.

Alternatively, you could try striking (and releasing) the pad as normal, allowing it to "vibrate" for a while, and then using your palm as a "mute" (or lightly pressing the pad with the stick) to stop the vibrations (i.e. send the note-off event). I don't know whether any USB drum pad has this feature, but it's worth a try.

If your drum pad cannot sustain notes then you will need to change my code to add an option to ignore MIDI note-off events. When you hit the drum, you'll want MuseScore to keep extending the note until something else happens, such as:

  • You hit a different note
  • The cursor reaches the end of the measure
  • You hit a special note (or pad) that is assigned to "Rest" in MuseScore's MIDI remote control (found in Preferences > Note Input).

Of course MIDI note-off events should be respected by default so that real-time input continues to work as expected on keyboard instruments.

Pedal

You don't have to use a pedal to move the cursor forwards in real-time model. If your USB drum set has multiple pads (or multiple pitches) then you can assign one of those pads to move the cursor forwards. This is already supported via MuseScore's MIDI remote control.

Simply follow the instructions at 4:05 in this video and hit the relevant pad (or MIDI pitch) instead of the pedal. Once assigned, you can use your left hand to tap metronome beats on the "advance pad" and your right hand to do the rhythm on the other pads (or vice versa).

If your USB drum set only has one pad then you will need to use the Enter key (or different keyboard key) as I explained above) or keep to Automatic mode.

Plugins

As a technical aside, my real-time code is not a plug-in. When we talk about plugins for MuseScore, we usually mean QML (JavaScript) files that are loaded in MuseScore via the Plugins menu. My real-time input is actually part of MuseScore's internal C++ code, which is where you will want to implement your code to enable drum input if required.

Reply

• Nov 18, 2020 - 17:29

@jmueller5524, recording with a microphone is great for users (if it works) but I fail to see how that will make things easier for developers. As a developer, you would need to do some fancy processing on the audio waveform to detect when notes begin and end, whereas MIDI gives you this information for free!

Also, a microphone won't work for percussion instruments, which is what this discussion is about. It might tell you where the drumbeat starts, sure, but how will you decide where it ends? How will you know whether to notate the drumbeat as an eighth note, quarter note, or half note?

Reply

• Nov 19, 2020 - 19:52

Amplitude can't tell you the duration of a percussion note. Drumbeats are over almost instantly so the duration is effectly zero, yet they are notated as though they have a duration (half, quarter, etc.) so you need some way to determine the duration other than through sound.

Reply

• Nov 19, 2020 - 02:51

Hi Peter,

Thank you so much for taking the time to respond and write such a detailed description of your work and how to approach this addition! It really means a lot to me to provide all this help and guidance. I hope that we can stay in contact through this thread as the team and I work on this.

After reading through, it seems like the concept we will need to address is: "If your drum pad cannot sustain notes then you will need to change my code to add an option to ignore MIDI note-off events." The goal will be to apply this idea to the code, and find some way to have this note "on/off" event have a reference in time for a given beat and measure. Ideally, the metronome would be running the entire time through the score and the MIDI inputs would be played against it. Do you think this is possible?

I will in touch over the next few days as we start to shape this idea into code. Again, thank you so much! I really appreciate all the information you have provided.

-drumzzzT

Reply

• Nov 19, 2020 - 20:07

You're welcome!

It is certainly possible to have the input metronome run through the entire score. If you press the pedal (or alternative keys mentioned above) in Automatic mode then it will start the metronome counting, and that count will contine until you press the pedal again. So the pedal is not just used to send individual beats in Manual mode, it also functions as start/stop metronome in Automatic mode. This enables you to enter rests in Automatic mode.

So you are correct that the only thing you need to do is add an option to ignore MIDI note-off events, and add an alternative way to guess where those events should occur instead (e.g. end of measure, start of next note, hit on "Rest" pad). You could also use a low velocity hit on the pad to signal the end of a note without starting a new one.

Reply

• Nov 19, 2020 - 08:10

There is a simple way to integrate the pedal: Buy a cheap midi keyboard with a sustain input. Make a recording for the sustain and the bitmi together with my MidiRecorder (or other).

Sorry, I got it wrong with the pedal! Unfortunately, my English is not that good.

Reply

Do you still have an unanswered question? Please log in first to post your question.