"sing your melody" - Is there MIDI and/or audio input for plugins?
(Apologies, if this has been suggested before - please direct me there then.)
Is a MIDI input or audio input available for plugins?
I am sporadically thinking about starting tinkering with a "sing/hum your melody" type plugin. It would record audio, analyze (find pitched notes), then interactively transcribe it in cooperation with the user.
First the plugin draws the notes on a continuous time (no tempo, no measures, no beats) vs. MIDI-note-number-graph, and allow the user to pull the notes into place in pitch and in loudness/velocity and in time - with the possibility of playback of the current state - until they are satisfied with the notes sounding as they want them.
Second comes the transcription into a score. Here the user overlays tempo/beats/measures/clicktrack/whathaveyou grid on the time axis, and flexibly massages the played notes and/or the tempo/measures into the musically correct and meaningful timings and lengths; if there is a tempo change, the user can "cut" the tempo grid into sections, and drag the grid wider or narrower if needed, with some musically meaningful snapping of the tempo grid to the existing notes (in an advanced version, even gradual tempo changes could be drawn like this). Ideally, a mini-score/staff is continuously shown below to show (at least partial) feedback about how the transcribed score would look, so that the user can be satisfied with the note lengths and positioning, and that the score concisely but precisely captures their actual musical idea.
Finally they can commit the thusly transcribed piece into the underlying actual score.
There could be an option to keep the audio recording (or perhaps just the extracted pitch contour) and other settings attached to the score, so that the user can return to it to amend transcription parameters when later needed.
Useful as a practical tool for getting musical ideas more directly from instrument to score - but even more importantly as an educational tool, to very graphically help beginner musicians/composers learn how notes/pitches they have in their head and can sing or play, actually convert into rhythms and notes in a score.
For MIDI input, the pitch estimation step is omitted, but beyond that the experience would be pretty much the same.
Not a small feature for sure. But we could work through it in multiple iterations, start with an MVP (the smallest set of features that are already useful), and gradually enhance.