Recognition of score from image or PDF
Hey! I'm Max from internal research team. I thinking about adding feature "recognition of score from image or PDF" to MuseScore (editor).
How it may works:
- User select image or PDF in MuseScore.
- Start processing...
- User see this in score window:
- Green: recognized elements.
- Purple: recognition engine recognized that there is an element, but didn't recognize which one.
- Purple elements aren't valid symbols, they are just placeholders. In place of these placeholders, user must drag the real elements from MuseScore editor.
- When user click save file then all green elements and new elements placed from user are saved to file.
What do you think? Will be it helpfull for transcribers? Will you use it?
How it works from developer point?
It will be use Audiveris engine for recognition. When user select image file, when we run console version of Audiveris and get .omr file from here. Then we:
- read .omr file,
- create new blank score,
- put recognized elements
- draw placeholders from points
I have few unresolved issues:
- Audiveris is licenced by AGPL v3.0 which incompatible with GPL 2.0. I thinking about adding this feature as separate plugin or make cloud version of Audiveris.
- Audiveris is written on Java and have large size. I wouldn't want to increase size of MuseScore distribution.
- I need to make .omr file reader for MuseScore. Audiveris can export files to MusicXML, but some part of data may be lost.
P.S. Yes, I know that MuseScore 2 had own OMR engine. But it can recognize only staves, which is clearly not enough. Also there's no placeholders for unrecognized elements.
P.S.S. Yes, I know that musescore.com have pdf2score feature, but it works on Audiveris 4 and didn't works well. When user sees emptiness instead of the recognized elements, he thinks that something is broken. If we show: "here's should be some element, but we don't know which one" - I think it be much more user-friendly.