[Accessibility] Announce when arpeggio crosses staves or voices

• Oct 25, 2020 - 02:13
Reported version
3.5
Type
Functional
Frequency
Once
Severity
S5 - Suggestion
Reproducibility
Always
Status
active
Regression
No
Workaround
No
Project

For accessibility purposes, there needs to be some feedback that can be accessed through NVDA or other screen readers when an arpeggio is selected. There is no indication that an arpeggio has been extended to staff 2 or modified visually in any way.

Relates to #277496: [EPIC] Accessibility issues/suggestions and #288373: [EPIC] Arpeggio issues


Comments

The announcement should be, if the cursor is tabbed to the upper or bottom handle of the arpeggio, which staff/voice is it extended when pressing a defined key, so that blind musicians can easily make cross-staff and cross-voice arpeggios without any sight assistance.

Since we don't actually support cross-staff arepggios, there really is not useful feedback we could provide aside from a number representing the length of the arpeggio - that's what you are actually manipulating. But there really would not be any way to deduce from that when you have reached the point you want to reach.

So for accessibility purposes, really, there is no substitute for actually implementing cross-staff arpeggios. But adding a representation of the length to the Inspector or the status / screenreader feedback wouldn't hurt of course.

In reply to by Marc Sabatella

Thank you, this can partially make sense. I'm not going to impact the development of Musescore 3.x, but if we work for Musescore 4, can we add this as a feature by both changing the internal syntax? The arpeggio should not be an articulation coded in mscx format, instead, it should be a spanner like slur and tie, then the solution is here. Make it an id and specify the starting and ending point. In Musicxml, nobody has used the start and stop attributes of arpeggio, but Michael Good said this is correct. When we enter an arpeggio like this, the braille transcription software will also correctly recognize if the arpeggio is a cross-staff object, and give correct braille symbols. Just my thought to Musescore 4's groundbreaking feature.

When you apply an arpeggio to the top staff, select the bottom grab box and press either shift+down arrow or ctrl+shift+down arrow, the arpeggio is extended to notes in the next staff down (grand staff only) in the same voice and the anchor point (red line) when you select the bottom grab box points to the bottom note. MuseScore knows the arpeggio is cross staff. I haven't looked at how this is represented in the .mscx file but it also survives save and reload. This also doesn't affect playback so there is a separate issue for that.

In reply to by mike320

I just looked into my Jasmine Flower arrangement. The cross-staff arpeggios do have a "span" attribute with the value 2, indicating it spans for two staves. But when the arpeggio applies on two voices withing a single staff, the confusion comes: there's no "span" value, but a "userLen1" with undefinite values, sometimes 2, sometimes 3 or 4, which can't be inferred. So the feature request of redefining arpeggio as a spanner is not needed, but:
1. We should have Musescore say the arpeggio is spanned to another staff since the span tag is here, and it should know this;
2. if multiple voices in a staff is met, sthe userLen1 is used, and Musescore announces voice crossed.
This may solve the accessibility issue here. However, I don't know how Musescore will code if an arpeggio spans from right hand voice 2 to left hand voice 1 when there are both two voices in both hands :-)

I'm attaching the test file Mike320 helped me to do. In measure 1, arpeggio spans through two inner voices, and the code shows <span>2</span>; in measure 2, arpeggio spans through the lower 3 voices, both span and userLen1 are used; in measure 3, there's only a very large userLen1 value, which will prevent us from inferring how many voices it spans. in fact, the last one spans through all 4 voices, and when I looked into other scores by other editors, I found the <span>2</span> doesn't appear at all, since only Mike320 is using the keyboard method to span arpeggios instead of pure mouse dragging. This still raises the problem of both accessibility announcement and braille transcription.
Here can be another solution, which will less impact the syntax of Mscx coding: Keep arpeggio as current element, then set a connection attribute, and every voice having this arpeggio will contain this connection attribute. I got this method from Lilypond, where there's a command PianoStaff.connectArpeggios to set to true or false, and when cross-staff arpeggio is used, every voice and staff should write arpeggio command, and Lilypond will automatically connect them together. Also, for Musescore playback, this command will generate correct Midi by looking at the command. If this can be a good solution, I can add this as a separate issue or feature request for Musescore 4.

Haipeng

Attachment Size
Arpeggio test (1).mscz 8.76 KB

I'm curious if I press shift+down arrow, nothing happened then I dragged it. I'll send it to you again making sure I don't do that.

Here's the second file Mike helped me to try. At measure 1, he now just uses drag, no keystrokes. The result is, only userLen1 is here, no span attribute at all. So, the situation is, a dragged arpeggio will do nothing good to braille, and only cross-staff arpeggios can benefit braille by using keystrokes. Should we do feature request?

Attachment Size
Arpeggio test (2).mscz 8.76 KB
Title [Accessibility] Arpeggios need feedback in either inspector or status bar [Accessibility] Announce when arpeggio crosses staves or voices

As Marc said, if this is too difficult then the fallback option is to just say when the arpeggio has been made visually longer than normal