Mixer: Designing a more compact and accessible UI
MuseScore 3's Mixer looks much better than it did in MuseScore 2, and the side-by-side volume sliders are particularly useful when you have a second monitor on which to display them. However, if you only have one monitor then the Mixer's relatively large dialog gets in the way of the score and other parts of MuseScore's interface.
Size is not the only problem. There are various usability issues, and the UI is almost completely inaccessible to the keyboard users (e.g. people who are blind or motor impaired, or those who simply prefer using the keyboard over the mouse).
Current Mixer
The relationship between the top and bottom half of the dialog is not immediately obvious. Once it is understood, you realise that volume and pan controls for the selected instrumental part (Oboe II) are provided twice, which is inefficient. (It's actually three times if you include the number spinners, but those are admittedly quite useful.)
Proposed new Mixer
Users are encouraged to first select an instrument from the list, and then to adjust controls for that instrument alone. This saves space and avoids duplication. Master Gain control has been removed; overall volume can be set in the Play Panel.
How I came up with the proposed design
I seem to remember one of @ericfont's many proposed Mixer redesigns included a list (possibly as a dropdown combo box) to select the instrument, but I can't find the post in question. Anyway, it is actually possible to arrive at the new design simply by considering accessibility requirements. If you are not blind (I am not) then try to imagine that you are, and that you rely on a screen reader program to tell you what is happening on the screen...
As a blind user, imagine that you open the Mixer and the screen reader tells you that the “mute button” currently has focus. This is no good! Which instrument is it for? The first thing a blind user must be presented with is an instrument, and a quick way to choose a different one if necessary. This means a list of instruments. Once the blind user has selected an instrument they should be presented with controls for that instrument and no others.
Aside: If you think about it, an instrument list is the only way to arrange the Mixer that makes sense for keyboard users. If controls for all instruments were visible at once then keyboard users would have to press the Tab key many times to navigate between them. However, if controls are only displayed for one instrument at a time then keyboard users don't have to press the Tab key nearly as often.
Visual benefits
The proposed design obviously takes up a lot less screen space, but there are other benefits:
- The instrument list scrolls vertically instead of horizontally (easier to scroll with a mouse)
- We can see more instruments than we could previously (less scrolling is needed)
- Instrument names are vertically aligned (easier to spot groupings like “Corno” and “Fagotto”)
- Names can be longer before they get cut off (e.g. wouldn’t confuse Tenor Trombones 1 and 2)
These visual improvements all happened because of the instrument list, but the list was introduced to help blind users. In other words, the visual problems with the current design are simply byproducts of the fact that it is inaccessible to blind users! If you fix accessibility problems for blind users then the design becomes more usable for everyone.
Further improvements to the design
We can take the accessibility analysis a step further and consider the fact that blind users might want quick access to frequently used controls without having to leave the list. We could introduce new list columns with checkboxes for mute and solo, and of course this benefits sighted users too by making those options visible regardless of which instrument is selected.
Navigating the list with the keyboard becomes tiring if there are lots of instruments, and blind users need to remember the order of every instrument in the list (otherwise they might try to go up the list to get to the instrument they want when it is quicker to go down). We can help blind users by grouping related instruments together in a tree structure, with categories like woodwind, brass, strings, etc. Naturally, this grouping is also beneficial to sighted users.
Aside: A tree structure also enables settings to be adjusted for categories as well as for the instruments. Now users can adjust settings for all brass instruments at once, or expand the tree to adjust only the horns, expand the horns to adjust Horn II, expand again to adjust Horn II in C, and so on to whatever level of detail is required!
So far we have only looked at the list. We could apply the accessibility analysis to the other controls in the Mixer, and eventually everything would be presented in the way that makes most sense to blind users as they Tab over all the controls in the dialog. The optimum arrangement for blind users puts frequently used controls closest to the list, and this naturally results in the shortest cursor travel distance for mouse users. It also means controls that are needed less often would appear on the right of the dialog, so this part of the dialog could be hidden until an Advanced button is pressed, thereby saving even more space. As it stands, the current design is not far off optimum anyway, so the remaining adjustments won't alter the layout significantly.
Restoring the sliders
Having made all possible accommodations for blind users (all of which benefited sighted users too), it is finally time to consider whether there are any enhancements that could be made specifically for sighted users. The only people who will be unhappy with the proposed design are users who own a second monitor. These users will miss the side-by-side volume sliders, but we can simply reintroduce the sliders as an optional feature via a button to expand the Mixer downwards.
Viewing the designs in context
The attachments below show the two Mixer designs with MuseScore's interface in the background to provide context.
The screenshot of the current Mixer was taken in MuseScore 3.0.5 running on a 2015 MacBook Pro (background has been lightened and blurred). The image of the proposed Mixer is a mock-up at the same scale.
| Attachment | Size |
|---|---|
| Current_Mixer_in_context.png | 244.05 KB |
| Proposed_Mixer_in_context.png | 289.67 KB |
Comments
I basically like the design you are proposing, there is one thing that is not addressed at all. There is no apparent way to view and adjust the different channels (like mute and open on a trumpet).
In reply to I basically like the design… by mike320
Actually there is. I talk about the list becoming a tree structure where instruments are grouped by family (woodwind, brass, strings, etc.). It is equally possible to expand instruments into sub-instruments for different channels.
If you look closely at the list you will see a little triangle next to each instrument name. If you clicked that triangle then it would expand to show the sub-instruments, just like the template categories in the New Score Wizard.
In reply to Actually there is. I talk… by shoogle
I see, that wasn't clear in your description. Like I said, I really like the design better than the bulky redundant mixer we currently have.
A couple more things I would like to have clarified. Where does the category list come from, the instruments.xml file? That would be fine, but there are issues to consider. Currently the instruments are listed in the same order as in the score. There are people who do not follow standard orders for instruments and may mix woodwinds and brass for example, how will this correlation be handled, especially when it comes to categories of instruments. Also in the case of percussion, it's standard to have all unpitched percussion listed after the timani (a pitched percussion) but before all other pitched percussion (like marimbas and tubular bells).
In reply to I see, that wasn't clear in… by mike320
You're right, it wasn't clear. I guess I got caught up in all the new benefits of the tree structure and forgot that its most important job would be to replicate the channel control that that current design already provides. :)
The instrument categories would indeed be defined in
instruments.xmlaccording to the standard orchestral ordering, but I have plans to overhaul the Edit Instruments dialog to enable users to pick other groupings too (e.g. wind band) or even to define custom groupings of their own. This would take some time to implement, but there is no need to enable groupings in the Mixer until it is finished.In reply to You're right, it wasn't… by shoogle
Since I have no insider knowledge, I can contribute little. As for the grouping of instruments ('or even to define custom groupings'), I would like to point out (for the purpose of brainstorming) that existing brackets already represent a grouping.
Perhaps this becomes clearer with the example of polyphonic music. See e.g. Wikipedia 'Venetian polychoral style'. A work here consists of several choruses, each consisting of voices and/or instruments, which are grouped in the score with brackets. An example can be found in the appendix. The mixer would serve here to determine the position of the choruses in the room (e.g., chorus 1 to the left, chorus 2 to the right, etc.).
If you can do this per group, it is certainly easier than per single voice.
In reply to Since I have no insider… by Nikolaus Hold
You are quite right. Brackets do indeed represent a grouping, and one of the objectives of my planned overhaul of the Edit Instruments dialog is to enable groups to be defined there, and enable brackets to be defined for those groups. However, that is a discussion for another thread. As far as the Mixer is concerned, all that matters is that these groups are defined somewhere.
In reply to You are quite right… by shoogle
I'm missing something here. Why is the order of staves in the score (including hidden) not the indisputably correct order for the instrument column in the mixer?
In reply to I'm missing something here. … by [DELETED] 1831606
The order would be the same either way. I don't really want to discuss the Edit Instruments dialog in this thread, but basically groups created in that dialog would be bracketed together in the score, but equally, brackets added in the score would also create groups in the Edit Instrument dialog. The two would be linked, and instruments would be displayed in the same order (and with the same groupings) in the Mixer. This is, as you say, the only correct order.
In reply to Since I have no insider… by Nikolaus Hold
@Nikolaus, let's say you were able to set the positions as you wanted (chorus 1 to the left, chorus 2 to the right). We could represent this as positions of the Pan sliders between -1 (100% Left) and +1 (100% Right):
Having made that adjustment, where would you expect the Pan sliders for voice parts within each chorus to end up? Should they stay at zero (the default) or should they all have moved to match the parent group?
To put it a different way, how would you then set Chorus 1 Sopranos to be slightly more to the left than Chorus 1 Altos?
Would you want to set relative values?
Or would you prefer to set absolute values?
What would you expect to happen if you later changed Chorus 1 Pan back to zero (centered), or -1 (100% Left)?
In reply to @Nikolaus, let's say you… by shoogle
Personally, I would be content to imagine myself standing in a large church with one choir to the left and the other to the right. (The reverb would make a more accurate location difficult for me.)
But I understand your questions and would probably ask them myself if I had to implement the thing. So let's hope someone with finer ears and more mixer experience can answer.
In reply to Personally, I would be… by Nikolaus Hold
What is preventing you from setting the pan controls any way you want? I've scored double-choir movements easily with MS. This discussion is about how, possibly, to allow control of them as groups.
In reply to What is preventing you from… by [DELETED] 1831606
Yes exactly
I endorse this proposal! Good job Peter.
In reply to I endorse this proposal!… by Thomas
Suspect would need to change docking behaviour so it could only dock at the foot of the screen as minimum width of this would be pretty wide.
In reply to Suspect would need to change… by Startled Bee
Docking to the bottom of the screen makes more sense in my opinion as other areas are already used by the Palettes and Inspector. However, permit me to draw your attention to this sentence in the proposal:
controls that are needed less often would appear on the right of the dialog, so this part of the dialog could be hidden until an Advanced button is pressed.
I applaud this effort as well, from a general usability perspective and for accessibility in particular!
In reply to I applaud this effort as… by Marc Sabatella
Thank you! What I think is most interesting is not the design itself but the fact that:
In other words, the element of subjectivity in good UI design is practically zero.
In reply to Thank you! What I think is… by shoogle
Shoogle, you are definitely on the right track. Everyone has known for years that the mixer needs fixed, and the latest "fix" made somethings easier, its size is atrocious. As you are well aware, I am a big fan of making MuseScore as accessible as possible and this will definitely improve that area.
Very nice and efficient design indeed :)
I second the effort!
One suggestion though...
Sometimes it is useful to have as many "console sliders" as possible in front of you, to get an overall picture of the instrument adjustments (plus a master volume slider).
So how about instead of disappearing the sliders all together, just moving them inside the left pane, horizontally aligned. Just put them where the instrument names are? No extra space needed. The names of the instruments could be shown in each slider.
Unfortunately I can't draw a picture of this but you get the idea, right?
In reply to Very nice and efficient… by vasmakk
I get what you mean. I did consider putting actual sliders in the list itself in a new column to the right of the instrument names, but it's not clear how keyboard users could operate them (the arrow keys would be needed to navigate between rows and columns in the list, so they couldn't also be used for the sliders).
If we can't put actual sliders in the list then there might be a way to give a visual indication of volume, perhaps by coloring the region behind each instrument name. For example, if the Flute is on zero volume then the background region behind its name would be entirely white, but as you start to increase the volume a green bar would start to appear on the left hand side, and once you reach 100% volume the entire background region behind the Flute's name would be fully green. You wouldn't be able to adjust volumes in the list itself, but you would be able to see all the relative volumes for comparison purposes, and then quickly click on an Instrument to adjust its volume using the separate volume slider elsewhere in the dialog.
I don't know how customisation Qt allows over the appearence of a listview/treeview, so it may turn out that partial coloring is not possible (or not worth the effort), but even without it the existing proposal would still allow you to compare instruments very quickly by clicking on the first instrument, then pressing the Down arrow repeatedly while keeping an eye on the position of the volume slider. You can then go back and click on instruments in any order to compare their slider positions, and if you want an exact numeric value then you can use the number spinner.
In reply to I get what you mean. I did… by shoogle
How about the use of the TAB key to move from column to column.
And inside each column, we could use the arrow keys, to move from slider to slider and adjust the volume of each one of them.
I'm not sure though, if the TAB key is reserved by some OS, for other purposes...
In reply to How about the use of key to… by vasmakk
> "How about the use of the TAB key to move from column to column."
Part of the benefit of having the list structure is that it counts as one big widget, so you can navigate it with the arrow keys instead of Tab. This means that Tab is still available to navigate to other widgets, and the total number of keypresses required to reach any widget is reduced.
In a "flat" design, where every control is a separate widget, you only have one degree of freedom:
This means you have to cycle through all Flute controls (mute, solo, volume, etc.) before you get to the controls for the next instrument.
A list adds a second "dimension", giving two degrees of freedom:
Now you can get from the Flute to the next instrument without tabbing through all of the Flute controls.
A table or grid adds a third degree of freedom:
Now you can access mute and solo without leaving the grid, and you can access volume sliders without tabbing through mute and solo, so the number of keypresses required to access any control is reduced.
If we used Tab to navigate within the grid then we lose a degree of freedom, and more keypresses are required to reach any given control. Equally, if we put all controls in the grid then you can no longer access some controls with Tab and Shift+Tab, so you lose a degree of freedom that way too. The optimum arrangment puts some controls in the grid and some outside the grid.
In reply to I get what you mean. I did… by shoogle
QTreeWidget has a method setItemWidget(). This can be used to add a widget to the the row so you don't just get text. So you could definitely create a custom widget that drew a bar indicating the volume level using this approach.
If the additional vertical space were tolerable, it's also possible to add a slider to each item in the tree view. I did a proof of concept on this by modifying the existing mixer code which allowed me to generate the following screenshot.
In reply to QTreeWidget has a method… by Startled Bee
Or perhaps more like this:
![Screenshot 2019-06-01 at 13.13.15.png]( "Screenshot 2019-06-01 at 13.13.15.png")
In reply to Or perhaps more like this: … by Startled Bee
Yes! Pretty much, that is what I had in mind. With the addition of the (fixed) "Master Volume control" on top of them.
All the other controls should be scrollable, in case the available space is insufficient!
Thank you :)
In reply to Or perhaps more like this: … by Startled Bee
Looks good! I would definitely use a separate column for Volume. However two questions remain for keyboard users:
By default, Qt assigns the Up and Down arrow keys to change rows in the grid (i.e. select an instrument) and Right and Left to expand or contract branches of the tree. Qt does not provide a default way to change column via the keyboard alone.
I recommend keeping Up and Down for changing row, and implementing Left and Right to change column. As for the two questions:
Alternatively (or in addition) you could use Shift+Right and Shift+Left to expand and contract branches and to adjust sliders.
There is a current limitation that I would like to see addressed in a new mixer. Currently, if you change something in the iteration of the instrument that's visible before you click the arrow button (I'll call this the "master" instrument), then only the first sub-instrument after it is changed when you click the arrow and expand the rest of the instruments.
In the case of instruments like clarinet and horn, it is common that the only change is the pitch of the instrument and the sound from the font does not change. If you change the volume or even the sound in the font, you must go through each instrument change rather than being able to change the "master" instrument and it applying to all of it's sub-instruments. You would still have the freedom to adjust the pizz or tremolo channels on strings and any other sub-instruments you need to adjust. I'm currently working on a score with 6 clarinet changes on one staff and it was a long process to change the soundfont for that clarinet.
In reply to There is a current… by mike320
@mike320, that is a very good point. It can be partially addressed through time-saving features:
Presumably some of your 6 clarinet changes were actually changes back to a previous clarinet (e.g. Clarinet in A --> Clarinet in C --> Clarinet in A again). You don't really need independent control for all 6 changes, just one setting for each unique instrument. This is addressed in #42301: Instrument Changes: add ability to change back to a previous instrument.
Back in the Mixer, it could be made possible to select multiple instruments at once (e.g. by holding the Ctrl or Shift key). If you selected both the Clarinet in A and the Clarinet in C then you could adjust volume and patch sounds for both instruments at the same time.
With these time-saving features in place, the issue of how sub-instruments should "inherit" settings from their parent becomes less important. In fact, we might decide it is best to disable settings for parent instruments and force users to expand the tree and adjust sub-instrument settings instead. This makes parents (and groups) a purely organisational feature, which is a bit of a shame but at least it avoids confusion.
In reply to This is a very good point… by shoogle
If you do want some form of inheritance, then there are a few ways I could see it working. Let's say you have a parent Violin that expands to show sub-instruments for Arco and Pizzicato.
Now you try to adjust the parent Violin's volume to 50%. What should happen to the sub-instruments?
Case 0: Nothing
It is not possible to adjust the volume for parent instruments (i.e. the Violin has no volume slider, or it is disabled).
Case 1: Overwrite
All sub-instruments are now at 50% volume. Relative volume adjustments for sub-instruments are lost.
Case 2: Scale on playback
Arco and Pizzicato are still displayed at 60% and 70% volume in the Mixer, but playback is at half of those values because the parent instrument is set to 50% volume.
Case 3: Scale in Mixer (upper limit)
Assuming the Violin was initally at 100% volume, Arco is now explicitly displayed at 30% volume in the Mixer, and Pizzicato at 35% volume. Playback is at the displayed volumes.
The sub-instrument sliders cannot now be moved above the 50% limit set by the parent instrument.
Case 4: Scale in Mixer (enforced mean)
The parent volume is always the mean average of all of its children. The Violin must have initially been at a volume of 65% (mean of 60 and 70). This has now been reduced to 50%, so the volumes of the Arco and Pizzicato sub-instruments are reduced in proportion to give 46% and 54% respectively.
Note that the 60:70 ratio cannot be maintained if the Violin volume is allowed to go above 93%.
Where is the all-important Master Gain in the proposal?
In reply to Where is the all-important… by [DELETED] 1831606
It says under the image:
"Master Gain control has been removed; overall volume can be set in the Play Panel."
But the image is only a "compact view". The proposal later says:
"users will miss the side-by-side volume sliders, but we can simply reintroduce the sliders as an optional feature via a button to expand the Mixer downwards."
Naturally this could include the Master Gain control.
In reply to > "Master Gain control has… by shoogle
The mixer settings are part of the score. Is Master Gain currently part of the score? Are the Play Panel settings?
In reply to The mixer settings are part… by [DELETED] 1831606
FWIW, you've got my vote for this significant improvement scheme.
@shoogle I recently opened an issue regarding some improvements to the layout of the mixer and another user has brought to my attention your post here; see #290458: Improvements to the mixer GUI.
If your design would be implemented, I think that some of the suggestions I have made are still very relevant, particularly using QT push buttons instead of those svg files, displaying the voice mute buttons only as either depressed or not using a default grey colour (instead of using voice colour), etc.
But I must say I am not too convinced about this proposed new mixer here. I do get that there are some good things about the design you propose, but there are several advantages of having all faders and knobs form all tracks displayed side by side; it makes it very intuitive to change panning and volume and mute/solo in a sensible way, comparing the overall settings. For instance, say you have 10 instruments, it's very easy to see if their pan is distributed in the way you wish just by glancing a single screen as opposed to having to memorise 10 pan values since you can't see them side by side. Same for muting or soloing, you can see at a quick glance which of your instruments are currently playing or not. My point is that there is a good reason for the mixer layout that virtually every DAW uses; and while I see that your solution is elegant and intuitive in a certain way, I believe that a lot of the advantages are lost.
In reply to @shoogle I recently opened… by gsagostinho
Getting an overview of volume or pan settings is not actually as difficult as you might think with the proposed design: simply select the first instrument in the list, then press and hold the Down arrow key while looking at the position of the volume or pan slider. (Notice how you would never think to use the keyboard with the current Mixer design - unless, of course, you are blind and have no choice - but with the proposed design the keyboard because the obvious way to interact with it.) Having gotten a rough idea of slider positions, you can then go back and click on individual instruments to look at their settings in more detail.
Of course this doesn't fully replace side-by-side controls, but if you read the section of the proposal titled "restoring the sliders", you will see that my plan was not to remove the DAW layout at all, but simply to hide it until the user clicks an "expand" button. Perhaps it's my fault for not including this button in the image of the proposed design, but that image was only really there to illustrate the improvement made in the very first step of the accessibility analysis. The analysis didn't stop there though, so I encourage you to read the rest of the proposal to find out about the other improvements in the design.
In reality, my design is actually very similar to the existing Mixer. The only real differences are the addition of an instrument list, and swapping the behaviour of the current Mixer's "collapse" button so that it hides the sliders rather than the controls at the top of the dialog. There are many other improvements that I would like to make, but they are relatively minor in comparison to the list and the expand/collapse button.
Other users have suggested putting volume sliders in the instrument list itself to provide side-by-side control in a more compact form. This is something that I had considered but rejected due to the difficulty in controlling grid sliders with the keyboard. However, further experimentation has led me to believe that this is not an insurmountable problem. Naturally, if it can be done with volume then it can also be done with pan.
I address the points about button styles in the other thread. I am more concerned with functionality than style, so I would probably just do whatever is easiest and leave the styling to somebody who cares about that sort of thing. I don't think the existing styles are too bad, but I agree yours are better.
In reply to Getting an overview of… by shoogle
@shoogle Apologies, I've missed the part about restoring the sliders.
> Naturally, if it can be done with volume then it can also be done with pan.
I think this would be a great idea, see the mock up I created:
Adding panning and solo/muting add just a bit to the size of the panel, but the reverb slider can also shrink a lot. I personally think this looks really good, would keep the current functionality and allow for a ocmplete overview of all tracks, at the same time as have a much improved design. What do you think?
As for the styling, I completely understand your point. But perhaps if it would not be too much trouble, would you consider to at least replace the solo/mute svg with QT buttons? They look so much more natural and should be fairly trivial to add.
There seems to be some enthusiasm for new design. @shoogle are you working on a PR for this? If not, I'd be interested in making a start on implementing the proposed design, i.e. making the quick hack-work I did to generate my screenshots into (a) tidy code and (b) actually functional. I suspect that implementing the whole re-design might involve a few staged steps to get right 😀. Of course, I don't want to duplicate effort if you're already on it.
In reply to There seems to be some… by Startled Bee
🎉
In reply to There seems to be some… by Startled Bee
@StartledBee. Sure, feel free!
In terms of making the code tidy, the best thing you can do is to use a QTreeView instead of a QTreeWidget. Views are bit more tricky to understand at first, but they have huge benefits down the line in terms of code maintenance and reusability. (In fact, it appears that you may not have a choice in this matter as setItemWidget is can't be used for editing, so you need to use QTreeView and QItemDelegate instead.)
The GSoC student I am currently mentoring for the Palette Accessibility project has created a small example Qt project that demonstrates how to create a simple tree structure, firstly as a QTreeWidget, and then as a QTreeView. If you've not used a QTreeView before then playing around with this example will really help you to understand the differences between the two.
> I suspect that implementing the whole re-design might involve a few staged steps to get right
I agree. i'd just try to create a single-column list of instruments in the first stage (like in my original image). I'd leave the columns of checkboxes and sliders until later, after you have got the hang of creating a model for a TreeView.
Ping me in MuseScore's developer channel on Telegram if you need help getting started.
In reply to Sure, go ahead! In terms of… by shoogle
This is starting to come together. I did look at QTreeView (and the GSoC student's code) - and it was very helpful to play around in a small project that compiles more or less instantly. And, for now, I've stuck with the QTreeWidget. The "for editing" comment in QT documentation is, as I understand, about swizzling out one widget for another, e.g. to edit text in the tree view. As far as I can see there is no need for that here. While there's a fair bit of work yet to do, the TreeWidget mixer shown in screenshot is pretty functional. Even with various bugs and as yet un-squashed quirks it's really nice to use. Will do a [WIP] PR in the next wee while, once current bugs / quirks are squashed and I've completed a bit more re-factoring (not least to improve code readability - the current mixer code is not so easy to follow :) ). Other ideas discussed in this thread (how MAIN instrument slider interacts with the child channels; possibility of having pan (by default, as an option) in the TreeView are all very do-able).
In reply to This is started to come… by Startled Bee
Looks fantastic! I can't wait to try it out!
The tooltip and custom mute and solo controls are nice touches, but I recommend concentrating on getting the layout right before you worry about the details. You might spend time customising the appearance of a button only to find out later that it would have been better to use a checkbox instead (e.g. to meet accessibility requirements). That would mean the time you spent customising the button would have been wasted, whereas if we test first and settle on the checkbox then you would have all the time in the world to spend making the checkbox look nice. (You could even style it to look like a button and get the best of both worlds!)
Anyway, a fantastic effort so far! Don't worry about messy code at this stage. I recommend sharing it sooner rather than later so that I can check it from an accessibility standpoint, and I might be able to help a bit with the refractoring too.
In reply to Looks fantastic! I can't… by shoogle
(Don't take this as an instruction to replace any buttons with checkboxes. That was just an example to illustrate why it is important to make sure you have the right tool for the job before you try to customise it.)
In reply to (Don't take this as an… by shoogle
Thanks for positive feedback. (And, don't worry, the mute / solo styling took all of a minute and a half - it's other stuff that's taking time.) Aiming to do a WIP PR at some point tomorrow.
I have spent a bit of time looking at the QT accessibility classes, but not finding the documentation easy to follow. Have you come across any example of QTTreeWidget (or any of the related classes) getting some accessibility souping up?
From what I can see, list and tress don't behave that well out of the box. If you move between rows you don't, as a matter of course, get told the row you've moved to. Though I might be getting something basic wrong. In any case, that's what I'm finding when I put VoiceOver on on my Mac. (The non-list, non-tree stuff is, on the other hand, not at all bad from what I can tell.)
I did find on GitHub a tiny example project that you'd put up, but apart from that, and a fairly helpful YouTube clip (https://www.youtube.com/watch?v=Dl9AB1IEQRc&t=1864s) (but with no code to accompany it), I've not been able to track down much in the way of examples of using the Accessible classes. It'd be good to get a tiny example project of a customised widget (like in the video), ideally for a list or tree. Of course, if there's no need to provide customisation and I've just missed something obvious then all the better.
In reply to Thanks for positive feedback… by Startled Bee
> "that's what I'm finding when I put VoiceOver on on my Mac"
Thank you very much for going to the trouble of testing with a screen reader! This makes me very happy indeed! (OK, it's not actually much trouble but still more than most people would bother with.)
> "From what I can see, list and tress don't behave that well out of the box."
This is unfortunately true on VoiceOver on macOS, but QTreeViews (and QTreeWidgets) behave perfectly well without modification on Windows (under Narrator or NVDA), and Linux (under the Orca screen reader program).
The Finder application in macOS uses a tree structure, so know that VoiceOver can work with tree structures. The fact that it doesn't work with QTreeViews is probably Qt's fault rather than Apple's fault. It seems that Qt is not providing the necessary information to tell VoiceOver what is going on in the tree. There is nothing that you can do to fix this in MuseScore, and even if there was, it would only benefit MuseScore. This needs to be fixed in Qt so that all applications using QTreeViews work well with VoiceOver.
In the meantime, if you want to test accessibility of the tree then you will need access to a Windows or Linux machine (perhaps via dual-boot or VM), but don't bother setting one up specifically for this purpose as I can test it easily enough myself once you have created a PR. You can still test keyboard control without a screen reader.
> "I have spent a bit of time looking at the QT accessibility classes, but not finding the documentation easy to follow."
If you stick to using Qt's widgets (QTreeView, etc.) and controls (QPushButton, etc.) then you don't need to read the accessibility documentation. All of Qt's widgets should be accessible by default. If any are not (such as the QTreeView in VoiceOver) then it is a Qt bug and should be reported to Qt.
Qt's accessibility documentation is there for when you want to create custom widgets, which you should avoid doing unless absolutely necessary precisely because it makes accessibility so difficult. The only time it is necessary in MuseScore is for the score view (though other widgets like the palettes are custom when they shouldn't be, hence the GSoC project to fix this).
As long as you stick to using Qt widgets, all you need to worry about is ensuring:
The accessibility analysis I provided in the initial post should help with (1), and I can help with (2) once I've seen the PR.
In reply to > "that's what I'm finding… by shoogle
It's good to know that the QTreeWidget behaves better on other systems. Thanks for your insights. I don't have ready access to a non-Mac platform at the moment. Definitely want to avoid doing custom accessibility work if at all possible.
In reply to This is started to come… by Startled Bee
This looks stunning! Really really nice :-) I had previously suggested adding a slider for panning in between the volume slider and the solo/mute buttons but perhaps a better approach would be to add a small dial of the same size of the solo/mute buttons. See: https://doc.qt.io/qt-5/qml-qtquick-controls2-dial.html
This would allow for a quick overview of the overall panning.
And in your screenshot, the Mute Voice buttons are doubled, is that a small bug or is there a reason for it?
In reply to This looks stunning! Really… by gsagostinho
It would be QDial rather than QML Dial.
I have other comments for @Startled Bee but I will save them for when we see the PR, because getting the code right is more important than the design at this stage.
In reply to This looks stunning! Really… by gsagostinho
The Mute Voice buttons have two rows because there are two staves for that instrument. This is what the current mixer does too. In my 'work in progress' I've amended the tooltip - see below.
Thanks for positive feedback. I'm looking at options for panning in the main mixer bit. I've not tried out a dial yet. (I'm not sure how it will work if it's really small.) Also going to look at having the OPTION of either showing an extra panning slider in the MixerTree or swapping the Volume slider's role to Pan (or Chorus or Reverb) with a modified key or a button. With all these things, it's hard to know what it'll actually feel like to use without trying it out.
Aiming to get a WIP PR up at some point tomorrow. Various bugs aside it's nice to use. Quite a lot more to do to fulfil the Accessibility goals though. It's very keyboard friendly, but my tests so far with VoiceOver on the Mac suggest that the TreeWidget is going to need some custom code to "talk" effectively for visual impaired users.
In reply to The Mute Voice buttons have… by Startled Bee
I recommend not using a dial because (as you say) it will either be too small to adjust properly, or else it will simply cause the list items to become taller hence you will see fewer of them. People might be used to seeing dials on real mixer desks, but that doesn't necessarily mean it makes sense to use them as part of a virtual interface because the usability requirements are completely different.
It should be possibly to allow users to show and hide additional columns by right-clicking on the header row at the top of the tree view, but I wouldn't bother with for the initial implementation.
I said I wasn't going to comment until the PR, but I see you have made changes already...
I would recommend putting Mute and Solo immediately after the instrument names. These are things that users will want regular access to, whereas balance (volume and pan) are things that the score creator will set once and then hopefully never change again. Also, it makes sense to put smaller controls on the left so that the eye doesn't have to travel as far to get to the other controls.
I wouldn't bother with patch colour. It has no physical meaning, so the fact it was needed for the current Mixer is a big clue that the Mixer's design is poor. However, if you want to include it then it should probably be next to the instrument name, or better yet it should affect the colour of the instrument names themselves (or the background behind them), and maybe also the colour of the sliders, etc.
I wonder if there is a way to combine Mute and Solo into a single tristate control (i.e. Mute, Solo, Normal) that users would click on three times (or press spacebar three times) to cycle through. However, this is something that would definitely need to be tested with a screen reader before we can commit to using it.
In reply to I recommend not using a dial… by shoogle
PR is up now. I'm not sure I'm keen on the patch colours either. At the moment there's a right click option so you can see the UI with or without them. Re-ordering the controls is very easily done. My thinking with mute and solo being where they are is to do with the idea of being at the edge of thing making things a better target. So I was thinking that if they were next to the instrument name they'd be stuck in the middle. For a mouse user, it seems to work well.
some of the geometry and resizing is still off - but it works well for some case and is an improvement.
the issue with what the main (top level) slider does to the secondary (child level) sliders do is really interesting. I've not got as far as setting them up - it requires a tiny bit more refactoring of the code so that different options for the relationship can be plumbed in.
I'm keen to find a nice way of swapping the role of the slider to do panning (or even chorus or reverb) - I've a couple of idea, but not had a chance to implement them yet - there's a teasing (not impl yet) option shown in the contextual menu...
I regularly take a midi file from MuseScore into Logic Pro for mixing and mastering which works well except that some instruments are not recognised correctly (trombones appearing as saxes, etc.). It would be useful if I could batch-select all tracks in the mixer to turn off volume and pan. The faders and pan dials in Logic have a mind of their own if this is not done. Also, it is counter-intuitive to have to turn the pan dials all over to the left in order to set them to zero in Logic. I expected the default setting (in the middle) to yield this. Also, in Logic I can option-click (Mac) a control or batch-select controls to set to zero.
In reply to I regularly take a midi file… by John Morton
Batch-select is a good idea and something that has been requested before.
If the middle value of Pan is not exported to MIDI as zero (i.e. centred) then it is a bug and should be reported on the tracker.
In reply to Batch-select is a good idea… by shoogle
del
In reply to Batch-select is a good idea… by shoogle
Probably related - double-click sets the value to 0, which is hard-left, rather than 064, which is centered. I vaguely recall some discussion of the range of this back in the previous redesign. It's fine with me to call the range 0 to 127 (although I personally would expect 0 to be centered, negative to be left, etc) if that maps to MIDI better, but then, we need to actually export to MIDI correctly if we aren't currently (I'm not 100% sure the problem isn't at the Logic end)
In reply to Probably related - double… by Marc Sabatella
I believe that most DAWs handle this by mapping the 0-127 MIDI range into a 90L-90R range, which is used only in the interface with the user. So 0 is center (MIDI 64), 90L is full left (MIDI 0) and 90R is full right (MIDI 127). That's the way REAPER handles it, and if my memory serves me well both Logic and Pro Tools also handle things in the same way.
In reply to I believe that most DAWs… by gsagostinho
Just a correction, I checked now and both REAPER and Kontakt use a range from 100 L to 100 R, not 90 as I previously stated. Everything else stands. I also noticed that both programs map the volume slider from the traditional MIDI range 0-127 into a dB range of -inf to +12, in a scale where MIDI 064 is being mapped into -6 dB.
In reply to Just a correction, I checked… by gsagostinho
90 make sense when interpreted as °, 100 makes sense when interpreted as %
In reply to 90 make sense when… by Jojo-Schmitz
Sure, either would make sense but I meant to say that the programs I mentioned use 100 (as in %) instead of what I had initially thought (which was °).
In reply to Sure, either would make… by gsagostinho
The current details view doesn't handle panning very well. My WIP keeps the slider (because the dial control just doesn't sit right), but it's tweaked so:
* double click centres it (that's what happens at the end of the GIF sequence)
* the blue bar indicates the degree of left or right.
I've stuck with 0 to 127 as a range, but in the spin box it's shown as -63 to 63. It's converted back before it's sent off to the MuseScore synthesise or MIDI.
In reply to The current details view… by Startled Bee
Erk, the volumes lack the numbers, which makes it impossible to e.g. dial two instruments to the precisely same but lowered volume.
In reply to Erk, the volumes lack the… by mirabilos
Are you the Erkkönig!? Yes, I, too, have always felt that need (see numerical values of all settings).
In reply to The current details view… by Startled Bee
if you move Volume and Pan to the right side of the instruments. We will have a slightly improved (and foldable) 2.x mixer. :)
In reply to I believe that most DAWs… by gsagostinho
Logic doesn't. Tried yesterday.
In reply to I regularly take a midi file… by John Morton
Does your use case required selecting only some of the instruments to force volume to be zero and pan to be centred or is it sufficient to have the option to do it for all instruments?
The latter would be a quick tweak to the WIP - an option on the Details button menu.
The former means allowing for multiple items to be selected. That's do-able, but switching on selection for multiple items will then raise also sorts of issues about how the dialogue should behave in that state. It might also be that some (many) users would be confused / puzzled about what to do with multiple selections.
The current pan control is not so great. See another reply to this thread for what the WIP is currently doing.
In reply to Does your use case required… by Startled Bee
An option to do it for all is nice but not, to me, as necessary as restoring the ability to do it for a single instrument, which had been the case until the recent redesign.
Nice!
But please add an expand/collapse all button for the LHS tree, and do something about the Ⓜ and Ⓢ thingies nobody knows what they are for (at least not intuitively).
Also, considering instrument name lengths, please make the divider between the LHS tree and RHS dialogue dynamic, so people with longer instrument names can expand it a little.
I’m aiming for usability on a device (netbook) with a 1024x600 px screen.
In reply to Nice! But please add an… by mirabilos
Current divider behaviour in the WIP is shown in the GIF.
Expand / collapse button would be easy to add - difficulty is where to put it without cluttering things up. Not sure about doing M and S differently. Some folks will be used to them from other contexts. But if you've not come across them before, they are far from intuitive.
Oops - looking at the graphic I see the selection changes back to top of tree when you edit the instrument name. Will need to fix that.
In reply to Current divider behaviour in… by Startled Bee
My 'Logic doesn't' reply was to the claim that Logic handles top dead centre as '0'. This site misplaced my reply. As a general reply to all, I need to save time in Logic having to set everything to zero but, without a batch select facility, and some means of setting all to zero with some key command (doesn't matter which, as long as I know), it takes too long In MuseScore anyway. I'm not actually interested with MuseScore's mixing ability because of the existence of DAW's nowadays..
See comments at #292641: Display problem with Mixer patch list.