Revising categories of the issues in the issue tracker
We are pleased to announce upcoming changes going to happen with the Issue Tracker.
Current categories describing the issues lead to misunderstanding in terms of priority and importance. Lack of predefined categories led to creating separate hit lists for particular versions summing up the most important issues to fix.
Taking the best practices from existing bug trackers, I prepared an update for the categories we have in the issue tracker. The most important goal here is to make categories absolutely clear to a reporter. We will drop all not popular categories and introduce specific and transparent ones instead.
One more important thing is that almost all new fields will use default values when creating new issue which means less time spending on filling all categories.
All these changes are about improving the experience of using the tracker for mature contributors as well as attracting new ones. New categories are believed to simplify the workflow from creating an issue to merging the PR with the solution.
We are full speed ahead with the new MuseScore release. The main goal we all have right now is to make the most stable release ever with new groundbreaking features. The most important challenge in this way is to fix all critical and important issues which blocking further testing scenarios and having great quality in the end.
By the way, some of the issues marked as critical are not really critical to fix in this release in fact. It means we need to revise Priority field to mark absolutely critical features which must be fixed in the release. We used to use Critical priority for crashes even if the scenario is hard to reproduce or could be reproduced on one machine several years ago. It means we need a separate field to specify severity (level of impact) in such cases. So, such issues could have “Critical” severity, but “Low” priority. Prioritizing with separate field could solve the problem of separately existing hit lists created several times, to sum up,really important issues which should be fixed in the release.
Along with the severity and priority we still have clear categories which help to prioritize the issues. More categories could specify whether a workaround exists for the issue or not, whether this is a regression or not. The important category is a level of visibility (Frequency) of the issue defining how many users are potentially affected by the issue. The issues which could not always be reproduced often have less priority as well.
The last but not the least category is a type of the issue. Most issues have “Code” category right now, which may mean everything but doesn’t help to prioritize the issue. Introducing clear categories will allow filter issues and assign corresponding priorities according to the type.
The list of new categories is described below. Each category is provided with a description and each value which can be specified in the category also is described.
Severity is the level of impact the bug has on the product.
There is something subjective about severity: when you choosing a level, always think about the product first. How much will the bug affect the core purpose of the product?
S1 - Blocker - Any kind of crash and hang, data loss, security issues, 3.0 score cannot be opened, something cannot be saved
S2 - Critical - Features work incorrectly, UI elements are lost
S3 - Major - Features work, but give unexpected results, UI elements are incorrect, operations which make score unusable
S4 - Minor - All other cases
This category defines whether the issue could be reproduced in a particular scenario.
Always - The steps to reproduce are probably easy to identify and to write. A scenario is always expected.
Randomly - They appear only sometimes meaning that the conditions to reproduce the bugs are not yet identified
Once - The bug happened once and even reporter cannot reproduce it again
A frequency of reporting (Visibility)
This category specifies whether the issue has been reported once or this is a frequent topic.
Once - 1 report in the issue tracker or forum
Few - 2-5 reports in the tracker or forum
Many - More than 5 reports
The category specifies whether the issue is a regression or not.
Yes - The bug cannot be reproduced in the previous version of the software. The bug breaks the logic which works in the previous version.
No - The bug can be reproduced in the previous version of the software as well.
This category defines whether there is an easy workaround for the issue.
Yes - There is easy (up to 3 steps) workaround for the reported bug. It means the user can perform the desired action using a different scenario.
No - There is no workaround to achieve the result.
The type of a bug is the nature of a bug. This categorization is objective: a bug will always have the same nature for whatever product. Nevertheless, some bugs are tricky and you may hesitate between 2 categories. Type helps us to categorize and prioritize issues.
Functional - A functional bug is a dynamic bug related to an action you're doing. You can only find it while doing an action on the product. The product’s reaction is not as expected.
Graphical bug (UI) - A graphical bug is a static bug related to UI (User Interface) issues.
Wording bug - A wording bug is related to the text content including translation issues
Ergonomics (UX) - The issues related to various user scenarios and proper placement of the UI elements
Performance - The issues related to the performance of the software
Layout bug - Layout bug relates to everything related to displaying score, elements.
musescore.org - Issues related to the MuseScore editor website
Plugins - The issues related to QML plugins framework and related code in MuseScore
Development - The issues related to the code, infrastructure, deploy
This category defines the importance of fixing the issue. The issues with high priority should be fixed first. The priority can be changed by core team members only.
P0 - Critical - The issues must be fixed asap and block currently scheduled release.
P1 - High - The issues should be fixed in currently scheduled release. They do not block currently scheduled release, but may block the next one.
P2 - Medium - Nice to have this issue fixed/implemented in the next release. The issue won’t stop the release.
P3 - Low - The issue can be fixed/implemented, but if someone wants to do it, it is better to spend time on P1, P2 issues instead