Musescore 3: full backward-compatibility, please

• Sep 21, 2016 - 22:24

I've been meaning to post about this for some time and just never got around to it. Anyway, my biggest peeve about Musescore 2, improved as it is, is the REALLY annoying problem of initial backward-incompatibility with files created in Musescore 1, which forces you to save your old files all over again in Musescore 2 before you can make any changes with them. Essentially, it turns them into read-only files until you re-save them.

I know it seems like a simple operation, but it irritates me sufficiently that it acts as a significant deterrent when I contemplate using older Musescore files posted in the Sheet Music section. There's a lot of great public domain/CCzero stuff here. Unfortunately, that that initial-save issue, along with the frequent "corruption" that ensues with the older files (when Musescore says that it found X number of beats when it expected Y), holds me back from taking advantage of a good deal of the available material that I might otherwise download and use.

So if at all possible, please avoid this issue in Musescore 3. I have boatloads of Musescore 2 files, and if I have to reopen and resave every single one of them at the transition, you will probably lose this user.

Another item on my wishlist, though secondary, is that you get rid of the hidden files automatically generated when saving. I exercise complete control over my system, so of course I have it set to show hidden files, and these things are a significant annoyance; I spend a lot of time deleting them so they don't cause confusion. If it's not too much trouble, please get rid of them. Or at least let us turn them off.


Comments

This was deliberate for MuseScorew 2, because there was concern that people might otherwise accidentally overwrite their originals and be unable to open them in MuseScore 1 any more. While there is the very minor nuisance of needing to confirm the save the first time you open a file in MuseScore 2, surely it pales compared to the wailing and gnashing of teeth we'd be hearing if we overwrote people's originals and made it impossible to open their originals in MuseScore 1 any more. And I have to imagine the same concern will hold with MuseScore 3.

I don't understand what you mean about having to save before making changes. Nothing forces you to do that. Make your changes firs,t then hit save, then if you like simply hit Enter to confirm. It's exactly *one* extra keystroke to gain that piece of mind that your original scores that you worked hard on in a previous version won't be accidentally overwritten. That said, for people who don't mind us handing you a loaded gun pointed directly at your foot, I suppose a "don't ask again" option would be a possibility.

EDIT:

Hmm, maybe I misunderstood. I thought you were asking us to remove the safewgaurd that prevents you from accidentally overwriting your original file without at least hitting one extra key to confirm. Mayne you are really saying you wish it were possible to add significant new features to a program and somehow have older versions of the program still be able to actually open the files saved from it? That's just not feasible. not just MuseScore, it's true for most software. Adding significant new features means updating the file format of the files saved to record the new information, and older versions of a program will almost never be able to handle that. Sometimes you get lucky and the changes are mild enough that an older version can still open the file, just maybe missing some things you added using a feature that doesn't exist in the old version, but normally that happens only if the changes are relatively minor. We usually keep these minor changes to minor releases - 2.0.1, 2.0.2, 2.0.3, who knows, maybe even 2.1 - but whenever MuseScore 3 comes out, I would *hope* it would have major improvements to justify the version numebr change. And major improvements almost always mean older versions cannot open the files created by the new one. That's just a fact of life with any changing technology.

In reply to by Marc Sabatella

> I don't understand what you mean about
> having to save before making changes

If I just used the names that original Sheet Music posters gave their files, I suppose it wouldn't be that much trouble. But that's impossible to do, really; most of the time their filenames are fairly useless (e.g. "Tchaikovsky Rondo"). So whenever I download a file, I always rename it to my own style so it fits alphabetically with all my other files, as follows:

[Composer surname]_[catalog designation if available][title of piece][movement or part]-[copyright license]

BUT what happens after I open the (renamed) score and make a change, then attempt to save?

Nothing, if it's a Musescore 2 file. Everything works just fine. But if it's a Musescore 1 file, the d---d program's dialog box insists on saving the file with the old, useless name. So I have to navigate to the folder where the bleeping file resides, manually copy the new filename I gave it at download time, then paste it into the dialog box. It's a genuine PITA, and I hate it.

You're probably going to recommend that I just download the file without renaming it in the process, then immediately open/save, and only then rename it. For other reasons too complicated to go into here, this would make an alteration in my workflow which would slow productivity. What I need is a program that simply doesn't cause the problem in the first place.

I don't use Musescore 1 at all, and if Musescore 3 is all it's cracked up to be, I won't use Musescore 2 either after the upgrade. So yes, please give me the gun, or whatever, to avoid this problem in the future.

In reply to by Ironword

Instead of first renaming then saving, why not simply rename *while* you save?

Oops, I guess you already saw that coming. I don't understand the problem with this approach. Maybe if you explained your usage we could understand better. But in normal usage - sounds like you for some unknown reason are downloading and modifying large numbers of files you didn't create yourself?

In reply to by Marc Sabatella

> And major improvements almost always mean older versions
> cannot open the files created by the new one.
> That's just a fact of life with any changing technology.

Not at all true;-or at least not true in terms of the issue of the dialog furnishing the old filename instead of whatever new one you may have manually given it in the meantime. I have used Corel WordPerfect from the beginning, and the latest version can still open and then save files created in the very first Windows version, WP6.1, back in 1996 or so. It does require one to choose whether to save in WP6 format or the format of the current version, but--significantly for me--it does NOT require you to use the original name of the WP6 file if you've manually given it another name (via Windows folder).

In reply to by Ironword

Right, but the *reverse* is not true. That is, old versions of Word Perfect cannot open files created by newer versions. That's why it is important to prevent people from accidentally overwriting their original files with newer versions the original version cannot open.

If you mean the backup files (.score.mscz,), they have saved many butts over the years if a score accidentally gets deleted or corrupted or whatever. Again, it's extremely unlikelky we would chosoe to deprive people of this very important safeguard. but for people who don't mind being handed that loaded gun, indeed, feel free to formally requewst (via the Issue Tracker) an option to do "unsafe saves" or whatever one miught call the option to make it clear how dangerous it would be to the average user.

Maybe a non realistic idea: In several programs (like "Word") you've the choice to save your file in older versions with the risk (and the warning), that some of the new features can't be use with older versions of the software. Is something like that conceivable?

In reply to by kuwitt

As I noted above, what's crucial for me is this: if I've given a file a different name than it had originally, the new version of Musescore should not insist on offering the old name in the dialog box when saving the old file for the first time.

So I guess my heading for the post is wrong. It really shouldn't be "full backward-compatibility", but something more like "Windows-naming compatability between Musescore versions."

In reply to by Ironword

That's definitely a much different thing. It's not even a compatibility issue at all - it's just a question of what filename is offered as a default when saving an imported file for the first time. In principle, wanting it to use the current filename rather than the original might make sense, although my guess is someone else will just as strongly want it the current way. Still seems the obvious answer for now is to not rename the file until you actually save the file. Without knowing more about your highly-unusual-sounding usage, that seems a no brainer to me.

In reply to by Marc Sabatella

Yes, I am downloading all of the public-domain/CCzero transcriptions I can find because I'm in the incipient stages of a theory project (harmonic analysis) for which I will require a large number of examples. (Actually I could probably grab just about anything, as the snippets I'll be extracting for analysis will clearly fall under all four points of the Fair Use doctrine, but better to avoid hassle).

It would take way too long--at least a pageworth of text--to explain the workflow procedure I've developed to do this most efficiently; it's complicated, involving multiple windows in various folders (organization is key) and simultaneously collating IMSLP scores with Musescores. Unfortunately, one crucial component has been the creation of the new, carefully constructed filename at the time of download (in the download dialog). It's supposed to be fire-and-forget. But then the filename-save problem comes back and bites me in the butt about half the time because (depending on the situation) I need to open the files, create a quick vertical frame at the beginning of the score, and add a few brief notes about the composer, and often the transcriptionist's name & link to his/her Musescore page. (If they give their real name somewhere, anyhow. I'm not going to bother crediting "pikachu99" and suchlike. If people are going to use unprofessional names on their posted scores, they'll get unprofessional treatment--yet another reason to stick to public domain releases; it allows me to get away with that.)

But I only have to do that some of the time, not all the time. Aside from other issues that would take too long to explain, the fact that opening the files and adding these brief notes is only done with some and not all files means that, to keep a good work rhythm going, I need to rename ALL the files at the same point--the point of download. It's like a factory assembly line: you need to have the whole procedure as standardized as possible in order to reach maximum efficiency. All files must be renamed, therefore that procedure should be standard, no exceptions. The note-adding procedure, because it occurs with only some files, should take place downstream from that. This is elementary logistics, but Musescore's insistence on using the old name when saving the old files mucks it up.

> my guess is someone else will just as
> strongly want it the current way

Doubtful. If you deliberately give a file a new name, why would you want the app to insist on saving with the old name which you yourself already overwrote, on purpose?

Another way to think about the issue is to consider how many other apps behave this way. Personally, I have none. Musescore is the first program I've ever used, at least as far as I can remember, that does this.

In reply to by Ironword

A simple solution to your special unusual use case:

Don't rename the files. Instead, use MuseScore from the command line (perhaps via a batch file) to rename and convert in one step: "musescore2 old.mscz -o new.mscz".

As for what other programs I've seen do this, it's pretty common in cases like this where people might want to hold on to old versions for compatibility to make sure their old.files are still laid out the same. Not so common in word processors where such changes are less likely. But Finale for instance does something similar.

In reply to by Marc Sabatella

+1 on using the -o command line option. What I did when V2 came out was to copy all my MuseScore files to new V2 folders, then ran a batch file that processed each .mscz file and saved all new files in V2 format. Then fixed all the corruptions I found doing that process. I ended up with old directories with files in 1.3 and new directories in V2 format.

Ok so long story short. Let's be constructive... Are these the steps to reproduce your problem?

  • Create a score in MuseScore 1.3 with a title "Test". Save it as "TestFilename.mscz" somewhere
  • Rename the file to TestFilename2.mscz
  • Open it in MuseScore 2
  • Change something and save the file again. MuseScore propose to save it as Test.mscz using the title of the score and in the default "Scores" directory.

Is that the behavior you don't like?

If yes, there is a rationale for it. We consider that we import the 1.3 file, we don't open it. So when the user wants to save it, it's a completely new file just like if it was created via File > New. We want to avoid that the user override the 1.3 file accidentally. Now, can you find a way that we can cater to your need and to the need of most users who don't want to override 1.3 files?

Second point. The corruptions. It's very likely that MuseScore 2 doesn't corrupt MuseScore 1.3 files but that the files are corrupted in the first place and MuseScore 2 detects them. I could be wrong and If you do report such problem with a given file, someone could take a look and maybe fix the problem. If you don't report it, the problem will be there in MuseScore 3 for sure...

In reply to by Nicolas

OK, I see--you've programmed it this way as an invisible failsafe. In WordPerfect, for example, when this issue comes up, the app actually asks you whether you want to save in the old format or the new format. You have opted not to ask the user up front whether to do this; you have simply made it harder to save, presenting the user with the old filename in a save-as dialog as a reminder that s/he's dealing with an old file.

> Is that the behavior you don't like?
Yup.

> Now, can you find a way that we can cater to your need
> and to the need of most users who don't want to override 1.3 files?

I see two possibilities:

==================

POSSIBILITY 1:

Why not adopt the WordPerfect way? It has the big advantage of being more transparent. First, do not retrieve the old filename as a subtle reminder that it's an old file. Use the new filename instead. But when the user attempts to save, ask the user right up front which format they wish to use for the save. Give them a message saying "This file is in an older format: Musescore [VERSION]. Please select the format in which you wish to save: " --and then offer a radio-button list with all Musescore versions, with the old format (the one the file is currently) as the default so that it's impossible to rush through the save process and accidentally save in a newer format.

And please make the list of versions a radio-button list. I know that they take up more screen space than drop-down listboxes, but when you're saving, you're not normally doing anything else anyway so it doesn't matter if a lot of the screen is taken up by the dialog. The problem with drop-down lists is that they require extra clicks or button-presses, first to open and then to arrow down. Radio-button lists require only a single button click--much more efficient.

==================

POSSIBILITY 2:

If you only want each Musescore version to save to its own format for ease of programming, no problem. Just detect which version the file is, and then IF it's not the current version, pop up a warning box which the user must go through prior to saving. For Musescore 3 the message on the warning box would read something like this...

"This file is in an older format: Musescore [1 OR 2]. The version of Musescore you are currently using is [CURRENT VERSION NUMBER], and it saves only to that format. It cannot save to Musescore [1 OR 2]. Instead, it will overwrite the Musescore [1 OR 2] format, making this file impossible to open in Musescore [1 OR 2]. If you wish to retain your file in the old format so it can be used in Musescore [1 OR 2], please press Cancel. Otherwise, press OK to continue."

...and if the user presses Cancel, dismiss not only the Warning dialog but also the entire Save dialog. If user presses OK, dismiss only the Warning dialog and go back to the Save dialog with no further interruption.

As far as default behavior, I guess the Warning dialog should give default focus to the Cancel button so an accidental or unthinking Enter doesn't inadvertently save over the old format. (Personally, I'd rather have the default be on OK so I could just click through fast, but that's because the Warning box wouldn't help people like me who don't ever plan on using anything but the current version of Musescore.)

Obviously, the IF statement's conditional could be easily expanded with each new version as more versions become deprecated.

==================

> The corruptions
I've been downloading lots of files but have noticed the problem only with 1.3 versions, so I assumed that this must already be a known issue. But of course you could be right; it only happens with one out of every ten files or so, and that's a low enough rate that it could just be corruption that's already there. But next time I come across it, if I have time I'll post it.

In reply to by Ironword

Having 3.0 preserve the ability to save in 2.0 (or 1.3) format means basically keeping thousandcs upon thousands line of code around, and maintaining it to still work with all other under the hood changes. There is pretty much no way a tiny open source project like MuseScore is going to be able to manage it. A big corporation like Corel can afford to pay people to do that, but it's just not likely with something like MuseScore.

The other option you mention might make sense for your special and extremely unusual use case, but it doesn't make nearly as much sense for the average users, who are generally editing *their own files*, not other people's. Wanting to save the file to a new name band not overwrite the original is the norm; you are very unusual in wanting to always overwrite the original because the original wasn't yours in the first place.

1.3 definitelty created a *lot* of corrupt files. 2.0 detects them. This is indeed known, and a good thing - 2.0 gives you the opportunity to identify and therefore fix the problem, which 1.3 never did. So this is not a problem with 2.0. Of course, there are also a small number of bugs in which 2.0.x releases can also create corrupt files. Some of these are known issues, most already fixed. If you find a case where 2.0.3 specifically is creating - not detecting the corruption caused by an older version, but actually corrupting a file itself - then please do report it, with precise steps to reproduce the problem so we can investigate further.

Each version of MuseScore is only backwards compatible with one version previous because maintaining backwards compatibility requires significant time and effort, makes the program grow very large, and restricts development of new features. If you need further backwards compatibility then you could always download an older version of MuseScore.

Perhaps MuseScore.com will allow uploads in older formats and do the conversion automatically?

In reply to by shoogle

Currently (since a few days) work is underway to make MuseScore 3.0 backwards compatible with 2.0 and 1.x. But still These would be traeed as imports, so the ussie at Hand here would still exist. Only it spares from having to import 1.x files into 2.0 first, save and then import into 3.0.

In reply to by Jojo-Schmitz

Indeed, and believe me, making sure that MuseScore 3 can open 1.3 and 2.0 well enough is (and will be) a hard job. I hope there will be more people than Werner and I looking at this and hunting bugs down. If not, we will need to kill the 1.3 import or live with a bad one.

In reply to by shoogle

> Perhaps MuseScore.com will allow uploads in older formats and
> do the conversion automatically?

Yes, this would become very important for people like me who do not want to clutter their systems with old programs. I maintain a clean, lean system and I only want the current version of Musescore on it, no previous versions. But if Musescore will only ever import one version backwards, as time goes on people like me will need some way to access files made two, three, or more iterations previously.

This is also an issue for each user separately, on their own machine, with all their personal Musescore files. I think the solution lies in a standalone, separate file-transformation utility that has only a single function: batch conversions between Musescore formats. Batching will be critical because if a user has hundreds of files in Musescore 2 (as I will when Musescore 3 is ready), it will be utterly impractical to open and save each one to convert it to Musescore 3. We will need a batch utility to do it all at once.

Indeed, now that I think more about it, I will go on record and say that this is an absolute necessity.

EDIT: just saw the command-line suggestion several posts above, which mentioned a batch. If you can already do a mass conversion with the command line, the utility would merely need to be a front end for that process. But giving it a front end is important. Even someone like me (I've done some programming in the [far] past) can't just whip up a batch command line in a minute or two. I'd have to dig out my old DOS manuals and refresh my memory on how it all works. After all necessary reading, it'd be at least an hour, maybe a bit more, before I was finished with the process. And by now there are a lot of users out there who have grown up never facing that blinking DOS prompt (Oh, nostalgia). If they're not geeks, they don't even know what a command line is.

So a GUI utility will still be necessary.

With reference to batch export plugin, I have used this extensively on thousands of files recently and can report that it does NOT work under Linux (Musescore 2.3 hangs after a few files requiring a restart of Musescore program).
I have managed more success under Windows 7 but still occasional crashes.
I finally used a Users' script program written in python which worked faultlessly.
HTH.

In reply to by crm114

I just ran the batch export plugin on more than 450 files producing more than a thousand converted files on my Kubuntu 14.04 and it works just fine.

Can you provide more detailed information when you see it hanging/crashing, like:

  • Which from/to formats did you select?
  • Did you let it work on subfolders?
  • Is there a certain file causing problems? If so, could you provide it?
  • Which version of the plugin are you using?
  • Which version of Linux and MuseScore (there is no 2.3 - did you mean 2.0.3?)

Thank you heuchi for your interest in this problem.
Specifics
Ubuntu 16.04 using Xfce
Called Musescore using Musescore-2.0.3x86_64AppImage
Batch export using Jojo-Schmitz-batch-export-a71e138
Hang occurs on progress screen with Abort button non functional.
Only option is to kill progress screen with X in corner and OS reports Musescore not responding do you wish to exit this application.
Since it works fine with Python script, that would point to error handling in GUI application or batch-export script itself.
HTH

In reply to by crm114

I found a possible reason while converting to wav files:
On one of my test files, the (old) Navigator bug appears (see for example here), which causes a cpu load of 100% for at least one core. This leads to a horribly slow wav export. However, exporting to wav shows a progress bar window with a separate abort button which still solves the situation.

It would probably help me to know, to which formats you are exporting, to see if I get the same problem here using this specific file.

Do you have the Navigator window enabled by default? If so, can you try exporting files with the Navigator window disabled (you might need to open some score first to change this using F12) and see if the problem remains.

Do you still have an unanswered question? Please log in first to post your question.