Scores corrupted beyond repair, containing nothing but zeros

• Apr 9, 2018 - 08:34
Priority
P1 - High
Type
Functional
Severity
S1 - Blocker
Reproducibility
Randomly
Status
needs info
Regression
No
Workaround
No
Project

We're pretty frequently are getting reports about score that MuseScore can't open anymore, frequently these files contain nothing but zeros, sometime also a zip header, but nothing more.

What is needed in order to get this fixed is steps to reproduce.


Comments

I don't have steps to reproduce but a couple of ideas to investigate:

  • Is it possible that autosave kicks in while saving a file via Save as ? What would happen ?
  • Review the save process and make sure we catch all possible errors and display a dialog.

Could it be a problem in thirdparty/qzip/? We seem to be using MQZipWriter to compress files while saving, rather than Qt's QZipWriter?
Related to https://bugreports.qt.io/browse/QTBUG-55016 and https://bugreports.qt.io/browse/QTBUG-3897 ?

From third/gzip/gzipwriter_p.h:

//  W A R N I N G
//  -------------
//
// This file is not part of the Qt API.  It exists for the convenience
// of the QZipWriter class.  This header file may change from
// version to version without notice, or even be removed.
//
// We mean it.

Didn't re just recently have an issue with having used private Qt stuff, that meanwhile had been changed?

Sometime a zip file header followed by all zeroes, other times just zeros. Points at (error during) compression to me, not permission or access. But that is just a gut feeling, nothing more.

In reply to by Jojo-Schmitz

Who knows, I think that unless some kind of detailed tracing is written, only for the code related to the autosave/the explicit user saves/the zip write, we depend on the user's memory for the actions done before the problem...

Ill let my MuseScore opened all the weekend in my computer, with a score opened of course. Let's see what happens after

Are these old issues solved? https://musescore.org/en/node/15059 , https://musescore.org/en/node/15264 ?

and https://musescore.org/en/node/150281 ?

What's the below TODO: in autoSaveTimer? (code from 2.2.1 branch) ,
https://github.com/musescore/MuseScore/blob/2.2.1/mscore/musescore.cpp#…

//---------------------------------------------------------

void MuseScore::autoSaveTimerTimeout()
{
bool sessionChanged = false;
foreach (Score* s, scoreList) {
if (s->autosaveDirty()) {
QString tmp = s->tmpName();
if (!tmp.isEmpty()) {
QFileInfo fi(tmp);
// TODO: cannot catch exeption here:
s->saveCompressedFile(fi, false);
}
else {
QDir dir;
dir.mkpath(dataPath);
QTemporaryFile tf(dataPath + "/scXXXXXX.mscz");
tf.setAutoRemove(false);
if (!tf.open()) {
qDebug("autoSaveTimerTimeout(): create temporary file failed");
return;
}
s->setTmpName(tf.fileName());
QFileInfo info(tf.fileName());
s->saveCompressedFile(&tf, info, false);
tf.close();
sessionChanged = true;
}
s->setAutosaveDirty(false);
}
}
if (sessionChanged)
writeSessionFile(false);
if (preferences.autoSave) {
int t = preferences.autoSaveTime * 60 * 1000;
autoSaveTimer->start(t);
}
}

In reply to by Jojo-Schmitz

Next corruption reports should tell, Jojo, in my opinion we must ask this one you mention, and perhaps other things about the user's environment (antivirus/ slowness etc.)

But If there would be still some points in file.cpp (or others) that are not catching or propagating exceptions "upside-down".. I would prefer code that fails or crash "blatantly" that "silently" (haha)

Pair of remarks from support system about the issue:

Hello, for the first time I've had serious problems with working on a musescore document (Version 2.1, MacOs 10.12.6). After two hours of composition work I saved and closed the document. Later, when I opened the file again, half of the work was gone, i. e. part of it had obviously not been saved (without a warning by closing).

This guy created the forum post: https://musescore.org/en/node/269405.
Hi there "Cannot read file C:/Users/kevin.LAPTOP-SV2AAG16/Desktop/HSC MUSIC/New Ragtime HSC.mscz:" happens when i attempt to open my score up!
I've been using the plain MusoScore2, and i was saving the same as i always do (when i was finished id hit save and close done.). Started this score around end of last year and everything was perfectly fine. This has happened to me before (with other scores) but i hadn't really cared much because i had finished the previous scores and had already handed them in, but i really need this one for an assessment. Ill attach the score it just says that it cannot read file thats all!

Yes, I have been using MuseScore for all 8 years. Have had this problem once before, but it was resolved through Cloud storage. Sadly, that didn't work this time.

Those files are of no use whatsoever. We need steps to reproduce or finding the common denominator in usage and ideas where to look in code

In reply to by Jojo-Schmitz

Jojo, do you know if any MuseScore developer is actually looking now at this issue, to the code pointed above (for example)?

I'm regrettably not a developer (programmer), but would like to help. Regarding the attachment, I think having a sample in the issue should be mandatory, to not lose time searching for examples in the forum threads. I know that apparently is of no use.

Another idea. It would be useful to ask the users experiencing this issue, if they were on Windows, to consider the chance to submit to this Issue an attachment containing an export (in text or CSV format for example) of the Application Event Log of Windows, more or less at the time of the problem (some time before, ideally). Sometimes, information in this log could give a hint of performance or instability problems in the background.

By exporting the log (or just a part of it) as a text or comma-separated (CSV) files, the user can sanitize information from his/her computer not relevant to the issue, before attaching here.

In reply to by Nicolas

Nicholas,

I just did this test (with 2.1), regarding your first point:

-Opened a large ( about 1000 measures) score. Naturally, it lasted several seconds to load. Changed the autosave time to 1 minute.

-Started a clock in my computer. Did a small change in the score, so to trigger the autosave at the 1 min mark.

-1 secons after the autosave kick in, checked via a cmd window, that MuseScore still didn't written the temp file. 5 seconds after the 1-minute mark, checked that MuseScore created a 0-byte size tempfile , and , at the same time, the UI was freezed (blue circle rotating). After 10 seconds, the tempfile was finally written (about 500 Kbytes), and the UI was again available (responding to menu selections, etc.)

-Lastly I did another change, waited until the kick-in, and when I was ready to enter a Control-S (to save) at the austosave time, again the UI was freezed (again until the tempfile was written)

So, at a first glance, it appears that a collision between a UI Save/Save As and the autosave tempfile, per se, is not feasible. iSeems the running thread who writes the tempfile prevents any action from the UI, blocks it, until the tempfile is written on disk.

So, it appears (to me) that Score::saveCompressedFile in scorefile.cpp would be a good candidate to put a log or trace. I don't think it would that difficult to implement, if you are writing for example the "session" file, you could just write (only to disk, in a logfile) a log of the actions performed inside this function, so, in the case of a user reporting a 0-bytes score, at least have a trace of the inner workings to pinpoint where it halted.

Am I wrong?

In reply to by mdi1972

@mid1972

If I tried the test you just did, my system would freeze. It would take more than a minute to auto save the file and every minute it would attempt to once again auto save the file and I would get the windows blue circle of death until I forced MuseScore to close. I know, I've tried it. So on your system, you cannot interrupt the autosave, that doesn't mean someone with a slower operating system like mine would not be able to do that.

In reply to by mike320

@Mike, could you actually did a Save while the blue circle was visible? if so, tell me.

The blue rotating circle indicates MuseScore is busy doing the operation, and it won't accept any input from the user at all (unless you forcibly kill the MuseScore process of course). What I was interested is to check is the first point @lasconic suggested, if the autosave operation is blocking, and it's appears that yes (at least in my system!)

I can reproduce this every time: until the autosave finish, no UI input is allowed, I think this is good.

Again, this is not to prove the problem is that delay or blocking behaviour of the autosave. My point is to check that maybe a further investigation (ie: traces) would be needed on scorefile.cpp - Score::saveCompressedFile, which is a common function, not only used by the autosave AFAIK.

In reply to by mdi1972

I don't know how you tested whether you can have the save and auto save interrupting one another. Explain and I will run tests.

I turned the auto save on my computer off, because the score I'm working on right now takes about 4 minutes to save, so I NEVER want auto save to kick in. I save it when I'm ready to take a break. I'm curious what would happen if I had auto save turned on and it kicked in while my score was saving. You mentioned a delay between auto save kicking in and you being locked out, I wonder if the delay is long enough on my system to cause problems. I'm willing to test these possibilities, but of course they will take a little more time on my system than yours.

While the circle of death is active I use the task manager to force MuseScore to stop. Fortunately I get the "Would you like to restore your last session?" message.

In reply to by mike320

Re: "I don't know how you tested whether you can have the save and auto save interrupting one another. Explain and I will run tests."

Mike, I didn't said that! it's precisely the contrary I'm verifying. I was precisely interested to test that, and I couldn't, due to the behaviour presented.

Re: "I'm curious what would happen if I had auto save turned on and it kicked in while my score was saving. You mentioned a delay between auto save kicking in and you being locked out, I wonder if the delay is long enough on my system to cause problems."

In my system, what it happens is that the autosave kick-in blocks the UI (and presents the blue circle). And, just the moment it finally writes the tempfile, the blue circle dissapears and the UI resumes to be responsive. So I couldn't, of course, make a Save/Save As during that blockage. That's all.

And that's the reason I think that tracing/logging the actions of scorefile::saveCompressedFile would give a hint to what would happened in the case of a "0-bytes" score, simply.

In reply to by mdi1972

I understand what you are doing now. I'm sure I can test if autosave can kick in while a save as is being executed and find the results. I need a day or two to test it. I'm almost finished with my score, I'll save it to MuseScore.com so it won't matter if it gets clobbered and replaced by zeros.

Severity S2 - Critical S1 - Blocker
Priority P1 - High
Regression No
Reproducibility Randomly
Workaround No