sporadic "xvfb-run: error: Xvfb failed to start" in make tests causes Travis to mistakenly report "Broken"

• Mar 14, 2016 - 23:37
Type
Functional
Severity
S4 - Minor
Status
closed
Project
MuseScore

This is a minor annoyance, but I'd like to get fixed so Travis doesn't send email when it happens, since as far as I can tell the error is just with the test scripts, not with the MuseScore application.

When make tests, line
https://travis-ci.org/ericfont/MuseScore/jobs/115985176#L4531 reports error for Test #16:

16/45 Test #16: tst_instrumentchange .............***Exception: Other  0.86 sec

Lines https://travis-ci.org/ericfont/MuseScore/jobs/115985176#L4595-L4598 describe the error as:

The following tests FAILED:
16 - tst_instrumentchange (OTHER_FAULT)
Errors while running CTest
xvfb-run: error: Xvfb failed to start

Now it seems that other projects have had their error on Travis. The problem is due to concurrent tests jobs trying to run simultaneously on framebuffer (I did add "-j2" to make tests...so I take responsiblitlity for that), see:
https://github.com/travis-ci/travis-ci/issues/4387 (Error: "xvfb-run: error: Xvfb failed to start" #4387)
https://github.com/travis-ci/docs-travis-ci-com/pull/311/files (Recommend -a option for xvfb #311)
which says the solution is: 

Use `-a` option to `xvfb-run` so that it selects a free display automatically and starts the server. Do not start `xfvb` manually, this will cause problems with concurrent runs of `xfvb-run`.

I'll try to make the relevant changes and submit PR (although since this is a sporatic bug, it will be hard to know if actually is fixed...all I can hope for is that the error doesn't pop up anymore).

Comments

• Mar 14, 2016 - 23:54

Note, the readme on https://github.com/musescore/MuseScore/tree/master/mtest needs correct the line in "Building the Tests" which currently reads:

make debug install

to be corrected and read:

make installdebug

since the proper command is "make installdebug" according to:
* https://github.com/musescore/MuseScore/blob/master/Makefile
* https://github.com/musescore/MuseScore/blob/master/Makefile.osx

FYI, what "make debug install" does is first "make debug", then "make install", but since make install first does "make release", that means that means that what gets installed is the *release* build, not the *debug* build!!! Also note that additionally does an unnecessary release build!!!

I'll include this readme correction as a sepearate commit in the PR.

Reply

• Mar 15, 2016 - 00:20

Actually, I'm just realizing that the Makefile.osx doesn't include the a make debug at beginning of make installdebug:

installdebug:
	cd build.debug; \
	xcodebuild -project ${XCODEPROJ} -configuration Debug -target

while the linux Makefile does include a make debug at the beginning of make installdebug:

installdebug: debug
	cd build.debug \
	&& make install \
	&& if [ ${UPDATE_CACHE} = "TRUE" ]; then \
	     update-mime-database "${PREFIX}/share/mime"; \
	     gtk-update-icon-cache -f -t "${PREFIX}/share/icons/hicolor"; \
	fi

So those two makefiles are inconsistent in that specific aspect. But to make the readme cleaner and cross platform, I'm going to add debug to first line of Makefile.osx for make installdebug: 

installdebug: debug
	cd build.debug; \
	xcodebuild -project ${XCODEPROJ} -configuration Debug -target

This way can have consistent readme.

I'll also remove the superflous line in https://github.com/musescore/MuseScore/blob/master/build/travis/job1_Te… which makes debug, since it handled by make installdebug. That will all be in a seperate commit in the PR.

Reply

• Mar 15, 2016 - 00:28

oh...well I'm realzing now that it is better to not include "make debug" inside the "make installdebug" since if on home computer, then it is better to run the compilation "make debug" as non-root user, and then only run the *install* portion as sudo. So I'll keep them seperate...in which case that means I'll update the linux Makefile only...

EDIT 1: But on the other hand, if I'm building on my home account, then that means I simply use

make debug
sudo make installdebug

and the installdebug will notice that debug is already built. So now I'm back to saying that the Makefile.osx should be the one that is changed.

EDIT 2: I'm now realizing that the Makefile.mingw adheres to the way Makefile.osx does it...in that it doesn't automatically make debug at beginning:

installdebug:
	cd build.debug & $(MAKE) install

So I'm at point now where I think I will not change the makefiles for now, unless someone else wants to. I'll just fix the readme to indicate how to build test on home machine using "make debug && sudo make installdebug"

Reply

• Mar 17, 2016 - 08:17

The tests are not run concurrently as far as I understand, even with j45 :). That will just build them concurrently. They are built, all of them, and then run one by one. A possible problem is that the first failure blocks xvfb-run, so adding -a can't hurt.

Modification of the REAME in mtest is welcome! Thanks!

Reply

• Mar 17, 2016 - 08:45

I'm noticing that line is just performing a *Single* xvfb-run command (but running a bunch of tests for that one xvfb-run). So that makes me think that it is not the concurrency that is casuing the sporadic error. So maybe that sporadic error also occured sometimes with -j1.

Anyway, I'm thinking we should make that "xvfb-run -a ctest -j2 --output-on-failure" and hope that that fixes it. That is what the official directions https://github.com/travis-ci/docs-travis-ci-com/blob/0bc92eabe46348c6fc… tell us...

Reply