Add Capability to Produce Testable PR Builds to Test Issue Fixes.

• Jul 8, 2019 - 22:05
Reported version
S5 - Suggestion

As a new C++ contributor I think it would be really great to be able to spin a PR's build to product a test install so the issue's fix can be checked by its submitter without them have to build the program.

For example, I've been adding a few items to the PluginAPI and I'm faced with getting a, possibly non-programmer type, to try the change. For them, they have to wait for a next release to exercise the fix. That may be too late.

Here is an actual recent case where having a PR build prevented a release of code with problems because I could iterate the fix with the issue's original author.

I think a method available at the Github interface of creating throw-away installs for PR's would be very helpful to developers and tester's alike. It would make the process more iterative, productive and reliable.

It turns out there is a roundabout way to produce a PR test build for WIndows using Appveyor and putting the magic text "collect_artifacts" in the commit message (not the PR message.) While this is great to have it is cumbersome and ignores testing on non-Windows targets.

I have yet another issue PR that can't be exercised by the submitter because he's Mac based. I'm note equipped to do Mac builds.

I believe that the most motivated tester is the one that reported the problem--so long as they can do the testing easily. I doubt that most issue submitters are equipped (or motivated) to build the software. They just want to see it fixed.

My original comments on this topic can be found here:

My 2 cents anyway.



Another example of why this feature is important:

i.e., The issue's author can't test the fix implementation without the PR being absorbed into 'master' so it can be downloaded in the nightly builds.

Usually a merge to master means the feature is ready to go. This shouldn't happen until all are happy with the implementation in the PR. A bit of a catch-22.