Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Running the testsuite several times fails #7792

Closed
vicuna opened this issue May 5, 2018 · 4 comments
Closed

Running the testsuite several times fails #7792

vicuna opened this issue May 5, 2018 · 4 comments
Assignees
@gasche
Labels
bug configure-related

Comments

@vicuna
Copy link

vicuna commented May 5, 2018

Original bug ID: 7792
Reporter: @gasche
Assigned to: @gasche
Status: resolved (set by @gasche on 2018-05-30T13:15:46Z)
Resolution: fixed
Priority: low
Severity: minor
Version: 4.07.0+dev/beta2/rc1/rc2
Fixed in version: 4.08.0+dev/beta1/beta2
Category: configure and build/install

Bug description

Running the testsuite several times results in spurious failures, due to the way some tests write to their log outputs -- they append to the log of the previous run instead of replacing them.

A workaround exists, namely running "make clean" in-between runs, but this behavior still results in confusing behaviors and some wasted time.

Additional information

I have privately reported the issue to Sébastien Hinderer, the ocamltest author and maintainer, so I believe that the issue will eventually be fixed -- but it seems to be non-trivial. I am opening this issue for him to have a concrete MPR to refer to if useful, and in case other contributors would run into the same behavior and wonder if the issue is known.
@vicuna
Copy link
Author

vicuna commented May 5, 2018

Comment author: @gasche

For the record, below is the list of spurious failures that currently occur on subsequent runs:

List of failed tests:
tests/no-alias-deps/'aliases.ml' with 1.1.1.1.1.1 (check-program-output)
tests/tool-ocamlobjinfo/'question.ml' with 1.1.1.1.1.1.1 (check-program-output)
tests/tool-debugger/basic/'debuggee.ml' with 1.1.1.1.1.1 (check-program-output)
tests/tool-debugger/no_debug_event/'noev.ml' with 1.1.1.1.1.1.1.1.1 (check-program-output)

List of unexpected errors:
tests/tool-debugger/find-artifacts/'debuggee.ml' with 1.1.1 (script)

@vicuna
Copy link
Author

vicuna commented May 30, 2018

Comment author: @gasche

It appears that the issue is now fixed -- it was fixed by Sébastien at some point.

@vicuna vicuna closed this as completed May 30, 2018
@vicuna
Copy link
Author

vicuna commented May 31, 2018

Comment author: @gasche

Note: as of today, the issue still affects the 4.07 release and maintenance branch.

@vicuna
Copy link
Author

vicuna commented Jun 1, 2018

Comment author: @gasche

(This is now fixed in 4.07 as well.)

@vicuna vicuna added the bug label Mar 20, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gasche gasche

Labels
bug configure-related
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@gasche @vicuna