Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0007792OCamlconfigure and build/installpublic2018-05-05 16:352018-06-01 10:28
Reportergasche 
Assigned Togasche 
PrioritylowSeverityminorReproducibilityalways
StatusresolvedResolutionfixed 
PlatformOSOS Version
Product Version4.07.0+dev/beta2/rc1/rc2 
Target VersionFixed in Version4.08.0+dev 
Summary0007792: Running the testsuite several times fails
DescriptionRunning the testsuite several times results in spurious failures, due to the way some tests write to their log outputs -- they append to the log of the previous run instead of replacing them.

A workaround exists, namely running "make clean" in-between runs, but this behavior still results in confusing behaviors and some wasted time.
Additional InformationI have privately reported the issue to S├ębastien Hinderer, the ocamltest author and maintainer, so I believe that the issue will eventually be fixed -- but it seems to be non-trivial. I am opening this issue for him to have a concrete MPR to refer to if useful, and in case other contributors would run into the same behavior and wonder if the issue is known.
TagsNo tags attached.
Attached Files

- Relationships

-  Notes
(0019103)
gasche (administrator)
2018-05-05 16:36

For the record, below is the list of spurious failures that currently occur on subsequent runs:


List of failed tests:
    tests/no-alias-deps/'aliases.ml' with 1.1.1.1.1.1 (check-program-output)
    tests/tool-ocamlobjinfo/'question.ml' with 1.1.1.1.1.1.1 (check-program-output)
    tests/tool-debugger/basic/'debuggee.ml' with 1.1.1.1.1.1 (check-program-output)
    tests/tool-debugger/no_debug_event/'noev.ml' with 1.1.1.1.1.1.1.1.1 (check-program-output)

List of unexpected errors:
    tests/tool-debugger/find-artifacts/'debuggee.ml' with 1.1.1 (script)
(0019146)
gasche (administrator)
2018-05-30 15:15

It appears that the issue is now fixed -- it was fixed by S├ębastien at some point.
(0019147)
gasche (administrator)
2018-05-31 10:05

Note: as of today, the issue still affects the 4.07 release and maintenance branch.
(0019151)
gasche (administrator)
2018-06-01 10:28

(This is now fixed in 4.07 as well.)

- Issue History
Date Modified Username Field Change
2018-05-05 16:35 gasche New Issue
2018-05-05 16:35 gasche Status new => acknowledged
2018-05-05 16:36 gasche Note Added: 0019103
2018-05-30 15:15 gasche Note Added: 0019146
2018-05-30 15:15 gasche Status acknowledged => resolved
2018-05-30 15:15 gasche Fixed in Version => 4.08.0+dev
2018-05-30 15:15 gasche Resolution open => fixed
2018-05-30 15:15 gasche Assigned To => gasche
2018-05-31 10:05 gasche Note Added: 0019147
2018-06-01 10:28 gasche Note Added: 0019151


Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker