Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0007792OCamlconfigure and build/installpublic2018-05-05 16:352018-05-05 16:36
Reportergasche 
Assigned To 
PrioritylowSeverityminorReproducibilityalways
StatusacknowledgedResolutionopen 
PlatformOSOS Version
Product Version4.07.0+dev/beta2 
Target VersionFixed in Version 
Summary0007792: Running the testsuite several times fails
DescriptionRunning the testsuite several times results in spurious failures, due to the way some tests write to their log outputs -- they append to the log of the previous run instead of replacing them.

A workaround exists, namely running "make clean" in-between runs, but this behavior still results in confusing behaviors and some wasted time.
Additional InformationI have privately reported the issue to S├ębastien Hinderer, the ocamltest author and maintainer, so I believe that the issue will eventually be fixed -- but it seems to be non-trivial. I am opening this issue for him to have a concrete MPR to refer to if useful, and in case other contributors would run into the same behavior and wonder if the issue is known.
TagsNo tags attached.
Attached Files

- Relationships

-  Notes
(0019103)
gasche (developer)
2018-05-05 16:36

For the record, below is the list of spurious failures that currently occur on subsequent runs:


List of failed tests:
    tests/no-alias-deps/'aliases.ml' with 1.1.1.1.1.1 (check-program-output)
    tests/tool-ocamlobjinfo/'question.ml' with 1.1.1.1.1.1.1 (check-program-output)
    tests/tool-debugger/basic/'debuggee.ml' with 1.1.1.1.1.1 (check-program-output)
    tests/tool-debugger/no_debug_event/'noev.ml' with 1.1.1.1.1.1.1.1.1 (check-program-output)

List of unexpected errors:
    tests/tool-debugger/find-artifacts/'debuggee.ml' with 1.1.1 (script)

- Issue History
Date Modified Username Field Change
2018-05-05 16:35 gasche New Issue
2018-05-05 16:35 gasche Status new => acknowledged
2018-05-05 16:36 gasche Note Added: 0019103


Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker