Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0007473OCamlplatform support (windows, cross-compilation, etc)public2017-01-31 18:522017-02-06 20:00
Reporterprotz 
Assigned To 
PrioritynormalSeveritymajorReproducibilityalways
StatusresolvedResolutionfixed 
PlatformOSOS Version
Product Version4.04.0 
Target Version4.05.0 +dev/beta1/beta2/beta3/rc1Fixed in Version4.05.0 +dev/beta1/beta2/beta3/rc1 
Summary0007473: caml_expand_command_line behaves incorrectly
DescriptionConsider this:

D:\cygwin\home\protz\Code\hacl-star\code\curve25519>.\camlprog.exe Hacl.Spec.*
.\camlprog.exe
Hacl.Spec.*Hacl.Spec.Bignum.Crecip.fst
Hacl.Spec.*Hacl.Spec.Bignum.Crecip.fst.hints
Hacl.Spec.*Hacl.Spec.Bignum.Fmul.Lemmas.fst
Hacl.Spec.*Hacl.Spec.Bignum.Fsquare.fst
Hacl.Spec.*Hacl.Spec.Bignum.Fsquare.fst.hints
Hacl.Spec.*Hacl.Spec.Bignum.Modulo.fst
Hacl.Spec.*Hacl.Spec.Bignum.Modulo.fst.hints
Hacl.Spec.*Hacl.Spec.EC.AddAndDouble.fst
Hacl.Spec.*Hacl.Spec.EC.AddAndDouble.fst.hints
Hacl.Spec.*Hacl.Spec.EC.AddAndDouble2.fst
Hacl.Spec.*Hacl.Spec.EC.AddAndDouble2.fst.hints
Hacl.Spec.*Hacl.Spec.EC.Format.fst
Hacl.Spec.*Hacl.Spec.EC.Format.fst.hints
Hacl.Spec.*Hacl.Spec.EC.fst
Hacl.Spec.*Hacl.Spec.EC.Ladder.fst
Hacl.Spec.*Hacl.Spec.EC.Ladder.fst.hints
Hacl.Spec.*Hacl.Spec.EC.Point.fst
Hacl.Spec.*Hacl.Spec.EC.Point.fst.hints

D:\cygwin\home\protz\Code\hacl-star\code\curve25519>type test.ml
let _ = Array.iter print_endline Sys.argv


This is the native windows toolchain, compiled with x86_64-w64-mingw32.

This is the contents of the directory:
D:\cygwin\home\protz\Code\hacl-star\code\curve25519>dir
 Volume in drive D is DATADRIVE1
 Volume Serial Number is BA7C-B54F

 Directory of D:\cygwin\home\protz\Code\hacl-star\code\curve25519

01/31/2017  09:24 AM    <DIR>          .
01/31/2017  09:24 AM    <DIR>          ..
01/31/2017  09:24 AM             6,697 .vimsession
01/31/2017  09:24 AM           142,833 a.exe
01/31/2017  09:16 AM           634,551 camlprog.exe
01/31/2017  08:35 AM               581 Curve25519.fst
01/31/2017  08:54 AM                26 FStar.c
01/31/2017  08:54 AM             1,057 FStar.h
01/30/2017  03:25 PM               431 Hacl.Bignum.Constants.fst
01/30/2017  03:25 PM               755 Hacl.Bignum.Constants.fst.hints
01/30/2017  03:25 PM            12,781 Hacl.Bignum.Crecip.fst
01/30/2017  03:25 PM            72,855 Hacl.Bignum.Crecip.fst.hints
01/30/2017  03:25 PM             5,860 Hacl.Bignum.Fsquare.fst
01/30/2017  03:25 PM            25,175 Hacl.Bignum.Fsquare.fst.hints
01/30/2017  03:25 PM             3,850 Hacl.Bignum.Modulo.fst
01/30/2017  03:25 PM            29,194 Hacl.Bignum.Modulo.fst.hints
01/30/2017  03:25 PM            10,024 Hacl.Bignum.Parameters.fst
01/30/2017  03:25 PM            61,043 Hacl.Bignum.Parameters.fst.hints
01/30/2017  04:03 PM            28,499 Hacl.EC.AddAndDouble.fst
01/30/2017  03:25 PM            38,849 Hacl.EC.AddAndDouble.fst.hints
01/30/2017  03:47 PM             9,843 Hacl.EC.Format.fst
01/30/2017  03:25 PM            57,407 Hacl.EC.Format.fst.hints
01/31/2017  08:37 AM             3,378 Hacl.EC.fst
01/30/2017  03:25 PM            19,832 Hacl.EC.fst.hints
01/30/2017  03:47 PM             1,139 Hacl.EC.Ladder.BigLoop.fst
01/30/2017  03:25 PM             6,384 Hacl.EC.Ladder.BigLoop.fst.hints
01/30/2017  03:47 PM             5,071 Hacl.EC.Ladder.fst
01/30/2017  03:25 PM            35,464 Hacl.EC.Ladder.fst.hints
01/30/2017  03:25 PM             6,820 Hacl.EC.Ladder.SmallLoop.fst
01/30/2017  03:25 PM            24,478 Hacl.EC.Ladder.SmallLoop.fst.hints
01/30/2017  03:25 PM             6,970 Hacl.EC.Point.fst
01/30/2017  03:25 PM            37,136 Hacl.EC.Point.fst.hints
01/30/2017  03:25 PM             2,482 Hacl.Spec.Bignum.Crecip.fst
01/30/2017  03:25 PM            12,340 Hacl.Spec.Bignum.Crecip.fst.hints
01/30/2017  03:25 PM               787 Hacl.Spec.Bignum.Fmul.Lemmas.fst
01/30/2017  03:25 PM            39,477 Hacl.Spec.Bignum.Fsquare.fst
01/30/2017  03:25 PM           127,079 Hacl.Spec.Bignum.Fsquare.fst.hints
01/30/2017  03:25 PM            11,630 Hacl.Spec.Bignum.Modulo.fst
01/30/2017  03:25 PM            78,665 Hacl.Spec.Bignum.Modulo.fst.hints
01/30/2017  03:25 PM            43,828 Hacl.Spec.EC.AddAndDouble.fst
01/30/2017  03:25 PM           231,152 Hacl.Spec.EC.AddAndDouble.fst.hints
01/30/2017  03:25 PM             5,088 Hacl.Spec.EC.AddAndDouble2.fst
01/30/2017  03:25 PM            13,539 Hacl.Spec.EC.AddAndDouble2.fst.hints
01/30/2017  03:47 PM             7,025 Hacl.Spec.EC.Format.fst
01/30/2017  03:25 PM            31,046 Hacl.Spec.EC.Format.fst.hints
01/30/2017  03:25 PM               578 Hacl.Spec.EC.fst
01/30/2017  03:47 PM             3,668 Hacl.Spec.EC.Ladder.fst
01/30/2017  03:25 PM             7,644 Hacl.Spec.EC.Ladder.fst.hints
01/30/2017  03:25 PM             2,841 Hacl.Spec.EC.Point.fst
01/30/2017  03:25 PM            16,367 Hacl.Spec.EC.Point.fst.hints
01/30/2017  03:47 PM             2,384 Hacl.Test.X25519.fst
01/31/2017  08:54 AM             1,056 Hacl_Bignum.c
01/31/2017  08:54 AM             1,698 Hacl_Bignum.h
01/31/2017  08:54 AM               314 Hacl_Bignum_Constants.c
01/31/2017  08:54 AM               651 Hacl_Bignum_Constants.h
01/31/2017  08:54 AM             2,477 Hacl_Bignum_Crecip.c
01/31/2017  08:54 AM             1,273 Hacl_Bignum_Crecip.h
01/31/2017  08:54 AM               363 Hacl_Bignum_Fdifference.c
01/31/2017  08:54 AM             1,337 Hacl_Bignum_Fdifference.h
01/31/2017  08:54 AM             1,782 Hacl_Bignum_Fmul.c
01/31/2017  08:54 AM             1,594 Hacl_Bignum_Fmul.h
01/31/2017  08:54 AM             2,682 Hacl_Bignum_Fproduct.c
01/31/2017  08:54 AM             1,691 Hacl_Bignum_Fproduct.h
01/31/2017  08:54 AM               540 Hacl_Bignum_Fscalar.c
01/31/2017  08:54 AM             1,471 Hacl_Bignum_Fscalar.h
01/31/2017  08:54 AM             3,111 Hacl_Bignum_Fsquare.c
01/31/2017  08:54 AM             1,357 Hacl_Bignum_Fsquare.h
01/31/2017  08:54 AM               328 Hacl_Bignum_Fsum.c
01/31/2017  08:54 AM             1,112 Hacl_Bignum_Fsum.h
01/31/2017  08:54 AM             2,700 Hacl_Bignum_Limb.c
01/31/2017  08:54 AM             2,350 Hacl_Bignum_Limb.h
01/31/2017  08:54 AM             1,811 Hacl_Bignum_Modulo.c
01/31/2017  08:54 AM             1,480 Hacl_Bignum_Modulo.h
01/31/2017  08:54 AM             5,626 Hacl_Bignum_Parameters.c
01/31/2017  08:54 AM             4,634 Hacl_Bignum_Parameters.h
01/31/2017  08:54 AM             3,554 Hacl_Bignum_Wide.c
01/31/2017  08:54 AM             2,686 Hacl_Bignum_Wide.h
01/31/2017  08:54 AM             2,251 Hacl_Cast.c
01/31/2017  08:54 AM             1,602 Hacl_Cast.h
01/31/2017  08:54 AM             1,307 Hacl_EC.c
01/31/2017  08:54 AM             1,607 Hacl_EC.h
01/31/2017  08:54 AM             3,232 Hacl_EC_AddAndDouble.c
01/31/2017  08:54 AM             1,530 Hacl_EC_AddAndDouble.h
01/31/2017  08:54 AM             5,020 Hacl_EC_Format.c
01/31/2017  08:54 AM             1,545 Hacl_EC_Format.h
01/31/2017  08:54 AM               879 Hacl_EC_Ladder.c
01/31/2017  08:54 AM             1,983 Hacl_EC_Ladder.h
01/31/2017  08:54 AM               518 Hacl_EC_Ladder_BigLoop.c
01/31/2017  08:54 AM             1,651 Hacl_EC_Ladder_BigLoop.h
01/31/2017  08:54 AM             2,043 Hacl_EC_Ladder_SmallLoop.c
01/31/2017  08:54 AM             1,675 Hacl_EC_Ladder_SmallLoop.h
01/31/2017  08:54 AM             1,505 Hacl_EC_Point.c
01/31/2017  08:54 AM             1,728 Hacl_EC_Point.h
01/31/2017  08:54 AM               568 Hacl_Spec_Bignum.c
01/31/2017  08:54 AM               990 Hacl_Spec_Bignum.h
01/31/2017  08:54 AM             1,950 Hacl_Spec_Bignum_Bigint.c
01/31/2017  08:54 AM             1,800 Hacl_Spec_Bignum_Bigint.h
01/31/2017  08:54 AM             1,178 Hacl_Spec_Bignum_Crecip.c
01/31/2017  08:54 AM             1,511 Hacl_Spec_Bignum_Crecip.h
01/31/2017  08:54 AM               420 Hacl_Spec_Bignum_Fdifference.c
01/31/2017  08:54 AM               962 Hacl_Spec_Bignum_Fdifference.h
01/31/2017  08:54 AM             1,927 Hacl_Spec_Bignum_Field.c
01/31/2017  08:54 AM             2,058 Hacl_Spec_Bignum_Field.h
01/31/2017  08:54 AM             2,064 Hacl_Spec_Bignum_Fmul.c
01/31/2017  08:54 AM             2,147 Hacl_Spec_Bignum_Fmul.h
01/31/2017  08:54 AM             3,022 Hacl_Spec_Bignum_Fproduct.c
01/31/2017  08:54 AM             2,630 Hacl_Spec_Bignum_Fproduct.h
01/31/2017  08:54 AM               680 Hacl_Spec_Bignum_Fscalar.c
01/31/2017  08:54 AM             1,085 Hacl_Spec_Bignum_Fscalar.h
01/31/2017  08:54 AM             7,572 Hacl_Spec_Bignum_Fsquare.c
01/31/2017  08:54 AM             6,507 Hacl_Spec_Bignum_Fsquare.h
01/31/2017  08:54 AM               371 Hacl_Spec_Bignum_Fsum.c
01/31/2017  08:54 AM               949 Hacl_Spec_Bignum_Fsum.h
01/31/2017  08:54 AM             1,503 Hacl_Spec_Bignum_Modulo.c
01/31/2017  08:54 AM             1,480 Hacl_Spec_Bignum_Modulo.h
01/31/2017  08:54 AM             5,081 Hacl_Spec_EC_AddAndDouble.c
01/31/2017  08:54 AM             4,828 Hacl_Spec_EC_AddAndDouble.h
01/31/2017  08:54 AM             2,546 Hacl_Spec_EC_AddAndDouble2.c
01/31/2017  08:54 AM             2,543 Hacl_Spec_EC_AddAndDouble2.h
01/31/2017  08:54 AM             1,836 Hacl_Spec_EC_Format.c
01/31/2017  08:54 AM             1,899 Hacl_Spec_EC_Format.h
01/31/2017  08:54 AM             5,947 Hacl_Spec_EC_Ladder.c
01/31/2017  08:54 AM             2,741 Hacl_Spec_EC_Ladder.h
01/31/2017  08:54 AM             3,283 Hacl_Spec_EC_Point.c
01/31/2017  08:54 AM             2,593 Hacl_Spec_EC_Point.h
01/31/2017  08:54 AM             4,045 Hacl_Test_X25519.c
01/31/2017  08:54 AM             1,592 Hacl_Test_X25519.h
01/31/2017  08:54 AM             3,437 Hacl_UInt128.c
01/31/2017  08:54 AM             2,433 Hacl_UInt128.h
01/31/2017  08:54 AM             2,545 Hacl_UInt32.c
01/31/2017  08:54 AM             2,106 Hacl_UInt32.h
01/31/2017  08:54 AM             2,545 Hacl_UInt64.c
01/31/2017  08:54 AM             1,992 Hacl_UInt64.h
01/31/2017  08:54 AM             2,429 Hacl_UInt8.c
01/31/2017  08:54 AM             1,965 Hacl_UInt8.h
01/31/2017  09:15 AM             2,623 Makefile
01/31/2017  08:54 AM                26 Prims.c
01/31/2017  08:54 AM               126 Prims.h
01/31/2017  09:24 AM               119 test.c
01/31/2017  09:16 AM               234 test.cmi
01/31/2017  09:16 AM               287 test.cmx
01/31/2017  09:16 AM                42 test.ml
01/31/2017  09:16 AM             1,112 test.o
01/31/2017  08:55 AM    <DIR>          x25519-c
01/31/2017  08:55 AM           654,423 x25519.exe
             142 File(s)      2,767,109 bytes
               3 Dir(s)  899,522,285,568 bytes free


Note that I'm running this within a Windows shell to rule out Cygwin behavior.

Note also:

D:\cygwin\home\protz\Code\hacl-star\code\curve25519>type test.c
#include <stdio.h>

int main(int argc, char *argv[]) {
  for (int i = 0; i < argc; ++i)
    printf("%s\n", argv[i]);
}

D:\cygwin\home\protz\Code\hacl-star\code\curve25519>.\a.exe Hacl.Spec.*
.\a.exe
Hacl.Spec.*
TagsNo tags attached.
Attached Files

- Relationships

-  Notes
(0017216)
protz (manager)
2017-01-31 18:54

For the record, I'm trying to add a new option to my tool of the form:

krml -drop Hacl.Spec.*


and I'm just trying to find a way to pass the argument "as-is" without anyone trying to be smart about shell-expansion... if there's a way to bypass caml_expand_command_line I'd be very happy about it.

<personal-opinion>
perhaps we should just drop caml_expand_command_line altogether... it seems like a surprising behavior
</personal-opinion>
(0017218)
xleroy (administrator)
2017-01-31 19:31

We have two issues / questions here.

1- Globbing of file name patterns not working. Looks bad indeed, and I'm too lazy to get access to a Windows machine right now. What happens if you put a directory name? As in ".\camlprog.exe ./Hack.Spec.*"

2- Turning globbing off. Currently there is no way to do that.

2a- Why globbing, you ask? I believe(d) this is the behavior of a great many Windows command-line tools, including the MSVC compiler, and Cygwin commands when started from a bash shell. But maybe things have changed in the last 20 years...

2b- Should we give a way to turn globbing off? Perhaps. A flag in OCAMLRUNPARAM would do, for example, but is not terribly friendly.
(0017220)
protz (manager)
2017-01-31 19:46

Note: I managed to reproduce on 4.03.0 and 4.04.0 versions, that I built myself.

1- Your suggestion performs proper path expansion.

2a- Granted; I just always start my commands from a cygwin shell so I always relied on bash's shell expansion feature, and never felt the need to have OCaml do it.

2b- unclear... perhaps I'm the only one who wants that sort of options? Hacl.Spec.* denotes the set of F* modules that start with that given prefix...
(0017222)
xleroy (administrator)
2017-01-31 20:05

Thanks for the additional test and info. I strongly suspect bad string surgery on lines 367-370 of byterun/win32.c.
(0017223)
protz (manager)
2017-01-31 20:10

Some more info from Redmond. I asked an expert, and:
- yes, OCaml performing wildcard expansion is a desirable behavior
- you can actually have the Microsoft C runtime do it for you, if you wish to (https://msdn.microsoft.com/en-us/library/8bch7bkk.aspx [^]).
(0017224)
xleroy (administrator)
2017-01-31 20:15

So, just by linking with a Microsoft-provided library, all the bugs become theirs? Great! thanks for the tip! Just need to check that setargv.obj is available in Mingw as well as in MSVC...
(0017225)
protz (manager)
2017-01-31 20:22

https://github.com/ocaml/ocaml/pull/1025 [^]
(0017226)
protz (manager)
2017-01-31 20:24

Submitted a pull request for a quickfix that seems to work for me. Otherwise, yes, the more future-proof version is to use the .obj provided by MSVC... but this may prevent David Allsopp from building with circa 1997 versions of MSVC.

Feel free to discard the pull request if you want to go with the cleaner version, it didn't take me too long to fix :)
(0017227)
dra (developer)
2017-02-01 10:26

@protz the globbing expansion via setargv.obj in MSVCRT is as old the hills, you will not break my museum :p

@xleroy - your memory of Cygwin is correct, unless noglob is set in CYGWIN, it does a lot more work to expand wildcards when a Cygwin program is invoked from outside Cygwin (i.e. from cmd) and it's always and continues to be the case that Windows processes are expected to do globbing, not the shell.

It would be good if there were a way to prevent the expansion being done (or opt into it?) - we could do that in the same way as Microsoft by adding some kind of setenv.cmo/setenv.cmx or we could consider something less inelegant (like providing the original argv somewhere!)...
(0017233)
xleroy (administrator)
2017-02-06 19:58

Fixed on trunk by merge of commit [561b846].

- Issue History
Date Modified Username Field Change
2017-01-31 18:52 protz New Issue
2017-01-31 18:54 protz Note Added: 0017216
2017-01-31 19:27 protz Note Added: 0017217
2017-01-31 19:31 xleroy Note Added: 0017218
2017-01-31 19:31 xleroy Status new => feedback
2017-01-31 19:43 protz Note Deleted: 0017217
2017-01-31 19:46 protz Note Added: 0017220
2017-01-31 19:46 protz Status feedback => new
2017-01-31 20:05 xleroy Note Added: 0017222
2017-01-31 20:05 xleroy Status new => acknowledged
2017-01-31 20:05 xleroy Target Version => 4.05.0 +dev/beta1/beta2/beta3/rc1
2017-01-31 20:10 protz Note Added: 0017223
2017-01-31 20:15 xleroy Note Added: 0017224
2017-01-31 20:22 protz Note Added: 0017225
2017-01-31 20:24 protz Note Added: 0017226
2017-02-01 10:26 dra Note Added: 0017227
2017-02-06 19:58 xleroy Note Added: 0017233
2017-02-06 19:58 xleroy Status acknowledged => resolved
2017-02-06 19:58 xleroy Resolution open => fixed
2017-02-06 20:00 xleroy Fixed in Version => 4.05.0 +dev/beta1/beta2/beta3/rc1
2017-02-23 16:46 doligez Category OCaml windows => platform support (windows, etc)
2017-02-23 17:16 doligez Category platform support (windows, etc) => platform support (windows, cross-compilation, etc)


Copyright © 2000 - 2011 MantisBT Group
Powered by Mantis Bugtracker