Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incompatibility between ocamllex and compiler #5598

Closed
vicuna opened this issue Apr 22, 2012 · 5 comments
Closed

Incompatibility between ocamllex and compiler #5598

vicuna opened this issue Apr 22, 2012 · 5 comments
Assignees
Labels

Comments

@vicuna
Copy link

vicuna commented Apr 22, 2012

Original bug ID: 5598
Reporter: @mmottl
Assigned to: @damiendoligez
Status: closed (set by @xavierleroy on 2015-12-11T18:25:36Z)
Resolution: fixed
Priority: normal
Severity: minor
Platform: Any
OS: Any
OS Version: Any
Version: 3.12.1
Fixed in version: 4.00.0+dev
Category: ~DO NOT USE (was: OCaml general)
Related to: #6165

Bug description

It seems ocamllex and the compiler once shared the same parsing code for OCaml-strings, but have since slightly diverged even though the manual states that they both use the same syntax.

lexer.mll for ocamllex defines "backslash_escapes" that are different from the inline pattern used in lexer.mll (rule "string") for the compiler: the latter adds a space character. Furthermore, ocamllex uses '\010' as a pattern for newlines where the compiler uses "newline", which also supports '\013' and DOS-newlines and has hence a different rule body.

ocamllex apparently has some catching up to do...

@vicuna
Copy link
Author

vicuna commented May 2, 2012

Comment author: @damiendoligez

For backslash-space, I've fixed ocamllex (commit 12418 in 4.00, commit 12419 in trunk).

For newlines, ocamllex supports both Unix and DOS because it ignores \013 and takes \010 as the newline character.

The \013-as-newline support in the compiler is a leftover from the last century (support for Mac OS 9), which will be removed from the compiler after the 4.00 release (commit 12420 in trunk).

@vicuna
Copy link
Author

vicuna commented May 2, 2012

Comment author: @mmottl

It seems that the fix for this problem accidentally removed DOS-style newlines ("\013\010") rather than the Mac OS 9 "\013" from the definition of "newline" in parsing/lexer.mll. I guess this would lead to double counting of lines on Windows.

Update: "\013" may then probably need to be added to the definition of "blank" instead?

@vicuna
Copy link
Author

vicuna commented May 3, 2012

Comment author: @damiendoligez

Oops, thanks for catching this. Fixed (commit 12424 in trunk).

I don't think we want to add \013 to the definition of blank. I don't see any point in allowing it in source files.

@vicuna
Copy link
Author

vicuna commented Jan 22, 2014

Comment author: @damiendoligez

I've changed my mind. Now I think the perfect solution would be to utterly ignore all occurrences of \013 everywhere in the source file (even within identifiers or string literals). But that would be too much work for such a small problem.

@vicuna
Copy link
Author

vicuna commented Jan 22, 2014

Comment author: @damiendoligez

Following the fix of #6165, I have brought ocamllex in line with ocamlc/ocamlopt (commit 14406 in trunk).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants