Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ocamlopt + flambda requires a lot of memory to compile large array literal expressions #7244

Closed
vicuna opened this issue Apr 27, 2016 · 7 comments
Assignees
Milestone

Comments

@vicuna
Copy link

vicuna commented Apr 27, 2016

Original bug ID: 7244
Reporter: joris
Assigned to: @chambart
Status: closed (set by @damiendoligez on 2016-10-27T14:50:41Z)
Resolution: fixed
Priority: normal
Severity: minor
Target version: 4.04.0 +dev / +beta1 / +beta2
Fixed in version: 4.04.0 +dev / +beta1 / +beta2
Category: back end (clambda to assembly)
Monitored by: @trefis @gasche @ygrek @jmeber @hcarty @Chris00

Bug description

ocamlopt + flambda needs several GB of memory to build a file with a lot of (very) large array literals. ocaml 4.02.3 uses between 4 and 5 times less memory on the same files.

The attached file is generated by ragel, a parser generator which uses arrays to store the automata.

Steps to reproduce

build attached file with ocamlopt.opt -Oclassic ragel_file.ml. On my machine it requires 3.3GB of memory at peak to complete and several minutes. (-Oclassic or -O2 or anything else doesn't actually matter, there is no function in the test file)

(note the attached file is not the full ragel output but is enough to reproduce)

File attachments

@vicuna
Copy link
Author

vicuna commented Apr 27, 2016

Comment author: joris

The file to reproduce is too large and i fail to upload it to mantis, so i've put it here : https://gist.github.com/jorisgio/8e8687be610d1c80a01c696a1aa68f98

@vicuna
Copy link
Author

vicuna commented May 3, 2016

Comment author: @damiendoligez

Got the file, compressed it, and uploaded here.

@vicuna
Copy link
Author

vicuna commented May 17, 2016

Comment author: @Chris00

Not sure it is exactly the same but ocamlopt 4.03.0+flambda fails to compile the file https://github.com/Chris00/color_brewery/blob/master/src/color_brewery_palettes.ml with

Fatal error: exception Stack overflow
Command exited with code 2.

(without flambda, the file compiles fine).

@vicuna
Copy link
Author

vicuna commented Sep 8, 2016

Comment author: @mshinwell

We should check this again with the current 4.04

@vicuna
Copy link
Author

vicuna commented Sep 23, 2016

Comment author: @chambart

After looking a bit more at that, the problem seems to come directly from the file size. There is no specific inefficiency, but the conversion to a sequence of lets makes it too big. One way to handle that may be to circumvent the let conversion by directly producing the let_symbol during closure conversion. It is quite sad to have to special case this, but I can't find any better solution.

@vicuna
Copy link
Author

vicuna commented Sep 23, 2016

Comment author: @chambart

By the way, Christophe, your situation is not the same, you should file a separate bug.

@vicuna
Copy link
Author

vicuna commented Oct 27, 2016

Comment author: @damiendoligez

I've checked and #840 fixes the issue. ( #840 )

@vicuna vicuna closed this as completed Oct 27, 2016
@vicuna vicuna added this to the 4.04.0 milestone Mar 14, 2019
@vicuna vicuna added the bug label Mar 20, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants