0005221: Gram.token_info abstraction is too restrictrive, could be moved to Token

View Issue Details [ Jump to Notes ]

[ Issue History ] [ Print ]

Project

Category

View Status

Date Submitted

Last Update

0005221

OCaml

Camlp4

public

2011-02-05 13:58

2011-12-12 14:54

Reporter

bluestorm

Assigned To

xclerc

Priority

normal

Severity

feature

Reproducibility

always

Status

assigned

Resolution

open

Platform

OS Version

Product Version

3.12.0

Target Version

Fixed in Version

Summary

0005221: Gram.token_info abstraction is too restrictrive, could be moved to Token

Description

I'm having a type regression under 3.12 due to a formerly public type which is now private in Camlp4 interface.

My use case is to compose the Gram1.Entry.of_parser and Gram2.parse_tokens_after_input, to define a grammar entry from the action of a grammar entry of a *different* grammar. This was possible in 3.10/11 as both functions relied on the following type:

(Token.t * Loc.t) Stream.t

As both grammars relied on the same Lexer module -- of course it doesn't make sense to apply one grammar to an other input stream if the lexers are different -- the Token and Loc module are shared, the types are compatible and this works fine.

At 3.12 the type was changed to:

(Token.t * token_info) Stream.t

where token_info is a private type of the `Gram` module. Two private types of distinct modules are incompatible, so my code doesn't compile anymore.

I suggest the token_info type definition could be moved into the Token module. I don't think its semantic place it necessarily in the Grammar module, it's mostly a per-token information. If it was defined in Token, even abstractly, different grammars sharing the same Token and Loc module could communicate `token_info` again.

This change wouldn't break anything as it would only expose more sharing of the `token_info` type. Of course, for backward-compatibility, an alias inside Gram (type token_info = Token.token_info) would be fine.

Additional Information

Attached is a simple example file exhibiting the problem: it compiles under 3.10/11, but not under 3.12. It is actually faithfully reproduces my use case in the Macaque project.

Tags

No tags attached.

Attached Files

sample.ml [^] (981 bytes) 2011-02-05 13:58

Relationships

Notes
(0005796) ertai (developer) 2011-02-05 20:17	I committed a fix in the version/3.12 branch. I prefered to keep the token_info type close to the grammar code. However thanks to applicative functors, I've written the type in such a way that the type only depends on the location type. Please try this commit and tell us if it fixes your issue.

Issue History
Date Modified	Username	Field	Change
2011-02-05 13:58	bluestorm	New Issue
2011-02-05 13:58	bluestorm	Status	new => assigned
2011-02-05 13:58	bluestorm	Assigned To	=> ertai
2011-02-05 13:58	bluestorm	File Added: sample.ml
2011-02-05 20:17	ertai	Note Added: 0005796
2011-10-26 08:44	ertai	Assigned To	ertai => xclerc