Re: [wtp-dev] JSDT Parser

[Eugene Melekhov <emvv@xxxxxxx>]

Gorkem,

Thanks for prompt reply. I read your post about benchmarking with pleasure. I've never heard about Shift parser and it
looks very promising. I took a quick glimpse at sources and imho they are really smart, neat and "clean" :-). Great job.

BTW, they implemented location info, so the only missing part is error recovery / tolerant parsing. Speaking of which,
it seems that with their two-phase parser recovery is only required at syntax level. "Semantic/Early errors" are
handled separately, which on the one hand probably makes recovery easier to implement and on the other hand gives client
some freedom to use second pass only if needed, and speed of course...

I ran your benchmarks (w/o node. I didn't want to bother with configuring node benchmark properly on my Windows box).
Below are my results:

# Run complete. Total time: 01:26:54

Benchmark                           Mode  Cnt   Score   Error  Units
AcornWithJ2V8.testAngular125       thrpt  200  10,176 ? 0,045  ops/s
AcornWithJ2V8.testJQM142           thrpt  200   7,755 ? 0,023  ops/s
AcornWithNashorn.testAngular125    thrpt  200  11,398 ? 0,310  ops/s
AcornWithNashorn.testJQM142        thrpt  200   8,793 ? 0,257  ops/s
EsprimaWithJ2V8.testAngular125     thrpt  200  11,393 ? 0,135  ops/s
EsprimaWithJ2V8.testJQM142         thrpt  200   9,432 ? 0,029  ops/s
EsprimaWithNashorn.testAngular125  thrpt  200  15,459 ? 0,130  ops/s
EsprimaWithNashorn.testJQM142      thrpt  200  15,223 ? 0,110  ops/s
NashornParser.testAngular125       thrpt  200  41,306 ? 0,208  ops/s
NashornParser.testJQM142           thrpt  200  37,744 ? 0,104  ops/s
ShiftJava.testAngular125           thrpt  200  74,734 ? 0,203  ops/s
ShiftJava.testJQM142               thrpt  200  76,934 ? 0,413  ops/s

As you might notice I've added NashornParser (native JVM) test (sample borrowed from Nashorn sources). And it performs
quite well, not as good as Shift, but not bad.

So, according to the benchmarks Esprima with Nashorn and Nashorn "native JVM" parser are good options, but in the long
run I'd prefer to use Shift because it's fast, it's nice Java etc. It might worth trying to add error recovery to it if
it's not that hard....

I've managed to obtain, compile, run/test latest code that you've mentioned; I have more or less normal working
configuration and can do something useful. So, what would you suggest to look at? Some small/relatively easy task that
would allow me to get acquainted with code and bring some value? I thought that something like unit tests for particular
parser/ast related parts would be a good start. What do you think?

Thanks.

> Hi Eugene,

> Yes, help is definitely is definitely needed and appreciated.

> There is an ongoing work around JavaScript parser and you can see the 
> latest code at https://git.eclipse.org/r/#/c/67469/
> At this time we are trying to introduce the esprima parser by running it 
> with nashorn. We will
> be using the parser to generate what is known as DOM AST or external AST 
> and retire the old/internal AST that is tightly
> couple to the current parser.

> We did consider several options for a replacement parser [1]. I have 
> also looked at mending the JDT compiler based parser.
> Even though it is possible to fix the current parser as you have said 
> with a few good alternates around I do not think
> the effort is worth it.

> I am aware of the JEP 236, and we are architecting the new parser in 
> such a way that we can replace it relatively painless
> in the future. The replacement can easily be Nashorn’s parser provided 
> that it supports the latest Ecmascript specification.

> Please let me know if you have any further questions. I also recommend 
> reading our recently revamped contributing wiki [2].

> [1] 
> http://www.gorkem-ercan.com/2015/11/benchmarking-javascript-parsers-for.html
> [2] https://wiki.eclipse.org/JSDT/Development
> —
> Gorkem


> On 10 Mar 2016, at 2:51, Eugene Melekhov wrote:

>> Hi all,
>>
>> I'd like to help JSDT and I believe that Compiler/Parser is one of the 
>> places that needs change. Does anyone works on
>> it? Are there any new implementation ideas?
>>
>> I see that at the moment it's based on the JDT compiler code base and 
>> thus depends on jikes parser generator. Even
>> though using generator is OK in general it seems to me that 
>> maintaining and developing such parser is harder then
>> maintaining just hand-written one; especially when up-to-date 
>> open-source JS parsers are available at the moment.
>>
>> I looked at the Nashorn sources and they are quite clean, readable and 
>> maintainable, though I don't have any performance
>> comparison results. There is no Nashorn's public parser API yet, but 
>> JEP 236 addresses this issue and it will be
>> available in Java 9.
>>
>> What do you think of using Nashorn parser in JSDT?
>>
>> Thanks.
>>
>> -- 
>> Eugene Melekhov
>>
>> _______________________________________________
>> wtp-dev mailing list
>> wtp-dev@xxxxxxxxxxx
>> To change your delivery options, retrieve your password, or 
>> unsubscribe from this list, visit
>> https://dev.eclipse.org/mailman/listinfo/wtp-dev
> _______________________________________________
> wtp-dev mailing list
> wtp-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> https://dev.eclipse.org/mailman/listinfo/wtp-dev



-- 
Eugene Melekhov