Re: [jdt-dev] Performance degradation switching from 1.8 to 11

[Andrey Loskutov <loskutov@xxxxxx>]

All of the org.eclipse.jdt.internal.compiler.util classes are *memory* optimized, they were all written before Java 1.2 / modern non-synchronized Collections API appeared and are basically a various copies of SimpleLookupTable. Also HashtableOfArrayToObject that is responsible for the performance of org.eclipse.jdt.internal.core.NameLookup.isPackage(String[]) is another clone of this table. You can find overall in JDT clones of the original code - some with later fixes, some not.

In all our latest performance related bugs in this area we saw that the replacing use of these ancient jdt *Table structures with default HashMap/HahSet increases throughput but costs some more memory. Especially worrying is the worst case behavior of those tables/sets, they do not scale from some 10.000+ of entries anymore, and they rehash() function that is invoked if the arrays go out of free space is extremely inefficient and may block for minutes (!) on big input on fastest workstations.

So there is a big room for an improvement, but also a big chance to break JDT (I believe we did it every time we touched this code, there are very subtle implementation details that you only understand after you realized that you broke the build).

See https://bugs.eclipse.org/bugs/show_bug.cgi?id=563030#c9 for one example (10% memory trade off for > 1000 % speedup in some worst cases for huge tables).

See https://bugs.eclipse.org/bugs/show_bug.cgi?id=529984 discussion for some profiler results.

See https://bugs.eclipse.org/bugs/show_bug.cgi?id=564092 as one of many places where the current code could be changed.

Kind regards,
Andrey Loskutov

Спасение утопающих - дело рук самих утопающих

https://www.eclipse.org/user/aloskutov


> Gesendet: Donnerstag, 24. September 2020 um 18:35 Uhr
> Von: "Gunnar Wagenknecht" <gunnar@xxxxxxxxxxxxxxx>
> An: "Eclipse JDT general developers list." <jdt-dev@xxxxxxxxxxx>
> Betreff: Re: [jdt-dev] Performance degradation switching from 1.8 to 11
>
> Hi All,
> 
> I'm still investigating this one. I've attached two profiling samples of the same Code Assist operation to the bug. Based on this I have a couple of questions/ideas and looking for feedback whether it makes sense to continue/experimenting with something or not.
> 
> Obviously the challenge here is creating a public, reproducible data set. Hence my call for ideas of things I can/should try in the code base. Any ideas are welcome.
> 
> 
> LookupEnvironment.java:1620 org.eclipse.jdt.internal.compiler.util.HashtableOfPackage.get(char[]) 
> --> takes 1 second
> 
> Does this smell like potential hash collisions?
> 
> There are more similar classes in the util package. Are those performance optimized? Would it make sense to start looking into replacing some of these with HashMaps?
> I was thinking this because of https://openjdk.java.net/jeps/180, which looks promising. 
> 
> I'm looking for someone with triable knowledge in this area. 
> 
> 
> SearchableEnvironment.java:922 org.eclipse.jdt.internal.core.NameLookup.isPackage(String[], IPackageFragmentRoot[])
> --> takes 1 second in String.join
> 
> This is quite surprising to me. It shows up two times in a single sample and contributing two seconds here to the execution time. 
> 
> 
> Any thoughts?
> 
> -Gunnar
> 
> -- 
> Gunnar Wagenknecht
> gunnar@xxxxxxxxxxxxxxx, http://guw.io/
> 
> 
> > On Aug 29, 2020, at 11:03, Gunnar Wagenknecht <gunnar@xxxxxxxxxxxxxxx> wrote:
> > 
> > Greetings JDT Developers,
> > 
> > We are seeing a severe performance degradation with Eclipse switching a large code base from 1.8 to 11. It's noticeable across our entire Eclipse use base. Increasing Eclipse JVM heap to 32GB on desktops with >= 64GB memory does not improve the situation. 
> > 
> > We are currently running Eclipse with OpenJDK 1.8.0_212 and the following additional options:
> > -Xverify:none
> > -XX:+UseStringDeduplication
> > -XX:+DisableExplicitGC
> > -XX:+ParallelRefProcEnabled
> > -Dorg.eclipse.jdt.core.javamodelcache.ratio=1.5
> > -Dorg.eclipse.jdt.core.javamodelcache.jartyperatio=4
> > 
> > I've created https://bugs.eclipse.org/566498 for sharing logs and profiling results. I'd appreciate your input on thing to try and/or data you would like us to collect.
> > 
> > Please let me know if you prefer communication on the bug of on the mailing list.
> > 
> > Thanks a lot!
> > 
> > -Gunnar
> > 
> > -- 
> > Gunnar Wagenknecht
> > gunnar@xxxxxxxxxxxxxxx, http://guw.io/
> > 
> > 
> 
> _______________________________________________
> jdt-dev mailing list
> jdt-dev@xxxxxxxxxxx
> To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jdt-dev
>