|
Hi, Jay
I attach sample scanner for external jars from classpath.
Method ClasspathScanner.buildJars(String[] externalJars,
IJavaProject javaProject) iterates jars and all IType objects in
them. It checks IType.isInterface() and then does some specific
work for interfaces and other types. After IType.isInterface() is
once called, then yes, the Java model caches element info for the
type and all subsequent queries go fast. But it looks like at the
moment of this very first call there is no cached element info
yet, and Java model builds it by reading jar entry. Typical stack
trace is given in attached Stacktrace.txt - it starts at my
ClasspathScanner.buildJars goes through
JavaModelManager.getZipFile and reaches
java.util.zip.ZipFile.open. So, for every type new ZipFile object
is created and opened.
Method ClasspathScanner.buildJarsImproved(String[] externalJars,
IJavaProject javaProject) wraps the call to buildJars() with
JavaModelManager.cacheZipFiles() and
JavaModelManager.flushZipFiles() (I took this pattern from
org.eclipse.jdt.internal.core.hierarchy.IndexBasedHierarchyBuilder.build(boolean)).
In this case ZipFile is created only once for a jar and scan goes
100 times faster for large jars.
Yes, if some other code had scanned a jar before my scanner, I
would not need wrapper method buildJarsImproved(), but it seems
that my builder going after Java builder finds jars unscanned yet
by another party.
I see that I might start my scan with a call to org.eclipse.jdt.core.JavaCore.newTypeHierarchy(IRegion,
>> WorkingCopyOwner, IProgressMonitor) thus
forcing initialization of Java model for jars to be scanned. But
that does not look nice, to run intentionally some big unnecessary
job to use its side effect as a condition of good performance of
my own code. I would prefer an API method that would wrap my scan
as I do it in buildJarsImproved().
Best regards,
Slava Kabanovich
On 09/09/2015 10:11 PM, Jayaprakash Arthanareeswaran wrote:
I must admit I don't
understand your requirement fully, but let me state that once
the Java model (such as BinaryType etc.) is constructed, these
are cached by the
Java model manager. May be
there is a way you can do what you are doing by using the
cached java model?
Regards,
Jay
 Slava Kabanovich ---09/09/2015
11:16:02 AM---Hi, Stephan I have found the solution to my
problem by studying why
From: Slava Kabanovich
<vkabanovich@xxxxxxxxxx>
To: "Eclipse JDT general developers
list." <jdt-dev@xxxxxxxxxxx>
Date: 09/09/2015 11:16 AM
Subject: Re: [jdt-dev] Performance issues
using IType interface
Sent by: jdt-dev-bounces@xxxxxxxxxxx
Hi, Stephan
I have found the solution to my problem by studying why
IndexBasedHierarchyBuilder.build(boolean) works fast. It turns
out that
when I iterate over types in a jar, the first call to IType
(e.g.
IType.isInterface()) results in creating new ZipFile object
for each
type. That can be easily prevented by
JavaModelManager manager =
JavaModelManager.getJavaModelManager();
try {
manager.cacheZipFiles(this);
//All build over jars goes here
} finally {
manager.flushZipFiles(this);
}
JDT works fine and fast. Caching ZipFile objects makes all the
difference.
The only concern remaining now is that JavaModelManager is not
a part of
API. May I request to consider providing API methods for doing
similar
searches?
Thank you.
Best regards,
Slava Kabanovich
On 06/06/2015 08:32 PM, Slava Kabanovich wrote:
> Hi, Stephan
>
> Thank you for the reply.
>
>
> > What requests exactly to you make?
>
> For instance, the very first call IType.isInterface()
takes too much
> time. If after that I call any other methods of IType
they perform
> much faster. That is probably because creating element
info takes much
> time as it probably prepares data for all requests to
IType.
> Suppose, I need to look into all classes, but i know that
I do not
> need interfaces and abstract classes, only concrete
classes. Then, I
> try to be smart and get done with interfaces by a single
check
> IType.isInterface(). I expect it to be much faster than
getting all I
> need for a concrete class. But it does not work as
expected. As I have
> said, (the first) call to isInterface() takes almost as
long as
> everything else I need to get from a concrete class.
>
>
> > Not sure what you mean by Java build, since
org.eclipse.jdt.core
> does implement building Java programs.
>
> I may be mistaken, is not Java building implemented with
packages
> org.eclipse.jdt.internal.core.jdom and
*.internal.compiler.* packages?
> With 'org.eclipse.jdt.core' I meant not plugin (yes, the
plugin
> implements Java build), but package that includes IType
and other
> handles for querying Java model. When Java builder
completes, element
> info for IType is not yet created. Then I conclude that
Java builder
> does not need requesting anything from IType and uses the
internal
> model that is more efficient.
>
>
> > Also, I'm not sure I understand what exactly your
tool is doing?
>
> It is ok with incremental building, certainly I do not
need looking
> into a jar at every build, but when a project is imported
into
> workspace, the full build is performed for it, and Batch
tool needs to
> determine which jars are Batch archives. Unfortunately,
there is no
> required resource like batch.xml (it may be present, but
is ok to be
> missing). So I do not see another way than scanning, just
once, the
> jar in search for Batch artifacts. If none found, I mark
the jar as
> 'not a Batch archive', and it is ignored after that.
> The same with CDI 1.1. While in CDI 1.0 archives resource
beans.xml
> was required, it is not so since 1.1.
> Once more, the performance issue is critical for the case
of importing
> into workspace a large set of projects with many
dependencies. It may
> be critical also at workspace refresh if dependencies are
changed
> (e.g. versions of all jars are changed).
>
> >Have you considered, instead of iterating lot's of
potentially
> irrelevant elements,
> >to perform a search (e.g., for all types with a given
annotation)?
>
> In CDI 1.1+, there are many bean defining annotations and
most
> confusing is that they include custom (defined by
application) scopes
> and stereotypes, which the tool may either collect as the
result of
> build, or by checking annotations found at the type
during iteration.
>
>
> >Or just use a subtype hierarchy of a well-known
interface?
>
> At present, in Mars M7, when I start 'Open Type
Hierarchy' for IType,
> it takes about 1 minute to show in the view. In Luna it
took only a
> few seconds.
>
>
> >Have you seen the following API?
>
>org.eclipse.jdt.core.JavaCore.newTypeHierarchy(IRegion,
> WorkingCopyOwner, IProgressMonitor)
>
> Thank you, I will try it to see if it may give gain in
performance.
>
>
> Best regards,
> Slava Kabanovich
>
> On 06/06/2015 05:39 AM, Stephan Herrmann wrote:
>> Hi Slava,
>>
>> A few points in your description are not clear to me,
so it's
>> difficult to advise.
>>
>>
>> Questions:
>>
>> > At present this is implemented with requests to
IType interface.
>>
>> What requests exactly to you make?
>> Creating an IType handle has virtually no cost.
>> Some requests require to read and analyze the
underlying compilation
>> unit or class file etc.
>>
>> > makes one to assume that Java build uses
internally more efficient
>> model than org.eclipse.jdt.core.
>>
>> Not sure what you mean by Java build, since
org.eclipse.jdt.core does
>> implement building Java programs.
>>
>>
>> Also, I'm not sure I understand what exactly your
tool is doing? Why
>> do you have to inspect all
>> entries of all jars in the classpaths? When speaking
of incremental
>> buildings I'd assume
>> inspecting all compilation units in (affected
projects in) the
>> workspace should be enough.
>> Why isn't it?
>>
>> Have you considered, instead of iterating lot's of
potentially
>> irrelevant elements,
>> to perform a search (e.g., for all types with a given
annotation)? Or
>> just use a
>> subtype hierarchy of a well-known interface?
>> If a lot of hierarchy lookup is needed, perhaps doing
this in one big
>> bulk instead of
>> thousands of individual invocations might also speed
up things.
>> Have you seen the following API?
>>
>>
org.eclipse.jdt.core.JavaCore.newTypeHierarchy(IRegion,
>> WorkingCopyOwner, IProgressMonitor)
>>
>>
>> best,
>> Stephan
>>
>>
>> On 05/27/2015 02:38 AM, Slava Kabanovich wrote:
>>> Hello,
>>>
>>> I work on Batch and CDI models for JBossTools in
Eclipse.
>>> Incremental builders for the models look for
types that either
>>> implement
>>> some interfaces (Batch), or have bean defining
annotations (CDI). At
>>> present this is implemented with requests to
IType interface.
>>> In a large workspace, with a lot of jars in
classpath, at first
>>> build (or at a change in class path) jar entries
have to be checked
>>> for being Batch or CDI archive, there is no
simple way to do that
>>> (in CDI since 1.1, CDI 1.0 had required beans.xml
that made life
>>> much easier) other than checking all types in
jar. Tests show that
>>> in average about 2000 IType objects may be
checked in one second,
>>> which results in unacceptably long build time for
large workspaces.
>>> Batch and CDI build take 2 to 3 times longer than
Java build. It
>>> makes one to assume that Java build uses
internally more efficient
>>> model than org.eclipse.jdt.core.
>>>
>>> Could it be possible to get faster methods for
checking inheritance
>>> and annotations? It may be still ok to work with
IType interface
>>> for objects verified as Batch artifacts or CDI
beans/annotations,
>>> but it is of great importance to be able to
quickly rule out
>>> irrelevant types. Could you please advise me
already existing ways
>>> to solve this problem, or consider implementing
in JDT new high
>>> performance requests?
>>>
>>> Thank you.
>>>
>>> Best regards,
>>> Viacheslav Kabanovich
>>> _______________________________________________
>>> jdt-dev mailing list
>>> jdt-dev@xxxxxxxxxxx
>>> To change your delivery options, retrieve your
password, or
>>> unsubscribe from this list, visit
>>> https://dev.eclipse.org/mailman/listinfo/jdt-dev
>>
>> _______________________________________________
>> jdt-dev mailing list
>> jdt-dev@xxxxxxxxxxx
>> To change your delivery options, retrieve your
password, or
>> unsubscribe from this list, visit
>> https://dev.eclipse.org/mailman/listinfo/jdt-dev
>
_______________________________________________
jdt-dev mailing list
jdt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jdt-dev
_______________________________________________
jdt-dev mailing list
jdt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jdt-dev
|
