Re: [Dltk-dev] ScriptBuilder full build too slow because it reads all IR

[Gabriel Petrovay <gabipetrovay@xxxxxxxxx>]

Hi Alex,

First, sorry for duplicate! (After I posted it to the list I saw the
lack of activity and I though that you moved it or have other policies
now. So I decided to post it on the forum as well). I an at least very
happy that I received an answer so quickly. Thanks!

Back to the problem. Thanks for the wonderful trick ;). The 10k files
were files without extension. So my language toolkit were considering
them as candidate in
DLTKcontentTypeManager:isValidResourceForContentType

So, it works only by overwriting:
boolean canValidateContent(IResource resource);

and not allowing the processing of files without an extension. (Not
necessary the complete solution, but in this case it works). It is
also fast enough.


Thanks!


On Mon, Jul 12, 2010 at 2:32 PM, Alex Panchenko <alex@xxxxxxxxx> wrote:
> Hi Gabriel,
>
> I am not sure it makes sense to report issue in multiple places, as the discussion might become fragmented. Or some posts will have no answers.
> If you want to bring the attention to the bug posted in bugzilla - just post the link here.
>
> Now talking about the actual issue with 10k files.
> Have you tried to profile your application to understand what particular methods takes most of the time?
>
> If you think it scans the content of all files you can override the following methods in your org.eclipse.dltk.core.IDLTKLanguageToolkit
>
> boolean canValidateContent(IResource resource);
> boolean canValidateContent(File file);
> boolean canValidateContent(IFileHandle file);
>
> Could you please let us know if overriding these methods helps and if it isn't then it would be great to have the profiling results.
>
> Thank you,
> Alex
>
> ----- Original Message -----
> From: "Gabriel Petrovay" <gabipetrovay@xxxxxxxxx>
> To: "dltk-dev" <dltk-dev@xxxxxxxxxxx>
> Sent: Monday, July 12, 2010 6:35:39 PM GMT +06:00 Almaty, Novosibirsk
> Subject: [Dltk-dev] ScriptBuilder full build too slow because it reads all IReources in a project
>
> Hi,
>
> During a full build the ScriptBuilder will read all the resources in a
> project: Scriptuilder:getResourcesFrom(...). The ResourceVisitor
> visits all the resources in a given project. For a project with 10000
> files in it, out of hich only one is a ScriptModule, all the resources
> are scanned and DLTKCore.create(res) is called for each resource.
>
> This project takes for example about 1.5 hours on my machine. Most of
> the user actions are blocked by the "Building project" job.
>
> Is there a way to filter the resources that need to be processed. In
> my example I know that one folder contains 9990 resources that are not
> SourceModules so I can ignore that folder.
>
>
> Thanks!
>
>
> --
> MSc Gabriel Petrovay
> Mobile: +41(0)787978034
> www.28msec.com
> _______________________________________________
> dltk-dev mailing list
> dltk-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/dltk-dev
> _______________________________________________
> dltk-dev mailing list
> dltk-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/dltk-dev
>



-- 
MSc Gabriel Petrovay
Mobile: +41(0)787978034
www.28msec.com