[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
Re: [cdt-dev] Improved reliability of Index tests
|
Hi Marc-Andre
The effect of failing tests is completely random. The correlation I see
is, that the more the indexer is used, the more it happens and also that
if it runs on slower pc's, the failure go away (on our one-core
vm-build-server test do never fail, on my quad-core pc they do a lot).
Here at the HSR, I'm also seeing many others with the same problems
(lots of student projects, mostly cdt-refactoring stuff often heavily
making use of the indexer) at least until I show them what you see now
in my gerrit patch.
Basically any test involving the indexer/bindings fails sometimes. There
is no real pattern its just completely random. To me its pretty obvious
that cdt-indexing has a synchronization issue.
Failing happens not in waitForIndexer or waitUntilFileIsIndexed but
rather while asserting results produced by the indexer which are just
wrong. Meaning I see stuff like "expected:<2> but was:<1>" meaning the
indexer did not yield 2 but wrongly only one IIndexName.
I can understand your theory about the doing some random stuff might
make enough delay to help. But since test execution (i did many runs of
the full cdt-core-tests suite and tonns of 100s of executions of
IndexBugsTests) do not take longer (difference is marginal) I do not
think that your theory is right. Also, the removing of Thread.sleep
calls in a loop speaks for my solution.
Calling the refresh alone does not improve the situation as it does with
the workspace builds.
File contents in Eclipse get cached. Since these caches might only be
updated with resource changes I can see a good chance that rebuilding
effectively enforces some update of contents which will then get into
the indexer.
Here some stats (average over several 650 IndexBugsTest runs):
* Original: Failures 6-7, Time 72-81sec
* With -Dindexer.timeout=60: Failures 5-7, Time 75-81sec
* My solution: Failures 0-0, Time 76-80sec
Can you reproduce anything like this on your computer?
Basically, I also must say if adding/increasing some timeout somewhere
helps, then the solution is broken. To fix such issues, some
synchronisation/joining is missing. And thats the point where a fix
should come from. I hope that you see it that way too!
Lukas
On 31.03.2014 16:42, Marc-André Laperle wrote:
Hi Lukas,
I've seen tests fail before because of waitForIndexer timing out. Which tests are failing for you? Do they fail with an asserting failed in waitForIndexer or waitUntilFileIsIndexed? I think the reason why adding the workspace.build calls might be helping is that it adds an additional delay and gives the indexer a better chance to finish. But this is just a theory. I didn't think the index tests depended on the build and if they do, I'd be curious to know why. Perhaps it's only the refresh that's necessary? Then I wonder why that would be the case because I believe all the files are created in the workspace through the eclipse.resources APIs.
Could you try increasing the indexer timeout and see if that also improves for you? You can add -Dindexer.timeout=60 in your VM arguments (that's if you launch from inside Eclipse, for maven you have to add it in the pom.xml files).
Marc-Andre
________________________________________
From: cdt-dev-bounces@xxxxxxxxxxx [cdt-dev-bounces@xxxxxxxxxxx] on behalf of Lukas Felber [lfelber@xxxxxx]
Sent: Monday, 31 March 2014 8:08 AM
To: CDT General developers list.
Subject: Re: [cdt-dev] Improved reliability of Index tests
Hi Marc-André
Please see https://git.eclipse.org/r/#/c/24080/ again. It is now rebased
to master. The new build even got +1 from hudson.
I actually do not exactly know at what point the
workspace.build(IncrementalProjectBuilder.FULL_BUILD) method helps the
cdt index tests. It seems to refresh resources and hence get rid of
outdated cache stuff. This then makes the index more up to date than
without rebuilding.
Lukas
On 28.03.2014 17:59, Marc-André Laperle wrote:
Hi Lukas,
Can you explain what is the purpose of the build? Also, if you rebase
your patch, I made changed to CDescriptorTests and codan so that they
run more reliably.
Marc-André
On 14-03-28 12:58 PM, Lukas Felber wrote:
Hello Everyone
I created a small patch ( https://git.eclipse.org/r/24080 ) that
improves the reliability of index tests.
When running the 65 IndexBugsTests 50 times (total 3250 test runs)
from a test suite, I got 90 to 100 random test failures.
After my adaptions, failures are reduced to 0-5 failures (sadly not to
0). Meaning a failure reduction of ~95%. When running all core tests,
I never saw an index test fail (in 10+ runs). Those CDescriptorTests
do still fail (I did not touch these).
The adaptions do not have any impact on run-time (+/- 3%). And I was
able to remove the very ugly part that called Thread.sleep() in a loop.
The better result is achieved by having two additional calls to
workspace.build(IncrementalProjectBuilder.FULL_BUILD) which basically
is similar to a "clean..." call from the "Project" eclipse ui-menu.
I hope that some of you guys can try the patch and check if it does
not have any negative effects somewhere (I only checked the cdt-core
and codan tests).
Cheers
Lukas
PS: I will most likely not be active in CDT development in the near
future since I'm leaving HSR and moving on to IBM.
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev