| [jdt-dev] Almost there! But caution! |
Dear team,When looking at the test results of releases 4.8 - 4.10, JDT was accumulating a lot of technical debt = failures, some intermittent, some systematic.
As of I20190114-1800 results look much better, as the attached picture demonstrates, the remaining failures being:
1. ClassNameTests.testBug152841 I can't recall seeing this ever before, only one platform 2. TypeHierarchyNotificationTests.testAddAnonymousInRegion intermittent (infrequent) 3. LeakTestSuite complete failure (15) on MacOSI personally wouldn't pay much attention to (1), (2) is in one group with a few others that happened more than once but too rarely to badly worry about, IMHO.
With (3) Kalyan has a quite hard nut to crack. Let's wish him luck (or help him).Other than that, I think the current state is good enough for a little toast to the team and particularly to Andrey(!), who has invested a lot recently to get closer to the big zero.
Cheers!One reason for sending this mail: encourage *every one* in the team to try to keep this status, by monitoring build results below
https://download.eclipse.org/eclipse/downloads/and by reacting to any failures. Just watching your own gerrit job doesn't seem to suffice when many bugs only occur on specific machines / os / environment.
(Did Sasi provide his build-observing App to anybody?) Oh, what should actually be the compliance level of JDT/Core gerrit tests?Currently it's 10, but perhaps it should already be 11? What pace of update should the job have these days? Can we make this a part of the update-to-next-Java-version routine?
But, the next wave of technical debt could soon be upon us: our BETA gerrit job at https://ci.eclipse.org/jdt/job/eclipse.jdt.core-Gerrit-BETA/ isn't really in a very good shape with constantly 59 failures! Those have been captured in bug
https://bugs.eclipse.org/540704 & children. From these, one bug specifically worries me:https://bugs.eclipse.org/540922 - [12] ct.sym file has been restructured again for older releases
My latest comments are meant to indicate a severe bug in the JEP 247 process: we need to implement the "-release" option but the necessary information is only available in undocumented and changing formats. At the same time people expect JDT to compile against *future* JREs. This set of requirements is impossible to fulfill. Please join me in discussing the options we have (if any) in the bug.
Please also help to eliminate the other failures in BETA_JAVA_12. It is so much easier and nicer to fix individual failures when they occur rather than letting them pile up until we are overwhelmed by the mess. :)
At this point I hardly dare mentioning the back-to-back tests with javac: https://ci.eclipse.org/jdt/job/eclipse.jdt.core-run.javac-1.8/ https://ci.eclipse.org/jdt/job/eclipse.jdt.core-run.javac-10/which linger at > 200 differences between compilers, but I am monitoring those and didn't see many regressions over time, good.
The last job: https://ci.eclipse.org/jdt/job/eclipse.jdt.core-run.javac-11 doesn't even run regularly, I'm re-checking its status right now (recently it showed some nice crashes of javac ;p ).Sorry, if this mail sounds a bit negative. Generally I'm very happy to see the recent progress.
best, Stephan
Attachment:
test-results-I20190114-1800.png
Description: PNG image
