Re: [wtp-releng] Status of weekly I build

[David Carver <dcarver@xxxxxxxxxxxxxxxx>]

David M Williams wrote:
> Let me know if there's ways we can improve the predictability of our 
> builds .. we are always within a few days, but never by 3:00 on 
> Friday, like we say we will be.
From looking at the smoke test links here:

http://wiki.eclipse.org/WTP_Smoke_Test_Scenarios_R30

It looks like these are basically functional tests.  Making sure 
everything is working from a user stand point.   In the future we 
probably want to look at automating these tests where possible.   There 
are a couple of frameworks that can help with this.

The first is SWTBot, which is designed to help test the UI and has the 
ability to record and playback tests for the UI.

http://wiki.eclipse.org/SWTBot/UsersGuide

SWTBot has moved to eclipse, and is actively being developed and used.

The other being the TPTP framework also hosted at eclipse, and described 
in this Eclipse Corner article:

http://www.eclipse.org/articles/article.php?file=Article-TPTPAutomateFunctionalTesting/index.html

My recommendations though is the go with SWTBot once it's CQs have all 
cleared.   The main reason is that it is a very light weight testing 
framework, with little overhead.   It can be used within the current 
junit testing framework, so should be fairly easy to pick up.

The Smoke Tests should be run with every Milestone Integration build, 
not necessarily with every build.  However the more often they are run 
the better, so that errors can be caught sooner rather than later.

Moving away from the manual testing should help with the consistency of 
the builds and timeliness of getting them declared.

My two cents on the subject.

Dave