Hi,
To start with, indeed, the ecosystem isn't that big anymore, so whatever we do we have to take this fact into account. Was it still 2010 with larger amounts of users and available resources, other choices might have been made.
For Mojarra we have multiple versions of tests
2. Integration tests using Arquillian
3. Internal tests using Cactus
4. JUnit tests using Mock types
I'm going to delete the Cactus tests. It's simply undoable to either convert or maintain them.
The cargo/script based tests are very easy to convert to Arquillian (copy/paste, add arquillian runner and deploy annotations), but even that of course takes work. Finally there's 921 unit tests, where everything is mocked.
So where does that leave us? The Arquillian tests of Mojarra and Myfaces could be joined, but we both don't have that many of them. They can be, more or less, instantly rubber stamped as TCK. This is essentially how the CDI, Batch, Validation, and all MP TCKs work. I can spend a few days picking a selection of tests from the cargo reservoir, which should give us a reasonable base. I'm not sure if we will reach 100.
As for the unit tests, we've been working on a servlet/EE runtime implementation called Piranha, that at its lowest level can be used like a unit testing framework. This feels just like using a servlet mock, except that the servlet implementation is the same one the higher level server actually uses. Maybe this is an option for the future to use.
For now though my gut feeling says that if we can agree on rubber stamping the existing Arquillian tests as TCK we're essentially there. There's some specs that don't have a TCK, but those are always trouble, so it's best to have at least a small TCK.
Kind regards,
Arjan