Well technically JAXRS is always slower than servlet - in particular when implemented on top of servlet as required by the spec - until you make it 1-1 (a method called directly handling the input/output streams) where it is just about proposing different programming model but iso in terms of perf.
I think that servlet by itself is not bad and not that fat but dropping the xml requirement can be a big thing (enhancing initializers to have @Priority and other small details like that is sufficient).
Dropping some inter-dependencies like jaspic, jaas etc can be neat but think most vendors did their homework to enable it so not sure how much it would bring at spec level.
The other big thing I would hope can be to enable to embed the container from an application instead of deploying into the container (servlet-se API, a bit similar to EJBContainer or SeContainer of CDI. JAXRS and JAXWS had such features, not sure why servlet can't and it would help in all the microservices we write these days).
So to summarize:
1. Same API as today (compat is key)
2. Embeddable (SE "profile")
3. XML free (initializers enhancements + probably no xml in the "light" spec jar)
