Re: [che-dev] new single-host progress update (+ some concerns)

[Michal Vala <mvala@xxxxxxxxxx>]

Hello there,

Here's a quick update where we got from our last message.

 - che-server Kubernetes part PR is code-complete and currently in
review [1]. I'd like to merge the PR in the beginning of next week.
   - I've been able to fix the single-namespace limitation
   - single-user limitation should be fixed in the deployment part
   - I've created an issue to reduce the number of ConfigMaps in che
namespace [2]. This will be the limitation in first versions.
   - Work on OpenShift part did not started yet, but it should be
straight-forward and I believe we'll make it into 7.19

 - Lukas is working on the che-operator part. We're optimistic to have
it merged in 7.19.

To summarize, we'd like to deliver the first working version of
Gateway based single-host in 7.19. Of course it is not the end. There
is still a lot of work ahead of us. We need to update Helm charts,
Chectl, and fix issues that we'll meet or introduce. I've already
reported 2 single-host issues [3][4] that are not related to the
gateway single-host, but are probably reproducible for some time
already and are probably more issues like that. Also, considering the
amount of code and complexity we're introducing with all this, the
risk of some issues is almost certain. We'd like to cooperate with the
QE team to catch the issues as soon as possible and make the new
Gateway single-host stable.

As a bonus, I've prepared for you a small demo, where I'm comparing
workspace startup times with Gateway and with Ingresses. I've used the
exact same workspace for all parts. It's an empty devfile with
ephemeral storage. All container images are pre-pulled and
ImagePullPolicy is set to Never. Important to say here, that time
saved will be roughly constant, NOT proportional to full workspace
startup time. Enjoy: https://youtu.be/Fhzt8DiBajU

Michal

[1] - https://github.com/eclipse/che/pull/17557
[2] - https://github.com/eclipse/che/issues/17669
[3] - https://github.com/eclipse/che/issues/17694
[4] - https://github.com/eclipse/che/issues/17693


On Mon, Aug 10, 2020 at 5:19 PM Michal Vala <mvala@xxxxxxxxxx> wrote:
>
> Hello all,
>
> we're working hard on a new Gateway based single-host, using Traefik
> as the central component to do all the routing and only single
> Ingress/Route as an entrypoint to the whole Che including all the
> Workspaces.
>
> Some of you saw the demo already https://youtu.be/s3rXi-QstJw (no sound, sorry).
>
> So where are we now?
>
>  - we've released the Configbump tool which helps us to get routing
> configurations dynamically to Traefik pod
> https://github.com/che-incubator/configbump
>  - we did necessary refactorings of che-server so we can export
> services in more generic way + prepare che-server configuration
> https://github.com/eclipse/che/pull/17430
>  - we've started working on che-operator part to support new
> deployment with the traefik gateway
> https://github.com/eclipse/che-operator/pull/378
>  - we've started working on che-server part to actually configure
> routing with the Traefik https://github.com/eclipse/che/pull/17557
>
> I'd like to write more about the last point, the che-server part. The
> demo shows first working prototype implementation, but it has several
> limitations.
>
>  - It works only in a single namespace
>    - Currently, we're creating all workspace objects (like
> Deployments, ConfigMaps, Services, Ingresses, ...) in a single
> namespace, where workspace belongs, and beside some database entries,
> it lives only in that one namespace. For new single-host, we have to
> configure the Traefik, which lives in Che namespace. That means that
> we have to create ConfigMaps in Che namespace that will actually
> belong to the workspace. There are several issues with this:
>      - It breaks down this nice relationship where all workspace fits
> in a single namespace. It's now harder to manage them (create,
> cleanup).
>      - It creates an issue with permissions, where we're now using the
> workspace's serviceaccount/user to create these objects. For new
> single-host, to create objects in Che namespace, we would need
> permisisons to do that, so we would need to use "che" serviceaccount
> just to create this part of workspace. I think there is no mechanism
> to do something like this in current che-server implementation and it
> might get a bit complicated with OpenShift OAuth.
>
>  - It works only on single-user
>    - I was yet not able to configure Keycloak to run on subpath like
> `https://<che-host>/keycloak`. At some point, keycloak ignores the
> `/keycloak` part and uses it's own absolute urls. It's probably just a
> matter of configuration. I wasn't trying hard yet. (BTW does current
> Ingress based single-host even work on kubernes with multi-user? I
> didn't try yet.)
>
>  - We've basically replaced the Ingresses/Routes with ConfigMaps. That
> means that for each exposed server, we need to create 1 ConfigMap.
> That means, that for a clean workspace, we need to create 7 ConfigMaps
> (4 theia + 3 jwtproxy). All that will live in Che namespace, so that
> place will become quite crowded with many workspaces. We would like to
> get this down to 1 ConfigMap per Workspace, but it's definitely
> something we would like to keep for later.
>
>
> So, this is where we are. We have interesting technical problems in
> front of us. I have few ideas how to solve some of these and some idea
> how we could proceed to bring this big feature in meaningful
> incremental steps. I'd like to discuss it inside the team first. I'll
> keep you informed.
>
> If you have any thoughts or ideas, please share with us.
>
> Thanks
>
> --
> Michal Vala
> Software Engineer, Eclipse Che
> Red Hat Czech



--
Michal Vala
Software Engineer, Eclipse Che
Red Hat Czech