Re: [cf-dev] Moving Blockwise support out of the stack

["Hudalla Kai (INST/ESY1)" <Kai.Hudalla@xxxxxxxxxxxx>]

On Fr, 2016-08-26 at 03:04 +0000, Joe Magerramov wrote:
> What I would love to see is ability to compose my own stack. That is
> have ability to plug my own layers into CoapStack. That would make
> the framework more easily extensible, and I think could even make it
> possible to implement TCP support without making changes to
> californium core.
> 
That would be awesome, indeed.

> Today, that is difficult because various layers depend on each other,
> and need to be present in correct order. For example, I ran into bugs
> with my TCP implementation because reliability layer was initializing
> some required parameters that the rest of the layers needed (TCP
> didn't need reliability layer). I also observe that a lot of these
> coupling are incidentals, rather than a required property of the
> protocol, so removing them is possible.
> 
Maybe we can start with identifying these "incidental" dependencies and
start getting rid of them in order to prepare the stack for becoming
more "modular" in the way you describe it, i.e. being able to assemble
a stack dynamically ...

> Joe
> On Thu, Aug 25, 2016 at 9:47 AM Hudalla Kai (INST/ESY1) <Kai.Hudalla@
> bosch-si.com> wrote:
> > Simon, 
> > 
> > On Do, 2016-08-25 at 15:11 +0200, Simon Bernard wrote:
> > > Hi,
> > >
> > >    I agree with the analysis.
> > >
> > >    This is a bit out of topic, but from my point of view, using 
> > > `Exchange` in californium for naming relation between one request
> > > and 
> > > severals responses is a bit confusing because the CoAP RFC seems
> > to 
> > > mainly use `Exchange` for Message (CON+ACK).
> > >
> > I think this is at the very heart of the problem and that is one of
> > the
> > reasons I would like to drive it back to this much simpler model.
> > 
> > >    Keeping the stack as simple as possible sounds a good plan,
> > but
> > > It's 
> > > hard to know if the "blockwise transfer" could be extracted from
> > the 
> > > stack alone ? e.g. I'm not sure we can extract blockwise without 
> > > extracting observation too ?
> > >    Maybe the stack should be limited as a "message exchange
> > stack" ?
> > > and 
> > > All the request/response matching, blockwise, observe will be
> > handle
> > > at 
> > > higher level.  (Are we creating another stack on the stack ?)
> > >
> > That is pretty much what I have in mind. However, I do not think
> > that
> > we would need to implement a "stack on top of the stack" but
> > instead a
> > set of helper classes for managing e.g. the blockwise transfer
> > process.
> > 
> > Currently, I do not share your concerns regarding the close
> > relation
> > between observe and blockwise but maybe that's just because of my
> > lack
> > of understanding of the corresponding layers' current
> > implementation.
> > From a conceptual point of view, I think that the "house keeping"
> > of
> > observation related state should also be separated from the
> > Exchange.
> > We already discussed this in depth when we started the 2.0.x branch
> > ...
> > 
> > >    In all case this seems to be a lot of changes ;).
> > >
> > > Simon
> > >
> > >
> > > Le 25/08/2016 à 10:16, Hudalla Kai (INST/ESY1) a écrit :
> > > >
> > > > Hi,
> > > >
> > > > during the last weeks we have run into several issues regarding
> > the
> > > > generation and usage of MIDs and tokens and all sorts of
> > > > synchronization issues.
> > > >
> > > > One thing that has always struck me when trying to locate the
> > > > source of
> > > > the problems was the fact that throughout the stack we are
> > checking
> > > > for
> > > > the presence of block options in requests and responses. To me
> > this
> > > > indicates that we have no clean separation of concerns anymore
> > when
> > > > it
> > > > comes to handling of blockwise transfers.
> > > >
> > > > My feeling is that the main reason for this is the fact that we
> > are
> > > > using the Exchange not only for keeping track of a single
> > request
> > > > and
> > > > its corresponding single response (based on their shared MID)
> > but
> > > > instead use it for arbitrary other sequences of message
> > exchanges
> > > > that
> > > > span multiple MIDs (and tokens in the case of blockwise
> > transfers).
> > > >
> > > > My understanding is that the original Exchange object was
> > probably
> > > > meant only to represent a single request/response exchange but
> > then
> > > > was
> > > > extended in order to also cover the "automagical" and (for
> > tclient
> > > > code) transparent transfer of large payloads using blockwise
> > "under
> > > > the
> > > > hood", i.e. a client can create a POST request with a large
> > payload
> > > > and
> > > > send it using the CoapClient which, under the hood, will break
> > it
> > > > down
> > > > to multiple separate POSTs unsing blockwise. The same applies
> > for
> > > > the
> > > > other direction, i.e. a a Response with a large payload
> > returned by
> > > > a
> > > > Resource for a request is transparently broken down into
> > smaller
> > > > chunks
> > > > and transferred via blockwise by Californium under the hood.
> > > >
> > > > While this may seem very convenient for a client or Resource at
> > > > first
> > > > sight, it actually makes implementation of the stack much more
> > > > complicated than necessary AFAIK. We need to keep a lot of
> > state
> > > > for
> > > > both managing the blockwise transfer itself and also allocate a
> > lot
> > > > of
> > > > memory for "buffering" the full payload to be sent to the peer
> > > > before
> > > > it gets chunked up to be transferred using blockwise. It also
> > > > introduces all sorts of "special treatment" in the different
> > layers
> > > > of
> > > > the stack which now all need to be aware of the fact that such
> > a
> > > > thing
> > > > as a blockwise transfer exists.
> > > >
> > > > Instead, I would rather like to move the blockwise handling out
> > of
> > > > the
> > > > stack itself and e.g. provide it as a set of helper classes
> > that
> > > > can be
> > > > used by client code (i.e. at the application level) to get
> > easier
> > > > access to the block options and make chunking and reassembling
> > of
> > > > the
> > > > payload easier. That way, IMHO, we could reduce the stack to
> > its
> > > > original purpose of simply transferring simple messages in
> > return
> > > > to
> > > > each other.
> > > >
> > > > Any thoughts?
> > > >
> > > _______________________________________________
> > > cf-dev mailing list
> > > cf-dev@xxxxxxxxxxx
> > > To change your delivery options, retrieve your password, or
> > > unsubscribe from this list, visit
> > > https://dev.eclipse.org/mailman/listinfo/cf-dev
> > _______________________________________________
> > cf-dev mailing list
> > cf-dev@xxxxxxxxxxx
> > To change your delivery options, retrieve your password, or
> > unsubscribe from this list, visit
> > https://dev.eclipse.org/mailman/listinfo/cf-dev
> _______________________________________________
> cf-dev mailing list
> cf-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or
> unsubscribe from this list, visit
> https://dev.eclipse.org/mailman/listinfo/cf-dev