[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [ecf-dev] Efficient downloads
|
Hi Scott,
Thanks for a very elaborate answer. This looks very promising. Well
structured, low overhead. Only thing missing if I read you correctly is
true pause/resume on protocol level (and the failure recovery that I
think will come as a bonus). Do you have any sense what it would take to
implement that?
- thomas
Scott Lewis wrote:
Hi Thomas,
Thomas Hallgren wrote:
I'm looking for ways to improve Buckminsters download capabilities
and I have some questions regarding the ECF filetransfer provider. As
you might tell by my questions, I'm fairly new to the whole thing :-)
What is the current status for implementations (http, ftp, and
bittorrent in particular)?
We use apache httpclient 3.0.1 for implementing http/https support,
the jre URLConnection impl for ftp, and we have a working EPL
bittorrent implementation (created by committer Remy Suen as part of
the Google SOC project 2006). The URLConnection impl also supports
file and jar URLs (of course). These are all for file retrieval of
course...I assume this is the use case you are interested in (rather
than peer-to-peer file sending which the ECF file transfer API also
supports). The top-level container adapter for file retrieval is:
IRetrieveFileTransferContainerAdapter[1].
Is it possible to resume a download from a certain position (e.g.
using the http Range header attribute)?
We have an adapter interface called IFileTransferPausable, which
exposes semantics for pausing/resuming a file transfer. This is
currently implemented (with the protocols listed above except for
bittorrent) via the Job.sleep() and Job.wakeUp() methods. It doesn't
have to be implemented this way for a given protocol (i.e. it could
use protocol specific mechanisms where available), but we haven't done
that yet.
Does ECF provide any resume or recovery after failure capabilities
out-of-the-box?
Resume, yes via IFileTransferPausable adapter. The adapter is
optionally implemented by a given provider implementation.
Recovery from failure...no.
Do you have any abstractions for this?
How much overhead does ECF add in terms of size in a headless
environment assuming an eclipse runtime is active already?
It depends upon which ECF plugins are used. If you are using just the
file transfer api and impls then the filetransfer API bundle (jar) is
currently ~25k and it depends upon the ECF core API (61k), and the
ECF identity API (39k). For file transfer, these are small by
comparison with the size of the apache httpclient implementation
bundle (from Orbit...~290kb).
Just FYI, we went to some significant trouble to reduce the
dependencies of the ECF core APIs to the bare minimum and make them as
small as possible, meaning that for all of these (filetransfer API,
core API, identity API) they require only the CDC 1.0/Foundation 1.0
runtime + the equinox jobs bundle.
Unfortunately, I don't yet have detailed numbers of memory usage by
bundle...or for the runtime behavior of httpclient under various load
conditions separate from other analyses that people have done of
httpclient (I haven't searched to find these, but I imagine they do
exist...if you find them please make them known to us).
Also...FYI, here's some new subsystem-level docs on ECF that might
prove useful:
http://wiki.eclipse.org/index.php/ECF_API_Docs
There's also some pointers on those pages to test code for filetransfer.
Let us know if we can be of more help.
Scott
[1]
http://www.eclipse.org/ecf/org.eclipse.ecf.docs/api/org/eclipse/ecf/filetransfer/IRetrieveFileTransferContainerAdapter.html
Regards,
Thomas Hallgren
_______________________________________________
ecf-dev mailing list
ecf-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/ecf-dev
_______________________________________________
ecf-dev mailing list
ecf-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/ecf-dev
