[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
RE: [cdt-dev] First gotcha with add/exclude children of FFS
|
Title: Re: [cdt-dev] First gotcha with add/exclude children of FFS
Copying the platform-core-dev folks too. Is there someone
from the Platform who could attend a meeting at EclipseCon about flexible file
systems. John A, will you be there? John is Mr. EFS and we can use his
guidance.
Thanks,
Doug.
Doug,
Can we get together at EclipseCon to
discuss this issue specifically? Do you know the right platform people to invite
to the meeting? We really need to piece together a plan.
Im sure you
have enough to do so if you can tell me who to recruit I can help organize the
meeting.
Thanks - Ken
From: "ext Schaefer, Doug"
<Doug.Schaefer@xxxxxxxxxxxxx>
Reply-To: "CDT General developers
list." <cdt-dev@xxxxxxxxxxx>
Date: Wed, 20 Feb 2008 18:44:57
-0800
To: "CDT General developers list."
<cdt-dev@xxxxxxxxxxx>
Subject: RE: [cdt-dev] First gotcha with
add/exclude children of FFS
So, you know. The
more I think about what you guys are saying, I'm realizing that the EFS solution
probably is the right one. The objective should be to turn the IResource tree
into a logical project view and to remove all notions that it represents
physical file layout. That unfortunately starts with the .project and .cproject
files, but I think there are tricks we can do there. The .settings may be harder
but let me sleep on that.
At any rate, this has piqued my interest again and I'll work on
reviving it and see where it goes. I'll try to get a prototype working by
EclipseCon so we can talk about it more concretely.
Cheers,
Doug
From:
cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx]
On Behalf Of Brunauer, Walter
Sent: Monday, February 18, 2008
8:53 AM
To: CDT General developers list.
Subject: RE:
[cdt-dev] First gotcha with add/exclude children of FFS
Hi
Warren,
well, the confusion my origin from the different meanings of
what project setup is: for me, project setup is not equal to build setup. I.e.,
on our projects, the build setup is an independent step from the project setup.
We intentionally separated this to overcome all kind of issues you obviously
experienced as well. And this is how we see it:
1.
Create a project at the desired location (everything beneath this root location
is part of the project, but it can be an empty project just as well with linked
resources added to it later). By default, the build setup is identical to the
project content (there is one build target, linking/archiving everything
together).
2.
If (a) specific build setup(s) are needed, it is possible to specify as many
build targets with arbitrary contents as desired. This approach separates the
physical file system layout from logical build layouts, and it even works beyond
project boundaries. IOW, no matter from where source files are pulled in (the
same projects, nested projects, outer projects, sibling projects), one is able
to specify any build setup exactly are needed, as long as all source files are
known to Eclipse (as resources).
HTH,
Walter
From:
cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx]
On Behalf Of Warren.Paul@xxxxxxxxx
Sent: Freitag, 15.
Februar 2008 14:51
To: cdt-dev@xxxxxxxxxxx
Subject:
RE: [cdt-dev] First gotcha with add/exclude children of
FFS
Hi Walter,
I forgot all about the absolute paths issue
with linked resources. I'll update the
wiki.
I'm a bit confused about your comment about
this not being a project setup issue. We have our own builder as
well, and it will happily build whatever the build description says,
whether those files are under the project root or not. We even
have our own project explorer view which shows a logical representation
of the project rather than the physical file system layout. But we
still run into a lot of issues when files are not under the project root
- that is, when you can't get an IFile for them.
We have a wide range of user types from small
application developers to large system developers. In many cases,
a users code base consists of hundreds of directories with thousands of
source files. In such a source base, there are many hundreds of
build artifacts and almost as many "logical projects". It is a
huge problem for these users to be able to create projects currently.
They will typically have a few projects going at a time, but many
times the natural project root for all of them will be the same.
We've found ways to work around some of the other issues, but not
this one. It sounds like perhaps you guys have found a way.
Could you elaborate on how you setup your
projects?
Thanks,
Warren
From:
cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx]
On Behalf Of ext Brunauer, Walter
Sent: Friday,
February 15, 2008 1:56 AM
To: CDT General developers
list.
Subject: RE: [cdt-dev] First gotcha with add/exclude
children of FFS
Hi Warren,
FWIW, you did not mention anything about linked
resources and absolute paths these persist in the .project file by
default. Again a big issue around linked resources in combination with
sharing project within a team (even without team support), and one more
reason why they appear to be so cumbersome to handle. To me it seems
many times one has to unsell linked resources to users: Whereas linked
resources are (kind of!) nice for evaluation purposes (because, yes, in
this case you might not want to pollute your sources), as soon as you
start serious development, you run into all kind of troubles. The hurdle
to get everything right from the beginning is overwhelming for novices
(e.g. its not possible to change a linked resource to use a variable
later). Sorry, I don't know how to add this to the Wiki
page...
Having said that, the scenario you describe is really
about having the flexibility around build and indexer setup, not around
project setup, IMO.
It's rather classic: users have common code they want
to reuse in multiple applications - so they create one or a set of
libraries out of it, within one or a set of projects. Of course,
indexing should be able to handle only code going into these libraries,
and optionally ignore the rest. Then, they create their application
projects, which use the binary artifacts of the library project(s). Now
it would be great if they would have automatic support for application
linkage specification, i.e. some nice wizard or UI allowing to select
the library binaries of other projects to be linked in, without the need
to specify it manually in the linker options. And probably also desired:
during application code development, the public API's of all used
library projects should be the only thing they see WRT code completion,
etc. I guess, some UI would be needed for this as well.
And now think of all developers in the world.
Wouldn't it be great to give as many of them the freedom to choose how
to achieve this? Either everything in one project, or one project per
build artifact, or one project per module/application/product, or with
nested projects... its possible. Our commercial IDE based on CDT
supports all this, and we did not have to provide some EFS or work with
linked resources. Well, we had to override the build system, and this is
IMO the place to solve this in CDT as well.
Again, I don't see anything specific to project
setup. The issue around having the source tree polluted with project
files - I don't think this is the big thing. I would not leave the
Eclipse path in this area at all and allow to separate the project file
from the project location. Its a very general paradigm of Eclipse, and I
am pretty sure doing everything differently will generate lots and lots
of problems in all kind of areas (probably much more than you already
identified), unless you make it a new Eclipse way (add/change this in
the platform, not in CDT, that is).
Just my 2 or 3 cents again,
Walter
From:
cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx]
On Behalf Of Warren.Paul@xxxxxxxxx
Sent: Donnerstag,
14. Februar 2008 23:35
To:
cdt-dev@xxxxxxxxxxx
Subject: RE: [cdt-dev] First gotcha with
add/exclude children of FFS
I've updated the Wiki page http://wiki.eclipse.org/CDT:Flexible_Project_Structure
with some more thoughts on the issue. It would be great to get
feedback from other CDT users - both those shipping C/C++ IDE's and
end users. You'll see that I'm not convinced that the linked
resources route is a viable option. Maybe we can get the
platform team involved in the discussion to help find the best route
forward.
Thanks,
Warren
From:
cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx]
On Behalf Of ext Schaefer, Doug
Sent: Friday, January
25, 2008 11:15 AM
To: CDT General developers
list.
Subject: RE: [cdt-dev] First gotcha with add/exclude
children of FFS
I guess what my investigation has shown me
that the EFS solution and linked resources are pretty much identical.
I really noticed this when trying to figure out how to persist the
adds and found myself wishing I could add that to the .project file
just like linked resource are. And they are....
I think all the issues that we have with
linked resources would be equally as bad with the EFS solution,
possibly worse because the EFS adds are hidden. The CVS one is a great
example. I really doubt CVS would work with the EFS solution either.
And I don't want us to think EFS would be better since it's not in the
platform where we'll have a battle getting changes. Any platform
changes required to make linked resource work correctly would also
need to be done for EFS.
So my hope is to save the effort at
implementing the add/remove functionality since I believe that's
already there with linked/hidden resources. We can then focus on
making linked resources work where we need them and improving the
workflows. But this really needs to start now.
So, Warren, you've somewhat started a list of
workflows that we'd like to support with this solution. This is a
great place to start. I've created a Wiki page to start capturing
these. Please feel free to add more information, especially to the
workflow section. When we have that we may get a better idea of which
of the two solutions will work best.
http://wiki.eclipse.org/CDT:Flexible_Project_Structure
Thanks,
Doug
From:
cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx]
On Behalf Of Warren.Paul@xxxxxxxxx
Sent: Friday,
January 25, 2008 11:22 AM
To:
cdt-dev@xxxxxxxxxxx
Subject: RE: [cdt-dev] First gotcha with
add/exclude children of FFS
We've been working on Eclipse/CDT based
products for about three years now. I'm sorry to say that the
project model is still not satisfactory for our purposes. We've
tried many angles, but are still stuck with some pretty serious
limitations. I've volunteered to investigate the EFS route to
see if it will help at all. Based on this thread I'm assuming it
won't.
Let me give you a brief overview of how our
users work, and then discuss the problem we've run into. I don't
think any of this is specific to our users BTW.
Most of our users have existing code bases.
They simply want to "import" it into the IDE. Others will
create new projects from our templates. The new projects are
created in the workspace. Imported projects could be anywhere in
the file system. Often times they will import several projects
from the same source tree. This is where our biggest problem is.
Let's say the source base looks like
this:
C:\MyProjects\Project1\...
C:\MyProjects\Project2\...
C:\MyProjects\Common\...
Because
both projects share code in the Common directory, the logical root
project directory for both Project1 and Project 2 is C:\MyProjects\.
But in Eclipse you can't have two projects with the same root.
This is where the .project and .cproject files are created.
So currently our users would import Project1 with the natural
root (C:\MyProjects\), but Project2 has to be rooted at
C:\MyProjects\Project2\. This means that any source/headers from
the common directory are not under Project2. This means those
files are not in the project explorer for that project, are not
indexed, etc.. We logged this against the platform -
https://bugs.eclipse.org/bugs/show_bug.cgi?id=78438. Basically
if you put the .project file anywhere, but have a project root
attribute, this would cease to be a
problem.
Our first product actually
always created the .project in the workspace, and for imported
projects, would create links to files and folders. We ran into
so many issues with this that we had to change the model. I
don't recall all of the issues, but here's a list of
some:
- Version control simply didn't work
at all
- You can't make file system
changes with links. For example, if you want to rename a file or
folder, or move a file around, you can't do this with linked
resources. It only changes the link itself, not the underlying
resource.
- Creating new resources in
a project with links is confusing at best. Let's say you have a
project with a linked folder and file at the root. If you create
a new file or folder at the root, it is created in the workspace, not
where the other folder/file are in the file system. But if you
create a new file under the linked folder, it gets created where you'd
expect.
- The location of the
.project/.cproject files are problematic. Some users will want
to keep these in version control, while others won't. Those that
do want them created in the source tree, but those that don't want
them elsewhere, like the workspace. I forget now why this was a
problem with linked resources, but there was something weird going on
there.
I suppose it's worth noting
that the last time we really looked at this was in Eclipse 3.2, so
some of this may have been fixed by now. But I doubt it.
In general linked resources are second class citizens.
Some IResource API's just don't work for linked resources.
Just search for references to IResource#isLinked for "special
handling". I suspect that we'll run into similar issues with EFS
though - see getLocation vs
getLocationURI.
We also have the same
issue that Doug is trying to address (hiding some branches in a source
tree). This is much less of an issue for us though. You
can already reduce the scope of the indexer and the build. The
only real issue for us is for a very large source tree, you're going
to get IResource's for everything, which slows things down quite a
bit. There is actually somewhat of a problem in reducing the
indexer scope - see
https://bugs.eclipse.org/bugs/show_bug.cgi?id=178159.
The
hidden attribute addition sounds promising for hiding resources under
the project root, but doesn't really do anything to add flexibility to
the contents of a project. EFS sounds like it would though.
What I mean by that is, having resources under a project that
are real resources, not linked, but that don't live under the project
root in the file system. I've just started looking into EFS, so
maybe it's a bit of wishful thinking at this point, but I'm hoping we
could create a project anywhere, and when we create it we pass the URI
location from our own EFS. Then when asked for the children, we
could return URI's for files from anywhere in the file system, or on
other machines even. This would seem to hold the potential for
working around the issues listed above. We'd basically have an
EFS map from what we want under a project to the actual file
system.
So hopefully some of the
experts can chime in here. Is my hope for EFS unrealistic?
Is there a different approach we should look
at?
Thanks,
Warren
From:
cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx]
On Behalf Of ext Brunauer, Walter
Sent: Friday,
January 25, 2008 1:47 AM
To: CDT General developers
list.
Subject: RE: [cdt-dev] First gotcha with add/exclude
children of FFS
Hi,
after reading this rather long thread, I'll decided
to throw in my personal opinion.
I consider this approach to work against one of the
most general Eclipse platform paradigms, where a project is defined to
be a root directory and everything in it. IMO, the more workarounds are
introduced against this paradigm, the more problems will be faced, and
the more incompatibilities (or at least unawarenesses)
created.
Isn't the whole problem you try to solve here
rather about what files should go into the build (and probably into
the indexer) than what files are part of a project? I understand that
CDT has no separation of what a project and what the build input is
(well, IIRC one can exclude specific files from the build, but in
general, the project content defines the build input,
right?).
In our commercial IDE, we separated this. This not
only introduced much more powerful build setup capabilities in
general, but especially enabled users to setup build artifacts with
arbitrary contents (think of sources being compilable with different
compiler flags for different build artifacts, build input exclusion
patterns, build input from all over the workspace, multiple build
artifacts within the same project, reusable build artifacts accross
project boundaries, etc., etc., etc.). BTW, we call this build system
flexible managed build - because that's what it is:-)
Of course, one can setup CDT projects as of today
to exactly contain what is desired (with the help of linked
resources). However, I find linked resources to be cumbersome and
error prone, though many of our customers start out with them during
evaluation as well, mostly because they are looking for a way to
achieve what they did in the past with other non-Eclipse based IDEs,
but sooner or later I know of lots of them realizing its much easier
to use the features of our flexible build system instead, especially
if projects need to be shared in a team. And now think of virtual file
systems, the potential complexity of these, hidden assumptions,
restrictions, etc. Sounds worse than linked resources to
me.
I guess, the point I am trying to make is: whatever
you decide to do, make it understandable and transparent (and of
course as simple as possible to use) for the user.
As said, just my 2 cents,
Walter
From:
cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx]
On Behalf Of Schaefer, Doug
Sent: Donnerstag, 24.
Jänner 2008 23:17
To: CDT General developers
list.
Subject: RE: [cdt-dev] First gotcha with add/exclude
children of FFS
Jogging through the code, it really looks
like the HIDDEN feature is what I was looking for. What I haven't
found yet is UI to make a resource hidden or a navigator filter to
show hidden resources (in case you want to bring them back). Is this
planned?
Assuming we have the core features
available to link in and hide resource, I think we still have
workflow issues that need to be addressed. I like Ken's idea of a
file that controls the linking/hiding. We could have an
import/export mechanism for setting up projects based on this file.
A nice wizard for creating the file would also be good, similar to
the way the way the export file system wizard
works.
Given this, we may be further along than we
thought (BTW, the hidden stuff seems to have been added in 3.4
M4).
Cheers,
Doug
From:
cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx]
On Behalf Of Schaefer, Doug
Sent: Thursday, January
24, 2008 2:51 PM
To: CDT General developers
list.
Subject: RE: [cdt-dev] First gotcha with add/exclude
children of FFS
Thanks
Michael/Szymon,
Is there a bug describing the isHidden
feature?
Doug
From:
cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx]
On Behalf Of Michael Valenta
Sent: Thursday,
January 24, 2008 11:37 AM
To:
cdt-dev@xxxxxxxxxxx
Subject: RE: [cdt-dev] First gotcha
with add/exclude children of FFS
Doug et
al,
Szymon is really the person you want to bug on this but
I'll throw in my 2 cents ;-) First, I have to say that a
solution at the IResource level (e.g. using linked resources and the
new hidden folder support) is infinitely better from a repository
provider perspective than an EFS based solution. You may not
get all the Team support you want at the IResource level but a
solution at the EFS level would certainly break the existing CVS
client since the CVS client isn't EFS aware to any great extent. For
instance, if you tried to hide a folder using EFS, the CVS client
would probably try and recreate it the next time you performed a
Team>Update. It is also important to note that the Platform does
not provide all the hooks required by repository providers and I
know of at least one provider that has resorted to using it's own
EFS implementation under projects that are mapped to that provider
to get the capabilities it requires. I think it is important that
tooling in Eclipse stick to using the IResource layer as the layer
they operate on and let the repository provider (or any other
tooling whose responsibility it is to manage the available files)
control the underlying file system. If there are shortcomings or
enhancements required then you should push to get them in at the
IResource level.
As for the current state of Team support
for linked resources, I think the best approach is to enumerate some
specific scenarios of how you see linked resources and exclusions
working with descriptions of what you need to do today to get Team
support and what you would like to see. It is also important to know
if you expect all the links to come from the same repository (or at
least repository type) or whether a project could contain content
from different repository types (obviously the later would be more
difficult to accommodate than the former).
Hope this
helps,
Michael
_______________________________________________
cdt-dev
mailing
list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
