Bug 107682 - [implementation] Random documentProvider API must die!
Summary: [implementation] Random documentProvider API must die!
Status: VERIFIED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Text (show other bugs)
Version: 3.1   Edit
Hardware: PC Windows XP
: P2 normal (vote)
Target Milestone: 3.2 M5   Edit
Assignee: Platform-Text-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 49594 101846 (view as bug list)
Depends on:
Blocks: 107529
  Show dependency tree
 
Reported: 2005-08-22 20:44 EDT by Bob Foster CLA
Modified: 2006-07-06 14:55 EDT (History)
8 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Bob Foster CLA 2005-08-22 20:44:05 EDT
Or at least work.

When two or more plugins extend org.eclipse.ui.editors.documentProviders for the
same file extension and/or content type, Eclipse makes a random* choice of one
of them to create the document for the content. Since almost no editors work at
all when their own document class is not used to open a file, using this API
guarantees that all but one plugins in this situation are broken.

* The choice is random enough that under certain circumstances one
documentProvider will be used while at other times the other is. This leads to
_both_ plugins being broken for different use cases.
Comment 1 Bob Foster CLA 2005-08-22 20:47:55 EDT
To reproduce this, install the WTP editors and XMLBuddy (free download). Use
File > Open File to open an "external" XML file. Change the default editor back
and forth between XMLBuddy and the WTP XML editor and Open File. One of the
editors will fail. The stack trace when XMLBuddy is used to edit the file but
WTP is chosen to create the document looks like this:

org.eclipse.jface.util.Assert$AssertionFailedException: Assertion failed: 
	at org.eclipse.jface.util.Assert.isTrue(Assert.java:180)
	at org.eclipse.jface.util.Assert.isTrue(Assert.java:165)
	at com.objfac.xmleditor.partition.Partitioner.computePartitioning(Partitioner.java)
	at
org.eclipse.wst.sse.core.internal.text.BasicStructuredDocument.computePartitioning(BasicStructuredDocument.java:1059)
	at org.eclipse.jface.text.TextUtilities.computePartitioning(TextUtilities.java:426)
	at
org.eclipse.jface.text.presentation.PresentationReconciler.createPresentation(PresentationReconciler.java:443)
	at
org.eclipse.jface.text.presentation.PresentationReconciler.processDamage(PresentationReconciler.java:560)
	at
org.eclipse.jface.text.presentation.PresentationReconciler.access$3(PresentationReconciler.java:558)
	at
org.eclipse.jface.text.presentation.PresentationReconciler$InternalListener.inputDocumentChanged(PresentationReconciler.java:118)
	at org.eclipse.jface.text.TextViewer.fireInputDocumentChanged(TextViewer.java:2439)
	at org.eclipse.jface.text.TextViewer.setDocument(TextViewer.java:2488)
	at org.eclipse.jface.text.source.SourceViewer.setDocument(SourceViewer.java:491)
	at org.eclipse.jface.text.source.SourceViewer.setDocument(SourceViewer.java:435)
	at
org.eclipse.ui.texteditor.AbstractTextEditor.initializeSourceViewer(AbstractTextEditor.java:2882)
	at
org.eclipse.ui.texteditor.AbstractTextEditor.createPartControl(AbstractTextEditor.java:2649)
	at
org.eclipse.ui.texteditor.StatusTextEditor.createPartControl(StatusTextEditor.java:53)
	at
org.eclipse.ui.texteditor.AbstractDecoratedTextEditor.createPartControl(AbstractDecoratedTextEditor.java:314)
	at com.objfac.xmleditor.BaseEditor.createPartControl(BaseEditor.java)
	at com.objfac.xmleditor.XMLEditor.createPartControl(XMLEditor.java)
	at
org.eclipse.ui.internal.EditorReference.createPartHelper(EditorReference.java:585)
	at org.eclipse.ui.internal.EditorReference.createPart(EditorReference.java:365)
	at
org.eclipse.ui.internal.WorkbenchPartReference.getPart(WorkbenchPartReference.java:552)
	at org.eclipse.ui.internal.PartPane.setVisible(PartPane.java:283)
	at
org.eclipse.ui.internal.presentations.PresentablePart.setVisible(PresentablePart.java:126)
	at
org.eclipse.ui.internal.presentations.util.PresentablePartFolder.select(PresentablePartFolder.java:268)
	at
org.eclipse.ui.internal.presentations.util.LeftToRightTabOrder.select(LeftToRightTabOrder.java:65)
	at
org.eclipse.ui.internal.presentations.util.TabbedStackPresentation.selectPart(TabbedStackPresentation.java:391)
	at
org.eclipse.ui.internal.PartStack.refreshPresentationSelection(PartStack.java:1102)
	at org.eclipse.ui.internal.PartStack.setSelection(PartStack.java:1051)
	at org.eclipse.ui.internal.PartStack.showPart(PartStack.java:1256)
	at org.eclipse.ui.internal.PartStack.add(PartStack.java:442)
	at org.eclipse.ui.internal.EditorStack.add(EditorStack.java:109)
	at
org.eclipse.ui.internal.EditorSashContainer.addEditor(EditorSashContainer.java:60)
	at org.eclipse.ui.internal.EditorAreaHelper.addToLayout(EditorAreaHelper.java:212)
	at org.eclipse.ui.internal.EditorAreaHelper.addEditor(EditorAreaHelper.java:202)
	at org.eclipse.ui.internal.EditorManager.createEditorTab(EditorManager.java:753)
	at
org.eclipse.ui.internal.EditorManager.openEditorFromDescriptor(EditorManager.java:665)
	at org.eclipse.ui.internal.EditorManager.openEditor(EditorManager.java:628)
	at
org.eclipse.ui.internal.WorkbenchPage.busyOpenEditorBatched(WorkbenchPage.java:2323)
	at org.eclipse.ui.internal.WorkbenchPage.busyOpenEditor(WorkbenchPage.java:2258)
	at org.eclipse.ui.internal.WorkbenchPage.access$9(WorkbenchPage.java:2250)
	at org.eclipse.ui.internal.WorkbenchPage$9.run(WorkbenchPage.java:2236)
	at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:69)
	at org.eclipse.ui.internal.WorkbenchPage.openEditor(WorkbenchPage.java:2231)
	at org.eclipse.ui.internal.WorkbenchPage.openEditor(WorkbenchPage.java:2204)
	at
org.eclipse.ui.internal.editors.text.OpenExternalFileAction.run(OpenExternalFileAction.java:129)
	at
org.eclipse.ui.internal.editors.text.OpenExternalFileAction.run(OpenExternalFileAction.java:98)
	at org.eclipse.ui.internal.PluginAction.runWithEvent(PluginAction.java:246)
	at org.eclipse.ui.internal.WWinPluginAction.runWithEvent(WWinPluginAction.java:223)
	at
org.eclipse.jface.action.ActionContributionItem.handleWidgetSelection(ActionContributionItem.java:538)
	at
org.eclipse.jface.action.ActionContributionItem.access$2(ActionContributionItem.java:488)
	at
org.eclipse.jface.action.ActionContributionItem$5.handleEvent(ActionContributionItem.java:400)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:66)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:843)
	at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:3080)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:2713)
	at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:1699)
	at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:1663)
	at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:367)
	at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:143)
	at org.eclipse.ui.internal.ide.IDEApplication.run(IDEApplication.java:103)
	at
org.eclipse.core.internal.runtime.PlatformActivator$1.run(PlatformActivator.java:226)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:376)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:163)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.eclipse.core.launcher.Main.invokeFramework(Main.java:334)
	at org.eclipse.core.launcher.Main.basicRun(Main.java:278)
	at org.eclipse.core.launcher.Main.run(Main.java:973)
	at org.eclipse.core.launcher.Main.main(Main.java:948)

Note that org.eclipse.wst.sse.core.internal.text.BasicStructuredDocument has no
business being in this stack trace!
Comment 2 Dani Megert CLA 2005-08-23 04:51:34 EDT
*** Bug 49594 has been marked as a duplicate of this bug. ***
Comment 3 Marat Boshernitsan CLA 2005-08-23 12:35:52 EDT
(In reply to comment #2)
> *** Bug 49594 has been marked as a duplicate of this bug. ***

Bug 49594 is not quite a duplicate, it describes a similar problem with a
different extension point (org.eclipse.core.filebuffers.documentCreation).  I
assume same is true of org.eclipse.core.filebuffers.documentSetup and
org.eclipse.core.filebuffers.annotationModelCreation.
Comment 4 Bob Foster CLA 2005-08-23 12:52:09 EDT
Actually it is the same problem. org.eclipse.core.filebuffers.documentCreation
extensions can also conflict _and_ they can conflict with
org.eclipse.ui.editors.documentProviders extensions. (The latter is actually
what is happening in my earlier stack trace. WST uses documentCreation while
XMLBuddy uses documentProviders.)

Like you, I'm suspicious that
org.eclipse.core.filebuffers.annotationModelCreation has the same issue, but I
haven't actually seen an example. Since viewers are not shared it could be this
works ok.

I don't see a conflict problem with org.eclipse.core.filebuffers.documentSetup.
Comment 5 Bob Foster CLA 2005-09-01 13:04:32 EDT
See Bug 89195 for a horrible example. It should be possible for multiple
different content types to be detected for the same file without interference;
indeed, it is intrinsic that any file in XML syntax - Ant, Maven, J2EE, XSLT -
is also an XML file and can be edited by any XML editor. The conceptual problem
is that there is not, nor will ever be, a single "best" document creator
associated with a file. The document creator that must be used when a file is
opened is the one required by the plugin that opens it.

Historically, I know Text got in this state by trying to satisfy two requirements:

- If a user makes a change in one editor and the file is also open in another
editor, the change should be reflected in the other.

- It should be possible to open files outside the workspace in Eclipse.

while losing sight of this requirement:

- It should be possible for the user (or Eclipse) to choose any editor for any file.

The whole "content type" thing has just been a distraction. As Bug 89195 shows,
each editor is trying to use content types to force creation of the document
class appropriate for that editor. Completely wrong-headed.

I note this is marked for 3.2. Some discussion of the proposed solution would
sure be nice. 
Comment 6 Dani Megert CLA 2005-09-02 04:25:49 EDT
Would a solution be acceptable where the problem we see now only might happen if
two different editors are used to open the same file i.e.
- open a file with whatever editor would always work
- open the same file again with above editor would also always work and they would
  be in sync as long a file buffers and/or text file document provider is used
- open the same file again with a different editor might fail if that editor
  expects more from a document than what's speced in IDocument
Comment 7 David Williams CLA 2005-09-02 11:14:49 EDT
I'm not sure I follow your proposal, Dani. But, will chime in on some of the 
general issues in this area -- and there are many. 

1. The bug described in comment 2 ... "BasicStructuredDocument has no
business being in this stack trace" ... is really just a bug against 
wtp.sse ... we are not meeting the spec of IDocument and Partioners, 
I'd guess, and otherwise, an editor shouldn't mind (too much) who implements 
IDocument. 

2. bug 89195 is not really the same problem as this one, IMHO. That issue would
be a problem if no editors (or even documents) were involved at all. Its a case
that "ant" is not a very distinct "language" but the ant team is (rightfully)
trying to make things easier for users by identifying ant content, and providing
special functions for cases a resource is ant content. (And they are just an
over eager bunch :)

3. The participant pattern (and actually the whole plugin pattern) in Eclipse
does not have a well organized story for what to do if there are "conflicting"
contributions for something. Who wins? Ultimately, there will be cases where a
user will have to "pick one" .... but I have not seen any general infrastructure
for this, or good examples, or an overall design recommendation for how to
handle  it in a consistent fashion. [And, I know, everyone hopes it shouldn't be
needed, 
that we could be intuitive and know exactly what the users wants ... but ... ]

4. I know to "editor people" it seems no big deal, but plesae don't think of a
Document as just an editor construct. It should be a full fledged "text model"
object all in its own right, no matter what UI is modifying it. Which makes it
this general problem all the worse (but the solution needs to apply to
non-editor use cases as well). 

So ... where to from here? Has anyone else thought of any of the above issues? 
Would it help to take a step back and detail the requirements, design, and
direction of "Document"? and FileBuffers? and DocumentProviders? (And, apologies
in advance if these design documents already exist and I just don't know about
them). 

Thanks for reading. 


Comment 8 Bob Foster CLA 2005-09-03 19:47:54 EDT
(In reply to comment #6)
> Would a solution be acceptable where the problem we see now only might happen if
> two different editors are used to open the same file i.e.

Doesn't sound like a solution to me, especially now that it is possible (and
easy) to open a file again in the same window with a different editor.

Why can't we just establish as reality what is already true in practice? A
plugin that opens a file needs to be able to specify the document class used for
the file. Data sharing and multi-editor synchronization is great, but the
document belongs to the plugin, not to Eclipse. So a way needs to be found to
share data without forcing editors to share the IDocument object.
Comment 9 Bob Foster CLA 2005-09-03 20:20:24 EDT
(In reply to comment #7)

I very much agree with most of David's comments. I say that up front because it
is sometimes not obvious to people I am agreeing with them. ;-}

> 1. ...an editor shouldn't mind (too much) who implements 
> IDocument.

But the horse is already out of the barn. The Java editor won't work unless it
gets a synchronizable document. The WTP editors won't work unless they get a
structured document. It was quite a struggle to get XMLBuddy to be as
document-neutral as it is, and it is still crippled when it isn't able to use
its own DocumentProvider.

OTOH, if Text were to declare that an editor _must_ not depend on the class of
its IDocument (or IDocumentProvider) objects - and all existing editors could
get the functionality they currently depend on in other ways - that would be fine.

> 2. bug 89195 is not really the same problem as this one, IMHO. 

IMHO it is. ;-} The editors are trying to use content type to force selection of
their own document creators. I cry foul. This is just a hack to overcome the
problem described in this bug. Eliminate the need for the hack and you will
eliminate Bug 89195 as well.

> 3. The participant pattern (and actually the whole plugin pattern) in Eclipse
> does not have a well organized story for what to do if there are "conflicting"
> contributions for something. Who wins? Ultimately, there will be cases where a
> user will have to "pick one" .... but I have not seen any general infrastructure
> for this, or good examples, or an overall design recommendation for how to
> handle  it in a consistent fashion.

I snipped the part that seemed to suggest, gee, maybe we can't do this right. Of
course we can. For the editor case, user picks editor (or Eclipse picks default
editor) and everything else is determined by the selection of the editor. For
the programmable case, plugin picks editor (or whatever, see below) and
everything else is determined by the selection of the editor.

> 4. I know to "editor people" it seems no big deal, but plesae don't think of a
> Document as just an editor construct. It should be a full fledged "text model"
> object all in its own right, no matter what UI is modifying it. Which makes it
> this general problem all the worse (but the solution needs to apply to
> non-editor use cases as well).

I don't agree, but mostly because IDocument seems hopelessly contaminated by
history. There needs to be a text model, that's for sure, but it seems unlikely
it can be the document object. Water under the bridge.

> So ... where to from here? Has anyone else thought of any of the above issues? 
> Would it help to take a step back and detail the requirements, design, and
> direction of "Document"? and FileBuffers? and DocumentProviders? (And, apologies
> in advance if these design documents already exist and I just don't know about
> them).

Yes! The right thing to do is stop trying to make the current broken API work,
step back and find the right abstractions, move forward from there.

David is dead right that there needs to be an abstraction that is not
necessarily an editor but is a "file opener". There needs to be an abstraction
for the data model of a file to allow sharing transparent to the current
document object. From current requirements, the data model needs to be
synchronized to prevent collisions and allow atomic "transactions". What are its
other requirements?

I personally believe that error/warning/info annotations are not data model
properties but properties of the "file opener". Different views of the data
induce different error states. E.g., is the file an XML file, an XSLT file or a
specific type of transform?

Like David, if there are any design documentation or proposals, I would love to
see them. Just a requirements list would be great!
Comment 10 Dani Megert CLA 2005-09-09 12:07:46 EDT
I agree that the current situation is a problem.

File buffers intent was to offer access to a shared document to several clients
(actions, models, editors) that need the document and want to share it, however,
it was not the intent to make this editor specific or aware. The idea was that
this shared document is the common ground i.e. documentCreation and setup should
only be used if it is seen as *the* common ground (i.e. master). If an editor
would require a special document type and/or setup then the idea was that it
should achieve this using its own file document provider (eventually wrapping
the document it gets from file buffers) because the document provider was
invented to provide the correct document and setup for a given editor family. It
was not the intent that each editor uses the file buffer extension points. This
should only be done if the intent is to share the creation and setup to all
clients (which should be rare).

Having said that, the problems started  with implementation provided for the
Java editor (and others in the Eclipse SDK) which exactly gave the impression
that those extension points should be used by editor providers. In addition the
Java editor's implementation of IDocumentSetupParticipant contradicts to the
spec of IDocumentSetupParticipant by setting a partitioner for the default
partitioning in case the document doesn't implement IDocumentExtension3.

Until we have better solution (a fix without any changes on the client side will
most likely not be possible) there are several things clients could do:
1) if you can live with the fact that your editor's document is only shared
between your editors you can provide your own document provider which does not
use file buffers (e.g. FileDocumentProvider)
2) if you want sharing but expect a special document and setup you can implement
your own document provider that gets the document from file buffers and wraps it
and you would not contribute any file buffer extensions. Note that this solution
might still fail if there's are documentSetup contributions from different
providers for the same content type.
Comment 11 Bob Foster CLA 2005-09-11 13:30:51 EDT
(In reply to comment #10)

Dani, I'm not sure I understand your response. In the first place, document
creation and setup (the assignment of a partitioner) are very different cases.
An editor may be able to work with any IDocument implementation but I have never
seen one that would work with any partitioner other than its own. Unless I am
missing something I subtract "and setup" from your reply.

The suggestion that an editor provide its own documentprovider to wrap the
filebuffers-created document seems to forget that filebuffers also creates the
documentprovider for external files.

Lastly, "*the* common ground" makes no sense to me. IDocument is the common
ground. Everything else is a plugin extension, and you can hardly expect two
independently developed plugins to agree on what the common ground would be for
a file type. (Moreover, if an API is undocumented as to intent you can hardly
expect plugins to use it according to its intent.) I see the document creation
APIs as an ugly hack to satisfy the Java editor, which needed its document class
to be synchronized so multiple threads could use it at once. The true ugliness
is revealed by the fact it only works if there is only one document creator for
*.java files.

The root question is: why do editors/views/models need their own document type?
I see two use cases.

Every editor of any sophistication has multiple threads that need to
simultaneously access the document. There are two possible implementations: make
a copy of the document contents on the UI thread for each background thread to
use and validate the results of the background thread with the mod stamp, or
synchronize access to the document. The latter is the only performant solution
for large files (XML has use cases in the hundreds of megabytes). Therefore
synchronization should be implemented in the base documents created by file
buffers. This would eliminate the need for Java and Ant to have special document
classes (Ant's doesn't work, anyway).

The second important use case is implementing IDocument to add extensions
related to the editor-specific document model, which is apparently why the WTP
editors do it. In the MVC paradigm, the sourceviewer is the viewer, the document
is the model and the editor (or whatever) is the controller. So naturally when
applications have more sophisticated models than plain text, e.g., Java's AST or
XML's tree, they tend to lump these together. I personally believe it is a
mistake to extend the IDocument implementation for this purpose; all that is
required is that the editor-specific model be 1-1 to the shared document. I
believe I have seen a comment to this effect from David. It would be simple
enough to hang the editor-specific model on the editor-specific partitioner
already stored in the document object.

Therefore, lacking any valid use cases, the document creator extension points
should be deprecated in 3.2 and all misleading examples removed from the core
distribution. Plugins should not provide document creators _or_
documentproviders - the latter don't work for external files, for which
filebuffers creates a documentprovider that actually works.

While you are up, please deprecate setDocumentPartitioner() and
getDocumentPartitioner() in IDocument. These have no place in the world of
shared documents and I don't believe there are any valid use cases that don't
want sharing. The fact that there are some cases where users don't *get* sharing
is just an implementation artifact; these cases should be eliminated. If there
is a need to read/write a file in a non-shared way, java.io.File is there for
the purpose. It should not be possible using IDocument.
Comment 12 David Williams CLA 2005-09-12 03:21:14 EDT
Let me see if I can summarize one short term solution to at least do better than
we are now for the specific "interested parties". 

1. WTP document start using documentChanged2 instead of the very old
documentChange (I'm not really sure what this accomplishes, I guess we are 
supposed to do something with the region that's returned from it, but its unclear
what the constract is, exactly. 
2. WTP stop setting default partition at all (and be on the look out for NPE's). 
3. WTP AND XML Buddy implement getFileDocumentProvider in their editor to return 
an instance of their document provider (and * not use * the plugin.xml extension
point for it. (I'm not sure what XMLBuddy needs for a document, I assume the
"generic" one, but with their own partioner?). 
5. WTP's documentProvider would still "force" use the common StrucutredDocument
and all we need from it. When an editor is not used, we'd still use document
factory so it would still be the common ground shared document. When someone
besides an editor got a document. (And, recall, that's actually our greatest
concern ... editors could work perfectly, but if our document is not used when
someone tries to use our model, then they are hosed). 

Not sure the above will work as all expect, but I think that's the next step to
try. And we'll try to sort out the higher level theoretical things later? 


As for our documents special needs, there is quite a few. Thread safety (job
safety) is one. Another is to be able to call 'replace' (or add positions) from
non-Dispaly thread. 

More imporantly, even then document is changed, we 'reparse' a certain amount of
our strucuture, and send out more specific document events than generic document
does. (I do not like the reconciler thread concept). Its for this reason, I do
not think we could ever simply "wrap" an existing document that others might
have access to (in an unwrapped fashion) since we could not then control the
exact sequence of events and parses. And, recall, we often need to get a
document, even when we are not editing! To build other models, refactor links, etc. 

Lastly, there's some improvements I think need to be made to document that I
assumed another implemetation would be the best way to solve. But that's a long
story, so let's focus on the specific short term solution to see if any better. 

Comment 13 Bob Foster CLA 2005-09-13 01:48:33 EDT
(In reply to comment #12)
Sounds good and moves in the right direction, but I think there are at least two
issues.

- If XMLBuddy opens a file and then WTP opens the same file, either there is not
going to be any sharing or WTP isn't going to work. (XMLBuddy is pretty
document-agnostic as we do the stupid thing and copy the document for background
threads.)

- For external files, I think Eclipse creates the documentprovider and the
editor doesn't really get any say in the matter. Could be wrong. Dani?

I still don't understand why WTP needs to specify the document class. The part
of your requirements I understood all seem to deal with concurrent access. If
the default document created by filebuffers provided synchronization, what else
would you need?
Comment 14 Dani Megert CLA 2005-09-16 12:42:41 EDT
re comment 11:

Bob, let me try to answer your comment:

>An editor may be able to work with any IDocument implementation but I have never
>seen one that would work with any partitioner other than its own.
An editor requires his own partitioning being in place but it should be
irrelevant whether there are other partitionings around. Problems arise if the
editor does not have its own partitioning but uses the default partitioning and
someone else also registers a partitioner for the default partitioning. I
IDocumentSetupParticipant specifies this and those who do otherwise basically
violate the spec. Initially we considered not to call the participant if the
document doesn't implement IDocumentExtension3 but there might be other aspects
that someone would like to setup (be it simply to know when this happens). Up to
know the extension point itself only mentioned that IDocumentSetupParticipant
must be implemented. I've now also explicitly written that into the extension
point doc.

>The suggestion that an editor provide its own documentprovider to wrap the
>filebuffers-created document seems to forget that filebuffers also creates the
>documentprovider for external files.
File buffers don't create documentProviders. But yes, we provide a default
document provider for external files for those using the implicit document
provider but every editor can define its own (explicit) document provider by
calling AbstractTextEditor.setDocumentProvider(IDocumentProvider provider).

>Lastly, "*the* common ground" makes no sense to me. IDocument is the common
>ground. Everything else is a plugin extension, 
This is true for IDEs and other tools that are developed to be extensible but
many clients use Eclipse to build their applications (see rcp) and they might
well have some other common ground.

>...for large files (XML has use cases in the hundreds of megabytes). Therefore
>synchronization should be implemented in the base documents created by file
>buffers.
That's my current thinking i.e. you'll get synchronized document that's shared
and that implements IDocument. Whoever wants more can do more based on that
document.

>Therefore, lacking any valid use cases, the document creator extension points
>should be deprecated in 3.2
That's again my plan.

>While you are up, please deprecate setDocumentPartitioner() and
>getDocumentPartitioner() in IDocument. These have no place in the world of
>shared documents 
This can't happen because lots of clients will never need/use that sharing and
are fully happy with the default partitioning.
Comment 15 Dani Megert CLA 2005-09-16 12:44:42 EDT
>- For external files, I think Eclipse creates the documentprovider and the
>editor doesn't really get any say in the matter. Could be wrong. Dani?
As outline in the previous comment every editor is free to use its own document
provider.

>I still don't understand why WTP needs to specify the document class.
David, can you give a short explanation for Bob?
Comment 16 Dani Megert CLA 2005-09-16 12:45:21 EDT
Answer to comment 12 will follow this weekend.
Comment 17 Bob Foster CLA 2005-09-17 13:08:21 EDT
(In reply to comment #14)
> Problems arise if the
> editor does not have its own partitioning but uses the default partitioning
> and someone else also registers a partitioner for the default partitioning.

What default partitioning? In my experience the default partitioning is no
partitioning, i.e., getDocumentPartitioner() returns null.

> Up to
> know the extension point itself only mentioned that IDocumentSetupParticipant
> must be implemented. I've now also explicitly written that into the extension
> point doc.

That's good, but I fear you're not writing to the right audience. The people who
need to know there's a collision problem with setDocumentPartitioner() and
shared documents are those who don't know about the extension point.

> File buffers don't create documentProviders. But yes, we provide a default
> document provider for external files for those using the implicit document
> provider

;-}

> but every editor can define its own (explicit) document provider by
> calling AbstractTextEditor.setDocumentProvider(IDocumentProvider provider).

Sure you can, but the external file example shows that editors that do define
their own document providers wind up broken because of assumptions in the
document provider about specific IEditorInput types. When the next release
introduces an input type that uses a URI instead of any sort of file, all
existing document providers will be hopelessly broken. It's not even possible to
adjust to the external file input type - it's not API. This whole (input) area
is crap. It should be possible to read and write documents without having to
cast, excuse me, adapt, down to specific input types. But that's a digression.

Leaving aside the impossibility of writing a general-purpose document provider,
what is the provider supposed to do? One would guess from the name it should
provide a document. But if it creates its own document object, it won't get
sharing. What is the code sequence that allows a document provider to fetch a
shared document object from filebuffers?

You can probably answer the last question, but do you see why I think that
document users are better off not implementing document providers? They have far
better insulation from future extensions if they leave the matter to Eclipse.

> >Lastly, "*the* common ground" makes no sense to me. IDocument is the common
> >ground. Everything else is a plugin extension, 
> This is true for IDEs and other tools that are developed to be extensible but
> many clients use Eclipse to build their applications (see rcp) and they might
> well have some other common ground.

They might want UI that doesn't assume toolbars, too, but they don't get one
from RCP. RCP is not a blank slate.

> That's my current thinking i.e. you'll get synchronized document that's shared
> and that implements IDocument. Whoever wants more can do more based on that
> document.

Hooray! Please note my comment to Bug 75767.

> >Therefore, lacking any valid use cases, the document creator extension points
> >should be deprecated in 3.2
> That's again my plan.

+1

> >While you are up, please deprecate setDocumentPartitioner() and
> >getDocumentPartitioner() in IDocument. These have no place in the world of
> >shared documents 
> This can't happen because lots of clients will never need/use that sharing and
> are fully happy with the default partitioning.

Of course it can happen. The alternative, using a named document partitioner, is
just as easy to use and has no hidden pitfalls. It's not like you would be
removing functionality.

Comment 18 Dani Megert CLA 2005-09-18 08:14:09 EDT
>What default partitioning? In my experience the default partitioning is no
>partitioning, i.e., getDocumentPartitioner() returns null.
Correct. What I meant was: clients registering their partitioner either via
setDocumentPartitioner(IDocumentPartitioner) or as
IDocumentExtension3.DEFAULT_PARTITIONING.

> It's not even possible to
>adjust to the external file input type - it's not API.
This should probably be changed so that clients can use it.

If we'd deprecate get/setDocumentPartitioner(IDocumentPartitioner) then we
should also deprecate the IDocumentExtension3.DEFAULT_PARTITIONING constant
otherwise this could result in the same conflict again. The problem with the
deprecation is that those who want to get rid of the deprecation are force to
implement IDocumentExtension3 even if the eventually live in a closed world.
Comment 19 Dani Megert CLA 2005-09-18 08:28:34 EDT
In reply to comment 12:


>1. WTP document start using documentChanged2 instead of the very old
>documentChange (I'm not really sure what this accomplishes, I guess we are 
Is a good idea but not really connected to this PR.

>supposed to do something with the region that's returned from it,
For example the DocumentPartitioningChangedEvent.setPartitionChange(...) can be
reduced to the region that changed instead of using 0..length.

> but its unclear what the constract is, exactly. 
The minimal region that includes all partition changes caused by the invocation
of the document partitioner.

>3. WTP AND XML Buddy implement getFileDocumentProvider in their editor to >return 
>an instance of their document provider (and * not use * the plugin.xml
>extension point for it. 
Do not overwrite getDocumentProvider() [I assume that's what you meant with
getFileDocumentProvider]. Instead call setDocumentProvider(...) in your editor's
constructor to set the explicit document provider.

>5. WTP's documentProvider would still "force" use the common StrucutredDocument
>and all we need from it. When an editor is not used, we'd still use document
>factory so it would still be the common ground shared document.
Stay away form using the documentCreation extension point. I plan to deprecate
it for 3.2. If you need your special document type then you could introduce your
own document factory from where clients can get the specialized document.
Comment 20 Bob Foster CLA 2005-09-18 11:34:35 EDT
(In reply to comment #18)

What is the API that allows a document provider to fetch a shared document
object from filebuffers (so it can be shadowed)?

> > It's not even possible to
> >adjust to the external file input type - it's not API.
> This should probably be changed so that clients can use it.

You might want to change its name, as well, to e.g. PathEditorInput, to stop
needing to explain what "JavaFile" means.

> If we'd deprecate get/setDocumentPartitioner(IDocumentPartitioner) then we
> should also deprecate the IDocumentExtension3.DEFAULT_PARTITIONING constant
> otherwise this could result in the same conflict again. The problem with the
> deprecation is that those who want to get rid of the deprecation are force to
> implement IDocumentExtension3 even if the eventually live in a closed world.

That kind of inconvenience is all over Eclipse. "Deprecated; use
IDocumentExtension3.setDocumentPartitioner()". No biggie. (There are a number of
other methods in IDocumentExtension3 that should be deprecated in IDocument, as
well. IIRC, they must all be used together.

As for "closed world", I think of that as a state of blissful ignorance that
ends the first time a document must be opened by two different processes. ;-}
Comment 21 Bob Foster CLA 2005-09-18 11:40:38 EDT
Just thought of this: When you make JavaFileEditorInput API, you should also
provide a single documentprovider for JavaFileEditorInput, IFileEditorInput and
IStorageEditorInput. IIRC, except for getting the input and output streams most
of the code is common, including getting encoding right per content types. Then
"external files" wouldn't need to be a special case.
Comment 22 Bob Foster CLA 2005-09-18 11:45:08 EDT
Question for David: If the shared document object produced by filebuffers allows
synchronization, why do need your own IDocument implementation? If you hang
other "model" data off it, why don't you just hang that data elsewhere? Maybe
subclassing isn't the best solution. (It's certainly not the easiest.)
Comment 23 Missing name CLA 2005-10-03 18:13:15 EDT
Please refere to bug 111313
Comment 24 Dani Megert CLA 2006-01-05 09:51:33 EST
Fixed in HEAD.
Available in builds > N20060105-0010.

See for details see bug 96917, comment 17.
Comment 25 Benno Baumgartner CLA 2006-02-14 12:07:40 EST
verifying...
Comment 26 Benno Baumgartner CLA 2006-02-14 12:35:34 EST
verified in I20060214-0010
Comment 27 Dani Megert CLA 2006-03-27 11:26:32 EST
*** Bug 101846 has been marked as a duplicate of this bug. ***
Comment 28 David Coppit CLA 2006-04-07 11:04:29 EDT
Sorry to ask a stupid question, but: Can someone please summarize what the right course of action should be for plugin authors that are implementing editors derived from CompilationUnitEditor and need to create custom documents? Do we extend at org.eclipse.core.internal.filebuffers.ExtensionsRegistry? Do we need to define a content type as well? Or do we need to implement our own document provider that does buffering, compilation unit creation, etc. just to add some additional state to the Document object?
Comment 29 Dani Megert CLA 2006-04-09 10:46:33 EDT
It depends whether your editor edits Java files or soemthing different (e.g. non *.java files). If so, a different content type makes sense.

Best way is to have your own document provider that adapts the file buffer document.