This issue is about more than just URI encoding it is about naming things and defining concepts. This stuff is fundamental.
It will likely also fix a bunch of issues that have been opened about URI encoding and non-ascii character encoding bugs, but it is not a bug report in and of itself.
Thanks to both @duncdrum and @ccheraa for helping me reason this out...
I will use xmldb:create-collection only to demonstrate problems...
Currently:
xmldb:create-collection("/db", "hello world")Raises the error:
org.xmldb.api.base.XMLDBException: err:FORG0001 failed to convert hello world into an XmldbURI: Illegal character in path at index 5: hello world [at line 1, column 1, source: String/2869744519086207067]
However, the signature of xmldb:create-collection is:
xmldb:create-collection($target-collection-uri as xs:string, $new-collection as xs:string) as xs:string?
The parameter $new-collection is documented as: The name of the new collection to create.
A "Collection" is technically ONLY a mapping from an xs:string (interpreted as an xs:anyURI) to a item()*.
https://www.w3.org/TR/xpath-functions-31/#func-collection
So the first problem here is that I can't see a reason why hello world should not be a valid "name" for a Collection. Now of course, I could argue the opposite because the XQuery spec. doesn't have the notion of a Collection name. Neither does eXist-db define anywhere what a Collection name is.
The following XQuery does execute: xmldb:create-collection("/db", "hello%20world") however you end up with a Collection that is named: hello%20world.
IMHO "name" does not imply URI, rather it should be much closer to the concept of a "file name" in a file system. Note, eXist-db does not define "file name" either.
The second and third issues are the return values from xmldb:create-collection, it is documented as: the path to the new collection if successfully created, otherwise the empty sequence.
- The function in practical terms never returns the empty sequence, instead it raises errors if the collection can't be created. What those errors are is undocumented.
- When the function returns a value, it returns a URI but as an
xs:string, e.g. /db/hello%20world. The documentation says it returns a "path", however nowhere is it defined what is meant by "path". Is a path a URI or just a string, i.e. should the return value actually be (a) /db/hello%20world, or (b) /db/hello world? I don't know, but /db/hello%20world feels kind of right here. If that is the case, then the function should be documented instead as returns the database URI of a Collection expressed as a xs:string.
The above is just one of many examples that I could formulate. However, before we start fixing things, what we really need to do is work on strict definitions of various concepts used by eXist-db.
These concepts in the context of eXist-db need to be properly defined and we need to consider if they are (1) expressed in a URI encoded or decoded way, and (2) how they are expressed as either or both xs:anyURI or xs:string.
- Collection
- Collection Name
- Collection URI
- Document Name
- Document URI
- Path
- URI as an xs:string
- URI as a xs:anyURI
My thoughts...
If a Collection Name is just a name like how a File Name is just a name, then it should be totally valid to create these two distinct collections:
xmldb:create-collection("/db", "hello world"),
xmldb:create-collection("/db", "hello%20world")
In the second, the actual name is "hello%20world" and is not short for something that was URI encoded. I would expect the above to return:
("/db/hello%20world", "/db/hello%2520world")
If we were to transition to Collection Names and Document Names being just names, then eXist-db needs some fixes. At the moment it unnecessarily stores just the Collection Name (not the Collection URI) as an XmldbURI.
If we were to store names rather as strings rather than XmldbURI, then this would require a change in storage-format for collections.dbx and therefore necessitates a major release.
This issue is about more than just URI encoding it is about naming things and defining concepts. This stuff is fundamental.
It will likely also fix a bunch of issues that have been opened about URI encoding and non-ascii character encoding bugs, but it is not a bug report in and of itself.
Thanks to both @duncdrum and @ccheraa for helping me reason this out...
I will use
xmldb:create-collectiononly to demonstrate problems...Currently:
Raises the error:
However, the signature of
xmldb:create-collectionis:The parameter
$new-collectionis documented as:The name of the new collection to create.A "Collection" is technically ONLY a mapping from an
xs:string(interpreted as anxs:anyURI) to aitem()*.https://www.w3.org/TR/xpath-functions-31/#func-collection
So the first problem here is that I can't see a reason why
hello worldshould not be a valid "name" for a Collection. Now of course, I could argue the opposite because the XQuery spec. doesn't have the notion of a Collection name. Neither does eXist-db define anywhere what a Collection name is.The following XQuery does execute:
xmldb:create-collection("/db", "hello%20world")however you end up with a Collection that is named:hello%20world.IMHO "name" does not imply URI, rather it should be much closer to the concept of a "file name" in a file system. Note, eXist-db does not define "file name" either.
The second and third issues are the return values from
xmldb:create-collection, it is documented as:the path to the new collection if successfully created, otherwise the empty sequence.xs:string, e.g./db/hello%20world. The documentation says it returns a "path", however nowhere is it defined what is meant by "path". Is a path a URI or just a string, i.e. should the return value actually be (a)/db/hello%20world, or (b)/db/hello world? I don't know, but/db/hello%20worldfeels kind of right here. If that is the case, then the function should be documented instead asreturns the database URI of a Collection expressed as a xs:string.The above is just one of many examples that I could formulate. However, before we start fixing things, what we really need to do is work on strict definitions of various concepts used by eXist-db.
These concepts in the context of eXist-db need to be properly defined and we need to consider if they are (1) expressed in a URI encoded or decoded way, and (2) how they are expressed as either or both
xs:anyURIorxs:string.My thoughts...
If a Collection Name is just a name like how a File Name is just a name, then it should be totally valid to create these two distinct collections:
In the second, the actual name is "hello%20world" and is not short for something that was URI encoded. I would expect the above to return:
If we were to transition to Collection Names and Document Names being just names, then eXist-db needs some fixes. At the moment it unnecessarily stores just the Collection Name (not the Collection URI) as an XmldbURI.
If we were to store names rather as strings rather than XmldbURI, then this would require a change in storage-format for
collections.dbxand therefore necessitates a major release.