XML database

Definition

In this article, we are primairly interested in Native XML Databases, “Databases that store XML in "native" form, generally as some variant of the DOM mapped to an underlying data store. This includes the category formerly known as persistent DOM (PDOM) implementations.For data- and document-centric applications.”(R. Bourret, retrieved 17:59, 3 November 2006 (MET))

When do we need an XML database

Ronald Bourret (2006) summarizes the major different XML and databases scenarios:

To store and retrieve the data in data-centric documents, what kind of software you need will depend on how well structured your data is. For highly structured data, such as the white pages in a telephone book, you will need an XML-enabled database that is tuned for data storage, such as a relational or object-oriented database, and some sort of data transfer software. This may be built in to the database (in which case the database is said to be XML-enabled) or might be third-party software, such as middleware or an XML server. If your data is semi-structured, such as the yellow pages in a telephone book or health data, you have two choices. You can try to fit your data into a well-structured database, such as a relational database, or you can store it in a native XML database, which is designed to handle semi-structured data.

(R. Bourret, retrieved 17:59, 3 November 2006 (MET))

Standards

• XQuery 1.0: An XML Query Language (the central document). XQuery can only query (not update).
• There are some other parts, e.g. XML Query Use Cases and XML Query Requirements and XML Query Data Model (W3C Working Draft). XQuery is a full programming language and comes with 2 syntaxes: one humain readable and on in XML ;)
• xqupdate - XQuery Update Facility. This will replace Xupdate (a very informal "standard")
• Xpath 2.0 is used both by XSLT 2.0 and XQuery. XQuery is an extension of XPath 2.0 and both XSLT 2.0 and XQuery 2.0 share the same data model.
• Xupdate can be used to update some XML databases (not updated since 2000). Xupdate looks a lot like XSLT.

Software

Native XML databases

(just some popular free ones)

According to R. Bourret, retrieved 13:38, 24 November 2007 (MET), native XML databases differ from XML-enabled databases in three main ways:

• Native XML databases can preserve physical structure (entity usage, CDATA sections, etc.) as well as comments, PIs, DTDs, etc.
• Native XML databases can store XML documents without knowing their schema (DTD), assuming one even exists.
• The only interface to the data in native XML databases is XML and related technologies (such as XQuery, XPath, the DOM) or an XML-specific API, such as the XML:DB API. XML-enabled databases, on the other hand, offer direct access to the data, such as through ODBC.

List of some XML databases:

• eXist XML database and retrieval engine, an alternatives to popular Xindice. Features: XQuery, XUpdate, XML:DB API, xml-rpc support, REST support, WebDav, PHP API (needs dowloading from CVS), good Cocoon Integration, etc. This is the only one DKS tested in recent month. Perfectly works.
  • A side note for qualitative analysis people: see XML DBs as Research Tools (by darcusblog)

• Xindice is a native XML database. Supports XPath queries and XML:DBXUpdate. Java and has a XML-RPC API plugin. It has grown out from dbXML is a native XML database (JDK 1.3, LGPL licence). See also the Xindice Wiki at Apache for more information, e.g. a pointer to

• Sedna is an open source XML database management system. It is a XML-native system developed from scratch in C/C++ and Scheme. Sedna is a full-featured database system that supports queries, updates, ACID transactions, security, etc. Sedna is designed to be fast, reliable and easy-to-use for production-grade applications. Apache license (free).

• dbXML, GPL license. dbXML is a Native XML Database (NXD). NXDs are databases that store XML using an internalized format for faster overall processing. dbXML was developed using the Java 2 Standard Edition version 1.4. Dead project, became Xindice, but also

• Oracle Berkeley DB XML. Open source, embeddable XML database with XQuery-based access to documents stored in containers and indexed based on their content. Oracle Berkeley DB XML is built on top of Oracle Berkeley DB.

• MonetDB database system with XQuery front-end. Supports XQuery and XUpdate

Other

• DB2XML: A tool for transforming relational databases into XML documents As Servlet or standalone. Xindice is a fork of this (don't know if db2xml development continues).

XIndicePHPAdmin. There exist other PHP libraries, but most projects seem to be inactive. Note: this project seems to stall since sept 2004, see eXist or Sedna)

Discussion

DSchneider believes that native XML databases like eXist "will take off" in our field as soon as there are sufficient PHP APIs. For example eXist is both a database server and a portal written. They are writtin in Java which makes this technology not very accessible to the "bricoleur" world of education....

The advantage of an XML-database is that you can just "stick in" XML contents and then retrieve them with XQuery expressions. Native XML databases are most commonly used to store document-centric documents and that's something we may see a lot in education (e.g. descriptions of pedagogical scenarios, lesson plans, contents other than IMS Content Packaging. Adding new information structures may require writing interfaces to XQuery, but this kind of flexibility comes at a much lower price. Most of these database now support standardized XQuery Update. Lack of an updating standard was a problem that is now almost solved - 21:26, 1 November 2007 (MET)

See some discussion in the DITA article.

Links

• (more needed here)

• XML Database Products by V. Bourret (best resource)

• Yuli Vasiliev (2007), PHP Oracle Web Development: Data processing, Security, Caching, XML, Web Services, and Ajax (Paperback). ISBN-10: 1847193633. Book excepts are available at webreference.com ("XML enabled applications").

• Elliotte Rusty Harold (2007), Native XML Databases, eXQuisite or eXcruciating?, New York PHP Users Group, Presentation slides, HTML pages - HTML 1 page

• XQuery performance on the XMark benchmark, Several DBs compared by MonetDB