Types in programming can help structure your code, remind you of data invariants and push the compiler to detect errors and apply optimizations. A type system should be considered a simple and effective form of program verification.

Data Types in XML specifications are concerned with assigning meaning to sequences of characters -- not with programming. The types thus introduced express some form of structural invariants of XML documents and fragments. This, and not more, is what standards like "Document Type Definition" (DTD)[xml], the more recent XML Schema Definitions (XSD) [xsd1][xsd2] and Relax NG (RNG) [rng] schemata achieve. Less well-known alternatives are schematron and Document Structure Description (DSD) [dsd]. An XML document conforming to such a schema is called valid or schema-valid. For the programmer, the job does not end at data definition, it begins there. And then there are a whole number of XML programs that don't need this datatype business at all.

I believe, it is wrong to impose one perspective and completely neglect the other. Probably, most users of scala.xml are interested in generating (X)HTML: These users need support for almost all details of the XML and XHTML spec, plus some knowledge about browser incompatibilities. The Scala language thus supports cut-and-paste compatibility for XML literals.

On the other hand, there are benefits of using type information. Today there is some static type checking for XQuery and XSLT, but they are somewhat all-or-nothing, forcing the developer to decide whether he wants to live in a typed or an untyped XML world. The scala.xml library keeps all options open: one can manipulate XML without worrying about a schema, but there are ways to convert to or represent some attribute or element text as a Scala/Java integer.

XML programming is a placeholder for many different approaches developers have for XML processing. We will call such a perspective "generic" if it does not depend on any particular XML application (nothing to do with Java generics). A generic approach can deal with XHTML just as it can deal with a markup language for cooking recipes. There are several points of view that can be taken:

Using scala.xml feels somewhere between using some DOM API and having an XML specific language. Besides the literal syntax, there is actually no language support. In Scala, most "features" are realized not as language extensions but as libraries. Even the literal XML syntax is desugared into code that constructs objects -- so it is possible to do express everything that can be expressed in XML literals (and even more) without actually using XML syntax, programmatically.

Thanks go to Martin Odersky for giving me the freedom in designing this library. Also, without the past and present LAMP staff, Scala would not be what it is today. Matthias Zenger, Michel Schinz, Philippe Altherr, Vincent Cremet, Erik Stenman, Gilles Dubochet, Stéphane Micheloud, Lex Spoon, Sean McDirmid, Nikolay Mihaylov, Iulian Dragos. Some of these guys were pretty ardent XML detractors, which is sometimes good as it reminds one that no XML API is a silver-bullet.

Jamie Webb and Jon Pretty of Sygneca gave a lot of feedback and a couple of features were suggested by them. Students that took undergraduate projects helped to weed out bugs and improve performance and usability -- thank you to Simon Barbey, Fatemeh Borran, Susann Bucher, Badr Hejira, Florian Hof, Clément Hongler and Lukas Rytz.

Update: For the latest iteration of this draft's release, Jonas Bonér, David Pollak, David Hall, Michael Fortson deserve thanks for reporting bugs in the code and the document.

The Scala programming language offers a wide range of constructions and library routines that make dynamic XML processing simple and effective. This section contains an overview of the most common ways to construct XML.

/* examples/phonebook/phonebook1.scala */
package phonebook  

object phonebook1 {

  val labPhoneBook = 
    <phonebook>
      <descr>
        This is the <b>phonebook</b> of the 
        <a href="http://acme.org">ACME</a> corporation.
      </descr>
      <entry>
        <name>Burak Emir</name> 
        <phone where="work">+41 21 693 68 67</phone>
      </entry>
    </phonebook>;

  def main(args: Array[String]) =
    Console.println( labPhoneBook )

}

$ scalac -d /tmp examples/xml/phonebook/phonebook1.scala
$ scala -classpath /tmp phonebook.phonebook1
<phonebook>
      <descr>
        This is the <b>phonebook</b> of the
        <a href="http://acme.org">ACME</a> corporation.
      </descr>
      <entry>
        <name>Burak Emir</name>
        <phone where="work">+41 21 693 68 67</phone>
      </entry>
    </phonebook>
    

case class Elem(val prefix: String,          // namespace prefix
                val label: String,           // (local) tag name
                val attributes: MetaData,
                val scope: NamespaceBinding, // namespace bindings
                val child: Node*) extends Node { ... }

/* examples/xml/phonebook/verboseBook.scala */
package phonebook 

object verboseBook {

  import scala.xml.{ UnprefixedAttribute, Elem, Node, Null, Text, TopScope } 

  val pbookVerbose = 
    Elem(null, "phonebook", Null, TopScope,
       Elem(null, "descr", Null, TopScope,
            Text("This is a "), 
            Elem(null, "b", Null, TopScope, Text("sample")),
            Text("description")
          ),
       Elem(null, "entry", Null, TopScope,
            Elem(null, "name", Null, TopScope, Text("Burak Emir")),
            Elem(null, "phone", new UnprefixedAttribute("where","work", Null), TopScope, 
                 Text("+41 21 693 68 67"))
          )
       )

  def main(args: Array[String]) = 
    Console.println( pbookVerbose )
}

$ scalac -d /tmp examples/xml/phonebook/verboseBook.scala
$ scala -classpath /tmp phonebook.verboseBook
<phonebook><descr>This is a <b>sample</b>description</descr><entry><name>Burak Emir</name><phone where="work">+41 21 693 68 67</phone></entry></phonebook>

  def main(args: Array[String]) = 
    Console.println( new PrettyPrinter(80 /*width*/,3 /*indent*/).format(pbookVerbose) )
      

val myElem = Elem(null, "baz", Null, TopScope, computeList(42,"froz"):_* );

val myElem = Elem(null, "baz", Null, TopScope, 
                  Elem(null, "foo", Null, TopScope), 
                  Elem(null, "bar", Null, TopScope) )

/* examples/phonebook/embeddedBook.scala */
package phonebook  

object embeddedBook {

  val company  = <a href="http://acme.org">ACME</a>
  val first    = "Burak"
  val last     = "Emir"
  val location = "work"

  val embBook = 
    <phonebook>
      <descr>
        This is the <b>phonebook</b> of the 
        {company} corporation.
      </descr>
      <entry>
        <name>{ first+" "+last }</name> 
        <phone where={ location }>+41 21 693 68 {val x = 60 + 7; x}</phone>
      </entry>
    </phonebook>;

  def main(args: Array[String]) =
    Console.println( embBook )

}

  if(cond)
    <foo bar="pizza">{ /*lots of code*/ }</foo>
  else
    <foo>{ /*lots of code*/ }</foo>

    <foo bar={if (cond) "pizza" else null}>{ /*lots of code*/ }</foo>

    val z = if (cond) { Some(Text("pizza")) } else { None }
    <foo bar={z}>{ /*lots of code*/ }</foo>

Although the above is sufficient for most purposes, there are a couple of other nodes that can be used.

<foo name= "s&uuml;ss" life={Atom(42)}>

Elem(null, 
     foo,  
     new UnprefixedAttribute("name",List(Text("s"),EntityRef("uuml"),Text("ss")), 
       new UnprefixedAttribute("life", Atom(42), Null), TopScope)
          

Scala provides pattern matching to search and decompose sequences. Pattern matching can also be used to decompose XML.

For instance to find out whether a variable contains an "entry" element which has as last child a "foo" with no children, this pattern will do:

x match {
          case Elem(_,"entry", _, _, _*, Elem(_, "foo", _)) => true
          case _ => false
}

This also works using XML syntax:

x match {
          case <entry>{ _* }<foo/></entry> => true
          case _ => false
}

However, there is no support for testing presence or values of attributes. This can be achieved using guards, for instance like in the following example

x match {
          case link @ <a>{ _* }</a> if link.attribute("href").isEmpty => "href missing"
}

The Scala XML API takes a functional approach to representing data, eschewing imperative updates where possible. Since nodes as used by the library are immutable, updating an XML tree can a bit verbose, as the XML tree has to be copied. Here is an example how this could be done.

/* examples/xml/phonebook/phonebook2.scala */
package phonebook;

object phonebook2 {

  import scala.xml.Node

  /** adds an entry to a phonebook */
  def add( p: Node, newEntry: Node ): Node = p match {

      case <phonebook>{ ch @ _* }</phonebook> => 

        <phonebook>{ ch }{ newEntry }</phonebook>
  }

  val pb2 = 
    add( phonebook1.labPhoneBook, 
         <entry>
           <name>Kim</name> 
           <phone where="work">+41 21 111 11 11</phone>
         </entry> );

  def main( args: Array[String] ) = 
    Console.println( pb2 )
}

This code will throw a MatchError in add exception if the node does not have phonebook tag. It is also possible to express it using only method calls:

          def add( p: Node, e: Node ) = Elem(null, p.label, Null, TopScope, (p.child ++ e):_*)
        

Here we assume that our element representing a phonebook will never have a namespace prefix (null), never have attributes (Null) and never define namespace bindings (TopScope). Without these assumptions, we would have copied p.prefix, p.attributes and p.scope over to the new element as well. The _* ("sequence escape") has been explained before: see Passing sequences to Elem.

Changing the phone number of an entry is similar. First we lookup an entry by traversing the tree and and copying it. Then we provide an updated copy of the element we wish to change.

package phonebook;

object phonebook3 {

  import scala.xml.{Elem, Node, Text} ;
  import scala.xml.PrettyPrinter ;
  import Node.NoAttributes ;

  /* this method "changes" (returns an updated copy) of the phonebook when the
   *   entry for Name exists. If it has an attribute "where" whose value is equal to the
   *   parameter Where, it is changed, otherwise, it is added.
   */
  def change ( phonebook:Node, Name:String, Where:String, newPhone:String ) = {

    /** this nested function walks through tree, and returns an updated copy of it  */
    def copyOrChange ( ch: Iterator[Node] ) = {

      import xml.Utility.{trim,trimProper} //removes whitespace nodes, which are annoying in matches

      for( val c <- ch ) yield 
        trimProper(c) match {

          // if the node is the particular entry we are looking for, return an updated copy

          case x @ <entry><name>{ Text(Name) }</name>{ ch1 @ _* }</entry> => 

            var updated = false;
            val ch2 = for(val c <- ch1) yield c match { // does it have the phone number?

              case y @ <phone>{ _* }</phone> if y \ "@where" == Where => 
                updated = true
                <phone where={ Where }>{ newPhone }</phone>
              
              case y => y
              
            }
            if( !updated ) { // no, so we add as first entry
            
              <entry>
                <name>{ Name }</name>
                <phone where={ Where }>{ newPhone }</phone>
                { ch1 }
              </entry>
              
            } else {         // yes, and we changed it as we should
              
              <entry>
                { ch2 }
              </entry>
        
            } 
          // end case x @ <entry>...
          
          // other entries are copied without changing them

          case x =>           
            x
          
        }
    } ; // for ... yield ... returns an Iterator[Node]
    
    // decompose phonebook, apply updates
    phonebook match {
      case <phonebook>{ ch @ _* }</phonebook> =>
        <phonebook>{ copyOrChange( ch.elements ) }</phonebook>
    }
    
  }

  val pb2 = 
    change( phonebook1.labPhoneBook, "John", "work", "+41 55 555 55 55" );

  val pp = new PrettyPrinter( 80, 5 );

  def main( args:Array[String] ) = {
    Console.println("---before---");
    Console.println( pp.format( phonebook1.labPhoneBook ));
    Console.println("---after---");
    Console.println( pp.format( pb2 ));
  }
}

Namespaces [names1.0][names1.1] have been introduced into extensible markup long after the XML specicifaction was out. The intention is to provide a means of 'packaging' related names by associating them with a URL. The association happens indirectly by (1) binding URIs to prefixes and (2) prefixing names using the syntax 'prefix:localname', i.e. using the colon as a separator. Consequently, the colon is not a part of names anymore.

To avoid clutteredness, the standard allows a 'default namespace' to be declared, which implicitly associates unprefixed names with a certain URI. Finally it is possible to undeclare namespaces by binding them to the empty prefix. (v1.0 only allowed to undeclare the default namespaces, but in v1.1 this has been generalized to any prefix).

How does this look in Scala? Namespace bindings are treated in a class aptly named NamespaceBinding, which is a linked list of prefix-URI pairs. A default namespace is synonymous with a namespace for the null prefix (not the empty string), and undeclaring a namespacebinding is done by assigning the empty string as URI. The TopScope is the most common top-level scope, the empty prefix-URI mapping that does not contain any binding.

Here is an example what the compiler does with scala.xml.NamespaceBinding. Assuming we had a internal variable $scope containing the active bindings at each element. Then for the following fragment

val foobar = <foo:bar xmlns:foo="http://foo.com" foo:key="value" xmlns="urn:default" attr="42"><a/></foo:bar>

the compiler has to take the following steps to updates the scope, translating everything roughly into:

val foobar = { // add bindings to scope
  scope = new NamespaceBinding(null,  "urn:default", 
             NamespaceBinding("foo", "http://foo.com",  scope))

  // make attributes
  val md = new UnprefixedAttribute("attr","42", 
              new PrefixedAttribute("foo","key","value", Null))

  // make element
  Elem("foo","bar", md, scope, Elem(null, "a", Null, scope))
}
  

The element labeled bar uses a prefix which tells us it is in the namespace http://foo.com. The element a is nested under bar, this is affected by the same namespace bindings. It is in the namespace urn:default.

Namespace binding is properly scoped over the child nodes: Unless a descendant undeclares a prefix, the prefix is bound to URI according to the bindings defined for the parent. As can be seen, namespace bindings are treated differently from regular attributes -- this seems a good compromise since they are shared, have different properties and there is an important class of users that is simply not concerned with namespaces. The library is design to handle namespace bindings by itself, and where namespace manipulations are needed, they are effected on the scope members and NamespaceBinding classes.

Attributes without a prefix are not implicitly put in the same namespace as the element in which they occur. This is the reason why there is UnprefixedAttribute and a PrefixedAttribute class. UnprefixedAttributes have no namespace at all.

Scala has a XML parser of its own, which can be invoked like this

import scala.xml.parsing.ConstructingParser

val p = ConstructingParser.fromFile(file, true /*preserve whitespace*/)
val d: xml.Document = p.document
  

The advantages of this parser is that the developer has more fine-grained control over what to parse. It is for instance possible to parse a sequence of elements from a stream (the XML spec allows only one), or to obtain the entity declarations from the internal subset of the DTD.

The native XML parser can also be used for pull parsing. An experimental API is accessible via scala.xml.pull.XMLEventReader. You need to provide a scala.io.Source, just like for the constructing parser.

The abstract superclass of all XML nodes as represented in the Scala library. Nodes have an optional prefix (null = no prefix), a namespace binding scope, a list of metadata (attributes), and a sequence of children. A node can be considered as a singleton sequence containing the node, because it inherits from NodeSeq.

Sequences of nodes are pretty common in XML processing. The main use of this class is to add XPath methods \ and \\ to any sequence of nodes, regardless of its concrete representation. It is a wrapper class, which gets automatically created by means of Scala's view mechanism.

A class implementing scala.xml.Node with a case class. XML literals embedded in Scala code will get turned into Elem instances. Also, most default parsing factories will produce Elem instances. By contrast, most library routines (like e.g. the PrettyPrinter) expect instances of Node, so it is possible to call them with custom XML representations.

To store data values like ints and dates.

To represent entity references. It is possible to output entity declarations using the classes in scala.xml.dtd.

The abstract superclass of attribute nodes. Attributes are realized as an immutable linked list. Since the attribute order does not matter in XML, default parser factories may actually change (typically reverse) the order when they parse XML.

This object is used to `ground' linked attribute lists. It is also the representation of empty attribute lists.

A prefixed attribute has a prefix, a name, a value and a pointer to the tail of the attribute list. It answers to getValue(uri,scope,key) calls with its value if the its own prefix matches the uri in the given scope (typically the scope of the parent element). It will not answer getValue(key) calls, because the Namespaces spec considers it distinct from an unprefixed attribute.

An unprefixed attribute has a name, a value and a pointer to the tail of the attribute list. It answers getValue(key) calls, but not the namespace aware ones describe above.

This class is for representing namespace bindings using a linked list of namespace binding nodes.

This class is used to `ground' a linked list of namespace binding nodes. It also stands for a top-level scope in which no namespaces are bound.

This tool is an adaption of Eliotte Rusty Harold's SAXXIncluder to Scala. It builds on top of the relevant JAVA API classes and was crucial for including the code samples in this document.

At this point some information (the Scaladoc description) is available at url xinc homepage

Despite great APIs, data represented in XML tends to be converted to and from object representations. This task is called data binding. It can in prinicple be coded manually if the data representations are intricate. But often, conversion has to bridge fairly straightforward XML types and fairly basic "pure data" classes. The latter scenario is a case for automation.

For the sake of an example consider the following way to represent bugreports

<bugReport id="42">
  <dateSubmitted>2003-06-25</dateSubmitted>
  <status>     fixed   </status>
  <submitter>  Matthias</submitter>
  <assignedTo> Michel  </assignedTo>
  <code>        ...    </code>
  <whatHappened>...</whatHappened>
  <whatExpected>...</whatExpected>
</bugreport>

<!ELEMENT bugReport (dateSubmitted,
                     status,
                     submitter,
                     assignedTo,
                     code,
                     whatHappened,
                     whatExpected)>
<!ELEMENT dateSubmitted #PCDATA>
<!ELEMENT status        #PCDATA>
<!ELEMENT submitter     #PCDATA>
<!ELEMENT assignedTo    #PCDATA>
<!ELEMENT code          #PCDATA>
<!ELEMENT whatHappened  #PCDATA>
<!ELEMENT whatExpected  #PCDATA>

There are many scenarios, where we would like to programmatically manipulate bug reports in ways that cannot be handled by XML tools. We might want to store and retrieve them in a relational database, access and compile the source in the code element, or notify the submitter of changes by email.

Using the data binding tool schema2src it is possible to generate the following classes from the DTD above. We can invoke the schema2src with its DTD module in the following way

This will result in a sourcefile being generated. The sourcefile contains mainly one object, one cases class and one method definition for each element declaration found in the DTD.

object BugReportDefs {
  import scala.xml.{MetaData, Node};

  object bugReport { def validate(atts: MetaData, child: Node*)    ... }
  case class bugReport    (atts: MetaData, child: Node*)         { ... }
  def bugReport(s:String) = new bugReport(s);

  object dateSubmitted { def validate(atts: MetaData, child: Node*) ...}
  case class dateSubmitted(atts: MetaData, child: Node*)          { ... }
  def dateSubmitted(s:String) = new dateSubmitted(s);

  ...
}

Importing the types from BugReportDefs has several benefits for the programmer working with bug reports. She can