5 Functions on strings

This section specifies functions and operators on the [XML Schema Part 2: Datatypes Second Edition] xs:string datatype and the datatypes derived from it.

5.1 String types

The operators described in this section are defined on the following types. Each type whose name is indented is derived from the type whose name appears nearest above with one less level of indentation.

xs:string          
xs:normalizedString        
xs:token      
xs:language    
xs:NMTOKEN    
xs:Name    
xs:NCName  
xs:ID
xs:IDREF
xs:ENTITY

They also apply to user-defined types derived by restriction from the above types.

5.2 Functions to assemble and disassemble strings

Function Meaning
fn:codepoints-to-string Returns an xs:string whose characters have supplied ·codepoints·.
fn:string-to-codepoints Returns the sequence of ·codepoints· that constitute an xs:string value.

5.2.1 fn:codepoints-to-string

Summary

Returns an xs:string whose characters have supplied ·codepoints·.

Signature
fn:codepoints-to-string(
$values as xs:integer*
) as xs:string
Properties

This function is ·deterministic·, ·context-independent·, and ·focus-independent·.

Rules

The function returns the string made up from the ·characters· whose Unicode ·codepoints· are supplied in $values. This will be the zero-length string if $values is the empty sequence.

Error Conditions

A dynamic error is raised [err:FOCH0001] if any of the codepoints in $values is not a permitted XML character.

Examples

The expression fn:codepoints-to-string((66, 65, 67, 72)) returns "BACH".

The expression fn:codepoints-to-string((2309, 2358, 2378, 2325)) returns "अशॊक".

The expression fn:codepoints-to-string(()) returns "".

The expression fn:codepoints-to-string(0) raises error FOCH0001.

5.2.2 fn:string-to-codepoints

Summary

Returns the sequence of ·codepoints· that constitute an xs:string value.

Signature
fn:string-to-codepoints(
$value as xs:string?
) as xs:integer*
Properties

This function is ·deterministic·, ·context-independent·, and ·focus-independent·.

Rules

The function returns a sequence of integers, each integer being the Unicode ·codepoint· of the corresponding ·character· in $value.

If $value is a zero-length string or the empty sequence, the function returns the empty sequence.

Examples

The expression fn:string-to-codepoints("Thérèse") returns (84, 104, 233, 114, 232, 115, 101).

5.3 Comparison of strings

Function Meaning
fn:compare Returns -1, 0, or 1, depending on whether $value1 collates before, equal to, or after $value2 according to the rules of a selected collation.
fn:codepoint-equal Returns true if two strings are equal, considered codepoint-by-codepoint.
fn:collation-key Given a string value and a collation, generates an internal value called a collation key, with the property that the matching and ordering of collation keys reflects the matching and ordering of strings under the specified collation.
fn:contains-token Determines whether or not any of the supplied strings, when tokenized at whitespace boundaries, contains the supplied token, under the rules of the supplied collation.

5.3.1 Collations

A collation is a specification of the manner in which ·strings· are compared and, by extension, ordered. When values whose type is xs:string or a type derived from xs:string are compared (or, equivalently, sorted), the comparisons are inherently performed according to some collation (even if that collation is defined entirely on codepoint values). The [Character Model for the World Wide Web 1.0: Fundamentals] observes that some applications may require different comparison and ordering behaviors than other applications. Similarly, some users having particular linguistic expectations may require different behaviors than other users. Consequently, the collation must be taken into account when comparing strings in any context. Several functions in this and the following section make use of a collation.

Collations can indicate that two different codepoints are, in fact, equal for comparison purposes (e.g., "v" and "w" are considered equivalent in some Swedish collations). Strings can be compared codepoint-by-codepoint or in a linguistically appropriate manner, as defined by the collation.

Some collations, especially those based on the Unicode Collation Algorithm (see [UTS #10]) can be "tailored" for various purposes. This document does not discuss such tailoring, nor does it provide a mechanism to perform tailoring. Instead, it assumes that the collation argument to the various functions below is a tailored and named collation.

The ·Unicode codepoint collation· is a collation available in every implementation, which sorts based on codepoint values. For further details see 5.3.2 The Unicode Codepoint Collation.

Collations may or may not perform Unicode normalization on strings before comparing them.

This specification assumes that collations are named and that the collation name may be provided as an argument to string functions. Functions that allow specification of a collation do so with an argument whose type is xs:string but whose lexical form must conform to an xs:anyURI. If the collation is specified using a relative URI reference, it is resolved relative to the value of the static base URI property from the static context. This specification also defines the manner in which a default collation is determined if the collation argument is not specified in calls of functions that use a collation but allow it to be omitted.

This specification does not define whether or not the collation URI is dereferenced. The collation URI may be an abstract identifier, or it may refer to an actual resource describing the collation. If it refers to a resource, this specification does not define the nature of that resource. One possible candidate is that the resource is a locale description expressed using the Locale Data Markup Language: see [UTS #35].

Functions such as fn:compare and fn:max that compare xs:string values use a single collation URI to identify all aspects of the collation rules. This means that any parameters such as the strength of the collation must be specified as part of the collation URI. For example, suppose there is a collation http://www.example.com/collations/French that refers to a French collation that compares on the basis of base characters. Collations that use the same basic rules, but with higher strengths, for example, base characters and accents, or base characters, accents and case, would need to be given different names, say http://www.example.com/collations/French1 and http://www.example.com/collations/French2. Note that some specifications use the term collation to refer to an algorithm that can be parameterized, but in this specification, each possible parameterization is considered to be a distinct collation.

The XQuery/XPath static context includes a provision for a default collation that can be used for string comparisons and ordering operations. See the description of the static context in Section 2.1.1 Static Context XP31. If the default collation is not specified by the user or the system, the default collation is the ·Unicode codepoint collation·.

Note:

XML allows elements to specify the xml:lang attribute to indicate the language associated with the content of such an element. This specification does not use xml:lang to identify the default collation because using xml:lang does not produce desired effects when the two strings to be compared have different xml:lang values or when a string is multilingual.

5.3.2 The Unicode Codepoint Collation

[Definition] The collation URI http://www.w3.org/2005/xpath-functions/collation/codepoint identifies a collation which must be recognized by every implementation: it is referred to as the Unicode codepoint collation (not to be confused with the Unicode collation algorithm).

The Unicode codepoint collation does not perform any normalization on the supplied strings.

The collation is defined as follows. Each of the two strings is converted to a sequence of integers using the fn:string-to-codepoints function. These two sequences $A and $B are then compared as follows:

  • If both sequences are empty, the strings are equal.

  • If one sequence is empty and the other is not, then the string corresponding to the empty sequence is less than the other string.

  • If the first integer in $A is less than the first integer in $B, then the string corresponding to $A is less than the string corresponding to $B.

  • If the first integer in $A is greater than the first integer in $B, then the string corresponding to $A is greater than the string corresponding to $B.

  • Otherwise (the first pair of integers are equal), the result is obtained by applying the same rules recursively to fn:tail($A) and fn:tail($B)

Note:

While the Unicode codepoint collation does not produce results suitable for quality publishing of printed indexes or directories, it is adequate for many purposes where a restricted alphabet is used, such as sorting of vehicle registrations.

5.3.3 The Unicode Collation Algorithm

This specification defines a family of collation URIs representing tailorings of the Unicode Collation Algorithm (UCA) as defined in [UTS #10]. The parameters used for tailoring the UCA are based on the parameters defined in the Locale Data Markup Language (LDML), defined in [UTS #35].

This family of URIs use the scheme and path http://www.w3.org/2013/collation/UCA followed by an optional query part. The query part, if present, consists of a question mark followed by a sequence of zero or more semicolon-separated parameters. Each parameter is a keyword-value pair, the keyword and value being separated by an equals sign.

All implementations must recognize URIs in this family in the collation argument of functions that take a collation argument.

If the fallback parameter is present with the value no, then the implementation must either use a collation that conforms with the rules in the Unicode specifications for the requested tailoring, or fail with a static or dynamic error indicating that it does not provide the collation (the error code should be the same as if the collation URI were not recognized). If the fallback parameter is omitted or takes the value yes, and if the collation URI is well-formed according to the rules in this section, then the implementation must accept the collation URI, and should use the available collation that most closely reflects the user's intentions. For example, if the collation URI requested is http://www.w3.org/2013/collation/UCA?lang=se;fallback=yes and the implementation does not include a fully conformant version of the UCA tailored for Swedish, then it may choose to use a Swedish collation that is known to differ from the UCA definition, or one whose conformance has not been established. It might even, as a last resort, fall back to using codepoint collation.

If two query parameters use the same keyword then the last one wins. If a query parameter uses a keyword or value which is not defined in this specification then the meaning is ·implementation-defined·. If the implementation recognizes the meaning of the keyword and value then it should interpret it accordingly; if it does not recognize the keyword or value then if the fallback parameter is present with the value no it should reject the collation as unsupported, otherwise it should ignore the unrecognized parameter.

The following query parameters are defined. If any parameter is absent, the default is ·implementation-defined· except where otherwise stated. The meaning given for each parameter is non-normative; the normative specification is found in [UTS #35].

Keyword Values Meaning
fallback yes | no (default yes) Determines whether the processor uses a fallback collation if a conformant collation is not available.
lang language code: a string in the lexical space of xs:language. The language whose collation conventions are to be used.
version string The version number of the UCA to be used.
strength (default tertiary) primary | secondary | tertiary | quaternary | identical, or 1|2|3|4|5 as synonyms The collation strength as defined in UCA. Primary strength takes only the base form of the character into account (so A=a=Ä=ä); secondary strength ignores case but considers accents and diacritics as significant (so A=a and Ä=ä but ä≠a); tertiary considers case as significant (A≠a≠Ä≠ä); quaternary considers spaces and punctuation that would otherwise be ignored (for example data-base=database).
maxVariable space | punct | symbol | currency (default punct) Indicates that all characters in the specified group and earlier groups are treated as "noise" characters to be handled as defined by the alternate parameter. For example, maxVariable=punct indicates that characters classified as whitespace or punctuation get this treatment.
alternate non-ignorable | shifted | blanked (default non-ignorable) Controls the handling of characters such as spaces and hyphens; specifically, the "noise" characters in the groups selected by the maxVariable parameter. The value non-ignorable indicates that such characters are treated as distinct at the primary level (so data base sorts before datatype); shifted indicates that they are used to differentiate two strings only at the quaternary level, and blanked indicates that they are taken into account only at the identical level.
backwards yes | no (default no) The value backwards=yes indicates that the last accent in the string is the most significant.
normalization yes | no (default no) Indicates whether strings are converted to normalization form D.
caseLevel yes | no (default no) When used with primary strength, setting caseLevel=yes has the effect of ignoring accents while taking account of case.
caseFirst upper | lower Indicates whether upper-case precedes lower-case or vice versa.
numeric yes | no (default no) When numeric=yes is specified, a sequence of consecutive digits is interpreted as a number, for example chap2 sorts before chap12.
reorder a comma-separated sequence of reorder codes, where a reorder code is one of space, punct, symbol, currency, digit, or a four-letter script code defined in [ISO 15924 Register], the register of scripts maintained by the Unicode Consortium in its capacity as registration authority for [ISO 15924]. Determines the relative ordering of text in different scripts; for example the value digit,Grek,Latn indicates that digits precede Greek letters, which precede Latin letters.

Note:

This list excludes parameters that are inconvenient to express in a URI, or that are applicable only to substring matching.

5.3.4 The HTML ASCII Case-Insensitive Collation

The collation URI http://www.w3.org/2005/xpath-functions/collation/html-ascii-case-insensitive must be recognized by every implementation. It is used to refer to the HTML ASCII case-insensitive collation as defined in [HTML5: A vocabulary and associated APIs for HTML and XHTML] (section 2.5, Case sensitivity and string comparison). It is used, for example, when matching HTML class attribute values.

Note:

The definition of the collation is paraphrased here for convenience:

Comparing two strings in an ASCII case-insensitive manner means comparing them exactly, codepoint for codepoint, except that the characters in the range x41 to x5A (A-Z) and the corresponding characters in the range x61 to x7A (a-z) are considered to also match.

HTML5 defines the semantics of equality matching using this collation; it does not define rules for ordering. If the collation is used for ordering, the results are ·implementation-defined·. The collation supports collation units and can therefore be used with functions such as fn:contains; each Unicode codepoint is a single collation unit.

5.3.5 Choosing a collation

Many functions have two signatures, where one signature includes a $collation argument and the other omits this argument.

The collation to use for these functions is determined by the following rules:

  1. If the function specifies an explicit collation, CollationA (e.g., if the optional collation argument is specified in a call of the fn:compare function), then:

    • If CollationA is supported by the implementation, then CollationA is used.

    • Otherwise, a dynamic error is raised [err:FOCH0002].

  2. If no collation is explicitly specified for the function and the default collation in the XQuery/XPath static context is CollationB, then:

    • If CollationB is supported by the implementation, then CollationB is used.

    • Otherwise, a dynamic error is raised [err:FOCH0002].

Note:

Because the set of collations that are supported is ·implementation-defined·, an implementation has the option to support all collation URIs, in which case it will never raise this error.

If the value of the collation argument is a relative URI reference, it is resolved against the base-URI from the static context. If it is a relative URI reference and cannot be resolved, perhaps because the base-URI property in the static context is absent, a dynamic error is raised [err:FOCH0002].

Note:

There is no explicit requirement that the string used as a collation URI be a valid URI. Implementations will in many cases reject such strings on the grounds that do not identify a supported collation; they may also cause an error if they cannot be resolved against the static base URI.

5.3.6 fn:compare

Summary

Returns -1, 0, or 1, depending on whether $value1 collates before, equal to, or after $value2 according to the rules of a selected collation.

Signatures
fn:compare(
$value1 as xs:string?,
$value2 as xs:string?
) as xs:integer?
fn:compare(
$value1 as xs:string?,
$value2 as xs:string?,
$collation as xs:string
) as xs:integer?
Properties

The two-argument form of this function is ·deterministic·, ·context-dependent·, and ·focus-independent·. It depends on collations.

The three-argument form of this function is ·deterministic·, ·context-dependent·, and ·focus-independent·. It depends on collations, and static base URI.

Rules

Returns -1, 0, or 1, depending on whether $value1 is respectively less than, equal to, or greater than $value2, according to the rules of the collation that is used.

The collation used by this function is determined according to the rules in 5.3.5 Choosing a collation.

If either $value1 or $value2 is the empty sequence, the function returns the empty sequence.

This function, called with the first signature, defines the semantics of the "eq", "ne", "gt", "lt", "le" and "ge" operators on xs:string values.

Examples

The expression fn:compare('abc', 'abc') returns 0.

The expression fn:compare('Strasse', 'Straße') returns 0. (Assuming the default collation includes provisions that equate "ss" and the (German) character "ß" ("sharp-s"). Otherwise, the returned value depends on the semantics of the default collation.)

The expression fn:compare('Strasse', 'Straße', 'http://www.w3.org/2013/collation/UCA?lang=de;strength=primary') returns 0. (The specified collation equates "ss" and the (German) character "ß" ("sharp-s").)

The expression fn:compare('Strassen', 'Straße') returns 1. (Assuming the default collation includes provisions that treat differences between "ss" and the (German) character "ß" ("sharp-s") with less strength than the differences between the base characters, such as the final "n". ).

5.3.7 fn:codepoint-equal

Summary

Returns true if two strings are equal, considered codepoint-by-codepoint.

Signature
fn:codepoint-equal(
$value1 as xs:string?,
$value2 as xs:string?
) as xs:boolean?
Properties

This function is ·deterministic·, ·context-independent·, and ·focus-independent·.

Rules

If either argument is the empty sequence, the function returns the empty sequence.

Otherwise, the function returns true or false depending on whether $value1 is equal to $value2, according to the Unicode codepoint collation (http://www.w3.org/2005/xpath-functions/collation/codepoint).

Notes

This function allows xs:anyURI values to be compared without having to specify the Unicode codepoint collation.

Examples

The expression fn:codepoint-equal("abcd", "abcd") returns true().

The expression fn:codepoint-equal("abcd", "abcd ") returns false().

The expression fn:codepoint-equal("", "") returns true().

The expression fn:codepoint-equal("", ()) returns ().

The expression fn:codepoint-equal((), ()) returns ().

5.3.8 fn:collation-key

Summary

Given a string value and a collation, generates an internal value called a collation key, with the property that the matching and ordering of collation keys reflects the matching and ordering of strings under the specified collation.

Signatures
fn:collation-key(
$value as xs:string
) as xs:base64Binary
fn:collation-key(
$value as xs:string,
$collation as xs:string
) as xs:base64Binary
Properties

This function is ·deterministic·, ·context-dependent·, and ·focus-independent·. It depends on collations.

Rules

Calling the one-argument version of this function is equivalent to calling the two-argument version supplying the default collation as the second argument.

The function returns an ·implementation-dependent· value with the property that, for any two strings $K1 and $K2:

  • collation-key($K1, $C) eq collation-key($K2, $C) if and only if compare($K1, $K2, $C) eq 0

  • collation-key($K1, $C) lt collation-key($K2, $C) if and only if compare($K1, $K2, $C) lt 0

The collation used by this function is determined according to the rules in 5.3.5 Choosing a collation. Collation keys are defined as xs:base64Binary values to ensure unambiguous and context-free comparison semantics.

An implementation is free to generate a collation key in any convenient way provided that it always generates the same collation key for two strings that are equal under the collation, and different collation keys for strings that are not equal. This holds only within a single ·execution scope·; an implementation is under no obligation to generate the same collation keys during a subsequent unrelated query or transformation.

It is possible to define collations that do not have the ability to generate collation keys. Supplying such a collation will cause the function to fail. The ability to generate collation keys is an ·implementation-defined· property of the collation.

Error Conditions

An error is raised [err:FOCH0004] if the specified collation does not support the generation of collation keys.

Notes

The function is provided primarily for use with maps. If a map is required where codepoint equality is inappropriate for comparing keys, then a common technique is to normalize the key so that equality matching becomes feasible. There are many ways keys can be normalized, for example by use of functions such as fn:upper-case, fn:lower-case, fn:normalize-space, or fn:normalize-unicode, but this function provides a way of normalizing them according to the rules of a specified collation. For example, if the collation ignores accents, then the function will generate the same collation key for two input strings that differ only in their use of accents.

The result of the function is defined to be an xs:base64Binary value. Binary values are chosen because they have unambiguous and context-free comparison semantics, because the value space is unbounded, and because the ordering rules are such that between any two values in the ordered value space, an arbitrary number of further values can be interpolated. The choice between xs:base64Binary and xs:hexBinary is arbitrary; the only operation that behaves differently between the two binary data types is conversion to/from a string, and this operation is not one that is normally required for effective use of collation keys.

For collations based on the Unicode Collation Algorithm, an algorithm for computing collation keys is provided in [UTS #10]. Implementations are not required to use this algorithm.

The fact that collation keys are ordered can be exploited in XQuery, whose order by clause does not allow the collation to be selected dynamically. This restriction can be circumvented by rewriting the clause order by $e/@key collation "URI" as order by fn:collation-key($e/@key, $collation), where $collation allows the collation to be chosen dynamically.

Note that xs:base64Binary becomes an ordered type in XPath 3.1, making binary collation keys possible.

Examples
let $C := 'http://www.w3.org/2013/collation/UCA?strength=primary'

The expression map:merge((map{collation-key("A", $C):1}, map{collation-key("a", $C):2}), map{"duplicates":"use-last"})(collation-key("A", $C)) returns 2. (Given that the keys of the two entries are equal under the rules of the chosen collation, only one of the entries can appear in the result; the one that is chosen is the one from the last map in the input sequence.)

The expression let $M := map{collation-key("A", $C):1, collation-key("B", $C):2} return $M(collation-key("a", $C)) returns 1. (The strings "A" and "a" have the same collation key under this collation.)

As the above examples illustrate, it is important that when the collation-key function is used to add entries to a map, then it must also be used when retrieving entries from the map. This process can be made less error-prone by encapsulating the map within a function: function($k) {$M(collation-key($k, $collation)}.

5.3.9 fn:contains-token

Summary

Determines whether or not any of the supplied strings, when tokenized at whitespace boundaries, contains the supplied token, under the rules of the supplied collation.

Signatures
fn:contains-token(
$value as xs:string*,
$token as xs:string
) as xs:boolean
fn:contains-token(
$value as xs:string*,
$token as xs:string,
$collation as xs:string
) as xs:boolean
Properties

The two-argument form of this function is ·deterministic·, ·context-dependent·, and ·focus-independent·. It depends on collations.

The three-argument form of this function is ·deterministic·, ·context-dependent·, and ·focus-independent·. It depends on collations, and static base URI.

Rules

If $value is the empty sequence, the function returns false.

Leading and trailing whitespace is trimmed from $token. If the trimmed value of $token is a zero-length string, the function returns false.

The collation used by this function is determined according to the rules in 5.3.5 Choosing a collation.

The function returns true if and only if there is string in $value which, after tokenizing at whitespace boundaries, contains a token that is equal to the trimmed value of $token under the rules of the selected collation.

That is, the function returns the value of the expression:

some $t in $input!fn:tokenize(.) satisfies 
                 compare($t, fn:replace($token, '^\s*|\s*$', ''), $collation) eq 0)
Notes

Interior whitespace within $token will cause the function to return false, unless such whitespace is ignored by the selected collation.

This function can be used for processing space-separated attribute values (for example, the XHTML and DITA class attribute), where one often needs to test for the presence of a single token in a space-separated list. The function is designed to work both when the attribute has been validated against an XSD list type, and when it appears as a single untyped string. It differs from the HTML 5 definition in that HTML 5 recognizes form feed (x0C) as a separator. To reproduce the HTML token matching behavior, the HTML ASCII case-insensitive collation should be used: see 5.3.4 The HTML ASCII Case-Insensitive Collation.

Examples

The expression fn:contains-token("red green blue ", "red") returns true().

The expression fn:contains-token(("red", "green", "blue"), " red ") returns true().

The expression fn:contains-token("red, green, blue", "red") returns false().

The expression fn:contains-token("red green blue", "RED", "http://www.w3.org/2005/xpath-functions/collation/html-ascii-case-insensitive") returns true().

5.4 Functions on string values

The following functions are defined on values of type xs:string and types derived from it.

Function Meaning
fn:characters Splits the supplied string into a sequence of single-character strings.
fn:concat Returns the concatenation of the string values of the arguments.
fn:string-join Returns a string created by concatenating the items in a sequence, with a defined separator between adjacent items.
fn:substring Returns the part of $value beginning at the position indicated by $start and continuing for the number of ·characters· indicated by $length.
fn:string-length Returns the number of ·characters· in a string.
fn:normalize-space Returns $value with leading and trailing whitespace removed, and sequences of internal whitespace reduced to a single space character.
fn:normalize-unicode Returns $value after applying Unicode normalization.
fn:upper-case Converts a string to upper case.
fn:lower-case Converts a string to lower case.
fn:translate Returns $value modified by replacing or removing individual characters.

Notes:

When the above operators and functions are applied to datatypes derived from xs:string, they are guaranteed to return values that are instances of xs:string, but the value might or might not be an instance of the particular subtype of xs:string to which they were applied.

The strings returned by fn:concat and fn:string-join are not guaranteed to be normalized. But see note in fn:concat.

5.4.1 fn:characters

Summary

Splits the supplied string into a sequence of single-character strings.

Signature
fn:characters(
$value as xs:string?
) as xs:string*
Properties

This function is ·deterministic·, ·context-independent·, and ·focus-independent·.

Rules

The function returns a sequence of strings, each string having length 1, containing the corresponding ·character· in $value.

If $value is a zero-length string or the empty sequence, the function returns the empty sequence.

More formally, the function returns the result of the expression fn:string-to-codepoints($value) ! fn:codepoints-to-string(.)

Examples

The expression fn:characters("Thérèse") returns ("T", "h", "é", "r", "è", "s", "e").

The expression fn:characters("") returns ().

The expression fn:characters(()) returns ().

The expression fn:characters("Banana") => fn:index-of("a") returns (2, 4, 6).

The expression fn:characters("stretch") => fn:string-join("-") returns "s-t-r-e-t-c-h".

The expression "Banana" => fn:characters() => fn:reverse() => fn:string-join() returns "ananaB".

History

New in 4.0. Accepted 2022-09-20, subject to improving the description.

5.4.2 fn:concat

Summary

Returns the concatenation of the string values of the arguments.

Operator Mapping

The two-argument form of this function defines the semantics of the "||" operator.

Signature
fn:concat(
$value1 as xs:anyAtomicType?,
$value2 as xs:anyAtomicType?,
$... as xs:anyAtomicType?
) as xs:string
Properties

This function is ·deterministic·, ·context-independent·, and ·focus-independent·.

Rules

This function accepts two or more xs:anyAtomicType arguments and casts each one to xs:string. The function returns the xs:string that is the concatenation of the values of its arguments after conversion. If any argument is the empty sequence, that argument is treated as the zero-length string.

The fn:concat function is specified to allow two or more arguments, which are concatenated together. This is the only function specified in this document that allows a variable number of arguments. This capability is retained for compatibility with [XML Path Language (XPath) Version 1.0].

Notes

As mentioned in 5.1 String types Unicode normalization is not automatically applied to the result of fn:concat. If a normalized result is required, fn:normalize-unicode can be applied to the xs:string returned by fn:concat. The following XQuery:

let $v1 := "I plan to go to Mu"
let $v2 := "?nchen in September"
return concat($v1, $v2)

where the "?" represents either the actual Unicode character COMBINING DIARESIS (Unicode codepoint U+0308) or "̈", will return:

"I plan to go to Mu?nchen in September"

where the "?" represents either the actual Unicode character COMBINING DIARESIS (Unicode codepoint U+0308) or "̈". It is worth noting that the returned value is not normalized in NFC; however, it is normalized in NFD.

However, the following XQuery:

let $v1 := "I plan to go to Mu"
let $v2 := "?nchen in September"
return normalize-unicode(concat($v1, $v2))

where the "?" represents either the actual Unicode character COMBINING DIARESIS (Unicode codepoint U+0308) or "̈", will return:

"I plan to go to München in September"

This returned result is normalized in NFC.

Examples

The expression fn:concat('un', 'grateful') returns "ungrateful".

The expression fn:concat('Thy ', (), 'old ', "groans", "", ' ring', ' yet', ' in', ' my', ' ancient',' ears.') returns "Thy old groans ring yet in my ancient ears.".

The expression fn:concat('Ciao!',()) returns "Ciao!".

The expression fn:concat('Ingratitude, ', 'thou ', 'marble-hearted', ' fiend!') returns "Ingratitude, thou marble-hearted fiend!".

The expression fn:concat(01, 02, 03, 04, true()) returns "1234true".

The expression 10 || '/' || 6 returns "10/6".

5.4.3 fn:string-join

Summary

Returns a string created by concatenating the items in a sequence, with a defined separator between adjacent items.

Signatures
fn:string-join(
$values as xs:anyAtomicType*
) as xs:string
fn:string-join(
$values as xs:anyAtomicType*,
$separator as xs:string
) as xs:string
Properties

This function is ·deterministic·, ·context-independent·, and ·focus-independent·.

Rules

The effect of calling the single-argument version of this function is the same as calling the two-argument version with $separator set to a zero-length string.

The function returns an xs:string created by casting each item in the sequence $values to an xs:string, and then concatenating the result strings in order, using the value of $separator as a separator between adjacent strings. If $separator is the zero-length string, then the items in $values are concatenated without a separator.

Notes

If $values is the empty sequence, the function returns the zero-length string.

Examples

The expression fn:string-join(1 to 9) returns "123456789".

The expression fn:string-join(('Now', 'is', 'the', 'time', '...'), ' ') returns "Now is the time ...".

The expression fn:string-join(('Blow, ', 'blow, ', 'thou ', 'winter ', 'wind!'), '') returns "Blow, blow, thou winter wind!".

The expression fn:string-join((), 'separator') returns "".

The expression fn:string-join(1 to 5, ', ') returns "1, 2, 3, 4, 5".

let $doc := <doc>
  <chap>
    <section xml:id="xyz"/>
  </chap>
</doc>

The expression $doc//@xml:id ! fn:string-join((node-name(), '="', ., '"')) returns 'xml:id="xyz"'.

The expression $doc//section ! fn:string-join(ancestor-or-self::*/name(), '/') returns "doc/chap/section".

5.4.4 fn:substring

Summary

Returns the part of $value beginning at the position indicated by $start and continuing for the number of ·characters· indicated by $length.

Signatures
fn:substring(
$value as xs:string?,
$start as xs:double
) as xs:string
fn:substring(
$value as xs:string?,
$start as xs:double,
$length as xs:double
) as xs:string
Properties

This function is ·deterministic·, ·context-independent·, and ·focus-independent·.

Rules

If $value is the empty sequence, the function returns the zero-length string.

Otherwise, the function returns a string comprising those ·characters· of $value whose index position (counting from one) is greater than or equal to $start (rounded to an integer), and (if $length is specified) less than the sum of $start and $length (both rounded to integers).

The characters returned do not extend beyond $value. If $start is zero or negative, only those characters in positions greater than zero are returned.

More specifically, the three argument version of the function returns the characters in $value whose position $p satisfies:

fn:round($start) <= $p and $p < fn:round($start) + fn:round($length)

The two argument version of the function assumes that $length is infinite and thus returns the ·characters· in $value whose position $p satisfies:

fn:round($start) <= $p

In the above computations, the rules for op:numeric-less-than apply.

Notes

The first character of a string is located at position 1, not position 0.

The second and third arguments allow xs:double values (rather than requiring xs:integer) in order to achieve compatibility with XPath 1.0.

A surrogate pair counts as one character, not two.

The consequences of supplying values such as NaN or positive or negative infinity for the $start or $length arguments follow from the above rules, and are not always intuitive.

Examples

The expression fn:substring("motor car", 6) returns " car". (Characters starting at position 6 to the end of $sourceString are selected.)

The expression fn:substring("metadata", 4, 3) returns "ada". (Characters at positions greater than or equal to 4 and less than 7 are selected.)

The expression fn:substring("12345", 1.5, 2.6) returns "234". (Characters at positions greater than or equal to 2 and less than 5 are selected.)

The expression fn:substring("12345", 0, 3) returns "12". (Characters at positions greater than or equal to 0 and less than 3 are selected. Since the first position is 1, these are the characters at positions 1 and 2.)

The expression fn:substring("12345", 5, -3) returns "". (Characters at positions greater than or equal to 5 and less than 2 are selected.)

The expression fn:substring("12345", -3, 5) returns "1". (Characters at positions greater than or equal to -3 and less than 2 are selected. Since the first position is 1, this is the character at position 1.)

The expression fn:substring("12345", 0 div 0E0, 3) returns "". (Since 0 div 0E0 returns NaN, and NaN compared to any other number returns false, no characters are selected.)

The expression fn:substring("12345", 1, 0 div 0E0) returns "". (As above.)

The expression fn:substring((), 1, 3) returns "".

The expression fn:substring("12345", -42, 1 div 0E0) returns "12345". (Characters at positions greater than or equal to -42 and less than INF are selected.)

The expression fn:substring("12345", -1 div 0E0, 1 div 0E0) returns "". (Since the value of -INF + INF is NaN, no characters are selected.)

5.4.5 fn:string-length

Summary

Returns the number of ·characters· in a string.

Signatures
fn:string-length() as xs:integer
fn:string-length(
$value as xs:string?
) as xs:integer
Properties

The zero-argument form of this function is ·deterministic·, ·context-dependent·, and ·focus-dependent·.

The one-argument form of this function is ·deterministic·, ·context-independent·, and ·focus-independent·.

Rules

The function returns an xs:integer equal to the length in ·characters· of $value.

Calling the zero-argument version of the function is equivalent to calling fn:string-length(fn:string(.)).

If $value is the empty sequence, the function returns the xs:integer value zero (0).

Error Conditions

If $value is not specified and the context item is absentDM40, a dynamic error is raised: [err:XPDY0002]XP.

Notes

Unlike some programming languages, a ·codepoint· greater than 65535 counts as one character, not two.

There are situations where fn:string-length() has a different effect from fn:string-length(.). For example, if the context item is an attribute node typed as an xs:integer with the string value 000001, then fn:string-length() returns 6 (the length of the string value of the node), while fn:string-length(.) raises a type error (because the result of atomization is not an xs:string).

Examples

The expression fn:string-length("Harp not on that string, madam; that is past.") returns 45.

The expression fn:string-length(()) returns 0.

5.4.6 fn:normalize-space

Summary

Returns $value with leading and trailing whitespace removed, and sequences of internal whitespace reduced to a single space character.

Signatures
fn:normalize-space() as xs:string
fn:normalize-space(
$value as xs:string?
) as xs:string
Properties

The zero-argument form of this function is ·deterministic·, ·context-dependent·, and ·focus-dependent·.

The one-argument form of this function is ·deterministic·, ·context-independent·, and ·focus-independent·.

Rules

If $value is the empty sequence, the function returns the zero-length string.

The function returns a string constructed by stripping leading and trailing whitespace from $value, and replacing sequences of one or more adjacent whitespace characters with a single space, #x20.

The whitespace characters are defined in the metasymbol S (Production 3) of [Extensible Markup Language (XML) 1.0 (Fifth Edition)].

If no argument is supplied, then $value defaults to the string value (calculated using fn:string) of the context item (.).

Error Conditions

If no argument is supplied and the context item is absentDM40 then a dynamic error is raised: [err:XPDY0002]XP.

Notes

The definition of whitespace is unchanged in [Extensible Markup Language (XML) 1.1 Recommendation]. It is repeated here for convenience:

S ::= (#x20 | #x9 | #xD | #xA)+

Examples

The expression fn:normalize-space(" The    wealthy curled darlings                                         of    our    nation. ") returns "The wealthy curled darlings of our nation.".

The expression fn:normalize-space(()) returns "".

5.4.7 fn:normalize-unicode

Summary

Returns $value after applying Unicode normalization.

Signatures
fn:normalize-unicode(
$value as xs:string?
) as xs:string
fn:normalize-unicode(
$value as xs:string?,
$form as xs:string
) as xs:string
Properties

This function is ·deterministic·, ·context-independent·, and ·focus-independent·.

Rules

If $value is the empty sequence, the function returns the zero-length string.

If the single-argument version of the function is used, the result is the same as calling the two-argument version with $form set to the string "NFC".

Otherwise, the function returns $value normalized according to the rules of the normalization form identified by the value of $form.

The effective value of $form is the value of the expression fn:upper-case(fn:normalize-space($form)).

  • If the effective value of $form is "NFC", then the function returns $value converted to Unicode Normalization Form C (NFC).

  • If the effective value of $form is "NFD", then the function returns $value converted to Unicode Normalization Form D (NFD).

  • If the effective value of $form is "NFKC", then the function returns $value in Unicode Normalization Form KC (NFKC).

  • If the effective value of $form is "NFKD", then the function returns $value converted to Unicode Normalization Form KD (NFKD).

  • If the effective value of $form is "FULLY-NORMALIZED", then the function returns $value converted to fully normalized form.

  • If the effective value of $form is the zero-length string, no normalization is performed and $value is returned.

Normalization forms NFC, NFD, NFKC, and NFKD, and the algorithms to be used for converting a string to each of these forms, are defined in [UAX #15].

The motivation for normalization form FULLY-NORMALIZED is explained in [Character Model for the World Wide Web 1.0: Normalization]. However, as that specification did not progress beyond working draft status, the normative specification is as follows:

  • A string is fully-normalized if (a) it is in normalization form NFC as defined in [UAX #15], and (b) it does not start with a composing character.

  • A composing character is a character that is one or both of the following:

    • the second character in the canonical decomposition mapping of some character that is not listed in the Composition Exclusion Table defined in [UAX #15];

    • of non-zero canonical combining class (as defined in [The Unicode Standard]).

  • A string is converted to FULLY-NORMALIZED form as follows:

    • if the first character in the string is a composing character, prepend a single space (x20);

    • convert the resulting string to normalization form NFC.

Conforming implementations must support normalization form "NFC" and may support normalization forms "NFD", "NFKC", "NFKD", and "FULLY-NORMALIZED". They may also support other normalization forms with ·implementation-defined· semantics.

It is ·implementation-defined· which version of Unicode (and therefore, of the normalization algorithms and their underlying data) is supported by the implementation. See [UAX #15] for details of the stability policy regarding changes to the normalization rules in future versions of Unicode. If the input string contains codepoints that are unassigned in the relevant version of Unicode, or for which no normalization rules are defined, the fn:normalize-unicode function leaves such codepoints unchanged. If the implementation supports the requested normalization form then it must be able to handle every input string without raising an error.

Error Conditions

A dynamic error is raised [err:FOCH0003] if the effective value of the $form argument is not one of the values supported by the implementation.

5.4.8 fn:upper-case

Summary

Converts a string to upper case.

Signature
fn:upper-case(
$value as xs:string?
) as xs:string
Properties

This function is ·deterministic·, ·context-independent·, and ·focus-independent·.

Rules

If $value is the empty sequence, the zero-length string is returned.

Otherwise, the function returns $value after translating every ·character· to its upper-case correspondent as defined in the appropriate case mappings section in the Unicode standard [The Unicode Standard]. For versions of Unicode beginning with the 2.1.8 update, only locale-insensitive case mappings should be applied. Beginning with version 3.2.0 (and likely future versions) of Unicode, precise mappings are described in default case operations, which are full case mappings in the absence of tailoring for particular languages and environments. Every lower-case character that does not have an upper-case correspondent, as well as every upper-case character, is included in the returned value in its original form.

Notes

Case mappings may change the length of a string. In general, the fn:upper-case and fn:lower-case functions are not inverses of each other: fn:lower-case(fn:upper-case($s)) is not guaranteed to return $s, nor is fn:upper-case(fn:lower-case($s)). The Latin small letter dotless i (as used in Turkish) is perhaps the most prominent lower-case letter which will not round-trip. The Latin capital letter i with dot above is the most prominent upper-case letter which will not round trip; there are others, such as Latin capital letter Sharp S (#1E9E) which was introduced in Unicode 5.1.

These functions may not always be linguistically appropriate (e.g. Turkish i without dot) or appropriate for the application (e.g. titlecase). In cases such as Turkish, a simple translation should be used first.

Because the function is not sensitive to locale, results will not always match user expectations. In Quebec, for example, the standard uppercase equivalent of "è" is "È", while in metropolitan France it is more commonly "E"; only one of these is supported by the functions as defined.

Many characters of class Ll lack uppercase equivalents in the Unicode case mapping tables; many characters of class Lu lack lowercase equivalents.

Examples

The expression fn:upper-case("abCd0") returns "ABCD0".

5.4.9 fn:lower-case

Summary

Converts a string to lower case.

Signature
fn:lower-case(
$value as xs:string?
) as xs:string
Properties

This function is ·deterministic·, ·context-independent·, and ·focus-independent·.

Rules

If t$value is the empty sequence, the zero-length string is returned.

Otherwise, the function returns $value after translating every ·character· to its lower-case correspondent as defined in the appropriate case mappings section in the Unicode standard [The Unicode Standard]. For versions of Unicode beginning with the 2.1.8 update, only locale-insensitive case mappings should be applied. Beginning with version 3.2.0 (and likely future versions) of Unicode, precise mappings are described in default case operations, which are full case mappings in the absence of tailoring for particular languages and environments. Every upper-case character that does not have a lower-case correspondent, as well as every lower-case character, is included in the returned value in its original form.

Notes

Case mappings may change the length of a string. In general, the fn:upper-case and fn:lower-case functions are not inverses of each other: fn:lower-case(fn:upper-case($s)) is not guaranteed to return $s, nor is fn:upper-case(fn:lower-case($s)). The Latin small letter dotless i (as used in Turkish) is perhaps the most prominent lower-case letter which will not round-trip. The Latin capital letter i with dot above is the most prominent upper-case letter which will not round trip; there are others, such as Latin capital letter Sharp S (#1E9E) which was introduced in Unicode 5.1.

These functions may not always be linguistically appropriate (e.g. Turkish i without dot) or appropriate for the application (e.g. titlecase). In cases such as Turkish, a simple translation should be used first.

Because the function is not sensitive to locale, results will not always match user expectations. In Quebec, for example, the standard uppercase equivalent of "è" is "È", while in metropolitan France it is more commonly "E"; only one of these is supported by the functions as defined.

Many characters of class Ll lack uppercase equivalents in the Unicode case mapping tables; many characters of class Lu lack lowercase equivalents.

Examples

The expression fn:lower-case("ABc!D") returns "abc!d".

5.4.10 fn:translate

Summary

Returns $value modified by replacing or removing individual characters.

Signature
fn:translate(
$value as xs:string?,
$replace as xs:string,
$with as xs:string
) as xs:string
Properties

This function is ·deterministic·, ·context-independent·, and ·focus-independent·.

Rules

If $value is the empty sequence, the function returns the zero-length string.

Otherwise, the function returns a result string constructed by processing each ·character· in $value, in order, according to the following rules:

  1. If the character does not appear in $replace then it is added to the result string unchanged.

  2. If the character first appears in $replace at some position M, where the value of $with is M or more characters in length, then the character at position M in $with is added to the result string.

  3. If the character first appears in $replace at some position M, where $with is less than M characters in length, then the character is omitted from the result string.

Notes

If $replace is the zero-length string then the function returns $value unchanged.

If a character occurs more than once in $replace, then the first occurrence determines the action taken.

If $with is longer than $replace, the excess characters are ignored.

Examples

The expression fn:translate("bar","abc","ABC") returns "BAr".

The expression fn:translate("--aaa--","abc-","ABC") returns "AAA".

The expression fn:translate("abcdabc", "abc", "AB") returns "ABdAB".

5.5 Functions based on substring matching

The functions described in the section examine a string $arg1 to see whether it contains another string $arg2 as a substring. The result depends on whether $arg2 is a substring of $arg1, and if so, on the range of ·characters· in $arg1 which $arg2 matches.

When the ·Unicode codepoint collation· is used, this simply involves determining whether $arg1 contains a contiguous sequence of characters whose ·codepoints· are the same, one for one, with the codepoints of the characters in $arg2.

When a collation is specified, the rules are more complex.

All collations support the capability of deciding whether two ·strings· are considered equal, and if not, which of the strings should be regarded as preceding the other. For functions such as fn:compare, this is all that is required. For other functions, such as fn:contains, the collation needs to support an additional property: it must be able to decompose the string into a sequence of collation units, each unit consisting of one or more characters, such that two strings can be compared by pairwise comparison of these units. ("collation unit" is equivalent to "collation element" as defined in [UTS #10].) The string $arg1 is then considered to contain $arg2 as a substring if the sequence of collation units corresponding to $arg2 is a subsequence of the sequence of the collation units corresponding to $arg1. The characters in $arg1 that match are the characters corresponding to these collation units.

This rule may occasionally lead to surprises. For example, consider a collation that treats "Jaeger" and "Jäger" as equal. It might do this by treating "ä" as representing two collation units, in which case the expression fn:contains("Jäger", "eg") will return true. Alternatively, a collation might treat "ae" as a single collation unit, in which case the expression fn:contains("Jaeger", "eg") will return false. The results of these functions thus depend strongly on the properties of the collation that is used.

In addition, collations may specify that some collation units should be ignored during matching. If hyphen is an ignored collation unit, then fn:contains("code-point", "codepoint") will be true, and fn:contains("codepoint", "-") will also be true.

In the definitions below, we refer to the terms match and minimal match as defined in definitions DS2 and DS4 of [UTS #10]. In applying these definitions:

It is possible to define collations that do not have the ability to decompose a string into units suitable for substring matching. An argument to a function defined in this section may be a URI that identifies a collation that is able to compare two strings, but that does not have the capability to split the string into collation units. Such a collation may cause the function to fail, or to give unexpected results or it may be rejected as an unsuitable argument. The ability to decompose strings into collation units is an ·implementation-defined· property of the collation.

Function Meaning
fn:contains Returns true if the string $value contains $substring as a substring, taking collations into account.
fn:starts-with Returns true if the string $value contains $substring as a leading substring, taking collations into account.
fn:ends-with Returns true if the string $value contains $substring as a trailing substring, taking collations into account.
fn:substring-before Returns the part of $value that precedes the first occurrence of $substring, taking collations into account.
fn:substring-after Returns the part of $value that follows the first occurrence of $substring, taking collations into account.

5.5.1 fn:contains

Summary

Returns true if the string $value contains $substring as a substring, taking collations into account.

Signatures
fn:contains(
$value as xs:string?,
$substring as xs:string?
) as xs:boolean
fn:contains(
$value as xs:string?,
$substring as xs:string?,
$collation as xs:string
) as xs:boolean
Properties

The two-argument form of this function is ·deterministic·, ·context-dependent·, and ·focus-independent·. It depends on collations.

The three-argument form of this function is ·deterministic·, ·context-dependent·, and ·focus-independent·. It depends on collations, and static base URI.

Rules

If $value or $substring is the empty sequence, or contains only ignorable collation units, it is interpreted as the zero-length string.

If $substring is the zero-length string, then the function returns true.

If $value is the zero-length string, the function returns false.

The collation used by this function is determined according to the rules in 5.3.5 Choosing a collation.

The function returns an xs:boolean indicating whether or not $value contains (at the beginning, at the end, or anywhere within) at least one sequence of collation units that provides a minimal match to the collation units in $substring, according to the collation that is used.

Note:

Minimal match is defined in [UTS #10].

Error Conditions

A dynamic error may be raised [err:FOCH0004] if the specified collation does not support collation units.

Examples

The collation used in these examples, http://example.com/CollationA is a collation in which both "-" and "*" are ignorable collation units.

"Ignorable collation unit" is equivalent to "ignorable collation element" in [UTS #10].

The expression fn:contains ( "tattoo", "t") returns true().

The expression fn:contains ( "tattoo", "ttt") returns false().

The expression fn:contains ( "", ()) returns true(). (The first rule is applied, followed by the second rule.)

The expression fn:contains ( "abcdefghi", "-d-e-f-", "http://www.w3.org/2013/collation/UCA?lang=en;alternate=blanked;strength=primary") returns true().

The expression fn:contains ( "a*b*c*d*e*f*g*h*i*", "d-ef-", "http://www.w3.org/2013/collation/UCA?lang=en;alternate=blanked;strength=primary") returns true().

The expression fn:contains ( "abcd***e---f*--*ghi", "def", "http://www.w3.org/2013/collation/UCA?lang=en;alternate=blanked;strength=primary") returns true().

The expression fn:contains ( (), "--***-*---", "http://www.w3.org/2013/collation/UCA?lang=en;alternate=blanked;strength=primary") returns true(). (The second argument contains only ignorable collation units and is equivalent to the zero-length string.)

5.5.2 fn:starts-with

Summary

Returns true if the string $value contains $substring as a leading substring, taking collations into account.

Signatures
fn:starts-with(
$value as xs:string?,
$substring as xs:string?
) as xs:boolean
fn:starts-with(
$value as xs:string?,
$substring as xs:string?,
$collation as xs:string
) as xs:boolean
Properties

The two-argument form of this function is ·deterministic·, ·context-dependent·, and ·focus-independent·. It depends on collations.

The three-argument form of this function is ·deterministic·, ·context-dependent·, and ·focus-independent·. It depends on collations, and static base URI.

Rules

If $value or $substring is the empty sequence, or contains only ignorable collation units, it is interpreted as the zero-length string.

If $substring is the zero-length string, then the function returns true. If $value is the zero-length string and $substring is not the zero-length string, then the function returns false.

The collation used by this function is determined according to the rules in 5.3.5 Choosing a collation.

The function returns an xs:boolean indicating whether or not $value starts with a sequence of collation units that provides a match to the collation units of $substring according to the collation that is used.

Note:

Match is defined in [UTS #10].

Error Conditions

A dynamic error may be raised [err:FOCH0004] if the specified collation does not support collation units.

Examples

The collation used in these examples, http://example.com/CollationA is a collation in which both "-" and "*" are ignorable collation units.

"Ignorable collation unit" is equivalent to "ignorable collation element" in [UTS #10].

The expression fn:starts-with("tattoo", "tat") returns true().

The expression fn:starts-with ( "tattoo", "att") returns false().

The expression fn:starts-with ((), ()) returns true().

The expression fn:starts-with ( "abcdefghi", "-a-b-c-", "http://www.w3.org/2013/collation/UCA?lang=en;alternate=blanked;strength=primary") returns true().

The expression fn:starts-with ( "a*b*c*d*e*f*g*h*i*", "a-bc-", "http://www.w3.org/2013/collation/UCA?lang=en;alternate=blanked;strength=primary") returns true().

The expression fn:starts-with ( "abcd***e---f*--*ghi", "abcdef", "http://www.w3.org/2013/collation/UCA?lang=en;alternate=blanked;strength=primary") returns true().

The expression fn:starts-with ( (), "--***-*---", "http://www.w3.org/2013/collation/UCA?lang=en;alternate=blanked;strength=primary") returns true(). (The second argument contains only ignorable collation units and is equivalent to the zero-length string.)

The expression fn:starts-with ( "-abcdefghi", "-abc", "http://www.w3.org/2013/collation/UCA?lang=en;alternate=blanked;strength=primary") returns true().

5.5.3 fn:ends-with

Summary

Returns true if the string $value contains $substring as a trailing substring, taking collations into account.

Signatures
fn:ends-with(
$value as xs:string?,
$substring as xs:string?
) as xs:boolean
fn:ends-with(
$value as xs:string?,
$substring as xs:string?,
$collation as xs:string
) as xs:boolean
Properties

The two-argument form of this function is ·deterministic·, ·context-dependent·, and ·focus-independent·. It depends on collations.

The three-argument form of this function is ·deterministic·, ·context-dependent·, and ·focus-independent·. It depends on collations, and static base URI.

Rules

If $value or $substring is the empty sequence, or contains only ignorable collation units, it is interpreted as the zero-length string.

If $substring is the zero-length string, then the function returns true. If $value is the zero-length string and the value of $substring is not the zero-length string, then the function returns false.

The collation used by this function is determined according to the rules in 5.3.5 Choosing a collation.

The function returns an xs:boolean indicating whether or not $value ends with a sequence of collation units that provides a match to the collation units of $substring according to the collation that is used.

Note:

Match is defined in [UTS #10].

Error Conditions

A dynamic error may be raised [err:FOCH0004] if the specified collation does not support collation units.

Examples

The collation used in these examples, http://example.com/CollationA is a collation in which both "-" and "*" are ignorable collation units.

"Ignorable collation unit" is equivalent to "ignorable collation element" in [UTS #10].

The expression fn:ends-with ( "tattoo", "tattoo") returns true().

The expression fn:ends-with ( "tattoo", "atto") returns false().

The expression fn:ends-with ((), ()) returns true().

The expression fn:ends-with ( "abcdefghi", "-g-h-i-", "http://www.w3.org/2013/collation/UCA?lang=en;alternate=blanked;strength=primary") returns true().

The expression fn:ends-with ( "abcd***e---f*--*ghi", "defghi", "http://www.w3.org/2013/collation/UCA?lang=en;alternate=blanked;strength=primary") returns true().

The expression fn:ends-with ( "abcd***e---f*--*ghi", "defghi", "http://www.w3.org/2013/collation/UCA?lang=en;alternate=blanked;strength=primary") returns true().

The expression fn:ends-with ( (), "--***-*---", "http://www.w3.org/2013/collation/UCA?lang=en;alternate=blanked;strength=primary") returns true(). (The second argument contains only ignorable collation units and is equivalent to the zero-length string.)

The expression fn:ends-with ( "abcdefghi", "ghi-", "http://www.w3.org/2013/collation/UCA?lang=en;alternate=blanked;strength=primary") returns true().

5.5.4 fn:substring-before

Summary

Returns the part of $value that precedes the first occurrence of $substring, taking collations into account.

Signatures
fn:substring-before(
$value as xs:string?,
$substring as xs:string?
) as xs:string
fn:substring-before(
$value as xs:string?,
$substring as xs:string?,
$collation as xs:string
) as xs:string
Properties

The two-argument form of this function is ·deterministic·, ·context-dependent·, and ·focus-independent·. It depends on collations.

The three-argument form of this function is ·deterministic·, ·context-dependent·, and ·focus-independent·. It depends on collations, and static base URI.

Rules

If $value or $substring is the empty sequence, or contains only ignorable collation units, it is interpreted as the zero-length string.

If $substring is the zero-length string, then the function returns the zero-length string.

If $value does not contain a string that is equal to $substring, then the function returns the zero-length string.

The collation used by this function is determined according to the rules in 5.3.5 Choosing a collation.

The function returns the substring of $value that precedes in $value the first occurrence of a sequence of collation units that provides a minimal match to the collation units of $substring according to the collation that is used.

Note:

Minimal match is defined in [UTS #10].

Error Conditions

A dynamic error may be raised [err:FOCH0004] if the specified collation does not support collation units.

Examples

The collation used in these examples, http://example.com/CollationA is a collation in which both "-" and "*" are ignorable collation units.

"Ignorable collation unit" is equivalent to "ignorable collation element" in [UTS #10].

The expression fn:substring-before ( "tattoo", "attoo") returns "t".

The expression fn:substring-before ( "tattoo", "tatto") returns "".

The expression fn:substring-before ((), ()) returns "".

The expression fn:substring-before ( "abcdefghi", "--d-e-", "http://www.w3.org/2013/collation/UCA?lang=en;alternate=blanked;strength=primary") returns "abc".

The expression fn:substring-before ( "abc--d-e-fghi", "--d-e-", "http://www.w3.org/2013/collation/UCA?lang=en;alternate=blanked;strength=primary") returns "abc--".

The expression fn:substring-before ( "a*b*c*d*e*f*g*h*i*", "***cde", "http://www.w3.org/2013/collation/UCA?lang=en;alternate=blanked;strength=primary") returns "a*b*".

The expression fn:substring-before ( "Eureka!", "--***-*---", "http://www.w3.org/2013/collation/UCA?lang=en;alternate=blanked;strength=primary") returns "". (The second argument contains only ignorable collation units and is equivalent to the zero-length string.)

5.5.5 fn:substring-after

Summary

Returns the part of $value that follows the first occurrence of $substring, taking collations into account.

Signatures
fn:substring-after(
$value as xs:string?,
$substring as xs:string?
) as xs:string
fn:substring-after(
$value as xs:string?,
$substring as xs:string?,
$collation as xs:string
) as xs:string
Properties

The two-argument form of this function is ·deterministic·, ·context-dependent·, and ·focus-independent·. It depends on collations.

The three-argument form of this function is ·deterministic·, ·context-dependent·, and ·focus-independent·. It depends on collations, and static base URI.

Rules

If $value or $substring is the empty sequence, or contains only ignorable collation units, it is interpreted as the zero-length string.

If $substring is the zero-length string, then the function returns the value of $value.

If $value does not contain a string that is equal to $substring, then the function returns the zero-length string.

The collation used by this function is determined according to the rules in 5.3.5 Choosing a collation.

The function returns the substring of $value that follows in $value the first occurrence of a sequence of collation units that provides a minimal match to the collation units of $substring according to the collation that is used.

Note:

Minimal match is defined in [UTS #10].

Error Conditions

A dynamic error may be raised [err:FOCH0004] if the specified collation does not support collation units.

Examples

The collation used in these examples, http://example.com/CollationA is a collation in which both "-" and "*" are ignorable collation units.

"Ignorable collation unit" is equivalent to "ignorable collation element" in [UTS #10].

The expression fn:substring-after("tattoo", "tat") returns "too".

The expression fn:substring-after("tattoo", "tattoo") returns "".

The expression fn:substring-after((), ()) returns "".

The expression fn:substring-after("abcdefghi", "--d-e-", "http://www.w3.org/2013/collation/UCA?lang=en;alternate=blanked;strength=primary") returns "fghi".

The expression fn:substring-after("abc--d-e-fghi", "--d-e-", "http://www.w3.org/2013/collation/UCA?lang=en;alternate=blanked;strength=primary") returns "-fghi".

The expression fn:substring-after ( "a*b*c*d*e*f*g*h*i*", "***cde***", "http://www.w3.org/2013/collation/UCA?lang=en;alternate=blanked;strength=primary") returns "*f*g*h*i*".

The expression fn:substring-after ( "Eureka!", "--***-*---", "http://www.w3.org/2013/collation/UCA?lang=en;alternate=blanked;strength=primary") returns "Eureka!". (The second argument contains only ignorable collation units and is equivalent to the zero-length string.)

5.6 String functions that use regular expressions

The three functions described in this section make use of a regular expression syntax for pattern matching. This is described below.

Function Meaning
fn:matches Returns true if the supplied string matches a given regular expression.
fn:replace Returns a string produced from the input string by replacing any substrings that match a given regular expression with a supplied replacement string, provided either literally, or by invoking a supplied function.
fn:tokenize Returns a sequence of strings constructed by splitting the input wherever a separator is found; the separator is any substring that matches a given regular expression.
fn:analyze-string Analyzes a string using a regular expression, returning an XML structure that identifies which parts of the input string matched or failed to match the regular expression, and in the case of matched substrings, which substrings matched each capturing group in the regular expression.

5.6.1 Regular expression syntax

The regular expression syntax used by these functions is defined in terms of the regular expression syntax specified in XML Schema (see [XML Schema Part 2: Datatypes Second Edition]), which in turn is based on the established conventions of languages such as Perl. However, because XML Schema uses regular expressions only for validity checking, it omits some facilities that are widely used with languages such as Perl. This section, therefore, describes extensions to the XML Schema regular expressions syntax that reinstate these capabilities.

Note:

It is recommended that implementers consult [UTS #18] for information on using regular expression processing on Unicode characters.

The regular expression syntax and semantics are identical to those defined in [XML Schema Part 2: Datatypes Second Edition] with the additions described in the following sub-sections.

Note:

In [Schema 1.1 Part 2] there are no substantive technical changes to the syntax or semantics of regular expressions relative to XSD 1.0, but a number of errors and ambiguities have been resolved. For example, the rules for the interpretation of hyphens within square brackets in a regular expression have been clarified; and the semantics of regular expressions are no longer tied to a specific version of Unicode.

Implementers, even in cases where XSD 1.1 is not supported, are advised to consult the XSD 1.1 regular expression specification for guidance on how to handle cases where the XSD 1.0 specification is unclear or inconsistent.

5.6.1.1 Matching the Start and End of the String

Two meta-characters, ^ and $ are added. By default, the meta-character ^ matches the start of the entire string, while $ matches the end of the entire string. In multi-line mode, ^ matches the start of any line (that is, the start of the entire string, and the position immediately after a newline character), while $ matches the end of any line (that is, the end of the entire string, and the position immediately before a newline character). Newline here means the character #x0A only.

This means that the production in [XML Schema Part 2: Datatypes Second Edition]:

[10] Char ::= [^.\?*+()|#x5B#x5D]

is modified to read:

[10] Char ::= [^.\?*+{}()|^$#x5B#x5D]

The XSD 1.1 grammar for regular expressions uses the same production rule, but renumbered and renamed [73] NormalChar; it is affected in the same way.

The characters #x5B and #x5D correspond to "[" and "]" respectively.

Note:

The definition of Char (production [10]) in [XML Schema Part 2: Datatypes Second Edition] has a known error in which it omits the left brace ("{") and right brace ("}"). That error is corrected here.

The following production:

[11] charClass ::= charClassEsc | charClassExpr | WildCardEsc

is modified to read:

[11] charClass ::= charClassEsc | charClassExpr | WildCardEsc | "^" | "$"

Using XSD 1.1 as the baseline the equivalent is to change the production:

[74] charClass ::= SingleCharEsc | charClassEsc | charClassExpr | WildCardEsc

to read:

[74] charClass ::= SingleCharEsc | charClassEsc | charClassExpr | WildCardEsc | "^" | "$"

Single character escapes are extended to allow the $ character to be escaped. The following production is changed:

[24]SingleCharEsc ::= '\' [nrt\|.?*+(){}#x2D#x5B#x5D#x5E]

to

[24]SingleCharEsc ::= '\' [nrt\|.?*+(){}$#x2D#x5B#x5D#x5E]

(In the XSD 1.1 version of the regular expression grammar, the production rule for SingleCharEsc is unchanged, but is renumbered [84])

5.6.1.2 Reluctant Quantifiers

Reluctant quantifiers are supported. They are indicated by a ? following a quantifier. Specifically:

  • X?? matches X, once or not at all

  • X*? matches X, zero or more times

  • X+? matches X, one or more times

  • X{n}? matches X, exactly n times

  • X{n,}? matches X, at least n times

  • X{n,m}? matches X, at least n times, but not more than m times

The effect of these quantifiers is that the regular expression matches the shortest possible substring consistent with the match as a whole succeeding. Without the ? , the regular expression matches the longest possible substring.

To achieve this, the production in [XML Schema Part 2: Datatypes Second Edition]:

[4] quantifier ::= [?*+] | ( '{' quantity '}' )

is changed to:

[4] quantifier ::= ( [?*+] | ( '{' quantity '}' ) ) '?'?

(In the XSD 1.1 version of the regular expression grammar, this rule is unchanged, but is renumbered [67])

Note:

Reluctant quantifiers have no effect on the results of the boolean fn:matches function, since this function is only interested in discovering whether a match exists, and not where it exists.

5.6.1.3 Captured Sub-Expressions

Sub-expressions (groups) within the regular expression are recognized. The regular expression syntax defined by [XML Schema Part 2: Datatypes Second Edition] allows a regular expression to contain parenthesized sub-expressions, but attaches no special significance to them. Some operations associated with regular expressions (for example, back-references, and the fn:replace function) allow access to the parts of the input string that matched a sub-expression (called captured substrings).

[Definition] A left parenthesis is recognized as a capturing left parenthesis provided it is not immediately followed by ?: (see below), is not within a character group (square brackets), and is not escaped with a backslash. The sub-expression enclosed by a capturing left parenthesis and its matching right parenthesis is referred to as a capturing sub-expression.

More specifically, the ·capturing sub-expression· enclosed by the Nth capturing left parenthesis within the regular expression (determined by its character position in left-to-right order, and counting from one) is referred to as the Nth capturing sub-expression.

For example, in the regular expression A(BC(?:D(EF(GH[()])))), the string matched by the sub-expression BC(?:D(EF(GH[()]))) is capturing sub-expression 1, the string matched by EF(GH[()]) is capturing sub-expression 2, and the string matched by GH[()] is capturing sub-expression 3.

When, in the course of evaluating a regular expression, a particular substring of the input matches a capturing sub-expression, that substring becomes available as a captured substring. The string matched by the Nth capturing sub-expression is referred to as the Nth captured substring. By convention, the substring captured by the entire regular expression is treated as captured substring 0 (zero).

When a ·capturing sub-expression· is matched more than once (because it is within a construct that allows repetition), then only the last substring that it matched will be captured. Note that this rule is not sufficient in all cases to ensure an unambiguous result, especially in cases where (a) the regular expression contains nested repeating constructs, and/or (b) the repeating construct matches a zero-length string. In such cases it is implementation-dependent which substring is captured. For example given the regular expression (a*)+ and the input string "aaaa", an implementation might legitimately capture either "aaaa" or a zero length string as the content of the captured subgroup.

Parentheses that are required to group terms within the regular expression, but which are not required for capturing of substrings, can be represented using the syntax (?:xxxx). To achieve this, the production rule for atom in [XML Schema Part 2: Datatypes Second Edition] is changed to replace the alternative:

( '(' regExp ')' )

with:

( '(' '?:'? regExp ')' )

(For the new versions of the XSD 1.0 and XSD 1.1 production rules for atom, see below.)

In the absence of back-references (see below), the presence of the optional ?: has no effect on the set of strings that match the regular expression, but causes the left parenthesis not to be counted by operations (such as fn:replace and back-references) that number the capturing sub-expressions within a regular expression.

5.6.1.4 Back-References

Back-references are allowed outside a character class expression. A back-reference is an additional kind of atom. The construct \N where N is a single digit is always recognized as a back-reference; if this is followed by further digits, these digits are taken to be part of the back-reference if and only if the resulting number NN is such that the back-reference is preceded by the opening parenthesis of the NNth capturing left parenthesis. The regular expression is invalid if a back-reference refers to a capturing sub-expression that does not exist or whose closing right parenthesis occurs after the back-reference.

A back-reference with number N matches a string that is the same as the value of the Nth captured substring.

For example, the regular expression ('|").*\1 matches a sequence of characters delimited either by an apostrophe at the start and end, or by a quotation mark at the start and end.

If no string has been matched by the Nth capturing sub-expression, the back-reference is interpreted as matching a zero-length string.

Combining this change with the introduction of non-capturing groups (see above), back-references change the following production:

[9] atom ::= Char | charClass | ( '(' regExp ')' )

to

[9] atom ::= Char | charClass | ( '(' '?:'? regExp ')' ) | backReference

[9a] backReference ::= "\" [1-9][0-9]*

With respect to the XSD 1.1 version of the regular expression grammar, the effect is to change:

[72] atom ::= NormalChar | charClass | ( '(' regExp ')' )

to

[72] atom ::= NormalChar | charClass | ( '(' '?:'? regExp ')' ) | backReference

[72a] backReference ::= "\" [1-9][0-9]*

Note:

Within a character class expression, \ followed by a digit is invalid. Some other regular expression languages interpret this as an octal character reference.

5.6.1.5 Unicode Block Names

A regular expression that uses a Unicode block name that is not defined in the version(s) of Unicode supported by the processor (for example \p{IsBadBlockName}) is deemed to be invalid [err:FORX0002].

Note:

XSD 1.0 does not say how this situation should be handled; XSD 1.1 says that it should be handled by treating all characters as matching.

5.6.2 Flags

All these functions provide an optional parameter, $flags, to set options for the interpretation of the regular expression. The parameter accepts a xs:string, in which individual letters are used to set options. The presence of a letter within the string indicates that the option is on; its absence indicates that the option is off. Letters may appear in any order and may be repeated. If there are characters present that are not defined here as flags, then a dynamic error is raised [err:FORX0001].

The following options are defined:

  • s: If present, the match operates in "dot-all" mode. (Perl calls this the single-line mode.) If the s flag is not specified, the meta-character . matches any character except a newline (#x0A) or carriage return (#x0D) character. In dot-all mode, the meta-character . matches any character whatsoever. Suppose the input contains "hello" and "world" on two lines. This will not be matched by the regular expression "hello.*world" unless dot-all mode is enabled.

  • m: If present, the match operates in multi-line mode. By default, the meta-character ^ matches the start of the entire string, while $ matches the end of the entire string. In multi-line mode, ^ matches the start of any line (that is, the start of the entire string, and the position immediately after a newline character other than a newline that appears as the last character in the string), while $ matches the end of any line (that is, the position immediately before a newline character, and the end of the entire string if there is no newline character at the end of the string). Newline here means the character #x0A only.

  • i: If present, the match operates in case-insensitive mode. The detailed rules are as follows. In these rules, a character C2 is considered to be a case-variant of another character C1 if the following XPath expression returns true when the two characters are considered as strings of length one, and the ·Unicode codepoint collation· is used:

    fn:lower-case(C1) eq fn:lower-case(C2) or fn:upper-case(C1) eq fn:upper-case(C2)

    Note that the case-variants of a character under this definition are always single characters.

    1. When a normal character (Char) is used as an atom, it represents the set containing that character and all its case-variants. For example, the regular expression "z" will match both "z" and "Z".

    2. A character range (production charRange in the XSD 1.0 grammar, replaced by productions charRange and singleChar in XSD 1.1) represents the set containing all the characters that it would match in the absence of the "i" flag, together with their case-variants. For example, the regular expression "[A-Z]" will match all the letters A-Z and all the letters a-z. It will also match certain other characters such as #x212A (KELVIN SIGN), since fn:lower-case("#x212A") is "k".

      This rule applies also to a character range used in a character class subtraction (charClassSub): thus [A-Z-[IO]] will match characters such as "A", "B", "a", and "b", but will not match "I", "O", "i", or "o".

      The rule also applies to a character range used as part of a negative character group: thus [^Q] will match every character except "Q" and "q" (these being the only case-variants of "Q" in Unicode).

    3. A back-reference is compared using case-blind comparison: that is, each character must either be the same as the corresponding character of the previously matched string, or must be a case-variant of that character. For example, the strings "Mum", "mom", "Dad", and "DUD" all match the regular expression "([md])[aeiou]\1" when the "i" flag is used.

    4. All other constructs are unaffected by the "i" flag. For example, "\p{Lu}" continues to match upper-case letters only.

  • x: If present, whitespace characters (#x9, #xA, #xD and #x20) in the regular expression are removed prior to matching with one exception: whitespace characters within character class expressions (charClassExpr) are not removed. This flag can be used, for example, to break up long regular expressions into readable lines.

    Examples:

    fn:matches("helloworld", "hello world", "x") returns true()

    fn:matches("helloworld", "hello[ ]world", "x") returns false()

    fn:matches("hello world", "hello\ sworld", "x") returns true()

    fn:matches("hello world", "hello world", "x") returns false()

  • q: if present, all characters in the regular expression are treated as representing themselves, not as metacharacters. In effect, every character that would normally have a special meaning in a regular expression is implicitly escaped by preceding it with a backslash.

    Furthermore, when this flag is present, the characters $ and \ have no special significance when used in the replacement string supplied to the fn:replace function.

    This flag can be used in conjunction with the i flag. If it is used together with the m, s, or x flag, that flag has no effect.

    Examples:

    fn:tokenize("12.3.5.6", ".", "q") returns ("12", "3", "5", "6")

    fn:replace("a\b\c", "\", "\\", "q") returns "a\\b\\c"

    fn:replace("a/b/c", "/", "$", "q") returns "a$b$c"

    fn:matches("abcd", ".*", "q") returns false()

    fn:matches("Mr. B. Obama", "B. OBAMA", "iq") returns true()

5.6.3 fn:matches

Summary

Returns true if the supplied string matches a given regular expression.

Signatures
fn:matches(
$value as xs:string?,
$pattern as xs:string
) as xs:boolean
fn:matches(
$value as xs:string?,
$pattern as xs:string,
$flags as xs:string
) as xs:boolean
Properties

This function is ·deterministic·, ·context-independent·, and ·focus-independent·.

Rules

The effect of calling the first version of this function (omitting the argument $flags) is the same as the effect of calling the second version with the $flags argument set to a zero-length string. Flags are defined in 5.6.2 Flags.

If $value is the empty sequence, it is interpreted as the zero-length string.

The function returns true if $value or some substring of $value matches the regular expression supplied as $pattern. Otherwise, the function returns false. The matching rules are influenced by $flags if present.

Error Conditions

A dynamic error is raised [err:FORX0002] if $pattern is invalid according to the rules described in 5.6.1 Regular expression syntax.

A dynamic error is raised [err:FORX0001] if $flags is invalid according to the rules described in 5.6.2 Flags.

Notes

Unless the metacharacters ^ and $ are used as anchors, the string is considered to match the pattern if any substring matches the pattern. But if anchors are used, the anchors must match the start/end of the string (in string mode), or the start/end of a line (in multi-line mode).

This is different from the behavior of patterns in [XML Schema Part 2: Datatypes Second Edition], where regular expressions are implicitly anchored.

Regular expression matching is defined on the basis of Unicode code points; it takes no account of collations.

Examples

The expression fn:matches("abracadabra", "bra") returns true().

The expression fn:matches("abracadabra", "^a.*a$") returns true().

The expression fn:matches("abracadabra", "^bra") returns false().

Given the source document:

let $poem := 
<poem author="Wilhelm Busch">
Kaum hat dies der Hahn gesehen,
Fängt er auch schon an zu krähen:
Kikeriki! Kikikerikih!!
Tak, tak, tak! - da kommen sie.
</poem>

the following function calls produce the following results, with the poem element as the context node:

The expression fn:matches($poem, "Kaum.*krähen") returns false().

The expression fn:matches($poem, "Kaum.*krähen", "s") returns true().

The expression fn:matches($poem, "^Kaum.*gesehen,$", "m") returns true().

The expression fn:matches($poem, "^Kaum.*gesehen,$") returns false().

The expression fn:matches($poem, "kiki", "i") returns true().

5.6.4 fn:replace

Summary

Returns a string produced from the input string by replacing any substrings that match a given regular expression with a supplied replacement string, provided either literally, or by invoking a supplied function.

Signature
fn:replace(
$value as xs:string?,
$pattern as xs:string,
$replacement as xs:string? := (),
$flags as xs:string? := '',
$action as (function(xs:string, xs:string*) as xs:string?)? := ()
) as xs:string
Properties

This function is ·deterministic·, ·context-independent·, and ·focus-independent·.

Rules

If the $flags argument is omitted, the effect is the same as supplying a zero-length string. Flags are defined in 5.6.2 Flags. The $flags argument is interpreted in the same manner as for the fn:matches function.

The replacement string is determined by the values of the $replacement and/or $action arguments. If both these arguments are present and non-empty, an error is reported.

If $value is the empty sequence, it is interpreted as the zero-length string.

The function returns the xs:string that is obtained by replacing each non-overlapping substring of $value that matches the given $pattern with a replacement string.

If two overlapping substrings of $value both match the $pattern, then only the first one (that is, the one whose first ·character· comes first in the $value string) is replaced.

The replacement string is determined as follows:

  • If the $replacement argument is present and is not an empty sequence, then the value of $replacement.

  • If the $action argument is present and is not an empty sequence, then the string obtained by calling the $action function.

    The first argument to the $action function is the string to be replaced.

    The second argument to the $action function is the sequence of captured substrings. The Nth item in this sequence is the substring captured by the Nth parenthesized sub-expression. If the Nth parenthesized sub-expression was not matched, the Nth item will be the zero-length string.

    Note that the rules for function coercion mean that the function actually supplied for the $action parameter may be an arity-1 function: the second argument does not need to be declared if it is not used.

    If the function returns an empty sequence, then the replacement string is the zero-length string.

  • If neither of the $replacement or $action arguments is present, or if both are empty, then the replacement string is the zero-length string.

If the q flag is present, or if the replacement string was obtained by calling the $action function, then the replacement string is used as is.

Otherwise, within the replacement string, a variable $N may be used to refer to the substring captured by the Nth parenthesized sub-expression in the regular expression. For each match of the pattern, these variables are assigned the value of the content matched by the relevant sub-expression, and the modified replacement string is then substituted for the ·characters· in $value that matched the pattern. $0 refers to the substring captured by the regular expression as a whole.

More specifically, the rules are as follows, where S is the number of parenthesized sub-expressions in the regular expression, and N is the decimal number formed by taking all the digits that consecutively follow the $ character:

  1. If N=0, then the variable is replaced by the substring matched by the regular expression as a whole.

  2. If 1<=N<=S, then the variable is replaced by the substring captured by the Nth parenthesized sub-expression. If the Nth parenthesized sub-expression was not matched, then the variable is replaced by the zero-length string.

  3. If S<N<=9, then the variable is replaced by the zero-length string.

  4. Otherwise (if N>S and N>9), the last digit of N is taken to be a literal character to be included "as is" in the replacement string, and the rules are reapplied using the number N formed by stripping off this last digit.

For example, if the replacement string is "$23" and there are 5 substrings, the result contains the value of the substring that matches the second sub-expression, followed by the digit 3.

Unless the q flag is used, a literal $ character within the replacement string must be written as \$, and a literal \ character must be written as \\.

If two alternatives within the pattern both match at the same position in the $input, then the match that is chosen is the one matched by the first alternative. For example:

 fn:replace("abcd", "(ab)|(a)", "[1=$1][2=$2]") returns "[1=ab][2=]cd"
Error Conditions

A dynamic error is raised [err:FORX0002] if the value of $pattern is invalid according to the rules described in section 5.6.1 Regular expression syntax.

A dynamic error is raised [err:FORX0001] if the value of $flags is invalid according to the rules described in section 5.6.2 Flags.

A dynamic error is raised [err:FORX0003] if the pattern matches a zero-length string, that is, if the expression fn:matches("", $pattern, $flags) returns true. It is not an error, however, if a captured substring is zero-length.

In the absence of the q flag, a dynamic error is raised [err:FORX0004] if the value of $replacement contains a dollar sign ($) character that is not immediately followed by a digit 0-9 and not immediately preceded by a backslash (\).

In the absence of the q flag, a dynamic error is raised [err:FORX0004] if the value of $replacement contains a backslash (\) character that is not part of a \\ pair, unless it is immediately followed by a dollar sign ($) character.

A dynamic error is raised [err:FORX0005] if both the $replacement and $action arguments are supplied, and neither is an empty sequence.

Notes

If the input string contains no substring that matches the regular expression, the result of the function is a single string identical to the input string.

Examples

The expression fn:replace("abracadabra", "bra", "*") returns "a*cada*".

The expression fn:replace("abracadabra", "a.*a", "*") returns "*".

The expression fn:replace("abracadabra", "a.*?a", "*") returns "*c*bra".

The expression fn:replace("abracadabra", "a", "") returns "brcdbr".

The expression fn:replace("abracadabra", "a(.)", "a$1$1") returns "abbraccaddabbra".

The expression fn:replace("AAAA", "A+", "b") returns "b".

The expression fn:replace("AAAA", "A+?", "b") returns "bbbb".

The expression fn:replace("darted", "^(.*?)d(.*)$", "$1c$2") returns "carted". (The first d is replaced.)

The expression fn:replace("abracadabra", "bra", action: ->{"*"}) returns "a*cada*".

The expression fn:replace("abracadabra", "bra", action: fn:upper-case#1) returns aBRAcadaBRA.

The expression fn:replace("Chapter 9", "[0-9]+", action: ->{string(number(.)+1)}) returns "Chapter 10".

The expression fn:replace("LHR to LAX", "[A-Z]{3}", action: map{'LAX': 'Los Angeles', 'LHR': 'London'}) returns "London to Los Angeles".

The expression fn:replace("57°43′30″", "([0-9]+)°([0-9]+)′([0-9]+)″)", action: ->($s, $groups){string(number($groups[1]) + number($groups[2])÷60 + number($groups[3])÷3600)||'°'}) returns "57.725°".

The expression fn:replace("abracadabra", ".*?", "$1") raises an error, because the pattern matches the zero-length string

5.6.5 fn:tokenize

Summary

Returns a sequence of strings constructed by splitting the input wherever a separator is found; the separator is any substring that matches a given regular expression.

Signatures
fn:tokenize(
$value as xs:string?
) as xs:string*
fn:tokenize(
$value as xs:string?,
$pattern as xs:string
) as xs:string*
fn:tokenize(
$value as xs:string?,
$pattern as xs:string,
$flags as xs:string
) as xs:string*
Properties

This function is ·deterministic·, ·context-independent·, and ·focus-independent·.

Rules

The one-argument form of this function splits the supplied string at whitespace boundaries. More specifically, calling fn:tokenize($value) is equivalent to calling fn:tokenize(fn:normalize-space($value), ' ')) where the second argument is a single space character (x20).

The effect of calling the two-argument form of this function (omitting the argument $flags) is the same as the effect of calling the three-argument version with the $flags argument set to a zero-length string. Flags are defined in 5.6.2 Flags.

The following rules apply to the three-argument form of the function:

  • The $flags argument is interpreted in the same way as for the fn:matches function.

  • If $value is the empty sequence, or if $value is the zero-length string, the function returns the empty sequence.

  • The function returns a sequence of strings formed by breaking the $value string into a sequence of strings, treating any substring that matches $pattern as a separator. The separators themselves are not returned.

  • Except with the one-argument form of the function, if a separator occurs at the start of the $value string, the result sequence will start with a zero-length string. Similarly, zero-length strings will also occur in the result sequence if a separator occurs at the end of the $value string, or if two adjacent substrings match the supplied $pattern.

  • If two alternatives within the supplied $pattern both match at the same position in the $value string, then the match that is chosen is the first. For example:

     fn:tokenize("abracadabra", "(ab)|(a)") returns ("", "r", "c", "d", "r", "")
Error Conditions

A dynamic error is raised [err:FORX0002] if the value of $pattern is invalid according to the rules described in section 5.6.1 Regular expression syntax.

A dynamic error is raised [err:FORX0001] if the value of $flags is invalid according to the rules described in section 5.6.2 Flags.

A dynamic error is raised [err:FORX0003] if the supplied $pattern matches a zero-length string, that is, if fn:matches("", $pattern, $flags) returns true.

Notes

If the input string is not zero length, and no separators are found in the input string, the result of the function is a single string identical to the input string.

The one-argument form of the function has a similar effect to the two-argument form with \s+ as the separator pattern, except that the one-argument form strips leading and trailing whitespace, whereas the two-argument form delivers an extra zero-length token if leading or trailing whitespace is present.

The function returns no information about the separators that were found in the string. If this information is required, the fn:analyze-string function can be used instead.

The separator used by the one-argument form of the function is any sequence of tab (x09), newline (x0A), carriage return (x0D) or space (x20) characters. This is the same as the separator recognized by list-valued attributes as defined in XSD. It is not the same as the separator recognized by list-valued attributes in HTML5, which also treats form-feed (x0C) as whitespace. If it is necessary to treat form-feed as a separator, an explicit separator pattern should be used.

Examples

The expression fn:tokenize(" red green blue ") returns ("red", "green", "blue").

The expression fn:tokenize("The cat sat on the mat", "\s+") returns ("The", "cat", "sat", "on", "the", "mat").

The expression fn:tokenize(" red green blue ", "\s+") returns ("", "red", "green", "blue", "").

The expression fn:tokenize("1, 15, 24, 50", ",\s*") returns ("1", "15", "24", "50").

The expression fn:tokenize("1,15,,24,50,", ",") returns ("1", "15", "", "24", "50", "").

fn:tokenize("abba", ".?") raises the dynamic error [err:FORX0003].

The expression fn:tokenize("Some unparsed <br> HTML <BR> text", "\s*<br>\s*", "i") returns ("Some unparsed", "HTML", "text").

5.6.6 fn:analyze-string

Summary

Analyzes a string using a regular expression, returning an XML structure that identifies which parts of the input string matched or failed to match the regular expression, and in the case of matched substrings, which substrings matched each capturing group in the regular expression.

Signatures
fn:analyze-string(
$value as xs:string?,
$pattern as xs:string
) as element(fn:analyze-string-result)
fn:analyze-string(
$value as xs:string?,
$pattern as xs:string,
$flags as xs:string
) as element(fn:analyze-string-result)
Properties

This function is ·nondeterministic·, ·context-independent·, and ·focus-independent·.

Rules

The effect of calling the first version of this function (omitting the argument $flags) is the same as the effect of calling the second version with the $flags argument set to a zero-length string. Flags are defined in 5.6.2 Flags.

The $flags argument is interpreted in the same way as for the fn:matches function.

If $value is the empty sequence the function behaves as if $value were the zero-length string. In this situation the result will be an element node with no children.

The function returns an element node whose local name is analyze-string-result. This element and all its descendant elements have the namespace URI http://www.w3.org/2005/xpath-functions. The namespace prefix is ·implementation-dependent·. The children of this element are a sequence of fn:match and fn:non-match elements. This sequence is formed by breaking the $value string into a sequence of strings, returning any substring that matches $pattern as the content of a match element, and any intervening substring as the content of a non-match element.

More specifically, the function starts at the beginning of the input string and attempts to find the first substring that matches the regular expression. If there are several matches, the first match is defined to be the one whose starting position comes first in the string. If several alternatives within the regular expression both match at the same position in the input string, then the match that is chosen is the first alternative that matches. For example, if the input string is The quick brown fox jumps and the regular expression is jump|jumps, then the match that is chosen is jump.

Having found the first match, the instruction proceeds to find the second and subsequent matches by repeating the search, starting at the first ·character· that was not included in the previous match.

The input string is thus partitioned into a sequence of substrings, some of which match the regular expression, others which do not match it. Each substring will contain at least one character. This sequence is represented in the result by the sequence of fn:match and fn:non-match children of the returned element node; the string value of the fn:match or fn:non-match element will be the corresponding substring of $input, and the string value of the returned element node will therefore be the same as $input.

The content of an fn:non-match element is always a single text node.

The content of a fn:match element, however, is in general a sequence of text nodes and fn:group element children. An fn:group element with a nr attribute having the integer value N identifies the substring captured by the Nth parenthesized sub-expression in the regular expression. For each capturing subexpression there will be at most one corresponding fn:group element in each fn:match element in the result.

If the function is called twice with the same arguments, it is ·implementation-dependent· whether the two calls return the same element node or distinct (but deep equal) element nodes. In this respect it is ·non-deterministic with respect to node identity·.

The base URI of the element nodes in the result is ·implementation-dependent·.

A schema is defined for the structure of the returned element: see C.1 Schema for the result of fn:analyze-string.

The result of the function will always be such that validation against this schema would succeed. However, it is ·implementation-defined· whether the result is typed or untyped, that is, whether the elements and attributes in the returned tree have type annotations that reflect the result of validating against this schema.

Error Conditions

A dynamic error is raised [err:FORX0002] if the value of $pattern is invalid according to the rules described in section 5.6.1 Regular expression syntax.

A dynamic error is raised [err:FORX0001] if the value of $flags is invalid according to the rules described in section 5.6.2 Flags.

A dynamic error is raised [err:FORX0003] if the supplied $pattern matches a zero-length string, that is, if fn:matches("", $pattern, $flags) returns true.

Notes

It is recommended that a processor that implements schema awareness should return typed nodes. The concept of "schema awareness", however, is a matter for host languages to define and is outside the scope of the function library specification.

The declarations and definitions in the schema are not automatically available in the static context of the fn:analyze-string call (or of any other expression). The contents of the static context are host-language defined, and in some host languages are implementation-defined.

The schema defines the outermost element, analyze-string-result, in such a way that mixed content is permitted. In fact the element will only have element nodes (match and non-match) as its children, never text nodes. Although this might have originally been an oversight, defining the analyze-string-result element with mixed="true" allows it to be atomized, which is potentially useful (the atomized value will be the original input string), and the capability has therefore been retained for compatibility with the 3.0 version of this specification.

Examples

In the following examples, the result document is shown in serialized form, with whitespace between the element nodes. This whitespace is not actually present in the result.

The expression fn:analyze-string("The cat sat on the mat.", "\w+") returns (with whitespace added for legibility):

<analyze-string-result xmlns="http://www.w3.org/2005/xpath-functions">
  <match>The</match>
  <non-match> </non-match>
  <match>cat</match>
  <non-match> </non-match>
  <match>sat</match>
  <non-match> </non-match>
  <match>on</match>
  <non-match> </non-match>
  <match>the</match>
  <non-match> </non-match>
  <match>mat</match>
  <non-match>.</non-match>
</analyze-string-result>

The expression fn:analyze-string("2008-12-03", "^(\d+)\-(\d+)\-(\d+)$") returns (with whitespace added for legibility):

<analyze-string-result xmlns="http://www.w3.org/2005/xpath-functions">
  <match><group nr="1">2008</group>-<group nr="2"
          >12</group>-<group nr="3">03</group></match>
</analyze-string-result>

The expression fn:analyze-string("A1,C15,,D24, X50,", "([A-Z])([0-9]+)") returns (with whitespace added for legibility):

<analyze-string-result xmlns="http://www.w3.org/2005/xpath-functions">                  
  <match><group nr="1">A</group><group nr="2">1</group></match>
  <non-match>,</non-match>
  <match><group nr="1">C</group><group nr="2">15</group></match>
  <non-match>,,</non-match>
  <match><group nr="1">D</group><group nr="2">24</group></match>
  <non-match>, </non-match>
  <match><group nr="1">X</group><group nr="2">50</group></match>
  <non-match>,</non-match>
</analyze-string-result>