# EXSLT - str:tokenize

Implementer Page: str.tokenize.html Function Package: str.tokenize.zip

#### Function Syntax

    node-set str:tokenize(string, string?)

#### Template Syntax

    <xsl:call-template name="str:tokenize">
       <xsl:with-param name="string" select="string" />
       <xsl:with-param name="delimiters" select="string" />?
    </xsl:call-template>

The `str:tokenize` function splits up a string and returns a node set of `token` elements, each containing one token from the string.

The first argument is the string to be tokenized. The second argument is a string consisting of a number of characters. Each character in this string is taken as a delimiting character. The string given by the first argument is split at any occurrence of any of these characters. For example:

    str:tokenize('2001-06-03T11:40:23', '-T:')

Gives the node set consisting of:

    <token>2001</token>
    <token>06</token>
    <token>03</token>
    <token>11</token>
    <token>40</token>
    <token>23</token>

If the second argument is omitted, the default is the string `'&#x9;&#xA;&#xD;&#x20;'` (i.e. whitespace characters). Thus:

    str:tokenize('date math str')

Gives the node set consisting of:

    <token>date</token>
    <token>math</token>
    <token>str</token>

If the second argument is an empty string, the function returns a set of `token` elements, each of which holds a single character. Thus:

    str:tokenize('foo', '')

Gives the node set consisting of:

    <token>f</token>
    <token>o</token>
    <token>o</token>

The template version of this function returns a result tree fragment consisting of a number of `token` elements.

## Implementations

The following XSLT processors support `str:tokenize`:

- 4XSLT, from 4Suite. (version 0.12.0a3)
- libxslt from Daniel Veillard et al. (version 1.0.19)

Implementations of `str:tokenize` are available in the following languages:

- EXSLT Function (by Jeni Tennison)
- XSLT Template (by Jeni Tennison)
- Javascript (by Chris Bayes)
- Msxsl (by Chris Bayes)

## Examples

### Function

The following example shows how to use the `str:tokenize` function:

#### Source

    <a>
       <c>Is this EXSLT? No. no</c>
    </a>

#### Stylesheet

    <xsl:template match="a">
       <xsl:apply-templates />
    </xsl:template>
    <xsl:template match="*">
       <xsl:value-of select="." />
     -
          <xsl:value-of select="str:tokenize(string(.), ' ')" />
       <xsl:value-of select="str:tokenize(string(.), '')" />
       <xsl:for-each select="str:tokenize(string(.), ' ')">
          <xsl:value-of select="." />
       </xsl:for-each>
       <xsl:apply-templates select="*" />
    </xsl:template>

http://www.exslt.org/str/functions/tokenize/index.html last modified 2002-11-12