Performance

1. XSLT benchmark
2. Benchmark for XPath - Update
3. Eight tips for how to use XSLT efficiently:
4. Improving Performance
5. Predicate or if clause?
6. Match or match then test?
7. Performance hints

XSLT benchmark

Eugene Kuznetsov

DataPower announces XSLTmark, a XSLT benchmark and a small compliance testing suite. Version 1.1.0 is available now and is the first release to the general public. There are about 40 different testcases in this release (see documentation for descriptions and several third-party credits). A variety of java and C/C++ processors are supported, and drivers for other XSLT engines are easy to add. Source and makefiles are being released (with an emphasis on Linux X86, although Win32 X86 and Solaris SPARC are also supported, and other platforms should be fairly straightforward).

For more information, please see: DataPower

We are also making available some initial benchmark results for several popular and well-regarded XSLT processors. We welcome comments, benchmark results submissions and new test drivers for other XSLT processors. (See list for drivers that already exist).

XSLTMark -- First XSLT Benchmark

DataPower's XSLTMark is the first comprehensive benchmark for measuring the performance of XSL processors. It can be used to test the XSLT performance of XSL processors for XML-to-XML and XML-to-HTML transformations. It also provides basic compliance testing to ensure that benchmark results are not distorted by incorrectly functioning processors. The benchmark is a java application that uses a "Driver" class to communicate with the XSL processor under test. Both java and native (C/C++) processors are supported, with driver modules available for many popular XSLT engines on a variety of platforms. XSLTMark is currently being used for performance and compliance testing at DataPower, but also has a core suite of tests to yield benchmark figures for external comparison purposes.

Features:

Processing throughput measurement

Normalized score calculation

Balanced test suite

Optional standards compliance testing

Support for processors written in both java and C/C++

Cross-platform operation

Test drivers for most popular XSLT processors:

System Requirements:

* Java runtime compliant with JDK 1.1.8 or later

* Platform suitable for processor to be tested

Availability:

Version 1.1.0 available for evaluation download

Kevin Jones adds (July 2003)

We (www.sarvega.com) also maintain XSLT and XML Parser benchmarks. The XSLT results are for late 2002 although I believe an update is being prepared at the moment which should be ready soon. Ednote: The tests are undated (as of Feb 2005)

Benchmark for XPath - Update

Massimo Franceschet

I updated the XPath benchmark (science.uva.nl) according to some of the comments I have received from Saxon people. In particular, the major changes are as follows:

1) Query Q21, which was problematic because of the format of the output, has been slightly modified.

2) Whitespaces that are not expected in the query result have been removed from the results of the queries. The format of the result of each query is now the same you obtain running Saxon with the serialization parameter '!indent=no' (and without wrapping).

3) A version of the benchmark including a DTD has been added. In order to declare in the DTD only the elements of the benchmark, the benchmark documents and the query results have been encapsulated into a CDATA section. Moreover, the DTD for the XMark target document has been included.

4) The technical report describing the benchmark has been updated with sections about the evaluation methodology. In particular, several performance indexes to evaluated an XML engine have been proposed.

My next task is to evaluate Galax and Saxon according to such an evaluation strategy.

Any comment is always welcome.

Massimo University of Amsterdam

Eight tips for how to use XSLT efficiently:

Mike Kay

Keep the source documents small. If necessary split the document first.
Keep the XSLT processor (and Java VM) loaded in memory between runs
If you use the same stylesheet repeatedly, compile it first.
If you use the same source document repeatedly, keep it in memory.
If you perform the same transformation repeatedly, don't. Store the result instead.
Keep the output document small. For example, if you're generating HTML, use CSS.
Never validate the same source document more than once.
Split complex transformations into several stages.

Eight tips for how to write efficient XSLT:

Avoid repeated use of "//item".
Don't evaluate the same node-set more than once; save it in a variable.
Avoid <xsl:number> if you can. For example, by using position().
Use <xsl:key>, for example to solve grouping problems.
Avoid complex patterns in template rules. Instead, use <xsl:choose> within the rule.
Be careful when using the preceding[-sibling] or following[-sibling] axes. This often indicates an algorithm with n-squared performance.
Don't sort the same node-set more than once. If necessary, save it as a result tree fragment and access it using the node-set() extension function.
To output the text value of a simple #PCDATA element, use <xsl:value-of> in preference to <xsl:apply-templates>.

Sebastian updated this, July 2000

I have spent some time making a more rigorous set of XSLT performance statistics, which are visible at http://users.ox.ac.uk/~rahtz/xsltest/Report.html

I should say that I don't entirely believe some of what I did. I'll repeat it tonight. The issue of file cacheing is a worry. Anyway, all the files I used are on the web, if anyone with some patience wants to try it, or test other processors. The file Test.pl is a (horrid) Perl script that does the work.

Some trends are fairly obvious in this sort of `number-crunching' XSLT:

XT remains the fastest processor

Saxon is the obvious best choice by almost any criterion other than raw speed

Xalan does poorly when the file gets big

Sablotron has pretty serious problems with large-scale tree traversal

On *my* computer setup, Oracle (with Sun JDK) and Sablotron are memory hogs which effectively kill the machine while running on a decent-sized file. I know Oracle can do much better if other setups.

None of this surprised me. My biggest disappointment was Sablotron, which had impressed me earlier; the 'apply-templates select="//*' really upset it!

I know these tests are not of much interest to the web servlet brigade, but for those of us who want to run radical transforms on big documents, it may help.

The next step must be to see what happens when different XML parsers are used with each processor.

I just ran a test to see how different XSLT processors coped with reading a 3mbyte XML file and going over the tree twice: (All processors are current at July 2000)

Table 1. Comparison

rea	user	sys
XT:	0m23.131s	0m19.030s	0m0.510s
Saxon:	0m35.125s	0m28.900s	0m0.730s
Sablotron:	0m47.187s	0m39.320s	0m1.300s
Xalan:	0m55.277s	0m50.540s	0m1.280s
Oracle:	6m56.811s	1m15.790s	0m10.920s

the interesting results here are

Oracle has some real problem in working well for me :-} Sablotron is slower than Saxon, despite being compiled C++

bizarrely, Xalan produced the results even though it (rightly) said my XML file was not valid against the DTD. Is this to be expected?

Scott Boag answers for Xalan

There just validation warnings I think... the XML file is still well formed, and can be processed. We just happen to not stop if a validation error occurs. We just happen to not specifically stop if a validation warning occurs. I think you can override this behavior. In retrospect, it might be more correct to stop processing by default if a validation warning occurs.

Improving Performance

Michael Kay

The best practices are the same as with any other technology:

- define your performance requirements

- prototype to assess the risk

- allow enough time on the project to acquire the skills

- make measurements at every stage of development

- don't optimize your code unless it works

- don't optimize your code unless it gives measurable benefits

Most of the advice for XSLT is also fairly obvious:

- cache compiled stylesheets rather than compiling them each time

- keep your source documents small

- don't do the same transformation more than once, save the results instead

- don't transform anything unless someone is going to use the result

- remember that XSLT is not the only tool in your kitbag

At the coding level:

- think carefully about anything that deviates from a serial pass through the data

- use keys where direct access to nodes is needed (often suggested by "//")

Some tips:

- do fewer transformations (don't do the same transformation twice; cache the results)

- compile stylesheets and reuse them

- reduce startup costs by not reloading the Java VM etc

- supply the source document as a SAX stream or byte stream, not as a DOM

- avoid serializing a document and then reparsing it

- if you transform the same source document more than once, don't reparse it each time

- avoid doing XSLT transformations on very large documents if you can; consider splitting the document into smaller pieces. 10Mb is about the limit, depending of course on how much memory you have.

- look at any XPath expressions that use axes other than child, parent, or attribute. How many nodes will they access? How often will they be executed? Can you use variables or keys to reduce the number of nodes visited? Can you replace a complex single-phase transformation with a simpler two-phase transformation?

- look at any XPath expressions that use the "=" operator to compare node-sets (especially with non-singular node-sets on both sides). How many comparisons will be needed? Can the number be reduced using keys or variables?

- look carefully at any instructions that use xsl:number or count(). They may need to access many nodes. Can you achieve the same effect using position()?

- try to replace expressions that use "//" with a more explicit path.

- look in particular at the template rules that will be executed most frequently. How often will the rule be executed, and how many nodes in the source tree will it access?

- look at any template rules with complex match patterns. Combining several template rules and using xsl:choose might help.

- set up a repeatable benchmark for measuring performance, that is representative of your real workload. Measure results, record them, make changes one at a time, measure the effect of each change, record the results, reverse the change if it gives no benefit.

- use tracing facilities (e.g. Saxon's -T) to understand which instructions in your stylesheet are being executed most often. If necessary, do detailed profiling of where the time is being spent (but that's usually difficult).

Obviously the details are processor-dependent. In particular, different processors do different optimisations.

Predicate or if clause?

W. Eliot Kimber

>>My question is where, in general, is the best place to use these
>>functions:
>>
>>- In apply-templates specifications?
>>
>>- In match specifications?
>>
>>- As IF blocks within templates?

I just stumbled onto a subtle (at least to me) difference between these two nominally equivalent forms:

<xsl:template match="foo[util:is_applicable()]">

and

<xsl:template match="foo">
   <xsl:if test="util:is_applicable()">

Which is that in the first case all *inapplicable* foo elements fall through to the default template, which if there's no explicit template for "foo", means that the content of foo will likely flow to the output, therefore failing to suppress inapplicable foo elements. Doh!

Given that, it suggests that putting the check in the match= value is the least attractive as it requires at least a single separate template with a lower priority to catch all elements that fail the applicability check, while doing the check at select time ensures that only applicable elements will be processed at all.

Match or match then test?

Eliot Kimber

>My question is where, in general, is the best place to use these
>functions:
>
>- In apply-templates specifications?
>
>- In match specifications?
>
>- As IF blocks within templates?

I just stumbled onto a subtle (at least to me) difference between these two nominally equivalent forms:

<xsl:template match="foo[util:is_applicable()]">

and

<xsl:template match="foo">
   <xsl:if test="util:is_applicable()">

Performance hints

Jim Fuller

generic types;

- always deliver any text based type thing from a server with gzip-encoding; apache has a module for doing this and instructions on how to define which mime-types/ext to deliver thusly...you will be amazed with the results.

- if you must do client side transforms refactor your browser specific javascripts to manage transformations instead of loading stylesheets with an XML PI calling a stylesheet: there are more mechanisms in javascript for pre-loading or caching stylesheets (ex. http://www.perfectxml.com/articles/xml/XSLTInMSXML.asp?pg=2)...this of course complicates matters.

- if you are interested in server side xslt performance, both in pre-publishing and dynamic server side xslt processing you can investigate compiled stylesheets...most of these technologies just convert your stylesheet into a java object (translets I think they were called...XSLTC being built into xalan)

- pre publish as much as possible on the server to its final format....if things need to change, determine if the change is really dynamic or lets say you could schedule publishing every 15 minutes?

- there are hardware appliances with XSLT processing built in mind these days, for all but the most serious situations I would imagine

- like it or not, XSLT may not be the right tool for every publishing job....investigate refactoring using other techniques like SAX or perhaps your XML could be refactored to generate a more appropriate structure from source, designing out xslt transform steps.

- make use of the nice timing mechanisms in SAXON to measure what parts of your stylesheet are slowing things down also depending on your xml size you can choose which tree model SAXON uses...which can have significant effects on speed.

a few perf related tips at the XSL FAQ; this site

a recent perf article for .NET...very informative if this is your env fawcette.com

if you live in a java app serv env I found this article somewhat useful sys-con.com

there is lots you can do to simplify your XSLT...though with no examples it hard to suggest anything.

ps: add RAM/ more Processors/ get faster hard drives and use any other filesystem other then that provided by microsoft

	Home	Feedback
Copyright © 1999-2012 Dave Pawson.