Friday, April 30, 2010

replace newline new line character \n to

Hold down Alt and type 010 on the numeric keypad.

Excel uses standard ASCII characters.

Saturday, April 24, 2010

7 Free Windows Password Recovery Tools

Windows password recovery tools are used to recover, or reset lost user and administrator passwords used to log on to Windows operating systems.

Password recovery tools are often called "password cracker" tools because they are sometimes used to "crack" passwords by hackers. Legally cracking or unlocking your own Windows password is certainly a legitimate practice!

Note: Using a Windows password recovery program is just one of several ways to find a lost Windows password.

Important: Please read my Windows Password Recovery Programs FAQ for more information. I also have an easy-to-read comparison of these programs too that might help.

Need to Crack a Different Kind of Password? See my list of free password crackers for free programs to crack PDF files, Word & Excel documents, RAR & ZIP archives, and more.

Here are the top 7 free Windows password recovery programs available. I also keep a list of premium Windows password recovery tools.

1. Ophcrack

The Ophcrack Windows password cracker is by far the best free Windows password recovery tool available. It's fast and easy enough for a first time Windows password cracker with a basic knowledge of Windows.

With Ophcrack, you don't need any access to Windows to be able to recover your lost passwords. Simply visit the site, download the free ISO image, burn it to a CD and boot from the CD. The Ophcrack program starts, locates the Windows user accounts, and proceeds to recover (crack) the passwords - all automatically.

In a test on a Windows 7 PC, Ophcrack recovered the 10-character password to my administrator account in 40 seconds. Ophcrack supports Windows 7, Windows Vista, and Windows XP.

2. Offline NT Password & Registry Editor

Offline NT Password & Registry Editor works differently than most password recovery programs in that it erases your Windows password instead of recovering it. You can think of it as more of a Windows password reset tool.

Like Ophcrack, you boot to a burned CD created with the Offline NT Password & Registry Editor ISO file. After running the program, you can log in to your Windows account without entering a password at all.

If you like this "password deleting" strategy then I highly recommend this program. Offline NT Password & Registry Editor works basically the same way as PC Login Now, listed below.

I tried Offline NT Password & Registry Editor on a Windows 7 PC and it reset the password immediately without problem. It should work equally well with Windows 7, Windows Vista, Windows 2000, and Windows NT. It should also work with 64-bit versions of these operating systems.

3. PC Login Now

PC Login Now is one of the better free Windows password recovery programs I've tested... even though it doesn't actually recover passwords.

Instead of discovering and displaying your current Windows password, PC Login Now deletes it, allowing you to access Windows without a password, after which you can create a new one.

PC Login Now works just like the free and popular Offline NT Password & Registry Editor (listed above) only it's much easier to use.

The trade-off with PC Login Now over Offline NT Password & Registry Editor is that PC Login Now causes Windows to detect a possible hard drive problem on the first reboot after deleting the password. I've never seen nor heard of a problem actually happening but it's still a little bothersome.

In a test on a Windows 7 PC, PC Login Now deleted the 15-character password to my user account instantly. PC Login Now supports Windows 7, Windows Vista, and Windows XP.

4. Kon-Boot

Kon-Boot is yet another free password reset program, much like ONTP&RE and PC Login Now. Just burn the program to a disc, boot to it, and you're off.

Kon-Boot works differently than the two password reset tools above, so if you have problems using them, give Kon-Boot a try. It's very, very easy to use and probably the fastest password reset, and certainly password recovery, tool available.

Unfortunately, Kon-Boot 1.0 does not work with 64-bit versions of Windows.

I used Kon-Boot v1.0 to successfully delete the password on a Windows Vista PC and then also on a Windows XP PC. Kon-Boot is reported to also reset Windows 7 passwords but I was not able to make it work on two different Windows 7 PCs.

5. Cain & Abel

Cain & Abel is a free, fast and effective Windows password recovery tool.

Unlike Ophcrack and other popular Windows password hacking programs, Cain & Abel requires access to Windows under an administrator account. Due to this fact, Cain & Abel is a valuable resource to recover passwords to accounts other than the one you're using.

Add that to the fact that Cain & Abel is a bit more complicated to use than other password recovery apps and you have what is, in my book, a pretty advanced tool. Check it out if you think it might be useful to you.

Cain & Abel was able to recover the 10-character password to the Windows XP "Administrator" account in ten seconds. I couldn't get it to work properly Windows Vista. One user said it works great with Windows 7.

6. LCP

LCP is yet another free password recovery software tool for Windows.

Like Cain & Abel, LCP is a standard Windows program you download from LCPSoft's website and install inside of Windows, meaning you'll need access to a user account on the computer.

LCP can be a little intimidating to those new to Windows password recovery tools so some prior knowledge is very helpful to avoid even bigger problems with your PC.

I haven't been able to get LCP to work for me. If you've successfully used LCP and would like to share your experiences, please let me know.

7. John The Ripper
John the Ripper is a very popular free password recovery tool that can be used to find Windows account passwords.

While the password recovery application itself is free, the wordlists used by John the Ripper to discover passwords do cost and are required for the software to work.

Note: I'm told there are free wordlist alternatives that work with John the Ripper which is why this Windows password recovery tool is still listed as free. However, I have not tested any of them.

John the Ripper is operated at the command line making it a password cracking tool reserved for the very advanced user.

If you have experience with John the Ripper that you would like to share, please let me know.

John The Ripper Free Download
Windows Password Recovery Tools Aren't Necessary if You're Proactive!

These Windows password recovery tools are great if you need them, but there's a much easier way to access your account if you forget your password - a password reset disk!

How To Create a Password Reset Disk

A password reset disk is a special disk you can insert in your PC during the logon process that will allow you to change your Windows password without knowing your current password. You will need to create this disk before you lose access to your account!


Tuesday, April 20, 2010

Processing Large XML Documents with PHP 5

Processing Large XML Documents with PHP 5

There are different ways to process XML Documents in PHP 5. One can process them with SimpleXML, SAX, XMLReader or DOM and they all have their pro and cons (See my "XML in PHP 5" workshop slides for more details about them). But when it comes to large XML documents, the choices look quite limited.
Therefore I did some benchmark testing with the different extensions. The XML document is approx. 10MB big and consists of a lot of blog-entries from Planet PHP. The to-be-solved excercise was to get the title of the entry with the ID 4365. Not that much of a complicated task and with more complicated questions,
the results may differ.

The results (as text file) were actually not that surprising. SAX and XMLReader were very low on memory usage, but slower than with DOM/XPath. Here's a chart of the initial
results (parsing the full document)

But if we assume, there's only one ID = 4365, then we don't have to process the full document and we can stop after the first one (aka FO or firstonly in the results) was found. As this entry is in the first 10% in our example, the results are quite different. To no
surprise. With this approach and some luck with the entries order, we can cut down the processing time considerably, which is not possible with the DOM approach. There, it's all-or-nothing.

In the result charts you maybe also recognized the option "Expand" and "Expand & SimpleXML". I added a new method to XMLReader this weekend called "expand()" (it's in CVS now). With this method, you can convert a node catched with XMLReader to a DOMElement. See also the libxml2 page for more information. This can be very useful, if you want to do DOM Operations on only a little part of a huge XML document. With the "Expand" script, we expand the node matching ID = 4365 with XMLReader and then apply an XPath operation on it. As you can see, it needs some lines
of code (the expand() method only returns a node, but we need a document for XPath), but after that, we can use every XPath expression and DOM Method we want. Even convert it to SimpleXML, as we do in the "Expand & SimpleXML" script. It's maybe a little bit useless in this case, as we don't save a lot of coding or time, but if your subtrees
are more complex or you want to build a new XML document, this can be quite useful. The time and memory used is approx. the same as with the plain XMLReader script (no surprise, since most of the time is spent in traversing the XML document and not parsing the subtree).

I also did some benchmarks with XSLT ( the chart). First I did the traditional method with loading the whole XML document into memory and then transform it. Time and memory used is more or less the same as with plain DOM processing, which is no surprise, since the
task this script has to do is almost the same as we did with the XPath stuff. But it gets interesting with the expand() feature of XMLReader. As we just want to transform the one entry, we search for it with XMLReader, create a DOMElement, resp. a DOMDocument and feed only that to the XSLT processor. This saves a lot of memory and scales very well
on the memory side. It takes longer time-wise (if you parse the full document, but that's the worst case scenario anyway), but if your XML documents are really huge (more than your available RAM for example), then this (or other XMLReader approaches) is the only feasible solution, IMHO.

To sum up: XMLReader is a powerfull extension to parse large XML documents, it's usually much faster than SAX (twice as fast), while still scaling without problems on the memory side. With the expand() method, it's now also possible to mix the features of DOM/SimpleXML/XSLT with XMLReader, if you only have to process parts of an XML

Here are the scripts for reference:




Expand & SimpleXML

DOM & XPath


XSLT w/ XMLReader
Comments (26) |


Bill Humphries

@ 11.05.2004 08:22 CEST

I need to dedicate a box to PHP5 testing.

Thanks for the write up on this, because it gives me some ideas for some strategies to use with XML Reader.
Daniel Veillard

@ 15.05.2004 19:33 CEST

Interesting, at the libxml2 level, SAX is like

twice as fast as the xmlReader, but your

experiment point out one more problem of

the SAX API which is when you cross languages

boundaries, the callbacks are extermely

expensive especially if converting strings

is needed. That's why SAX is not a good

API to export a fast parser to say PHP or

Python, any advantage you may gain with

the C parser is lost in the marshalling process

of the strings. The reader in comparison

allows to minimize the marshalling, you have

far more integer (cheaper), all attributes and

their values are marshalled only if asked for,

and checking the element type allows to

short-circuit potentially expensive operations.

I think that adding Reader operations like

NextElement(Name?) or NextType(type) would

have even more potential for fast processing,

and would be very convenient for the kind of

operations you describe NextElement(title)

would stop only once per article (and there

is glob of optimization possible at the libxml2

level for such searching).

@ 24.03.2005 17:39 CEST

Hai Daniel,

I want to try you reference scripts but in need the memreport.php, it would be nice if you could zend it to me pleas???

thank in advance


Oskar Austegard
@ 18.07.2005 20:23 CEST

Has anyone run a test on HUGE (multi-GB) XML files? Would XMLReader scale to this size?
@ 18.07.2005 20:28 CEST

Hi Oskar: XMLReader scales to any size as it only parses chunk by chunk.
@ 26.08.2005 12:38 CEST

very usefull thanks!!

i am currently using the combination of xmlreader+expand and then manage the single needed node with DOM

great speed!!
Ben Margolin
@ 04.10.2005 00:01 CEST

This was enormously useful info, thanks! Appreciate the examples. While it's slightly clunky to expand/SimpleXML import, it's VERY convenient; I was pleasantly surprised to see there isn't much of a performance penalty, either.

Looking forward to using XMLReader for processing giant feeds... (150MB+ XML...)
@ 10.01.2006 18:42 CEST


Problem:I have to parse an xml file of size greater than 5gb.If i do that using DOM it throws Out of memory exception.

Which parser should i use...should i go for SAX...Or will i face the same problem.....
@ 10.01.2006 19:39 CEST

If you have PHP 5, i'd recommend XMLReader
@ 26.02.2006 12:04 CEST

The reference scripts have been removed. Would much appreciate to have em available again.

Anyhow, i would like to make some speed/memory comparisons between DOM, SAX, XMLReader and

SimpleXML as implementations of an xml2array parser. The structure should be the one given by PEAR:XML_Unserializer. Any idea or prediction which of them is most useful for this job ? The XML structure

i would work with is not very deep folded. Attributes are

rare too.

Thx so far
@ 14.03.2006 12:34 CEST


I read your blog it seems intresting and show me a way to find solution for parsing and storing large XML in DB (upto 200 MB)

can send the example scripts that you mentioned.
@ 14.03.2006 13:09 CEST

The examples are online again, sorry to all who didn't find them...
@ 21.03.2006 02:56 CEST

I'm trying to use XMLReader on a Windows install of PHP 5.1.2 but can't find a way to enable it. Does it require that PHP be recompiled with --enable-xmlreader added to the configure line or is there a build of PHP out there with this already done? I'm doing this for a client and don't have the time to figure out how to compile PHP myself.

I'm trying to parse a 28MB XML file and load it into a MySQL database. Is there another way to do it that won't require me to recompile PHP?

Any help would greatly be appreciated. Thanks.
@ 23.08.2006 02:39 CEST

ashook , memreport.php is available at
Kanthan Arul
@ 30.08.2006 12:43 CEST

Does this work with crossdomain fetching, without crossdomain.xml

My testing showed varied results.

someone clarify pls
@ 21.12.2006 01:35 CEST

Nice article but couldn't you give any example about xml reading with xmlreader. I was looking for an example.
@ 21.12.2006 08:53 CEST

bwdow: and ff. has some examples
@ 26.02.2007 18:53 CEST

i think it's wierd that we are going in circles. the reason large data volumes got broken down into a relational db was for this EXACT situation. fast searching through giant data.'

now we've gone full circle, back to flat files, and needing a way to search them quickly again?

why not just structure your file hierarchy like a database, where the folder names represent tables, and the data in the xml is structured in a relational manner - this keeps the sizes down, and your main title file, would only have two elements, name, and location.

technology seems to chase it's tail an awful lot. i mean, if you have php5, then why not just use a db if you suspect your files are going to become huge???

that said, if your files are reasonable enough and you don't want a db - this advice is great, and thanks a lot!

@ 30.06.2008 09:24 CEST

Could I ask you 'memreport.php' and Xml/Xsl files to test your benchmark ?

Your benchmark and blog are really interesting

Thank you very much

@ 08.07.2008 15:03 CEST

Hey Christian

Great post and thanks for doing the work, saved me some time and effort.

All the files you need are here:
Not Web Design
@ 21.07.2008 14:24 CEST

Perfect information - thank you. Saved me a lot of time. XMLReader it is then :)

@trevor - I agree that we are going in circles, but XML is often used to transport data and not for storage. So sometimes huge XML files has to be read and imported.
@ 16.01.2009 09:38 CEST

I'd just like to queue up with all the others.

I'm using XMLReader to do exact the thing "Not Web Design" mentioned - and it works great. Especially in combination with expand and SimpleXML.

Satya Prakash
@ 02.12.2009 11:23 CEST

I have read this presentation

@ 15.03.2010 12:57 CEST

Great test results,

but where can I find the planet.xml?

Thanks in advance,

@ 24.03.2010 22:34 CEST

Great article!

someone has tried to parse large xml files using XMLReader + XMLWriter?
@ 26.03.2010 11:38 CEST

nice post,

with this information i made my xml operation work faster, i never knew that my xmlparsing could be done so fast...

expand() -> nice idea

XML in PHP 5 - What's New?

Author: Christian Stocker
Title: XML in PHP 5 - What's New?
Date: 18th March 2004

Intended Audience


XML in PHP 4

XML in PHP 5

•  Streams support



•  Reading the DOM

•  XPath

•  Writing to the DOM

•  Extending Classes




•  Writing to SimpleXML Documents

•  Interoperability


•  Calling PHP Functions



•  PHP 4 Specific

•  PHP 5 Specific

•  Standards

•  Tools

Intended Audience

This article is intended for PHP developers at all levels who are interested in using the new XML functionality in PHP 5. Only basic, general knowledge about XML is assumed. However, it's an advantage if you have already worked with XML in PHP.


In today's Internet world, XML isn't just a buzzword anymore, but a widely accepted and used standard. Therefore XML support was taken more seriously for PHP 5 than it was in PHP 4. In PHP 4 you were almost always faced with non-standard, API-breaking, memory leaking, incomplete functionality. Although some of these deficiencies were dealt with in the 4.3 series of PHP 4, the developers nevertheless decided to dump almost everything and start from scratch in PHP 5.

This article will give an introduction to all the new exciting features PHP 5 has to offer regarding XML.

XML in PHP 4

PHP has had XML support from its early days. While this was "only" a SAX based interface, it did at least allow parsing any XML documents without too much hassle. Further XML support came with PHP 4 and the domxml extension. Later the XSLT extension, with Sablotron as backend, was added. During the PHP 4 life cycle, additional features like HTML, XSLT and DTD-validation were added to the domxml extension. Unfortunately, since the xslt and domxml extensions never really left the experimental stage, and changed their API more than once, they were not enabled by default, and frequently not installed on hosts. Furthermore, the domxml extension did not implement the DOM standard defined by the W3C, but had its own method-naming. While this was improved in the 4.3 series of PHP, together with a lot of memory leak and other fixes, it never reached a truly stable stage, and it was almost impossible to really fix the deeper issues. Also, only the SAX extension was enabled by default, so the other extensions never achieved widespread distribution.

For all these reasons, the PHP XML developers decided to start from scratch for PHP 5, and to follow commonly used standards.

XML in PHP 5

Almost everything regarding XML support was rewritten for PHP 5. All the XML extensions are now based on the excellent libxml2 library by the GNOME project. This allows for interoperability between the different extensions, so that the core developers only need to work with one underlying library. For example, the quite complex and now largely improved memory management had to be implemented only once for all XML-related extensions.

In addition to the better-known SAX support inherited from PHP 4, PHP 5 supports DOM according to the W3C standard and XSLT with the very fast libxslt engine. It also incorporates the new PHP-specific SimpleXML extension and a much improved, standards-compliant SOAP extension. Given the increasing importance of XML, the PHP developers decided to enable more XML support by default. This means that you now get SAX, DOM and SimpleXML enabled out of the box, which ensures that they will be installed on many more servers in the future. XSLT and SOAP support, however, still need to be explicitly configured into a PHP build.

Streams support

All the XML extensions now support PHP streams throughout, even if you try to access a stream not directly from PHP. In PHP 5 you can access a PHP stream, for example, from an <xsl:include> or from an <xi:xinclude> directive. Basically, you can access a PHP stream everywhere where you can access a normal file.

Streams in general were introduced in PHP 4.3 and were further improved in PHP 5 as a way of generalizing file-access, network-access, and other operations that share a common set of functions. You can even implement your own streams with PHP code, and thus unify and simplify access to your data. See the PHP documentation for more details about that.


SAX stands for Simple API for XML. It's a callback-based interface for parsing XML documents. SAX support has been available since PHP 3 and hasn't changed a lot since then. For PHP 5 the API is unchanged, so your old code should still work. The only difference is that it's not based on the expat library anymore, but on the libxml2 library.

This change introduced some problems with namespace support, which are currently resolved in libxml2 2.6, but not in older versions of libxml2. Therefore, if you use xml_parser_create_ns(), you are strongly advised to install libxml2 2.6 or above on your system.


DOM (Document Object Model) is a standard for accessing XML document trees, defined by the W3C. In PHP 4, the domxml extension was used for doing just that. The main problem with the domxml extension was that it didn't follow the standard method names. It also had memory leak issues for a long time (they were fixed in PHP 4.3).

The new DOM extension is completely based on the W3C standard, including method and property names. If you're familiar with DOM from other languages, for example in JavaScript, it will be much easier for you to code similar functionality in PHP. You don't have to check the documentation all the time, because the methods and parameters are identical.

As a consequence of this new W3C compatibility, your old domxml-based scripts won't work anymore. The API is quite different in PHP 5. But if you used the "almost W3C compatible" method names available in PHP 4.3, porting isn't such a big deal. You only need to change the loading and saving methods, and remove the underscore in the method names (the DOM standard uses studlyCaps). Other adjustments here and there may be necessary, but the main logic can stay the same.

Reading the DOM

I will not explain all the features of the DOM extension in this article; that would be overkill. You may want to bookmark the documentation available at, which basically corresponds to the implementation in PHP 5.

For most of examples in this article we will use the same XML file; a much-simplified version of the RSS available at Paste the following into a text file and save it as articles.xml:

<?xml version="1.0" encoding="iso-8859-1"


        <title>PHP Weekly: Issue # 172</title>  




        <title>Tutorial: Develop rock-solid code in PHP: Part three</title>  




To load this example into a DOM object, you have to create a DomDocument object, and then load the XML file:

$dom = new DomDocument();

As mentioned above, you could use a PHP stream to load an XML document. You would do this by writing:


(or any other type of stream, as appropriate).

If you want to output the XML document to the browser or as standard output, use:

print $dom->saveXML();

If you want to save it to a file, use:

print $dom->save("newfile.xml");

(Note that this action will send the filesize to stdout.)

There's not much functionality in this example, of course, so let's do something more useful: let's grab all the titles. There are different ways to do this, the easiest one being to use getElementsByTagname($tagname):

$titles = $dom->getElementsByTagName("title");

$titles as $node) {

$node->textContent . "


The property textContent isn't actually a W3C standard. It's a convenience property to access all the text nodes of an element quickly. The W3C way to read this would have been:


(but only if you were sure that firstChild was the text node you needed, otherwise you would have to loop through all the child nodes to find that).

One other thing to notice is that getElementsByTagName() returns a DomNodeList, and not an array as the similar function get_elements_by_tagname() did in PHP 4. But as you can see in the example, you can easily loop through it with a foreach directive. You could also directly access the nodes with $titles->item(0). This would return the first title element.

Another approach to getting all the titles would be to loop through the nodes starting with the root element. As you can see, this is way more complicated, but it's also more flexible should you need more than just the title elements.

foreach ($dom->documentElement->childNodes as $articles) {

//if node is an element (nodeType == 1) and the name is "item" loop further

if ($articles->nodeType == 1 && $articles->nodeName == "item") {

        foreach (
$articles->childNodes  as $item) {

//if node is an element and the name is "title", print it.

if ($item->nodeType == 1 && $item->nodeName == "title") {

$item->textContent . "






XPath is something like SQL for XML. With XPath you can query an XML document for a specific node matching some criteria. To get all the title nodes with XPath, just do the following:


= new domxpath($dom);
$titles = $xp->query("/articles/item/title");

foreach (
$titles as $node) {

$node->textContent . "


This is almost the same code as with getElementsByTagName(), but XPath is much more powerful. For example, if we had a title element as a child of the article element (instead of being the child of an item element), getElementsByTagname() would return it. With /articles/item/title we only pick up the title elements that are placed at the desired level. This is just a simple example; further possibilities might be:

  • /articles/item[position() = 1]/title returning the title element of the first item element.
  • /articles/item/title[@id = '23'] returning all title elements having an attribute id with the value 23
  • /articles//title returning all title elements that are placed below articles

You can also query for elements which have a specific sibling element, or which have a certain text content, or using namespaces, etc. If you have to query XML documents a lot, learning to use XPath properly will save you a lot of time. It's much easier to use, faster in execution, and requires less code than the standard DOM methods.

Writing to the DOM

The Document Object Model can not only be read and queried; you can also manipulate and write to it. (The DOM standard is a little verbose, because its writers tried to support just about every imaginable situation, but it does the job very well). See the next example, where a new element is added to our articles.xml:

$item = $dom->createElement("item");
$title = $dom->createElement("title");
$titletext = $dom->createTextNode("XML in PHP5");


First, we create all the needed nodes: an item element, a title element and a text node containing the title of the item. Then we chain all the nodes together by appending the text node to the title element and appending the title element to the item element. Finally we insert the item element into the root element articles, and voilà! - we have a new article listed in our XML document.

Extending Classes

While the above examples were all doable with PHP 4 and the domxml extension (only the API was a little bit different), the ability to extend DOM classes with your own code is a new feature of PHP 5. This makes it possible to write more readable code. Here's the whole example again, re-written to use the DomDocument class:

class Articles extends DomDocument {

__construct() {

//has to be called!




addArticle($title) {

$item = $this->createElement("item");

$titlespace = $this->createElement("title");

$titletext = $this->createTextNode($title);





$dom = new Articles();
$dom->addArticle("XML in PHP5");



An often-overlooked feature in PHP 4 is the HTML support in libxml2. You can not only load well-formed XML documents with the DOM extension, but you can also load not-well-formed HTML documents, treat them as regular DomDocument objects, and use all the available methods and features such as XPath and SimpleXML.

This HTML capability is very useful if you need to access content from a website you don't control. With the help of XPath, XSLT or SimpleXML you avoid a lot of coding, as compared with using regular expressions or a SAX parser. This is especially useful if the HTML document is not well structured (a frequent problem!).

The code below fetches the index page, parses it and returns the name of the first title element:

$dom = new DomDocument();
$title = $dom->getElementsByTagName("title");


Note that you may get errors as part of your output when expected elements are not found.

If you're one of those people still outputting HTML 4 code on their web pages, there is good news for you, too. The DOM extension cannot only load HTML documents, but can also save them as HTML 4. Just use $dom->saveHTML() after you have built up your DOM document. Note that, for simply making HTML code W3C standards compliant, you're far better off using the Tidy extension. The HTML support in libxml2 is not tuned for every possible case, and doesn't cope well with input in uncommon formats.


Validation of XML documents is getting more and more important. For example, if you get an XML document from some foreign source, you need to verify that it follows a certain format before you can process it. Luckily it's not necessary to write your own validating code in PHP, because you can use one of the three widely used standards for doing this: DTD, XML Schema or RelaxNG.

  • DTD is a standard that comes from SGML days, and lacks some of the newer XML features (like namespaces). Also, because it's not written in XML, it's not easily parsed and/or transformed.
  • XML Schema is a standard defined by the W3C. It's very extensive and has taken care of almost every imaginable need for validating XML documents.
  • RelaxNG was an answer to the complex XML Schema standard, and was created by an independent group. More and more programs support RelaxNG, since it's much easier to implement than XML Schema.

If you don't have legacy schema documents, or overly complex XML documents, go for RelaxNG. It's easier to write, easier to read, and more and more tools support it. There's even a tool called Trang, which automatically creates a RelaxNG document from sample XML document(s). Furthermore, only RelaxNG (and the aging DTDs) is fully supported by libxml2, although full XML Schema support is coming along.

The syntax for validating XML documents is quite simple:

  • $dom->validate('articles.dtd');
  • $dom->relaxNGValidate('articles.rng');
  • $dom->schemaValidate('articles.xsd');

At present, these all simply return true or false. Errors are dumped out as PHP warnings. Obviously this is not the ideal way to give good feedback to the user, and it will be enhanced in one of the releases after PHP 5.0.0. The exact implementation is currently under discussion, but will certainly lead to better error reporting for parse errors and so on.


SimpleXML is the latest addition to the XML family in PHP. The goal of the SimpleXML extension is to provide easy access to XML documents using standard object properties and iterators. This extension doesn't have many methods, but it's quite powerful nonetheless. Getting all the title nodes from our document requires even less code than before:

$sxe = simplexml_load_file("articles.xml");

$sxe->item as $item) {

$item->title ."


What does this do? It first loads articles.xml into a SimpleXML object. Then it gets all elements named item with the property $sxe->item. Finally $item->title gives us the content of the title element. That's it. You could also query attributes with associative arrays, using: $item->title['id'].

As you can see, there's a lot of magic behind this, and there are different ways to get the desired result. For example, $item->title[0] returns the same result as the example. On the other hand, foreach($sxe->item->title as $item) only returns the first title, and not all the titles stored in the document (as I - coming from XPath - would have expected).

SimpleXML is actually one of the first extensions to use most of the new features available with Zend Engine 2. It's therefore also the testing ground for these new features. You should be aware that bugs and unexpected behavior are not uncommon during this stage of development.

Besides the traditional "loop through all the nodes" approach, as shown in the example above, there's also an XPath interface in SimpleXML, which provides even easier access to individual nodes:

foreach($sxe->xpath('/articles/item/title') as $item) {

$item . "


Admittedly the code isn't shorter than in the previous example, but given more complex or deeply nested XML documents you'll find that using XPath together with SimpleXML saves you a lot of typing.

Writing to SimpleXML documents

You can not only parse and read, but also change SimpleXML documents. At least, to some extent:

$sxe->item->title = "XML in PHP5 ";  //new text content for the title element
$sxe->item->title['id'] = 34; // new attribute for the title element
$xmlString = $sxe->asXML(); // returns the SimpleXML object as a serialized XML string
print $xmlString;


As SimpleXML is also based on libxml2, you can easily convert SimpleXML objects to DomDocument objects and vice versa without a big impact on speed (the document doesn't have to be copied internally). With this mechanism you can have the best of both worlds, using the tool best suited for the job in hand. It works with the following methods:

  • $sxe = simplexml_import_dom($dom);
  • $dom = dom_import_simplexml($sxe);


XSLT is a language for transforming XML documents into other XML documents. XSLT is itself written in XML, and belongs to the family of functional languages, which have a different approach to that of procedural and object-orientated languages like PHP.

There were two different XSLT processors implemented in PHP 4: Sablotron (in the more widely used and known xslt extension), and libxslt (within the domxml extension). The two APIs were not compatible with each other, and their feature sets were also different.

In PHP 5, only the libxslt processor is supported. Libxslt was chosen because it's also based on libxml2 and therefore fits perfectly into the XML concept of PHP 5.

It would theoretically be possible to port the Sablotron binding to PHP 5 as well, but unfortunately no one did this yet. Therefore, if you're using Sablotron you will have to switch to the libxslt processor for PHP 5. libxslt is - with the exception of the JavaScript support - feature-equivalent to Sablotron. Even the useful Sablotron-specific scheme handlers can be reimplemented with the much more powerful and portable PHP streams. In addition, libxslt is one of the fastest XSLT implementations available, so you'll get a nice speed boost for free (the execution speed can be double that of Sablotron).

As with all the other extensions discussed in this article, you can exchange XML documents from the XSL extension to the DOM extension and vice versa. In fact you have to, as ext/xsl doesn't have an interface to load and save XML documents, but uses the one from the DOM extension.

You don't need many methods for starting an XSLT transformation, and there is no W3C standard for it, therefore the API was "borrowed" from Mozilla.

First, you need an XSLT stylesheet. Paste the following into a new file and save it as articles.xsl:

<?xml version=1.0" encoding="iso-8859-1"

<xsl:stylesheet version="1.0" xmlns:xsl="">

  <xsl:param name="titles" />

  <xsl:template match="/articles">

    <h2><xsl:value-of select="$titles" /></h2>

    <xsl:for-each select=".//title">

      <h3><xsl:value-of select="." /></h3>




Then call it with a PHP script:


/* load the xml file and stylesheet as domdocuments */
$xsl = new DomDocument();
$inputdom = new DomDocument();

/* create the processor and import the stylesheet */
$proc = new XsltProcessor();
$xsl = $proc->importStylesheet($xsl);
$proc->setParameter(null, "titles", "Titles");

/* transform and output the xml document */
$newdom = $proc->transformToDoc($inputdom);



The above example first loads the XSLT stylesheet articles.xsl with the help of the DOM method load(). Then it creates a new XsltProcessor object, which imports the loaded XSLT stylesheet for later execution. Parameters can be set with setParameter(namespaceURI, name, value), and finally it starts the transformation with transformToDoc($inputdom), which returns a new DomDocument.

This API has the advantage that you can make dozens of XSLT transformations with the same stylesheet, just loading it once and reusing it, as transormToDoc() can be applied to different XML documents.

Besides transformToDoc(), there are two other transformation methods; transformToXML($dom), which returns a string, and transformToURI($dom, $uri), which saves the transformation to a file or a PHP stream. Note that if you want to use an XSLT feature such as <xsl:output method="html"> or indent="yes", you can't use transformToDoc(), because the DomDocument cannot retain this information. These directives will be used only if you output the transformation directly to a string or a file.

Calling PHP Functions

One of the latest features added to the XSLT extension is the ability to call any PHP function from within an XSLT stylesheet. While XML/XSLT purists will certainly dislike this (such stylesheets won't be portable anymore, and could easily mix logic and design), it can be very useful in some special cases. XSLT is very limited when it comes down to functions. Even outputting a date in different languages can be painful to implement - but with this feature it's no more complicated than with PHP itself. Here's the PHP snippet for adding a function into XSLT:


function dateLang () {



$xsl = new DomDocument();
$inputdom = new DomDocument();

$proc = new XsltProcessor();

// Load the documents and process using $xslt
$xsl = $proc->importStylesheet($xsl);

/* transform and output the xml document */
$newdom = $proc->transformToDoc($inputdom);



Here's the XSLT stylesheet, datetime.xsl, that will call that function:

<?xml version="1.0" encoding="iso-8859-1"

<xsl:stylesheet version="1.0" xmlns:xsl="" xmlns:php="">

<xsl:template match="/">

  <xsl:value-of select="php:function('dateLang')" />



And here's an absolute minimal XML file, today.xml, to pass through the stylesheet (although articles.xml would achieve the same result):

<?xml version="1.0" encoding="iso-8859-1"


The stylesheet above, together with the PHP script and any xml file loaded, will output the current weekday name in the language defined in the locale settings. You could add more arguments to php:function(), which would also be passed to the PHP function. Additionally, there's php:functionString(). This function automatically converts all input parameters to strings, so that you don't need to convert them when they reach PHP.

Note that you will need to call $xslt->registerPhpFunctions(); before the transformation, otherwise the PHP function-calls will not work for security reasons (can you always trust your XSLT stylesheets?). A more refined access system (i.e. one that limits access to specific methods) is not available yet, but would not be impossible to implement in a future PHP 5 release.


XML support in PHP has taken a great step forward. It is standards-compliant, well behaved, feature-rich, interoperable - and enabled-by-default functionality can now be taken for granted.

PHP 4's much-disliked domxml extension has been completely rewritten. The new DOM extension follows the W3C standard almost to the dot, and has also resolved a lot of internal memory problems. With the added support of some general PHP features, such as class inheritance and stream support, even more powerful and tightly integrated XML applications will be possible.

The newly added SimpleXML extension is an easy and fast way to access XML documents. It can save you a lot of coding, especially if you have structured documents or are able to use the power of XPath.

Thanks to libxml2, the underlying library used for all PHP 5 XML extensions, validation of XML documents using DTD, RelaxNG or (to some extent) XML Schema is now supported.

XSLT support also got a facelift and now uses the libxslt library, which should improve performance over the old Sablotron library. Furthermore, the ability to call PHP functions from within XSLT stylesheets allows you to write more powerful (though unfortunately less portable) XSLT code.

If you used XML in PHP 4 or in another language, you will love the XML support in PHP 5. XML in PHP 5 is much improved, is standards-compliant, and is finally on a par with other tools and languages.


PHP 4 specific

PHP 5 specific



Friday, April 16, 2010

5.5.0 smtp;554 : Relay access denied

5.5.0 smtp;554 : Relay access denied
5.5.0 smtp;554 : Relay access denied
gunther.imbrechts@... - 10/03/07 Hello,

Some users within our organisation receive NDR's like this:

Your message did not reach some or all of the intended recipients.

Sent: 3/10/2007 16:04

The following recipient(s) could not be reached: on 3/10/2007 16:05
There was a SMTP communication problem with the recipient's email server. Please contact your system administrator.
: Relay access denied>

We have this issue only with a few mail recipients. My configuration:

Exchange 2003 SP2 with FE and BE configuration. There are 2 WAN IP's, #1 and #2, first one is for outbound traffic, second is for inbound. When doing a reverse lookup of our #1 IP, tells us that the A record of our domain published in the PTR (=same as the HELO hostname) is not pointing back to the originating IP. But when I google a bit I found that normally there shouldn't be any problem using different WAN IP's on inbound and outbound. Does it maybe mean that I should publish our SMTP clients on the second IP address, or should I change our DNS configuration so that the A record points to #1 IP?

If I do a reverse lookup for the PTR of our #2 IP address, it gives us the same domain as when I do it for #1. No errors concerning A records for this reverse lookup, because the A record for this domain points back to the same IP, #2.

One last thing, if I don't want to change any of the two options above, could an SPF record maybe help me out?

Thanks a lot for your help,


1. Exactly the same problem! - mailserver HELO?
crastinblack@... - 10/05/07 Hi, I'm having exactly the same problem as you in that we are also getting the 554 Relay access denied NDR when we try to send email to certain domains.

I have two internet connections (although SMTP is bound to only one), and spam filtering relays (MX properly configured so that all incoming mail goes through those).

I know that receiving mailservers perform a reverse-dns lookup on the source IP to get the hostname, then perform a forward-dns lookup to get the IP of that hostname and they double-check that the resulting IP is the same as the source IP. I've manually done this to verify, it all checks out. Which leads me to think that these domains that I can't send email to must perform an additional check on the address that our mailserver gives in it's EHLO.

At the moment, my mailserver identifies itself as in it's EHLO statement. I think perhaps that it should be fully-qualified (as in
1.1. Progression...
gunther.imbrechts@... - 10/06/07 Hey crastinblack,

I read that you should use indeed a FQHN. Did you set up a SPF record for your domain? You can let disappear certain relaying issues (mostly due to spam filters) using an SPF record. There is even a little tool which helps you out configuring a correct spf record, but the link is on my company's computer.

For my issue, last Friday I changed our configuration so that inbound and outbound SMTP traffic pass over the same IP. (#1) Our ISP should only change the A record to point to that IP#1


2. 554 Errors
wmadoty@... - 04/29/09 What I have found is that this issue is frequently related to errors in DNS. Some ISP's (the list is growing) will do different types of verification to check if a message is valid before receiving it. I am sure we are all familiar with the lack of a reverse lookup for a mail server, but some ISP's are also checking to see if an email account is valid before receiving the message.

This validation is done by looking up the MX record for a given domain and then checking the mail server listed in the MX record to verify that the sender has an account on that machine. Most mail servers will respond to this query. Once it is known that the user does exist in the mail domain, the mail is received and delivered to the appropriate box.

If the server does not respond to this query or the senders account does not exist on the server referenced in the DNS MX record for that domain, the 554 DNR error message is issued.

Check your MX Records and your PTR records on your DNS server. If you have more than one MX record for domain make sure the records are valid and the priorities are properly set. This will fix a lot of the 554 problems.

Friday, April 9, 2010

Introduction to Exchange Server 2003 - NDR (Non-Delivery Report)

Introduction to Exchange Server 2003 - NDR (Non-Delivery Report)
Let us suppose that Outlook sends an email to an Exchange server, but that server realizes that it cannot deliver the message, what happens next? The answer is the server sends a NDR (Non-Delivery Report) back to the sender.

At first, it comes as a surprise when you realize there is more than one type of NDR. Then you discover that NDRs have status codes. Moreover, each status code gives you valuable information about the cause of the email problem.

Topics for Exchange Server 2003 NDR (Non-Delivery report)
Interpreting the Exchange 2003 NDR
Exchange 2007 NDRs
NDR Example
Tips for troubleshooting NDRs
NDR List of Codes and their meanings
Master Settings to Enable or Disable NDRs
See Chris Lehr's thoughts on aqadmcli.exe
NDR - Summary

Interpreting the Windows Exchange 2003 NDR
As you examine a NDR message, look out for a three-digit code, for example, 5.2.1. If the first number begins with 5.y.z, then it means you are dealing with a permanent error; this message will never be delivered. Occasionally, you get NDRs beginning with 4.y.z, in which case there is hope that email will eventually get through. The place to look for the NDR status code is on the very last line of the report.

NDR codes like 5.5.0 or 4.3.1 may remind you of SMTP errors 550 and 431. Indeed, the 500 series in SMTP has a similar meaning to the 5.y.z codes in an NDR - failure. I expect that you have worked out why there are no 2.y.z NDRs? The reason is that the 2.y.z series mean success, whereas Non-Delivery Reports, by definition, are failures.

NDR Classification
As you are beginning to appreciate, these status codes are not random. The first number 4.y.z, or 5.y.z refers to the class of code, for example, 5.y.z is permanent error. Incidentally, I have not seen any status codes beginning with 1.y.z, 3.y.z, or indeed any numbers greater than 5.7.z.

The second number x.1.z means subject. This second digit, 1 in the previous example, gives generic information where as the third digit (z) gives detail. Unfortunately, I have not cracked the complete code for the second digit. However, I have discovered a few useful patterns, for instance, 5.1.x indicates a problem with the email address, as apposed to server or connector problem. In addition, 5.2.x means that the email is too big, somewhere there is a message limit.

Conclusion, it is best to look up your three-digit error in the status code, see NDR table below.

Guy Recommends: A Free Trial of the Orion Network Performance Monitor (NPM) 9.5
Solarwinds' Orion performance monitor will help you discover what's happening on your network. Also this utility will guide you through troubleshooting; the dashboard will indicate whether the root cause is a broken link, faulty equipment or resource overload. Because it produces network-centric views, the NPM is intuitive to navigate, and as result you can see easily what's working and what's not.

Perhaps Orion's best feature is the way it suggests solutions. Moreover, if problems arise out of the blue, then you can configure Orion NPM 9.5 to notify members of your team what's changed and how to fix it.

If you are interested in creating network maps, then I recommend that you take advantage of Solarwinds' offer of a download a free trial of Orion's Network Performance Monitor.

NDR Example
Here is an example of an email sent to user who does not exist. There is no guyx mailbox on the paris server. At the bottom of the NDR, you can see the name of the domain (, the server (paris) and the NDR status code (5.1.1). In your examples always seek out the servername in the last line of your NDR.

Your message did not reach some or all of the intended recipients.

Subject: Cisco Kid
Sent: 12/15/2004 11:09 PM

The following recipient(s) could not be reached: on 12/15/2004 11:09 PM
The e-mail account does not exist at the organization this message was sent to. Check the e-mail address, or contact the recipient directly to find out the correct address.

Tips for troubleshooting Exchange NDRs
The key to any troubleshooting is to isolate the problem. In the case of an NDR, discover if the fault lies with the sender, the recipient or the Exchange 2003 server. To gather more clues, send more emails to the same recipient but from different accounts. In addition, send emails to different accounts from the original sender.

Expand the search area by sending email to different sites, or to internet users. Does this tactic narrow the problem to a particular Server, Mailstore or Routing Group Connector?

If it's just one email address that produces the Non-Delivery report, do you type in the SMTP address manually, or do you click the user account in the GAL?

One ISP will only troubleshoot NDRs if you use Outlook Express, which alerted me to the fact that you get different responses from different email clients. So try a different version of Outlook.

I always mean to do this first when I troubleshoot - look in the Application log for errors. A variation of this tip is to increase the Diagnostic Logging see here.

You could also gather more clues with Regtrace, which you find on the Exchange 2003 CD in the support\utils\i386 folder. Regtrace gives you detailed information e.g. homeMDB =
CN=Mailbox Store (GuyMail-Managers),CN=First Storage Group,CN=InformationStore,CN=GuyMail-Managers,CN=Servers,CN=First Administrative Group,CN=Administrative Groups,CN=First Organization,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=GuyMail,DC=com

Kiwi Syslog Server - Free Utility to Analyze Your Network Messages
Syslog messages are full of information for troubleshooting network problems. When something goes wrong then surely there will be an error message in the syslog datagram - if only we can find that record and interpret the event. What will help to capture and analyze such network messages is the Kiwi Syslog Server.

Free Download of Kiwi Syslog Server

NDR List of Codes and their meanings
Explanation of Non-Delivery Report error codes for Exchange Server
(See here for Exchange 2007 NDR codes)
4.2.2 The recipient has exceeded their mailbox limit. It could also be that the delivery directory on the Virtual server has exceeded its limit. (Default 22 MB)
4.3.1 Not enough disk space on the delivery server. Microsoft say this NDR maybe reported as out-of-memory error.
4.3.2 Classic temporary problem, the Administrator has frozen the queue.
4.4.1 Intermittent network connection. The server has not yet responded. Classic temporary problem. If it persists, you will also a 5.4.x status code error.
4.4.2 The server started to deliver the message but then the connection was broken.
4.4.6 Too many hops. Most likely, the message is looping.
4.4.7 Problem with a timeout. Check receiving server connectors.
4.4.9 A DNS problem. Check your smart host setting on the SMTP connector. For example, check correct SMTP format. Also, use square brackets in the IP address [] You can get this same NDR error if you have been deleting routing groups.
4.6.5 Multi-language situation. Your server does not have the correct language code page installed.
5.0.0 SMTP 500 reply code means an unrecognised command. You get this NDR when you make a typing mistake when you manually try to send email via telnet.
More likely, a routing group error, no routing connector, or no suitable address space in the connector. (Try adding * in the address space)
This status code is a general error message in Exchange 2000. In fact Microsoft introduced a service pack to make sure now get a more specific code.
Guy Recommends: SolarWinds Engineer's Toolset v10
The Engineer's Toolset v10 provides a comprehensive console of utilities for troubleshooting computer problems. Guy says it helps me monitor what's occurring on the network, and the tools teach me more about how the system itself operates.

There are so many good gadgets, it's like having free rein of a sweetshop. Thankfully the utilities are displayed logically: monitoring, discovery, diagnostic, and Cisco tools. Download your copy of the Engineer's Toolset v 10

5.1.x Problem with email address.
5.1.0 Often seen with contacts. Check the recipient address.
5.1.1 Another problem with the recipient address. Possibly the user was moved to another server in Active Directory. Maybe an Outlook client replied to a message while offline.
5.1.2 SMTP; 550 Host unknown. An error is triggered when the host name can’t be found. For example, when trying to send an email to bob@
[Example kindly sent in by Paul T.]
5.1.3 Another problem with contacts. Address field maybe empty. Check the address information.
5.1.4 Two objects have the same address, which confuses the categorizer.
5.1.5 Destination mailbox address invalid.
5.1.6 Problem with homeMDB or msExchHomeServerName - check how many users are affected. Sometimes running RUS (Recipient Update Service) cures this problem. Mailbox may have moved.
5.1.7 Problem with senders mail attribute, check properties sheet in ADUC.
5.2.x NDR caused by a problem with the large size of the email.
5.2.1 The message is too large. Else it could be a permissions problem. Check the recipient's mailbox.
5.2.2 Sadly, the recipient has exceeded their mailbox limit.
5.2.3 Recipient cannot receive messages this big. Server or connector limit exceeded.
5.2.4 Most likely, a distribution list or group is trying to send an email. Check where the expansion server is situated.
5.3.0 Problem with MTA, maybe someone has been editing the registry to disable the MTA / Store driver.
5.3.1 Mail system full. Possibly a Standard edition of Exchange reached the 16 GB limit.
5.3.2 System not accepting network messages. Look outside Exchange for a connectivity problem.
5.3.3 Remote server has insufficient disk space to hold email. Check SMTP log.
5.3.4 Message too big. Check limits, System Policy, connector, virtual server.
5.3.5 Multiple Virtual Servers are using the same IP address and port. See Microsoft TechNet article: 321721 Sharing SMTP. Email probably looping.
5.4.0 DNS Problem. Check the Smart host, or check your DNS. It means that there is no DNS server that can resolve this email address. Could be Virtual Server SMTP address.
5.4.1 No answer from host. Not Exchange's fault check connections.
5.4.2 Bad connection.
5.4.3 Routing server failure. No available route.
5.4.4 Cannot find the next hop, check the Routing Group Connector. Perhaps you have Exchange servers in different Routing Groups, but no connector.
5.4.6 Tricky looping problem, a contact has the same email address as an Active Directory user. One user is probably using an Alternate Recipient with the same email address as a contact.
5.4.7 Delivery time-out. Message is taking too long to be delivered.
5.4.8 Microsoft advise, check your recipient policy. SMTP address should be
5.5.0 Underlying SMTP 500 error. Our server tried ehlo, the recipient's server did not understand and returned a 550 or 500 error. Set up SMTP logging.
5.5.2 Possibly the disk holding the operating system is full. Or could be a syntax error if you are executing SMTP from a telnet shell.
5.5.3 More than 5,000 recipients. Check the Global Settings, Message Delivery properties.
5.5.5 Wrong protocol version
5.6.3 More than 250 attachments.
5.7.1 Permissions problem. For some reason the sender is not allowed to email this account. Perhaps an anonymous user is trying to send mail to a distribution list.
Check SMTP Virtual Server Access Tab. Try checking this box: Allow computers which successfully authenticate to relay
User may have a manually created email address that does not match a System Policy.
5.7.2 Distribution list cannot expand and so is unable to deliver its messages.
5.7.3 Check external IP address of ISA server. Make sure it matches the SMTP publishing rule.
5.7.4 Extra security features not supported. Check delivery server settings
5.7.5 Cryptographic failure. Try a plain message with encryption.
5.7.6 Certificate problem, encryption level maybe to high.
5.7.7 Message integrity problem.

Monitor Your Network with the Real-time Traffic Analyzer
The main reason to monitor your network is to check at a glance which of your servers are available. If there is a network problem you want an interface to show the scope of the problem immediately.

Even when all servers and routers are available, sooner or later you will be curious to know who, or what, is hogging the precious network's bandwidth. A GUI showing the top 10 users makes interesting reading.

Another reason to monitor network traffic is to learn more about your server's response times and the consumption of resources. To take the pain out of capturing frames and analysing the raw data, Guy recommends that you download a copy of the SolarWinds free Real-time NetFlow Analyzer.

Tips for isolating the NDR problem

Master Setting to Enable or Disable NDR Exchange
See here for turning off NDR Exchange 2007

It is possible to control whether or not your Exchange 2003 server will send an NDR.

Launch the Exchange System Manager, navigate to Global Settings, Internet Message Format. Select the Advanced tab. (Un)check Allow non-delivery reports. See more here.

Summary - NDR Reports for Microsoft Exchange Server 2003
There are many reasons why Microsoft Exchange 2003 sends an NDR (Non-Delivery Report). If you examine an NDR carefully you will find a status code number, for example 5.3.1 Two points follow from this discovery, there are more than one error code and therefore, more than one possible cause. My aim is to provide reasons why Exchange sends a particular NDR.