Introduction - If you have any usage issues, please Google them yourself
For XML files that don t declare entity in Document Type Declaration (e.g. SOAP), tokenization can be done by only recording starting offset and length. To make it work, one also needs to maintain XML in memory intact and un-decoded. This has led to the design of a binary encoding specification we called Virtual Token Descriptor (VTD). VTD records are 64-bit integers that encode the starting offsets, lengths, token types and nesting depths of tokens in the XML document.