[xmlsec] Extremely odd situation.

[Lars Goldschlager]

Greetings.

I'm trying to help solve a bug in third party software (opendcp) when using
libxmlsec to sign and generate files.

An introduction on the bug can be seen here:
https://code.google.com/p/opendcp/issues/detail?id=136

Basically, when using a 64 bit debian system, trying to sign the xml file
ends up on a segmentation fault in libcrypto with a long stack (seen there)
of xmlsec calls.

So far I've come to this spot in the program's code:

key = xmlSecCryptoAppKeyLoadMemory((unsigned char *)opendcp_private_key,
strlen((char *)opendcp_private_key), xmlSecKeyDataFormatPem, NULL, NULL,
NULL);

key is defined thus:
xmlSecKeyPtr      key;

debugging this with gdb, on the 32 bit system, when I enter
xmlSecOpenSSLAppKeyLoadMemory it looks the same in both 64 and 32 bits.

Inside the keys, specially key.value.id (for example) looks perfectly sane
by the moment I reach line 213 of src/openssl/app.c:  (as per gdb: print *
key.value.id):

32bit:

{klassSize = 88, objSize = 16, name = 0xb7d6a4ba "rsa", usage = 28, href =
0xb7d6a480 "http://www.w3.org/2000/09/xmldsig#RSAKeyValue",
  dataNodeName = 0xb7d6a4ae "RSAKeyValue", dataNodeNs = 0x81a2a40 "
http://www.w3.org/2000/09/xmldsig#", initialize = 0xb7d8f21e
<xmlSecOpenSSLKeyDataRsaInitialize>,
  duplicate = 0xb7d8f0fc <xmlSecOpenSSLKeyDataRsaDuplicate>, finalize =
0xb7d8f05d <xmlSecOpenSSLKeyDataRsaFinalize>, generate = 0xb7d9009e
<xmlSecOpenSSLKeyDataRsaGenerate>,
  getType = 0xb7d90d4b <xmlSecOpenSSLKeyDataRsaGetType>, getSize =
0xb7d90a79 <xmlSecOpenSSLKeyDataRsaGetSize>, getIdentifier = 0,
  xmlRead = 0xb7d8f8f3 <xmlSecOpenSSLKeyDataRsaXmlRead>, xmlWrite =
0xb7d90474 <xmlSecOpenSSLKeyDataRsaXmlWrite>, binRead = 0, binWrite = 0,
  debugDump = 0xb7d90c44 <xmlSecOpenSSLKeyDataRsaDebugDump>, debugXmlDump =
0xb7d90b3d <xmlSecOpenSSLKeyDataRsaDebugXmlDump>, reserved0 = 0x0,
reserved1 = 0x0}

64bit:

{klassSize = 168, objSize = 32, name = 0x7ffff711ee9a "rsa", usage = 28,
href = 0x7ffff711ee60 "http://www.w3.org/2000/09/xmldsig#RSAKeyValue",
  dataNodeName = 0x7ffff711ee8e "RSAKeyValue", dataNodeNs = 0x7772c0 "
http://www.w3.org/2000/09/xmldsig#", initialize = 0x7ffff7348f62
<xmlSecOpenSSLKeyDataRsaInitialize>,
  duplicate = 0x7ffff7348e4f <xmlSecOpenSSLKeyDataRsaDuplicate>, finalize =
0x7ffff7348dcf <xmlSecOpenSSLKeyDataRsaFinalize>,
  generate = 0x7ffff7349caa <xmlSecOpenSSLKeyDataRsaGenerate>, getType =
0x7ffff734a844 <xmlSecOpenSSLKeyDataRsaGetType>,
  getSize = 0x7ffff734a5bc <xmlSecOpenSSLKeyDataRsaGetSize>, getIdentifier
= 0, xmlRead = 0x7ffff7349561 <xmlSecOpenSSLKeyDataRsaXmlRead>,
  xmlWrite = 0x7ffff734a00c <xmlSecOpenSSLKeyDataRsaXmlWrite>, binRead = 0,
binWrite = 0, debugDump = 0x7ffff734a750
<xmlSecOpenSSLKeyDataRsaDebugDump>,
  debugXmlDump = 0x7ffff734a65c <xmlSecOpenSSLKeyDataRsaDebugXmlDump>,
reserved0 = 0x0, reserved1 = 0x0}

But now, after lines

214: BIO_free(bio);

and

215: return(key);

On 32 bit the key pointer returned from the function call contains the
same, sane info:

{klassSize = 88, objSize = 16, name = 0xb7d6a4ba "rsa", usage = 28, href =
0xb7d6a480 "http://www.w3.org/2000/09/xmldsig#RSAKeyValue",
  dataNodeName = 0xb7d6a4ae "RSAKeyValue", dataNodeNs = 0x81a2a40 "
http://www.w3.org/2000/09/xmldsig#", initialize = 0xb7d8f21e
<xmlSecOpenSSLKeyDataRsaInitialize>,
  duplicate = 0xb7d8f0fc <xmlSecOpenSSLKeyDataRsaDuplicate>, finalize =
0xb7d8f05d <xmlSecOpenSSLKeyDataRsaFinalize>, generate = 0xb7d9009e
<xmlSecOpenSSLKeyDataRsaGenerate>,
  getType = 0xb7d90d4b <xmlSecOpenSSLKeyDataRsaGetType>, getSize =
0xb7d90a79 <xmlSecOpenSSLKeyDataRsaGetSize>, getIdentifier = 0,
  xmlRead = 0xb7d8f8f3 <xmlSecOpenSSLKeyDataRsaXmlRead>, xmlWrite =
0xb7d90474 <xmlSecOpenSSLKeyDataRsaXmlWrite>, binRead = 0, binWrite = 0,
  debugDump = 0xb7d90c44 <xmlSecOpenSSLKeyDataRsaDebugDump>, debugXmlDump =
0xb7d90b3d <xmlSecOpenSSLKeyDataRsaDebugXmlDump>, reserved0 = 0x0,
reserved1 = 0x0}

But the pointer in the 64 bit system contains something entirely crazy in
key.value.id:

{klassSize = 137438953640, objSize = 140737338535578, name = 0x1c <Address
0x1c out of bounds>, usage = 4145147488, href = 0x7ffff711ee8e
"RSAKeyValue",
  dataNodeName = 0x7772c0 "http://www.w3.org/2000/09/xmldsig#", dataNodeNs
= 0x7ffff7348f62
"USH\203\354\030H\211\375H\205\377t'H\213\037H\205\333t\037\201;\247",
  initialize = 0x7ffff7348e4f <xmlSecOpenSSLKeyDataRsaDuplicate>, duplicate
= 0x7ffff7348dcf <xmlSecOpenSSLKeyDataRsaFinalize>,
  finalize = 0x7ffff7349caa <xmlSecOpenSSLKeyDataRsaGenerate>, generate =
0x7ffff734a844 <xmlSecOpenSSLKeyDataRsaGetType>,
  getType = 0x7ffff734a5bc <xmlSecOpenSSLKeyDataRsaGetSize>, getSize = 0,
getIdentifier = 0x7ffff7349561 <xmlSecOpenSSLKeyDataRsaXmlRead>,
  xmlRead = 0x7ffff734a00c <xmlSecOpenSSLKeyDataRsaXmlWrite>, xmlWrite = 0,
binRead = 0, binWrite = 0x7ffff734a750 <xmlSecOpenSSLKeyDataRsaDebugDump>,
  debugDump = 0x7ffff734a65c <xmlSecOpenSSLKeyDataRsaDebugXmlDump>,
debugXmlDump = 0, reserved0 = 0x0, reserved1 = 0x0}

At first i suspected the call on line 214: BIO_free(bio); of havign
troubles because the work space was being freed before the key pointer
being returned (I do not know nothing of openssl and BIO calls as a
library, sorry). but this should fail always, not work in one architecture
and not the other.

And in opendcp (you can check in the same repository where the ticket I
linked is. this file:
https://code.google.com/p/opendcp/source/browse/libopendcp/opendcp_xml_sign.cline
385) I check the returned key pointer as soon as it's returned (by
line 386). So nothing else was run on that pointer by that time in theory.

I can't grasp how or why this ocurrs. But this unchains a chain of events
that ends up causing a segfault which I posted there under the name Lars.
(second or third post)....

Versions I'm using:

xmlsec: 1.2.19 (this happens too with debian's default 1.2.18, I compiled
1.2.19 to enable debug symbols and have source).
openssl: 1.0.1e-2+deb7u4 (Basically 1.0.1e patched to close heartbleed).
gcc: 4.7.2-1

Everything is adm64 arch. The workign i386 versions of the libraries are
exactly the same versions as the 64 bit ones (including my own compiled
xmlsec with the same confiure flags).

Any feedback is highly welcome.

Lars.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.aleksey.com/pipermail/xmlsec/attachments/20140425/d330296f/attachment.html>