[xmlsec] upgrading from xmlsec1-1.2.16 to xmlsec1-1.2.29

[Floodeenjr, Thomas]

After some digging, it seems the problem is in the xmlsec key management.

In xmlsec, in app.c, xmlSecOpenSSLAppDefaultKeysMngrInit (), this is called:
keysStore = xmlSecKeyStoreCreate(xmlSecSimpleKeysStoreId);

When the simple keystore is initialized, there is a difference:

Works (xmlsec1-1.2.16):
-                              id            0x00007ff7e52a22a0 {xmlsec_test.exe!_xmlSecKeyStoreKlass xmlSecSimpleKeysStoreKlass} {klassSize=64 objSize=...}              const _xmlSecKeyStoreKlass *
                                klassSize              64           unsigned __int64
                                objSize 64           unsigned __int64
+                             name    0x00007ff7e5332470 "simple-keys-store"            const unsigned char *
                                initialize                0x00007ff7e5001e30 {xmlsec_test.exe!xmlSecSimpleKeysStoreInitialize(_xmlSecKeyStore *)}  int(*)(_xmlSecKeyStore *)
                                finalize  0x00007ff7e50020a0 {xmlsec_test.exe!xmlSecSimpleKeysStoreFinalize(_xmlSecKeyStore *)}                void(*)(_xmlSecKeyStore *)
                                findKey 0x00007ff7e50021b0 {xmlsec_test.exe!xmlSecSimpleKeysStoreFindKey(_xmlSecKeyStore *, const unsigned char *, _xmlSecKeyInfoCtx *)}     _xmlSecKey *(*)(_xmlSecKeyStore *, const unsigned char *, _xmlSecKeyInfoCtx *)
                                reserved0           0x0000000000000000     void *
                                reserved1           0x0000000000000000     void *
                                ret          0              int
+                             store     0x0000000000000000 <NULL>  _xmlSecKeyStore *

Fails (xmlsec1-1.2.29):
-                              id            0x00007ff6c2833f28 {xmlsec_test.exe!_xmlSecKeyStoreKlass xmlSecSimpleKeysStoreKlass} {klassSize=240518168632 ...}      const _xmlSecKeyStoreKlass *
                                klassSize              240518168632    unsigned __int64
                                objSize 140697802938688            unsigned __int64
+                             name    0x00007ff6c2554000 "H‰L$\bHƒìxHƒ¼$€"           const unsigned char *
                                initialize                0x00007ff6c25541f0 {xmlsec_test.exe!xmlSecSimpleKeysStoreFinalize(_xmlSecKeyStore *)}    int(*)(_xmlSecKeyStore *)
                                finalize  0x00007ff6c2554300 {xmlsec_test.exe!xmlSecSimpleKeysStoreFindKey(_xmlSecKeyStore *, const unsigned char *, _xmlSecKeyInfoCtx *)}     void(*)(_xmlSecKeyStore *)
                                findKey 0x0000000000000000     _xmlSecKey *(*)(_xmlSecKeyStore *, const unsigned char *, _xmlSecKeyInfoCtx *)
                                reserved0           0x0000000000000000     void *
                                reserved1           0x0000000000000000     void *
                                ret          48           int
+                             store     0x0000000000000001 {id=??? reserved0=??? reserved1=??? }    _xmlSecKeyStore *

Note the large size and the messed up name.

What would cause this? Is there some definition I am missing?

Thanks,
-Tom

From: xmlsec <xmlsec-bounces at aleksey.com> On Behalf Of Floodeenjr, Thomas
Sent: Tuesday, February 4, 2020 6:32 AM
To: xmlsec at aleksey.com
Subject: [xmlsec] upgrading from xmlsec1-1.2.16 to xmlsec1-1.2.29

Hello,

We are in the process of upgrading from openssl-1.0.2g to openssl-1.1.1d and from libxml2-2.7.8 to libxml2-2.9.9. We are also upgrading from xmlsec1-1.2.16 to xmlsec1-1.2.29.

The code we have been using for years to encrypt our xml is no longer working.

static bool S_EncryptXMLNode(xmlDocPtr _doc, xmlNodePtr _node, xmlSecKeysMngrPtr _keysMngr, xmlChar* _keyName)
{
    bool            localRet = false;
    xmlNodePtr      encDataNode = NULL;
    xmlSecEncCtxPtr encCtx = NULL;

    localRet = SDDInfrasecUtil::CreateEncryptedDataNode(_doc, xmlSecTypeEncElement, _keyName, &encDataNode);

    if (true == localRet) {
        encCtx = xmlSecEncCtxCreate(_keysMngr);
        if (NULL == encCtx) {
            fprintf(stderr,"Error: failed to create encryption context\n");
            xmlFreeNode(encDataNode);   encDataNode = NULL;
            localRet = false;
        } else {
            encCtx->defEncMethodId = xmlSecTransformAes128CbcId;
        }
    }

    if ((NULL != encDataNode) && (NULL != encCtx)) {
       if (xmlSecEncCtxXmlEncrypt(encCtx, encDataNode, _node) < 0) {
            fprintf(stderr,"Error: encryption failed\n");
            xmlFreeNode(encDataNode);       encDataNode = NULL;
            xmlSecEncCtxDestroy(encCtx);    encCtx = NULL;
            localRet = false;
        } else {
            encDataNode = NULL;
        }
    }
    if (NULL != encCtx)         xmlSecEncCtxDestroy(encCtx);
    if (NULL != encDataNode)    xmlFreeNode(encDataNode);

    return localRet;
}

Encryption fails on this line: if (xmlSecEncCtxXmlEncrypt(encCtx, encDataNode, _node) < 0) {
It worked before the upgrade.

Is there something we need to change?

Thanks,
-Tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.aleksey.com/pipermail/xmlsec/attachments/20200204/d639b603/attachment-0001.htm>