[xmlsec] xmlsec1-1.2.16 to xmlsec1-1.2.29 decrypt problems

[Floodeenjr, Thomas]

Alexsey,

It seems we found the problem. Before we decrypt, we call doc = xmlReadFile( filePath, NULL, XML_PARSE_HUGE ); It seems the XML_PARSE_HUGE flag is not honored in libxml2. I am not sure if this is a bug in libxml2, or if it is a new "feature". We are looking for a way to work around this. If we hard-code it in xmlCtxtUseOptionsInternal(), i.e., ctxt->options |= XML_PARSE_HUGE;, then our code decrypts fine.

Thanks for your replies.

-Tom



-----Original Message-----
From: Aleksey Sanin <aleksey at aleksey.com> 
Sent: Friday, March 27, 2020 10:32 AM
To: Floodeenjr, Thomas <thomas_floodeenjr at mentor.com>; xmlsec at aleksey.com
Subject: Re: [xmlsec] xmlsec1-1.2.16 to xmlsec1-1.2.29 decrypt problems

Hi Thomas,

I am not aware of any limitations on file sizes. I would suggest to either set a breakpoint or dump the decrypted data before the call to xmlParseInNodeContext() to see what's wrong. There are a couple options I can think of:
1) Decryption is completely incorrect and you will see junk in the buffer.
2) Decryption is correct but the data is truncated.

Let me know what do you find.

Aleksey

On 3/27/20 6:07 AM, Floodeenjr, Thomas wrote:
> Aleksey,
> 
> Is there a limitation with xmlsec1-1.2.29 on the size of the encrypted 
> XML that is being decrypted? The file seems to encrypt OK, but will 
> not decrypt. (290 MB). Smaller files work fine both ways. We are using 
> libxml2-2.9.9 and openssl-1.1.1d.  (Previously we used libxml2-2.7.8 
> and openssl-1.0.2g, and it worked fine.)
> 
> Thanks,
> -Tom
> 
> -----Original Message-----
> From: xmlsec <xmlsec-bounces at aleksey.com> On Behalf Of Floodeenjr, 
> Thomas
> Sent: Thursday, March 26, 2020 12:52 PM
> To: Aleksey Sanin <aleksey at aleksey.com>; xmlsec at aleksey.com
> Subject: Re: [xmlsec] xmlsec1-1.2.16 to xmlsec1-1.2.29 decrypt 
> problems
> 
> It returns here with -1
> 
>      if(ret != XML_ERR_OK) {
>          xmlSecXmlError("xmlParseInNodeContext", NULL);
>          return(-1);
>      }
> 
> -----Original Message-----
> From: Aleksey Sanin <aleksey at aleksey.com>
> Sent: Thursday, March 26, 2020 11:05 AM
> To: Floodeenjr, Thomas <thomas_floodeenjr at mentor.com>; 
> xmlsec at aleksey.com
> Subject: Re: [xmlsec] xmlsec1-1.2.16 to xmlsec1-1.2.29 decrypt 
> problems
> 
> What error do you get?
> 
> Aleksey
> 
> On 3/26/20 8:26 AM, Floodeenjr, Thomas wrote:
>> Hello,
>>
>> We recently upgraded from xmlsec1-1.2.16 to xmlsec1-1.2.29. We have 
>> some data created in 1.2.16 that we can decrypt, but it fails to 
>> decrypt in 1.2.29.
>>
>>                 Application.exe!xmlSecCheckNodeName(_xmlNode * const 
>> cur, const unsigned char * name, const unsigned char * ns) Line 210 C
>>
>>>                
>>> Application.exe!xmlSecEncCtxEncDataNodeRead(_xmlSecEncCtx * encCtx, 
>>> _xmlNode * node) Line 696      C
>>
>>                 
>> Application.exe!xmlSecEncCtxDecryptToBuffer(_xmlSecEncCtx
>> * encCtx, _xmlNode * node) Line 597      C
>>
>>                 Application.exe!xmlSecEncCtxDecrypt(_xmlSecEncCtx * 
>> encCtx, _xmlNode * node) Line 524      C
>>
>> Other items do decrypt fine with both versions. Any ideas what we can 
>> look into?
>>
>> Thanks,
>>
>> -Tom
>>
>>
>> _______________________________________________
>> xmlsec mailing list
>> xmlsec at aleksey.com
>> http://www.aleksey.com/mailman/listinfo/xmlsec
>>
> _______________________________________________
> xmlsec mailing list
> xmlsec at aleksey.com
> http://www.aleksey.com/mailman/listinfo/xmlsec
>