Prev: AES counter content and same key/nonce
Next: Another informative pronouncement from Smug Doug Gwyn
From: Peac on 7 Feb 2010 01:02

"Ilmari Karonen" <usenet2(a)vyznev.invalid> wrote in message
news:slrnhmnrh6.eet.usenet2(a)melkki.cs.helsinki.fi...
> On 2010-02-05, Peac <peac1972(a)hotmail_munged.com> wrote:
>> "Ilmari Karonen" <usenet2(a)vyznev.invalid> wrote in message
>> news:slrnhmlaic.eet.usenet2(a)melkki.cs.helsinki.fi...
>>> On 2010-02-04, Peac <peac1972(a)hotmail_munged.com> wrote:
>>>>
>>>> Can I get compatibility for the same in my Java Program. i.e. if the
>>>> Windows Native program generates the HMAC-MD5, will a client written
>>>> in Java be able to validate the HMAC computed my the Microsoft APIs
>>>> & sent over the wire to the client?
>>>
>>> I don't see any reason it shouldn't. There's nothing ambiguous about
>>> the definition of HMAC-MD5, so the result should be the same on any
>>> platform. Of course, it's always prudent to test such things, but I
>>> would be quite surprised if there were any incompatibilities.
>>
>> Thanks everyone for your help. I have this code working between 2 Native
>> C Programs. However, as I suspected the Java stuff doesn't match.
>>
>> From googling it looks like the Microsoft Crypto API function
>> CryptDeriveKey
>> is the problem. I call this function to derive a key from my password.
>
> Hmm. Internally, the keys to HMAC-MD5 are 512 bit strings. According
> to RFC 2104, if the supplied key is shorter than 512 bits, it should
> be padded with zeroes to the full length, while a longer key should be
> first hashed and then padded.
>
> It seems Microsoft is doing something else; from what I could tell by
> looking at the docs, it looks like they might simply be always hashing
> the key, but I couldn't find anything really definite about what goes
> on under the hood. You could test it with keys longer than 512 bits
> and see if they give the same results then.

This may be the case, but I think the problem is happening even before that.
The Key Derivation function in the MS API seems to be non-standard.
Is there a way I can write derive a key from a password which can be
standard across MS & Java.

> I'm not sure if there's any way to make the Microsoft implementation
> behave in an RFC-compliant manner. If not, you could always implement
> the HMAC construction yourself. It's not very difficult. In
> pseudo-Java/C#:
>
> byte[64] ipad = ((byte) 0x5c).repeat(64);
> byte[64] opad = ((byte) 0x36).repeat(64);
>
> byte[16] HMAC_MD5 (byte[] key, byte[] input) {
> if (key.length > 64) {
> MD5Hash keyHash = new MD5Hash ();
> keyHash.addData(key);
> key = keyHash.getResult();
> }
> byte[64] expKey = key.padToLength(64);
>
> MD5Hash innerHash = new MD5Hash ();
> innerHash.addData(key.xor(ipad));
> innerHash.addData(input);
> byte[16] temp = innerHash.getResult();
>
> MD5Hash outerHash = new MD5Hash ();
> outerHash.addData(key.xor(opad));
> outerHash.addData(temp);
> return outerHash.getResult();
> }
>
> That's off the top of my head, but I think it should do it unless I've
> made some silly mistake.
>
> --
> Ilmari Karonen
> To reply by e-mail, please replace ".invalid" with ".net" in address.


From: Ilmari Karonen on 7 Feb 2010 12:50
On 2010-02-07, Peac <peac1972(a)hotmail_munged.com> wrote:
> "Ilmari Karonen" <usenet2(a)vyznev.invalid> wrote in message
> news:slrnhmnrh6.eet.usenet2(a)melkki.cs.helsinki.fi...
>>
>> Hmm. Internally, the keys to HMAC-MD5 are 512 bit strings. According
>> to RFC 2104, if the supplied key is shorter than 512 bits, it should
>> be padded with zeroes to the full length, while a longer key should be
>> first hashed and then padded.
>>
>> It seems Microsoft is doing something else; from what I could tell by
>> looking at the docs, it looks like they might simply be always hashing
>> the key, but I couldn't find anything really definite about what goes
>> on under the hood. You could test it with keys longer than 512 bits
>> and see if they give the same results then.
>
> This may be the case, but I think the problem is happening even before that.
> The Key Derivation function in the MS API seems to be non-standard.
> Is there a way I can write derive a key from a password which can be
> standard across MS & Java.

I'm sure there is, but the problem would seem to lie in getting the MS
crypto API to accept that key. As far as I can tell, Microsoft
doesn't document what goes inside a HCRYPTKEY, so you can't create one
yourself. You could try to import one from a PUBLICKEYSTRUC blob, but
I couldn't so far find any documentation about how the key is actually
stored inside one of those, either.

Googling for "HCRYPTKEY" did turn up this thread:
<http://www.ureader.com/msg/1659409.aspx>. Maybe it'll do as a
starting point.

Anyway, you could also use the code I suggested in my last post; that
should be compatible with any RFC 2104 -compliant implementation, and
would save you from having to deal with HCRYPTKEYs at all. All you'd
need is a plain old unkeyed MD5 implementation, which even Microsoft
should be able to provide.

Here's the pseudocode again, for reference:

>> byte[64] ipad = ((byte) 0x5c).repeat(64);
>> byte[64] opad = ((byte) 0x36).repeat(64);
>>
>> byte[16] HMAC_MD5 (byte[] key, byte[] input) {
>> if (key.length > 64) {
>> MD5Hash keyHash = new MD5Hash ();
>> keyHash.addData(key);
>> key = keyHash.getResult();
>> }
>> byte[64] expKey = key.padToLength(64);
>>
>> MD5Hash innerHash = new MD5Hash ();
>> innerHash.addData(key.xor(ipad));
>> innerHash.addData(input);
>> byte[16] temp = innerHash.getResult();
>>
>> MD5Hash outerHash = new MD5Hash ();
>> outerHash.addData(key.xor(opad));
>> outerHash.addData(temp);
>> return outerHash.getResult();
>> }

(Ps. If you're designing this from the ground up, why not use a more
modern hash than MD5? Even though none of the currently known attacks
on MD5 actually apply when used in HMAC, they still show that its
security isn't quite as solid as it was meant to be.)

--
Ilmari Karonen
To reply by e-mail, please replace ".invalid" with ".net" in address.
From: Peac on 8 Feb 2010 08:13

On 7 Feb, 22:50, Ilmari Karonen <usen...(a)vyznev.invalid> wrote:
> Anyway, you could also use the code I suggested in my last post; that
> should be compatible with any RFC 2104 -compliant implementation, and
> would save you from having to deal with HCRYPTKEYs at all. All you'd
> need is a plain old unkeyed MD5 implementation, which even Microsoft
> should be able to provide.

Thank you for your reply.
Pardon, my ignorance - but what's the difference between a keyed hash & an
unkeyed hash?

> Here's the pseudocode again, for reference:
>
> >> byte[64] ipad = ((byte) 0x5c).repeat(64);
> >> byte[64] opad = ((byte) 0x36).repeat(64);
>
> >> byte[16] HMAC_MD5 (byte[] key, byte[] input) {


Does the key parameter to the HMAC_MD5 function represent
a password or a symmetric key derived from a password?

> >> if (key.length > 64) {
> >> MD5Hash keyHash = new MD5Hash ();
> >> keyHash.addData(key);
> >> key = keyHash.getResult();
> >> }
> >> byte[64] expKey = key.padToLength(64);

You don't seem to be using this expKey anywhere?

>
> >> MD5Hash innerHash = new MD5Hash ();
> >> innerHash.addData(key.xor(ipad));
> >> innerHash.addData(input);
> >> byte[16] temp = innerHash.getResult();
>
> >> MD5Hash outerHash = new MD5Hash ();
> >> outerHash.addData(key.xor(opad));
> >> outerHash.addData(temp);
> >> return outerHash.getResult();
> >> }
>
> (Ps. If you're designing this from the ground up, why not use a more
> modern hash than MD5?

Thank you, will look into it.

> Even though none of the currently known attacks
> on MD5 actually apply when used in HMAC, they still show that its
> security isn't quite as solid as it was meant to be.)


From: Ilmari Karonen on 8 Feb 2010 09:36
On 2010-02-08, Peac <peac1972(a)hotmail_munged.com> wrote:
> On 7 Feb, 22:50, Ilmari Karonen <usen...(a)vyznev.invalid> wrote:
>> Anyway, you could also use the code I suggested in my last post; that
>> should be compatible with any RFC 2104 -compliant implementation, and
>> would save you from having to deal with HCRYPTKEYs at all. All you'd
>> need is a plain old unkeyed MD5 implementation, which even Microsoft
>> should be able to provide.
>
> Thank you for your reply.
> Pardon, my ignorance - but what's the difference between a keyed hash & an
> unkeyed hash?

I was using the term "unkeyed hash" simply to emphasize the difference
between an ordinary hash like MD5 (which takes only one parameter: the
input string) and a MAC like HMAC-MD5 (which takes an additional
parameter: the key).

>> Here's the pseudocode again, for reference:
>>
>> >> byte[64] ipad = ((byte) 0x5c).repeat(64);
>> >> byte[64] opad = ((byte) 0x36).repeat(64);
>>
>> >> byte[16] HMAC_MD5 (byte[] key, byte[] input) {
>
> Does the key parameter to the HMAC_MD5 function represent
> a password or a symmetric key derived from a password?

It can be basically anything, as long as it has sufficient entropy not
to be guessable by brute force. It could be a passphrase, or some key
derived from a passphrase, or a random key stored in a keyfile, etc.
HMAC isn't too picky about what you give it as the key, since that key
is going to be fed to a hash function anyway.

>> >> if (key.length > 64) {
>> >> MD5Hash keyHash = new MD5Hash ();
>> >> keyHash.addData(key);
>> >> key = keyHash.getResult();
>> >> }
>> >> byte[64] expKey = key.padToLength(64);
>
> You don't seem to be using this expKey anywhere?

Oops, sorry. Both of the occurrences of "key" below this line should
read "expKey" instead. I've made the change below:

>> >> MD5Hash innerHash = new MD5Hash ();
>> >> innerHash.addData(expKey.xor(ipad));
>> >> innerHash.addData(input);
>> >> byte[16] temp = innerHash.getResult();
>>
>> >> MD5Hash outerHash = new MD5Hash ();
>> >> outerHash.addData(expKey.xor(opad));
>> >> outerHash.addData(temp);
>> >> return outerHash.getResult();
>> >> }

BTW, I just thought of something... _if_ the Microsoft implementation
is simply hashing the key regardless of its length, while the Java
implementation follows RFC 2104 and only hashes it if it's longer than
64 bytes, then you might be able to make the Java implementation
behave the same as the Microsoft one (at a small performance cost)
simply by hashing the key in advance. It might be worth a try. I'm
not sure why I didn't think of that when I wrote my last post.

--
Ilmari Karonen
To reply by e-mail, please replace ".invalid" with ".net" in address.
From: Peac on 8 Feb 2010 10:00

"Ilmari Karonen" <usenet2(a)vyznev.invalid> wrote in message
news:slrnhmtvb2.au7.usenet2(a)melkki.cs.helsinki.fi...
> On 2010-02-07, Peac <peac1972(a)hotmail_munged.com> wrote:
>> "Ilmari Karonen" <usenet2(a)vyznev.invalid> wrote in message
>> news:slrnhmnrh6.eet.usenet2(a)melkki.cs.helsinki.fi...
>>>
>>> Hmm. Internally, the keys to HMAC-MD5 are 512 bit strings. According
>>> to RFC 2104, if the supplied key is shorter than 512 bits, it should
>>> be padded with zeroes to the full length, while a longer key should be
>>> first hashed and then padded.
>>>
>>> It seems Microsoft is doing something else; from what I could tell by
>>> looking at the docs, it looks like they might simply be always hashing
>>> the key, but I couldn't find anything really definite about what goes
>>> on under the hood. You could test it with keys longer than 512 bits
>>> and see if they give the same results then.
>>
>> This may be the case, but I think the problem is happening even before
>> that.
>> The Key Derivation function in the MS API seems to be non-standard.
>> Is there a way I can write derive a key from a password which can be
>> standard across MS & Java.
>
> I'm sure there is, but the problem would seem to lie in getting the MS
> crypto API to accept that key. As far as I can tell, Microsoft
> doesn't document what goes inside a HCRYPTKEY, so you can't create one
> yourself. You could try to import one from a PUBLICKEYSTRUC blob, but
> I couldn't so far find any documentation about how the key is actually
> stored inside one of those, either.

Here is code to convert an array of bytes into a HCRYPTKEY
http://msdn.microsoft.com/en-us/library/aa382383(VS.85).aspx

Now that is avaialable - can I write my own key derivation function
to generate a key from a password using MS Crypto API - which can
be replicated in Java?

Is there some algorithm which I use to generate the key?


First  |  Prev  | 
Pages: 1 2
Prev: AES counter content and same key/nonce
Next: Another informative pronouncement from Smug Doug Gwyn