[RFC][VOTE] Improve hash_hkdf() parameter

php.internals

Yasuo Ohgaki

9 years ago
Hi all, Since hash_hkdf() is in PHP 7.1.2, I restarted vote. I posted previous announce in discussion thread by mistake. https://wiki.php.net/rfc/improve_hash_hkdf_parameter Vote start: 2017-03-26 Vote end: 2017-04-07 UTC 23:59:59 Current hash_hkdf() function signature does not make sense. - HKDF is KEY derivation function, yet derivation KEY is the last option. - hash_hkdf() is simple hash_hmac() extension, yet it has totally different signature. - Return value is binary unlike other hash functions. - The signature is INSECURE. Current signature is overly optimized very limited crypto operation and cannot be optimal by above reasons. Fortunately, almost all users are not using current hash_hkdf(). It's only in 7.1.2/7.1.3 now. We should avoid yet another new inconsistent and insecure function. It would be better to be fixed ASAP, IMHO. I suggest you to disclose the reason why against this change. Otherwise, you may be considered you don't understand crypto basic. i.e. HKDF(IKM) security depends on PRK being secure. To make PRK secure or more secure, "salt" parameter is required. "length" is irrelevant for security. Thank you for voting.
-- Yasuo Ohgaki yohgaki@ohgaki.net

Yasuo Ohgaki

9 years ago
Hi all, On Sun, Mar 26, 2017 at 7:29 AM, Yasuo Ohgaki <yohgaki@ohgaki.net> wrote:
> I suggest you to disclose the reason why against this change. > Otherwise, you may be considered you don't understand crypto basic. > i.e. HKDF(IKM) security depends on PRK being secure. To make PRK > secure or more secure, "salt" parameter is required. "length" is > irrelevant > for security. >
I'll try to explain a bit more by examples. HKDF is designed to obtain the best possible "cryptographically strong hash value" (key) for various key derivation operations. Current signature could lead to insecure/wrong usages. (We have similar experience with our PHP functions. e.g. uniqid, crypt, etc) Example #1 : Deriving strong 256 bit AES key from 128 bit AES key. $new_key = hash_hkdf('sha256', $AES_128bit_key, 32); // Derive 256 bit AES key from 128 bit key // No additional entropy, thus $new_key is not strong 256 bit AES key. // Far from the best possible. Users must not do this with HKDF. The same $new_key quality can be obtained by simple SHA-256 hashing which is faster. Without "strong derivation key", HKDF is not useful at all. The optimal way is $new_key = hash_hkdf('sha256', $AES_128bit_key, 0, '', $strong_derivation_key); // where $strong_derivation_key = random_bytes(32); or like. Example #2 : Deriving strong key from week key such as user entered password $new_key = hash_hkdf('sha256', 'p@ssword'); // Almost the same as hash('sha256', 'p@ssword'); All of us should know how bad this is. // Far from the best possible. Users must not do this with HKDF. The same could be done with simple hash(). Users must provide cryptographically strong derivation key, otherwise HKDF is useless. $new_key = hash_hkdf('sha256', 'p@ssword', 0, '', $strong_derivation_key); // where $strong_derivation_key = random_bytes(32); or like. // Since input key material is weak, $strong_derivation_key must be secret Example #3 : Deriving CSRF token from secret seed $new_key = hash_hkdf('sha256', $secret_seed, 0, $version); // Almost the same as hash('sha256', $secret_seed . $version); // Far from the best possible. Users must not do this with HKDF. The same could be done with simple hash(). Users must provide cryptographically strong derivation key, otherwise HKDF is useless. $new_key = hash_hkdf('sha256', $secret_seed, 0, $version, $strong_derivation_key); // where $strong_derivation_key = random_bytes(32); or like. There are looong lists of this kind of insecure/wrong usage with current signature. If you understand how to derive "strong key" by HKDF, you should realize current hash_hkdf() function signature is far from the best. Detailed rationale is explained the PHP RFC, but it seems many of us does not understand this. HKDF is supposed to derive "strong key", why should we encourage "weak key" derivations with non optimal signature? Regards, P.S. I strongly objected the current signature before 7.1 merge. Shouldn't committer write RFC before commit in the first place? Especially for released versions.
-- Yasuo Ohgaki yohgaki@ohgaki.net

Stephen Reay

9 years ago
> > I'll try to explain a bit more by examples. >
Hi Yasuo, It sounds to me like it is *possible* to currently use hash_hkdf() in a secure manner, but that you (and some others?) feel the arg order and default args are not conducive to safe/secure usage. Given that the function is live in the wild, massively changing the order of things and defaults is an instant red flag for myself, and I believe a lot of other people. To me this sounds more like an issue that could be relatively quickly improved by a documentation update that highlights how to securely use the function. Yes, if there are more secure defaults that would be nice, but that ship has sailed, and the function was on it. Just my 2 cents. Cheers Stephen

Yasuo Ohgaki

9 years ago
Hi Stephen, On Mon, Mar 27, 2017 at 1:09 PM, Stephen Reay <php-lists@koalephant.com> wrote:
> > It sounds to me like it is *possible* to currently use hash_hkdf() in a > secure manner, but that you (and some others?) feel the arg order and > default args are not conducive to safe/secure usage. >
It's _possible_, of course. Problem is _new_ function has - insecure signature (it ignores strong RFC 5689 recommendation) - inconsistent signature and return value (hash() and hash_hmac()) - no major use(application) for PHP apps (Length has almost no use with PHP apps) If users would like to generate arbitrary length hash from existing hash value with _insecure_ way, they should use new SHA-3 standards, i.e. SHA-3 already has 2 SHAKE algorithms that generate arbitrary length hash value, SHAKE128(M, d) and SHAKE256(M, d). No reason to encourage less secure HKDF usage to obtain arbitrary length hash value. Current hash_hkdf() signature does not make much sense with regard to cryptographically, consistency and expected usage. Given that the function is live in the wild, massively changing the order
> of things and defaults is an instant red flag for myself, and I believe a > lot of other people. >
Aside from it should not be merged into PHP 7.1 in the first place. There are only 2 (or 3) bug fix versions released. Fixing mistake ASAP is better. IMHO. To me this sounds more like an issue that could be relatively quickly
> improved by a documentation update that highlights how to securely use the > function. >
While documentation may work, it seems silly for me to write, Even if "salt" is the last optional parameter, users must set appropriate "salt" whenever it is possible for maximum key security. for new function. Yes, if there are more secure defaults that would be nice, but that ship
> has sailed, and the function was on it.
Thank you for your comment. I would like to try to fix it at least. To avoid this kind of confusions, we are better to have RFC if there is strong objection. Regards,
-- Yasuo Ohgaki yohgaki@ohgaki.net

Yasuo Ohgaki

9 years ago
Hi all, - insecure signature (it ignores strong RFC 5689 recommendation) s/RFC 5689/RFC 5869/ On Sat, Apr 1, 2017 at 11:27 AM, Yasuo Ohgaki <yohgaki@ohgaki.net> wrote:
> > Given that the function is live in the wild, massively changing the order >> of things and defaults is an instant red flag for myself, and I believe a >> lot of other people. >> > > Aside from it should not be merged into PHP 7.1 in the first place. > There are only 2 (or 3) bug fix versions released. Fixing mistake ASAP is > better. IMHO. > > > To me this sounds more like an issue that could be relatively quickly >> improved by a documentation update that highlights how to securely use the >> function. >> > > While documentation may work, it seems silly for me to write, > > Even if "salt" is the last optional parameter, users must set > appropriate "salt" whenever it is possible for maximum key security. >
Another possible resolution could be reverting hash_hkdf() merge from 7.1 branch. Basic hash_hkdf() operation could be done by hash_hmac() easily. The merge should have had PHP RFC. Reverting hash_hkdf() merge may work better. Regards,
-- Yasuo Ohgaki yohgaki@ohgaki.net

Joe Watkins

9 years ago
Morning, This RFC was left open for 5 days past the end of voting as declared on the RFC. I have closed the vote, and moved it out of voting section on RFC index. Cheers Joe On Sat, Apr 1, 2017 at 3:50 AM, Yasuo Ohgaki <yohgaki@ohgaki.net> wrote:

Yasuo Ohgaki

9 years ago
Hi Joe, On Wed, Apr 12, 2017 at 7:46 PM, Joe Watkins <pthreads@pthreads.org> wrote:
> This RFC was left open for 5 days past the end of voting as declared on > the RFC. >
Thank you, I forgot about this. IMHO, it's a shame for us we should have inconsistent and insecure function signature for a new function. I'm going to update the manual to add warning notes and example usages like advanced CRFS token dedicated for specific URL with expiration time. I can think of length option only usage, but I cannot think usage that could be useful for majority of PHP users like advanced CSRF token. Andrey, Could you give us some length only and length/info only example that could be useful for most PHP users. It should be safe and recommended usage. I suppose you should have some good examples. Thank you.
-- Yasuo Ohgaki yohgaki@ohgaki.net

Pieter Hordijk

9 years ago
----- Original Message -----
> From: "Yasuo Ohgaki" <yohgaki@ohgaki.net> > To: "Joe Watkins" <pthreads@pthreads.org>, "Andrey Andreev" <narf@devilix.net> > Cc: internals@lists.php.net > Sent: Thursday, April 13, 2017 1:07:19 AM > Subject: Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
> Hi Joe, > > On Wed, Apr 12, 2017 at 7:46 PM, Joe Watkins <pthreads@pthreads.org> wrote: > >> This RFC was left open for 5 days past the end of voting as declared on >> the RFC. >> > > Thank you, I forgot about this. > IMHO, it's a shame for us we should have inconsistent and insecure function > signature for a new function. > > I'm going to update the manual to add warning notes and example usages > like advanced CRFS token dedicated for specific URL with expiration time. > > I can think of length option only usage, but I cannot think usage that could > be useful for majority of PHP users like advanced CSRF token.
Is this really something we need in our official docs instead of for example on a personal blog? To be honest I am afraid of ending up with something like the current state of the session docs. Which are imo way too broad / opinionated, non English, contains utterly confusing examples and / or flat out wrong and broken examples. Above already resulted in a stream of docs bugs regarding session pages and a lot of confused readers. By all means describe how functions work, but don't confuse readers with things most people won't ever need or are better suited as a (series of) blog posts / Stack Overflow post(s). My €0.02 cc-ing docs discussion to get them also involved in case somebody of the docs team has an opinion.

Yasuo Ohgaki

9 years ago
Hi Peiter, On Thu, Apr 13, 2017 at 5:11 PM, Pieter Hordijk <info@pieterhordijk.com> wrote:
> To be honest I am afraid of ending up with something like the current state > of the session docs. Which are imo way too broad / opinionated, non > English, > contains utterly confusing examples and / or flat out wrong and broken > examples. > Above already resulted in a stream of docs bugs regarding session pages > and a lot of confused readers. >
You may consider my opinion as my personal opinion. I don't know of other than me who had that opinion then. After our session discussion, it seems OWASP adopted most of discussed elements in their doc ;) https://www.owasp.org/index.php/Session_Management_Cheat_Sheet Regards, P.S. My opinion is based on RFC 5869. In addition, it's totally nonsense to me to have completely different signature for hash_hkdf(). See the difference hash_hmac() and hash_pbkdf2(). hash_pbkdf2() is older KDF function. I should have mention in the RFC :(
-- Yasuo Ohgaki yohgaki@ohgaki.net

Yasuo Ohgaki

9 years ago
Hi Pieter, On Thu, Apr 13, 2017 at 5:38 PM, Yasuo Ohgaki <yohgaki@ohgaki.net> wrote:
> > On Thu, Apr 13, 2017 at 5:11 PM, Pieter Hordijk <info@pieterhordijk.com> > wrote: > >> To be honest I am afraid of ending up with something like the current >> state >> of the session docs. Which are imo way too broad / opinionated, non >> English, >> contains utterly confusing examples and / or flat out wrong and broken >> examples. >> Above already resulted in a stream of docs bugs regarding session pages >> and a lot of confused readers. >> > > You may consider my opinion as my personal opinion. I don't know of other > than > me who had that opinion then. > > After our session discussion, it seems OWASP adopted most of discussed > elements in their doc ;) >
I'm not exactly sure which part you consider as personal blog. Current session management is too loose and insecure in many ways. Since mandatory features for precise session management are not implemented, the doc is intermediate. I'm willing to improve the doc and appreciate improvement suggestions always. Feel free to send to my personal mail address. Required information for precise and secure session management should be in Precise Session Management RFC https://wiki.php.net/rfc/precise_session_management I appreciate if one could add missing documentation for precise session management. Regards,
-- Yasuo Ohgaki yohgaki@ohgaki.net

wout van gils

9 years ago
Kan iemand mij eindelijk eens uitschrijven.?

Jan Ehrhardt

9 years ago
wout van gils in php.internals (Thu, 13 Apr 2017 15:13:40 +0000):
>Kan iemand mij eindelijk eens uitschrijven.?
Dat moet je zelf doen: http://php.net/mailing-lists.php Onderaan.
-- Jan

wout van gils

9 years ago
Kan iemand mij eindelijk eens uitschrijven.? ?? ________________________________ Van: Pieter Hordijk <info@pieterhordijk.com> Verzonden: donderdag 13 april 2017 08:11 Aan: Yasuo Ohgaki CC: Joe Watkins; Andrey Andreev; internals@lists.php.net; phpdoc@lists.php.net Onderwerp: [PHP-DOC] Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter ----- Original Message -----
> From: "Yasuo Ohgaki" <yohgaki@ohgaki.net> > To: "Joe Watkins" <pthreads@pthreads.org>, "Andrey Andreev" <narf@devilix.net> > Cc: internals@lists.php.net > Sent: Thursday, April 13, 2017 1:07:19 AM > Subject: Re: [PHP-DEV] [RFC][VOTE] Improve hash_hkdf() parameter
> Hi Joe, > > On Wed, Apr 12, 2017 at 7:46 PM, Joe Watkins <pthreads@pthreads.org> wrote: > >> This RFC was left open for 5 days past the end of voting as declared on >> the RFC. >> > > Thank you, I forgot about this. > IMHO, it's a shame for us we should have inconsistent and insecure function > signature for a new function. > > I'm going to update the manual to add warning notes and example usages > like advanced CRFS token dedicated for specific URL with expiration time. > > I can think of length option only usage, but I cannot think usage that could > be useful for majority of PHP users like advanced CSRF token.
Is this really something we need in our official docs instead of for example on a personal blog? To be honest I am afraid of ending up with something like the current state of the session docs. Which are imo way too broad / opinionated, non English, contains utterly confusing examples and / or flat out wrong and broken examples. Above already resulted in a stream of docs bugs regarding session pages and a lot of confused readers. By all means describe how functions work, but don't confuse readers with things most people won't ever need or are better suited as a (series of) blog posts / Stack Overflow post(s). My €0.02 cc-ing docs discussion to get them also involved in case somebody of the docs team has an opinion.

Yasuo Ohgaki

9 years ago
Hi Pieter and all, On Thu, Apr 13, 2017 at 5:11 PM, Pieter Hordijk <info@pieterhordijk.com> wrote:
> Is this really something we need in our official docs instead of for > example > on a personal blog? >
I wrote draft doc patch. Please verify. Index: en/reference/hash/functions/hash-hkdf.xml =================================================================== --- en/reference/hash/functions/hash-hkdf.xml (リビジョン 342317) +++ en/reference/hash/functions/hash-hkdf.xml (作業コピー) @@ -3,7 +3,7 @@ <refentry xml:id="function.hash-hkdf" xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink"> <refnamediv> <refname>hash_hkdf</refname> - <refpurpose>Generate a HKDF key derivation of a supplied key input</refpurpose> + <refpurpose>Derive secure new key from existing key by using HKDF</refpurpose> </refnamediv> <refsect1 role="description"> &reftitle.description; @@ -16,6 +16,20 @@ <methodparam choice="opt"><type>string</type><parameter>salt</parameter><initializer>''</initializer></methodparam> </methodsynopsis> + <para> + RFC 5869 defines HKDF (HMAC based Key Derivation Function) which + is general purpose KDF. HKDF could be useful for many PHP + applications that require temporary keys, such CSRF token, + pre-signed key for URI, password for password protected + URI, and so on. + </para> + <note> + <para> + When info and length + is not required for your program, more efficient + <function>hash_hmac</function> could be used instead. + </para> + </note> </refsect1> <refsect1 role="parameters"> &reftitle.parameters; @@ -25,7 +39,7 @@ <term><parameter>algo</parameter></term> <listitem> <para> - Name of selected hashing algorithm (i.e. "sha256", "sha512", "haval160,4", etc..) + Name of selected hashing algorithm (i.e. "sha3-256", "sha3-512", "sha256", "sha512", "haval160,4", etc..) See <function>hash_algos</function> for a list of supported algorithms. <note> <para> @@ -39,7 +53,7 @@ <term><parameter>ikm</parameter></term> <listitem> <para> - Input keying material (raw binary). Cannot be empty. + Input keying material. Cannot be empty. </para> </listitem> </varlistentry> @@ -60,7 +74,8 @@ <term><parameter>info</parameter></term> <listitem> <para> - Application/context-specific info string. + Application/context-specific info string. Info is intended for + public information such as user ID, protocol version, etc. </para> </listitem> </varlistentry> @@ -71,8 +86,32 @@ Salt to use during derivation. </para> <para> - While optional, adding random salt significantly improves the strength of HKDF. + While optional, adding random salt significantly improves the + strength of HKDF. Salt could be either secret or + non-secret. It is used as "Pre Shared Key" in many use cases. + Strong value is preferred. e.g. Use <function>random_bytes</function>. + Optimal salt size is size of used hash algorithm. </para> + <warning> + <para> + Although salt is the last optional parameter, salt is the + most important parameter for key security. Omitted salt is + indication of inappropriate design in most cases. Users must + set appropriate salt value whenever it is possible. Omit salt + only when it cannot be used. + </para> + <para> + Strong salt is mandatory and must be kept secret when input + key is weak, otherwise input key security will not be kept. + Even when input key is strong, providing strong salt is the + best practice for the best possible key security. + </para> + <para> + Salt must not be able to be controlled by users. i.e. User + must not be able to set salt value and get derived key. User + controlled salt allows input key analysis to attackers. + </para> + </warning> </listitem> </varlistentry> </variablelist> @@ -101,6 +140,99 @@ &reftitle.examples; <para> <example> + <title>URI specific CSRF token that supports expiration by <function>hash_hkdf</function></title> + <programlisting role="php"> +<![CDATA[ +<?php +define('CSRF_TOKEN_EXPIRE', 180); // CSRF token expiration +define('CSRF_TOKENS', 5); // Last 5 CSRF tokens are valid + +/************************************** + * Implementation note + * + * It uses "counter" for CSRF expiration management. + * "counter" is very low entropy, but input key is strong and + * CSRF_TOKEN_SEED is short term key. It should be OK. + * + * This CSRF token implementation has pros and cons + * + * Pros + * - A CSRF token is valid only for specific URI. + * - No database is required for URI specific CSRF tokens. + * - Only CSRF token is required. i.e. No timestamp parameter. + * - When user is active, a CSRF token is valid upto CSRF_TOKEN_EXPIRE * CSRF_TOKENS sec. + * - Even when user had long idle time, CSRF token is valid. + * - CSRF token will expire eventually. + * - Invalidating all active CSRF tokens could be done by unset($_SESSION['CSRF_TOKEN_SEED']). + * It is recommended to reset CSRF tokens by login/logout event at least. + * It may be good idea to invalidate all of older CSRF tokens when idle time is long. + * + * Cons + * - There could be no CSRF expiration time. + * + * Precise CSRF token expiration is easy. Just add timestamp parameter + * as "info" and check it. + **************************************/ + +session_start(); +if (empty($_SESSION['CSRF_TOKEN_SEED'])) { + $_SESSION['CSRF_TOKEN_SEED'] = random_bytes(32); + $_SESSION['CSRF_TOKEN_COUNT'] = 1; + $_SESSION['CSRF_TOKEN_EXPIRE'] = time(); +} + + +function csrf_get_token($uri) { + // Check expiration + if ($_SESSION['CSRF_TOKEN_EXPIRE'] + CSRF_TOKEN_EXPIRE < time()) { + $_SESSION['CSRF_TOKEN_COUNT']++; + $_SESSION['CSRF_TOKEN_EXPIRE'] = time(); + } + // Equivalent(NOT exactly the same) value by using hash_hmac() + // return hash_hmac('sha3-256', hash_hmac('sha3-256', $_SESSION['CSRF_TOKEN_SEED'], $_SESSION['CSRF_TOKEN_COUNT']), $uri); + return hash_hkdf('sha3-256', $_SESSION['CSRF_TOKEN_SEED'], 0, $uri, $_SESSION['CSRF_TOKEN_COUNT']); +} + +function csrf_validate_token($csrf_token, $uri) { + for($i = 0; $i < CSRF_TOKENS; $i++) { + // Equivalent(NOT exactly the same) value by using hash_hmac() + // $token = hash_hmac('sha3-256', hash_hmac('sha3-256', $_SESSION['CSRF_TOKEN_SEED'], $_SESSION['CSRF_TOKEN_COUNT'] - $i), $uri); + $token = hash_hkdf('sha3-256', $_SESSION['CSRF_TOKEN_SEED'], 0, $uri, $_SESSION['CSRF_TOKEN_COUNT'] - $i); + if (hash_equals($csrf_token, $token)) { + return TRUE; + } + } + return FALSE; +} + + +//// Generating CSRF token //// +// $uri is target URI that browser POSTs form data +$uri = 'https://example.com/some_form/'; +$csrf_token = csrf_get_token($uri); +// embed $csrf_token to your form + +//// Validating CSRF token //// +$csrf_token = $_POST['csrf_token'] ?? ''; +if (!csrf_validate_token($csrf_token, $_SERVER['REQUEST_URI'])) { + // Invalid CSRF token + throw new Exception('CSRF token validation error'); +} +// valid request +?> +]]> + </programlisting> + <para> + Common CSRF token uses the same token value for a session and all + URI. This example CSRF token expires and is specific to a + URI. i.e. CSRF token http://example.com/form_A/ is not valid for + http://example.com/form_B/ Since token value is computed, no + database is required. + </para> + </example> + </para> + <para> + <example> <title><function>hash_hkdf</function> example</title> <programlisting role="php"> <![CDATA[ @@ -124,6 +256,30 @@ </para> </example> </para> + <para> + <example> + <title><function>hash_hkdf</function> bad example</title> + <para> + Users must not simply extend input key material length. HKDF does + not add additional entropy automatically. Therefore, weak key + remains weak unless strong salt is supplied. Following is bad + example. + </para> + <programlisting role="php"> +<![CDATA[ +<?php +$inputKey = get_my_aes128_key(); // AES 128 bit key + +// Derive AES 256 key from AES 128 key +$encryptionKey = hash_hkdf('sha256', $inputKey, 32, 'aes-256-encryption'); +// Users should not do this. $encryptionKey only has 128 bit +// entropy while it should have 256 bit entropy. +// To derive strong AES 256 key, strong enough salt is required. +?> +]]> + </programlisting> + </example> + </para> </refsect1> <refsect1 role="seealso"> @@ -130,6 +286,7 @@ &reftitle.seealso; <para> <simplelist> + <member><function>hash_hmac</function></member> <member><function>hash_pbkdf2</function></member> <member><link xlink:href="&url.rfc;5869">RFC 5869</link></member> <member><link xlink:href="&url.git.hub;narfbg/hash_hkdf_compat">userland implementation</link></member>
-- Yasuo Ohgaki yohgaki@ohgaki.net

Yasuo Ohgaki

9 years ago
Hi all, On Fri, Apr 14, 2017 at 6:22 AM, Yasuo Ohgaki <yohgaki@ohgaki.net> wrote:
> > On Thu, Apr 13, 2017 at 5:11 PM, Pieter Hordijk <info@pieterhordijk.com> > wrote: > >> Is this really something we need in our official docs instead of for >> example >> on a personal blog? >> > > I wrote draft doc patch. > Please verify. >
I used "very low entropy salt" for this CSRF token because "Input key is strong, very low entropy salt is acceptable". To avoid confusions, I revised the doc patch. Index: en/reference/hash/functions/hash-hkdf.xml =================================================================== --- en/reference/hash/functions/hash-hkdf.xml (リビジョン 342317) +++ en/reference/hash/functions/hash-hkdf.xml (作業コピー) @@ -3,7 +3,7 @@ <refentry xml:id="function.hash-hkdf" xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink"> <refnamediv> <refname>hash_hkdf</refname> - <refpurpose>Generate a HKDF key derivation of a supplied key input</refpurpose> + <refpurpose>Derive secure new key from existing key by using HKDF</refpurpose> </refnamediv> <refsect1 role="description"> &reftitle.description; @@ -16,6 +16,20 @@ <methodparam choice="opt"><type>string</type><parameter>salt</ parameter><initializer>''</initializer></methodparam> </methodsynopsis> + <para> + RFC 5869 defines HKDF (HMAC based Key Derivation Function) which + is general purpose KDF. HKDF could be useful for many PHP + applications that require temporary keys, such CSRF token, + pre-signed key for URI, password for password protected + URI, and so on. + </para> + <note> + <para> + When info and length + is not required for your program, more efficient + <function>hash_hmac</function> could be used instead. + </para> + </note> </refsect1> <refsect1 role="parameters"> &reftitle.parameters; @@ -25,7 +39,7 @@ <term><parameter>algo</parameter></term> <listitem> <para> - Name of selected hashing algorithm (i.e. "sha256", "sha512", "haval160,4", etc..) + Name of selected hashing algorithm (i.e. "sha3-256", "sha3-512", "sha256", "sha512", "haval160,4", etc..) See <function>hash_algos</function> for a list of supported algorithms. <note> <para> @@ -39,7 +53,7 @@ <term><parameter>ikm</parameter></term> <listitem> <para> - Input keying material (raw binary). Cannot be empty. + Input keying material. Cannot be empty. </para> </listitem> </varlistentry> @@ -60,7 +74,8 @@ <term><parameter>info</parameter></term> <listitem> <para> - Application/context-specific info string. + Application/context-specific info string. Info is intended for + public information such as user ID, protocol version, etc. </para> </listitem> </varlistentry> @@ -71,8 +86,34 @@ Salt to use during derivation. </para> <para> - While optional, adding random salt significantly improves the strength of HKDF. + While optional, adding random salt significantly improves the + strength of HKDF. Salt could be either secret or + non-secret. It is used as "Pre Shared Key" in many use cases. + Strong value is preferred. e.g. Use <function>random_bytes</ function>. + Optimal salt size is size of used hash algorithm. </para> + <warning> + <para> + Although salt is the last optional parameter, salt is the + most important parameter for key security. Omitted salt is + indication of inappropriate design in most cases. Users must + set appropriate salt value whenever it is possible. Omit salt + only when it cannot be used. + </para> + <para> + Strong salt is mandatory and must be kept secret when input + key is weak, otherwise input key security will not be kept. + When input key is strong, low entropy salt is acceptable. + However, providing strong salt is the best practice for the + best possible key security. Strong salt is strongly recommended + long life input keys. + </para> + <para> + Salt must not be able to be controlled by users. i.e. User + must not be able to set salt value and get derived key. User + controlled salt allows input key analysis to attackers. + </para> + </warning> </listitem> </varlistentry> </variablelist> @@ -101,6 +142,99 @@ &reftitle.examples; <para> <example> + <title>URI specific CSRF token that supports expiration by <function>hash_hkdf</function></title> + <programlisting role="php"> +<![CDATA[ +<?php +define('CSRF_TOKEN_EXPIRE', 180); // CSRF token expiration +define('CSRF_TOKENS', 5); // Last 5 CSRF tokens are valid + +/************************************** + * Implementation note + * + * It uses "counter" for CSRF expiration management. + * "counter" is very low entropy, but input key is strong and + * CSRF_TOKEN_SEED is short term key. It should be OK. + * + * This CSRF token implementation has pros and cons + * + * Pros + * - A CSRF token is valid only for specific URI. + * - No database is required for URI specific CSRF tokens. + * - Only CSRF token is required. i.e. No timestamp parameter. + * - When user is active, a CSRF token is valid upto CSRF_TOKEN_EXPIRE * CSRF_TOKENS sec. + * - Even when user had long idle time, CSRF token is valid. + * - CSRF token will expire eventually. + * - Invalidating all active CSRF tokens could be done by unset($_SESSION['CSRF_TOKEN_SEED']). + * It is recommended to reset CSRF tokens by login/logout event at least. + * It may be good idea to invalidate all of older CSRF tokens when idle time is long. + * + * Cons + * - There could be no CSRF expiration time. + * + * Precise CSRF token expiration is easy. Just add timestamp parameter + * as "info" and check it. + **************************************/ + +session_start(); +if (empty($_SESSION['CSRF_TOKEN_SEED'])) { + $_SESSION['CSRF_TOKEN_SEED'] = random_bytes(32); + $_SESSION['CSRF_TOKEN_COUNT'] = 1; + $_SESSION['CSRF_TOKEN_EXPIRE'] = time(); +} + + +function csrf_get_token($uri) { + // Check expiration + if ($_SESSION['CSRF_TOKEN_EXPIRE'] + CSRF_TOKEN_EXPIRE < time()) { + $_SESSION['CSRF_TOKEN_COUNT']++; + $_SESSION['CSRF_TOKEN_EXPIRE'] = time(); + } + // Equivalent(NOT exactly the same) value by using hash_hmac() + // return hash_hmac('sha3-256', hash_hmac('sha3-256', $_SESSION['CSRF_TOKEN_SEED'], $_SESSION['CSRF_TOKEN_COUNT']), $uri); + return hash_hkdf('sha3-256', $_SESSION['CSRF_TOKEN_SEED'], 0, $uri, $_SESSION['CSRF_TOKEN_COUNT']); +} + +function csrf_validate_token($csrf_token, $uri) { + for($i = 0; $i < CSRF_TOKENS; $i++) { + // Equivalent(NOT exactly the same) value by using hash_hmac() + // $token = hash_hmac('sha3-256', hash_hmac('sha3-256', $_SESSION['CSRF_TOKEN_SEED'], $_SESSION['CSRF_TOKEN_COUNT'] - $i), $uri); + $token = hash_hkdf('sha3-256', $_SESSION['CSRF_TOKEN_SEED'], 0, $uri, $_SESSION['CSRF_TOKEN_COUNT'] - $i); + if (hash_equals($csrf_token, $token)) { + return TRUE; + } + } + return FALSE; +} + + +//// Generating CSRF token //// +// $uri is target URI that browser POSTs form data +$uri = 'https://example.com/some_form/'; +$csrf_token = csrf_get_token($uri); +// embed $csrf_token to your form + +//// Validating CSRF token //// +$csrf_token = $_POST['csrf_token'] ?? ''; +if (!csrf_validate_token($csrf_token, $_SERVER['REQUEST_URI'])) { + // Invalid CSRF token + throw new Exception('CSRF token validation error'); +} +// valid request +?> +]]> + </programlisting> + <para> + Common CSRF token uses the same token value for a session and all + URI. This example CSRF token expires and is specific to a + URI. i.e. CSRF token http://example.com/form_A/ is not valid for + http://example.com/form_B/ Since token value is computed, no + database is required. + </para> + </example> + </para> + <para> + <example> <title><function>hash_hkdf</function> example</title> <programlisting role="php"> <![CDATA[ @@ -124,6 +258,30 @@ </para> </example> </para> + <para> + <example> + <title><function>hash_hkdf</function> bad example</title> + <para> + Users must not simply extend input key material length. HKDF does + not add additional entropy automatically. Therefore, weak key + remains weak unless strong salt is supplied. Following is bad + example. + </para> + <programlisting role="php"> +<![CDATA[ +<?php +$inputKey = get_my_aes128_key(); // AES 128 bit key + +// Derive AES 256 key from AES 128 key +$encryptionKey = hash_hkdf('sha256', $inputKey, 32, 'aes-256-encryption'); +// Users should not do this. $encryptionKey only has 128 bit +// entropy while it should have 256 bit entropy. +// To derive strong AES 256 key, strong enough salt is required. +?> +]]> + </programlisting> + </example> + </para> </refsect1> <refsect1 role="seealso"> @@ -130,6 +288,7 @@ &reftitle.seealso; <para> <simplelist> + <member><function>hash_hmac</function></member> <member><function>hash_pbkdf2</function></member> <member><link xlink:href="&url.rfc;5869">RFC 5869</link></member> <member><link xlink:href="&url.git.hub;narfbg/hash_hkdf_compat">userland implementation</link></member>
-- Yasuo Ohgaki yohgaki@ohgaki.net

Nikita Popov

9 years ago
On Thu, Apr 13, 2017 at 11:22 PM, Yasuo Ohgaki <yohgaki@ohgaki.net> wrote:
> Hi Pieter and all, > > On Thu, Apr 13, 2017 at 5:11 PM, Pieter Hordijk <info@pieterhordijk.com> > wrote: > > > Is this really something we need in our official docs instead of for > > example > > on a personal blog? > > > > I wrote draft doc patch. > Please verify. > > Index: en/reference/hash/functions/hash-hkdf.xml > =================================================================== > --- en/reference/hash/functions/hash-hkdf.xml (リビジョン 342317) > +++ en/reference/hash/functions/hash-hkdf.xml (作業コピー) > @@ -3,7 +3,7 @@ > <refentry xml:id="function.hash-hkdf" xmlns="http://docbook.org/ns/ > docbook" > xmlns:xlink="http://www.w3.org/1999/xlink"> > <refnamediv> > <refname>hash_hkdf</refname> > - <refpurpose>Generate a HKDF key derivation of a supplied key > input</refpurpose> > + <refpurpose>Derive secure new key from existing key by using > HKDF</refpurpose> > </refnamediv> > <refsect1 role="description"> > &reftitle.description; > @@ -16,6 +16,20 @@ > <methodparam > choice="opt"><type>string</type><parameter>salt</ > parameter><initializer>''</initializer></methodparam> > </methodsynopsis> > > + <para> > + RFC 5869 defines HKDF (HMAC based Key Derivation Function) which > + is general purpose KDF. HKDF could be useful for many PHP > + applications that require temporary keys, such CSRF token, > + pre-signed key for URI, password for password protected > + URI, and so on. > + </para> > + <note> > + <para> > + When info and length > + is not required for your program, more efficient > + <function>hash_hmac</function> could be used instead. > + </para> > + </note> > </refsect1> > <refsect1 role="parameters"> > &reftitle.parameters; > @@ -25,7 +39,7 @@ > <term><parameter>algo</parameter></term> > <listitem> > <para> > - Name of selected hashing algorithm (i.e. "sha256", "sha512", > "haval160,4", etc..) > + Name of selected hashing algorithm (i.e. "sha3-256", "sha3-512", > "sha256", "sha512", "haval160,4", etc..) > See <function>hash_algos</function> for a list of supported > algorithms. > <note> > <para> > @@ -39,7 +53,7 @@ > <term><parameter>ikm</parameter></term> > <listitem> > <para> > - Input keying material (raw binary). Cannot be empty. > + Input keying material. Cannot be empty. > </para> > </listitem> > </varlistentry> > @@ -60,7 +74,8 @@ > <term><parameter>info</parameter></term> > <listitem> > <para> > - Application/context-specific info string. > + Application/context-specific info string. Info is intended for > + public information such as user ID, protocol version, etc. > </para> > </listitem> > </varlistentry> > @@ -71,8 +86,32 @@ > Salt to use during derivation. > </para> > <para> > - While optional, adding random salt significantly improves the > strength of HKDF. > + While optional, adding random salt significantly improves the > + strength of HKDF. Salt could be either secret or > + non-secret. It is used as "Pre Shared Key" in many use cases. > + Strong value is preferred. e.g. Use > <function>random_bytes</function>. > + Optimal salt size is size of used hash algorithm. > </para> > + <warning> > + <para> > + Although salt is the last optional parameter, salt is the > + most important parameter for key security. Omitted salt is > + indication of inappropriate design in most cases. Users must > + set appropriate salt value whenever it is possible. Omit salt > + only when it cannot be used. > + </para> > + <para> > + Strong salt is mandatory and must be kept secret when input > + key is weak, otherwise input key security will not be kept. > + Even when input key is strong, providing strong salt is the > + best practice for the best possible key security. > + </para> > + <para> > + Salt must not be able to be controlled by users. i.e. User > + must not be able to set salt value and get derived key. User > + controlled salt allows input key analysis to attackers. > + </para> > + </warning> > </listitem> > </varlistentry> > </variablelist> > @@ -101,6 +140,99 @@ > &reftitle.examples; > <para> > <example> > + <title>URI specific CSRF token that supports expiration by > <function>hash_hkdf</function></title> > + <programlisting role="php"> > +<![CDATA[ > +<?php > +define('CSRF_TOKEN_EXPIRE', 180); // CSRF token expiration > +define('CSRF_TOKENS', 5); // Last 5 CSRF tokens are valid > + > +/************************************** > + * Implementation note > + * > + * It uses "counter" for CSRF expiration management. > + * "counter" is very low entropy, but input key is strong and > + * CSRF_TOKEN_SEED is short term key. It should be OK. > + * > + * This CSRF token implementation has pros and cons > + * > + * Pros > + * - A CSRF token is valid only for specific URI. > + * - No database is required for URI specific CSRF tokens. > + * - Only CSRF token is required. i.e. No timestamp parameter. > + * - When user is active, a CSRF token is valid upto CSRF_TOKEN_EXPIRE * > CSRF_TOKENS sec. > + * - Even when user had long idle time, CSRF token is valid. > + * - CSRF token will expire eventually. > + * - Invalidating all active CSRF tokens could be done by > unset($_SESSION['CSRF_TOKEN_SEED']). > + * It is recommended to reset CSRF tokens by login/logout event at > least. > + * It may be good idea to invalidate all of older CSRF tokens when idle > time is long. > + * > + * Cons > + * - There could be no CSRF expiration time. > + * > + * Precise CSRF token expiration is easy. Just add timestamp parameter > + * as "info" and check it. > + **************************************/ > + > +session_start(); > +if (empty($_SESSION['CSRF_TOKEN_SEED'])) { > + $_SESSION['CSRF_TOKEN_SEED'] = random_bytes(32); > + $_SESSION['CSRF_TOKEN_COUNT'] = 1; > + $_SESSION['CSRF_TOKEN_EXPIRE'] = time(); > +} > + > + > +function csrf_get_token($uri) { > + // Check expiration > + if ($_SESSION['CSRF_TOKEN_EXPIRE'] + CSRF_TOKEN_EXPIRE < time()) { > + $_SESSION['CSRF_TOKEN_COUNT']++; > + $_SESSION['CSRF_TOKEN_EXPIRE'] = time(); > + } > + // Equivalent(NOT exactly the same) value by using hash_hmac() > + // return hash_hmac('sha3-256', hash_hmac('sha3-256', > $_SESSION['CSRF_TOKEN_SEED'], $_SESSION['CSRF_TOKEN_COUNT']), $uri); > + return hash_hkdf('sha3-256', $_SESSION['CSRF_TOKEN_SEED'], 0, $uri, > $_SESSION['CSRF_TOKEN_COUNT']); > +} > + > +function csrf_validate_token($csrf_token, $uri) { > + for($i = 0; $i < CSRF_TOKENS; $i++) { > + // Equivalent(NOT exactly the same) value by using hash_hmac() > + // $token = hash_hmac('sha3-256', hash_hmac('sha3-256', > $_SESSION['CSRF_TOKEN_SEED'], $_SESSION['CSRF_TOKEN_COUNT'] - $i), $uri); > + $token = hash_hkdf('sha3-256', $_SESSION['CSRF_TOKEN_SEED'], 0, > $uri, $_SESSION['CSRF_TOKEN_COUNT'] - $i); > + if (hash_equals($csrf_token, $token)) { > + return TRUE; > + } > + } > + return FALSE; > +} > + > + > +//// Generating CSRF token //// > +// $uri is target URI that browser POSTs form data > +$uri = 'https://example.com/some_form/'; > +$csrf_token = csrf_get_token($uri); > +// embed $csrf_token to your form > + > +//// Validating CSRF token //// > +$csrf_token = $_POST['csrf_token'] ?? ''; > +if (!csrf_validate_token($csrf_token, $_SERVER['REQUEST_URI'])) { > + // Invalid CSRF token > + throw new Exception('CSRF token validation error'); > +} > +// valid request > +?> > +]]> > + </programlisting> > + <para> > + Common CSRF token uses the same token value for a session and all > + URI. This example CSRF token expires and is specific to a > + URI. i.e. CSRF token http://example.com/form_A/ is not valid for > + http://example.com/form_B/ Since token value is computed, no > + database is required. > + </para> > + </example> > + </para> > + <para> > + <example> > <title><function>hash_hkdf</function> example</title> > <programlisting role="php"> > <![CDATA[ > @@ -124,6 +256,30 @@ > </para> > </example> > </para> > + <para> > + <example> > + <title><function>hash_hkdf</function> bad example</title> > + <para> > + Users must not simply extend input key material length. HKDF does > + not add additional entropy automatically. Therefore, weak key > + remains weak unless strong salt is supplied. Following is bad > + example. > + </para> > + <programlisting role="php"> > +<![CDATA[ > +<?php > +$inputKey = get_my_aes128_key(); // AES 128 bit key > + > +// Derive AES 256 key from AES 128 key > +$encryptionKey = hash_hkdf('sha256', $inputKey, 32, 'aes-256-encryption'); > +// Users should not do this. $encryptionKey only has 128 bit > +// entropy while it should have 256 bit entropy. > +// To derive strong AES 256 key, strong enough salt is required. > +?> > +]]> > + </programlisting> > + </example> > + </para> > </refsect1> > > <refsect1 role="seealso"> > @@ -130,6 +286,7 @@ > &reftitle.seealso; > <para> > <simplelist> > + <member><function>hash_hmac</function></member> > <member><function>hash_pbkdf2</function></member> > <member><link xlink:href="&url.rfc;5869">RFC 5869</link></member> > <member><link > xlink:href="&url.git.hub;narfbg/hash_hkdf_compat">userland > implementation</link></member> >
Strong -1 on these docs changes. They are wrong and they will confuse users about when and how HKDF should be used. I have no idea where you got the idea that HKDF is supposed to be used for CSRF token generation, but it isn't. I did not check whether your code is correct and secure, but CSRF token generation is certainly not a common or typical application of HKDF and as such should not be present in the documentation. Your "bad example" is actually pretty much the textbook use-case for HKDF. The way you wrote it (get a AES-256 key from an AES-128 key) doesn't make much sense, but the general principle of extracting two keys (for encryption and authentication) from a single key is one of *the* use-cases of HKDF. It is also, contrary to your statement in the documentation snippet, perfectly cryptographically sound. A salt is not required for this case. A salt *may* be beneficial, but for entirely different reasons (as Scott pointed out, for many block cipher modes fixed encryption keys only have a lifetime of around 2^64 encryptions, past which point IV collisions are to be expected -- a salt in key derivation could mitigate this.) Nikita