Everything

php

Reseeding rand()/mt_rand()

php.internals

Yasuo Ohgaki

9 years ago

Hi all,

Following code is problematic and needs proper reseeding to work.

// We need the same random numbers here
mt_srand(1234);
for ($i=0; $i < 10; $i++) {
$my_rand[] = mt_rand();
}

Somewhere later in code

// We need somewhat random numbers for non CS purpose
for ($i=0; $i < 10; $i++) {
$my_other_rand[] = mt_rand();
}

$my_other_rand array is not random at all for the app. This applies to
rand()/srand() as well as all functions that use rand and rand/MT rand
internally. e.g. shuffle(). rand() is alias of mt_rand() in PHP 7.1, it can
cause serious problem. i.e. srand(1234) forces mt_rand() to generate non
random numbers.

In addition, this behavior persists across requests because once
BG(mt_rand_is_seeded)/BG(rand_is_seeded) are set to 1, they are kept for
the process. Therefore, subsequent mt_rand()/rand(), including any
functions use rand/MT rand internally such as shuffle(), call in other
requests are not random.

In order to get random numbers, we need to reseed RNG.

Currently, it is possible to reseed like (int overflow is ignored)

$seed = unpack("l", file_get_contents("/dev/urandom", false, NULL, 0, 4));
mt_srand($seed[1]);
for ($i=0; $i < 10; $i++) {
$my_other_rand[] = mt_rand();
}

OR for PHP 7

mt_srand(random_int(PHP_INT_MIN, PHP_INT_MAX));
for ($i=0; $i < 10; $i++) {
$my_other_rand[] = mt_rand();
}

Requiring these reseeding code for most mt_rand()/rand() calls is not
preferred behavior.

Proposal:
1) Add BG(mt_rand_is_user_seeded) and BG(rand_is_user_seeded). If they are
1, set BG(mt_rand_is_seeded)=0 and BG(rand_is_seeded)=0.
2) Make srand(0) and mt_srand(0) to force RNG reseeding by PHP.

Outcome:
1) resolves "across requests" non random numbers.
2) simplifies reseeding.

Problem:
1) Added new BG values are BC for released versions. Simply reseeding by
current GENERATE_SEED() macro is weak and too easy to be guessed even with
MT rand. i.e. Setting BG(mt_rand_is_seeded)=0 and BG(rand_is_seeded)=0 at
RINIT is not exactly a good idea. (There is improvement discussion in
"Improving mt_rand() seed" thread)
2) Manual reseeding API, srand(0)/mt_srand(0), is not compatible with older
versions.

Open Issue:
1) and 2), apply these to released versions or not.

This idea is acceptable, but I don't like this idea myself.

It seems we should do something for this, documentation for released
versions at least.
Any better ideas are appreciated.

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net

Christoph Becker

9 years ago

On 30.01.2017 at 13:32, Yasuo Ohgaki wrote:

> Hi all,
>
> Following code is problematic and needs proper reseeding to work.
>
> // We need the same random numbers here
> mt_srand(1234);
> for ($i=0; $i < 10; $i++) {
>    $my_rand[] = mt_rand();
> }
>
> Somewhere later in code
>
> // We need somewhat random numbers for non CS purpose
> for ($i=0; $i < 10; $i++) {
>    $my_other_rand[] = mt_rand();
> }
>
> $my_other_rand array is not random at all for the app. This applies to
> rand()/srand() as well as all functions that use rand and rand/MT rand
> internally. e.g. shuffle(). rand() is alias of mt_rand() in PHP 7.1, it can
> cause serious problem. i.e. srand(1234) forces mt_rand() to generate non
> random numbers.
>
> In addition, this behavior persists across requests because once
> BG(mt_rand_is_seeded)/BG(rand_is_seeded) are set to 1, they are kept for
> the process. Therefore, subsequent mt_rand()/rand(), including any
> functions use rand/MT rand internally such as shuffle(), call in other
> requests are not random.
>
> In order to get random numbers, we need to reseed RNG.
>
> Currently, it is possible to reseed like (int overflow is ignored)
>
> $seed = unpack("l", file_get_contents("/dev/urandom", false, NULL, 0, 4));
> mt_srand($seed[1]);
> for ($i=0; $i < 10; $i++) {
>    $my_other_rand[] = mt_rand();
> }
>
> OR for PHP 7
>
> mt_srand(random_int(PHP_INT_MIN, PHP_INT_MAX));
> for ($i=0; $i < 10; $i++) {
>    $my_other_rand[] = mt_rand();
> }
>
> Requiring these reseeding code for most mt_rand()/rand() calls is not
> preferred behavior.
>
> Proposal:
> 1) Add BG(mt_rand_is_user_seeded) and BG(rand_is_user_seeded). If they are
> 1, set BG(mt_rand_is_seeded)=0 and BG(rand_is_seeded)=0.
> 2) Make srand(0) and mt_srand(0) to force RNG reseeding by PHP.
>
> Outcome:
> 1) resolves "across requests" non random numbers.
> 2) simplifies reseeding.
>
> Problem:
> 1) Added new BG values are BC for released versions. Simply reseeding by
> current GENERATE_SEED() macro is weak and too easy to be guessed even with
> MT rand. i.e. Setting BG(mt_rand_is_seeded)=0 and BG(rand_is_seeded)=0 at
> RINIT is not exactly a good idea. (There is improvement discussion in
> "Improving mt_rand() seed" thread)
> 2) Manual reseeding API, srand(0)/mt_srand(0), is not compatible with older
> versions.
>
> Open Issue:
> 1) and 2), apply these to released versions or not.
>
> This idea is acceptable, but I don't like this idea myself.
>
> It seems we should do something for this, documentation for released
> versions at least.
> Any better ideas are appreciated.

Just a quick idea:

<?php

class PNRG {
public function __construct($seed = null) {…}
public function get() {…}
}

--
Christoph M. Becker

Andrew Faulds

9 years ago

Hi Christoph,

Christoph M. Becker wrote:

> Just a quick idea:
>
> <?php
>
> class PNRG {
>     public function __construct($seed = null) {…}
>     public function get() {…}
> }
>

I've long favoured an API along these lines. One size does not fit all,
you need two APIs for (non-crytographic) random numbers:
- automatically-seeded, non-reproducible numbers (a global function like
rand())
- manually-seeded, reproducible number sequences (objects)

Currently we mix these two, and it's a mess. The easy solution would be
a random number-generating object like you propose, to cover the second
use-case. We could also introduce a non-manually-seedable global PRNG
also, if desired. Perhaps as a static method of that class.

One thought: it'd be nice if you had two different methods, one that
mutates in places, and one that doesn't, i.e.:

$prng1 = new PRNG(SEED);
[$randomNumber, $prng2] = $prng2->getWithNewPRNG();

v.s.:

$prng = new PRNG(SEED);
$randomNumber = $prng->get();
$randomNumber2 = $prng->get();

Thanks!

--
Andrea Faulds
https://ajf.me/

Yasuo Ohgaki

9 years ago

Hi Andrea,

On Tue, Jan 31, 2017 at 1:29 AM, Andrea Faulds <ajf@ajf.me> wrote:

>
> Christoph M. Becker wrote:
>
> Just a quick idea:
>>
>> <?php
>>
>> class PNRG {
>>     public function __construct($seed = null) {…}
>>     public function get() {…}
>> }
>>
>>
> I've long favoured an API along these lines. One size does not fit all,
> you need two APIs for (non-crytographic) random numbers:
> - automatically-seeded, non-reproducible numbers (a global function like
> rand())
> - manually-seeded, reproducible number sequences (objects)
>

I agree. We should have PRNG object.

>
> Currently we mix these two, and it's a mess. The easy solution would be a
> random number-generating object like you propose, to cover the second
> use-case. We could also introduce a non-manually-seedable global PRNG also,
> if desired. Perhaps as a static method of that class.
>

Mixing system and user seeded PRNG is mess. I totally agree.
Let's divide them into separated states.

>
> One thought: it'd be nice if you had two different methods, one that
> mutates in places, and one that doesn't, i.e.:
>
> $prng1 = new PRNG(SEED);
> [$randomNumber, $prng2] = $prng2->getWithNewPRNG();
>
> v.s.:
>
> $prng = new PRNG(SEED);
> $randomNumber = $prng->get();
> $randomNumber2 = $prng->get();

My current objective is to make existing API to work, so resource may be
used
to set/get PRNG state.

*** Initialize and Create new PRNG state resource ***
resource mt_srand([int|string $seed])

resource: MT rand state resource.
$seed: I would like to allow large seed like Python and Ruby.
When seed is string, use all bits for MT rand state
initialization.
Example usage: mt_srand(random_bytes(2000));

*** Get random number from state resource ***
int mt_rand() // System state
int mt_rand(resource $state) // Specified user state
int mt_rand(int $min , int $max [,resource $state]) // Specified user state
if $state is passed



I suppose most codes do not use mt_srand()/srand(). With new API, users may
write
code like

===============
// We need the same random numbers here
$my_state = mt_srand(1234);
// NOTE: mt_srand(), w/o seed, creates new PRNG state.
for ($i=0; $i < 10; $i++) {
// Use my PRNG state
$my_rand[] = mt_rand($my_state);
}

Somewhere later in code

// We need somewhat random numbers for non CS purpose
for ($i=0; $i < 10; $i++) {
// System PRNG state is affected by mt_srand()/srand()
// and $my_other_rand will have proper random values
$my_other_rand[] = mt_rand();
}
===============

This requires code modifications, but it isn't too bad.
Problem is this change would be master only and
other functions, e.g. shuffle(), have optional state resource
parameter or not.

Thoughts? Comments?

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net

Andrew Faulds

9 years ago

Hi,

Yasuo Ohgaki wrote:

> My current objective is to make existing API to work, so resource may be
> used
> to set/get PRNG state.

I would prefer it if we not introduce a new usage of resources. An
object would suffice.

> *** Initialize and Create new PRNG state resource ***
> resource mt_srand([int|string $seed])
>
> resource: MT rand state resource.
> $seed: I would like to allow large seed like Python and Ruby.
>             When seed is string, use all bits for MT rand state
> initialization.
>             Example usage: mt_srand(random_bytes(2000));
>
> *** Get random number from state resource ***
> int mt_rand() // System state
> int mt_rand(resource $state) // Specified user state
> int mt_rand(int $min , int $max [,resource $state]) // Specified user state
> if $state is passed

While we could retrofit this into the existing API, it would be cleaner
and friendlier to provide a new one based on objects.

--
Andrea Faulds
https://ajf.me/

Yasuo Ohgaki

9 years ago

Hi all,

On Wed, Feb 1, 2017 at 5:48 AM, Andrea Faulds <ajf@ajf.me> wrote:

> Yasuo Ohgaki wrote:
>
>> My current objective is to make existing API to work, so resource may be
>> used
>> to set/get PRNG state.
>>
>
> I would prefer it if we not introduce a new usage of resources. An object
> would suffice.
>
> *** Initialize and Create new PRNG state resource ***
>> resource mt_srand([int|string $seed])
>>
>> resource: MT rand state resource.
>> $seed: I would like to allow large seed like Python and Ruby.
>>             When seed is string, use all bits for MT rand state
>> initialization.
>>             Example usage: mt_srand(random_bytes(2000));
>>
>> *** Get random number from state resource ***
>> int mt_rand() // System state
>> int mt_rand(resource $state) // Specified user state
>> int mt_rand(int $min , int $max [,resource $state]) // Specified user
>> state
>> if $state is passed
>>
>
> While we could retrofit this into the existing API, it would be cleaner
> and friendlier to provide a new one based on objects.

Object based implementation is nicer. I agree.

I like multi paradigm programming also. We should provide API that users
can adopt with
minimal cost. IMHO.  Therefore, I would like to provide working PRNG for
both procedural
and object based. I wrote new RFC that fixes issues discussed here.

https://wiki.php.net/rfc/improve_predictable_prng_random

I used RandomState object so that OO based Random API can use it seamlessly
in the future.

Comments are appreciated!

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net

Christoph Becker

9 years ago

On 30.01.2017 at 17:29, Andrea Faulds wrote:

> Christoph M. Becker wrote:
>
>> Just a quick idea:
>>
>> <?php
>>
>> class PNRG {
>>     public function __construct($seed = null) {…}
>>     public function get() {…}
>> }
>
> I've long favoured an API along these lines. One size does not fit all,
> you need two APIs for (non-crytographic) random numbers:
> - automatically-seeded, non-reproducible numbers (a global function like
> rand())
> - manually-seeded, reproducible number sequences (objects)
>
> Currently we mix these two, and it's a mess. The easy solution would be
> a random number-generating object like you propose, to cover the second
> use-case. We could also introduce a non-manually-seedable global PRNG
> also, if desired. Perhaps as a static method of that class.
>
> One thought: it'd be nice if you had two different methods, one that
> mutates in places, and one that doesn't, i.e.:
>
> $prng1 = new PRNG(SEED);
> [$randomNumber, $prng2] = $prng2->getWithNewPRNG();
>
> v.s.:
>
> $prng = new PRNG(SEED);
> $randomNumber = $prng->get();
> $randomNumber2 = $prng->get();

Thanks, Andrea!  I like all your suggestions, and I'd be happy to see a
respective RFC – unfortunately, I won't have time to do this myself in
the near future.

--
Christoph M. Becker

Yasuo Ohgaki

9 years ago

Hi Christoph,

On Mon, Jan 30, 2017 at 10:20 PM, Christoph M. Becker <cmbecker69@gmx.de>
wrote:

> Just a quick idea:
>
> <?php
>
> class PNRG {
>     public function __construct($seed = null) {…}
>     public function get() {…}
> }
>

Object based implementation is a lot cleaner. I'm willing to write one
but not now... I would like to make current API work better first.

If anyone could write OO based predictable PRNG, I appreciate a lot!

Regards,

--
Yasuo Ohgaki
yohgaki@ohgaki.net