Everything

php

RFC proposal: fsync support for file resources

php.internals

David Gebler

6 years ago

Exactly as the subject says, I would like to propose an RFC for adding an
fsync() function for file resources, which would in essence be a thin
wrapper around C's fsync on UNIX systems and _commit on Windows.

It seems to me an odd oversight that this has never been implemented in PHP
and means PHP has no way to perform durable file write operations, making
it inherently unsuitable for any systems requiring more intensive I/O,
mission critical logs, auditing, etc.

I am not really a C programmer and I have been able to implement a simple
working prototype of this as a compiled extension in merely a few hours, so
I'm sure it wouldn't be difficult to bring in to the language core where
the functionality really belongs.

Every other major programming language otherwise comparable to PHP in
features supports a way of providing durability.

Thanks.

Dennis Birkholz

6 years ago

Hello David,

isn't fflush exactly this:
https://www.php.net/manual/en/function.fflush.php

Greets
Dennis

Am 01.06.20 um 18:56 schrieb David Gebler:

David Gebler

6 years ago

No, fflush will flush the PHP application buffers out to the operating
system, it does in contrast to fsync not provide a guarantee the operating
system will immediately persist to the underlying storage device.

On Mon, 1 Jun 2020, 18:49 Dennis Birkholz, <php@dennis.birkholz.biz> wrote:

David Gebler

6 years ago

Just to add and clarify; operating systems have their own internal write
buffers. When you call fflush() you are saying "immediately flush this data
from my program buffers to the operating system".

When you call fsync() you are saying "immediately flush this data from my
program buffers to the operating system *and tell the operating system to
immediately flush its buffer to disk too*", so when this function returns
success status you can be as reliably sure as possible that data has been
physically persisted. fflush does not provide this assurance.

On Mon, 1 Jun 2020, 18:52 David Gebler, <davidgebler@gmail.com> wrote:

Christian Schneider

6 years ago

Am 01.06.2020 um 18:56 schrieb David Gebler <davidgebler@gmail.com>:

> It seems to me an odd oversight that this has never been implemented in PHP
> and means PHP has no way to perform durable file write operations, making
> it inherently unsuitable for any systems requiring more intensive I/O,
> mission critical logs, auditing, etc.

I think there is a misconception of the usefulness of fsync() here.

First of all it does not *really* guarantee the data is safe from power outages etc, see
https://stackoverflow.com/a/706688
https://stackoverflow.com/a/725742
and other answers in that thread.

Secondly I'd rather rely on something like a proper DB if not losing any data is crucial.
Trying to implement it in pure PHP and plain old files seems like using the wrong tools for the job to me.

While there is little reason not to add fsync() to PHP the importance and benefits are IMHO not that big.

- Chris

David Gebler

6 years ago

fsync() does not *guarantee* data has been persisted because no software
mechanism which exists can do that. It is however the *best assurance* we
can have that data has been physically persisted by the time the function
returns.

fsync() is the best operating-system level assurance we can get regarding
data persistence and PHP is the only major programming language I can think
of which doesn't support a means of doing it. I've had use cases in web
systems and microservices written in PHP where the ability to provide
durable file writes would have been very valuable, other cases where
services have had to be written in Java or some other language specifically
because they require durability where PHP might otherwise have been a good
candidate.

> While there is little reason not to add fsync() to PHP the importance and

benefits are IMHO not that big.

Arguably so, but neither that big is the effort required to implement this
feature.

On Mon, Jun 1, 2020 at 8:47 PM Christian Schneider <cschneid@cschneid.com>
wrote:

Nikita Popov

6 years ago

On Mon, Jun 1, 2020 at 6:57 PM David Gebler <davidgebler@gmail.com> wrote:

> Exactly as the subject says, I would like to propose an RFC for adding an
> fsync() function for file resources, which would in essence be a thin
> wrapper around C's fsync on UNIX systems and _commit on Windows.
>
> It seems to me an odd oversight that this has never been implemented in PHP
> and means PHP has no way to perform durable file write operations, making
> it inherently unsuitable for any systems requiring more intensive I/O,
> mission critical logs, auditing, etc.
>
> I am not really a C programmer and I have been able to implement a simple
> working prototype of this as a compiled extension in merely a few hours, so
> I'm sure it wouldn't be difficult to bring in to the language core where
> the functionality really belongs.
>
> Every other major programming language otherwise comparable to PHP in
> features supports a way of providing durability.
>
> Thanks.
>

No objections from my side. I assume you'd want to add both fsync() and
fdatasync()?

I think all it takes for this one is a PR...

Regards,
Nikita

David Gebler

6 years ago

It's interesting that you mention fdatasync - perhaps a better C programmer
than I am can correct me, but I don't think it has an equivalent on
Windows, where _commit() is a wrap on FlushFileBuffers API. So in respect
of PHP implementation, the options would be:

fdatasync() is not available on Windows, but fsync() is. fdatasync() is
available on UNIX builds.

fdatasync() on Windows is an alias of fsync(), on UNIX fdatasync() is as
expected.

Only fsync() is implemented for both Windows and Unix.

-Dave

On Wed, 3 Jun 2020, 11:16 Nikita Popov, <nikita.ppv@gmail.com> wrote:

David Gebler

6 years ago

As a side-note, I am unable to register a wiki.php.net account to raise
this RFC, I get a very unhelpful error message which just says "That wasn't
the answer we were expecting" when I submit my details. Anyone have any
idea what that is?

On Thu, Jun 4, 2020 at 3:19 PM David Gebler <davidgebler@gmail.com> wrote:

Christoph Becker

6 years ago

On 06.06.2020 at 00:00, David Gebler wrote:

> As a side-note, I am unable to register a wiki.php.net account to raise
> this RFC, I get a very unhelpful error message which just says "That wasn't
> the answer we were expecting" when I submit my details. Anyone have any
> idea what that is?

I assume you're stumbling upon the CAPTCHA (fourth input field).  The
point is, after registration you are supposed to send a mail to this
mailing list; and that email address is asked for. :)

--
Christoph M. Becker

David Gebler

6 years ago

Thanks, my browser was automatically populating the wrong email, I've now
registered. Apparently I need to request karma now? Username dwgebler.

On Sat, Jun 6, 2020 at 12:19 AM Christoph M. Becker <cmbecker69@gmx.de>
wrote:

Christoph Becker

6 years ago

On 06.06.2020 at 01:39, David Gebler wrote:

> Thanks, my browser was automatically populating the wrong email, I've now
> registered. Apparently I need to request karma now? Username dwgebler.

RFC karma granted.

I'm not sure whether this feature needs an RFC, though.  Maybe just
start with a pull request, and see if that turns out to be controversial.

Thanks,
Christoph