Everything
php
Learn Community Releases The PHP Project
Home » Project » Newsgroups » php.internals » [Discussion] FFI in PHP »

[Discussion] FFI in PHP

php.internals

Alex Bowers

9 years ago
Hello All, I'd like to start a discussion around an FFI RFC FFI RFC ====== There are many languages that support an FFI implementation. NodeJS Python C++ Ruby FFI allows you to call a native C function without requiring the boilerplate of an extension to be written. There are several benefits to having FFI - Performance - Shareability / Bundling - Common functionality between languages Performance === Although less pronounced than the 5.x versions, there is still a performance benefit to having native C code over PHP code. For example, you could utilise threading inside of your FFI methods, which PHP does not expose the ability to do. Shareability === If you wish to implement some of your source code in C, the current way to share it is to build it as an extension. This is cumbersome, and restricts use-cases such as shared hosting, where the ability to install your own extensions is probably restricted. However, with FFI, the shared object can be loaded from any location, and so that restriction is no longer in place. They could even be distributed via composer. Common functionality between languages === If you have some complex logic that needs to be replicated in several languages for whatever reason; implementing it several times over would lead to uncertain bugs and technical debt increasing heavily. If you could share the same logic amongst them all using FFI, then this is no longer an issue. Example === Take an example of a rust program that takes two numbers in and gives you the sum of them. ```rust #[no_mangle] pub extern fn add(a: i32, b: i32) -> i32 { a + b } ``` with the Cargo.toml file containing: ``` [package] name = "math" version = "0.1.0" authors = ["Alex Bowers <bowersbros@gmail.com>"] [dependencies] [lib] name = "math" crate-type = ["dylib"] ``` `cargo build --release` will create `.so`, `.dylib`, or `.dll` files depending on your system. These should be usable within PHP using the exposed functions. ```php $math = ffi("/path/to/math.so"); $result = $math->add(1, 5); echo $result; // 6 ``` With the implementation at its most basic level, calling the `add` method with incorrect parameters would likely cause a segfault. A way around that could be that the methods are not immediately exposed, but have to be configured. Something like: ```php $math = ffi("/path/to/math.so"); $math->add(1, 5); // Throws RuntimeException, method not configured $math->configure('add', int $a, int $b); $math->add(1, 5); // 6 $math->add('a', 5); Fatal error: Uncaught TypeError: Argument 1 passed to add() must be of the type integer, string given ``` Prior art: === https://pecl.php.net/package/ffi - Last release > 13 years https://github.com/mgdm/MFFI - Not stable, last commit > 1 year, no releases

Alex Bowers

9 years ago
And here is the previous messaging without borked formatting. Sorry folks. FFI RFC ====== There are many languages that support an FFI implementation. NodeJS Python C++ Ruby FFI allows you to call a native C function without requiring the boilerplate of an extension to be written. There are several benefits to having FFI - Performance - Shareability / Bundling - Common functionality between languages Performance === Although less pronounced than the 5.x versions, there is still a performance benefit to having native C code over PHP code. For example, you could utilise threading inside of your FFI methods, which PHP does not expose the ability to do. Shareability === If you wish to implement some of your source code in C, the current way to share it is to build it as an extension. This is cumbersome, and restricts use-cases such as shared hosting, where the ability to install your own extensions is probably restricted. However, with FFI, the shared object can be loaded from any location, and so that restriction is no longer in place. They could even be distributed via composer. Common functionality between languages === If you have some complex logic that needs to be replicated in several languages for whatever reason; implementing it several times over would lead to uncertain bugs and technical debt increasing heavily. If you could share the same logic amongst them all using FFI, then this is no longer an issue. Example === Take an example of a rust program that takes two numbers in and gives you the sum of them. ```rust #[no_mangle] pub extern fn add(a: i32, b: i32) -> i32 { a + b } ``` with the Cargo.toml file containing: ``` [package] name = "math" version = "0.1.0" authors = ["Alex Bowers <bowersbros@gmail.com>"] [dependencies] [lib] name = "math" crate-type = ["dylib"] ``` `cargo build --release` will create `.so`, `.dylib`, or `.dll` files depending on your system. These should be usable within PHP using the exposed functions. ```php $math = ffi("/path/to/math.so"); $result = $math->add(1, 5); echo $result; // 6 ``` With the implementation at its most basic level, calling the `add` method with incorrect parameters would likely cause a segfault. A way around that could be that the methods are not immediately exposed, but have to be configured. Something like: ```php $math = ffi("/path/to/math.so"); $math->add(1, 5); // Throws RuntimeException, method not configured $math->configure('add', int $a, int $b); $math->add(1, 5); // 6 $math->add('a', 5); Fatal error: Uncaught TypeError: Argument 1 passed to add() must be of the type integer, string given ``` Prior art: === https://pecl.php.net/package/ffi - Last release > 13 yearshttps://github.com/mgdm/MFFI - Not stable, last commit > 1 year, no releases

Michael Wallner

9 years ago
On 05/02/17 23:25, Alex Bowers wrote:
> And here is the previous messaging without borked formatting. Sorry folks. > > > FFI RFC > ====== > > There are many languages that support an FFI implementation. > > NodeJS > Python > C++ > Ruby > > FFI allows you to call a native C function without requiring the > boilerplate of an extension to be written. > > There are several benefits to having FFI > > - Performance > - Shareability / Bundling > - Common functionality between languages > > Performance > === > Although less pronounced than the 5.x versions, there is still a > performance benefit to having native C code over PHP code. For > example, you could utilise threading inside of your FFI methods, which > PHP does not expose the ability to do. > > Shareability > === > > If you wish to implement some of your source code in C, the current > way to share it is to build it as an extension. This is cumbersome, > and restricts use-cases such as shared hosting, where the ability to > install your own extensions is probably restricted. However, with FFI, > the shared object can be loaded from any location, and so that > restriction is no longer in place. > > They could even be distributed via composer. > > Common functionality between languages > === > > If you have some complex logic that needs to be replicated in several > languages for whatever reason; implementing it several times over > would lead to uncertain bugs and technical debt increasing heavily. If > you could share the same logic amongst them all using FFI, then this > is no longer an issue. > > Example > === > > Take an example of a rust program that takes two numbers in and gives > you the sum of them. > > ```rust > #[no_mangle] > pub extern fn add(a: i32, b: i32) -> i32 { > a + b > } > > ``` > > with the Cargo.toml file containing: > > ``` > [package] > name = "math" > version = "0.1.0" > authors = ["Alex Bowers <bowersbros@gmail.com>"] > > [dependencies] > > [lib] > name = "math" > crate-type = ["dylib"] > ``` > > `cargo build --release` will create `.so`, `.dylib`, or `.dll` files > depending on your system. > > These should be usable within PHP using the exposed functions. > > ```php > $math = ffi("/path/to/math.so"); > $result = $math->add(1, 5); > > echo $result; // 6 > ``` > > With the implementation at its most basic level, calling the `add` > method with incorrect parameters would likely cause a segfault. > > A way around that could be that the methods are not immediately > exposed, but have to be configured. > > Something like: > > ```php > $math = ffi("/path/to/math.so"); > $math->add(1, 5); // Throws RuntimeException, method not configured > > $math->configure('add', int $a, int $b); > > $math->add(1, 5); // 6 > $math->add('a', 5); Fatal error: Uncaught TypeError: Argument 1 passed > to add() must be of the type integer, string given > ``` > > Prior art: > === > https://pecl.php.net/package/ffi - Last release > 13 > yearshttps://github.com/mgdm/MFFI - Not stable, last commit > 1 year, > no releases >
Trying to add answers later today, in the meantime, please have a look at https://github.com/m6w6/ext-psi
-- Regards, Mike

Michael Wallner

9 years ago
On 05/02/17 23:25, Alex Bowers wrote:
> And here is the previous messaging without borked formatting. Sorry folks. > > > FFI RFC > ======
...
> Example > === > > Take an example of a rust program that takes two numbers in and gives > you the sum of them. > > ```rust > #[no_mangle] > pub extern fn add(a: i32, b: i32) -> i32 { > a + b > } > > ``` > > with the Cargo.toml file containing: > > ``` > [package] > name = "math" > version = "0.1.0" > authors = ["Alex Bowers <bowersbros@gmail.com>"] > > [dependencies] > > [lib] > name = "math" > crate-type = ["dylib"] > ``` > > `cargo build --release` will create `.so`, `.dylib`, or `.dll` files > depending on your system. > > These should be usable within PHP using the exposed functions. > > ```php > $math = ffi("/path/to/math.so"); > $result = $math->add(1, 5); > > echo $result; // 6 > ```
With ext/psi [1], that would work differently. You wouldn't define the interface at runtime, like the above example, pecl/ffi [2] or MFFI [3]. First, you would create an interface description file (a PSI file [4]), where you specify the native declaration and the library to load: ``` // https://github.com/m6w6/ext-psi#lib lib "/path/to/math.so"; // https://github.com/m6w6/ext-psi#declarations extern int32_t add(int32_t a, int32_t b); ``` The PHP interface to this function would usually be defined in the same file possibly looking along the following lines: ``` // https://github.com/m6w6/ext-psi#implementations function math\add(int $a, int $b) : int { let a = intval($a); let b = intval($b); return to_int(add); } ``` Those PSI files would be loaded from a directory configured in php.ini (psi.directory) and loaded at startup, i.e. the interfaces would be available to user land for the lifetime of the process, not the request. Have a look at the complete libidn example [5] of the README or the sqlite test [6] utilizing a user land callback. I've been working on ext/psi for the good parts of more than a year now, and I would be more than happy about insights, suggestions or collaboration with other interested parties. The (currently) two major to-dos are decent CPP support in the parser, so one could basically just parse C headers, and levelling out support for input/output marshalling, which currently differ in sophistication, AFAIR, while quite a lot is already possible. Other random bits can be found in the TODO [7]. It has been successfully built and run on Linux and OSX but YMMV; also, grab a coffee when building the first time, because `configure` performs a ton of checks for POSIX builtins (still like, 10% only yet), so don't forget the -C/--cache-file switch. [1] https://github.com/m6w6/ext-psi [2] https://pecl.php.net/ffi [3] https://github.com/mgdm/MFFI [4] https://github.com/m6w6/ext-psi#psi-files [5] https://github.com/m6w6/ext-psi#complete-example [6] https://github.com/m6w6/ext-psi/tree/master/tests/sqlite [7] https://github.com/m6w6/ext-psi/blob/master/TODO
-- Regards, Mike

Michał Brzuchalski

9 years ago
2017-02-08 12:51 GMT+01:00 Michael Wallner <mike@php.net>:
> On 05/02/17 23:25, Alex Bowers wrote: > > And here is the previous messaging without borked formatting. Sorry > folks. > > > > > > FFI RFC > > ====== > ... > > Example > > === > > > > Take an example of a rust program that takes two numbers in and gives > > you the sum of them. > > > > ```rust > > #[no_mangle] > > pub extern fn add(a: i32, b: i32) -> i32 { > > a + b > > } > > > > ``` > > > > with the Cargo.toml file containing: > > > > ``` > > [package] > > name = "math" > > version = "0.1.0" > > authors = ["Alex Bowers <bowersbros@gmail.com>"] > > > > [dependencies] > > > > [lib] > > name = "math" > > crate-type = ["dylib"] > > ``` > > > > `cargo build --release` will create `.so`, `.dylib`, or `.dll` files > > depending on your system. > > > > These should be usable within PHP using the exposed functions. > > > > ```php > > $math = ffi("/path/to/math.so"); > > $result = $math->add(1, 5); > > > > echo $result; // 6 > > ``` > > With ext/psi [1], that would work differently. You wouldn't define the > interface at runtime, like the above example, pecl/ffi [2] or MFFI [3]. > > First, you would create an interface description file (a PSI file [4]), > where you specify the native declaration and the library to load: > > ``` > // https://github.com/m6w6/ext-psi#lib > lib "/path/to/math.so"; > > // https://github.com/m6w6/ext-psi#declarations > extern int32_t add(int32_t a, int32_t b); > ``` > > The PHP interface to this function would usually be defined in the same > file possibly looking along the following lines: > > ``` > // https://github.com/m6w6/ext-psi#implementations > function math\add(int $a, int $b) : int { > let a = intval($a); > let b = intval($b); > return to_int(add); > } > ``` >
What is the purpose of `intval` call if both variables were declared as int already? Also what to_int stands for if we already declared int return type?
> > Those PSI files would be loaded from a directory configured in php.ini > (psi.directory) and loaded at startup, i.e. the interfaces would be > available to user land for the lifetime of the process, not the request. > > Have a look at the complete libidn example [5] of the README or the > sqlite test [6] utilizing a user land callback. > > I've been working on ext/psi for the good parts of more than a year now, > and I would be more than happy about insights, suggestions or > collaboration with other interested parties. > > The (currently) two major to-dos are decent CPP support in the parser, > so one could basically just parse C headers, and levelling out support > for input/output marshalling, which currently differ in sophistication, > AFAIR, while quite a lot is already possible. Other random bits can be > found in the TODO [7]. > > It has been successfully built and run on Linux and OSX but YMMV; also, > grab a coffee when building the first time, because `configure` performs > a ton of checks for POSIX builtins (still like, 10% only yet), so don't > forget the -C/--cache-file switch. > > > [1] https://github.com/m6w6/ext-psi > [2] https://pecl.php.net/ffi > [3] https://github.com/mgdm/MFFI > [4] https://github.com/m6w6/ext-psi#psi-files > [5] https://github.com/m6w6/ext-psi#complete-example > [6] https://github.com/m6w6/ext-psi/tree/master/tests/sqlite > [7] https://github.com/m6w6/ext-psi/blob/master/TODO > > -- > Regards, > Mike > >
-- regards / pozdrawiam, -- Michał Brzuchalski about.me/brzuchal brzuchalski.com

Michael Wallner

9 years ago
On 08/02/17 14:54, Michał Brzuchalski wrote:
> 2017-02-08 12:51 GMT+01:00 Michael Wallner <mike@php.net>: > >> >> ``` >> // https://github.com/m6w6/ext-psi#implementations >> function math\add(int $a, int $b) : int { >> let a = intval($a); >> let b = intval($b); >> return to_int(add); >> } >> ``` >> > > What is the purpose of `intval` call if both variables were declared as int > already? > Also what to_int stands for if we already declared int return type? >
While those type casts might seem superfluous for simple functions, marshalling between PHP and C quickly becomes complicated and requiring explicit casts just reduces the possibility of tripping points even if the types match on both sides. Omitting them wouldn't change anything internally, but just bloat the parser.
-- Regards, Mike