PHP Modules

php.internals

Michael Morris

3 years ago
This will be long. I've read over the Future Stability thread and taken it in, and decided to mull over an idea I touched on over a decade ago that I think might help. Also, in the interceding years the JavaScript community has overcome a compatibility issue using this technique, so we might do the same. The crux of the stability problem is the need to update functions without introducing a BC break. Adding in new features without removing old ones also makes it confusing to incoming programmers as to what to use. I propose PHP Modules to hold new features. The existing namespace and functions would be left alone. Existing files would also not be affected - the behavior of PHP modules would be entirely opt in. They would work similar JavaScript modules - they would use the import keyword and the include, require, include_once and require_once directives would pitch an error if used in a PHP module. PHP Modules also would not parse as templates at all - no opening <?php tag needed (and yes, I know that will be a headache for IDE makers, but it's not insurmountable). PHP would have a new ini directive to use them similar to the "type": "module" directive that npm uses now. If that ini directive isn't set, php files would be handled as they always have. mphp files would always be handled as php modules though and lphp (legacy php) would always be handled as legacy php. I expect some objection to file extensions affecting parsing behavior, but over the last 5 years this approach has worked for the JavaScript community with cjs, mjs and js files. Here the existing php, phtml, php3, php4 extensions are handled as the directive instructs them to be handled, only the two new never before used extensions of lphp and mphp would do their thing. In a PHP module a function has to be imported before it's used with a handful of exceptions. The root namespace of php modules is utterly empty, or as close to empty as makes sense. The existing functions would be gathered into modules so that they can be imported. While this is done the headache inducing inconsistencies like haystack, needle for strings and needle, haystack for arrays can be addressed. I really don't care which is adopted, but having one order for this language wide would be nice. Also, decide once and for all - camelCase or underscore_case. The above alone would be massive. Maybe it's impossible given the number of devs available. The one thing modules can do that the current system cannot is allow devs to pick which version of the module to use: import split from 'php.mb' // imports the mbsplit() function from the current version of PHP. Say the mb module gets some bc breaks. We can put them into a new module so that the behavior is once again opt in. The strongest way to do this is to make composer a 1st class part of the language and let it specify the version of the module that is loaded. The import command would be able to pull from the file system or from the import map as JavaScript does currently. For ease of dev headaches I'd recommend hewing to the JS model as close as possible as it is proven, and many of us already use it anyway when developing browser code. I hope this helps, or at least spurs a conversation to come up with something to address this issue.

Sara Golemon

3 years ago
> On Apr 10, 2023, at 20:40, Michael Morris <tendoaki@gmail.com> wrote: > > This will be long.
Yes, it's long, so I'm going to focus on a couple things and not quote too much.
> I propose PHP Modules to hold new features.
What you seem to have described is improved lexical scoping. Rather than a shared "global" scope for classes, functions, constants, and variables, you'd like to allow files, or perhaps even statement lists (open brace to closed brace) to work as either independent or compositional scopes. Basically, make scopes in a certain mode of PHP act more like Javascript. I'll be honest, if we could throw away PHP's old behavior and make this the only mode, I'd be pretty happy. We can't of course, at best we'd have to mix (and yes, I know that's part of your suggestion). This is a LOT of complexity. Not impossible, nothing is, but I'm fairly confident it would make maintaining the runtime harder, and would notably slow execution. Not saying "no", since I lack that power, but I do want to make it clear how HARD this feature would be to implement well.
> PHP Modules also would not > parse as templates at all - no opening <?php tag needed (and yes, I know > that will be a headache for IDE makers, but it's not insurmountable).
I would suggest, as a compromise, keep the initial open tag (or maybe an alternative one, like <?phm ). Require it come first, and not permitting closing tag. This would simplify webserver config (and avoid making file extensions special) while simplifying the experience in editors. Just a suggestion.
> I hope this helps, or at least spurs a conversation to come up with > something to address this issue.
Lastly, like the stability thread, this isn't a new suggestion, and I'd recommend googling back through the archives for prior discussions. Past failures don't mean it won't work today... Opinions change and so does the runtime, but it may be helpful to understand the complexities presented by this strategy, and what the objections to it will likely boil down to. See also "PHP Editions" which is a different, but closely related proposal put forth by Andi as recently as the 7.x era. -Sara

Arvids Godjuks

3 years ago
On Tue, 11 Apr 2023 at 04:41, Michael Morris <tendoaki@gmail.com> wrote:
> This will be long. I've read over the Future Stability thread and taken it > in, and decided to mull over an idea I touched on over a decade ago that I > think might help. Also, in the interceding years the JavaScript community > has overcome a compatibility issue using this technique, so we might do the > same. > > The crux of the stability problem is the need to update functions without > introducing a BC break. Adding in new features without removing old ones > also makes it confusing to incoming programmers as to what to use. > > I propose PHP Modules to hold new features. The existing namespace and > functions would be left alone. Existing files would also not be affected - > the behavior of PHP modules would be entirely opt in. > > They would work similar JavaScript modules - they would use the import > keyword and the include, require, include_once and require_once directives > would pitch an error if used in a PHP module. PHP Modules also would not > parse as templates at all - no opening <?php tag needed (and yes, I know > that will be a headache for IDE makers, but it's not insurmountable). > > PHP would have a new ini directive to use them similar to the "type": > "module" directive that npm uses now. If that ini directive isn't set, php > files would be handled as they always have. mphp files would always be > handled as php modules though and lphp (legacy php) would always be handled > as legacy php. > > I expect some objection to file extensions affecting parsing behavior, but > over the last 5 years this approach has worked for the JavaScript community > with cjs, mjs and js files. Here the existing php, phtml, php3, php4 > extensions are handled as the directive instructs them to be handled, only > the two new never before used extensions of lphp and mphp would do their > thing. > > In a PHP module a function has to be imported before it's used with a > handful of exceptions. The root namespace of php modules is utterly empty, > or as close to empty as makes sense. > > The existing functions would be gathered into modules so that they can be > imported. While this is done the headache inducing inconsistencies like > haystack, needle for strings and needle, haystack for arrays can be > addressed. I really don't care which is adopted, but having one order for > this language wide would be nice. Also, decide once and for all - camelCase > or underscore_case. > > The above alone would be massive. Maybe it's impossible given the number > of devs available. The one thing modules can do that the current system > cannot is allow devs to pick which version of the module to use: > > import split from 'php.mb' // imports the mbsplit() function from the > current version of PHP. > > Say the mb module gets some bc breaks. We can put them into a new module so > that the behavior is once again opt in. The strongest way to do this is to > make composer a 1st class part of the language and let it specify the > version of the module that is loaded. > > The import command would be able to pull from the file system or from the > import map as JavaScript does currently. For ease of dev headaches I'd > recommend hewing to the JS model as close as possible as it is proven, and > many of us already use it anyway when developing browser code. > > I hope this helps, or at least spurs a conversation to come up with > something to address this issue. >
One thing I like about PHP is that it has one mainline engine implementation with a single somewhat unfragmented ecosystem and it's an autoloading killer feature with Composer backing it up. I consider that combo to be the unique trait of PHP as the language and ecosystem. You change that, you basically convert PHP into JavaScript, Go, Ruby or any other modern language that's used widely in webdev. It loses its unique traits and is why many people like it - the ecosystem. I dabbled with NPM ecosystem enough to know that I'm gonna be the NYMBY type person screaming "GET OF MY LAWN!". Just look at Python and its 2 => 3 transition saga. PHP, the project, just does not have the resources to deal with something like this and maintain multiple major versions of the language. Don't forget - JavaScript's development is backed by some of the biggest corporations on the planet with functionally limitless budgets. They can afford have 1000 people working on the JS spec, engines, browsers and nodejs to sustain it all. Not even talking the amount of resources Google sunk into V8 to the point even Microsoft just gave up on Internet Explorer and migrated now runs Edge instead. Frankly, I think a lot of people have wholly unrealistic expectations of the scale and resources available to the PHP project - it's about 8 figure sum away on a yearly basis to be able to afford all the things people want PHP to do. And it will take at least half a decade just to onboard and get people familiar with the engine enough to even start major projects. And then there are reasons that Sara has pointed out - the technical complexity alone is massive.
-- Arvīds Godjuks +371 26 851 664 arvids.godjuks@gmail.com Telegram: @psihius https://t.me/psihius

Jannes Blume

3 years ago
I’d like to give my two cents on this as I’m currently exploring “dependency hell" as part of my dissertation, including a PoC implementation of a first-class module system for PHP. The hardest part was probably to find the least invasive way of limiting the lexical scope of global declarations like classes, free-standing functions and constants, similar to what Sara has described. I’ve ultimately decided to go with a “module.php” (manifest) file, that hosts an “export” array of said symbols that should be available to dependents. The file defines the boundaries of a “module” and all files loaded from within its directory tree will be treated according to the scoping rules defined in the manifest. While all of this is certainly possible, I can see a lot more problems, even with a more sophisticated implementation that is supposed to be adapted by the sheer amount of Composer packages to become useful for the community. PHP developers, rightfully, make the assumption that a definition will only exist once in the current execution context, as implied by namespaces and the convention of having one prefix for a Composer package (I’m aware that we’ve had some versioning in Namespace paths previously).
> Say the mb module gets some bc breaks. We can put them into a new module so > that the behavior is once again opt in. The strongest way to do this is to > make composer a 1st class part of the language and let it specify the > version of the module that is loaded.
If we’re introducing a possibility to have two or more versions of a function, class, constant, …, how can we make sure they do not clash across module boundaries? Is this the responsibility of the author when defining the public API? I think JavaScript thrives with its module system because it’s dynamically typed and I’d argue, that because of this, developers can get away with most minor inconsistencies in behaviour between versions. How would we combine this with strict types in PHP, that become more prevalent with each release? I’m a supporter of the idea in general but would agree with my predecessors, that it might not be viable right now due to the scale of this. However, I am curious to hear more thoughts on this. Jannes B

Larry Garfield

3 years ago
On Tue, Apr 11, 2023, at 1:40 AM, Michael Morris wrote:
> This will be long. I've read over the Future Stability thread and taken it > in, and decided to mull over an idea I touched on over a decade ago that I > think might help. Also, in the interceding years the JavaScript community > has overcome a compatibility issue using this technique, so we might do the > same. > > The crux of the stability problem is the need to update functions without > introducing a BC break. Adding in new features without removing old ones > also makes it confusing to incoming programmers as to what to use. > > I propose PHP Modules to hold new features. The existing namespace and > functions would be left alone. Existing files would also not be affected - > the behavior of PHP modules would be entirely opt in. > > They would work similar JavaScript modules - they would use the import > keyword and the include, require, include_once and require_once directives > would pitch an error if used in a PHP module. PHP Modules also would not > parse as templates at all - no opening <?php tag needed (and yes, I know > that will be a headache for IDE makers, but it's not insurmountable). > > PHP would have a new ini directive to use them similar to the "type": > "module" directive that npm uses now. If that ini directive isn't set, php > files would be handled as they always have. mphp files would always be > handled as php modules though and lphp (legacy php) would always be handled > as legacy php. > > I expect some objection to file extensions affecting parsing behavior, but > over the last 5 years this approach has worked for the JavaScript community > with cjs, mjs and js files. Here the existing php, phtml, php3, php4 > extensions are handled as the directive instructs them to be handled, only > the two new never before used extensions of lphp and mphp would do their > thing. > > In a PHP module a function has to be imported before it's used with a > handful of exceptions. The root namespace of php modules is utterly empty, > or as close to empty as makes sense. > > The existing functions would be gathered into modules so that they can be > imported. While this is done the headache inducing inconsistencies like > haystack, needle for strings and needle, haystack for arrays can be > addressed. I really don't care which is adopted, but having one order for > this language wide would be nice. Also, decide once and for all - camelCase > or underscore_case. > > The above alone would be massive. Maybe it's impossible given the number > of devs available. The one thing modules can do that the current system > cannot is allow devs to pick which version of the module to use: > > import split from 'php.mb' // imports the mbsplit() function from the > current version of PHP. > > Say the mb module gets some bc breaks. We can put them into a new module so > that the behavior is once again opt in. The strongest way to do this is to > make composer a 1st class part of the language and let it specify the > version of the module that is loaded. > > The import command would be able to pull from the file system or from the > import map as JavaScript does currently. For ease of dev headaches I'd > recommend hewing to the JS model as close as possible as it is proven, and > many of us already use it anyway when developing browser code. > > I hope this helps, or at least spurs a conversation to come up with > something to address this issue.
The main question, in my mind, is what problem is this solving. From the description here, it sounds like you're targeting "clean up inconsistencies in the stdlib", which could be done much more easily by putting new versions in a namespace, something that's been discussed many times. (And usually run into a wall of how much redesigning we should do in the process, and then people lose interest.) The previous times modules have been proposed, the big selling point was module-level visibility. Eg, object properties or methods that are visible to other classes/functions in the same module, but not outside of it. That's a problem space that would be solved by modules, which namespaces cannot solve. (Whether worth the effort or not is a separate question.) Most of the BC discussion right now (and generally) isn't about function or class redesign. It's about changes in language syntax behavior, such as undefined-var/key being promoted from Notice to Warning (8.0), or making dynamic properties opt-in via an attribute, or having stdlib functions react to null parameters the same way as userspace functions, etc. Regardless of whether you consider those good or bad changes, they're not the sort of thing that modules would solve. There has been discussion in the past of adding more declare statements to allow opt in/out of certain changes, like declare(warn_on_missing_var=1) or something. To whatever extent that's feasible, it's less work than doing it that way (per-file) than introducing a module system to handle that switch. However, as previous discussions have shown that is also a huge amount of work, as we have to maintain two different behaviors in the engine and switch per file... and then also figure out when the default changes, and when it becomes just the new approach always. Which doesn't solve the issue that at some point code that ignores missing variables *will still break*, it's just pushing that date off even longer at the cost of still more engine complexity. But the same people will still complain just as loudly whenever that is, because they haven't done anything about it for the past 17 years so they're not going to now. So, what are the problems to be solved that only a good module system would be able to? Start there, before thinking about how a module system would look. I do think there are potential benefits to a formal module/package system, but that's where the discussion would need to start, not with what the <? tag looks like. --Larry Garfield

Deleu

3 years ago
On Tue, Apr 11, 2023, 11:32 AM Larry Garfield <larry@garfieldtech.com> wrote:
> But the same people will still complain just as loudly whenever that is, > because they haven't done anything about it for the past 17 years so > they're not going to now. >
Do you know that for a fact or should this statement be classified as, and I'm quoting here, "BS"?