Everything
php
Namespaces internal refactoring
php.internals
Michał Brzuchalski
10 years ago
Hi internals,
I would like to present my idea for namespaces refactoring and provide an
RFC for them.
Currently in PHP namespaces are implemented as a prefix of class or
function name. Using ReflectionClass or ReflectionFunction we can retrieve
class/function name and its namespace-name. In both situations it's
implemented as substring operation on class/function name:
ReflectionClass::getNamespaceName():
https://github.com/php/php-src/blob/master/ext/reflection/php_reflection.c#L5283
ReflectionFunction::getNamespaceName():
https://github.com/php/php-src/blob/master/ext/reflection/php_reflection.c#L3427
In both we can see the result of method is part of name.
IMHO those namespaces should be implemented as a struct like zend_namespace
struct with flags and name and should not be a part of class/function name.
Such change could increase performance on those Reflection* methods but not
only. I can see a good usage of such struct in implementing new features
like class/function visibility modifiers like: private, protected,
package-private or internal.
Some time ago there was discussion about final-properties I have started
https://marc.info/?t=145979255800003&r=1&w=2
In this discussion there was also talk about private class and internal
methods http://marc.info/?l=php-internals&m=146005798902985&w=2 as we can
see in PR https://github.com/php/php-src/pull/947#discussion_r21542750
which is little outdated there is implementation of class modifiers, and
there is exactly the same thing I was using in my package-private
implementation which uses string subctring comparison as we can see here in
ZEND_API int zend_is_same_namespace(const char *fc, const char *sc)
https://github.com/php/php-src/pull/947/files#diff-5459151fc119e0faf6f26162999c0cfdR2411
IMHO those hack's are result of badly implemented namespaces and leaving
them as they are won't bring new features highly optimized against
execution performance, because of string comparison.
Reimplementing namespaces as a struct could bring optimization level breat
boost as it can simply comapre pointer to zend_namespace struct.
An advantage of struct coult bring us new features like internal namespaces
in near future simply by flags field in zend_namespace struct - in feature
there could be some cleaning of build in classes and functions and move
into internal system namespace eg. \PHP\DateTime or whatever - not
important right now.
Another adwantage is comparing namespaces in class/function range modifiers
like private,protected,package-private or whatever.
There could be even some kind of extension flag for namespace providing
protection of namespace used in some extension awoiding to missusage and
providing some extension-private class/function implementations which could
be protected and the user would get an error while implementing, extending
- private classes from extension or smth like that.
I can see also some unconsistencies in class/function usage in PHP we can
use A/B::class to retrieve whole-class name with namespace, there could be
also some magic const like A/B::ns and it won't need to substring class
name, maybe in a future ther would be need to provide such syntax for
functions? A/c::ns where A/c is function, right now there is only way to
pass function name into ReflectionFunction as a string, classes have
::class magic const and it's easy to refactor such code, eg. change
namespace name which could lead to errors in IDE where functions are in
namespaces! Because searching for whole function name with namespace can
exists in code only as a string.
For eg. code like that:
<?php
namespace A {
class B {}
function c() {}
}
namespace {
$cr = new ReflectionClass(A\B::class);
var_dump($cr->getName(), $cr->getNamespaceName());
$fr = new ReflectionFunction('A\c');
var_dump($fr->getName(), $fr->getNamespaceName());
}
Will execute string substringing for class/function getNamespaceName()
reflection method which is simply SLOW! Such information isn't even cached!
Implementing such struct as zend_namespace could bring namespaces registry
at run-time. Such namespaces could have wheir parents as pointers to other
structs like in tree so they could optimize memory usage for eg. having
1000 classes in Test namespace preserves right now 1000 zend_string's with
class name and 1000 * 4 chars("Test") in quite longer namespace names those
numbers could be significant! The same is with function names.
I now it's not an easy task because there are lot of source code lines
which use zend_class->name and same for function pointers.
Nowadays probably none of extension uses namespaces so it's not a real
problem right now to provide such change.
Thanks for reading,
regards,
--
Michał Brzuchalski
Julien Pauli
10 years ago
On Mon, Aug 8, 2016 at 1:08 PM, Michał Brzuchalski
<michal@brzuchalski.com> wrote:
> Hi internals,
>
> I would like to present my idea for namespaces refactoring and provide an
> RFC for them.
>
> Currently in PHP namespaces are implemented as a prefix of class or
> function name. Using ReflectionClass or ReflectionFunction we can retrieve
> class/function name and its namespace-name. In both situations it's
> implemented as substring operation on class/function name:
>
> ReflectionClass::getNamespaceName():
> https://github.com/php/php-src/blob/master/ext/reflection/php_reflection.c#L5283
> ReflectionFunction::getNamespaceName():
> https://github.com/php/php-src/blob/master/ext/reflection/php_reflection.c#L3427
>
> In both we can see the result of method is part of name.
>
> IMHO those namespaces should be implemented as a struct like zend_namespace
> struct with flags and name and should not be a part of class/function name.
> Such change could increase performance on those Reflection* methods but not
> only. I can see a good usage of such struct in implementing new features
> like class/function visibility modifiers like: private, protected,
> package-private or internal.
>
> Some time ago there was discussion about final-properties I have started
> https://marc.info/?t=145979255800003&r=1&w=2
> In this discussion there was also talk about private class and internal
> methods http://marc.info/?l=php-internals&m=146005798902985&w=2 as we can
> see in PR https://github.com/php/php-src/pull/947#discussion_r21542750
> which is little outdated there is implementation of class modifiers, and
> there is exactly the same thing I was using in my package-private
> implementation which uses string subctring comparison as we can see here in
> ZEND_API int zend_is_same_namespace(const char *fc, const char *sc)
> https://github.com/php/php-src/pull/947/files#diff-5459151fc119e0faf6f26162999c0cfdR2411
>
> IMHO those hack's are result of badly implemented namespaces and leaving
> them as they are won't bring new features highly optimized against
> execution performance, because of string comparison.
> Reimplementing namespaces as a struct could bring optimization level breat
> boost as it can simply comapre pointer to zend_namespace struct.
>
> An advantage of struct coult bring us new features like internal namespaces
> in near future simply by flags field in zend_namespace struct - in feature
> there could be some cleaning of build in classes and functions and move
> into internal system namespace eg. \PHP\DateTime or whatever - not
> important right now.
>
> Another adwantage is comparing namespaces in class/function range modifiers
> like private,protected,package-private or whatever.
>
> There could be even some kind of extension flag for namespace providing
> protection of namespace used in some extension awoiding to missusage and
> providing some extension-private class/function implementations which could
> be protected and the user would get an error while implementing, extending
> - private classes from extension or smth like that.
>
> I can see also some unconsistencies in class/function usage in PHP we can
> use A/B::class to retrieve whole-class name with namespace, there could be
> also some magic const like A/B::ns and it won't need to substring class
> name, maybe in a future ther would be need to provide such syntax for
> functions? A/c::ns where A/c is function, right now there is only way to
> pass function name into ReflectionFunction as a string, classes have
> ::class magic const and it's easy to refactor such code, eg. change
> namespace name which could lead to errors in IDE where functions are in
> namespaces! Because searching for whole function name with namespace can
> exists in code only as a string.
>
> For eg. code like that:
>
> <?php
> namespace A {
> class B {}
> function c() {}
> }
> namespace {
> $cr = new ReflectionClass(A\B::class);
> var_dump($cr->getName(), $cr->getNamespaceName());
> $fr = new ReflectionFunction('A\c');
> var_dump($fr->getName(), $fr->getNamespaceName());
> }
>
> Will execute string substringing for class/function getNamespaceName()
> reflection method which is simply SLOW! Such information isn't even cached!
>
>
> Implementing such struct as zend_namespace could bring namespaces registry
> at run-time. Such namespaces could have wheir parents as pointers to other
> structs like in tree so they could optimize memory usage for eg. having
> 1000 classes in Test namespace preserves right now 1000 zend_string's with
> class name and 1000 * 4 chars("Test") in quite longer namespace names those
> numbers could be significant! The same is with function names.
>
> I now it's not an easy task because there are lot of source code lines
> which use zend_class->name and same for function pointers.
>
> Nowadays probably none of extension uses namespaces so it's not a real
> problem right now to provide such change.
It is.
I agree with your thoughts.
Back in 2008, namespaces were designed and added to PHP 5.3 as just
substrings in class names.
That is not the right way to do things, but we had not that much time
to design them otherwise, and we did not want to break the API too
much by adding new structures.
So, namespaces were designed as "just" some substrings in class names,
with all the problems you spot.
I guess Dmitry will have some words about that, as he lead this
feature back in time.
Refactoring namespaces could happen, but many extensions use them, and
they are heavilly used as well in OPCache and its optimizer.
The task won't be easy, and the break will be massive for internal
(some extensions would need to be rewritten, as well as many parts of
the engine).
As such, you may target next PHP major, aka PHP 8 for your ideas.
Julien.P
Stas Malyshev
10 years ago
Hi!
> It is.
>
> I agree with your thoughts.
> Back in 2008, namespaces were designed and added to PHP 5.3 as just
> substrings in class names.
That is correct (except that namespaces first appeared in 2007, first
commit f32ffe9b430b718628f868e449c1fcbdc8ec9ef6 if you're interested, of
course were many since then that changed stuff :). However, I do not
think what follows is.
> That is not the right way to do things, but we had not that much time
> to design them otherwise, and we did not want to break the API too
> much by adding new structures.
> So, namespaces were designed as "just" some substrings in class names,
> with all the problems you spot.
Designing namespaces as they are was a deliberate design decision, and
it was not because we "didn't want to break APIs" or because we did not
know better or were afraid of adding another struct, but because it was
decided to choose a simple solution that allows to achieve a lot without
introducing a lot of complexity. I certainly object to the blanket
characterization of it as "not the right way to do things".
I took part in the implementation and though it was almost 9 years ago,
I think I still remember at least something :)
We did consider using something like Java or Python packages, and we
decided it's much too complex and not very suitable for dynamic world of
PHP where classes and class files can be loaded at any time at any order.
I think some people misunderstand this concept and think PHP namespaces
are like half-baked Java packages but without some features because we
were too short-sighted or too hasty to implement those. No, they are not
like Java packages and they are implemented as not being separate
entities but being name prefixes very consciously and deliberately.
Now, 9 years later, we can reopen this question and think maybe our
needs changed and maybe different model would serve us better. This is
certainly possible and there's nothing wrong with that. But I think we
need to approach it with right understanding of why we took certain
decisions and where these decisions come from. It is very wrong to
refactor something without understanding why it was built like that in
the first place, and without understanding the initial design and
requirements. We spent a real lot of time discussing all this stuff, and
it'll be a pity if we spend this time again without using our previous
experience.
I also don't see any need in having "namespace registry", and, in fact,
if you understand conceptual model of PHP namespaces as they are now, it
doesn't make much sense to me.
--
Stas Malyshev
smalyshev@gmail.com
Fleshgrinder
10 years ago
The current approach to namespaces blocks new features like you see it
in the following PR. The link already points to the two most relevant
comments that should give you full context:
https://github.com/php/php-src/pull/947/#issuecomment-224912697
You might know other approaches to solve this. What I want to say is, we
should not re-discuss everything that was discussed before, we should
discuss the proposed features and refactor the namespaces IF it is required.
--
Richard "Fleshgrinder" Fussenegger
Stas Malyshev
10 years ago
Hi!
> The current approach to namespaces blocks new features like you see it
> in the following PR. The link already points to the two most relevant
> comments that should give you full context:
>
> https://github.com/php/php-src/pull/947/#issuecomment-224912697
I don't think this - private classes, etc. - is a good idea. It looks
like Javaization of the language and introducing more and more
complexity without any clear benefit, and AFAIK none of the mainstream
scripting languages have such concept. Anyway, this doesn't even have an
RFC, so I don't see why we should refactor everything to accommodate it.
But it is certainly does not fit the concept of namespaces. It needs
completely different packages concept - one that currently does not
exist on PHP. Trying to bolt it on existing concept is not going to do
any good. I think this is what it should start with - developing a good
concept, not trying to go from code modification to design. Design
should come first, then code modification.
> You might know other approaches to solve this. What I want to say is, we
> should not re-discuss everything that was discussed before, we should
> discuss the proposed features and refactor the namespaces IF it is required.
Indeed.
--
Stas Malyshev
smalyshev@gmail.com
Guilherme Blanco
10 years ago
Hi Stas,
I'll answer your message as it directly refers to links with my name.
I understand the reasons of why namespaces were implemented that way, as I
follow this list for quite a long time. I never complained about the
implementation itself as it was publicly introduced originally back in
August 2008, when first PHP 5.3 alpha 1 got released.
Today I see 2 sides in PHP Internals. One that truly believes that PHP
should adopt more concepts of object orientation, such as Annotations,
Generics, Overloading, Class visibility, Collections, Exceptions, etc (the
side I support) and the other side that wants to make the engine more
robust, stable and faster, without entering in a rollercoaster of new
functionality to be introduced that brings instability.
One thing that didn't change is the constant increase of developers writing
and rewriting software using the language. Now these developers are also
improving their skills, and demanding more flexible ways of achieving their
needs. Language evolution should also be in compass with these new needs,
and that's where I think PHP is struggling to keep up, for a variety of
reasons.
Being a constant contributor to open source and many libraries that now
drives most of what people consider as "Modern PHP", I constantly face
problems with the language itself being unable to achieve my needs. When I
say my needs, I mean needs of heavily used libraries, with hundreds or
thousands of other projects depending on them.
I'll highlight the main driver of "private classes" conceptualization, in
an attempt to make this email short (which I already failed). I wrote a DAG
sorter as part of one of Doctrine packages. Many people started to
instantiate (and use this class, even though it was marked as an @internal)
and consume the benefits it provides, until I changed its API working on a
bug fix. If this class was never intended to be used outside of the scope
of this project, how can restrict its usage by outside packages? Feel free
to provide me answers or alternatives, and I'll happily agree with you that
private classes is a bad idea.
Namespace implementation is now the blocker of many new features that could
help solve complex problems that you don't see on a daily basis, but
library developers face daily. For example: allowing to assess
imported/used structs when declaring a class would drastically reduce the
complexity of Doctrine's annotation parser, which required token_get_all
and several logic to check for what you're including. Is there a better
solution for that too? Well, I'd argue Annotations has a market, and people
are just blind if they don't wanna see it. Answering that implementing
Annotations in userland is a bad idea is like telling a corporation
shouldn't create a product because you don't like it. Makes absolutely no
sense.
I do have a lengthy, detailed and complex plan of how namespace refactoring
could be made, what it would enable feature-wise, and how it would benefit
many projects. I'm open to share with you or any other internals developer,
assess impact over PHP source code and even help implementing it. I'm just
unable to do it all by myself due to my poor C knowledge. What I don't want
to see is blind resistance to language evolution and blaming over something
as badly designed without understanding the circumstances that drove to
that implementation decision.
Regards,
On Sat, Aug 13, 2016 at 3:20 PM, Stanislav Malyshev <smalyshev@gmail.com>
wrote:
> Hi!
>
> > The current approach to namespaces blocks new features like you see it
> > in the following PR. The link already points to the two most relevant
> > comments that should give you full context:
> >
> > https://github.com/php/php-src/pull/947/#issuecomment-224912697
>
> I don't think this - private classes, etc. - is a good idea. It looks
> like Javaization of the language and introducing more and more
> complexity without any clear benefit, and AFAIK none of the mainstream
> scripting languages have such concept. Anyway, this doesn't even have an
> RFC, so I don't see why we should refactor everything to accommodate it.
>
> But it is certainly does not fit the concept of namespaces. It needs
> completely different packages concept - one that currently does not
> exist on PHP. Trying to bolt it on existing concept is not going to do
> any good. I think this is what it should start with - developing a good
> concept, not trying to go from code modification to design. Design
> should come first, then code modification.
>
> > You might know other approaches to solve this. What I want to say is, we
> > should not re-discuss everything that was discussed before, we should
> > discuss the proposed features and refactor the namespaces IF it is
> required.
>
> Indeed.
> --
> Stas Malyshev
> smalyshev@gmail.com
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>
--
Guilherme Blanco
Lead Architect at E-Block
Stas Malyshev
10 years ago
Hi!
> Today I see 2 sides in PHP Internals. One that truly believes that PHP
> should adopt more concepts of object orientation, such as Annotations,
> Generics, Overloading, Class visibility, Collections, Exceptions, etc
Object orientation is a very broad term. You can be very well object
oriented language and not be Java. In fact, almost all OO languages are
not Java ;) And a lot of OO languages (most of ones I know) don't have
Javaesque features like private classes, many also do fine without
generics.
> One thing that didn't change is the constant increase of developers
> writing and rewriting software using the language. Now these developers
> are also improving their skills, and demanding more flexible ways of
> achieving their needs. Language evolution should also be in compass with
> these new needs, and that's where I think PHP is struggling to keep up,
> for a variety of reasons.
Please do not forget that PHP was designed as beginner's language with
low barrier of entry. Making EJB out of it goes contrary to that, and
introducing features that would serve 0.01% of very complex cases while
complicating the language model for 100% of the users does not seem to
be worth it to me.
Especially if in those 0.01% cases I don't see a lot of real
improvement, but rather a "good feeling" that you can write "really
strict" code even though there's nothing better in that code than if it
were written without all the complexities.
> Being a constant contributor to open source and many libraries that now
> drives most of what people consider as "Modern PHP", I constantly face
> problems with the language itself being unable to achieve my needs. When
> I say my needs, I mean needs of heavily used libraries, with hundreds or
> thousands of other projects depending on them.
I would like to see what is impossible to implement in PHP without
private classes. I mean useful functionality, not circular reference "I
need private classes because I need classes to be private" of course.
What application can not be implemented without it? I have very hard
time imagining such application.
> working on a bug fix. If this class was never intended to be used
> outside of the scope of this project, how can restrict its usage by
> outside packages? Feel free to provide me answers or alternatives, and
> I'll happily agree with you that private classes is a bad idea.
You can document it and talk to your users. PHP is an open-source
software, with open code and open development model. If the code is out
there, people can use it. If they use it contrary to maintainer's
recommendations, they take a risk. But that's it.
Obviously, if the users used that class, it provided some functionality
they needed. If you removed that functionality, now the users don't have
it. It's not a problem that can be solved with access modifiers. In
fact, if there were access modifiers preventing users from using
necessary functionality, they'd probably fork the code and removed them.
Also, working around namespace-private classes is very easy - just put
your own code in the same namespace. Works for Java too IIRC.
> Namespace implementation is now the blocker of many new features that
> could help solve complex problems that you don't see on a daily basis,
> but library developers face daily. For example: allowing to assess
> imported/used structs when declaring a class would drastically reduce
> the complexity of Doctrine's annotation parser, which required
> token_get_all and several logic to check for what you're including. Is
This is a whole bunch of things here. First of all, annotation parser is
a solution for a problem that shouldn't exist - namely, lack of
annotation support. Second, the number of code that needs to pre-process
other PHP code in PHP is minuscule compared to code doing other things.
Third, I'm not even sure what "allowing to assess imported/used structs
when declaring a class" means - why Doctrine annotation parser declares
a class? What are "structs"? What use has to do with class declaration?
What does Doctrine parser needs to assess, actually? I'm confused.
Fourth, I very much question the assertion that this is the use case
library developers face *daily*. I've developed many libraries in my
career, and I never had to face this particular issue (unless, of
course, in my confusion - see above - I misunderstood what the issue is).
> there a better solution for that too? Well, I'd argue Annotations has a
> market, and people are just blind if they don't wanna see it. Answering
> that implementing Annotations in userland is a bad idea is like telling
> a corporation shouldn't create a product because you don't like it.
Not because *I* don't *like* it - but because this is not the right way
to do it. If you create a car with square wheels, I'd say it won't drive
well not because I hate squares but because I know round wheels drive
better. It's not a question of my personal aesthetic preferences.
> I do have a lengthy, detailed and complex plan of how namespace
> refactoring could be made, what it would enable feature-wise, and how it
Before getting to the question of how, we need to pass the question of
why. And that I think is much more important.
> knowledge. What I don't want to see is blind resistance to language
> evolution and blaming over something as badly designed without
> understanding the circumstances that drove to that implementation decision.
Why do you assume this resistance is "blind"? Please do not fall into
temptation of imagining your opponents as incapable of reason. This
would not get us very far. No, there are very good *reasons* for
opposing complicating the language, copying Java features that make very
little sense in PHP context and overhauling the engine and introducing
massive complications to serve a tiny percentage of use cases, even
those probably produced by lack of a different feature (also lacking
because of rejection of simple solutions in a quest of building
complicated one that serves 0.01% of most complex cases).
--
Stas Malyshev
smalyshev@gmail.com
Lester Caine
10 years ago
On 14/08/16 10:14, Stanislav Malyshev wrote:
>> > I do have a lengthy, detailed and complex plan of how namespace
>> > refactoring could be made, what it would enable feature-wise, and how it
> Before getting to the question of how, we need to pass the question of
> why. And that I think is much more important.
On top of just how useful IS namespace and if it's needed at all?
--
Lester Caine - G8HFL
-----------------------------
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk
Rainbow Digital Media - http://rainbowdigitalmedia.co.uk
Fleshgrinder
10 years ago
On 8/14/2016 11:14 AM, Stanislav Malyshev wrote:
> Hi!
>
>> Today I see 2 sides in PHP Internals. One that truly believes that PHP
>> should adopt more concepts of object orientation, such as Annotations,
>> Generics, Overloading, Class visibility, Collections, Exceptions, etc
>
> Object orientation is a very broad term. You can be very well object
> oriented language and not be Java. In fact, almost all OO languages are
> not Java ;) And a lot of OO languages (most of ones I know) don't have
> Javaesque features like private classes, many also do fine without
> generics.
>
This is actually not accurate:
- PHP 7 has private classes through anonymous/inner classes.
- C++ has private (and protected) classes through nesting and friend
classes.
- C# and VB.NET have private classes and many more modifiers.
- Everything is private by default in Rust.
- Golang (another beginners language) has private and protected (internal).
- Swift has private and more classes.
- Ceylon has the shared annotation.
- Ruby has private classes through some tricks.
- ...
I stop here but you will find more.
On 8/14/2016 11:14 AM, Stanislav Malyshev wrote:
> Please do not forget that PHP was designed as beginner's language with
> low barrier of entry. Making EJB out of it goes contrary to that, and
> introducing features that would serve 0.01% of very complex cases while
> complicating the language model for 100% of the users does not seem to
> be worth it to me.
> Especially if in those 0.01% cases I don't see a lot of real
> improvement, but rather a "good feeling" that you can write "really
> strict" code even though there's nothing better in that code than if it
> were written without all the complexities.
>
I know that the numbers are figurative but I think that 80 to 20 might
be more accurate.
On 8/14/2016 11:14 AM, Stanislav Malyshev wrote:
> I would like to see what is impossible to implement in PHP without
> private classes. I mean useful functionality, not circular reference "I
> need private classes because I need classes to be private" of course.
> What application can not be implemented without it? I have very hard
> time imagining such application.
>
Impossible is such a strong word but lets consider for a second the
following example:
> Create an entity and builder. The entity has to be immutable and
> the builder is the only object that is allowed to create that
> entity.
<?php
abstract class Data {
protected $id;
protected $name;
protected function __construct() {}
protected function __clone() {}
}
final class Entity extends Data {
public function getId() {
return $this->id;
}
public function getName() {
return $this->name;
}
}
final class Builder extends Data {
private $entity;
public function build() {
return clone $this->entity;
}
public function setId($id) {
$this->entity->id = $id;
}
public function setName($name) {
$this->entity->name = $name;
}
}
?>
The biggest problem is that we are leaking the abstract base class that
is required to allow the entity and builder access and thus sharing of
properties and methods. This is very bad because that class has no value
for users but its part of our public API. Hence, any change to it
requires special care AND it gives anyone extending that class access to
the properties as well.
Now consider the whole thing with friend classes or private classes and
its super easy to solve.
Another example that is not solvable with friend classes would be the
strategy pattern of hidden strategies as we find it for example in
Symfony's process component.
https://github.com/symfony/process/tree/master/Pipes
The pipes are purely internal to work around differences between PHP on
Unix and Windows. They are not meant for consumption through users, they
actually never face the user in any circumstance. Their only purpose is
it to abstract these differences between operating systems away from the
user to make it easier for them.
On 8/14/2016 11:14 AM, Stanislav Malyshev wrote:
>> working on a bug fix. If this class was never intended to be used
>> outside of the scope of this project, how can restrict its usage by
>> outside packages? Feel free to provide me answers or alternatives, and
>> I'll happily agree with you that private classes is a bad idea.
>
> You can document it and talk to your users. PHP is an open-source
> software, with open code and open development model. If the code is out
> there, people can use it. If they use it contrary to maintainer's
> recommendations, they take a risk. But that's it.
>
At trivago we have hundreds of applications, packages, libraries and
developers who work in a very fast moving environment. Reading
documentation is not something most people spend time with. This is sad
but nothing anyone can solve. Hence, they use stuff that they are not
supposed to use. Having more control over access would greatly help to
improve control over how other developers use a library.
On 8/14/2016 11:14 AM, Stanislav Malyshev wrote:
> Obviously, if the users used that class, it provided some functionality
> they needed. If you removed that functionality, now the users don't have
> it. It's not a problem that can be solved with access modifiers. In
> fact, if there were access modifiers preventing users from using
> necessary functionality, they'd probably fork the code and removed them.
>
Yes, and that is exactly what they should do. It would be best if they
contribute that code back and let the maintainers know about their use
case. Either its completely wrong what they do and one can help them to
do it the right way or the library really needs changing.
On 8/14/2016 11:14 AM, Stanislav Malyshev wrote:
> Also, working around namespace-private classes is very easy - just put
> your own code in the same namespace. Works for Java too IIRC.
>
It's also easy to work around property and method access modifiers.
However, that is no argument to remove them now.
--
Richard "Fleshgrinder" Fussenegger
Stas Malyshev
10 years ago
Hi!
> - PHP 7 has private classes through anonymous/inner classes.
It's not exactly the same, and I suspect the same is true for Ruby. It's
true that anonymous classes can not be instantiated by other code. But
that is not what we were discussing here. This particular effect is
somewhat similar, the purpose of the feature is different.
Of course, some languages have them. Others don't. The point is saying
"it's not OO unless it implements $favorite_feature_of_mine" is not very
meaningful, at least at this point.
> abstract class Data {
> protected $id;
> protected $name;
> protected function __construct() {}
> protected function __clone() {}
> }
>
> final class Entity extends Data {
> public function getId() {
> return $this->id;
> }
> public function getName() {
> return $this->name;
> }
> }
>
> final class Builder extends Data {
> private $entity;
> public function build() {
> return clone $this->entity;
> }
> public function setId($id) {
> $this->entity->id = $id;
> }
> public function setName($name) {
> $this->entity->name = $name;
> }
> }
>
> ?>
I'm afraid I don't understand this code. Why both entity and builder are
Data? It looks like combining factory object with actual data object.
This is why __construct/__clone exists - so the object could take care
of the details and the factory didn't have to know about it in details.
I presume you want to say that Data object should be private class. I
think - at least from my partial understanding of the code - that it
shouldn't exist at all, especially given you don't want to expose it to
the user.
> Another example that is not solvable with friend classes would be the
> strategy pattern of hidden strategies as we find it for example in
> Symfony's process component.
>
> https://github.com/symfony/process/tree/master/Pipes
>
> The pipes are purely internal to work around differences between PHP on
> Unix and Windows. They are not meant for consumption through users, they
Sure, they aren't useful for most, but I see no problem in somebody
using them if necessary. Say, you may want to use Unix pipe on Windows
because you are in unix emulation environment, or you have a weird OS
that is not Unix and now Windows and want to reuse parts of these
classes to deal with it. There could be many cases we can't foresee when
we design it for our ideal use case.
> At trivago we have hundreds of applications, packages, libraries and
> developers who work in a very fast moving environment. Reading
> documentation is not something most people spend time with.
I find so many things not right with this statement I don't know where
to start... So I just leave it with a note that having no time to do
things right usually means doing things wrong many times, and *then*
being forced to spend time to do things right because the whole jenga
tower crashes on your head. That is not a criticism of you or your
employer, but rather the result of experience learned the hard way. You
can get away with it from time to time, true, but it's just luck.
> It's also easy to work around property and method access modifiers.
> However, that is no argument to remove them now.
Not very easy. The only way to work around private is to use heavy
machinery like reflection. This is why, btw, private should be used very
carefully - when if you're *sure* nobody will need or is advised to
access that thing directly. Overuse of private, especially for methods,
leads to classes that are not extendable without horrendous amounts of
copypaste.
Also, there's a difference IMO between classes and class' properties.
Class details may be - indeed, need to be - hidden because this is what
the class is, one of it's reasons to exist - a tool to reduce complexity
by hiding irrelevant information and exposing only relevant information
as class' public API. You hide things not because you're afraid evil
users would abuse them - but as a service to the user to not overload
their mental models with irrelevant details and allow to deal with the
class by only considering the important things.
However, if we're talking about collection of classes sharing a
namespace, there's no such thing as "namespace's API", at least not in
common use. There's a concept of library's API, but it's not derived
from classes alone - it's usually what documentation does, while class
API can be yet figured from it's methods, library API is usually
impossible to figure out just looking at the classes - you need the next
level, documentation.
So in my opinion, if you hide some classes from instantiation from
outside namespace (which by itself carries a lot of assumptions, such as
every class that will ever need it will share the same namespace),
reduction in complexity is almost none, since nobody uses "list of
available classes in package/namespace" as a way to work with the
package anyway.
P.S. BTW I still don't know why you need to refactor namespaces to have
namespace-private classes anyway. Isn't it just a matter of the match
between current scope and target class name?
--
Stas Malyshev
smalyshev@gmail.com
Christoph Becker
10 years ago
On 15.08.2016 at 00:11, Stanislav Malyshev wrote:
> Also, there's a difference IMO between classes and class' properties.
> Class details may be - indeed, need to be - hidden because this is what
> the class is, one of it's reasons to exist - a tool to reduce complexity
> by hiding irrelevant information and exposing only relevant information
> as class' public API. You hide things not because you're afraid evil
> users would abuse them - but as a service to the user to not overload
> their mental models with irrelevant details and allow to deal with the
> class by only considering the important things.
If it were only about suppressing unimportant details, documentation
would suffice, e.g. /** @private */ or the leading underscore
convention. I am, however, quite happy to have language support to
catch *inadvertent* accesses.
--
Christoph M. Becker
Guilherme Blanco
10 years ago
Hi Stas,
I'll comment your PS, since I'm the author of the PR.
On Sun, Aug 14, 2016 at 6:11 PM, Stanislav Malyshev <smalyshev@gmail.com>
wrote:
> Hi!
>
> > - PHP 7 has private classes through anonymous/inner classes.
>
> It's not exactly the same, and I suspect the same is true for Ruby. It's
> true that anonymous classes can not be instantiated by other code. But
> that is not what we were discussing here. This particular effect is
> somewhat similar, the purpose of the feature is different.
>
> Of course, some languages have them. Others don't. The point is saying
> "it's not OO unless it implements $favorite_feature_of_mine" is not very
> meaningful, at least at this point.
>
> > abstract class Data {
> > protected $id;
> > protected $name;
> > protected function __construct() {}
> > protected function __clone() {}
> > }
> >
> > final class Entity extends Data {
> > public function getId() {
> > return $this->id;
> > }
> > public function getName() {
> > return $this->name;
> > }
> > }
> >
> > final class Builder extends Data {
> > private $entity;
> > public function build() {
> > return clone $this->entity;
> > }
> > public function setId($id) {
> > $this->entity->id = $id;
> > }
> > public function setName($name) {
> > $this->entity->name = $name;
> > }
> > }
> >
> > ?>
>
> I'm afraid I don't understand this code. Why both entity and builder are
> Data? It looks like combining factory object with actual data object.
> This is why __construct/__clone exists - so the object could take care
> of the details and the factory didn't have to know about it in details.
>
> I presume you want to say that Data object should be private class. I
> think - at least from my partial understanding of the code - that it
> shouldn't exist at all, especially given you don't want to expose it to
> the user.
>
> > Another example that is not solvable with friend classes would be the
> > strategy pattern of hidden strategies as we find it for example in
> > Symfony's process component.
> >
> > https://github.com/symfony/process/tree/master/Pipes
> >
> > The pipes are purely internal to work around differences between PHP on
> > Unix and Windows. They are not meant for consumption through users, they
>
> Sure, they aren't useful for most, but I see no problem in somebody
> using them if necessary. Say, you may want to use Unix pipe on Windows
> because you are in unix emulation environment, or you have a weird OS
> that is not Unix and now Windows and want to reuse parts of these
> classes to deal with it. There could be many cases we can't foresee when
> we design it for our ideal use case.
>
> > At trivago we have hundreds of applications, packages, libraries and
> > developers who work in a very fast moving environment. Reading
> > documentation is not something most people spend time with.
>
> I find so many things not right with this statement I don't know where
> to start... So I just leave it with a note that having no time to do
> things right usually means doing things wrong many times, and *then*
> being forced to spend time to do things right because the whole jenga
> tower crashes on your head. That is not a criticism of you or your
> employer, but rather the result of experience learned the hard way. You
> can get away with it from time to time, true, but it's just luck.
>
> > It's also easy to work around property and method access modifiers.
> > However, that is no argument to remove them now.
>
> Not very easy. The only way to work around private is to use heavy
> machinery like reflection. This is why, btw, private should be used very
> carefully - when if you're *sure* nobody will need or is advised to
> access that thing directly. Overuse of private, especially for methods,
> leads to classes that are not extendable without horrendous amounts of
> copypaste.
>
> Also, there's a difference IMO between classes and class' properties.
> Class details may be - indeed, need to be - hidden because this is what
> the class is, one of it's reasons to exist - a tool to reduce complexity
> by hiding irrelevant information and exposing only relevant information
> as class' public API. You hide things not because you're afraid evil
> users would abuse them - but as a service to the user to not overload
> their mental models with irrelevant details and allow to deal with the
> class by only considering the important things.
>
> However, if we're talking about collection of classes sharing a
> namespace, there's no such thing as "namespace's API", at least not in
> common use. There's a concept of library's API, but it's not derived
> from classes alone - it's usually what documentation does, while class
> API can be yet figured from it's methods, library API is usually
> impossible to figure out just looking at the classes - you need the next
> level, documentation.
> So in my opinion, if you hide some classes from instantiation from
> outside namespace (which by itself carries a lot of assumptions, such as
> every class that will ever need it will share the same namespace),
> reduction in complexity is almost none, since nobody uses "list of
> available classes in package/namespace" as a way to work with the
> package anyway.
>
> P.S. BTW I still don't know why you need to refactor namespaces to have
> namespace-private classes anyway. Isn't it just a matter of the match
> between current scope and target class name?
>
I wanted to come up with a solution before writing an RFC.
The problem that is not yet solved in that PR is the check of instantiation
without being scoped. Example:
<?php
namespace Foo;
$bar = new ThisIsAPrivateClass();
This should work... but this code:
<?php
$bar = new \Foo\ThisIsAPrivateClass();
Should break.
The problem is that opline have no scope context, so at both of them the
scope = NULL.
We could easily add a scope every time a namespace is declared and remove
when ended, but as Nikita highlighted (here:
https://github.com/php/php-src/pull/947#issuecomment-224934615 ), this is
not the correct solution.
I hit a dead end on that PR. Changing opline (or oparray) to support a new
struct _zend_namespace is over my internals knowledge. I'd love if someone
could take over and complete it. I can write the RFC while someone else
finalize the code.
>
>
--
> Stas Malyshev
> smalyshev@gmail.com
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>
--
Guilherme Blanco
Lead Architect at E-Block
Fleshgrinder
10 years ago
On 8/15/2016 12:11 AM, Stanislav Malyshev wrote:
> Hi!
>
>> - PHP 7 has private classes through anonymous/inner classes.
>
> It's not exactly the same, and I suspect the same is true for Ruby. It's
> true that anonymous classes can not be instantiated by other code. But
> that is not what we were discussing here. This particular effect is
> somewhat similar, the purpose of the feature is different.
>
It is part of what we are discussing.
On 8/15/2016 12:11 AM, Stanislav Malyshev wrote:
> Of course, some languages have them. Others don't. The point is saying
> "it's not OO unless it implements $favorite_feature_of_mine" is not very
> meaningful, at least at this point.
>
I never said that I agree with that particular comment since you are
completely right that this is not true. Visibility modifiers do not
require OO and OO does not require visibility modifiers.
On 8/15/2016 12:11 AM, Stanislav Malyshev wrote:
>> abstract class Data {
>> protected $id;
>> protected $name;
>> protected function __construct() {}
>> protected function __clone() {}
>> }
>>
>> final class Entity extends Data {
>> public function getId() {
>> return $this->id;
>> }
>> public function getName() {
>> return $this->name;
>> }
>> }
>>
>> final class Builder extends Data {
>> private $entity;
>> public function build() {
>> return clone $this->entity;
>> }
>> public function setId($id) {
>> $this->entity->id = $id;
>> }
>> public function setName($name) {
>> $this->entity->name = $name;
>> }
>> }
>>
>> ?>
>
> I'm afraid I don't understand this code. Why both entity and builder are
> Data? It looks like combining factory object with actual data object.
> This is why __construct/__clone exists - so the object could take care
> of the details and the factory didn't have to know about it in details.
>
> I presume you want to say that Data object should be private class. I
> think - at least from my partial understanding of the code - that it
> shouldn't exist at all, especially given you don't want to expose it to
> the user.
>
You need to imagine that the entity has more properties of course and
no, the data class should not exist at all since it is only used to
emulate something that is currently simply impossible: allow some class
access to parts of another class while not allowing anyone else access
to it.
Again our goal:
> Create a builder that builds an immutable entity.
A possible solution as I could imagine it:
<?php
// This is the magic that enabled backwards compatibility and changes
// semantics of leaving the visibility modifier out.
(assembly|group|module|package|...) Fleshgrinder\Internals;
namespace Examples\Builder;
public final class Entity {
//const class = 'Fleshgrinder\Internals\Examples\Builder\Entity';
number $id;
string $name;
DateTimeImmutable $created;
DateTimeImmutable $changed;
SomeValueObject1 $some_property_1;
SomeValueObject2 $some_property_2;
// ...
SomeValueObjectn $some_property_n;
function __construct() {}
function __clone() {}
// Getters for everything ...
}
public final class EntityBuilder {
private Entity $entity;
public function __construct() {
$this->entity = new Entity;
}
public function build() {
return clone $this->entity;
}
// Setters for everything ...
// Other useful stuff while building ...
}
?>
Currently the only way to achieve this is through an interface for the
entity and hoping that everyone uses that interface and never finds out
that the class actually offers much more functionality. Or you add a
shit load of arguments to the constructors. Or, my preferred approach,
via the magic __set_state method that is not auto-completed via the IDE.
That approach is still not ideal but one of the best. The problem is
users who find out that that __set_state method exists and they might
start using it. The last resort are hacks ... more about that later.
On 8/15/2016 12:11 AM, Stanislav Malyshev wrote:
>> Another example that is not solvable with friend classes would be the
>> strategy pattern of hidden strategies as we find it for example in
>> Symfony's process component.
>>
>> https://github.com/symfony/process/tree/master/Pipes
>>
>> The pipes are purely internal to work around differences between PHP on
>> Unix and Windows. They are not meant for consumption through users, they
>
> Sure, they aren't useful for most, but I see no problem in somebody
> using them if necessary. Say, you may want to use Unix pipe on Windows
> because you are in unix emulation environment, or you have a weird OS
> that is not Unix and now Windows and want to reuse parts of these
> classes to deal with it. There could be many cases we can't foresee when
> we design it for our ideal use case.
>
The joke is, you cannot do that. Both the Process and the ProcessBuilder
do not allow you to change the Pipes that are in use. Unless you hack it.
https://github.com/symfony/process/blob/master/Process.php
https://github.com/symfony/process/blob/master/ProcessBuilder.php
These are purely internal classes that serve a single purpose.
On 8/15/2016 12:11 AM, Stanislav Malyshev wrote:
>> At trivago we have hundreds of applications, packages, libraries and
>> developers who work in a very fast moving environment. Reading
>> documentation is not something most people spend time with.
>
> I find so many things not right with this statement I don't know where
> to start... So I just leave it with a note that having no time to do
> things right usually means doing things wrong many times, and *then*
> being forced to spend time to do things right because the whole jenga
> tower crashes on your head. That is not a criticism of you or your
> employer, but rather the result of experience learned the hard way. You
> can get away with it from time to time, true, but it's just luck.
>
Thoughtful API design is bad? Pretty much any- and everything in PHP is
a breaking change because many things cannot be restricted. The addition
of visibility modifiers to constants in PHP 7.1 is a blessing because as
of now any change to a constant is directly a breaking change. This
sucks and only helps to increment the MAJOR part of our SemVer version
strings. Of course it also means that we need to touch a bunch of
packages because our composer is restricted to ^1.0 and now needs to be
^2.0.
On 8/15/2016 12:11 AM, Stanislav Malyshev wrote:
>> It's also easy to work around property and method access modifiers.
>> However, that is no argument to remove them now.
>
> Not very easy. The only way to work around private is to use heavy
> machinery like reflection. This is why, btw, private should be used very
> carefully - when if you're *sure* nobody will need or is advised to
> access that thing directly. Overuse of private, especially for methods,
> leads to classes that are not extendable without horrendous amounts of
> copypaste.
>
This is not true, closures are a lightweight work around here.
Nonetheless, heavy usage of private for methods does not make much sense
in any context or programming language but it does for any other kind of
member (we are left with constants and properties).
On 8/15/2016 12:11 AM, Stanislav Malyshev wrote:
> Also, there's a difference IMO between classes and class' properties.
> Class details may be - indeed, need to be - hidden because this is what
> the class is, one of it's reasons to exist - a tool to reduce complexity
> by hiding irrelevant information and exposing only relevant information
> as class' public API. You hide things not because you're afraid evil
> users would abuse them - but as a service to the user to not overload
> their mental models with irrelevant details and allow to deal with the
> class by only considering the important things.
>
Now exchange class with package in the above paragraph and all of this
is equally true.
On 8/15/2016 12:11 AM, Stanislav Malyshev wrote:
> However, if we're talking about collection of classes sharing a
> namespace, there's no such thing as "namespace's API", at least not in
> common use. There's a concept of library's API, but it's not derived
> from classes alone - it's usually what documentation does, while class
> API can be yet figured from it's methods, library API is usually
> impossible to figure out just looking at the classes - you need the next
> level, documentation.
> So in my opinion, if you hide some classes from instantiation from
> outside namespace (which by itself carries a lot of assumptions, such as
> every class that will ever need it will share the same namespace),
> reduction in complexity is almost none, since nobody uses "list of
> available classes in package/namespace" as a way to work with the
> package anyway.
>
A namespace might not have any relevance for you but it has to the PHP
community at large who are using PSR-0/4 and Composer. All of my
namespaces start with Fleshgrinder and all of trivago's namespaces start
with Trivago. The same is true for Symfony, Drupal, Laravel, Zend, ...
so yes, my classes share the same namespace and they will always share
the same namespace unless someone else starts using my namespace.
Also your last paragraph is completely untrue. Every time one interacts
with the IDE auto-completion a listing of all available declarations
that fit the context is given. Hence, if I type `new ` followed by a
Ctrl + Space everything is being listed. The only thing I can do now is
adding a filter by starting to type a few characters.
Your approach of making everything public from the very beginning means
that you add everything to your public API. It also means that you need
to support everything in a backwards compatible manner. What does that
mean for us as library developers?
Traits are the worst since we cannot change anything.
Interfaces cannot be changed in any way other than moving members up
into another interface that we extend but the other interface is not
allowed to add any other members to our interface.
Classes cannot be changed in any way because someone might extend them.
We cannot introduce new members since extending classes might already
have them. We can however introduce new private things and fiddle around
with magic methods to some extend.
Final classes give one more freedom but are a pain for people who like
to rely on mocking and learned what interface explosion means.
It is about API design and not only about class API design. It is a
combination of documentation and modifiers.
On 8/15/2016 12:11 AM, Stanislav Malyshev wrote:
> P.S. BTW I still don't know why you need to refactor namespaces to have
> namespace-private classes anyway. Isn't it just a matter of the match
> between current scope and target class name?
>
This is better explained by nikic and others.
--
Richard "Fleshgrinder" Fussenegger
Guilherme Blanco
10 years ago
Hi Stas,
Answers inline.
On Sun, Aug 14, 2016 at 5:14 AM, Stanislav Malyshev <smalyshev@gmail.com>
wrote:
> Hi!
>
> > Today I see 2 sides in PHP Internals. One that truly believes that PHP
> > should adopt more concepts of object orientation, such as Annotations,
> > Generics, Overloading, Class visibility, Collections, Exceptions, etc
>
> Object orientation is a very broad term. You can be very well object
> oriented language and not be Java. In fact, almost all OO languages are
> not Java ;) And a lot of OO languages (most of ones I know) don't have
> Javaesque features like private classes, many also do fine without
> generics.
>
I agree that we shouldn't copy and paste features as is. However, whenever
a need is found and there are common agreement over the existence of a
feature, we need to understand the decisions over each one and how are they
valid for PHP.
I'll draw 2 simple examples:
1- Generics: They solve a very interesting problem that is highlighted by
the usage of strict types.
It also hardens the implementation code, allowing to write more robust and
reliable libraries that could be used by others.
However, should it be a blind copy of Java generics? Of course no. In the
context of PHP, lower bounds, lower bounded wildcards, unbounded wildcards
and upper bounded wildcards make absolutely no sense.
Now let's suppose people decide to implement polymorphism in PHP and
everyone agrees with it. All of a sudden, lower bounded wildcards would
become a necessity.
>
> > One thing that didn't change is the constant increase of developers
> > writing and rewriting software using the language. Now these developers
> > are also improving their skills, and demanding more flexible ways of
> > achieving their needs. Language evolution should also be in compass with
> > these new needs, and that's where I think PHP is struggling to keep up,
> > for a variety of reasons.
>
> Please do not forget that PHP was designed as beginner's language with
> low barrier of entry. Making EJB out of it goes contrary to that, and
> introducing features that would serve 0.01% of very complex cases while
> complicating the language model for 100% of the users does not seem to
> be worth it to me.
> Especially if in those 0.01% cases I don't see a lot of real
> improvement, but rather a "good feeling" that you can write "really
> strict" code even though there's nothing better in that code than if it
> were written without all the complexities.
>
A realization that needs to be made is that beginners would be using
libraries that requires to make valid restrictions, preventing those
beginners to mess up with code they shouldn't. So even if the use case is
only valid for 0.01% of code producers, it might be valid for 20%+ of
consumers.
>
> > Being a constant contributor to open source and many libraries that now
> > drives most of what people consider as "Modern PHP", I constantly face
> > problems with the language itself being unable to achieve my needs. When
> > I say my needs, I mean needs of heavily used libraries, with hundreds or
> > thousands of other projects depending on them.
>
> I would like to see what is impossible to implement in PHP without
> private classes. I mean useful functionality, not circular reference "I
> need private classes because I need classes to be private" of course.
> What application can not be implemented without it? I have very hard
> time imagining such application.
>
It's not about being possible or impossible. It's about to prevent
unexpected breakages and headaches that sometimes you're not even aware.
It's about defensive programming.
I never knew TopologicalSorter was used by several users until I actually
broke its API. Imagine the frustration of all those developers with tight
deadlines having to deal with a sudden breakage, because the language it
too open to notify them about something shouldn't be used.
Every time someone questions me about the validity of private classes, I
like to exercise replacing the term "private" with "final". Final is part
of PHP language and it also there's non application that is prevented
without it.
Doctrine's code, taking advantage of "final" modifier also allowed Zend
Framework to take the right path in their implementation. We carefully
think about everything we implement there, always making sure defensive
programming is used for every class. Now if it was not for the existence of
"final" modifier, https://mwop.net/blog/2012-06-28-oop-visibility.html
history would be completely different.
The same would happen for "private" classes. It prevents consumers of
taking the wrong approach when developing their apps, other libraries, etc.
>
> > working on a bug fix. If this class was never intended to be used
> > outside of the scope of this project, how can restrict its usage by
> > outside packages? Feel free to provide me answers or alternatives, and
> > I'll happily agree with you that private classes is a bad idea.
>
> You can document it and talk to your users. PHP is an open-source
> software, with open code and open development model. If the code is out
> there, people can use it. If they use it contrary to maintainer's
> recommendations, they take a risk. But that's it.
>
> Obviously, if the users used that class, it provided some functionality
> they needed. If you removed that functionality, now the users don't have
> it. It's not a problem that can be solved with access modifiers. In
> fact, if there were access modifiers preventing users from using
> necessary functionality, they'd probably fork the code and removed them.
>
> Also, working around namespace-private classes is very easy - just put
> your own code in the same namespace. Works for Java too IIRC.
>
That's not about preventing for no reason and it should be bypassed. It's
about somehow telling users that something shouldn't be used at all.
The amount of people that takes the wrong approach is insane, and any kind
of prevention to do it is beneficial.
I managed big teams of development where I work, and controlling code
produced by 40+ developers is not easy.
Countless classes are marked as final and I bet hundreds would be final if
I could.
Handling a company source code is easy through code reviews, and also walk
a few steps to ask why a certain class is final or (hypothetically,
private).
I've walked through several scenarios of peers coming to my desk asking to
remove final because reasons and going out with a much simpler solution
without the need to remove it.
>
> > Namespace implementation is now the blocker of many new features that
> > could help solve complex problems that you don't see on a daily basis,
> > but library developers face daily. For example: allowing to assess
> > imported/used structs when declaring a class would drastically reduce
> > the complexity of Doctrine's annotation parser, which required
> > token_get_all and several logic to check for what you're including. Is
>
> This is a whole bunch of things here. First of all, annotation parser is
> a solution for a problem that shouldn't exist - namely, lack of
> annotation support. Second, the number of code that needs to pre-process
> other PHP code in PHP is minuscule compared to code doing other things.
>
Problem of lack of Annotations support... I've tried several times, and
made sure I was accessible enough to answer every situation or scenario
Doctrine faced and reasons for every decision taken on the library. Still,
people seem to ignore expertise and decide to take their own methods which
only solves (I'll use your numbers now) 0.01% of use cases.
>
> Third, I'm not even sure what "allowing to assess imported/used structs
> when declaring a class" means - why Doctrine annotation parser declares
> a class? What are "structs"? What use has to do with class declaration?
> What does Doctrine parser needs to assess, actually? I'm confused.
>
When I mentioned allowing to assess imported classes, I mean this:
namespace Foo;
use Bar\Woo;
/**
* @Woo\Fuzz()
*/
class Lala {}
Annotation parser needs to understand what was "use"d, so it can properly
refer to FQCN. That way, we need to somehow discover something that is only
available at source (and never at post-compile time) to know that you used
"Bar\Woo" when I'm processing "Lala" class. This is what TokenParser class
does:
https://github.com/doctrine/annotations/blob/master/lib/Doctrine/Common/Annotations/TokenParser.php
By the lack of language support, library developers need to implement
creative ways of solving problems that shouldn't exist.
> Fourth, I very much question the assertion that this is the use case
> library developers face *daily*. I've developed many libraries in my
> career, and I never had to face this particular issue (unless, of
> course, in my confusion - see above - I misunderstood what the issue is).
>
> > there a better solution for that too? Well, I'd argue Annotations has a
> > market, and people are just blind if they don't wanna see it. Answering
> > that implementing Annotations in userland is a bad idea is like telling
> > a corporation shouldn't create a product because you don't like it.
>
> Not because *I* don't *like* it - but because this is not the right way
> to do it. If you create a car with square wheels, I'd say it won't drive
> well not because I hate squares but because I know round wheels drive
> better. It's not a question of my personal aesthetic preferences.
>
> > I do have a lengthy, detailed and complex plan of how namespace
> > refactoring could be made, what it would enable feature-wise, and how it
>
> Before getting to the question of how, we need to pass the question of
> why. And that I think is much more important.
>
> > knowledge. What I don't want to see is blind resistance to language
> > evolution and blaming over something as badly designed without
> > understanding the circumstances that drove to that implementation
> decision.
>
> Why do you assume this resistance is "blind"? Please do not fall into
> temptation of imagining your opponents as incapable of reason. This
> would not get us very far. No, there are very good *reasons* for
> opposing complicating the language, copying Java features that make very
> little sense in PHP context and overhauling the engine and introducing
> massive complications to serve a tiny percentage of use cases, even
> those probably produced by lack of a different feature (also lacking
> because of rejection of simple solutions in a quest of building
> complicated one that serves 0.01% of most complex cases).
>
> --
> Stas Malyshev
> smalyshev@gmail.com
>
--
Guilherme Blanco
Lead Architect at E-Block
Stas Malyshev
10 years ago
Hi!
> A realization that needs to be made is that beginners would be using
> libraries that requires to make valid restrictions, preventing those
> beginners to mess up with code they shouldn't. So even if the use case
> is only valid for 0.01% of code producers, it might be valid for 20%+ of
> consumers.
I don't think PHP needs more bondage-and-discipline. If beginners "mess
up with code they shouldn't" that is usually, in my experience, a sign
of library authors not serving their use case well, and not implementing
functionality that they need. Either because they didn't know, or
because their opinion about user's needs differs from user's opinion
about their needs. In this case, I'm on the user's side. Hacker's ethos
is doing things other people think can't be done.
> It's about defensive programming.
> I never knew TopologicalSorter was used by several users until I
> actually broke its API. Imagine the frustration of all those developers
> with tight deadlines having to deal with a sudden breakage, because the
> language it too open to notify them about something shouldn't be used.
I'd rather say "imagine the frustration of all those developers that
rely on the library authors who are oblivious to their true needs and
still the developers at the mercy of them every day". I agree that can
be frustrating. The solution is not more detachment from the needs of
the users and more fencing out so these pesky users would have
absolutely no way to use my code in ways that I didn't foresee, because
I'm capable of foreseeing 100% of user's needs and if I'm not, it's
their fault. At least, I don't think it is.
> Every time someone questions me about the validity of private classes, I
> like to exercise replacing the term "private" with "final". Final is
> part of PHP language and it also there's non application that is
> prevented without it.
I think most of the usages of final are wrong too, and for the same
reasons. It's a claim you have foreseen all user's needs and they'd
never need anything else. It's usually wrong.
> modifier, https://mwop.net/blog/2012-06-28-oop-visibility.html history
> would be completely different.
There Matthew says pretty much what I say, so I agree:
I feel that frameworks and libraries should use private and final sparingly.
> That's not about preventing for no reason and it should be bypassed.
> It's about somehow telling users that something shouldn't be used at all.
> The amount of people that takes the wrong approach is insane, and any
> kind of prevention to do it is beneficial.
If the amount of people using your library in a wrong way is insane,
maybe something needs to be changed in the library? ;) If you build a
nice slick walkway and people keep walking over the grass 100 meters
aside, it may just be you've built the walkway in a wrong place.
> I managed big teams of development where I work, and controlling code
> produced by 40+ developers is not easy.
> Countless classes are marked as final and I bet hundreds would be final
> if I could.
> Handling a company source code is easy through code reviews, and also
> walk a few steps to ask why a certain class is final or (hypothetically,
> private).
Maybe it makes sense to a company which works on its own product, never
to be used by others, and has every use of the code tightly controlled,
foreseen and signed in triplicate with TPS reports on top. My opinion
though it is more of a burden than a benefit to an open-source
environment built on extending and modifying existing code.
> I've walked through several scenarios of peers coming to my desk asking
> to remove final because reasons and going out with a much simpler
> solution without the need to remove it.
That implies you are always smarter than any user of your classes and
have foreseen their need in advance before they even were aware of them.
It is nice to be a genius, unfortunately, there's only one of you, and
maybe another dozen of similar geniuses in existence (I'm certainly not
one of them, not even close). Others need to deal with situations where
their code is used in situations where they did not foresee 100% of
users' needs and some modification/extension should be made. Betting on
being a prophetical genius is usually a losing bet. Unless, of course,
you are. But I would rather prefer the flexibility of being able to
correct my mistakes and shortsightedness than betting on that. Which
means flexibility beats bondage-and-discipline.
> Problem of lack of Annotations support... I've tried several times, and
> made sure I was accessible enough to answer every situation or scenario
> Doctrine faced and reasons for every decision taken on the library.
> Still, people seem to ignore expertise and decide to take their own
> methods which only solves (I'll use your numbers now) 0.01% of use cases.
Unfortunately I am of opposite opinion - that the Doctrine needs -
important as they are - are 0.01% of all PHP user's needs. And the fact
that we don't have solution we could build on (again, you see the
leitmotif I'm harping on - being flexible instead of trying to be ideal
at the start and freezing there) because that solution won't serve *all*
the needs appears very unfortunate to me.
> When I mentioned allowing to assess imported classes, I mean this:
>
> namespace Foo;
>
> use Bar\Woo;
>
> /**
> * @Woo\Fuzz()
> */
> class Lala {}
>
> Annotation parser needs to understand what was "use"d, so it can
> properly refer to FQCN. That way, we need to somehow discover something
I see what you mean. This seems to be a problem because you are trying
to do compiler things without the support of the compiler. I'd say the
right solution is to let the compiler do these things...
--
Stas Malyshev
smalyshev@gmail.com
Marco Pivetta
10 years ago
Hey Stas
On 15 Aug 2016 00:35, "Stanislav Malyshev" <smalyshev@gmail.com> wrote:
>
> Hi!
>
> > A realization that needs to be made is that beginners would be using
> > libraries that requires to make valid restrictions, preventing those
> > beginners to mess up with code they shouldn't. So even if the use case
> > is only valid for 0.01% of code producers, it might be valid for 20%+ of
> > consumers.
>
> I don't think PHP needs more bondage-and-discipline. If beginners "mess
> up with code they shouldn't" that is usually, in my experience, a sign
> of library authors not serving their use case well, and not implementing
> functionality that they need. Either because they didn't know, or
> because their opinion about user's needs differs from user's opinion
> about their needs. In this case, I'm on the user's side. Hacker's ethos
> is doing things other people think can't be done.
*You* don't think so, but this happened many times also with non-beginners
(what Guilherme was talking about).
Conventions are not sufficient to keep a codebase consistent and safe:
explicit rules do that.
Guilherme Blanco
10 years ago
Hi Stas,
On Sun, Aug 14, 2016 at 6:35 PM, Stanislav Malyshev <smalyshev@gmail.com>
wrote:
> Hi!
>
> > A realization that needs to be made is that beginners would be using
> > libraries that requires to make valid restrictions, preventing those
> > beginners to mess up with code they shouldn't. So even if the use case
> > is only valid for 0.01% of code producers, it might be valid for 20%+ of
> > consumers.
>
> I don't think PHP needs more bondage-and-discipline. If beginners "mess
> up with code they shouldn't" that is usually, in my experience, a sign
> of library authors not serving their use case well, and not implementing
> functionality that they need. Either because they didn't know, or
> because their opinion about user's needs differs from user's opinion
> about their needs. In this case, I'm on the user's side. Hacker's ethos
> is doing things other people think can't be done.
>
It's a valid point of view, indeed.
But thinking from a different perspective, if you are using a tool that
does X, users shouldn't rely on Y to complete their job. Specially if Y is
already on a namespace marked as "Internal" with @private annotation and a
huge capital disclaimer that it was a private class.
It doesn't matter how much documentation you add, without language tooling
that prevents to use X as Y, all you can expect are weird use cases and
breakages, that re either not intended to work or are just not supposed to
be done at all. Like I always say "never underestimate someone's ignorance".
>
> > It's about defensive programming.
> > I never knew TopologicalSorter was used by several users until I
> > actually broke its API. Imagine the frustration of all those developers
> > with tight deadlines having to deal with a sudden breakage, because the
> > language it too open to notify them about something shouldn't be used.
>
> I'd rather say "imagine the frustration of all those developers that
> rely on the library authors who are oblivious to their true needs and
> still the developers at the mercy of them every day". I agree that can
> be frustrating. The solution is not more detachment from the needs of
> the users and more fencing out so these pesky users would have
> absolutely no way to use my code in ways that I didn't foresee, because
> I'm capable of foreseeing 100% of user's needs and if I'm not, it's
> their fault. At least, I don't think it is.
>
Taking from this perspective (and also inline with your initial comment),
what you are suggesting is basically break down common, reusable piece of
functionality on its own package. Yes, that is valid, and I have nothing
against it.
The lack of breaking X into smaller packages (such as decoupling Y) doesn't
mean users should use Y at will and suddenly expect/blame about API breaks.
>
> > Every time someone questions me about the validity of private classes, I
> > like to exercise replacing the term "private" with "final". Final is
> > part of PHP language and it also there's non application that is
> > prevented without it.
>
> I think most of the usages of final are wrong too, and for the same
> reasons. It's a claim you have foreseen all user's needs and they'd
> never need anything else. It's usually wrong.
>
> > modifier, https://mwop.net/blog/2012-06-28-oop-visibility.html history
> > would be completely different.
>
> There Matthew says pretty much what I say, so I agree:
>
> I feel that frameworks and libraries should use private and final
> sparingly.
>
Here you basically agreed with everything I said. I'm not saying every
single class should be private (or final), but you just agreed with Matthew
and I that there are valid use cases for each one.
>
> > That's not about preventing for no reason and it should be bypassed.
> > It's about somehow telling users that something shouldn't be used at all.
> > The amount of people that takes the wrong approach is insane, and any
> > kind of prevention to do it is beneficial.
>
> If the amount of people using your library in a wrong way is insane,
> maybe something needs to be changed in the library? ;) If you build a
> nice slick walkway and people keep walking over the grass 100 meters
> aside, it may just be you've built the walkway in a wrong place.
>
It's always a balance between what you can do in such a small spare time
and benefits you get out of these contributions that you could use to get
your job done.
I could perfectly decouple 10 sub-packages out of ORM, but that only brings
more maintenance burden that will consume more time that I already don't
have.
>
> > I managed big teams of development where I work, and controlling code
> > produced by 40+ developers is not easy.
> > Countless classes are marked as final and I bet hundreds would be final
> > if I could.
> > Handling a company source code is easy through code reviews, and also
> > walk a few steps to ask why a certain class is final or (hypothetically,
> > private).
>
> Maybe it makes sense to a company which works on its own product, never
> to be used by others, and has every use of the code tightly controlled,
> foreseen and signed in triplicate with TPS reports on top. My opinion
> though it is more of a burden than a benefit to an open-source
> environment built on extending and modifying existing code.
>
Not really. I worked as Chief Architect and controlling integration
conflicts consumed most of my time while designing and planning the
platform.
Every feature had critical technical architecture points highlighted and
another series of decisions to be made by developers. When handling large
volumes of traffic, some technology decisions are required in order to keep
business online, and some critical pieces of code were heavily controlled
by key developers.
>
> > I've walked through several scenarios of peers coming to my desk asking
> > to remove final because reasons and going out with a much simpler
> > solution without the need to remove it.
>
> That implies you are always smarter than any user of your classes and
> have foreseen their need in advance before they even were aware of them.
> It is nice to be a genius, unfortunately, there's only one of you, and
> maybe another dozen of similar geniuses in existence (I'm certainly not
> one of them, not even close). Others need to deal with situations where
> their code is used in situations where they did not foresee 100% of
> users' needs and some modification/extension should be made. Betting on
> being a prophetical genius is usually a losing bet. Unless, of course,
> you are. But I would rather prefer the flexibility of being able to
> correct my mistakes and shortsightedness than betting on that. Which
> means flexibility beats bondage-and-discipline.
>
By far I'm not the most genius person I've worked with. I tend to relate
this to scope knowledge and the amount of time invested during strategy,
architecting and planning a solution. If I spend 100h exercising edge cases
out of a solution, I may have covered a substantial amount of potential
problems that may come up by developers who are purely focused on
delivering a small feature.
That doesn't mean I have covered 100% of all edge cases, but surely I've
exercised things out of a solution more than others. If I haven't, I
wouldn't be doing my architect's job.
Anyway, this is already derailing into a completely separate topic. Let's
stick to the main subject, please.
>
> > Problem of lack of Annotations support... I've tried several times, and
> > made sure I was accessible enough to answer every situation or scenario
> > Doctrine faced and reasons for every decision taken on the library.
> > Still, people seem to ignore expertise and decide to take their own
> > methods which only solves (I'll use your numbers now) 0.01% of use cases.
>
> Unfortunately I am of opposite opinion - that the Doctrine needs -
> important as they are - are 0.01% of all PHP user's needs. And the fact
> that we don't have solution we could build on (again, you see the
> leitmotif I'm harping on - being flexible instead of trying to be ideal
> at the start and freezing there) because that solution won't serve *all*
> the needs appears very unfortunate to me.
>
Doctrine needs seems like a 0.01% of use cases, but they are also shared by
Symfony, Zend Framework and many other projects too.
We even downscaled a lot the suggested solution too, but it was ignored
somehow. For example, typed properties would reduce significantly our
implemented solution, because we'd rely on language support to the work for
us.
The only problem I find is that PHP is full of half-baked solutions. People
focus too much on a specific feature and do not consider a holistic
approach. Small example: people found it seriously cool to adopt ?Foo as a
nullable type. However, it complicated a lot the implementation of typed
properties (Joe had to work on a triple variable state to make it work).
Now implementation of typed properties was to blame, but I seriously think
it was the introduction of nullable types the culript.
>
> > When I mentioned allowing to assess imported classes, I mean this:
> >
> > namespace Foo;
> >
> > use Bar\Woo;
> >
> > /**
> > * @Woo\Fuzz()
> > */
> > class Lala {}
> >
> > Annotation parser needs to understand what was "use"d, so it can
> > properly refer to FQCN. That way, we need to somehow discover something
>
> I see what you mean. This seems to be a problem because you are trying
> to do compiler things without the support of the compiler. I'd say the
> right solution is to let the compiler do these things...
>
Yes, yes and yes. When can we get that?
I'll again reinforce that library developers have to come up with creative
ways to handle lack of support in the language. That was just one example.
Instead, the language should offer proper support and let library
developers to come up with creative tools (and now creative ways) to
improve and ease end user's development.
> --
> Stas Malyshev
> smalyshev@gmail.com
>
--
Guilherme Blanco
Lead Architect at E-Block