Order of class resolution with namespaces and autoload

php.internals

Chuck Hagenbuch

18 years ago
Hi folks- I've hit upon something with namespaces and autoloading that I think is a little out of order. If you take the attached 3 files, put them in the same directory, and run demo.php, you will see this: Fatal error: Uncaught exception 'Exception' in /Users/chuck/Desktop/test.php:7 If you look at the code, test.php defines a Test:: namespace with a Tester class that throws an exception, test_exception.php provides an Exception derivative (Test::Exception) for that namespace, and demo.php includes test.php, sets up an __autoload function to load the exception class, and triggers the exception throwing code. However autoload is never called and the exception is of type ::Exception, because the class resolution order seems to look like this: - does the class exist (_already defined_) in the current namespace? - does the class exist in the global (non-prefixed or "::") namespace? - then try autoload What I think should happen is: - does the class exist in the current namespace? - can the class be autoloaded with the current namespace? - does the class exist in the global scope - global autoload Otherwise if you are re-defining a global class, which I imagine will be quite common with Exception at least, you will have to explicitly load your Exception classes or they'll never be used. The other alternative would be to say that if you want to use a global class inside a namespace you MUST prefix it with ::. I think this is reasonable since if you have a namespaced file it's already not going to work with anything before PHP 5.3, and that would remove the two potential autoload checks. Thanks, -chuck

Greg Beaver

18 years ago
Chuck Hagenbuch wrote:
> Hi folks- > > I've hit upon something with namespaces and autoloading that I think is > a little out of order. If you take the attached 3 files, put them in the > same directory, and run demo.php, you will see this: > > Fatal error: Uncaught exception 'Exception' in > /Users/chuck/Desktop/test.php:7 > > > If you look at the code, test.php defines a Test:: namespace with a > Tester class that throws an exception, test_exception.php provides an > Exception derivative (Test::Exception) for that namespace, and demo.php > includes test.php, sets up an __autoload function to load the exception > class, and triggers the exception throwing code. > > However autoload is never called and the exception is of type > ::Exception, because the class resolution order seems to look like this: > > - does the class exist (_already defined_) in the current namespace? > - does the class exist in the global (non-prefixed or "::") namespace? > - then try autoload > > What I think should happen is: > > - does the class exist in the current namespace? > - can the class be autoloaded with the current namespace? > - does the class exist in the global scope > - global autoload
Hi Chuck, Are you suggesting there be namespace-specific autoload? This sounds interesting to me, but also quite complex, not sure the benefits would outweigh the difficulties for implementation. For instance, PEAR2 namespace would be different from PEAR2::Someclass, and one would need to register the same autoload handler for each possible namespace, otherwise the class resolution code would have to grep through each classname to find sub-namespaces, which is most likely an unacceptable performance hog.
> Otherwise if you are re-defining a global class, which I imagine will be > quite common with Exception at least, you will have to explicitly load > your Exception classes or they'll never be used.
There is another alternative: <?php namespace Test; // explicitly tell PHP that we want Test::Exception to be autoloaded (in essence) import Test::Exception; class Tester { public function fail() { throw new Exception(); } } ?> This calls autoload, as expected, and works wonderfully. It is also quite efficient, but at the expensive of intuitiveness. However, it also has the benefit of explicitly declaring external classes at the top of the file, which is a big plus for maintainability when others are looking at your code. For me, the fact that this coding practice is both more efficient than the alternative you suggest and more maintainable makes the choice quite appealing, even if it isn't immediately intuitive. Greg

Chuck Hagenbuch

18 years ago
Quoting Gregory Beaver <greg@chiaraquartet.net>:
> Are you suggesting there be namespace-specific autoload? This sounds > interesting to me, but also quite complex, not sure the benefits would > outweigh the difficulties for implementation. For instance, PEAR2 > namespace would be different from PEAR2::Someclass, and one would need > to register the same autoload handler for each possible namespace, > otherwise the class resolution code would have to grep through each > classname to find sub-namespaces, which is most likely an unacceptable > performance hog.
Yes, I agree, that sounds awful. :) I was thinking in terms of a duality - say, A::B::C::Exception vs. ::Exception - but when you throw in the possibility that A::B::Exception exists, it sure gets ugly.
>> Otherwise if you are re-defining a global class, which I imagine will be >> quite common with Exception at least, you will have to explicitly load >> your Exception classes or they'll never be used. > > There is another alternative: > > <?php > namespace Test; > // explicitly tell PHP that we want Test::Exception to be autoloaded (in > essence) > import Test::Exception; > > class Tester { > public function fail() { > throw new Exception(); > } > } > ?> > > This calls autoload, as expected, and works wonderfully. It is also > quite efficient, but at the expensive of intuitiveness. However, it > also has the benefit of explicitly declaring external classes at the top > of the file, which is a big plus for maintainability when others are > looking at your code.
Yes, this works. And I'll readily admit that I don't have a better idea right now. An observation, though: when combined with autoload this can change import from only affecting a single file to affecting all subsequent files. 1.php <? namespace Test; import Test::Exception; throw new Exception(); ?> 2.php <? namespace Test; throw new Exception(); ?> If I define a Test::Exception class and include those files in any order, both will create Test::Exception objects regardless of the import statement. However, if I set up autoload, then things become order-dependent: - If I include 1.php, then 2.php, both files throw Test::Exception - If I include 2.php, then 1.php, I get an Exception and then a Test::Exception I can live with this, but I wonder if taking a bit of magic away from nested namespaces would be better. What's wrong with: - If you are inside a namespace, all classnames that do not include a namespace separator (::) refer to classes of the current namespace. So, if I want a "global" exception, I use ::Exception. If I say Exception, then I mean my namespace's Exception with no fallback. And if I want another namespace's Exception, I import it (import Other::Exception). -chuck

Greg Beaver

18 years ago
Chuck Hagenbuch wrote:
> So, if I want a "global" exception, I use ::Exception. If I say > Exception, then I mean my namespace's Exception with no fallback. And > if I want another namespace's Exception, I import it (import > Other::Exception).
I (and PEAR) could live with this. I don't have a patch to back this up, so it is sort of up to Dmitry/Stas unless someone else feels like grokking the source and making one. Greg

Stanislav Malyshev

18 years ago
> <?php > namespace Test; > // explicitly tell PHP that we want Test::Exception to be autoloaded (in > essence) > import Test::Exception; > > class Tester { > public function fail() { > throw new Exception(); > } > } > ?>
This is even better than requiring, and makes the intent very clear. I don't think it decreases intuitiveness, on the contrary - from the look of the code it is immediately clear which exception would be used.
-- Stanislav Malyshev, Zend Software Architect stas@zend.com http://www.zend.com/ (408)253-8829 MSN: stas@zend.com

Chuck Hagenbuch

18 years ago
Quoting Stanislav Malyshev <stas@zend.com>:
>> <?php >> namespace Test; >> // explicitly tell PHP that we want Test::Exception to be autoloaded (in >> essence) >> import Test::Exception; >> >> class Tester { >> public function fail() { >> throw new Exception(); >> } >> } >> ?> > > This is even better than requiring, and makes the intent very clear. > I don't think it decreases intuitiveness, on the contrary - from the > look of the code it is immediately clear which exception would be > used.
Except that it makes it unclear what happens in _other_ files, which is even worse. Once you include the file above, any other file in the Test:: namespace that throws an Exception will throw Test::Exception, not Exception, even if it doesn't import Test::Exception. I think it makes much more sense to import classes that are _outside_ the current namespace. Having to import pieces of your own namespace makes namespaces less useful and intuitive. -chuck

Stanislav Malyshev

18 years ago
> Except that it makes it unclear what happens in _other_ files, which is
In other files you specify what happens to other files.
> even worse. Once you include the file above, any other file in the > Test:: namespace that throws an Exception will throw Test::Exception, > not Exception, even if it doesn't import Test::Exception.
I'm afraid I don't understand. Regardless of what you include where, the name resolution for the file is defined by the code and imports in this file. Since that file does not define test::exception, it would not influence resolution of any file it is included in.
> I think it makes much more sense to import classes that are _outside_ > the current namespace. Having to import pieces of your own namespace > makes namespaces less useful and intuitive.
Import is just a way to write names shorter. Thus, you can employ it to achieve different goals.
-- Stanislav Malyshev, Zend Software Architect stas@zend.com http://www.zend.com/ (408)253-8829 MSN: stas@zend.com

Chuck Hagenbuch

18 years ago
Quoting Stanislav Malyshev <stas@zend.com>:
>> Except that it makes it unclear what happens in _other_ files, which is > > In other files you specify what happens to other files. > >> even worse. Once you include the file above, any other file in the >> Test:: namespace that throws an Exception will throw >> Test::Exception, not Exception, even if it doesn't import >> Test::Exception. > > I'm afraid I don't understand. Regardless of what you include where, > the name resolution for the file is defined by the code and imports > in this file. Since that file does not define test::exception, it > would not influence resolution of any file it is included in.
This isn't true with the current implementation. Consider these four files: test_exception.php: ------------------- <?php namespace Test; class Exception extends ::Exception { } 1.php: ------ <?php namespace Test; import Test::Exception; throw new Exception(); 2.php: ------ <?php namespace Test; throw new Exception(); And finally the main file, say parent.php: ------------------------------------------ <?php function __autoload($class) { include './test_exception.php'; } try { include '2.php'; } catch (Exception $e) { echo get_class($e) . "\n"; } try { include '1.php'; } catch (Exception $e) { echo get_class($e) . "\n"; } If I run parent.php as written above, I see this: Exception Test::Exception ... which is correct as far as it goes, there are no side effects. However, if I flip the include order, including 1.php before 2.php, then I get this: Test::Exception Test::Exception This is the side effect I'm talking about. Because 1.php causes test_exception.php to be loaded, defining Test::Exception, it doesn't matter that 2.php doesn't import Test::Exception - I'm in the Test:: namespace, and Test::Exception exists, so it's used.
>> I think it makes much more sense to import classes that are >> _outside_ the current namespace. Having to import pieces of your >> own namespace makes namespaces less useful and intuitive. > > Import is just a way to write names shorter. Thus, you can employ it > to achieve different goals.
This is true. The goal I am looking at is making sure that when I use a namespace I have full control over the things inside my namespace. The way things work right now I either need to list every single class in my namespace in every file of that namespace - which is going to be a bit burdensome, and seems like a discouragement from using namespaces - or I am at the mercy of PHP adding a new class that might suddenly take the place of one of my namespace classes. Or presumably someone who used my code could include a different librry that suddenly overrode one of my classes with a global class. I understand the argument that namespaces should require minimal change to other code, but I think that PHP should encourage people to think a little bit when putting "namespace Foo" at the top of a file. That should really mean, in my opinion, that everything in that file is relative to that namespace by default. It's not hard to then import the global bit you want. Think about how variable scope works in PHP. The way I am proposing namespaces to work, they would match scope. You would have: $GLOBALS => :: - if you want to reference something from global scope without aliasing it, you can reference it explicitly global $variable => this is like import, but import lets you alias the name as well, for using things from global scope but making them look like they are local Everything else is in the local scope. I don't have a grand theory of why variable scope and class scope should match, but it seems reasonable. The main thing is that I think that the sort of people who are going to want to use namespaces are going to want namespaces to be as self-contained as possible, and that means explicitly importing external bits, not internal bits. Sincerely, -chuck

Stanislav Malyshev

18 years ago
> ... which is correct as far as it goes, there are no side effects. > However, if I flip the include order, including 1.php before 2.php, then > I get this: > > Test::Exception > Test::Exception
Of course, since once Test::Exception is defined, Exception in namespace Test refers to it. It does not influence imports though.
> namespace I have full control over the things inside my namespace. The > way things work right now I either need to list every single class in my > namespace in every file of that namespace - which is going to be a bit
Of course not - the problem even exists only for those classes that override internal class names, and only in the context where the overriding class is not defined but you still want to use it. If you either explicitly load it or specify full name - it would be OK. And so far we have one such name - Exception, so telling "every single class" is a gross overstatement unless your package does nothing but throwing exceptions :)
> The main thing is that I think that the sort of people who are going to > want to use namespaces are going to want namespaces to be as > self-contained as possible, and that means explicitly importing external > bits, not internal bits.
You can not be entirely self-contained as you are going to rely heavily on internal classes and functions anyway.
-- Stanislav Malyshev, Zend Software Architect stas@zend.com http://www.zend.com/ (408)253-8829 MSN: stas@zend.com

Chuck Hagenbuch

18 years ago
Quoting Stanislav Malyshev <stas@zend.com>:
>> ... which is correct as far as it goes, there are no side effects. >> However, if I flip the include order, including 1.php before 2.php, >> then I get this: >> >> Test::Exception >> Test::Exception > > Of course, since once Test::Exception is defined, Exception in > namespace Test refers to it. It does not influence imports though. > >> namespace I have full control over the things inside my namespace. >> The way things work right now I either need to list every single >> class in my namespace in every file of that namespace - which is >> going to be a bit > > Of course not - the problem even exists only for those classes that > override internal class names, and only in the context where the > overriding class is not defined but you still want to use it. If you > either explicitly load it or specify full name - it would be OK. And > so far we have one such name - Exception, so telling "every single > class" is a gross overstatement unless your package does nothing but > throwing exceptions :)
Right, it's not going to affect most classes most of the time, but my point is that you can't predict which classes will be affected. Exception is just the most obvious example. Looking at the built-in class list, Directory and DateTime jump out as ones that will probably come up a lot. Also looking at the global classes list, it's not that big. I really don't think importing your external dependencies would be a hardship. I know Stas gets a lot of weight here because he can write the code for it. I'd like to know if any other developers have strong feelings either way, because I can write tests or even (scary as it might be for others) look at the engine code to change this behavior if that's the only difference maker. -chuck

Stanislav Malyshev

18 years ago
> Right, it's not going to affect most classes most of the time, but my > point is that you can't predict which classes will be affected.
You can - set of internal classes is very well known, unless you are using some exotic extensions. In any case, I am not 100% happy with this solution, but the only alternative proposed so far (and we did consider it before deciding) is worse both in terms of easiness to use for the code and performance. If you can think of any way to solve this which not includes calling autoload on each instance of internal class used in namespace and requiring to always use :: to refer to internal classes - I'd be happy to hear it.
-- Stanislav Malyshev, Zend Software Architect stas@zend.com http://www.zend.com/ (408)253-8829 MSN: stas@zend.com

Stanislav Malyshev

18 years ago
> ::Exception, because the class resolution order seems to look like this: > > - does the class exist (_already defined_) in the current namespace? > - does the class exist in the global (non-prefixed or "::") namespace? > - then try autoload
That's exactly how it works, with the amendment that only internal classes are considered on step 2.
> What I think should happen is: > > - does the class exist in the current namespace? > - can the class be autoloaded with the current namespace? > - does the class exist in the global scope > - global autoload
If it worked this way, you couldn't easily use global classes and old applications would be much harder to convert to namespaces. We think that using global Exception is much more frequent than defining your own Exception - and in the latter case you have the choice to name it something that is not the same as existing class, while in the latter case you do not. So we decided to make resolution rules so that it makes easier to work with the most frequent case and makes easier to work with existing code.
> The other alternative would be to say that if you want to use a global > class inside a namespace you MUST prefix it with ::. I think this is
We consider it not the best choice. Using of global classes is ubiquitous in current applications, always requiring :: would make it very hard to convert them to namespaced code. We think it would require much more work than just having "require_once" in places when you override the internal classes.
> reasonable since if you have a namespaced file it's already not going to > work with anything before PHP 5.3, and that would remove the two
The problem is not working with PHP before 5.3, the problem is using code that was written before 5.3. We try to make this code as easy as possible to upgrade, and it seems that blocking access to internal classes would not do us much good.
-- Stanislav Malyshev, Zend Software Architect stas@zend.com http://www.zend.com/ (408)253-8829 MSN: stas@zend.com

Chuck Hagenbuch

18 years ago
Quoting Stanislav Malyshev <stas@zend.com>:
> If it worked this way, you couldn't easily use global classes and > old applications would be much harder to convert to namespaces. We > think that using global Exception is much more frequent than > defining your own Exception - and in the latter case you have the > choice to name it something that is not the same as existing class, > while in the latter case you do not.
If you look at the various PHP 5 class libraries or frameworks, pretty much all of them define an Exception for every package - Zend_Db_Exception, Zend_Http_Exception, etc. Calling exception classes something other than Exception is kind of like giving up on the namespace - if I have to name my classes, *inside* their namespace, specially in order to avoid global class names, then that seems very unintuitive.
>> The other alternative would be to say that if you want to use a >> global class inside a namespace you MUST prefix it with ::. I think >> this is > > We consider it not the best choice. Using of global classes is > ubiquitous in current applications, always requiring :: would make > it very hard to convert them to namespaced code. We think it would > require much more work than just having "require_once" in places > when you override the internal classes.
I agree. I think Greg and I have a better solution: if you want to use a global class, import it. You can use :: if you want, but again I will say that we are talking about code _inside_ of a namespace. It would not add much conversion burden to say that if you want to use global classes in a file you are adding a namespace to, then just add imports for those classes. If PHP provided a PHP:: namespace with all global classes in it (as aliases), then you could just do import PHP and be done with it. People who wanted their namespace to be used instead would just not do that. -chuck

Stanislav Malyshev

18 years ago
> Calling exception classes something other than Exception is kind of like > giving up on the namespace - if I have to name my classes, *inside*
Not entirely correct - namespaces needed to reconcile _multiple_ libraries, while in case of Exception you need to stay away only from _one_ set of classes - namely, internal ones. While indeed even that could be inconvenient, the problem is that it is not possible to avoid ambiguity in all cases successfully, so we had to choose how we solve ambiguous cases. We decided to solve it in a way that gives most old code most chance to work, while knowing that some people would find some code less convenient with any code. Out solution allows to explicitly specify your intent in one of these ways: 1. Defining the class 2. Importing the name
> say that we are talking about code _inside_ of a namespace. It would not > add much conversion burden to say that if you want to use global classes > in a file you are adding a namespace to, then just add imports for those > classes. If PHP provided a PHP:: namespace with all global classes in it
I think it would. It is much easier to keep track on your own classes than on system classes create by somebody else. And with exception of Exception (no pun intended :) I think there would be not many classes that would have names coinciding with internal class names.
> (as aliases), then you could just do import PHP and be done with it. > People who wanted their namespace to be used instead would just not do > that.
If you are OK with writing imports, then it shouldn't matter too much which imports you write. Right now you need to write your imports - or explicitly specify the class, or load the class. I would like to find better solution, but completely masking system classes with overloads and allowing only :: to work doesn't seem the best way to me currently. Also it would cause autoload call on each use of system classes even if you never had overridden them at all, which can have significant impact on the performance - exhaustive autoload search can be very expensive.
-- Stanislav Malyshev, Zend Software Architect stas@zend.com http://www.zend.com/ (408)253-8829 MSN: stas@zend.com

Chuck Hagenbuch

18 years ago
Quoting Stanislav Malyshev <stas@zend.com>:
> Not entirely correct - namespaces needed to reconcile _multiple_ > libraries, while in case of Exception you need to stay away only > from _one_ set of classes - namely, internal ones. While indeed even > that could be inconvenient, the problem is that it is not possible > to avoid ambiguity in all cases successfully, so we had to choose > how we solve ambiguous cases. We decided to solve it in a way that > gives most old code most chance to work, while knowing that some > people would find some code less convenient with any code. Out > solution allows to explicitly specify your intent in one of these > ways: > 1. Defining the class > 2. Importing the name
I agree with this approach, I just disagree about which set of names we should require people to import.
>> say that we are talking about code _inside_ of a namespace. It >> would not add much conversion burden to say that if you want to use >> global classes in a file you are adding a namespace to, then just >> add imports for those classes. If PHP provided a PHP:: namespace >> with all global classes in it > > I think it would. It is much easier to keep track on your own > classes than on system classes create by somebody else.
I agree and I think this is an excellent reason that external classes should be imported - that way they are easier to keep track of. Also, below you say:
> If you are OK with writing imports, then it shouldn't matter too > much which imports you write.
I think that goes both ways. I am okay with importing classes outside of my namespace. It doesn't make sense to me to have to import names inside my own namespace. And the fact that changes to what classes PHP provides can change the order of class resolution seems like unnecessary fragility to me. I really think people should think about the way their code is organized when they put it in a namespace. And I really don't think it's a bad thing if you need to list your external dependencies using import.
> And with exception of Exception (no pun intended :) I think there > would be not many classes that would have names coinciding with > internal class names.
Remember Date? (now DateTime)? This is mostly true right now because there was no concept of namespacing and only very well organized class libraries prefixed their classes. When people start moving their code into namespaces and then taking advantage of that to remove the prefixing, I think this will be more common.
> Also it would cause autoload call on each use of system classes even > if you never had overridden them at all, which can have significant > impact on the performance - exhaustive autoload search can be very > expensive.
I am not understanding why the extra autoload for system classes is necessary (I do understand why my initial re-ordering of include rules would cause two autoload calls and I agree that that is an awful idea now). - inside a namespace (Foo::), I try to create a new Exception - if no Foo::Exception class exists, we try to autoload "Foo::Exception". There is no need to autoload just "Exception". If what I meant was to create an ::Exception, then I just do import Exception; inside my namespace and the class exists - no autoload is necessary. Am I missing something? Sincerely, -chuck

Stanislav Malyshev

18 years ago
> my own namespace. And the fact that changes to what classes PHP provides > can change the order of class resolution seems like unnecessary > fragility to me.
I agree that it is not ideal way for all setups, however the alternative is worse - it leads to more problems.
> I am not understanding why the extra autoload for system classes is > necessary (I do understand why my initial re-ordering of include rules > would cause two autoload calls and I agree that that is an awful idea now).
Because if you say "Exception" inside any namespace, we should check if Foo::Exception not exists anywhere, which means full search of all autoloading possibilities. Only after we ensured that no such class exists anywhere, we might use Exception. Since using system classes is much more frequent than overloading them, performance would greatly suffer each time you use system class name inside namespace.
-- Stanislav Malyshev, Zend Software Architect stas@zend.com http://www.zend.com/ (408)253-8829 MSN: stas@zend.com

Chuck Hagenbuch

18 years ago
Quoting Stanislav Malyshev <stas@zend.com>:
> Because if you say "Exception" inside any namespace, we should check > if Foo::Exception not exists anywhere, which means full search of > all autoloading possibilities. Only after we ensured that no such > class exists anywhere, we might use Exception. Since using system > classes is much more frequent than overloading them, performance > would greatly suffer each time you use system class name inside > namespace.
But you shouldn't even look for Exception then. I'm saying that if you want the global Exception class you should import it. -chuck

Stanislav Malyshev

18 years ago
> But you shouldn't even look for Exception then. I'm saying that if you > want the global Exception class you should import it.
That would be double bad - that means you have to go over all your code and add :: to all instances of global classes, even though you never ever intended to override any of them. I don't think it's a good idea.
-- Stanislav Malyshev, Zend Software Architect stas@zend.com http://www.zend.com/ (408)253-8829 MSN: stas@zend.com

Chuck Hagenbuch

18 years ago
Quoting Stanislav Malyshev <stas@zend.com>:
>> But you shouldn't even look for Exception then. I'm saying that if >> you want the global Exception class you should import it. > > That would be double bad - that means you have to go over all your > code and add :: to all instances of global classes, even though you > never ever intended to override any of them. I don't think it's a > good idea.
No, you just have to import them into your namespace, right next to the namespace declaration. Which solves both problems - ease of using global classes and the extra autoload. Honestly I don't know if having a PHP:: namespace with an alias to the builtin classes is feasible, but if it is, then the simple, "put global classes into my namespace" ("register_globals" ;) option is: namespace Test; import PHP; -chuck

Stanislav Malyshev

18 years ago
> builtin classes is feasible, but if it is, then the simple, "put global > classes into my namespace" ("register_globals" ;) option is: > > namespace Test; > import PHP;
"import PHP" is a no-op now. However, I have another idea about it (besides the fix for the import thing that indeed looks like a bug to me) - what if we add some syntax to say "use this namespace" which would use autoload, so you'd say: namespace Test; throw new this::Exception(); and it would mean Test::Exception (including autoloading). Without concentrating on 'this' name (we may have another, better name for that) would it make things better?
-- Stanislav Malyshev, Zend Software Architect stas@zend.com http://www.zend.com/ (408)253-8829 MSN: stas@zend.com