destruction of cross-referenced objects

php.internals

Arnold Daniels

19 years ago
Hi, I'm struggling with the problem that cross-referenced objects don't get destroyed. I can off course write a method to break the cross-reference, but that's kind of a pain in the butt, especially if the cross-reference is not strait forward. To solve this I'm thinking of building one of 2 thing: 1.) Write a destroy() function which works much like a close function on a recourse, changing the object to a zval of type 'unknown'. 2.) Specify that a property of an object does not count for the reference count of a zval. 3.) Write code to find cross-references and destroy the objects still. The first solution should be quite easy to implement, but you need to call destroy() to free the object, introducing the whole alloc/free responsibility thing to PHP. The second solution would require some extra keyword within the class, but require no special code within the code using the object. Though I wouldn't have a clue how how to implement this or what would be the consequences. I don't think the third solution is really possible or would at least cause performance problems. I could use some comment on this problem. Or perhaps I'm just looking at it from a complete wrong angle. Thanks, Arnold PS. I don't want to start a 'cross-referencing is wrong' discussion. I use it, because it's useful in some occasions.

Richard Lynch

19 years ago
On Mon, December 4, 2006 7:21 pm, Arnold Daniels wrote:
> I'm struggling with the problem that cross-referenced objects don't > get > destroyed. I can off course write a method to break the > cross-reference, > but that's kind of a pain in the butt, especially if the > cross-reference > is not strait forward.
Seems to me that a simple unset() on the data members in userland coude so PHPs refcount is decremented ought to do it... If a user creates a monstrous data structure and doesn't prune it, how much responsibility can PHP take for that? But maybe I'm just being naive.
> To solve this I'm thinking of building one of 2 thing: > 1.) Write a destroy() function which works much like a close function > on > a recourse, changing the object to a zval of type 'unknown'. > 2.) Specify that a property of an object does not count for the > reference count of a zval.
I don't think this is a Good Idea... Seems like it would be WAY too easy for userland code to end up with a 0 for the refcount and then GC kicks in.
> 3.) Write code to find cross-references and destroy the objects still. > > The first solution should be quite easy to implement, but you need to > call destroy() to free the object, introducing the whole alloc/free > responsibility thing to PHP.
Just use unset() for this, rather than introduce yet another confusing function.
> The second solution would require some extra keyword within the class, > but require no special code within the code using the object. Though I > wouldn't have a clue how how to implement this or what would be the > consequences.
> I don't think the third solution is really possible or would at least > cause performance problems.
Finding a "dead branch" of cyclic structures is basic CS stuff. Add a second temp "refcount" and start walking the graph. Either you end up with something known to be in the GLOBAL scope, or you don't. It can take a long time, depending on the size of the graph, but it's just pointer-chasing and integer arithmetic, so shouldn't be THAT slow.
-- Some people have a "gift" link here. Know what I want? I want you to buy a CD from some starving artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So?

Arnold Daniels

19 years ago
Richard Lynch schreef:
> On Mon, December 4, 2006 7:21 pm, Arnold Daniels wrote: > >> I'm struggling with the problem that cross-referenced objects don't >> get >> destroyed. I can off course write a method to break the >> cross-reference, >> but that's kind of a pain in the butt, especially if the >> cross-reference >> is not strait forward. >> > > Seems to me that a simple unset() on the data members in userland > coude so PHPs refcount is decremented ought to do it... > > If a user creates a monstrous data structure and doesn't prune it, how > much responsibility can PHP take for that? > > But maybe I'm just being naive. > >
I myself run into problems when implementing parent/child relationships within object. I probably use PHP a bit different than the regular user and therefor run into different problems, but with more elaborate frameworks being build, I think more people will run into these kind of problems. Think about a DOM implementation fully in PHP (I'm not building that, but just to give you an idea). Using uset() on each child wouldn't do much good, since each also holds a parent/child relationship with its children and therefor the child is not destroyed, meaning the parent is not destroyed. To overcome this you need implement a method destroy(), which runs through all property to see if it is a DOM object and call the destroy() on it. But now imagine another class which has properties with objects of all kinds of classes, some of which have parent/child relationships. Now you need all these classes to implement some sort of destroyable interface. I don't really like a solution like this. With the destroy function you would still need to walk through the children and call destroy forcing an unset in the destructor of the class, but it's a step up.
>> To solve this I'm thinking of building one of 2 thing: >> 1.) Write a destroy() function which works much like a close function >> on >> a recourse, changing the object to a zval of type 'unknown'. >> 2.) Specify that a property of an object does not count for the >> reference count of a zval. >> > > I don't think this is a Good Idea... > > Seems like it would be WAY too easy for userland code to end up with a > 0 for the refcount and then GC kicks in. > >
Yes, that is what I want. If the parent object is unset, the parent property of the child object should be set to unknown or something. But I guess that the memory is freed and the zval isn't set to unknown and you would just get unexpected results.
>> 3.) Write code to find cross-references and destroy the objects still. >> >> The first solution should be quite easy to implement, but you need to >> call destroy() to free the object, introducing the whole alloc/free >> responsibility thing to PHP. >> > > Just use unset() for this, rather than introduce yet another confusing > function. > > >> The second solution would require some extra keyword within the class, >> but require no special code within the code using the object. Though I >> wouldn't have a clue how how to implement this or what would be the >> consequences. >> > > >> I don't think the third solution is really possible or would at least >> cause performance problems. >> > > Finding a "dead branch" of cyclic structures is basic CS stuff. > > Add a second temp "refcount" and start walking the graph. > > Either you end up with something known to be in the GLOBAL scope, or > you don't. > > It can take a long time, depending on the size of the graph, but it's > just pointer-chasing and integer arithmetic, so shouldn't be THAT > slow. > >
I don't really see this solution. I wouldn't unnecessarily find anything in the GLOBAL scope. When using unset or returning from a function, a check should be made to find if the variable is used within the scope of the call stack, global or statically. To my knowledge (which is small on the PHP internals subject) it is only possible to see how many references to the object there are, not where there references are. Thank you for commenting on my e-mail. I hope you understand where I'm coming from with this problem and understand why unset will not do. I hope that you or perhaps someone else might have some other comments or solutions I haven't yet thought of. Best regards, Arnold

Richard Quadling

19 years ago
In user land, I have an abstract_object like this (http://rquadling.php1h.com/abstract_object.html). I've come from a Delphi environment so some of the code you see is based upon some of the ideas I see in the TObject class. There are some comments, but the basic idea is there. On 05/12/06, Arnold Daniels <info@adaniels.nl> wrote:
> > > Richard Lynch schreef: > > On Mon, December 4, 2006 7:21 pm, Arnold Daniels wrote: > > > >> I'm struggling with the problem that cross-referenced objects don't > >> get > >> destroyed. I can off course write a method to break the > >> cross-reference, > >> but that's kind of a pain in the butt, especially if the > >> cross-reference > >> is not strait forward. > >> > > > > Seems to me that a simple unset() on the data members in userland > > coude so PHPs refcount is decremented ought to do it... > > > > If a user creates a monstrous data structure and doesn't prune it, how > > much responsibility can PHP take for that? > > > > But maybe I'm just being naive. > > > > > I myself run into problems when implementing parent/child relationships > within object. I probably use PHP a bit different than the regular user > and therefor run into different problems, but with more elaborate > frameworks being build, I think more people will run into these kind of > problems. > Think about a DOM implementation fully in PHP (I'm not building that, > but just to give you an idea). Using uset() on each child wouldn't do > much good, since each also holds a parent/child relationship with its > children and therefor the child is not destroyed, meaning the parent is > not destroyed. To overcome this you need implement a method destroy(), > which runs through all property to see if it is a DOM object and call > the destroy() on it. But now imagine another class which has properties > with objects of all kinds of classes, some of which have parent/child > relationships. Now you need all these classes to implement some sort of > destroyable interface. I don't really like a solution like this. With > the destroy function you would still need to walk through the children > and call destroy forcing an unset in the destructor of the class, but > it's a step up. > > >> To solve this I'm thinking of building one of 2 thing: > >> 1.) Write a destroy() function which works much like a close function > >> on > >> a recourse, changing the object to a zval of type 'unknown'. > >> 2.) Specify that a property of an object does not count for the > >> reference count of a zval. > >> > > > > I don't think this is a Good Idea... > > > > Seems like it would be WAY too easy for userland code to end up with a > > 0 for the refcount and then GC kicks in. > > > > > Yes, that is what I want. If the parent object is unset, the parent > property of the child object should be set to unknown or something. But > I guess that the memory is freed and the zval isn't set to unknown and > you would just get unexpected results. > >> 3.) Write code to find cross-references and destroy the objects still. > >> > >> The first solution should be quite easy to implement, but you need to > >> call destroy() to free the object, introducing the whole alloc/free > >> responsibility thing to PHP. > >> > > > > Just use unset() for this, rather than introduce yet another confusing > > function. > > > > > >> The second solution would require some extra keyword within the class, > >> but require no special code within the code using the object. Though I > >> wouldn't have a clue how how to implement this or what would be the > >> consequences. > >> > > > > > >> I don't think the third solution is really possible or would at least > >> cause performance problems. > >> > > > > Finding a "dead branch" of cyclic structures is basic CS stuff. > > > > Add a second temp "refcount" and start walking the graph. > > > > Either you end up with something known to be in the GLOBAL scope, or > > you don't. > > > > It can take a long time, depending on the size of the graph, but it's > > just pointer-chasing and integer arithmetic, so shouldn't be THAT > > slow. > > > > > I don't really see this solution. I wouldn't unnecessarily find anything > in the GLOBAL scope. When using unset or returning from a function, a > check should be made to find if the variable is used within the scope of > the call stack, global or statically. To my knowledge (which is small > on the PHP internals subject) it is only possible to see how many > references to the object there are, not where there references are. > > Thank you for commenting on my e-mail. I hope you understand where I'm > coming from with this problem and understand why unset will not do. I > hope that you or perhaps someone else might have some other comments or > solutions I haven't yet thought of. > > Best regards, > Arnold > >
-- ----- Richard Quadling Zend Certified Engineer : http://zend.com/zce.php?c=ZEND002498&r=213474731 "Standing on the shoulders of some very clever giants!"

Arnold Daniels

19 years ago
Hi, I think having to extend a base class is not a good idea. First of all you deprive yourself of extending a class of a library you haven't written (like something from Zend Framework or PEAR). Second it puts a huge constraint on the code of the users of your library, because any class they write needs to extend your base class. A slightly better solution would be to use an interface. A solution like this forces you to have some kind of framework with general interfaces/classes used in all of your libraries. To my opinion a user shouldn't have to set up an elaborate framework in PHP to prevent memory leaks. I've heard some good comments, but no real good solutions. So I think I should just write the destroy function. I hope it will find its way into PHP or (even better) that someone can find a way to prevent these memory leaks. Best regards, Arnold Richard Quadling schreef:

Richard Quadling

19 years ago
The reason I have this base class is that there is equivalent functionality within the built in object in PHP. No concept of ownership or relationship. In my rudimentary framework, nearly my components are related in some way. VERY little stands alone. Therefore, for me, the base object having ownership capabilities is perfect. The main advantage of having this functionality is that, in theory, you only ever destroy 1 object. All other objects are owned and are therefore destroyed in the correct order. If you notice from my code, the __destruct() method of the base object deals with destroying all owned objects. I am not sure I agree with you about having interfaces. PHP does not allow method inclusion into a class (I cannot build a class from a set of methods in different files). If PHP handled multiple inheritence, then this would be ideal, but as far as I know only a handful of specialist languages support true MI. I think I understand the issues that argue against it and I am not advocating that PHP SHOULD support it, it just would be nice if it did. It would be useful. An alternative way to building a deep class structure is to have a "flattening" script. So, take the deep classes and creat squashed classes via a script. You would have some constraints though. No longer would every class be of type abstract_object as well as constraints on the origin of class constants. Not an ideal solution. The basic reason I have this base class is to enforce the order of destruction. I try very hard to make my classes no more than 3 deep, but ownership goes WAY deep! (I have a particular instance of 23 levels of ownership!). Now trying to destroy all of those objects in the right order would be a nightmare without the base object dealing with it. As for using a third party framework (Zend / PEAR / eZ / Prado / etc), then each would have to be examined to see if it dealt with ownership. I would be truly surprised if it did not. On 06/12/06, Arnold Daniels <info@adaniels.nl> wrote:
> Hi, > > I think having to extend a base class is not a good idea. First of all > you deprive yourself of extending a class of a library you haven't > written (like something from Zend Framework or PEAR). Second it puts a > huge constraint on the code of the users of your library, because any > class they write needs to extend your base class. A slightly better > solution would be to use an interface. > A solution like this forces you to have some kind of framework with > general interfaces/classes used in all of your libraries. To my opinion > a user shouldn't have to set up an elaborate framework in PHP to prevent > memory leaks. > > I've heard some good comments, but no real good solutions. So I think I > should just write the destroy function. I hope it will find its way into > PHP or (even better) that someone can find a way to prevent these memory > leaks. > > Best regards, > Arnold > > > Richard Quadling schreef: > > In user land, I have an abstract_object like this > > (http://rquadling.php1h.com/abstract_object.html). > > > > I've come from a Delphi environment so some of the code you see is > > based upon some of the ideas I see in the TObject class. > > > > There are some comments, but the basic idea is there. > > > > On 05/12/06, Arnold Daniels <info@adaniels.nl> wrote: > >> > >> > >> Richard Lynch schreef: > >> > On Mon, December 4, 2006 7:21 pm, Arnold Daniels wrote: > >> > > >> >> I'm struggling with the problem that cross-referenced objects don't > >> >> get > >> >> destroyed. I can off course write a method to break the > >> >> cross-reference, > >> >> but that's kind of a pain in the butt, especially if the > >> >> cross-reference > >> >> is not strait forward. > >> >> > >> > > >> > Seems to me that a simple unset() on the data members in userland > >> > coude so PHPs refcount is decremented ought to do it... > >> > > >> > If a user creates a monstrous data structure and doesn't prune it, how > >> > much responsibility can PHP take for that? > >> > > >> > But maybe I'm just being naive. > >> > > >> > > >> I myself run into problems when implementing parent/child relationships > >> within object. I probably use PHP a bit different than the regular user > >> and therefor run into different problems, but with more elaborate > >> frameworks being build, I think more people will run into these kind of > >> problems. > >> Think about a DOM implementation fully in PHP (I'm not building that, > >> but just to give you an idea). Using uset() on each child wouldn't do > >> much good, since each also holds a parent/child relationship with its > >> children and therefor the child is not destroyed, meaning the parent is > >> not destroyed. To overcome this you need implement a method destroy(), > >> which runs through all property to see if it is a DOM object and call > >> the destroy() on it. But now imagine another class which has properties > >> with objects of all kinds of classes, some of which have parent/child > >> relationships. Now you need all these classes to implement some sort of > >> destroyable interface. I don't really like a solution like this. With > >> the destroy function you would still need to walk through the children > >> and call destroy forcing an unset in the destructor of the class, but > >> it's a step up. > >> > >> >> To solve this I'm thinking of building one of 2 thing: > >> >> 1.) Write a destroy() function which works much like a close function > >> >> on > >> >> a recourse, changing the object to a zval of type 'unknown'. > >> >> 2.) Specify that a property of an object does not count for the > >> >> reference count of a zval. > >> >> > >> > > >> > I don't think this is a Good Idea... > >> > > >> > Seems like it would be WAY too easy for userland code to end up with a > >> > 0 for the refcount and then GC kicks in. > >> > > >> > > >> Yes, that is what I want. If the parent object is unset, the parent > >> property of the child object should be set to unknown or something. But > >> I guess that the memory is freed and the zval isn't set to unknown and > >> you would just get unexpected results. > >> >> 3.) Write code to find cross-references and destroy the objects > >> still. > >> >> > >> >> The first solution should be quite easy to implement, but you need to > >> >> call destroy() to free the object, introducing the whole alloc/free > >> >> responsibility thing to PHP. > >> >> > >> > > >> > Just use unset() for this, rather than introduce yet another confusing > >> > function. > >> > > >> > > >> >> The second solution would require some extra keyword within the > >> class, > >> >> but require no special code within the code using the object. > >> Though I > >> >> wouldn't have a clue how how to implement this or what would be the > >> >> consequences. > >> >> > >> > > >> > > >> >> I don't think the third solution is really possible or would at least > >> >> cause performance problems. > >> >> > >> > > >> > Finding a "dead branch" of cyclic structures is basic CS stuff. > >> > > >> > Add a second temp "refcount" and start walking the graph. > >> > > >> > Either you end up with something known to be in the GLOBAL scope, or > >> > you don't. > >> > > >> > It can take a long time, depending on the size of the graph, but it's > >> > just pointer-chasing and integer arithmetic, so shouldn't be THAT > >> > slow. > >> > > >> > > >> I don't really see this solution. I wouldn't unnecessarily find anything > >> in the GLOBAL scope. When using unset or returning from a function, a > >> check should be made to find if the variable is used within the scope of > >> the call stack, global or statically. To my knowledge (which is small > >> on the PHP internals subject) it is only possible to see how many > >> references to the object there are, not where there references are. > >> > >> Thank you for commenting on my e-mail. I hope you understand where I'm > >> coming from with this problem and understand why unset will not do. I > >> hope that you or perhaps someone else might have some other comments or > >> solutions I haven't yet thought of. > >> > >> Best regards, > >> Arnold > >> > >> > > > > > >
-- ----- Richard Quadling Zend Certified Engineer : http://zend.com/zce.php?c=ZEND002498&r=213474731 "Standing on the shoulders of some very clever giants!"

Arnold Daniels

19 years ago
Hi, I don't think a discussion on programming methods is something for the PHP internals list. I understand how you do it, but I (and perhaps others) would like to do it differently. I have tried to explain why and I hope you understand. I would like to be able to prevent memory leaks without tricks. It's not there, so I will build it and I hope others will see the usefulness of it and perhaps even put it into PHP. Best regards, Arnold Richard Quadling schreef:

Richard Lynch

19 years ago
On Tue, December 5, 2006 4:45 pm, Arnold Daniels wrote:
> Richard Lynch schreef: >> On Mon, December 4, 2006 7:21 pm, Arnold Daniels wrote: >> >>> I'm struggling with the problem that cross-referenced objects don't >>> get >>> destroyed. I can off course write a method to break the >>> cross-reference, >>> but that's kind of a pain in the butt, especially if the >>> cross-reference >>> is not strait forward. >>> >> >> Seems to me that a simple unset() on the data members in userland >> coude so PHPs refcount is decremented ought to do it... >> >> If a user creates a monstrous data structure and doesn't prune it, >> how >> much responsibility can PHP take for that? >> >> But maybe I'm just being naive. >> >> > I myself run into problems when implementing parent/child > relationships > within object. I probably use PHP a bit different than the regular > user > and therefor run into different problems, but with more elaborate > frameworks being build, I think more people will run into these kind > of > problems. > Think about a DOM implementation fully in PHP (I'm not building that, > but just to give you an idea). Using uset() on each child wouldn't do > much good, since each also holds a parent/child relationship with its > children and therefor the child is not destroyed, meaning the parent > is > not destroyed. To overcome this you need implement a method destroy(), > which runs through all property to see if it is a DOM object and call > the destroy() on it. But now imagine another class which has > properties > with objects of all kinds of classes, some of which have parent/child > relationships. Now you need all these classes to implement some sort > of > destroyable interface. I don't really like a solution like this. With > the destroy function you would still need to walk through the children > and call destroy forcing an unset in the destructor of the class, but > it's a step up.
I could be full of it, as I don't do PHP OOP that much, but it seems to me that if your __dtor (or whatever it is in PHP) walks through the members and unset()s them to not point to the parent/child any more, in a structured way so that you don't chop off your own foot, then you will not create these giant cyclic un-salvagable data structures. If you don't do that, then, yes, PHP will most likely not be able to track your data storage for you.
>> Finding a "dead branch" of cyclic structures is basic CS stuff. >> >> Add a second temp "refcount" and start walking the graph. >> >> Either you end up with something known to be in the GLOBAL scope, or >> you don't. >> >> It can take a long time, depending on the size of the graph, but >> it's >> just pointer-chasing and integer arithmetic, so shouldn't be THAT >> slow. >> >> > I don't really see this solution. I wouldn't unnecessarily find > anything > in the GLOBAL scope. When using unset or returning from a function, a > check should be made to find if the variable is used within the scope > of > the call stack, global or statically. To my knowledge (which is > small > on the PHP internals subject) it is only possible to see how many > references to the object there are, not where there references are.
If it's on the function stack and it's not global nor static, then it should be free'd when the function ends. If it's global, then it's got to be pointed to by SOME data node within $_GLOBALS for you to be able to reach it, no? Hmmm, maybe not, with eval() and variable variables... Except that ultimately, eval() and variable variables can only refer back to some pre-existing variable to chase down the data, and that has to somehow be linked back to *something* in $_GLOBALS, doesn't it?... Let me think on that for awhile...
> Thank you for commenting on my e-mail. I hope you understand where I'm > coming from with this problem and understand why unset will not do. I > hope that you or perhaps someone else might have some other comments > or > solutions I haven't yet thought of.
I truly believe that if you are putting that much faith/reliance in PHP's GC and refcount values, then you've probably got a data structure that needs some more effort in maintenance and needs to clean up after itself, rather than rely on the semi-random timing of PHP's GC. So it's not that you couldn't improve PHP to do it for you -- It's that you'd be attacking the symptom rather than the disease. :-)
-- Some people have a "gift" link here. Know what I want? I want you to buy a CD from some starving artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So?

Ants Aasma

19 years ago
Richard Lynch wrote:
> I could be full of it, as I don't do PHP OOP that much, but it seems > to me that if your __dtor (or whatever it is in PHP) walks through the > members and unset()s them to not point to the parent/child any more, > in a structured way so that you don't chop off your own foot, then you > will not create these giant cyclic un-salvagable data structures. > > If you don't do that, then, yes, PHP will most likely not be able to > track your data storage for you. > > ... snip ... > > I truly believe that if you are putting that much faith/reliance in > PHP's GC and refcount values, then you've probably got a data > structure that needs some more effort in maintenance and needs to > clean up after itself, rather than rely on the semi-random timing of > PHP's GC. > > So it's not that you couldn't improve PHP to do it for you -- It's > that you'd be attacking the symptom rather than the disease. :-)
PHP's GC is not random nor semi-random. It's completely deterministic: when the last reference to a value is deleted, then the value itself is freed and if appropriate the destructor is called. If your datastructures are strictly hierarchical then yes, you can use finalizing methods that free the cyclic references. Unfortunately sometimes that is not the case. Sometimes you want to have references to objects from several places, which pretty much means that to avoid leaks you have to implement your own ref-counting. Only way to avoid that is to use the "random" variety of garbage counting that flags all values reachable via active scopes and deletes the rest. Another thing that's missing from PHP is weak references. If you want to have an identity map that doesn't keep everything ever used in memory, then you need to have references to objects that don't stop the GC from freeing the object. An identity map is a construct that ensures that you have at most one instance of every logical entity. I had to implement an identity map that releases unused objects in a JavaScript application recently. Because JavaScript doesn't have weak references I had to implement my own ref counting and I must say that that was probably the most complex and bug prone thing in the whole 30kLOC application. When you don't care about mem leaks, then you don't need "true" garbage collecting and weak references. This is the case when you use the PHP standard request-response model / everything gets freed when the request ends. But if you need to do batch processing involving complex datastructures and haven't thought memory management completely through, then you're pretty much *screwed* without them. This is precisely the reason why for serious stuff we at my employer prefer Python to PHP. We need to use our business logic in batch processing and manual memory management is just too complex, bug prone and time consuming. PHP doesn't have either weak refs nor true gc, the general consensus among core developers seems to be that it's not needed and we don't have enough resources to develop it ourselves. It's easier to rewrite the useful PHP libraries in Python (and get a more aesthetically pleasing language, ignoring the god awful semantically significant whitespace).
-- Ants Aasma