Smart pointers
-
I'd like to speak about pointers, I am trying to stay neutral and thoughtful, please don't transform this thread into a language or belief war. Pointers have several problems. 1 - they can be freed and they then point to garbage 2 - they can be freed twice 3 - sometime the programmer forget to free some memory 4 - probably more... The most common solution these days seem to be using garbage collectors. Garbage collector fix the issues 1,2, 3 above Unfortunately garbage collectors has come with some new problems: - you can't really release memory precisely when you want to - it has to freeze the computer for some time occasionally to count all its blocks - more... There has been a lot of middle way attempts. I know there are many flavour of smart pointers around, I am not claiming I know any of them and this is why I am writing here. Lets imagine we have a language between C++ and C# In this language : - every allocation requires an owner. The root owner would be the application itself. - free is available and work synchronously - when an object get freed, any child object is freed. - when an object is freed any other pointer to it get nullified This would solve our 3 original problems: 1 - they can be freed and they then point to garbage They would point to null 2 - they can be freed twice They would be freed once and then the pointer being null they would not be freed again 3 - sometime the programmer forget to free some memory When the owner is freed, the object is freed I was thinking of implementing those pointers and trying them on a free project like Apache or something like that. But first, I thought I would submit it to all, there is a good chance that it already exists ... Or perhaps the constraint of having an owner is too constraining. It is hard to say until you try on a real project. Having an owner per object could have other advantages in a multi-threading environment. I haven't finished on this but first, does anyone have come across anything like it?
First, the new C++11 standard does contain smart pointers, and they avoid all of the above problems as well as some that you introduced. Second, C++11 introduced move-semantics, and thereby the concept of passing ownership to another. Your core-premise of a singular ownership directly interferes with that concept. The ANSI commission didn't introduce this on a whim, they had some very good reasons, so why would you want to counter their efforts? Third, this...
Pascal Ganaye wrote:
- when an object is freed any other pointer to it get nullified
is a very bad idea! what are other parts of your application supposed to do when the pointer they rely on suddenly got nullified? How can you guarantee that your whole application isn't left in an undefined state because an operation couldn't be completed? This is an obvious concern in a multi-threaded application, but even single-threaded, it may cause big headaches. Sure, you can create a mutex for a pointer to make sure you can complete your operations, but that also means that the owner may not be able to nullify it's own pointer for an unspecified time. That doesn't quite mix with the conceept of ownership. Fourth, you mention that you know there are smart pointer implementations. Why don't you go have a look at them before implementing a concept of your own? They solve all the problems you listed plus the ones I pointed out above.
-
LOL. :) There are good reasons to use them on occasion. I've used them when I'm keeping multiple tables of objects in memory, that can be indexed and accessed from different points. I've had this come up in multi-threaded program where multi-threads are processing data at different entry points. In this case there is no clear owner of the object, so there's no clear way to determine when an object is out of scope and can be safely deleted.
JackDingler wrote:
In this case there is no clear owner of the object,
Unfortunately, the OP pretty much required a singular ownership. Just one of several reasons why he needs to rethink his premises...
-
In a decent design, smart pointers are needed when there's a chance of an exception being thrown.
I disagree - you don't need smart pointers to deal with exceptions. You just initialize all pointers with 0 and in the catch block delete all that are not. Of course, with smart pointers this is easier, but they are not specifically needed.
-
First, the new C++11 standard does contain smart pointers, and they avoid all of the above problems as well as some that you introduced. Second, C++11 introduced move-semantics, and thereby the concept of passing ownership to another. Your core-premise of a singular ownership directly interferes with that concept. The ANSI commission didn't introduce this on a whim, they had some very good reasons, so why would you want to counter their efforts? Third, this...
Pascal Ganaye wrote:
- when an object is freed any other pointer to it get nullified
is a very bad idea! what are other parts of your application supposed to do when the pointer they rely on suddenly got nullified? How can you guarantee that your whole application isn't left in an undefined state because an operation couldn't be completed? This is an obvious concern in a multi-threaded application, but even single-threaded, it may cause big headaches. Sure, you can create a mutex for a pointer to make sure you can complete your operations, but that also means that the owner may not be able to nullify it's own pointer for an unspecified time. That doesn't quite mix with the conceept of ownership. Fourth, you mention that you know there are smart pointer implementations. Why don't you go have a look at them before implementing a concept of your own? They solve all the problems you listed plus the ones I pointed out above.
Thanks Stefan for you answer. I think I got you on the wrong foot, I don't really suggest that what the C++11 have done is bad. I am only introducing something that is different and I am interested to see where it goes. When I wrote :
Stefan_Lang wrote:when an object is freed any other pointer to it get nullified
I know it is a quite an dangerous concept. In my mind I have this idea, like a little snow ball, I want to roll it and see where it goes.
what are other parts of your application supposed to do when the pointer they rely on suddenly got nullified?
They are not in a worst state than calling an object that has been freed. The . or -> operators can check if the pointer is null or not.
How can you guarantee that your whole application isn't left in an undefined state because an operation couldn't be completed?
I am not sure I follow your thoughts. the free function would basically do:
free(memory)
{
foreach pointer in the entire system who point to memory
*pointer = null
real free(memory);
}If the pointers got reused later by the rest of the application it is bound to crash anyway as you're not supposed to use a block of memory that have been freed. With this mechanism you can check very deterministically whether the block was freed by simply testing the pointer value. Again I am not here to start a religious war, I am not even saying this is the way to go. I am merely asking what if?
-
Thanks Stefan for you answer. I think I got you on the wrong foot, I don't really suggest that what the C++11 have done is bad. I am only introducing something that is different and I am interested to see where it goes. When I wrote :
Stefan_Lang wrote:when an object is freed any other pointer to it get nullified
I know it is a quite an dangerous concept. In my mind I have this idea, like a little snow ball, I want to roll it and see where it goes.
what are other parts of your application supposed to do when the pointer they rely on suddenly got nullified?
They are not in a worst state than calling an object that has been freed. The . or -> operators can check if the pointer is null or not.
How can you guarantee that your whole application isn't left in an undefined state because an operation couldn't be completed?
I am not sure I follow your thoughts. the free function would basically do:
free(memory)
{
foreach pointer in the entire system who point to memory
*pointer = null
real free(memory);
}If the pointers got reused later by the rest of the application it is bound to crash anyway as you're not supposed to use a block of memory that have been freed. With this mechanism you can check very deterministically whether the block was freed by simply testing the pointer value. Again I am not here to start a religious war, I am not even saying this is the way to go. I am merely asking what if?
I did not want to start a religious war either, but you did mention to use this in a multithreaded environment, and that is a very dangerous place to go! That is why my response may have seemed a bit radical. But my concerns are real enough.
Pascal Ganaye wrote:
They are not in a worst state than calling an object that has been freed. The . or -> operators can check if the pointer is null or not.
Ouch! First, your 'owner' has no idea at all what state other objects are in. None! For all you know they may have called a lengthy operation on the object that pointer points to and awaiting a response from somewhere else. Now your 'owner' nullifies all pointers, then the response the aforementioned operation waited for arrives. At this point, the execution continues. However, the object it was called upon has been destroyed, all member variables are undefined, the this pointer is invalid, causing any attempt to call another member function to crash! And you call this 'not in a worst state'? Second, the . operator can not be overloaded! You can overload -> , but apart from preventing an immediate crash, what should it do? And besides, you cannot even prevent a crash: operator -> does (at least) two things: it dereferences the pointer and then uses the resulting address to access the referenced member. If the nullifiying happens just after the dereferencing, but before accessing the member, this access will crash your application! (in truth, a lot more atomic operations happen upon dereferencing, I just picked the important two to make a point; the important thing to note is that C++ operations can - and will - be interrupted in mid-swing by multi-threading. Even a simple operation such as ++i can be interrupted, between reading i and incrementing the value and writing it back to storage) Third, even if you could overload both operators, and in a meaningful way, you would have to do that for every single class! As I said, I'm looking through the multi-threading glasses at this, and what I see isn't pretty. You might be able to fix a few of these concerns, but not all. And some might even apply to single-threaded apps. Again: do look at existing smart pointers. They're good. They're clever. A horde of extremely clever people created them, used them, improved them, and made them both easy to use and fool-proof. And yes, that includes multi-threading. There's really no reason to roll your own, certainly not when you're using a compiler
-
JackDingler wrote:
In this case there is no clear owner of the object,
Unfortunately, the OP pretty much required a singular ownership. Just one of several reasons why he needs to rethink his premises...
It would defintely be problematic for him to implement his approach in a current codebase. As you point out, it requires a that there be a clear definition of ownership. You can't just willy nilly, delete the resource. You won't know when other modules are done with it. You'll end up with attempts to resolve a null pointer, and the exceptions that come with that. So with his approach, you'll have to make sure that every user of the pointer is done with it, before deleting it. If you're doing this, then the smart pointer is unnecessary and just adds overhead.
-
It would defintely be problematic for him to implement his approach in a current codebase. As you point out, it requires a that there be a clear definition of ownership. You can't just willy nilly, delete the resource. You won't know when other modules are done with it. You'll end up with attempts to resolve a null pointer, and the exceptions that come with that. So with his approach, you'll have to make sure that every user of the pointer is done with it, before deleting it. If you're doing this, then the smart pointer is unnecessary and just adds overhead.
JackDingler wrote:
So with his approach, you'll have to make sure that every user of the pointer is done with it, before deleting it.
Agreed, that would work. However, that would require to adhere to certain coding standards - but if you have a strict coding standard, ...
JackDingler wrote:
then the smart pointer is unnecessary
-
I did not want to start a religious war either, but you did mention to use this in a multithreaded environment, and that is a very dangerous place to go! That is why my response may have seemed a bit radical. But my concerns are real enough.
Pascal Ganaye wrote:
They are not in a worst state than calling an object that has been freed. The . or -> operators can check if the pointer is null or not.
Ouch! First, your 'owner' has no idea at all what state other objects are in. None! For all you know they may have called a lengthy operation on the object that pointer points to and awaiting a response from somewhere else. Now your 'owner' nullifies all pointers, then the response the aforementioned operation waited for arrives. At this point, the execution continues. However, the object it was called upon has been destroyed, all member variables are undefined, the this pointer is invalid, causing any attempt to call another member function to crash! And you call this 'not in a worst state'? Second, the . operator can not be overloaded! You can overload -> , but apart from preventing an immediate crash, what should it do? And besides, you cannot even prevent a crash: operator -> does (at least) two things: it dereferences the pointer and then uses the resulting address to access the referenced member. If the nullifiying happens just after the dereferencing, but before accessing the member, this access will crash your application! (in truth, a lot more atomic operations happen upon dereferencing, I just picked the important two to make a point; the important thing to note is that C++ operations can - and will - be interrupted in mid-swing by multi-threading. Even a simple operation such as ++i can be interrupted, between reading i and incrementing the value and writing it back to storage) Third, even if you could overload both operators, and in a meaningful way, you would have to do that for every single class! As I said, I'm looking through the multi-threading glasses at this, and what I see isn't pretty. You might be able to fix a few of these concerns, but not all. And some might even apply to single-threaded apps. Again: do look at existing smart pointers. They're good. They're clever. A horde of extremely clever people created them, used them, improved them, and made them both easy to use and fool-proof. And yes, that includes multi-threading. There's really no reason to roll your own, certainly not when you're using a compiler
I am loving this conversation, you're certainly a lot more knowledgeable in C++ than I am. I am not against smart pointers, as I said I don't know them very well it might well be than one of them or a combination of them covers perfectly what I fuzzily thinking. I am taking your comments into consideration and either give up or come with a new proposition. I don't really want to fight against anyone on this, the idea is to accept some arbitrary and perhaps unorthodox concepts and see if it rolls. Hopefully it is yes. If not we can change our arbitrary and perhaps unorthodox concepts and see if it works. You're right the hypothesis I posted are too wild, I'll try to write something with less scope and see where it goes.
-
I am loving this conversation, you're certainly a lot more knowledgeable in C++ than I am. I am not against smart pointers, as I said I don't know them very well it might well be than one of them or a combination of them covers perfectly what I fuzzily thinking. I am taking your comments into consideration and either give up or come with a new proposition. I don't really want to fight against anyone on this, the idea is to accept some arbitrary and perhaps unorthodox concepts and see if it rolls. Hopefully it is yes. If not we can change our arbitrary and perhaps unorthodox concepts and see if it works. You're right the hypothesis I posted are too wild, I'll try to write something with less scope and see where it goes.
Well I don't 'fight' you, I just want to point out the problems. Consider this: 1. Dealing with pointers by yourself is tricky. But most people learn it soon enough. 2. Writing a class to automate this requires you to not only think of your code, but also the code of other users who will use that class. That is quite hard. Especially when you think of users who fail at 1. 3. Writing such a class to deal with multi-threading issues is mind-boggling! It's not that I judge your abilities as poor, just that you might have set your goals too high. Me, I've tried myself at level 2, and although I thought I did it reasonably well, I know it wasn't quite perfect. I might be willing to experiment with sth at the level of 3, but I don't think I'd offer to write an article about it until after I know it works ... On a sidenote, I did write an Object Pool implementation that hands out special smart pointers to make sure all pool items are released as soon as possible. In fact, the concept of releasing pool items ASAP runs a bit contrary to the idea of a pool, so in truth it's closer to a specialized garbage collector with 'immediate response'. That is about as close to garbage collection I dare go. And I only did it because I realized I could implement each function at a complexity of amortised O(1). Maybe I should put that up as an article, but I first have to fix memory alignment and then think up a new name...
-
This is a good example of dogma. You dont need smart pointers in exception handled code, since you can easilly deallocate an allocated pointer in any of the exception handling cases.
============================== Nothing to say.
Not dogma, just efficiency. The scope where an allocated pointer is allocated to may not be the best scope for handling an exception. Without a smart pointer, you have to cover everything with a try-catch, deallocate in all catch clauses (and you'd better not miss one or you'll be leaking), and then rethrow. So okay, smart pointer isn't needed, it just makes for sturdier, faster, and more concise code.
-
I disagree - you don't need smart pointers to deal with exceptions. You just initialize all pointers with 0 and in the catch block delete all that are not. Of course, with smart pointers this is easier, but they are not specifically needed.
That's why I said they were needed in a "decent design". To not use smart pointers when they are available makes for clunkier, unsafer code.
-
Not dogma, just efficiency. The scope where an allocated pointer is allocated to may not be the best scope for handling an exception. Without a smart pointer, you have to cover everything with a try-catch, deallocate in all catch clauses (and you'd better not miss one or you'll be leaking), and then rethrow. So okay, smart pointer isn't needed, it just makes for sturdier, faster, and more concise code.
-
It is perfectly OK to use normal pointers, provided you deallocate at every return from the function, exceptions included.
============================== Nothing to say.
Yes, agreed. But I wouldn't necessarily call that a decent design, which was the caveat in my original comment.