CoW and reference counting in the STL
in [C++]
|
From: Mathias Gaunard on 28 Apr 2010 04:15 On 27 avr, 23:25, Herb Sutter <herb.sut...(a)gmail.com> wrote: > If allocation fails up front, the program knows that there's not > enough memory to complete an operation, and then the program has the > choice of doing something about it, such as trying to allocate a > smaller buffer size or giving up on only that particular operation, or > at least attempting to clean up some things by unwinding the stack. > But if there's no way to know whether the allocation really worked, > then any attempt to read or write the memory may cause a halt - and > that halt can't be predicted, because it might happen on the first > attempt to use part of the buffer, or on the millionth attempt after > lots of successful operations have used other parts of the buffer. You can install a signal handler that gets called when you attempt to access overcommited memory that cannot be allocated, I suspect SIGSEGV would be the one. So strictly speaking, there is still a way to clean up, albeit not using the C++ error handling model; plus it can happen anywhere, meaning so you can't rely on anything being atomic or nothrow. -- [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ]
From: joe on 28 Apr 2010 16:51 Andre Kaufmann wrote: > > typedef map<string, pair<list<string>, set<string>>> T; > > T list1; > T list2; > > ..... > list2 = list1; > > Copying the complete list or adding elements might fail during copying, if the expansion of one of the elements fails. > It's hard to predict if the available memory if sufficient for a copy / add operation (in this case) due to memory fragmentation etc. > > To make a long story short: > > I have the feeling that think that most C++ applications can't handle insufficient memory conditions predictably in every situation. For huge contiguous memory blocks yes. > Or asking the other way round, is there a huge difference to recover from: > > list2 = list1; > > and > > list1["hello"] = element; I think there is a major difference between the above blowing out memory and: string a = "some big string...."; string b = a; // works b[3] = 'c'; // bam out of memory because cow allocated on me I suppose your mileage may vary, but you can get some pretty obscure looking errors. joe -- [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ]
From: Andre Kaufmann on 29 Apr 2010 04:04 joe wrote: > Andre Kaufmann wrote: > [...] > I think there is a major difference between the above blowing out memory > and: I wanted rather to point out that IMHO 99,9% of all C++ applications, can't recover 100% correctly from memory insufficiency under all conditions. There should be always a line of code, where the developer hasn't thought about insufficient memory and if the program doesn't crash, at least it will have unpredicted behavior or data loss. Stack unwinding helps to recover from a low memory condition, but data may be lost anyways. Commonly most applications don't run into insufficient memory conditions, because the OS and the application will get deadly slow, due to swapping before a malloc call could throw. When the application allocate huge memory blocks, it will be prepared that this allocation might fail. But IMHO that's quite different from running slowly (allocating few bytes) out of memory. > string a = "some big string...."; > string b = a; // works > b[3] = 'c'; // bam out of memory because cow allocated on me I think it depends if C++ memory allocation is involved or not. In standard C++ (not using OS COW mechanisms) this should be always the case. So b[3] should throws an exception, same as b = a; So anyways a C++ exception will be thrown, on assignment as on operator[] call. If malloc doesn't throw (older implementations), then at least the operator[] should be able to recover from malloc returning 0. > I suppose your mileage may vary, but you can get some pretty obscure > looking errors. Yes, depends. If OS COW functions are used you will get unpredicted results. But since it's an OS function, I would assume the OS to prompt an error or shutdown/abort the application. > joe Andre -- [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ]
From: red floyd on 29 Apr 2010 04:04 On Apr 29, 12:51 am, joe <jgr...(a)doubletake.com> wrote: > > string a = "some big string...."; > string b = a; // works > b[3] = 'c'; // bam out of memory because cow allocated on me > This example, right here, is the best argument I have seen against CoW. Thanks. -- [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ]
From: Mathias Gaunard on 29 Apr 2010 20:01
On 29 avr, 20:04, Andre Kaufmann <akfmn...(a)t-online.de> wrote: > There should be always a line of code, where the developer > hasn't thought about insufficient memory and if the program doesn't > crash, at least it will have unpredicted behavior or data loss. All code should be basic exception-safe, so unpredicted behavior shouldn't be possible; whatever happens invariants are maintained and the program remains in a valid state. Note basic exception-safe is guaranteed by the use of RAII. > Stack unwinding helps to recover from a low memory condition, but data > may be lost anyways. Indeed, no data loss is the strong exception-safety guarantee, which can be non-trivial to provide for certain operations, and can cost more than the basic guarantee. > > string a = "some big string...."; > > string b = a; // works > > b[3] = 'c'; // bam out of memory because cow allocated on me > > I think it depends if C++ memory allocation is involved or not. > In standard C++ (not using OS COW mechanisms) this should be always the > case. So b[3] should throws an exception, same as b = a; > > So anyways a C++ exception will be thrown, on assignment as on > operator[] call. If you can't rely on such primitives to be nothrow, it is almost impossible to write exception-safe code with the strong guarantee. > If malloc doesn't throw (older implementations) Malloc never throws. -- [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] |