Tcl_InvalidateStringRep(objPtr) causes later crash in TclCreateLiteral
in [TCL]
|
Prev: ANNOUNCE: Gnocl Release 0.9.95
Next: Tcl_InvalidateStringRep(objPtr) causes later crash in TclCreateLiteral
From: stsoe on 7 Apr 2010 18:43 Hi all, I am using tcl8.5.6 and have run into an issue when my extension is calling Tcl_InvalidateStringRep with a shared Tcl_Obj that happens to be stored in Tcl's literal table. When Tcl later looks up in the literal table for an object that matches the string rep of the earlier invalidated object, it can happen that it reaches the object that was earlier processed by Tcl_InvalidateStringRep. This object correctly has bytes==0x0 but the length was not reset and remains the original length, hence matches the length of the requested string literal. A crash occurs when TclCreateLiteral then deferences the null bytes ptr. Shouldn't Tcl_InvalidateStringRep remove the objPtr from the literal table if it is there? Or at the very least set the length to 0 at the same time it deletes and nulls the bytes ptr ? --Soren
From: stsoe on 8 Apr 2010 00:22 On Apr 7, 6:28 pm, miguel sofer <mso...(a)users.sf.net> wrote: > stsoe wrote: > > Hi all, > > > I am using tcl8.5.6 and have run into an issue when my extension is > > calling Tcl_InvalidateStringRep with a shared Tcl_Obj that happens to > > be stored in Tcl's literal table. When Tcl later looks up in the > > literal table for an object that matches the string rep of the earlier > > invalidated object, it can happen that it reaches the object that was > > earlier processed by Tcl_InvalidateStringRep. This object correctly > > has bytes==0x0 but the length was not reset and remains the original > > length, hence matches the length of the requested string literal. A > > crash occurs when TclCreateLiteral then deferences the null bytes ptr. > > > Shouldn't Tcl_InvalidateStringRep remove the objPtr from the literal > > table if it is there? Or at the very least set the length to 0 at the > > same time it deletes and nulls the bytes ptr ? > > Remove from the literal table: no. Depending on which literal table you > mean, this may even be disastrous: bytecodes index into this table. > > Set the length to 0: not really, irrelevant. There are Tcl_Objs with 0 > length anyway (the empty string), so length==0 is not a good signal for > 'invalid string rep'. That is signalled by bytes==NULL. > > The REAL question is: why are you invalidating the string rep of a > shared object? > > The literal code assumes that this is never done: it doesn't make a lot > of sense (or else we're missing something). Remember EIAS and the COW > principle: a shared object is read-only, and any changes (shimmering?) > that you do to it have to preserve the string rep (which carries the > real identity of the object). > > Miguel > > EIAS: Everything Is A String > COW : Copy On Write In my extension a Tcl_Obj wraps native C++ objects (using internalRep). % set x [create_object -name foo] The Tcl_Obj for the 'foo' argument is being 'shimmered' by the command and gets an internal representation for the corresponding native C++ object. The relationship between the C++ object and the Tcl_Obj is recorded by the extension. Some time later, another extension command may invoke C++ code that ends up deleting the C++ object and as part of this Tcl_InvalidateStringRep is called on the corresponding Tcl_Obj. Any following % puts $x will force Tcl to recompute the string representation, which could for example be "null" to indicate the object is no longer valid. The code that calls Tcl_InvalidateStringRep() cannot use Tcl_IsShared() to prevent the call, because the object legally may be shared, for example % set y $x Miguel, your answer makes me realize that I probably shouldn't be shimmering a shared object (the one that Tcl inserted in its literal table) if I later on may have to change its string representation. Preventing this does indeed fix the crash. Thanks for your answer. --Soren
From: Alexandre Ferrieux on 8 Apr 2010 02:54
On Apr 8, 6:22 am, stsoe <st...(a)gonsoe.com> wrote: > > Miguel, your answer makes me realize that I probably shouldn't be > shimmering a shared object (the one that Tcl inserted in its literal > table) if I later on may have to change its string representation. > Preventing this does indeed fix the crash. Warning, shimmering and invalidating the string rep are two different things. It turns out that shimmering on a shared object happens routinely, and does no harm: % namespace path ::tcl::unsupported % set x 13 13 % rep $x value is a pure string with a refcount of 6, object pointer at 0x8374a88, string representation "13". % expr {$x+2} 15 % rep $x value is a int with a refcount of 6, object pointer at 0x8374a88, internal representation 0xd:0x83c5f90, string representation "13". % lindex $x 0 13 % rep $x value is a list with a refcount of 6, object pointer at 0x8374a88, internal representation 0x839e340:(nil), string representation "13". The reason is that, as Miguel points out, COW means copy when you _modify_ the object in the sense of its script-accessible semantics, which is its string rep in our EIAS universe. In pure shimmering (above example) the string rep is not touched, hence the object keeps its identity (and high refcount), never being duplicated in the process. Now touching the string rep is somthing more radical: the power of EIAS is that you can reason entirely at the string level. Touch the string rep and you get a new value, and COW commands to duplicate. Then, we can look again at your problem globally as one of "transparent values vs. handles". EIAS (again) means that true Tcl value are strictly equivalent with their string representation, meaning that any internal state playing a role in their semantics can be _regenerated_ from the string rep alone. Those are "transparent values". On the other hand, "handles" (like file handles given by [open]) are only keys into an external "bag of state" (here the IO subsystem). Those _cannot_ be carelessly refcounted like transparent values without violating EIAS. Hence, to be clean you (1) mustn't touch the string rep and (2) prepare to detect "stale accesses" if someone uses that "handle" again after the C++ object's death. -Alex |