How to use dynamic-extent declarations...
in [Lisp]
|
Prev: online shopping
Next: python admin abuse complaint
From: Pascal Costanza on 2 Feb 2010 05:47 Hi, I recently submitted a report to the SBCL devel mailing list about what I believed to be a bug in their compiler, only to learn that my understanding of what dynamic-extent declarations do was just wrong. Here is the example: (defun definer (x) (list x)) (declaim (inline caller)) (defun caller (&rest args) (declare (dynamic-extent args) (optimize (speed 3) (debug 0) (safety 0) (compilation-speed 0))) (apply #'definer args)) (defun test () (caller (list 0))) If I run test in SBCL, I get the following output: ((SB-INT:STREAM-ENCODING-ERROR)) ....or worse. The problem that causes this is that the dynamic-extent declaration in 'caller doesn't reach only the list that makes up the &rest args, but also the list created in 'test - the HyperSpec states that dynamic-extent declarations cover everything directly or indirectly reachable that is not 'otherwise accessible', and since the list created in 'test is freshly consed, it is indeed not 'otherwise accessible' at the time 'caller is invoked. This screws up my mental model of the dynamic-extent declaration that I had, and I currently have problems coming up with a better mental model of when and how to use it. At the moment it seems to me that dynamic-extent declarations can be too 'aggressive' to the extent that I don't know what guarantees I can and cannot give to users of my libraries. Does this mean that I cannot use dynamic-extent at all? Or only deep down in the call chain of functions that are not exported to the outside? When and how do other people use dynamic-extent? What's a good rule of thumb here? Help! :) Pascal -- My website: http://p-cos.net Common Lisp Document Repository: http://cdr.eurolisp.org Closer to MOP & ContextL: http://common-lisp.net/project/closer/
From: Frode V. Fjeld on 2 Feb 2010 06:35 Pascal Costanza <pc(a)p-cos.net> writes: > [...] The problem that causes this is that the dynamic-extent > declaration in 'caller doesn't reach only the list that makes up the > &rest args, but also the list created in 'test - the HyperSpec states > that dynamic-extent declarations cover everything directly or > indirectly reachable that is not 'otherwise accessible', and since the > list created in 'test is freshly consed, it is indeed not 'otherwise > accessible' at the time 'caller is invoked. FWIW the sbcl behavior as you describe it I find unreasonable. I would say that the fresh list is in fact "otherwise accessible" as the returned value of the function call. The list is clearly "accessible" as (presumably) it's this access that causes your sbcl to crash, and it's "otherwise", i.e. not in any sense by going through the binding that was declared dynamic-extent. -- Frode V. Fjeld
From: Pascal J. Bourguignon on 2 Feb 2010 08:56 Pascal Costanza <pc(a)p-cos.net> writes: > Hi, > > I recently submitted a report to the SBCL devel mailing list about > what I believed to be a bug in their compiler, only to learn that my > understanding of what dynamic-extent declarations do was just wrong. > > Here is the example: > > (defun definer (x) > (list x)) > > (declaim (inline caller)) > > (defun caller (&rest args) > (declare (dynamic-extent args) > (optimize (speed 3) (debug 0) (safety 0) > (compilation-speed 0))) > (apply #'definer args)) > > (defun test () > (caller (list 0))) > > If I run test in SBCL, I get the following output: > > ((SB-INT:STREAM-ENCODING-ERROR)) > > ...or worse. The problem that causes this is that the dynamic-extent > declaration in 'caller doesn't reach only the list that makes up the > &rest args, but also the list created in 'test - the HyperSpec states > that dynamic-extent declarations cover everything directly or > indirectly reachable that is not 'otherwise accessible', and since the > list created in 'test is freshly consed, it is indeed not 'otherwise > accessible' at the time 'caller is invoked. > > This screws up my mental model of the dynamic-extent declaration that > I had, and I currently have problems coming up with a better mental > model of when and how to use it. At the moment it seems to me that > dynamic-extent declarations can be too 'aggressive' to the extent that > I don't know what guarantees I can and cannot give to users of my > libraries. > > Does this mean that I cannot use dynamic-extent at all? Or only deep > down in the call chain of functions that are not exported to the > outside? When and how do other people use dynamic-extent? What's a > good rule of thumb here? > > Help! :) My understand of this declaration is that it means "stack-allocated". The caveat, is while the declaration is done in the callee, it applies to the call sites, where the arguments are allocated. This declaration allows the allocation of these arguments onto the stack (or other space automatically deallocated on function return), bypassing the normal heap and garbage collector. -- __Pascal Bourguignon__
From: Pascal Costanza on 2 Feb 2010 09:00 On 02/02/2010 14:56, Pascal J. Bourguignon wrote: > Pascal Costanza<pc(a)p-cos.net> writes: > >> Hi, >> >> I recently submitted a report to the SBCL devel mailing list about >> what I believed to be a bug in their compiler, only to learn that my >> understanding of what dynamic-extent declarations do was just wrong. >> >> Here is the example: >> >> (defun definer (x) >> (list x)) >> >> (declaim (inline caller)) >> >> (defun caller (&rest args) >> (declare (dynamic-extent args) >> (optimize (speed 3) (debug 0) (safety 0) >> (compilation-speed 0))) >> (apply #'definer args)) >> >> (defun test () >> (caller (list 0))) >> >> If I run test in SBCL, I get the following output: >> >> ((SB-INT:STREAM-ENCODING-ERROR)) >> >> ...or worse. The problem that causes this is that the dynamic-extent >> declaration in 'caller doesn't reach only the list that makes up the >> &rest args, but also the list created in 'test - the HyperSpec states >> that dynamic-extent declarations cover everything directly or >> indirectly reachable that is not 'otherwise accessible', and since the >> list created in 'test is freshly consed, it is indeed not 'otherwise >> accessible' at the time 'caller is invoked. >> >> This screws up my mental model of the dynamic-extent declaration that >> I had, and I currently have problems coming up with a better mental >> model of when and how to use it. At the moment it seems to me that >> dynamic-extent declarations can be too 'aggressive' to the extent that >> I don't know what guarantees I can and cannot give to users of my >> libraries. >> >> Does this mean that I cannot use dynamic-extent at all? Or only deep >> down in the call chain of functions that are not exported to the >> outside? When and how do other people use dynamic-extent? What's a >> good rule of thumb here? >> >> Help! :) > > My understand of this declaration is that it means "stack-allocated". > > The caveat, is while the declaration is done in the callee, it applies > to the call sites, where the arguments are allocated. > > This declaration allows the allocation of these arguments onto the > stack (or other space automatically deallocated on function return), > bypassing the normal heap and garbage collector. Sure, that is understood. But how are you supposed to use that safely? Do I add documentation to my APIs stating that "don't allocate fresh conses directly in argument lists when calling my functions, otherwise you may get unpredictable results"? That doesn't sound reasonable. There should be a 'shallow dynamic-extent' declaration that doesn't reach out for objects allocated at the call site... Pascal -- My website: http://p-cos.net Common Lisp Document Repository: http://cdr.eurolisp.org Closer to MOP & ContextL: http://common-lisp.net/project/closer/
From: Pascal J. Bourguignon on 2 Feb 2010 09:24
Pascal Costanza <pc(a)p-cos.net> writes: > Sure, that is understood. But how are you supposed to use that safely? > > Do I add documentation to my APIs stating that "don't allocate fresh > conses directly in argument lists when calling my functions, otherwise > you may get unpredictable results"? That doesn't sound reasonable. > > There should be a 'shallow dynamic-extent' declaration that doesn't > reach out for objects allocated at the call site... Basically, you don't keep any reference to the arguments or any object reachable from them, including returning any of them. The data declared dynamic-extend, including all data that is reachable from it, must be considered purely kind of 'input' parameters, and should not leak out of the function (the dynamic scope actually). (defvar *example* nil) (defun f (arg) (declare (dynamic-extend arg)) (setf *example* (car arg)) ; bad (print (car arg)) ; ok (if (zerop (random 2)) (return arg) ; bad (return (reduce (function +) arg)))) ; ok The first bad is bad even for numbers because they could be bignums, allocated on the stack: (f (list (the bignum 12345678901234567890) 2 3)) -- __Pascal Bourguignon__ |