Prev: Call for papers (Deadline Extended): AIPR-10, USA, July 2010
Next: Reading *standard-input* as a byte stream
From: RG on 26 Mar 2010 05:43
In article <4bac0528$0$9897$80265adb(a)spool.cs.wisc.edu>,
Peter Keller <psilord(a)merlin.cs.wisc.edu> wrote:

> RG <rNOSPAMon(a)flownet.com> wrote:
> > In article <4babfc91$0$9897$80265adb(a)spool.cs.wisc.edu>,
> > Peter Keller <psilord(a)merlin.cs.wisc.edu> wrote:
> > Well, threads are an extension to the standard. So anything having to
> > do with threads is necessarily implementation-specific. Some
> > implementations provide this functionality (CCL for example, via the
> > PROCESS-INTERRUPT function), others don't.
>
> [snip]
>
> > The "right way" to solve this is to have the top-level thread keep track
> > of all the threads it spawns, and then kill them all in an
> > unwind-protect form. This isn't foolproof -- you can still lose if, for
> > example, you hit CTRL-C twice. But it's a reasonable first cut.
>
> In my server codes (which are for a tutorial), I have written exactly
> this use of unwind-protect. Maybe what I'll do is leave it like that and
> simply write a couple of paragraphs of text explaining the situation.
> I must admit I don't really know the right thing to do.

The "Right Thing" depends on what you really want to accomplish. There
are no easy answers when threading is involved. Threading is inherently
complicated (and some would say inherently evil -- e.g.
http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf)

> One of the problems I want to avoid are readers of the tutorial believing
> the threading idiom (it has the appearance of an idiom at any rate) I
> wrote is totally valid and then going off and writing code based upon it.
> Of course, I would like this to be exactly the case--that it is valid. :)
> But it seems not and I want to mitigate the effect of my decision in
> the pedagogical explanation of the code.

I would suggest just pointing out some of the problems and saying,
"Correcting these deficiencies is left as an exercise for the reader."

;-)

rg
From: Peter Keller on 26 Mar 2010 12:40
Johan Ur Riise <johan(a)riise-data.no> wrote:
> Peter Keller <psilord(a)merlin.cs.wisc.edu> writes:
>
>> It turns out that it "works"
>
> I wonder what is the difference of ""works"", as opposed to "works"
> here. May be it just looks to good to be true, to you.

The difference lies in this statement from the bordeaux-threads manual:

-------------------------
destroy-thread (thread)

Terminates the thread THREAD, which is an object as returned by MAKE-THREAD.
This should be used with caution: it is implementation-defined whether the
thread runs cleanup forms or releases its locks first.
-------------------------

On SBCL, it seems that the cleanup forms in the thread are called,
but that doesn't have to be true on other implementations. So, I say
""works"" because I don't want to give someone the idea that the construct
I present to them is consistent on all implementations.

The specific style of code I have is:

;; *client* is a thread specific special variable binding to the client socket.

(defun worker-thread ()
(unwind-protect

(loop forever doing something with client)

;; uw-p cleanup
(close *client*)))

I was hoping that the immediate start of the thread would be protected by
the unwind-protect so when I destroy-thread it will do the right thing.

It is becoming increasingly clear that I should just explain that my threading
solution is not correct, but sufficient for understanding the examples and
tinkering with them in a REPL. For production quality work, more attention
should be given to how threads work and that might require implementation
specific knowledge.

Thank you.

-pete


From: RG on 26 Mar 2010 14:25
In article <4bace399$0$9895$80265adb(a)spool.cs.wisc.edu>,
Peter Keller <psilord(a)merlin.cs.wisc.edu> wrote:

> It is becoming increasingly clear that I should just explain that my
> threading
> solution is not correct, but sufficient for understanding the examples and
> tinkering with them in a REPL. For production quality work, more attention
> should be given to how threads work and that might require implementation
> specific knowledge.

It's actually even worse than that. Maintaining invariants in a
threaded environment with interrupts is an open research problem. And
some well informed people have reached the conclusion that it's hopeless
and that threads ought to be abandoned altogether.

rg
From: Johan Ur Riise on 26 Mar 2010 16:16
Peter Keller <psilord(a)merlin.cs.wisc.edu> writes:

> Johan Ur Riise <johan(a)riise-data.no> wrote:
>> Peter Keller <psilord(a)merlin.cs.wisc.edu> writes:
>>
>>> It turns out that it "works"
>>
>> I wonder what is the difference of ""works"", as opposed to "works"
>> here. May be it just looks to good to be true, to you.
>
> The difference lies in this statement from the bordeaux-threads manual:
>
> -------------------------
> destroy-thread (thread)
>
> Terminates the thread THREAD, which is an object as returned by MAKE-THREAD.
> This should be used with caution: it is implementation-defined whether the
> thread runs cleanup forms or releases its locks first.
> -------------------------
>
> On SBCL, it seems that the cleanup forms in the thread are called,
> but that doesn't have to be true on other implementations. So, I say
> ""works"" because I don't want to give someone the idea that the construct
> I present to them is consistent on all implementations.
>
> The specific style of code I have is:
>
> ;; *client* is a thread specific special variable binding to the client socket.
>
> (defun worker-thread ()
> (unwind-protect
>
> (loop forever doing something with client)
>
> ;; uw-p cleanup
> (close *client*)))
>
> I was hoping that the immediate start of the thread would be protected by
> the unwind-protect so when I destroy-thread it will do the right thing.




I see. I think you can not rely on that, because if you succed in
killing the thread, nothing more happens, you could be cut off in the
middle of a setf, for instance.

I think that in general, you should not rely on automatic releasing of
resources. I think each thread should fall off the end of its thread
function (it doesn't really return). You should make sure that sockets
and files are closed, either by calling close or use with-open-file or
something, before you leave them for garbage collection. This is the
same in all languages.

About the variable *client* - most implementations treat the global
special variables as common between all threads, useful for
communication between threads. So after a new connection is accepted,
the old connection is lost, also for its worker thread. The solution
is to define the the thread-function as a lexical closure. For
instance:

(defun make-thread-func (some-socket)
(let ((socket some-socket))
(lambda () (worker-thread socket))
))

(you could also copy it with (let ((client *client*))) as the first
thing to do in the thread, but then you risk that a new connection is
accepted before your thread starts, and still loose)

Then change your function named worker-thread to take a socket
parameter (now the closure is the thread-func) and use that for your
socket calls. You don't need a global variable for the client socket,
that is only confusing. As soon as you have got it (from the accept),
call make-thread-func and give that to make-thread.

(let ((client (accept ...)))
(make-thread (make-thread-func client)))

> It is becoming increasingly clear that I should just explain that my
> threading solution is not correct, but sufficient for understanding
> the examples and tinkering with them in a REPL. For production
> quality work, more attention should be given to how threads work and
> that might require implementation specific knowledge.

You have moved away from lisp and now are more concerned with the
operating system's networking and threading interface, yes.

From: Peter Keller on 26 Mar 2010 17:32
Johan Ur Riise <johan(a)riise-data.no> wrote:
> I think that in general, you should not rely on automatic releasing of
> resources. I think each thread should fall off the end of its thread
> function (it doesn't really return). You should make sure that sockets
> and files are closed, either by calling close or use with-open-file or
> something, before you leave them for garbage collection. This is the
> same in all languages.

This is true, but if the thread is something like an echo server to a client
then until the client sends a pre-agreed upon sentinel value, the thread
will sit there happily copying bytes without any way to stop it. Well, there
is a way with your method before, via inspecting a global *stop* variable,
but even that has peril and it still isn't an active close from the point of
view of the client.

I actively want to clean up all of my non-memory resources, but killing
threads allows for situations where unwind-protect won't function,
thereby voiding the semantics of unwind-protect.

> About the variable *client* - most implementations treat the global
> special variables as common between all threads, useful for
> communication between threads.

That is true. But, in my haste I did not mention that I am using the
bordeaux threads special mechanism of creating thread specific global
bindings with *default-special-bindings*. It is there that I would set
up *client* bound to the client socket before starting the thread. In this
manner is *client* specific to each thread.

> You have moved away from lisp and now are more concerned with the
> operating system's networking and threading interface, yes.

That is very true, I want to be congnizant of how to solve the problem,
but not complicate the example code so much, that it dwarfs the purpose
of the example which is only slightly about threads. :)

Thank you.

-pete

First  |  Prev  |  Next  |  Last
Pages: 1 2 3 4
Prev: Call for papers (Deadline Extended): AIPR-10, USA, July 2010
Next: Reading *standard-input* as a byte stream