From: David Schwartz on
On Jan 25, 1:58 am, Jef Driesen <jefdrie...(a)hotmail.com.invalid>
wrote:

> I don't see why that can't work. My app opens the pty and sets the
> TIOCEXCL flag. Now any other process (including a new instance of my
> app) is not allowed to access that pty anymore. That's exactly what I
> want, thus so far no problem. But when my app has done its job, it
> closes the pty again. At that point the pty should become available
> again to other processes, regardless of whether that process is my app
> or another one. There is no need to treat my app as special.

Then just clear TIOCEXCL before closing the descriptor.

If your process crashes and then some process tries to open the pty,
still attached to the same other side, what can the kernel do? Your
last instructions to it were "don't let anyone else touch my pty". You
never revoked those instructions, and there's no way the kernel can
tell an authorized process from an unauthorized one.

If you have some way to tell, then you have to code that way. There is
no way the kernel can "just do it".

DS
From: Jef Driesen on
On 25/01/10 13:04, David Schwartz wrote:
> On Jan 25, 1:58 am, Jef Driesen<jefdrie...(a)hotmail.com.invalid>
> wrote:
>
>> I don't see why that can't work. My app opens the pty and sets the
>> TIOCEXCL flag. Now any other process (including a new instance of my
>> app) is not allowed to access that pty anymore. That's exactly what I
>> want, thus so far no problem. But when my app has done its job, it
>> closes the pty again. At that point the pty should become available
>> again to other processes, regardless of whether that process is my app
>> or another one. There is no need to treat my app as special.
>
> Then just clear TIOCEXCL before closing the descriptor.
>
> If your process crashes and then some process tries to open the pty,
> still attached to the same other side, what can the kernel do? Your
> last instructions to it were "don't let anyone else touch my pty". You
> never revoked those instructions, and there's no way the kernel can
> tell an authorized process from an unauthorized one.
>
> If you have some way to tell, then you have to code that way. There is
> no way the kernel can "just do it".

I'm not expecting the kernel do something magic. It's just that from my
point of view, the pty appears as a character device, just like a real
serial port (or even a regular file). And if I close a serial port (or
regular file) which was opened in exclusive mode, the kernel does unlock
it again. So why would that be any different for pty's? (I know now it's
due to the shared TIOCEXCL flag, but that's not the point I'm trying to
make.)
From: David Schwartz on
On Jan 26, 11:41 am, Jef Driesen <jefdrie...(a)hotmail.com.invalid>
wrote:

> > If you have some way to tell, then you have to code that way. There is
> > no way the kernel can "just do it".

> I'm not expecting the kernel do something magic. It's just that from my
> point of view, the pty appears as a character device, just like a real
> serial port (or even a regular file). And if I close a serial port (or
> regular file) which was opened in exclusive mode, the kernel does unlock
> it again. So why would that be any different for pty's? (I know now it's
> due to the shared TIOCEXCL flag, but that's not the point I'm trying to
> make.)

You are expecting the kernel to do something magic. You are expecting
the kernel to allow another instance of your process but deny an
related process, even though it has no way to tell them apart.

Your analogy to a serial port shows why the situations are not
analogous. You are comparing the last close on a serial port that
releases the resource to a non-last close on a pty that does not
release the resource. If, for example, you make another name for the
same serial port and have a process that has that name open, a close
on a serial port will not release the exclusive flag. Only the *last*
close that fully releases the resource will release the exclusive
flag.

The same is true of a regular file, a socket, or any other resource.
Only the 'last close' that fully releases the resource triggers the
automatic "on close" behavior.

DS
From: Jef Driesen on
On 27/01/2010 0:28, David Schwartz wrote:
> On Jan 26, 11:41 am, Jef Driesen<jefdrie...(a)hotmail.com.invalid>
> wrote:
>
>>> If you have some way to tell, then you have to code that way. There is
>>> no way the kernel can "just do it".
>
>> I'm not expecting the kernel do something magic. It's just that from my
>> point of view, the pty appears as a character device, just like a real
>> serial port (or even a regular file). And if I close a serial port (or
>> regular file) which was opened in exclusive mode, the kernel does unlock
>> it again. So why would that be any different for pty's? (I know now it's
>> due to the shared TIOCEXCL flag, but that's not the point I'm trying to
>> make.)
>
> You are expecting the kernel to do something magic. You are expecting
> the kernel to allow another instance of your process but deny an
> related process, even though it has no way to tell them apart.

I think you are misunderstanding me here.

I have one "server" process which job is to simulate a real hardware
device. This is a server like process (e.g. started once and keeps
running until killed) that connects to the master side of the pty.
(That's not 100% correct because I'm using socat to link two pty
together. Thus the master sides are managed by socat and my server
process connects to one of the clients sides. But it could just as well
use the master side directly.)

I have another "client" application that wants to communicate with the
simulated device, by opening the slave side of the pty. Since I don't
want any other process to interfere with the communication, I would like
to set TIOCEXCL. So that any other process is denied access, including
another instance of my own client app.

> Your analogy to a serial port shows why the situations are not
> analogous. You are comparing the last close on a serial port that
> releases the resource to a non-last close on a pty that does not
> release the resource. If, for example, you make another name for the
> same serial port and have a process that has that name open, a close
> on a serial port will not release the exclusive flag. Only the *last*
> close that fully releases the resource will release the exclusive
> flag.
>
> The same is true of a regular file, a socket, or any other resource.
> Only the 'last close' that fully releases the resource triggers the
> automatic "on close" behavior.

I fully understand that the kernel can only release resources at the
last close. But in my situation, when the client app closes the slave
pty, that was the last reference. Of course the server app still has the
master pty open. So the kernel obviously can't release the entire pty,
but it can release slave side resources such as the TIOCEXCL flag.

The current situation with a shared TIOCEXCL flag, is that the slave pty
remains locked, and any other process that attempts to communicate with
my server app will fail with EBUSY. Thus the server process is still
running, but it can't accept new connections. Since it doesn't know the
client app has disconnected, the only thing I can do is manually kill
the server process and restart it. But the kernel does know that the
client app has disconnected!

Now imagine that we replace the pty with a socket. If a client app
closes the socket, I expect that another client can connect. Not that
the server app remains locked.

If the kernel would clear the TIOCEXCL flag for the slave side,
everything would work as expected. Note that I have no if that is
technically possible. I can imagine difficulties if the kernel does not
track the number of references for each side separately, but only the
total number.
From: David Schwartz on
On Jan 29, 3:11 am, Jef Driesen <jefdrie...(a)hotmail.com.invalid>
wrote:

> I have another "client" application that wants to communicate with the
> simulated device, by opening the slave side of the pty. Since I don't
> want any other process to interfere with the communication, I would like
> to set TIOCEXCL. So that any other process is denied access, including
> another instance of my own client app.

But that's not what you want. You want another instance of your client
app to be allowed access if and only if the previous instance has
terminated. In other words, you want something unusual.

> > The same is true of a regular file, a socket, or any other resource.
> > Only the 'last close' that fully releases the resource triggers the
> > automatic "on close" behavior.

> I fully understand that the kernel can only release resources at the
> last close. But in my situation, when the client app closes the slave
> pty, that was the last reference. Of course the server app still has the
> master pty open. So the kernel obviously can't release the entire pty,
> but it can release slave side resources such as the TIOCEXCL flag.

That would make no sense in general, although it happens to do what
you want. It would simply be wrong, in general, for the kernel to
release exclusivity on the pty until the last reference to it was
dropped. It would be inconsistent with the way other objects are
handled.

> The current situation with a shared TIOCEXCL flag, is that the slave pty
> remains locked, and any other process that attempts to communicate with
> my server app will fail with EBUSY. Thus the server process is still
> running, but it can't accept new connections. Since it doesn't know the
> client app has disconnected, the only thing I can do is manually kill
> the server process and restart it. But the kernel does know that the
> client app has disconnected!

Right, so code what you want. You want something unusual, so code it.
It won't happen by magic.

> Now imagine that we replace the pty with a socket. If a client app
> closes the socket, I expect that another client can connect. Not that
> the server app remains locked.

Right, but if you add another process that also holds a handle to the
same connection, that's not what you'll get.

> If the kernel would clear the TIOCEXCL flag for the slave side,
> everything would work as expected. Note that I have no if that is
> technically possible. I can imagine difficulties if the kernel does not
> track the number of references for each side separately, but only the
> total number.

Everything will not work as expected, everything will work in the very
unsual way you happen to want. And ptys will behave different from all
other devices in that shutdown/release behavior will be invoked even
if another process still has a handle to the same underlying object.

You want something unusual. The system gives you the ability to do it.
You just need to *do* it. It cannot happen by magic -- as I explained,
the system cannot distinguish another instance of your process (which
should be allowed access) from an unrelated process trying to steal
your pty (which should not be).

Since the default protection behavior is not what you want, you either
have to give up protection or implement the behavior you *do* want.

The kernel provides a "one size fits most" protection scheme and gives
you the tools to implement your own if that doesn't fit. That doesn't
fit, so implement your own.

You'd be mighty pissed if you didn't realize this issue and some
malicious program managed to crash your client side and then open your
pty while it's still connected to the same master!

DS