Prev: How to Clean Tear Stains On a Dog
Next: Lisp on Mac OS X
From: Madhu on 11 Feb 2010 14:18

* Madhu <m3aavfabra.fsf(a)moon.robolove.meer.net> :
Wrote on Thu, 11 Feb 2010 23:35:45 +0530:

| * Zach Beane <87k4ujy8fu.fsf(a)hangup.portland.xach.com> :
| Wrote on Thu, 11 Feb 2010 12:43:33 -0500:
|
| | Madhu <enometh(a)meer.net> writes:
| |
| |> I'd prefer a slightly different level of abstraction - Split the
| |> job into two parts 1) the creation of a unique temporary file and
| |> 2) its use by the programmer via standard WITH-OPEN-FILE/OPEN
| |> macros. This would punt the complexity introduced in massaging
| |> OPEN-ARGS, etc., to `where it belongs'
| |>
| |> The idiom I'd use is
| |>
| |> (let (file)
| |> (unwind-protect (with-open-file (stream (setq file (HCL:MAKE-TEMP-FILE))
| |> :direction :io ...)
| |> ;; use stream
| |> )
| |> (when file (delete-file file))))
| |
| | It seems like this approach suffers from a race condition; I aimed
| | to avoid that. With SBCL on Unix, :if-exists :error and :if-exists
| | nil both eventually use the O_EXCL flag to open(), which guarantees
| | (modulo NFS and the like) the stream is opened on a fresh file.
|
| I'm not sure I understand what race contition you are seeing here,
| because the call to open(2) should NOT use the O_EXCL flag
| (i.e. create a fresh file) if you pass :IF-DOES-NOT-EXIST :ERROR, If
| it did that, it would be a bug in the implementation IMO.
|
| Note in the sketch above that the call to WITH-OPEN-FILE could or
| rather SHOULD be called with :IF-DOES-NOT-EXIST :ERROR, and the
| assumption is that WITH-OPEN-FILE opens an existing file which the
| MAKE-TEMP-FILE created.
|
| Only we (the callers of MAKE-TEMP-FILE) are responsible for deleting
| the created file, and no other file will be created with that name
| until we have deleted it.
|
| | I'd like to know if hcl:make-temp-file makes that a non-issue, somehow.
|
| If you split the job into the 2 pieces, MAKE-TEMP-FILE just needs to
| call creat(2), and it can possibly do this without going through CL:OPEN

I should clarify that here, creat(2) would not do the
job. MAKE-TEMP-FILE has to ensure that the file it is creating does not
exist. On Linux. If we used your sketch, and defined MAKE-TEMP-FILE
like this:

(loop for try = new-name()
when (with-open-stream (stream
(apply #'cl:open try
:if-does-not-exist :create
:if-exists NIL
:direction :output))
(if stream t))
return try)

And the implementation passes O_CREAT and O_EXCL to open(2) as you
noted, this would ensure that an existing file is not returned --- My
linux manpage specifies that when O_EXCL is specified with O_CREAT, the
call to open(2) will fail if the pathname already exists. It also states
that `The behavior of O_EXCL is undefined if O_CREAT is not specified.'

[So I assume SBCL will pass O_EXCL to open(2) only when it also passes
O_CREAT; In any case, there shouldnt be any race among its users as
long as MAKE-TEMP-FILE creates a unique file on each]

--
Madhu
From: Stelian Ionescu on 11 Feb 2010 14:44
On Thu, 11 Feb 2010 11:07:01 -0500, Zach Beane wrote:

> I need to work with a fresh, unique file that is deleted automatically
> when I'm done with it. Here are the functions I came up with, I'd love
> to hear any feedback.

(defparameter *tmpdir*
(concatenate 'string
(or (sb-posix:getenv "TMP")
(sb-posix:getenv "TMPDIR")
"/tmp")
"/"))

(defun call-with-temporary-file (fun &rest make-fd-stream-args
&key (element-type 'character)
(external-format :utf-8))
(let ((template (concatenate 'string *tmpdir* "XXXXXX")))
(multiple-value-bind (fd path)
(sb-posix:mkstemp template)
(sb-posix:unlink path)
(apply #'sb-sys:make-fd-stream fd :input t :output t
make-fd-stream-args))))

mkstemp() uses O_EXCL, and the file is deleted immediately(but still
referenced by the file descriptor) and will be removed completely when
you close the stream.

--
Stelian Ionescu a.k.a. fe[nl]ix
Quidquid latine dictum sit, altum videtur.
http://common-lisp.net/project/iolib
From: Madhu on 11 Feb 2010 15:20

* Zach Beane <87fx57y423.fsf(a)hangup.portland.xach.com> :
Wrote on Thu, 11 Feb 2010 14:18:12 -0500:

|> because the call to open(2) should NOT use the O_EXCL flag
|> (i.e. create a fresh file) if you pass :IF-DOES-NOT-EXIST :ERROR, If
|> it did that, it would be a bug in the implementation IMO.
|
| Right, when I mentioned O_EXCL, I was referring to my version, which
| uses ":if-exists nil", not :if-does-not-exist <anything>.

I don't think O_EXCL is an issue at all (if you see my the parallel
post); The only requirement I stated was that MAKE-TEMP-FILE should
create a unique temp file.

Now the objection seems to be against opening and using a temporary file
that was created by a different system call (in MAKE-TEMP-FILE), because
"someone else" may have deleted the file between those two calls

| I'm referring to
| http://www.dwheeler.com/secure-programs/Secure-Programs-HOWTO/avoid-race.html
| which says:
|
| Once you create the file atomically, you must alway use the returned
| file descriptor (or file stream, if created from the file descriptor
| using routines like fdopen()). You must never re-open the file, or use
| any operations that use the filename as a parameter - always use the
| file descriptor or associated stream. Otherwise, the tmpwatch race
| issues noted above will cause problems.

The tmpwatch race issues (specifically the farfetched scenarios in that
article) will still cause problems even if you follow these
recommendations. If someone else deletes your file, you're screwed. No
matter what. and that can still happen, everything else being equal,
since you typically won't be able to make your use of the tmp file an
atomic operation.

|> If you split the job into the 2 pieces, MAKE-TEMP-FILE just needs to
|> call creat(2), and it can possibly do this without going through CL:OPEN
|
| Is there a CL operation that corresponds closely to creat(2) like
| CL:OPEN with :if-exists :error-or-nil corresponds closely to open(2)
| with O_EXCL?

creat(2) was the wrong thing to mention here, sorry, I hope my parallel
reply cleared that up. The point I wanted to make was about ensuring
that MAKE-TEMP-FILE creates a NEW file (which did not exist before), and
I think CL:OPEN, as you used it would work.

--
Madhu
From: Michael Weber on 11 Feb 2010 17:51
On Feb 11, 8:44 pm, Stelian Ionescu <sione...(a)cddr.org> wrote:
> mkstemp() uses O_EXCL, and the file is deleted immediately(but still
> referenced by the file descriptor) and will be removed completely when
> you close the stream.

Besides the obvious typo that you don't funcall FUN, mkstemp does not
allow suffixes, and mkstemps is not supported widely enough.
From: Stelian Ionescu on 11 Feb 2010 18:19
On Thu, 11 Feb 2010 14:51:23 -0800, Michael Weber wrote:

> On Feb 11, 8:44 pm, Stelian Ionescu <sione...(a)cddr.org> wrote:
>> mkstemp() uses O_EXCL, and the file is deleted immediately(but still
>> referenced by the file descriptor) and will be removed completely when
>> you close the stream.
>
> Besides the obvious typo that you don't funcall FUN,

oops :)

> mkstemp does not allow suffixes, and mkstemps is not supported
> widely enough.

allowing suffixes is irrelevant, especially if you only want a private
temporary file(one unlinked right after creation)

--
Stelian Ionescu a.k.a. fe[nl]ix
Quidquid latine dictum sit, altum videtur.
http://common-lisp.net/project/iolib
First  |  Prev  |  Next  |  Last
Pages: 1 2 3
Prev: How to Clean Tear Stains On a Dog
Next: Lisp on Mac OS X