Prev: Have a question alike to 'sumif' in excel
Next: polar stereographic plot
From: Jan Simon on 15 Apr 2010 09:10
Dear Newsgroup!

Is there a method to create a safe file mutex in Matlab?

My problem:
1. Several Matlab sessions are working from different computers on the same file pool on a network drive.
2. All programs add a short comment to a single log file.
I want to avoid conflicts between the sessions, e.g. let a session wait until another has finished processing the data in a folder or writing its line to the log file.

Tried solution:
ComputerName = 'PC0815'; % dummy here
ProcessID = GetProcessIdFromOS; % dummy here
LockFile = fullfile(FolderName, '.lock');
if exist(LockFile, 'file') == 0 % Lock the folder
% *** race condition here
FID = fopen(LockFile, 'wb');
% ... check valid FID
fprintf(FID, '%s %s %s\n', datestr(now, 0), ComputerName, ProcessID);
fclose(FID);
else % Folder is locked
...
end

But exactly at *** there is a race condition: another session can open the same file for writing also.
If 2 Matlab sessions are writing a line in the same file, the file contains both lines, but the order seems to depend on which session *closes* the file at first, if the line length is less than 1023 characters. Therefore I could check, if the first line of the file equals the identifier of the current Matlab session after closing the file. But I'm really unhappy to check the contents of a file, which may be open for writing by another (or even several other) program(s).

The time between EXIST and FOPEN is tiny, but it still exists and it is not worth to spend time in programming an *almost* mutual exclusion method...

The easiest method would be a FOPEN permission which does:
create file for writing, fail if existing already
This is *not* FOPEN(r+)! The Windows API allows the wanted behaviour with CreateFile(GENERIC_WRITE, CREATE_NEW), but it would be nicer to keep this independent from the platform.
I do not have the parallel programming toolbox - does it contain a secure file mutex?

Any ideas? Thanks, Jan
From: Ashish Uthama on 15 Apr 2010 08:27
On Thu, 15 Apr 2010 10:10:22 -0300, Jan Simon
<matlab.THIS_YEAR(a)nminussimon.de> wrote:

> Dear Newsgroup!
>
> Is there a method to create a safe file mutex in Matlab?
>
> My problem:
> 1. Several Matlab sessions are working from different computers on the
> same file pool on a network drive.
> 2. All programs add a short comment to a single log file.
> I want to avoid conflicts between the sessions, e.g. let a session wait
> until another has finished processing the data in a folder or writing
> its line to the log file.
>
> Tried solution:
> ComputerName = 'PC0815'; % dummy here
> ProcessID = GetProcessIdFromOS; % dummy here
> LockFile = fullfile(FolderName, '.lock');
> if exist(LockFile, 'file') == 0 % Lock the folder
> % *** race condition here
> FID = fopen(LockFile, 'wb');
> % ... check valid FID
> fprintf(FID, '%s %s %s\n', datestr(now, 0), ComputerName,
> ProcessID);
> fclose(FID);
> else % Folder is locked
> ...
> end
>
> But exactly at *** there is a race condition: another session can open
> the same file for writing also.
> If 2 Matlab sessions are writing a line in the same file, the file
> contains both lines, but the order seems to depend on which session
> *closes* the file at first, if the line length is less than 1023
> characters. Therefore I could check, if the first line of the file
> equals the identifier of the current Matlab session after closing the
> file. But I'm really unhappy to check the contents of a file, which may
> be open for writing by another (or even several other) program(s).
>
> The time between EXIST and FOPEN is tiny, but it still exists and it is
> not worth to spend time in programming an *almost* mutual exclusion
> method...
>
> The easiest method would be a FOPEN permission which does:
> create file for writing, fail if existing already
> This is *not* FOPEN(r+)! The Windows API allows the wanted behaviour
> with CreateFile(GENERIC_WRITE, CREATE_NEW), but it would be nicer to
> keep this independent from the platform.
> I do not have the parallel programming toolbox - does it contain a
> secure file mutex?
>
> Any ideas? Thanks, Jan

http://java.sun.com/j2se/1.4.2/docs/api/java/nio/channels/FileLock.html ?
From: Jan Simon on 15 Apr 2010 14:14
Dear Ashish!

> http://java.sun.com/j2se/1.4.2/docs/api/java/nio/channels/FileLock.html ?

Good idea!
Unfortunately I have be program backward compatible with Matlab 6.5, which uses Java 1.3, which does not include the nio/channels (or at least I cannot find them).

Thanks, Jan
From: Walter Roberson on 15 Apr 2010 15:34
Jan Simon wrote:
> Dear Ashish!
>
>> http://java.sun.com/j2se/1.4.2/docs/api/java/nio/channels/FileLock.html ?
>
> Good idea!

Your original posting specified,

> Is there a method to create a safe file mutex in Matlab?

> My problem:
> 1. Several Matlab sessions are working from different computers on
> the same file pool on a network drive.

If you read the System Dependencies of the above URL, you will find that those
Java locks are only guaranteed at the single virtual machine level, not even
for multiple threads within the same virtual machine, and not for multiple
processes within the same machine, let alone different processes operating on
different machines with a shared filesystem.

If your shared filesystem is based upon NFS (any version) or SMB for unix-type
systems, or is running on any unix-type system (that is currently supported by
Mathworks) other than Trusted AIX, then the warnings about "advisory locks"
and "extreme care" for network file systems apply: in such cases, a networked
Mutex might "usually" work, but is not guaranteed, even if you are using lockd
or are using NFSv3 with integrated lockd: in particular lockd (or its
integrated equivalent) break down when nodes with locks are rebooted. And if
you aren't using lockd (or its integrated equivalent) then networked NFS locks
are not robust at all.

The POSIX function open() with O_CREAT|O_EXCL option is defined to be atomic
and "shall fail if the file exists" -- but NFSv2 even with lockd assistance
cannot guarantee these POSIX semantics without race conditions. NFSv3 with its
integrated lockd are _intended_ to follow these semantics, but as noted above,
can break with reboots (or network problems.)


I have read that real mandatory networked locking is available under XP (might
need SP2) and later MS operating systems (but I wouldn't guarantee it for
their "home" editions!), but I have never investigated that. Based upon what I
did chase down about XP and POSIX, I would be uncomfortable in relying upon
real networked locking with XP Home even with SP2.
From: Yair Altman on 15 Apr 2010 18:26
Another poor-man's solution: try using the fileattrib function to test & set a specific file attribute (for example: archive or execute). It's not guarantied to be "thread-safe", but it should work in practice unless you have extremely close race conditions.
 |  Next  |  Last
Pages: 1 2
Prev: Have a question alike to 'sumif' in excel
Next: polar stereographic plot