Prev: Decision Variable in the Matrix Form
Next: Why do deploytool generates different exe file
From: Max on 18 Jul 2010 00:20
Hi,

I'm writing a code which would allow multithreading on a cluster. For that I start several instances of Matlab: one of them is the "main" one, which put a request for jobs into a message file, "JobSubmit", while the rest of the instances (threads) are supposed to wait for until the jobs are posted. The number of jobs is supposed to be larger than the number of threads, and the the threads should be able to leave a message of what job number they are currently running, so that other thread would not take them.

Here comes the problem. I need to be able to lock the access to the file, while one of the threads is reading it and is leaving its marks, to exclude the possibility of two threads accessing the same file at once and thus confusing/corrupting the operation. I'm wondering if there is a way to implement it Matlab? Any advises on how it could be realized otherwise are much appreciated!

Thank you,
Max
From: Jan Simon on 19 Jul 2010 04:08
Dear Max,

> I need to be able to lock the access to the file, while one of the threads is reading it and is leaving its marks, to exclude the possibility of two threads accessing the same file at once and thus confusing/corrupting the operation.

If you search for "file lock thread" in this newsgroup, you find:
http://www.mathworks.com/matlabcentral/newsreader/view_thread/279510

Perhaps this helps, Jan
From: Max on 19 Jul 2010 12:32
Dear Jan,

Thank you for the link. Your question in the thread was indeed close to but more restrictive than mine. As I understood from reading your thread, the conclusion to your conundrum was to implement a database solution, right? Unfortunately, I don't think it will work for me - I run the program under Condor Management system and don't think I can work with databases there. But the lack of "simpler" solutions in your thread seems to indicate that locking might not be an option.

For my purposes, though, it would be sufficient to simply check whether the common file is being accessed at the moment, and if so - try again in a second. I considered using bash command lsof to obtain such information, but, unfortunately, it works slow and requires root access, which I don't have...



"Jan Simon" <matlab.THIS_YEAR(a)nMINUSsimon.de> wrote in message <i21155$kq9$1(a)fred.mathworks.com>...
> Dear Max,
>
> > I need to be able to lock the access to the file, while one of the threads is reading it and is leaving its marks, to exclude the possibility of two threads accessing the same file at once and thus confusing/corrupting the operation.
>
> If you search for "file lock thread" in this newsgroup, you find:
> http://www.mathworks.com/matlabcentral/newsreader/view_thread/279510
>
> Perhaps this helps, Jan
From: Jan Simon on 19 Jul 2010 16:27
Dear Max,

This was suggested by Ashish:
http://java.sun.com/j2se/1.4.2/docs/api/java/nio/channels/FileLock.html

And another idea was:
create a file with a tempname
if renaming (MOVEFILE) to a specific lock file works, the caller get access
else wait for some time and try again
delete the lock file
While renaming can be triggered to work only, if the destination does not exist, the creation/opening of a file cannot be done thread-safe, as far as I understood.

Kind regards, Jan
From: Max on 19 Jul 2010 17:34
> And another idea was:
> create a file with a tempname
> if renaming (MOVEFILE) to a specific lock file works, the caller get access
> else wait for some time and try again
> delete the lock file
> While renaming can be triggered to work only, if the destination does not exist, the creation/opening of a file cannot be done thread-safe, as far as I understood.

That's a great idea! But did you mean that the renaming (moving) can be triggered to work only if the ORIGIN exists?

One could use the presence of the destination file as a break on the access of other files:
fileattrib('test.lock','-w') % change the attribute so that if the file exists, the use of it as a destination will cause an error (status of copyfile==0). Windows platforms?
status = copyfile('test.lock','test.lock.lock') % copy the file. if cannot - try again later
<-- handing of a file -->
delete('test.lock.lock') % at the end: remove the roadblock

It seems to me that these two variants have the same chances to work...
 |  Next  |  Last
Pages: 1 2 3
Prev: Decision Variable in the Matrix Form
Next: Why do deploytool generates different exe file