Elephant experiences?
in [Lisp]
|
Prev: Amazing Video on How Computer Fashion Is Changing, Nice Work !!
Next: macro to expand into function body with declare at top
From: vanekl on 5 Mar 2010 22:38 On Mar 5, 10:16 pm, vanekl <va...(a)acd.net> wrote: > So I wrote a test program this morning. It turns out that this > architecture is absolutely terrible even with just a few threads > accessing the same file at the same time (which I had already assumed) > BUT only if the file is "small." Something interesting happens when > the size of the file increases to a "large" size. I could have 50 > threads accessing the same file and never get an integrity error. [I > was using something similar to your 'restore' function.] I'm trying to > figure out why this is. In any event, it surprised me. Follow up on my tests. I was allowing threads to randomly delete the file before doing a save. This one op (deletion) caused data integrity errors. When I removed this feature and made all threads do simple appends all errors went away. I'm sure either SBCL or Debian is holding a lock, but I can get this to work in a multithreaded environment without having to explicitly call a mutex as long as I never delete the file and use append mode.
From: Alex Mizrahi on 6 Mar 2010 04:48 v> went away. I'm sure either SBCL or Debian is holding a lock, but I can v> get this to work in a multithreaded environment without having to v> explicitly call a mutex as long as I never delete the file and use v> append mode. Multithreading errors only manifest themselves in certain situations. If you've checked this in one situation, this does not mean it will work in other situations too, unless you can prove otherwise. "I'm sure that something is holding lock" is not enough to prove that, sorry. I think small appends are, indeed, atomic. Larger appends won't be. I think with bufferization in place you need to write at least 4kb of data. Maybe more. Do you know how prin1 works? It goes throuh data structure, converts pieces of it into strings and writes them to stream. When there is enough stuff in buffer, it goes into a file. Even if each write is atomic, whole prin1 operation is not. If you have enough data, each prin1 will result in many writes when you have enough data. E.g. if you have 4KB buffer and two threads print 6KB of data, you can get this in your file: <first 4KB of thread1><first 4KB of thread2><second 2KB of thread1><second 2KB of thread2> That is, now you have a salad instead of your data. And I think depending on your data being small enough is just ridiculous.
From: vanekl on 6 Mar 2010 10:00 More tests. Sigh. It only works 99.9%+- of the time.
From: Giorgos Keramidas on 6 Mar 2010 10:15 On Fri, 5 Mar 2010 19:38:40 -0800 (PST), vanekl <vanek(a)acd.net> wrote: > On Mar 5, 10:16�pm, vanekl <va...(a)acd.net> wrote: >> So I wrote a test program this morning. It turns out that this >> architecture is absolutely terrible even with just a few threads >> accessing the same file at the same time (which I had already assumed) >> BUT only if the file is "small." Something interesting happens when >> the size of the file increases to a "large" size. I could have 50 >> threads accessing the same file and never get an integrity error. [I >> was using something similar to your 'restore' function.] I'm trying to >> figure out why this is. In any event, it surprised me. > > Follow up on my tests. > > I was allowing threads to randomly delete the file before doing a > save. This one op (deletion) caused data integrity errors. When I > removed this feature and made all threads do simple appends all errors > went away. I'm sure either SBCL or Debian is holding a lock, but I can > get this to work in a multithreaded environment without having to > explicitly call a mutex as long as I never delete the file and use > append mode. Yes, this makes sense. Writes to a file with a buffer size smaller than the internal buffer size of the kernel are usually 'mostly atomic'. If your kernel is using a buffer size for the underlying write(2) call that is 4 KB and you are appending in chunks less than 4 KB, then writes will tend to be in order, but there are still cases where two threads can race each other, e.g. when two threads try to seek to the end of the file to append: 1. Thread #1 seeks to current file size (at offset K). 2. Thread #2 seeks to current file size also at offset K. 3. Thread #1 writes a few Lisp forms at offset K. 4. Thread #2 writes a few forms of its own, at offset K too. Cooperative fcntl-style locking may help, but if you start 'bloating' the size of the small and elegant print / read functions that save and restore forms, you might as well start pushing the forms to a very light file-based database like SQLite.
From: vanekl on 6 Mar 2010 10:42
On Mar 6, 10:15 am, Giorgos Keramidas <keram...(a)ceid.upatras.gr> wrote: > On Fri, 5 Mar 2010 19:38:40 -0800 (PST), vanekl <va...(a)acd.net> wrote: > > On Mar 5, 10:16 pm, vanekl <va...(a)acd.net> wrote: > >> So I wrote a test program this morning. It turns out that this > >> architecture is absolutely terrible even with just a few threads > >> accessing the same file at the same time (which I had already assumed) > >> BUT only if the file is "small." Something interesting happens when > >> the size of the file increases to a "large" size. I could have 50 > >> threads accessing the same file and never get an integrity error. [I > >> was using something similar to your 'restore' function.] I'm trying to > >> figure out why this is. In any event, it surprised me. > > > Follow up on my tests. > > > I was allowing threads to randomly delete the file before doing a > > save. This one op (deletion) caused data integrity errors. When I > > removed this feature and made all threads do simple appends all errors > > went away. I'm sure either SBCL or Debian is holding a lock, but I can > > get this to work in a multithreaded environment without having to > > explicitly call a mutex as long as I never delete the file and use > > append mode. > > Yes, this makes sense. > > Writes to a file with a buffer size smaller than the internal buffer > size of the kernel are usually 'mostly atomic'. If your kernel is using > a buffer size for the underlying write(2) call that is 4 KB and you are > appending in chunks less than 4 KB, then writes will tend to be in > order, but there are still cases where two threads can race each other, > e.g. when two threads try to seek to the end of the file to append: > > 1. Thread #1 seeks to current > file size (at offset K). > > 2. Thread #2 seeks to current file size > also at offset K. > > 3. Thread #1 writes a few Lisp > forms at offset K. > > 4. Thread #2 writes a few forms of its > own, at offset K too. > > Cooperative fcntl-style locking may help, but if you start 'bloating' > the size of the small and elegant print / read functions that save and > restore forms, you might as well start pushing the forms to a very light > file-based database like SQLite. You must be right. No matter how hard I try, I cannot provoke an integrity error when the tuple is "small." When the tuple is "large," I get write errors about .1% of the time. |