threading and multicores, pros and cons
in [Python]
|
From: sjdevnull on 14 Feb 2007 16:24 On Feb 14, 1:44 am, Paul Rubin <http://phr...(a)NOSPAM.invalid> wrote: > > When a customer comes with his new beautiful dual-core server and > > get a basic plone install up and running, he will immediately > > compare it to J2EE and wonder why he should pay a consultant to make > > it work properly. At this time, it 's not easy to explain him that > > python is not flawed compared to Java, and that he will not regret > > his choice in the future. First impression may be decisive. > > That is true, parallelism is an area where Java is ahead of us. Java's traditionally been ahead in one case, but well behind in general. Java has historically had no support at all for real multiple process solutions (akin to fork() or ZwCreateProcess() with NULL SectionHandle), which should make up the vast majority of parallel programs (basically all of those except where you don't want memory protection). Has this changed in recent Java releases? Is there a way to use efficient copy-on-write multiprocess architectures?
From: Paul Rubin on 14 Feb 2007 16:37 "sjdevnull(a)yahoo.com" <sjdevnull(a)yahoo.com> writes: > Java has historically had no support at all for real multiple process > solutions (akin to fork() or ZwCreateProcess() with NULL > SectionHandle), which should make up the vast majority of parallel > programs (basically all of those except where you don't want memory > protection). I don't know what ZwCreateProcess is (sounds like a Windows-ism) but I remember using popen() under Java 1.1 in Solaris. That at least allows launching a new process and communicating with it. I don't know if there was anything like mmap. I think this is mostly a question of library functions--you could certainly write JNI extensions for that stuff. > Has this changed in recent Java releases? Is there a way to use > efficient copy-on-write multiprocess architectures? I do think they've been adding more stuff for parallelism in general.
From: sjdevnull on 14 Feb 2007 18:14 On Feb 14, 4:37 pm, Paul Rubin <http://phr...(a)NOSPAM.invalid> wrote: > "sjdevn...(a)yahoo.com" <sjdevn...(a)yahoo.com> writes: > > Java has historically had no support at all for real multiple process > > solutions (akin to fork() or ZwCreateProcess() with NULL > > SectionHandle), which should make up the vast majority of parallel > > programs (basically all of those except where you don't want memory > > protection). > > I don't know what ZwCreateProcess is (sounds like a Windows-ism) Yeah, it's the Window equivalent to fork. Does true copy-on-write, so you can do efficient multiprocess work. > but I > remember using popen() under Java 1.1 in Solaris. That at least > allows launching a new process and communicating with it. Yep. That's okay for limited kinds of applications. > I don't know if there was anything like mmap. That would be important as well. > I think this is mostly a > question of library functions--you could certainly write JNI > extensions for that stuff. Sure. If you're writing extensions you can work around the GIL, too. > > Has this changed in recent Java releases? Is there a way to use > > efficient copy-on-write multiprocess architectures? > > I do think they've been adding more stuff for parallelism in general. Up through 1.3/1.4 or so they were pretty staunchly in the "threads for everything!" camp, but they've added a select/poll-style call a couple versions back. That was a pretty big sticking point previously.
From: Paul Rubin on 14 Feb 2007 18:22 "sjdevnull(a)yahoo.com" <sjdevnull(a)yahoo.com> writes: > > question of library functions--you could certainly write JNI > > extensions for that stuff [access to mmap, etc.] > Sure. If you're writing extensions you can work around the GIL, too. I don't think that's comparable--if you have extensions turning off the GIL, they can't mess with Python data objects, which generally assume the GIL's presence. Python's mmap module can't do that either. > Up through 1.3/1.4 or so they were pretty staunchly in the "threads > for everything!" camp, but they've added a select/poll-style call a > couple versions back. That was a pretty big sticking point previously. They've gone much further now and they actually have some STM features: http://www-128.ibm.com/developerworks/java/library/j-jtp11234/
From: Paul Rubin on 15 Feb 2007 00:59
John Nagle <nagle(a)animats.com> writes: > If locking is expensive on x86, it's implemented wrong. > It's done right in QNX, with inline code for the non-blocking case. Acquiring the lock still takes an expensive instruction, LOCK XCHG or whatever. I think QNX is usually run on embedded cpu's with less extensive caching as these multicore x86's, so the lock prefix may be less expensive in the QNX systems. |