socket limitations question
in [TCL]
|
Prev: Announce: Runtime Library 3.0 Gestalt Items 1.1 and Hugelist 1.4,Tcl-Mode for XCode
Next: http::geturl intermittent hang - looks for ideas...
From: Uwe Klein on 12 Nov 2009 03:48 Alexandre Ferrieux wrote: > On Nov 11, 9:52 pm, Tomas Friml <instantni....(a)gmail.com> wrote: > >>On Nov 11, 11:07 am, Alexandre Ferrieux <alexandre.ferri...(a)gmail.com> >>wrote: >> >> >> >> >> >> >>>On Nov 10, 10:57 pm, Tomas Friml <instantni....(a)gmail.com> wrote: >> >>>>On Oct 15, 2:48 am, Alexandre Ferrieux <alexandre.ferri...(a)gmail.com> >>>>wrote: >> >>>>>On Oct 14, 3:30 pm, "Donal K. Fellows" >> >>>>><donal.k.fell...(a)manchester.ac.uk> wrote: >>>>> >>>>>>On 14 Oct, 01:41, Alexandre Ferrieux <alexandre.ferri...(a)gmail.com> >>>>>>wrote: >> >>>>>>>However, as a recent discussion on tclcore showed, there is another >>>>>>>one that kicks in, and that is not configurable at runtime: >>>>>>>FD_SETSIZE, which is the static size of the bit arrays passed to select. >>>>>>>IIRC it is 1024 for glibc. >> >>>>>>On at least some OSes, you can set it at build-time. >> >>>>>Yes. Recompile your libc, reboot, and pray. >> >>>>>-Alex >> >>>>Hi, I recently hit this same problem described here, after sock1024 is >>>>used thesocketcommand basically stops working. I was trying to find >>>>out what need to do get it working for more than 1024 sockets but I >>>>don't really understand the exact relation between tcl and libc (I'm >>>>tcl newbie). How do I find out which libc my tcl uses? >> >>>Well, it uses _the_ libc present on the system (unless you're >>>intentionally juggling with several versions, you have just one of >>>them). Then look at the value of the FD_SETSIZE in the associated >>>include files. >> >>>-Alex >> >>Thanks Alex, I recompiled the libc (setting FD_SETSIZE to 10000 for a >>start), updated, restarted, but it didn't help :(. Still the same... > > > Well, the kernel has some influence too ;-) If select is a syscall > (instead of being a library function implemented in terms of a poll/ > epoll syscall), then clearly FD_SETSIZE will have to be a constant in > the kernel itself. Which you can recompile too, if you have an open > source OS. > > -Alex look into 'ulimit -n' its usually 1024 by default me(a)home # ulimit -n 1024 me(a)home # ulimit -n 1048576 me(a)home # ulimit -n 1048577 bash: ulimit: open files: cannot modify limit: Die Operation ist nicht erlaubt so forex on Linux 2.6.5-7.111.19-default the hard limit for open files is 2**20 uwe
From: Tomas Friml on 12 Nov 2009 15:03 On Nov 12, 9:48 pm, Uwe Klein <uwe_klein_habertw...(a)t-online.de> wrote: > Alexandre Ferrieux wrote: > > On Nov 11, 9:52 pm, Tomas Friml <instantni....(a)gmail.com> wrote: > > >>On Nov 11, 11:07 am, Alexandre Ferrieux <alexandre.ferri...(a)gmail.com> > >>wrote: > > >>>On Nov 10, 10:57 pm, Tomas Friml <instantni....(a)gmail.com> wrote: > > >>>>On Oct 15, 2:48 am, Alexandre Ferrieux <alexandre.ferri...(a)gmail.com> > >>>>wrote: > > >>>>>On Oct 14, 3:30 pm, "Donal K. Fellows" > > >>>>><donal.k.fell...(a)manchester.ac.uk> wrote: > > >>>>>>On 14 Oct, 01:41, Alexandre Ferrieux <alexandre.ferri...(a)gmail.com> > >>>>>>wrote: > > >>>>>>>However, as a recent discussion on tclcore showed, there is another > >>>>>>>one that kicks in, and that is not configurable at runtime: > >>>>>>>FD_SETSIZE, which is the static size of the bit arrays passed to select. > >>>>>>>IIRC it is 1024 for glibc. > > >>>>>>On at least some OSes, you can set it at build-time. > > >>>>>Yes. Recompile your libc, reboot, and pray. > > >>>>>-Alex > > >>>>Hi, I recently hit this same problem described here, after sock1024 is > >>>>used thesocketcommand basically stops working. I was trying to find > >>>>out what need to do get it working for more than 1024 sockets but I > >>>>don't really understand the exact relation between tcl and libc (I'm > >>>>tcl newbie). How do I find out which libc my tcl uses? > > >>>Well, it uses _the_ libc present on the system (unless you're > >>>intentionally juggling with several versions, you have just one of > >>>them). Then look at the value of the FD_SETSIZE in the associated > >>>include files. > > >>>-Alex > > >>Thanks Alex, I recompiled the libc (setting FD_SETSIZE to 10000 for a > >>start), updated, restarted, but it didn't help :(. Still the same... > > > Well, the kernel has some influence too ;-) If select is a syscall > > (instead of being a library function implemented in terms of a poll/ > > epoll syscall), then clearly FD_SETSIZE will have to be a constant in > > the kernel itself. Which you can recompile too, if you have an open > > source OS. > > > -Alex > > look into 'ulimit -n' its usually 1024 by default > me(a)home # ulimit -n > 1024 > me(a)home # ulimit -n 1048576 > me(a)home # ulimit -n 1048577 > bash: ulimit: open files: cannot modifylimit: Die Operation ist nicht erlaubt > > so forex on Linux 2.6.5-7.111.19-default the hardlimitfor open files is 2**20 > > uwe Yep, ulimit was the first thing I increased - without it socket returns "too many files open", but even if you increase that the socket function just stops working when you hit sock1024. I recompiled libc as Alex suggested and I'm in the process of recompiling the kernel as well (I just needed to update to CentOS 5.4 first - we are using CentOS). Let you know how it went :) T.
From: Tomas Friml on 1 Dec 2009 14:58 It took a while till we reached the limit again but unfortunately it didn't work even with modified kernel :((. Any other ideas? On Nov 13, 9:03 am, Tomas Friml <instantni....(a)gmail.com> wrote: > On Nov 12, 9:48 pm, Uwe Klein <uwe_klein_habertw...(a)t-online.de> > wrote: > > > > > Alexandre Ferrieux wrote: > > > On Nov 11, 9:52 pm, Tomas Friml <instantni....(a)gmail.com> wrote: > > > >>On Nov 11, 11:07 am, Alexandre Ferrieux <alexandre.ferri...(a)gmail.com> > > >>wrote: > > > >>>On Nov 10, 10:57 pm, Tomas Friml <instantni....(a)gmail.com> wrote: > > > >>>>On Oct 15, 2:48 am, Alexandre Ferrieux <alexandre.ferri...(a)gmail.com> > > >>>>wrote: > > > >>>>>On Oct 14, 3:30 pm, "Donal K. Fellows" > > > >>>>><donal.k.fell...(a)manchester.ac.uk> wrote: > > > >>>>>>On 14 Oct, 01:41, Alexandre Ferrieux <alexandre.ferri...(a)gmail.com> > > >>>>>>wrote: > > > >>>>>>>However, as a recent discussion on tclcore showed, there is another > > >>>>>>>one that kicks in, and that is not configurable at runtime: > > >>>>>>>FD_SETSIZE, which is the static size of the bit arrays passed to select. > > >>>>>>>IIRC it is 1024 for glibc. > > > >>>>>>On at least some OSes, you can set it at build-time. > > > >>>>>Yes. Recompile your libc, reboot, and pray. > > > >>>>>-Alex > > > >>>>Hi, I recently hit this same problem described here, after sock1024 is > > >>>>used thesocketcommand basically stops working. I was trying to find > > >>>>out what need to do get it working for more than 1024 sockets but I > > >>>>don't really understand the exact relation between tcl and libc (I'm > > >>>>tcl newbie). How do I find out which libc my tcl uses? > > > >>>Well, it uses _the_ libc present on the system (unless you're > > >>>intentionally juggling with several versions, you have just one of > > >>>them). Then look at the value of the FD_SETSIZE in the associated > > >>>include files. > > > >>>-Alex > > > >>Thanks Alex, I recompiled the libc (setting FD_SETSIZE to 10000 for a > > >>start), updated, restarted, but it didn't help :(. Still the same... > > > > Well, the kernel has some influence too ;-) If select is a syscall > > > (instead of being a library function implemented in terms of a poll/ > > > epoll syscall), then clearly FD_SETSIZE will have to be a constant in > > > the kernel itself. Which you can recompile too, if you have an open > > > source OS. > > > > -Alex > > > look into 'ulimit -n' its usually 1024 by default > > me(a)home # ulimit -n > > 1024 > > me(a)home # ulimit -n 1048576 > > me(a)home # ulimit -n 1048577 > > bash: ulimit: open files: cannot modifylimit: Die Operation ist nicht erlaubt > > > so forex on Linux 2.6.5-7.111.19-default the hardlimitfor open files is 2**20 > > > uwe > > Yep, ulimit was the first thing I increased - without itsocket > returns "too many files open", but even if you increase that thesocketfunction just stops working when you hit sock1024. I recompiled > libc as Alex suggested and I'm in the process of recompiling the > kernel as well (I just needed to update to CentOS 5.4 first - we are > using CentOS). > > Let you know how it went :) > > T.
From: Cyan on 3 Dec 2009 04:00
On Oct 14, 1:50 am, keithv <kvet...(a)gmail.com> wrote: > This is probably more an OS question than > a tcl one, but does anybody know the how > many sockets one application can listen > on? > > The design I inherited (but is not yet > finalized) is to monitor a large number > of idle devices, all with a 2-way connection > via sockets. I was hoping to use tcl but > I need to know how large it can scale. Are > we talking hundreds, thousands, or what? > > The OS in question is some kind of Linux. > > Thanks, > Keith As everyone else pointed out a host of things effectively limit you to 1024 sockets. I recently needed to test a system that involved running just under 10000 network servers, connected in roughly a tree fashion, with at least one connection from a monitoring process to each server (all on a single machine) I got it going in the end by using a 2 tier system, where a master monitor process spawns slaves (open |slave.tcl) and communicates with it over it's stdin / stdout. At this level of fan-out there area about 180 connections from the master to the slaves. The master then sends jobs to the slaves to get them to spawn the servers and communicate the state back to the master over their stdout, threadpool style. I didn't use actual threads because I use tcl builds with threading disabled because it breaks other stuff I do, and using processes means I don't need to tweak any OS fd limits. In the end it worked well. It was throwaway code, only meant to support a single test, but if you want it use as the basis for what you need to do give me a shout. Cyan |