How many threads can be handled by the OS scheduler?
in [Win32 API]
|
Prev: Assign label to a drive letter and notify the Shell to update its cache
Next: handling user clicking x on console window on W2008
From: Timo Kunze on 25 Apr 2010 10:07 Hello Ulrich, Am 23.04.2010 08:59, schrieb Ulrich Eckhardt: > Yes, the OS can, though of course it presents overhead. If your program can > depends on how it uses the computer. It reads data from a database and sends it to other machines where other instances of this program write the received data back into a database. The sending part uses a single thread only, while the receiving part creates a thread for each incoming connection. The data is compressed and encrypted for the transfer. Since the methods used for compression and encryption both have been chosen by performance and not by compression rate/security, there is some CPU usage, but it's not as high as you might expect. > Why? I'd use the number of shared resources as base, plus one or two. Yes, you're right. Actually using the number of shared resources as base is more or less what I do (just with a different formula). I wrote about the number of CPUs, because so far my multi-threaded programs have mainly been limited by the CPU. Since the program we're talking about is mainly IO-limited, I wonder how many concurrent connections can be made over a single NIC (100 MBit, sometimes 1 GBit, mass-market hardware, no server components) until overhead outweights throughput gains. The ultimate target of this app is to distribute the data as fast as possible. The duration of the transfer to a single machine is much less important than the duration of the overall transfer to all machines in the network. Fortunately the transfer concept is robust. If the app can't send the data because the receiver rejects the connection, it will just try again later. Since the app writes received data into a database and these write accesses may block each other, I think it's wise to accept only a few incoming connections concurrently. So I think the fastest transfer can be reached if the app accepts x concurrent incoming connections and sends data to y different machines concurrently, with y being a bit larger than x, because reading from the database should be faster than writing to it. The best values of x and y would have to be evaluated, but I would start with x = 10 and y = 15 or 20. What do you think about my thoughts? Regards Timo -- www.TimoSoft-Software.de - Unicode controls for VB6 "Those who sacrifice freedom for safety deserve neither." "Demokratie ist per Definition unsicher. Ihr Schutz entsteht aus der Überzeugung, dass die demokratischen Kräfte überwiegen und sich – auf demokratischem Wege – durchsetzen." |