How hard is socket programming?
in [MFC]
|
Prev: Runtime Error does not specify program???
Next: windows 7 - imitating touch by sending wm_gesture ?
From: Joseph M. Newcomer on 17 Mar 2010 15:23 See below... On Tue, 16 Mar 2010 22:40:02 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >message news:r9h0q55c6oml5fhpkrcfs2jtbsof7rf1ld(a)4ax.com... >> See below... >> On Tue, 16 Mar 2010 20:50:58 -0500, "Peter Olcott" >> <NoSpam(a)OCR4Screen.com> wrote: >> >>> >>>"Hector Santos" <sant9442(a)nospam.gmail.com> wrote in >>>message >>>news:%23qdTxMXxKHA.4492(a)TK2MSFTNGP05.phx.gbl... >>>> Peter Olcott wrote: >>>> >>>> >>>>> You continue to ignore my repeatedly stated requirement >>>>> of 4.0 GB of data (a deterministic finite automaton) >>>>> that >>>>> must be loaded before execution can begin. If you >>>>> ignore >>>>> my fundamental requirements you can't possibly provide >>>>> good advice. >>>> >>>> >>>> Heed my advice if you want to get somewhere. Its coming >>>> for FREE! >>>> >>>> If you keeping 4 GB loaded under a 24x7 loaded exe , >>>> then >>>> the rest of your system will suffer and you will get >>>> paging delays during idle times. >>>> >>>> That calls for a memory map design requirement. It has >>>> nothing to do with FASTCGI unless youy think this >>>> "FastCGI >>>> Language" you wish to use provides a library for you to >>>> automatically do this for you - it does not. >>>> Independent >>>> concepts. >>>> >>>> -- >>>> HLS >>> >>>Fundamentally FastCGI differs from CGI in that the app is >>>held in memory rather than re-loaded every time. Although >>>this feature may have limited alternatives, I see no >>>reason >>>to consider them. >>> >> **** >> You obviously did not read the specs. NOWHERE does it say >> "the app is held in memory"; >> specifically, >> http://www.fastcgi.com/drupal/node/6?q=node/15 says >> * FastCGI processes are persistent: after finishing a >> request, they wait for a new >> request instead of exiting. > >So if you really want to get nitpicky it is held in virtual >memory (from the developer's point of view) that is mapped >to RAM, if there is no reason to page it out. **** See what I mean? You are insisting that because YOU don't see a reason that it should be paged out, the OPERATING SYSTEM will do what YOU think it ought to, and not page it out, which flies in the face of reality, in which an operating system will page out pages of an idle process. **** >Which >essentially boils down to it is held in RAM. (Even though it >can theoretically be paged out at any time, in actual >practice this rarely occurs if you have enough RAM. Virtual >Memory is an abstraction of RAM that can at times be >ignored. **** Are you ABSOLUTELY CERTAIN that the host OS behaves according to your dreams? It might not, and it might not actually have tunable parameters you can control! THis is what I mean about fact-finding; you have to talk to the experts (in this case, a linux expert) to understand what is going to REALLY happen to an idle process, how working set is managed, etc., and not rely upon your unfounded opinion of what you think SHOULD happen in the best-of-all-possible-worlds scenario. Reality has this really nasty habit of ignoring our wishes. joe **** > >> >> >> 1. The Web server creates FastCGI application processes to >> handle requests. The processes >> may be created at startup, or created on demand. >> 2. The FastCGI program initializes itself, and waits for a >> new connection from the Web >> server. >> 3. When a client request comes in, the Web server opens a >> connection to the FastCGI >> process. The server sends the CGI environment variable >> information and standard input over >> the connection. > >Not in the case of FastCGI, FastCGI does not use environment >variables. **** Perhaps you misread the point 3, which clearly states "sends the CGI environment variable INFORMATION...over the connection" (emphasis added, in case you missed it). ***** > >> 4. The FastCGI process sends the standard output and error >> information back to the server >> over the same connection. >> . When the FastCGI process closes the connection, the >> request is complete. The FastCGI >> process then waits for another connection from the Web >> server. >> >> NOWHERE in this spec does is make ANY claim that the "app >> is held in memory", only that a > >That is logically entailed by the statement that the app >does not terminate at the end of the request. With enough >RAM and few processes running one can virtually ignore >virtual memory. It may not be this way in theory, but, I >know that it is this way in practice, because in practice I >achieve the performance that could not be otherwise >achieved. ***** No, it is NOT "logically entailed" by the staement that the app does not terminate; this again shows your failure to understand how real operating systems work, which is how they are designed to work, not how you wish they had been designed to work. Windows, for example, pages out the pages of an idle process, particularly if they become LRU (but on general principles, will page them out). Unix used to work the same way. You have made this incredible leap of faith based on what you WANT it to do, with no input from a linux expert about how it ACTUALLY works. ***** > >> new app instance is not launched. Why you would leap to >> the (erroneous) conclusion that >> not starting a second app instance means the app is "held >> in memory" escapes me. What is >> "held in memory" is up to the platform's specific concepts >> of address space management for >> a process in the context of a certain physical memory >> configuration, and it is likely that >> some or all of the app's pages will be paged out to make >> room for other processes. This >> is your typical "design by wishful thinking" approach >> where yu make assertions about how > >I think that you may be getting just a little bit too >disrespectful. I do however really appreciate the support >that you provided on the quality of my patent. **** But we keep telling you where you are going wrong, and you keep insisting you must be right, because your success depends on your assumptions being right, even though most of us know they are actually wrong, based on our own experience. And when we tell you that you are wrong, and why you are wrong, you come back and tell us you *must* be right because you *want* the behavior you hypothesize, even though real systems don't work that way! We are getting very frustrated with having to tell you over and over again that your assumptions are faulty, and it annoys us that you ignore what we say and insist that the operating system must behave the way you erroneously state it behaves. I can see Hector getting frustrated for the same reason. I've been through this concerning memory allocation and TCP/IP with you, and my patience is running thin. You are now repeating the scenario with Web servers, where I am expert enough to know you are making nonsensical statements and Hector actually is an expert and is telling you (a) you are wrong and (b) how to do it right, and you keep making even more confused statements. joe ***** > >> things work, which are wrong, and then proceed to >> systematically ignore eveyone who tells >> you "but it doesn't work that way". This discussion is >> shaping up just like the "physical >> contiguous memory" discussion of a few months ago where >> you insisted that storage >> allocators *had* to work in a way that, historically, >> storage allocators have NEVER >> worked, and that virtual memory had to work in a fashion >> that has never been recognized as >> a way any operating system designer would implement >> virtual memory. Now you are telling >> us how FASTCGI works, which is, unfortunately, not the way >> it works at all. And that it >> will solve problems it is incapable of (not to mention >> irrelevant to) solving. >> >> So start paying attention to the answers. It will save >> you a lot of time and effort as >> you attempt to make technologies do things they never were >> designed to do, or try to get >> them to meet specifications they never pretended to meet. >> joe >> ***** >> Joseph M. Newcomer [MVP] >> email: newcomer(a)flounder.com >> Web: http://www.flounder.com >> MVP Tips: http://www.flounder.com/mvp_tips.htm > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Peter Olcott on 17 Mar 2010 15:31 "Hector Santos" <sant9442(a)nospam.gmail.com> wrote in message news:%23jOeaOgxKHA.3560(a)TK2MSFTNGP02.phx.gbl... > Why do you quote the entire message just a one line of > input? People can already follow the thread, if they > wish. No need to quote the entire thing, and worst, embed > one line response in the middle of it. > > Peter Olcott wrote: > >>> If you are have a separate service, you don't need this >>> or don't have to worry about FASTCGI. Use any web >>> server with an embedded language or CGI. I'm telling >>> ya, you are making this more complex than it is. >> >> It would be simplest to do this: >> OCR.cpp + mongoose.c = OCR_WebServer.exe > > > Probably, but I suspect not for you. :) > > if you are looking for scalability, follow for the > suggestions posted. > > -- > HLS Isn't the primary aspect of scalability how fast each request can be processed? Isn't this the controlling factor that determines how many concurrent requests can be processed? Of course the next most crucial factor would be how well this very fast single server can be divided up into multiple servers. According to my estimates a single server could provide $25G of monthly revenue before it would need to be upgraded. At this time I could hire some people to handle this.
From: Peter Olcott on 17 Mar 2010 15:32 "Hector Santos" <sant9442(a)nospam.gmail.com> wrote in message news:OUY3JTgxKHA.1692(a)TK2MSFTNGP04.phx.gbl... > Peter Olcott wrote: > >> http://mongoose.googlecode.com/svn/trunk/examples/authentication.c >> The answer is already complete in mongoose, also >> confirming that mongoose has cookies! > > > Mongoose doesn't have COOKIES. Your APPLICATION > implements cookies. > > It wouldn't be a web server if it didn't a have a > GET_HTTP_HEADER() function. In mongoose, its: > > const char * mg_get_header(conn, header_name); > > So to get the COOKIE from the HTTP block, it calls: > > const char *cookie; > cookie = mg_get_header(conn, "Cookie"); > > In that example, the NAME and PASS are part the form > fields, retrieved by: > > const char * mg_get_var(conn, field_name); > > Does it support? > > const char * mg_post_var(conn, field_name); > > Cookies are used here for hold a session condition. > > Of course, doens't work if the user has COOKIES disabled. > > Is this PCI ready? If you are going to charger people and > use credit cards, it needs to be PCI ready. > > > -- > HLS What is PCI? (I heard of PKI) I will be using Paypal as my payment processor.
From: Joseph M. Newcomer on 17 Mar 2010 15:34 See below... On Tue, 16 Mar 2010 22:19:25 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >message news:6ng0q5pbktq73efm697rl1tf18jv6dhknn(a)4ax.com... >> On Tue, 16 Mar 2010 20:16:44 -0500, "Peter Olcott" >> <NoSpam(a)OCR4Screen.com> wrote: >> >>> >>>"Hector Santos" <sant9442(a)nospam.gmail.com> wrote in >>>message >>>news:ueIgi6WxKHA.1692(a)TK2MSFTNGP04.phx.gbl... >>>> Peter Olcott wrote: >>>> >>>> >>>>> Fastest speed is highest priority, secondary to this is >>>>> portability, it looks like FastCGI provides both. >>>>> Another very high priority is minimal development >>>>> costs. >>>>> As far as I can tell FastCGI is good here too. >>>> >>>> Dude, FastCGI is not a "Language." Its an idea, a >>>> methodology that was designed with EARLY needs to speed >>>> up >>>> loading time. You won't have that problem today. >>> >>>You continue to ignore my repeatedly stated requirement of >>>4.0 GB of data (a deterministic finite automaton) that >>>must >>>be loaded before execution can begin. If you ignore my >>>fundamental requirements you can't possibly provide good >>>advice. >> *** >> And you continue to ignore our observations that there are >> no technologies that guarantee >> this, and the timings of program behavior can be swamped >> by communication times (did I >> tell you about the 70-second DNS resolution time? Did you >> know that you can expect delays >> in integer number of seconds to locate your server?) >> > >That is horrendous. I doubt if it is frequent, and local >caching would hopefully prevent it from recurring. **** Depends. Did the end user configure for local DNS caching? WHat was the cache retention time set by the end user? Note, again, YOU HAVE NO CONTROL OVER THESE PARAMETERS! **** > >> You fasten on some technology that says it is "faster" and >> assume it is the answer to your >> problem; you toss jargon around in ways that those of us >> who know the technology don't >> even recognize, such as assertions how FastCGI allows you >> to have browse buttons in the > >It seems to me that most of you guys continue to simply >ignore some of my most fundamental design requirements. App >load time for me can be several minutes, app execution time >of a large job is never more than 100 milliseconds. ***** No, we hear them, we just don't believe that FastCGI is going to be the solution that will solve every issue for you. Hector says that CGI has been improved to where it has comparable performance to FastCGI. I have no information to confirm or deny that statement. Again, you are not a server expert, so you should pay attention to the server expert. **** > >If the app is the only process besides the OS, by limiting >memory usage to no more than 75% of available RAM in actual >practice both the app and its data are paged out almost not >at all. On a real-time OS this behavior would be guaranteed. >On MS Windows it is close enough. Linux probably does not do >worse than Windows in this regard. **** I have no idea why you think this is true. This is not an assumption I would have considered valid until I talked to a linux expert (since you will be hosted on linux). Yet you make this assertion simply because you THINK it should work, as opposed to getting data that confirms that the OS actually works this way. **** > >> local Web server, and how it is the perfect answer to all >> your problems, when we know it >> isn't anything but another hack whose purpose is to >> compensate for bad initiali designs >> (CGI requiring a new process per request). You ignore >> core issues such as how you are >> going to port your app so it runs on a linux or Unix >> server, and how you are going to > >I am rewriting it to require very little OS specific code. >About the only thing that I will have to change is the way >that PNG files are read into memory. I will have to switch >to LibPNG. All of the MFC code is only development code and >will not be part of production. **** PNG files would not be "read into memory"; instead, you will need to convert the input string (however it is provided, most likely via stdin) to a bitmap, and that requires a library that does not read the PNG data from a file but from a text stream. **** > >> rewrite it to be platform-independent and possibly even >> GUI-free, and your unrealistic > >The web service needs no GUI. It is all PNG in, and UTF-8 >out. **** OK, that's good. You *will* have to write the FastCGI interface and make sure it works multithreaded so it can handle multiple requests. **** > >> expectations about how process memory is managed, and how >> every OS is magically going to >> behave in the way you wish, without any effort on your >> part. ANd you refuse to listen >> when people tell you that you don't understand either the >> technology or the intrinsic >> behavior of how an operating system works. >> >> You really need to do better research on these >> technologies before you start making such >> assertions, or ask questions and pay attention to the >> answers. I'm not even a Web server >> expert like Hector, and I can tell you are making >> nonsensical statements. Listen to him. >> He sounds like someone who does this for a living, and you >> should trust what he is telling >> you. >> joe > >All that I know for sure is that some of my fundamental >requirements are immutable. Some of Hector's advice seemed >to continue to ignore these requirements. I was envisioning >the web service as a single application. This may not have >been the way that he was envisioning it. **** You can do it by writing a "front end" that exports a known port and implements your appliction-specific protocol to get the data to send to your "business logic" component. It doesn't solve the paging problem, or the 4GB problem. joe **** > >> >>> >>> > Its nonsense to think this is "answer" to >>>> your problem. In addition, if you worry about >>>> processing >>>> speed, native C/C++ is your choice, so all you are doing >>>> is writing a CGI application, a console application that >>>> reads standard input. >>>> >>>> If you want productivity, learn PHP. Simple, well >>>> supported. Then if you really find out that FASTCGI is >>>> needed for your slow OS and machine, PHP supports it. >>>> >>>> The point is your SOLUTION is not FASTCGI, it is the >>>> actual application that does the work. >>>> >>>> -- >>>> HLS >>> >> Joseph M. Newcomer [MVP] >> email: newcomer(a)flounder.com >> Web: http://www.flounder.com >> MVP Tips: http://www.flounder.com/mvp_tips.htm > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Peter Olcott on 17 Mar 2010 15:40
"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in message news:u2a2q51uc73hcuqv16c4uld4s8957ud4hd(a)4ax.com... > See below... > On Tue, 16 Mar 2010 22:40:02 -0500, "Peter Olcott" > <NoSpam(a)OCR4Screen.com> wrote: > >> >>"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>message news:r9h0q55c6oml5fhpkrcfs2jtbsof7rf1ld(a)4ax.com... >>> See below... >>> On Tue, 16 Mar 2010 20:50:58 -0500, "Peter Olcott" >>> <NoSpam(a)OCR4Screen.com> wrote: >>> >>>> >>>>"Hector Santos" <sant9442(a)nospam.gmail.com> wrote in >>>>message >>>>news:%23qdTxMXxKHA.4492(a)TK2MSFTNGP05.phx.gbl... >>>>> Peter Olcott wrote: >>>>> >>>>> >>>>>> You continue to ignore my repeatedly stated >>>>>> requirement >>>>>> of 4.0 GB of data (a deterministic finite automaton) >>>>>> that >>>>>> must be loaded before execution can begin. If you >>>>>> ignore >>>>>> my fundamental requirements you can't possibly >>>>>> provide >>>>>> good advice. >>>>> >>>>> >>>>> Heed my advice if you want to get somewhere. Its >>>>> coming >>>>> for FREE! >>>>> >>>>> If you keeping 4 GB loaded under a 24x7 loaded exe , >>>>> then >>>>> the rest of your system will suffer and you will get >>>>> paging delays during idle times. >>>>> >>>>> That calls for a memory map design requirement. It has >>>>> nothing to do with FASTCGI unless youy think this >>>>> "FastCGI >>>>> Language" you wish to use provides a library for you >>>>> to >>>>> automatically do this for you - it does not. >>>>> Independent >>>>> concepts. >>>>> >>>>> -- >>>>> HLS >>>> >>>>Fundamentally FastCGI differs from CGI in that the app >>>>is >>>>held in memory rather than re-loaded every time. >>>>Although >>>>this feature may have limited alternatives, I see no >>>>reason >>>>to consider them. >>>> >>> **** >>> You obviously did not read the specs. NOWHERE does it >>> say >>> "the app is held in memory"; >>> specifically, >>> http://www.fastcgi.com/drupal/node/6?q=node/15 says >>> * FastCGI processes are persistent: after finishing a >>> request, they wait for a new >>> request instead of exiting. >> >>So if you really want to get nitpicky it is held in >>virtual >>memory (from the developer's point of view) that is mapped >>to RAM, if there is no reason to page it out. > **** > See what I mean? You are insisting that because YOU don't > see a reason that it should be > paged out, the OPERATING SYSTEM will do what YOU think it > ought to, and not page it out, > which flies in the face of reality, in which an operating > system will page out pages of an > idle process. So if I must I keep invoking in by another process to force it to keep its pages. Every five minutes add up all of your pixels, forcing it to hit every memory location. > **** >>Which >>essentially boils down to it is held in RAM. (Even though >>it >>can theoretically be paged out at any time, in actual >>practice this rarely occurs if you have enough RAM. >>Virtual >>Memory is an abstraction of RAM that can at times be >>ignored. > **** > Are you ABSOLUTELY CERTAIN that the host OS behaves > according to your dreams? It might > not, and it might not actually have tunable parameters you > can control! THis is what I > mean about fact-finding; you have to talk to the experts > (in this case, a linux expert) to > understand what is going to REALLY happen to an idle > process, how working set is managed, > etc., and not rely upon your unfounded opinion of what you > think SHOULD happen in the > best-of-all-possible-worlds scenario. Reality has this > really nasty habit of ignoring our > wishes. > joe > > **** >> >>> >>> >>> 1. The Web server creates FastCGI application processes >>> to >>> handle requests. The processes >>> may be created at startup, or created on demand. >>> 2. The FastCGI program initializes itself, and waits for >>> a >>> new connection from the Web >>> server. >>> 3. When a client request comes in, the Web server opens >>> a >>> connection to the FastCGI >>> process. The server sends the CGI environment variable >>> information and standard input over >>> the connection. >> >>Not in the case of FastCGI, FastCGI does not use >>environment >>variables. > **** > Perhaps you misread the point 3, which clearly states > "sends the CGI environment variable > INFORMATION...over the connection" (emphasis added, in > case you missed it). > ***** >> >>> 4. The FastCGI process sends the standard output and >>> error >>> information back to the server >>> over the same connection. >>> . When the FastCGI process closes the connection, the >>> request is complete. The FastCGI >>> process then waits for another connection from the Web >>> server. >>> >>> NOWHERE in this spec does is make ANY claim that the >>> "app >>> is held in memory", only that a >> >>That is logically entailed by the statement that the app >>does not terminate at the end of the request. With enough >>RAM and few processes running one can virtually ignore >>virtual memory. It may not be this way in theory, but, I >>know that it is this way in practice, because in practice >>I >>achieve the performance that could not be otherwise >>achieved. > ***** > No, it is NOT "logically entailed" by the staement that > the app does not terminate; this > again shows your failure to understand how real operating > systems work, which is how they > are designed to work, not how you wish they had been > designed to work. Windows, for > example, pages out the pages of an idle process, > particularly if they become LRU (but on > general principles, will page them out). Unix used to > work the same way. You have made > this incredible leap of faith based on what you WANT it to > do, with no input from a linux > expert about how it ACTUALLY works. > ***** >> >>> new app instance is not launched. Why you would leap to >>> the (erroneous) conclusion that >>> not starting a second app instance means the app is >>> "held >>> in memory" escapes me. What is >>> "held in memory" is up to the platform's specific >>> concepts >>> of address space management for >>> a process in the context of a certain physical memory >>> configuration, and it is likely that >>> some or all of the app's pages will be paged out to make >>> room for other processes. This >>> is your typical "design by wishful thinking" approach >>> where yu make assertions about how >> >>I think that you may be getting just a little bit too >>disrespectful. I do however really appreciate the support >>that you provided on the quality of my patent. > **** > But we keep telling you where you are going wrong, and you > keep insisting you must be > right, because your success depends on your assumptions > being right, even though most of I have seen my assumptions consistently be right, and I can force them to be right if needed. If the OS wants to page out an idle process then my process will automatically do busy work whenever it would otherwise be idle. If that does not work I will convert to a real-time OS. If that does not work I will write my own OS from scratch and make it work. Failure is not an option. > us know they are actually wrong, based on our own > experience. And when we tell you that > you are wrong, and why you are wrong, you come back and > tell us you *must* be right > because you *want* the behavior you hypothesize, even > though real systems don't work that > way! We are getting very frustrated with having to tell > you over and over again that your > assumptions are faulty, and it annoys us that you ignore > what we say and insist that the > operating system must behave the way you erroneously state > it behaves. I can see Hector > getting frustrated for the same reason. I've been through > this concerning memory > allocation and TCP/IP with you, and my patience is running > thin. You are now repeating > the scenario with Web servers, where I am expert enough to > know you are making nonsensical > statements and Hector actually is an expert and is telling > you (a) you are wrong and (b) > how to do it right, and you keep making even more confused > statements. > joe > > ***** >> >>> things work, which are wrong, and then proceed to >>> systematically ignore eveyone who tells >>> you "but it doesn't work that way". This discussion is >>> shaping up just like the "physical >>> contiguous memory" discussion of a few months ago where >>> you insisted that storage >>> allocators *had* to work in a way that, historically, >>> storage allocators have NEVER >>> worked, and that virtual memory had to work in a fashion >>> that has never been recognized as >>> a way any operating system designer would implement >>> virtual memory. Now you are telling >>> us how FASTCGI works, which is, unfortunately, not the >>> way >>> it works at all. And that it >>> will solve problems it is incapable of (not to mention >>> irrelevant to) solving. >>> >>> So start paying attention to the answers. It will save >>> you a lot of time and effort as >>> you attempt to make technologies do things they never >>> were >>> designed to do, or try to get >>> them to meet specifications they never pretended to >>> meet. >>> joe >>> ***** >>> Joseph M. Newcomer [MVP] >>> email: newcomer(a)flounder.com >>> Web: http://www.flounder.com >>> MVP Tips: http://www.flounder.com/mvp_tips.htm >> > Joseph M. Newcomer [MVP] > email: newcomer(a)flounder.com > Web: http://www.flounder.com > MVP Tips: http://www.flounder.com/mvp_tips.htm |