How hard is socket programming?
in [MFC]
|
Prev: Runtime Error does not specify program???
Next: windows 7 - imitating touch by sending wm_gesture ?
From: Joseph M. Newcomer on 19 Mar 2010 00:10 from my linux expert: linux uses a "two timer" approach. When the firt timer fires, the code runs thru the page list and notes which pages haven't changed. Any page that hasn't changed is marked as a candidate for page-out. any page that has changed is removed from the page-out list. When the second timer fires, pages that have been marked are paged out. If a page is used before it is paged out, because the first timer is still running, it is removed from the candidate list for page-out. Pages which are paged out are candidates for re-use. If the page frame is re-used for another page, the page will have to be brought back into memory; otherwise, it takes a "soft" page fault and the page fault finds the page which still happens to be in memory and does the appropriate fixup of the page table. So it is a hybrid of eager page-out and page retention. joe On Thu, 18 Mar 2010 11:40:43 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Hector Santos" <sant9442(a)nospam.gmail.com> wrote in message >news:OLClPfrxKHA.404(a)TK2MSFTNGP02.phx.gbl... >> Joe, Peter, is it just a habit, but you both have to quote >> the entire message just to embed a few lines of response >> in the middle of the message? It becomes a hunting game! >> Look for my replies below. >> > >(1) Its less trouble for me to do it this way. >(2) When I look for replies I use page-down, not >scroll-down, so finding the reply is very quick. >(3) It retains the entire context of every reply, thus >easily verifying who is at fault in communication errors. >(4) Text takes very little room or bandwidth, so the cost of >these resources is more than compensated by the savings in >human labor. > >> Honestly, the NNTP Server would work more efficiently. :) >> >> Peter Olcott wrote: >> >>> "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>> message >>> news:f9h4q5ltbdi70eot2n5me08gij5vg556bc(a)4ax.com... >>>> See below... >>>> On Wed, 17 Mar 2010 15:04:40 -0500, "Peter Olcott" >>>> <NoSpam(a)OCR4Screen.com> wrote: >>>> >>>>> "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>>>> message >>>>> news:43b2q51hukg9s1gofoqds7vodf0lcq4bhp(a)4ax.com... >>>>>> See below... >>>>>> On Tue, 16 Mar 2010 22:19:25 -0500, "Peter Olcott" >>>>>> <NoSpam(a)OCR4Screen.com> wrote: >>>>>> >>>>>>> "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>>>>>> message >>>>>>> news:6ng0q5pbktq73efm697rl1tf18jv6dhknn(a)4ax.com... >>>>>>>> On Tue, 16 Mar 2010 20:16:44 -0500, "Peter Olcott" >>>>>>>> <NoSpam(a)OCR4Screen.com> wrote: >>>>>>>> >>>>>>>>> "Hector Santos" <sant9442(a)nospam.gmail.com> wrote >>>>>>>>> in >>>>>>>>> message >>>>>>>>> news:ueIgi6WxKHA.1692(a)TK2MSFTNGP04.phx.gbl... >>>>>>>>>> Peter Olcott wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Fastest speed is highest priority, secondary to >>>>>>>>>>> this >>>>>>>>>>> is >>>>>>>>>>> portability, it looks like FastCGI provides both. >>>>>>>>>>> Another very high priority is minimal development >>>>>>>>>>> costs. >>>>>>>>>>> As far as I can tell FastCGI is good here too. >>>>>>>>>> Dude, FastCGI is not a "Language." Its an idea, a >>>>>>>>>> methodology that was designed with EARLY needs to >>>>>>>>>> speed >>>>>>>>>> up >>>>>>>>>> loading time. You won't have that problem today. >>>>>>>>> You continue to ignore my repeatedly stated >>>>>>>>> requirement >>>>>>>>> of >>>>>>>>> 4.0 GB of data (a deterministic finite automaton) >>>>>>>>> that >>>>>>>>> must >>>>>>>>> be loaded before execution can begin. If you ignore >>>>>>>>> my >>>>>>>>> fundamental requirements you can't possibly provide >>>>>>>>> good >>>>>>>>> advice. >>>>>>>> *** >>>>>>>> And you continue to ignore our observations that >>>>>>>> there >>>>>>>> are >>>>>>>> no technologies that guarantee >>>>>>>> this, and the timings of program behavior can be >>>>>>>> swamped >>>>>>>> by communication times (did I >>>>>>>> tell you about the 70-second DNS resolution time? >>>>>>>> Did >>>>>>>> you >>>>>>>> know that you can expect delays >>>>>>>> in integer number of seconds to locate your server?) >>>>>>>> >>>>>>> That is horrendous. I doubt if it is frequent, and >>>>>>> local >>>>>>> caching would hopefully prevent it from recurring. >>>>>> **** >>>>>> Depends. Did the end user configure for local DNS >>>>>> caching? WHat was the cache retention >>>>>> time set by the end user? Note, again, YOU HAVE NO >>>>>> CONTROL OVER THESE PARAMETERS! >>>>>> **** >>>>>>>> You fasten on some technology that says it is >>>>>>>> "faster" >>>>>>>> and >>>>>>>> assume it is the answer to your >>>>>>>> problem; you toss jargon around in ways that those >>>>>>>> of us >>>>>>>> who know the technology don't >>>>>>>> even recognize, such as assertions how FastCGI >>>>>>>> allows >>>>>>>> you >>>>>>>> to have browse buttons in the >> >> sure >> >>>>> It seems to me that most of you guys continue to >> >>>>> simply >> >>>>>>> ignore some of my most fundamental design >>>>>>> requirements. >>>>>>> App >>>>>>> load time for me can be several minutes, app >>>>>>> execution >>>>>>> time >>>>>>> of a large job is never more than 100 milliseconds. >>>>>> ***** >>>>>> No, we hear them, we just don't believe that FastCGI >>>>>> is >>>>>> going to be the solution that will >>>>>> solve every issue for you. Hector says that CGI has >>>>>> been >>>>>> improved to where it has >>>>>> comparable performance to FastCGI. I have no >>>>>> information >>>>>> to confirm or deny that >>>>>> statement. Again, you are not a server expert, so you >>>>>> should pay attention to the server >>>>>> expert. >>>>>> **** >>>>>>> If the app is the only process besides the OS, by >>>>>>> limiting >>>>>>> memory usage to no more than 75% of available RAM in >>>>>>> actual >>>>>>> practice both the app and its data are paged out >>>>>>> almost >>>>>>> not >>>>>>> at all. On a real-time OS this behavior would be >>>>>>> guaranteed. >>>>>>> On MS Windows it is close enough. Linux probably does >>>>>>> not >>>>>>> do >>>>>>> worse than Windows in this regard. >>>>>> **** >>>>>> I have no idea why you think this is true. This is >>>>>> not an >>>>>> assumption I would have >>>>>> considered valid until I talked to a linux expert >>>>>> (since >>>>>> you will be hosted on linux). Yet >>>>>> you make this assertion simply because you THINK it >>>>>> should >>>>>> work, as opposed to getting >>>>>> data that confirms that the OS actually works this >>>>>> way. >>>>>> **** >> >> whatever >> >>>>> An OS that unloads any part of an executing process >>>>> when it >>>>> has no need for additional memory would be an idiotic >>>>> OS. It >>>>> would be flat out stupid to do this. I am not convinced >>>>> that >>>>> either Windows or Linux is this stupid. >>>> *** >>>> But we learned DECADES ago that "eager page-out" >>>> improves overall system performance, >>>> because it reduces startup transients when a process (or >>>> one of its threads) becomes >>>> schedulable. Again, the "working set" has to be paged >>>> in, and this is easier if there are >>>> available page frames in memory. So what you are >>>> calling "stupid" is in fact the >>>> DEMONSTRATED INTELLIGENT way to build operating systems, >>>> and we've known this since they >>>> early 1970s. This is why I have never known a system >>>> that did not page out an idle >>>> process IN ANITICIPATION OF NEED. Windows does it; I'm >>>> trying to find out if linux does >>>> it, but again, your opinion of how you THINK an >>>> operating system SHOULD work flies in the >>>> fact of reality. Just because you don't see a reason >>>> doesn't mean there isn't one, and >>>> one that has been established by long practice (the >>>> "actual practice" you fantasize about >>>> doesn't exist). Windows indeed works this way, and so >>>> have other operating systems, all >>>> based on BEST practice (not someone's fantasized "actual >>>> practice"). So I would not >>>> predicate actual behavior on fantasies, but learn the >>>> facts. >>>> ***** >>> >>> I will simply do whatever it takes to achieve the >>> functional result that I require. If there is always an >>> extra 4.0 GB of RAM available, then your whole >>> anticipation of need concept should become moot. >>> >>>>> If this same OS unloads part of a process because the >>>>> process has been idle for a long time, AND needs more >>>>> memory >>>>> to do some housekeeping task, then this case I could >>>>> see a >>>>> reason that would not be stupid. >>>> **** >>>> It is the antiipation-of-need which in actual practice >>>> (not fantasized "actual practive", >>>> but ACTUAL practice) has been found to be the best >>>> solution for general purpose operating >>>> systems, to maximize overall performance. Until you >>>> have measured (there's that nasty >>>> phrase, implying the obtaining of facts) behavior, you >>>> have no reason to assume it >>>> corresponds to your fantasies. >>>> **** >>>>>>>> local Web server, and how it is the perfect answer >>>>>>>> to >>>>>>>> all >>>>>>>> your problems, when we know it >>>>>>>> isn't anything but another hack whose purpose is to >>>>>>>> compensate for bad initiali designs >>>>>>>> (CGI requiring a new process per request). You >>>>>>>> ignore >>>>>>>> core issues such as how you are >>>>>>>> going to port your app so it runs on a linux or Unix >>>>>>>> server, and how you are going to >>>>>>> I am rewriting it to require very little OS specific >>>>>>> code. >>>>>>> About the only thing that I will have to change is >>>>>>> the way >>>>>>> that PNG files are read into memory. I will have to >>>>>>> switch >>>>>>> to LibPNG. All of the MFC code is only development >>>>>>> code >>>>>>> and >>>>>>> will not be part of production. >>>>>> **** >>>>>> PNG files would not be "read into memory"; instead, >>>>>> you >>>>>> will need to convert the input >>>>>> string (however it is provided, most likely via stdin) >>>>>> to >>>>>> a bitmap, and that requires a >>>>>> library that does not read the PNG data from a file >>>>>> but >>>>>> from a text stream. >>>>>> **** >>>>> YES, or if I have to write is to disk and read it back >>>>> in >>>>> again. Reading and writing 2K files will not kill me. >>>>> >>>>>>>> rewrite it to be platform-independent and possibly >>>>>>>> even >>>>>>>> GUI-free, and your unrealistic >>>>>>> The web service needs no GUI. It is all PNG in, and >>>>>>> UTF-8 >>>>>>> out. >>>>>> **** >>>>>> OK, that's good. You *will* have to write the FastCGI >>>>>> interface and make sure it works >>>>>> multithreaded so it can handle multiple requests. >>>>> I am going with Hectors idea of a tiny embedded >>>>> webserver >>>>> built in to my code. There are several freeware one's >>>>> one >>>>> of these looks good enogh for my needs: mongoose. >>>>> >>>>>> **** >>>>>>>> expectations about how process memory is managed, >>>>>>>> and >>>>>>>> how >>>>>>>> every OS is magically going to >>>>>>>> behave in the way you wish, without any effort on >>>>>>>> your >>>>>>>> part. ANd you refuse to listen >>>>>>>> when people tell you that you don't understand >>>>>>>> either >>>>>>>> the >>>>>>>> technology or the intrinsic >>>>>>>> behavior of how an operating system works. >>>>>>>> >>>>>>>> You really need to do better research on these >>>>>>>> technologies before you start making such >>>>>>>> assertions, or ask questions and pay attention to >>>>>>>> the >>>>>>>> answers. I'm not even a Web server >>>>>>>> expert like Hector, and I can tell you are making >>>>>>>> nonsensical statements. Listen to him. >>>>>>>> He sounds like someone who does this for a living, >>>>>>>> and >>>>>>>> you >>>>>>>> should trust what he is telling >>>>>>>> you. >>>>>>>> joe >>>>>>> All that I know for sure is that some of my >>>>>>> fundamental >>>>>>> requirements are immutable. Some of Hector's advice >>>>>>> seemed >>>>>>> to continue to ignore these requirements. I was >>>>>>> envisioning >>>>>>> the web service as a single application. This may not >>>>>>> have >>>>>>> been the way that he was envisioning it. >>>>>> **** >>>>>> You can do it by writing a "front end" that exports a >>>>>> known port and implements your >>>>>> appliction-specific protocol to get the data to send >>>>>> to >>>>>> your "business logic" component. >>>>>> It doesn't solve the paging problem, or the 4GB >>>>>> problem. >>>>>> joe >>>>> I think that I will be going with some sort of >>>>> application >>>>> specific webserver such as mongoose. Hector tried to >>>>> sell me >>>>> on his webserver, but, it was too costly and not >>>>> flexible >>>>> enough for me right now. In the future when I scale up >>>>> to >>>>> multiple servers this may change. >>>>> >>>>>> **** >>>>>>>>>> Its nonsense to think this is "answer" to >>>>>>>>>> your problem. In addition, if you worry about >>>>>>>>>> processing >>>>>>>>>> speed, native C/C++ is your choice, so all you are >>>>>>>>>> doing >>>>>>>>>> is writing a CGI application, a console >>>>>>>>>> application >>>>>>>>>> that >>>>>>>>>> reads standard input. >>>>>>>>>> >>>>>>>>>> If you want productivity, learn PHP. Simple, well >>>>>>>>>> supported. Then if you really find out that >>>>>>>>>> FASTCGI >>>>>>>>>> is >>>>>>>>>> needed for your slow OS and machine, PHP supports >>>>>>>>>> it. >>>>>>>>>> >>>>>>>>>> The point is your SOLUTION is not FASTCGI, it is >>>>>>>>>> the >>>>>>>>>> actual application that does the work. >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> HLS >>>>>>>> Joseph M. Newcomer [MVP] >>>>>>>> email: newcomer(a)flounder.com >>>>>>>> Web: http://www.flounder.com >>>>>>>> MVP Tips: http://www.flounder.com/mvp_tips.htm >>>>>> Joseph M. Newcomer [MVP] >>>>>> email: newcomer(a)flounder.com >>>>>> Web: http://www.flounder.com >>>>>> MVP Tips: http://www.flounder.com/mvp_tips.htm >>>> Joseph M. Newcomer [MVP] >>>> email: newcomer(a)flounder.com >>>> Web: http://www.flounder.com >>>> MVP Tips: http://www.flounder.com/mvp_tips.htm >>> >>> >> >> >> >> -- >> HLS > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 19 Mar 2010 00:20 See below... On Thu, 18 Mar 2010 17:41:27 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: >>>process in memory even if I have to make changes to the OS >>>to achieve this. The simplest way that I can force the >>>data >>>to remain resident, is to access every memory location >>>whenever my process is otherwise idle. >> **** >> And you know linux support this? > >If Linux has threads, then it must support this. If it does >not have threads then IPC from another process would support >this. **** I know of nothing that would suggest this is possible with or without multithreading support. Or that lack of multithreading support would preclude it. I just don't follow the reasoning here. joe **** > >> >> And you know the host company will make the necessary >> configuration changes? >> joe > >Its only a tedious detail. I can always do the whole thing >myself (buy my own connection to the web) as a last resort, >its not really all that difficult. I am betting that the >half hour test will pass. **** Hosting companies are often very specific about what they provide. Tuning the OS parameters can well be an "added cost" support operation which is not available at the lower-cost price tier. I know my ISP has support tiers, and what you get at the lowest level is not all that much. Fortunately, my Web support requirements are minimal, but if I wanted to support Active Server Pages or needed anything beyond trivial CGI support, I would have to move to a higher-priced support package. Just a warnning, make sure you know what you are getting for the price you are paying, and be prepared to pay extra for customization. joe ****p://www.flounder.com/mvp_tips.htm >>>>> >> Joseph M. Newcomer [MVP] >> email: newcomer(a)flounder.com >> Web: http://www.flounder.com >> MVP Tips: http://www.flounder.com/mvp_tips.htm > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 19 Mar 2010 00:31 See below... On Thu, 18 Mar 2010 17:33:41 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >message news:k245q592jsdhrhg1ccberqg1440ltvv0fr(a)4ax.com... >> Apparently you aren't paying attention. Windows will swap >> out pages of an idle process. >> You can see this if you have a lot of processes, and >> switch back to Word or PowerPoint, or >> VS; there is a huge delay in reactivation. even if none of >> the other processes have >> executed, because it has to bring in the pages of the >> working set before it can activate >> the thread. > >So why would not simply keeping it always active with busy >work (whenever it would be otherwise idle) solve my problem? >I will try your little half hour thing. I am guessing that >your experience with this issue is derived from not having >plenty of excess RAM. **** All my base systems run with 4GB; my new Win7 system (dual quad-core Xeons) which will arrive this week will have 8GB. MY smallest system is my laptop that is a Core Duo with a mere 2GB of memory. BUt I mostly use it as a thin client to my desktops, and don't do serious work on it as a computer. joe **** > >> >> rather than guess at what you think should happen, rely on >> the experts, e.g., Russinovich >> & Solomon, WIndows Internals (5th edition) Micrsoft Press, >> 2009, start about page 812 >> ("Modified Page Writer") to understand how paging ACTUALLY >> works in Windows. Also read pp >> 822 ("WOrking Sets" and 823 ("Logical prefetcher"), page >> 824 which talks about the paging >> parameters in the Registry, page 828 ("Working Set >> Management") page 831 ("Balance Set >> Manager and Swapper") or page 836 ("Proactive Memory >> Management (SuperFetch)") or just >> read all of Chapter 9 ("Memory Management"). And it >> wouldn't hurt to read chapter 10 >> ("Cache Management"). >> >> You cannot pretend to be a serious developer who cares >> about WIndows Internals without >> owning this book. > >I care about windows internals about as much as my mother >cares about the details of how her car's transmission works. >I just want my OS to work, I don't care how it works. When >what I need it to do requires me to gain some understanding >about how it works, I gain just the bare minimum required to >get the job done. **** It is clear that you care a whole lot about Windows Internals, but instead of finding out what is really going on, you hypothesize about how it "ought" to work in some imaginary world. If you don't care about WIndows Internals, why are you caring so much about how, for example, paging works? You clearly care, but you have not used the proper available resources to read about the questions, and when those of us who HAVE read the books tell you what is going on, you keep insisting that our image is wrong and your image is right. So don't pretend you don't care; you care a lot, and have made it clear that you care a lot. And I know the basis of how an automatic transmission works, and I can tell you how (in theory) to adjust a timing belt using a strobe light synced to a particular spark plug, but it doesn't mean I do my own car maintenance. And I outsource my site management to experts, too. **** > >I am not at all interesting in memorizing thousands of >obscure facts, what I am interested in is creating new >software technology. **** At the interface, where your software technology meets the OS, it is important to know what is actually happening, because your metric for success REQUIRES this knowledge. I acquire the knowlege because I'm a factoid pack-rat and collect these things, but it also means that when I really, really care about something the first thing I do is look it up. joe **** > > >> Joseph M. Newcomer [MVP] >> email: newcomer(a)flounder.com >> Web: http://www.flounder.com >> MVP Tips: http://www.flounder.com/mvp_tips.htm > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 19 Mar 2010 00:40 Not correct, see below... On Thu, 18 Mar 2010 11:05:19 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >message news:m5h4q5du9bs9cev94k4etlsjv77f4o4r3s(a)4ax.com... >> Only if the working set size the OS maintains exceeds your >> application footprint. >> Otherwise, all you do is page-thrash your own app. >> joe >> > >You NEVER thrash if you always have far more physical memory >than you need. If you do thrash when you always have far >more physical memory than you need, the OS was designed by >morons. **** Providing you ignore fundamental concepts like "working set"; if you exceed your working set size you will thrash your own app, no matter how much physical storage you have. You keep insisting that operating systems work according to your fantasies. This is another case of the "wishful thinking" approach to understanding complex operating system behavior. In this case, you are ignoring a critical parameter, working set quota. But I have already explained working set, an idea Peter Denning explained in 1972. That's 38 year ago. Of course, since I've been doing this 47 years, I tend to remember the watershed events that changed how we built systems, and Denning's paper was historically one of the most important papers in the history of OS design. Every OS since has implemented the concept, and I don't understand why you think it can be ignored as a piece of the reality of how operating systems work. joe **** > >> Joseph M. Newcomer [MVP] >> email: newcomer(a)flounder.com >> Web: http://www.flounder.com >> MVP Tips: http://www.flounder.com/mvp_tips.htm > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Peter Olcott on 19 Mar 2010 11:12
"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in message news:71u5q59q4mi9k4uc4nmv7t6tkp0rdc7r7t(a)4ax.com... > from my linux expert: linux uses a "two timer" approach. > When the firt timer fires, the > code runs thru the page list and notes which pages haven't > changed. Any page that hasn't > changed is marked as a candidate for page-out. any page > that has changed is removed from > the page-out list. When the second timer fires, pages > that have been marked are paged > out. If a page is used before it is paged out, because > the first timer is still running, > it is removed from the candidate list for page-out. Pages > which are paged out are > candidates for re-use. If the page frame is re-used for > another page, the page will have > to be brought back into memory; otherwise, it takes a > "soft" page fault and the page fault > finds the page which still happens to be in memory and > does the appropriate fixup of the > page table. So it is a hybrid of eager page-out and page > retention. > joe > I extended your half hour test to 10 hours and nothing was paged out on Windows 7 with 8.0 GB RAM. To page out any processes with an extra 4.0 GB available would be like a guy with a million dollars in his bank account selling some of his furniture just in case he might need more money. All of your ideas and the ideas that you have presented from others are good ideas, but, probably should not apply when there are large amounts of excess RAM available. > On Thu, 18 Mar 2010 11:40:43 -0500, "Peter Olcott" > <NoSpam(a)OCR4Screen.com> wrote: > >> >>"Hector Santos" <sant9442(a)nospam.gmail.com> wrote in >>message >>news:OLClPfrxKHA.404(a)TK2MSFTNGP02.phx.gbl... >>> Joe, Peter, is it just a habit, but you both have to >>> quote >>> the entire message just to embed a few lines of response >>> in the middle of the message? It becomes a hunting >>> game! >>> Look for my replies below. >>> >> >>(1) Its less trouble for me to do it this way. >>(2) When I look for replies I use page-down, not >>scroll-down, so finding the reply is very quick. >>(3) It retains the entire context of every reply, thus >>easily verifying who is at fault in communication errors. >>(4) Text takes very little room or bandwidth, so the cost >>of >>these resources is more than compensated by the savings in >>human labor. >> >>> Honestly, the NNTP Server would work more efficiently. >>> :) >>> >>> Peter Olcott wrote: >>> >>>> "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>>> message >>>> news:f9h4q5ltbdi70eot2n5me08gij5vg556bc(a)4ax.com... >>>>> See below... >>>>> On Wed, 17 Mar 2010 15:04:40 -0500, "Peter Olcott" >>>>> <NoSpam(a)OCR4Screen.com> wrote: >>>>> >>>>>> "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>>>>> message >>>>>> news:43b2q51hukg9s1gofoqds7vodf0lcq4bhp(a)4ax.com... >>>>>>> See below... >>>>>>> On Tue, 16 Mar 2010 22:19:25 -0500, "Peter Olcott" >>>>>>> <NoSpam(a)OCR4Screen.com> wrote: >>>>>>> >>>>>>>> "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote >>>>>>>> in >>>>>>>> message >>>>>>>> news:6ng0q5pbktq73efm697rl1tf18jv6dhknn(a)4ax.com... >>>>>>>>> On Tue, 16 Mar 2010 20:16:44 -0500, "Peter Olcott" >>>>>>>>> <NoSpam(a)OCR4Screen.com> wrote: >>>>>>>>> >>>>>>>>>> "Hector Santos" <sant9442(a)nospam.gmail.com> wrote >>>>>>>>>> in >>>>>>>>>> message >>>>>>>>>> news:ueIgi6WxKHA.1692(a)TK2MSFTNGP04.phx.gbl... >>>>>>>>>>> Peter Olcott wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Fastest speed is highest priority, secondary to >>>>>>>>>>>> this >>>>>>>>>>>> is >>>>>>>>>>>> portability, it looks like FastCGI provides >>>>>>>>>>>> both. >>>>>>>>>>>> Another very high priority is minimal >>>>>>>>>>>> development >>>>>>>>>>>> costs. >>>>>>>>>>>> As far as I can tell FastCGI is good here too. >>>>>>>>>>> Dude, FastCGI is not a "Language." Its an idea, >>>>>>>>>>> a >>>>>>>>>>> methodology that was designed with EARLY needs >>>>>>>>>>> to >>>>>>>>>>> speed >>>>>>>>>>> up >>>>>>>>>>> loading time. You won't have that problem >>>>>>>>>>> today. >>>>>>>>>> You continue to ignore my repeatedly stated >>>>>>>>>> requirement >>>>>>>>>> of >>>>>>>>>> 4.0 GB of data (a deterministic finite automaton) >>>>>>>>>> that >>>>>>>>>> must >>>>>>>>>> be loaded before execution can begin. If you >>>>>>>>>> ignore >>>>>>>>>> my >>>>>>>>>> fundamental requirements you can't possibly >>>>>>>>>> provide >>>>>>>>>> good >>>>>>>>>> advice. >>>>>>>>> *** >>>>>>>>> And you continue to ignore our observations that >>>>>>>>> there >>>>>>>>> are >>>>>>>>> no technologies that guarantee >>>>>>>>> this, and the timings of program behavior can be >>>>>>>>> swamped >>>>>>>>> by communication times (did I >>>>>>>>> tell you about the 70-second DNS resolution time? >>>>>>>>> Did >>>>>>>>> you >>>>>>>>> know that you can expect delays >>>>>>>>> in integer number of seconds to locate your >>>>>>>>> server?) >>>>>>>>> >>>>>>>> That is horrendous. I doubt if it is frequent, and >>>>>>>> local >>>>>>>> caching would hopefully prevent it from recurring. >>>>>>> **** >>>>>>> Depends. Did the end user configure for local DNS >>>>>>> caching? WHat was the cache retention >>>>>>> time set by the end user? Note, again, YOU HAVE NO >>>>>>> CONTROL OVER THESE PARAMETERS! >>>>>>> **** >>>>>>>>> You fasten on some technology that says it is >>>>>>>>> "faster" >>>>>>>>> and >>>>>>>>> assume it is the answer to your >>>>>>>>> problem; you toss jargon around in ways that those >>>>>>>>> of us >>>>>>>>> who know the technology don't >>>>>>>>> even recognize, such as assertions how FastCGI >>>>>>>>> allows >>>>>>>>> you >>>>>>>>> to have browse buttons in the >>> >>> sure >>> >>>>> It seems to me that most of you guys continue to >>> >>>>> simply >>> >>>>>>>> ignore some of my most fundamental design >>>>>>>> requirements. >>>>>>>> App >>>>>>>> load time for me can be several minutes, app >>>>>>>> execution >>>>>>>> time >>>>>>>> of a large job is never more than 100 milliseconds. >>>>>>> ***** >>>>>>> No, we hear them, we just don't believe that FastCGI >>>>>>> is >>>>>>> going to be the solution that will >>>>>>> solve every issue for you. Hector says that CGI has >>>>>>> been >>>>>>> improved to where it has >>>>>>> comparable performance to FastCGI. I have no >>>>>>> information >>>>>>> to confirm or deny that >>>>>>> statement. Again, you are not a server expert, so >>>>>>> you >>>>>>> should pay attention to the server >>>>>>> expert. >>>>>>> **** >>>>>>>> If the app is the only process besides the OS, by >>>>>>>> limiting >>>>>>>> memory usage to no more than 75% of available RAM >>>>>>>> in >>>>>>>> actual >>>>>>>> practice both the app and its data are paged out >>>>>>>> almost >>>>>>>> not >>>>>>>> at all. On a real-time OS this behavior would be >>>>>>>> guaranteed. >>>>>>>> On MS Windows it is close enough. Linux probably >>>>>>>> does >>>>>>>> not >>>>>>>> do >>>>>>>> worse than Windows in this regard. >>>>>>> **** >>>>>>> I have no idea why you think this is true. This is >>>>>>> not an >>>>>>> assumption I would have >>>>>>> considered valid until I talked to a linux expert >>>>>>> (since >>>>>>> you will be hosted on linux). Yet >>>>>>> you make this assertion simply because you THINK it >>>>>>> should >>>>>>> work, as opposed to getting >>>>>>> data that confirms that the OS actually works this >>>>>>> way. >>>>>>> **** >>> >>> whatever >>> >>>>>> An OS that unloads any part of an executing process >>>>>> when it >>>>>> has no need for additional memory would be an idiotic >>>>>> OS. It >>>>>> would be flat out stupid to do this. I am not >>>>>> convinced >>>>>> that >>>>>> either Windows or Linux is this stupid. >>>>> *** >>>>> But we learned DECADES ago that "eager page-out" >>>>> improves overall system performance, >>>>> because it reduces startup transients when a process >>>>> (or >>>>> one of its threads) becomes >>>>> schedulable. Again, the "working set" has to be paged >>>>> in, and this is easier if there are >>>>> available page frames in memory. So what you are >>>>> calling "stupid" is in fact the >>>>> DEMONSTRATED INTELLIGENT way to build operating >>>>> systems, >>>>> and we've known this since they >>>>> early 1970s. This is why I have never known a system >>>>> that did not page out an idle >>>>> process IN ANITICIPATION OF NEED. Windows does it; >>>>> I'm >>>>> trying to find out if linux does >>>>> it, but again, your opinion of how you THINK an >>>>> operating system SHOULD work flies in the >>>>> fact of reality. Just because you don't see a reason >>>>> doesn't mean there isn't one, and >>>>> one that has been established by long practice (the >>>>> "actual practice" you fantasize about >>>>> doesn't exist). Windows indeed works this way, and so >>>>> have other operating systems, all >>>>> based on BEST practice (not someone's fantasized >>>>> "actual >>>>> practice"). So I would not >>>>> predicate actual behavior on fantasies, but learn the >>>>> facts. >>>>> ***** >>>> >>>> I will simply do whatever it takes to achieve the >>>> functional result that I require. If there is always an >>>> extra 4.0 GB of RAM available, then your whole >>>> anticipation of need concept should become moot. >>>> >>>>>> If this same OS unloads part of a process because the >>>>>> process has been idle for a long time, AND needs more >>>>>> memory >>>>>> to do some housekeeping task, then this case I could >>>>>> see a >>>>>> reason that would not be stupid. >>>>> **** >>>>> It is the antiipation-of-need which in actual practice >>>>> (not fantasized "actual practive", >>>>> but ACTUAL practice) has been found to be the best >>>>> solution for general purpose operating >>>>> systems, to maximize overall performance. Until you >>>>> have measured (there's that nasty >>>>> phrase, implying the obtaining of facts) behavior, you >>>>> have no reason to assume it >>>>> corresponds to your fantasies. >>>>> **** >>>>>>>>> local Web server, and how it is the perfect answer >>>>>>>>> to >>>>>>>>> all >>>>>>>>> your problems, when we know it >>>>>>>>> isn't anything but another hack whose purpose is >>>>>>>>> to >>>>>>>>> compensate for bad initiali designs >>>>>>>>> (CGI requiring a new process per request). You >>>>>>>>> ignore >>>>>>>>> core issues such as how you are >>>>>>>>> going to port your app so it runs on a linux or >>>>>>>>> Unix >>>>>>>>> server, and how you are going to >>>>>>>> I am rewriting it to require very little OS >>>>>>>> specific >>>>>>>> code. >>>>>>>> About the only thing that I will have to change is >>>>>>>> the way >>>>>>>> that PNG files are read into memory. I will have to >>>>>>>> switch >>>>>>>> to LibPNG. All of the MFC code is only development >>>>>>>> code >>>>>>>> and >>>>>>>> will not be part of production. >>>>>>> **** >>>>>>> PNG files would not be "read into memory"; instead, >>>>>>> you >>>>>>> will need to convert the input >>>>>>> string (however it is provided, most likely via >>>>>>> stdin) >>>>>>> to >>>>>>> a bitmap, and that requires a >>>>>>> library that does not read the PNG data from a file >>>>>>> but >>>>>>> from a text stream. >>>>>>> **** >>>>>> YES, or if I have to write is to disk and read it >>>>>> back >>>>>> in >>>>>> again. Reading and writing 2K files will not kill me. >>>>>> >>>>>>>>> rewrite it to be platform-independent and possibly >>>>>>>>> even >>>>>>>>> GUI-free, and your unrealistic >>>>>>>> The web service needs no GUI. It is all PNG in, and >>>>>>>> UTF-8 >>>>>>>> out. >>>>>>> **** >>>>>>> OK, that's good. You *will* have to write the >>>>>>> FastCGI >>>>>>> interface and make sure it works >>>>>>> multithreaded so it can handle multiple requests. >>>>>> I am going with Hectors idea of a tiny embedded >>>>>> webserver >>>>>> built in to my code. There are several freeware >>>>>> one's >>>>>> one >>>>>> of these looks good enogh for my needs: mongoose. >>>>>> >>>>>>> **** >>>>>>>>> expectations about how process memory is managed, >>>>>>>>> and >>>>>>>>> how >>>>>>>>> every OS is magically going to >>>>>>>>> behave in the way you wish, without any effort on >>>>>>>>> your >>>>>>>>> part. ANd you refuse to listen >>>>>>>>> when people tell you that you don't understand >>>>>>>>> either >>>>>>>>> the >>>>>>>>> technology or the intrinsic >>>>>>>>> behavior of how an operating system works. >>>>>>>>> >>>>>>>>> You really need to do better research on these >>>>>>>>> technologies before you start making such >>>>>>>>> assertions, or ask questions and pay attention to >>>>>>>>> the >>>>>>>>> answers. I'm not even a Web server >>>>>>>>> expert like Hector, and I can tell you are making >>>>>>>>> nonsensical statements. Listen to him. >>>>>>>>> He sounds like someone who does this for a living, >>>>>>>>> and >>>>>>>>> you >>>>>>>>> should trust what he is telling >>>>>>>>> you. >>>>>>>>> joe >>>>>>>> All that I know for sure is that some of my >>>>>>>> fundamental >>>>>>>> requirements are immutable. Some of Hector's advice >>>>>>>> seemed >>>>>>>> to continue to ignore these requirements. I was >>>>>>>> envisioning >>>>>>>> the web service as a single application. This may >>>>>>>> not >>>>>>>> have >>>>>>>> been the way that he was envisioning it. >>>>>>> **** >>>>>>> You can do it by writing a "front end" that exports >>>>>>> a >>>>>>> known port and implements your >>>>>>> appliction-specific protocol to get the data to send >>>>>>> to >>>>>>> your "business logic" component. >>>>>>> It doesn't solve the paging problem, or the 4GB >>>>>>> problem. >>>>>>> joe >>>>>> I think that I will be going with some sort of >>>>>> application >>>>>> specific webserver such as mongoose. Hector tried to >>>>>> sell me >>>>>> on his webserver, but, it was too costly and not >>>>>> flexible >>>>>> enough for me right now. In the future when I scale >>>>>> up >>>>>> to >>>>>> multiple servers this may change. >>>>>> >>>>>>> **** >>>>>>>>>>> Its nonsense to think this is "answer" to >>>>>>>>>>> your problem. In addition, if you worry about >>>>>>>>>>> processing >>>>>>>>>>> speed, native C/C++ is your choice, so all you >>>>>>>>>>> are >>>>>>>>>>> doing >>>>>>>>>>> is writing a CGI application, a console >>>>>>>>>>> application >>>>>>>>>>> that >>>>>>>>>>> reads standard input. >>>>>>>>>>> >>>>>>>>>>> If you want productivity, learn PHP. Simple, >>>>>>>>>>> well >>>>>>>>>>> supported. Then if you really find out that >>>>>>>>>>> FASTCGI >>>>>>>>>>> is >>>>>>>>>>> needed for your slow OS and machine, PHP >>>>>>>>>>> supports >>>>>>>>>>> it. >>>>>>>>>>> >>>>>>>>>>> The point is your SOLUTION is not FASTCGI, it is >>>>>>>>>>> the >>>>>>>>>>> actual application that does the work. >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> HLS >>>>>>>>> Joseph M. Newcomer [MVP] >>>>>>>>> email: newcomer(a)flounder.com >>>>>>>>> Web: http://www.flounder.com >>>>>>>>> MVP Tips: http://www.flounder.com/mvp_tips.htm >>>>>>> Joseph M. Newcomer [MVP] >>>>>>> email: newcomer(a)flounder.com >>>>>>> Web: http://www.flounder.com >>>>>>> MVP Tips: http://www.flounder.com/mvp_tips.htm >>>>> Joseph M. Newcomer [MVP] >>>>> email: newcomer(a)flounder.com >>>>> Web: http://www.flounder.com >>>>> MVP Tips: http://www.flounder.com/mvp_tips.htm >>>> >>>> >>> >>> >>> >>> -- >>> HLS >> > Joseph M. Newcomer [MVP] > email: newcomer(a)flounder.com > Web: http://www.flounder.com > MVP Tips: http://www.flounder.com/mvp_tips.htm |