Can extra processing threads help in this case?
in [MFC]
|
From: Joseph M. Newcomer on 29 Mar 2010 14:19 See below... On Mon, 29 Mar 2010 10:31:26 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >message news:lm00r5ddh8n84jmmd65o6n7do5crl4tenq(a)4ax.com... >> Ee below... >> On Thu, 25 Mar 2010 14:19:58 -0500, "Peter Olcott" >> <NoSpam(a)OCR4Screen.com> wrote: >> >>>Right and depending on what is being accomplished one >>>might >>>discard non crucial integrity for speed. If the sequence >>>of >>>steps can be easily reconstructed because the request is >>>definitely committed to disk, then making sure that >>>intervening steps are also committed is not as crucial. >> **** >> And how is this consistent with your new magical >> buzzphrase "fault tolerant"? Either you >> want reliability, or you don't. > >False dilemma fallacy. I want a specific degree of fault >tolerance, no more and no less. **** WHich apparently changes every time you define your "fault tolerance". And you have not indicated what price you are willing to pay for it (complexity, development time, performance) **** > >> If you want fault tolerance, you are going to have to hit >> the disk; > >At least to record incoming transactions, this disk hit can >not possibly be avoided and have minimal fault tolerance >maintained. This disk hit must be a fully committed write >operation. > **** Yep. And did you remember my story about the hard drives that claim "commit" when they mean "data is in the onboard cache and that's all you need to know" and will not guarantee or give a means to guarantee that the transaction has actually reached the magnetic surface. Have fun! joe **** > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 29 Mar 2010 14:24 See below... On Mon, 29 Mar 2010 10:33:29 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >message news:g710r593g0pnhaul2jkdc0rbvmosaqdj8e(a)4ax.com... >> See below... >> On Thu, 25 Mar 2010 09:35:19 -0500, "Peter Olcott" >> <NoSpam(a)OCR4Screen.com> wrote: >> >>>> EnqueueRequest() which takes a description of the >>>> request. >>>> And is implemented on each >>>> platform with platform-specific code that is optimized >>>> for >>>> that platform. Your concept of >>>> "Portable" is incorrect here. you want the FASTEST way >>>> that the platform supports, and >>>> that may be completely different from every other >>>> platform. But if you code has a single >>>> abstraction, you are free to implement it in the best >>>> way >>>> on each platform! >>> >>>Is this supported on every platform? >> *** >> That's my entire point: you determine, for every platform, >> what 'this' is, and you WRITE >> CODE TO SUPPORT IT! In the abstract model, you have a >> system, perhaps even MySQL, but it >> is considered a low-level transport mechanism completely >> invisible to you your application >> architecture! You have two abstraction: "Enqueue" and >> "Dequeue" and that's what you write >> your application in terms of. On some systems, if you >> don't care in the slightest about >> performance, you might implment them as MySQL calls. In >> systems that have >> high-performance interprocess communication, you implement >> a platform-specific mechanism >> that is blindingly fast. Or have you missed the >> fundamental concepts of "abstraction"? >> joe >> ***** > >I would prefer to start with something that already works if >there already is such a thing. **** I am not aware of anything that is truly portable and works on all operating systems. And if you use MySQL, is the initialization/connection code the same on all platforms? Have you looked at what it takes to establish a database connection on Windows? Do you know if linux works with identical APIs? [I doubt it] So I'm not at all sure how MySQL is "completely portable". I'd work from the premise that there is NO portable IPC mechanism, and work with experts on each platform to create one that implements my own abstraction. My experience was that this was actually fairly easy, especially if you are sitting across the table from the expert on the other system. You can design and implement in minutes. joe **** > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 29 Mar 2010 14:25 Actually, the correct mechanism is (a) embedded server: handle WM_POWERBROADCAST, gracefully shut down (b) external server: handle WM_POWERBROADCAST, use IPC mechanism to inform child processes to gracefully shut down. Pretty simple. joe On Mon, 29 Mar 2010 01:14:20 -0400, Hector Santos <sant9442(a)nospam.gmail.com> wrote: >Joseph M. Newcomer wrote: > >> My servers are all on UPS units and get notification if power is going to fail in the near >> future; robust code handles WM_POWERBROADCAST messages. > > >Exactly Joe, but he has to program for that! :) > >So unless he finds a utility that captures the signal and sends mouse >strokes to close his vaporware application, it ain't going to happen. Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 29 Mar 2010 14:27 See below... On Mon, 29 Mar 2010 10:47:43 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >message news:q230r5hp161iibaml6pif093en5dti0t2r(a)4ax.com... >> See below... >> On Thu, 25 Mar 2010 10:35:27 -0400, Hector Santos >> <sant9442(a)nospam.gmail.com> wrote: >> >>>Peter Olcott wrote: >>> >>>Yes, the world has already put aside this Kindergarden >>>concept! and NO >>>ONE IN THE WORLD uses SQL for a FIFO! If they do, they >>>are PRIMITIVE. >> **** >> Banking software uses features in products like ORACLE to >> guarantee transacted access to >> the database of pending transactions, and the audit trail >> that follows these transactions. >> So while they aren't always using SQL, they are using >> transacted databases. Slow as hell, >> of course, but utterly robust, because the transactions >> either get completed or rolled >> back, and the multiple servers that work on the queue can >> handle the fact that if server 1 >> removes the request, and then crashes, the transaction >> that removed the request is rolled >> back and the pending transaction reappears in the >> database. But this requirement is >> incosnsitent with the requirement that there be no disk >> accesses. But we expect >> consistent requirements? SIlly us! >> **** > >I will need fault tolerance that is as reliable as bank >fault tolerance. The specific transaction will be that every >client request will cost a dime. This dime will be deducted >from the customer's account balance. I can expect at most >100 transactions per second because this is the maximum >processing load of the OCR software on a single server when >processing requests are of minimal size. **** 100 ms = 10/sec. 500ms = 2/sec. Read my previous answers on approaches for detecting completion. joe > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Hector Santos on 29 Mar 2010 14:31
Kiddle stuff. :) Joseph M. Newcomer wrote: > Actually, the correct mechanism is > > (a) embedded server: handle WM_POWERBROADCAST, gracefully shut down > (b) external server: handle WM_POWERBROADCAST, use IPC mechanism to inform child processes > to gracefully shut down. > > Pretty simple. > joe > > On Mon, 29 Mar 2010 01:14:20 -0400, Hector Santos <sant9442(a)nospam.gmail.com> wrote: > >> Joseph M. Newcomer wrote: >> >>> My servers are all on UPS units and get notification if power is going to fail in the near >>> future; robust code handles WM_POWERBROADCAST messages. >> >> Exactly Joe, but he has to program for that! :) >> >> So unless he finds a utility that captures the signal and sends mouse >> strokes to close his vaporware application, it ain't going to happen. > Joseph M. Newcomer [MVP] > email: newcomer(a)flounder.com > Web: http://www.flounder.com > MVP Tips: http://www.flounder.com/mvp_tips.htm -- HLS |