What is the fastest sustainable web service response time?
in [MFC]
|
From: Peter Olcott on 23 Feb 2010 04:36 "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in message news:ss17o5ljcgg1cueekve99dvd30uh4v9t47(a)4ax.com... > The components involved in the response are: > Sender stack delay > Sender bandwidth to downstream server > Total multi-server latency to receiver downstream server > Receiver bandwidth to downstream server > Receiver stack delay > Receiver rendering time > > The stack delays exist and cannot be changed. They are > probably in hundreds of > microseconds but I have never measured them. > > "The last mile" certainly dominates. If you have dialup > internet vs. T3 hardwire you will > get different performance. > > But the interserver performance matters also. A ping, > which is a trivial packet, can turn > around in a few hundred milliseconds, e.g., > ================================================= > C:\Documents and Settings\email>ping google.com > > Pinging google.com [72.14.204.147] with 32 bytes of data: > > Reply from 72.14.204.147: bytes=32 time=145ms TTL=53 > Reply from 72.14.204.147: bytes=32 time=144ms TTL=53 > Reply from 72.14.204.147: bytes=32 time=142ms TTL=53 > Reply from 72.14.204.147: bytes=32 time=143ms TTL=53 > > Ping statistics for 72.14.204.147: > Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), > Approximate round trip times in milli-seconds: > Minimum = 142ms, Maximum = 145ms, Average = 143ms > > C:\Documents and Settings\email>ping verizon.net > > Pinging verizon.net [206.46.232.39] with 32 bytes of data: > > Reply from 206.46.232.39: bytes=32 time=203ms TTL=245 > Reply from 206.46.232.39: bytes=32 time=203ms TTL=245 > Reply from 206.46.232.39: bytes=32 time=200ms TTL=245 > Reply from 206.46.232.39: bytes=32 time=208ms TTL=245 > > Ping statistics for 206.46.232.39: > Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), > Approximate round trip times in milli-seconds: > Minimum = 200ms, Maximum = 208ms, Average = 203ms > > C:\Documents and Settings\email>ping > www.cityofsydney.nsw.gov.au > > Pinging www.cityofsydney.nsw.gov.au [203.147.135.212] with > 32 bytes of data: > > Reply from 203.147.135.212: bytes=32 time=389ms TTL=112 > Reply from 203.147.135.212: bytes=32 time=382ms TTL=112 > Reply from 203.147.135.212: bytes=32 time=381ms TTL=112 > Reply from 203.147.135.212: bytes=32 time=379ms TTL=112 > > Ping statistics for 203.147.135.212: > Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), > Approximate round trip times in milli-seconds: > Minimum = 379ms, Maximum = 389ms, Average = 382ms > > C:\Documents and Settings\email> > > ==================================== > > So note that a ping to halfway around the world takes > about 400ms. This suggests that > under similar conditions, at 3am local time, you *might* > be able to turn a packet around > in 500ms, but I wouldn't bet on it. Ah so ping can directly measure what I need to know. I just pinged seescreen.com and got 53 ms with 32 bytes and 56 ms with 1024 bytes, larger numbers of bytes timed out. What is the normal packet size for a web service? Is the total time that a file takes precisely proportional to the number of packets, times the ping time per packet? > > But it is unrealistic to assume such goals when they are > based on parameters you have no > control over. > > Your job is to turn the packet around in the minimum > compute time you can manage. > Everything else is beyond what you can control. > joe > ***** > > On Mon, 22 Feb 2010 23:54:04 -0600, "Peter Olcott" > <NoSpam(a)OCR4Screen.com> wrote: > >> >>"Ananth Ramasamy Meenachi" <msarm(a)live.com> wrote in >>message >>news:%23YtfKZEtKHA.3360(a)TK2MSFTNGP06.phx.gbl... >>> >>>> Is it possible for a very fast web service to >>>> consistently provide an average 500 millisecond >>>> response >>>> time? >>> It depends. The Internet Bandwidth is one of the major >>> factor which controls the effective response performance >>> (particularly the web Server). When you have enough >>> bandwidth @ both ends next comes the hardware (storage & >>> processing infrastructure). Best solution is going for >>> CLOUD computing. Check for AZURE platform. >>> >>> >>>> Is the internet itself too slow making this goal >>>> completely infeasible using current technology? >>> No way, any technology will fail to work with very low >>> internet connection. >>> >>> Let me know you objective so that I can explain some >>> more >>> on this. >> >>I want to provide a web service that takes a tiny image >>file >>(15 x (100 to 400) pixels) and returns 20 to 100 bytes of >>text that it derived using my proprietary OCR software. >>Ideally I want to do this with a real time limit of 500 >>milliseconds. The next best thing would be an average >>response rate less than 500 milliseconds. You can assume >>100 >>milliseconds for my process including translation to and >>from HTTP. >> >>> >>> "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote in message >>> news:W4mdnSJ8462Jxx7WnZ2dnUVZ_s-dnZ2d(a)giganews.com... >>>> Is it possible for a very fast web service to >>>> consistently provide an average 500 millisecond >>>> response >>>> time? >>>> >>>> Is the internet itself too slow making this goal >>>> completely infeasible using current technology? >>>> >>>> >> > Joseph M. Newcomer [MVP] > email: newcomer(a)flounder.com > Web: http://www.flounder.com > MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 23 Feb 2010 11:02 See below... On Tue, 23 Feb 2010 03:36:10 -0600, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >message news:ss17o5ljcgg1cueekve99dvd30uh4v9t47(a)4ax.com... >> The components involved in the response are: >> Sender stack delay >> Sender bandwidth to downstream server >> Total multi-server latency to receiver downstream server >> Receiver bandwidth to downstream server >> Receiver stack delay >> Receiver rendering time >> >> The stack delays exist and cannot be changed. They are >> probably in hundreds of >> microseconds but I have never measured them. >> >> "The last mile" certainly dominates. If you have dialup >> internet vs. T3 hardwire you will >> get different performance. >> >> But the interserver performance matters also. A ping, >> which is a trivial packet, can turn >> around in a few hundred milliseconds, e.g., >> ================================================= >> C:\Documents and Settings\email>ping google.com >> >> Pinging google.com [72.14.204.147] with 32 bytes of data: >> >> Reply from 72.14.204.147: bytes=32 time=145ms TTL=53 >> Reply from 72.14.204.147: bytes=32 time=144ms TTL=53 >> Reply from 72.14.204.147: bytes=32 time=142ms TTL=53 >> Reply from 72.14.204.147: bytes=32 time=143ms TTL=53 >> >> Ping statistics for 72.14.204.147: >> Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), >> Approximate round trip times in milli-seconds: >> Minimum = 142ms, Maximum = 145ms, Average = 143ms >> >> C:\Documents and Settings\email>ping verizon.net >> >> Pinging verizon.net [206.46.232.39] with 32 bytes of data: >> >> Reply from 206.46.232.39: bytes=32 time=203ms TTL=245 >> Reply from 206.46.232.39: bytes=32 time=203ms TTL=245 >> Reply from 206.46.232.39: bytes=32 time=200ms TTL=245 >> Reply from 206.46.232.39: bytes=32 time=208ms TTL=245 >> >> Ping statistics for 206.46.232.39: >> Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), >> Approximate round trip times in milli-seconds: >> Minimum = 200ms, Maximum = 208ms, Average = 203ms >> >> C:\Documents and Settings\email>ping >> www.cityofsydney.nsw.gov.au >> >> Pinging www.cityofsydney.nsw.gov.au [203.147.135.212] with >> 32 bytes of data: >> >> Reply from 203.147.135.212: bytes=32 time=389ms TTL=112 >> Reply from 203.147.135.212: bytes=32 time=382ms TTL=112 >> Reply from 203.147.135.212: bytes=32 time=381ms TTL=112 >> Reply from 203.147.135.212: bytes=32 time=379ms TTL=112 >> >> Ping statistics for 203.147.135.212: >> Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), >> Approximate round trip times in milli-seconds: >> Minimum = 379ms, Maximum = 389ms, Average = 382ms >> >> C:\Documents and Settings\email> >> >> ==================================== >> >> So note that a ping to halfway around the world takes >> about 400ms. This suggests that >> under similar conditions, at 3am local time, you *might* >> be able to turn a packet around >> in 500ms, but I wouldn't bet on it. > >Ah so ping can directly measure what I need to know. I just >pinged seescreen.com and got 53 ms with 32 bytes and 56 ms >with 1024 bytes, larger numbers of bytes timed out. What is >the normal packet size for a web service? Is the total time >that a file takes precisely proportional to the number of >packets, times the ping time per packet? **** Not quite. ping measures ONE instance of ONE protocol, a protocol that is handled fairly low in the stack and doesn't involve application response (which includes time to bring in pages of the application, etc.). Therefore, it is a sort-of-indicator of network traffic delays and nothing else. Note that doing a ping at different times of day could produce different results because of backbone traffic. For example, if done while hundreds of thousands of users are streaming live Winter Olympics feeds would probably give different results, especially if pinging to the Pacific Northwest. As such, it is an INDICATOR. It is not a precise measure, and therefore is only SUGGESTIVE of performance. Your Mileage May Vary. Does Not Include Dealer Prep. Offer Void Where Prohibited By Law. Backbone traffic, retransmit times if packets are received in error (e.g., potential satellite feeds where upper atmosphere effects could scramble a packet, or a flock of birds flying in front of a microwave tower), and similar vagaries will introduce additional delays. ping is a UDP-class protocol, not TCP/IP, so there are also ack/nak time issues involved when you use TCP/IP. Sliding window protocols and piggybacking of ack/nak improve TCP/IP performance. Packet reassembly and multipath delays decrease it. If your server is heavily loaded, it will delay senders by throttling transmission until it can handle the load. Also, ping is connectionless; in TCP/IP you have to establish a connection, which is a whole set of packets flying back and forth just to activate the connection. These packets will have delays comparable to ping, so you might have several roundtrip delays. Packets vary; while in theory, packets can be fairly large, in practice they can be split, and most hosts cannot send out packets > 1456 total bytes including headers (TCP/IP headers are larger than ping headers, for example; 1456 bytes is the maximum Ethernet packet size). But they can be further split depending on the MTU of any segment of the transmission path (Maximum Transmission Unit). So ping would represent "optimistic" times and not give you ANY GUARANTEES about TCP/IP performance. So, for example, with a 400ms delay to Sydney, Australia for ping, you may or may not be able to make a 500ms TCP/IP window. The best you can hope for is that TCP/IP will be close to ping time once connection is established, the app handles the activation, threads have been created (if necessary), etc. But ping does not account for connection/startup transient (and I don't mean that your app is launched by the connection, I'm referring to the "startup" required in *your* app once a connection is detected). Since TCP/IP is optimized for long transfers, short transfers have disproportionate overheads. I'm not sure why you think 500ms is a meaningful number at all; as I said, nearly everything involved is outside your control and therefore predicating end-to-end performance on environmental effects you cannot possibly take into account seems meaningless. Your goal is to turn that computation around as quickly as you can. Like google, you can even try to give an approximate measure. The user has to expect that other delays are in the network. Tough. Networks have delays, not your problem. Don't try to make it your problem, or you will always lose. Solve only the problem you can control. (You can work out the delays by doing things like figuring speed-of-light (about 80% of the speed of light in a vacuum), packet overheads, and payload times; I used to illustrate it using a plastic snake called "Packet"; times were measured from the time his head passed a point until his tail passed, so we had transmission time, reception time, and end-to-end delay, but that assumed ZERO routers in between, which is unrealistic. The point was to teach students that bandwidth matters. So I'd draw two lines on the board, and move "Packet" through those. Ultiimately, round-trip time was measured by head-to-tail-to-head-to-tail-to-head-to-tail-to-head-to-tail. "Packet" had articulated segments so I could make him shorter or longer as needed to illustrate packet overheads, the Nagle algorithm, and my only regret was that years ago I hadn't bought several plastic snakes so I could do sliding window protocol. Ultimately, the students had to demonstrate, mathematically, the consequences of packet size and demonstrate an understanding of the impact of processing delays on each side of the wire. So a typical exam question might be "You have two nodes, one in Los Angeles and one in Boston, separation 2500 miles for round numbers. The dedicated optical fiber has a speed of 0.8C. The response time for ack/nak is 200usec. Assume there are no delays caused by buffer waiting at either endpoint. What is the ideal sliding window size to maximize utilization of the fiber? Give the utilization you predict as a percentage of total fiber bandwidth") Of course, buffer delays are real, there are routers in between, there is other network traffic, etc. So no matter what your "perfect formula" predicts, in the real Internet it is AT BEST a HINT at what the performance MIGHT be under OPTIMISTIC conditions. I just tried several google searches on a variety of topics, While google took under 0.2 seconds for each, the approximate times from click-to-refresh were approximately 1 second as I perceived them. (Allowing for my delays on the stopwatch, I was getting times from 0.76 to 1.05 seconds across about a dozen experiments). To click into a page took from 1.5 to 7.6 seconds from click to display, except one that took ~22 seconds. joe **** >> >> But it is unrealistic to assume such goals when they are >> based on parameters you have no >> control over. >> >> Your job is to turn the packet around in the minimum >> compute time you can manage. >> Everything else is beyond what you can control. >> joe >> ***** >> >> On Mon, 22 Feb 2010 23:54:04 -0600, "Peter Olcott" >> <NoSpam(a)OCR4Screen.com> wrote: >> >>> >>>"Ananth Ramasamy Meenachi" <msarm(a)live.com> wrote in >>>message >>>news:%23YtfKZEtKHA.3360(a)TK2MSFTNGP06.phx.gbl... >>>> >>>>> Is it possible for a very fast web service to >>>>> consistently provide an average 500 millisecond >>>>> response >>>>> time? >>>> It depends. The Internet Bandwidth is one of the major >>>> factor which controls the effective response performance >>>> (particularly the web Server). When you have enough >>>> bandwidth @ both ends next comes the hardware (storage & >>>> processing infrastructure). Best solution is going for >>>> CLOUD computing. Check for AZURE platform. >>>> >>>> >>>>> Is the internet itself too slow making this goal >>>>> completely infeasible using current technology? >>>> No way, any technology will fail to work with very low >>>> internet connection. >>>> >>>> Let me know you objective so that I can explain some >>>> more >>>> on this. >>> >>>I want to provide a web service that takes a tiny image >>>file >>>(15 x (100 to 400) pixels) and returns 20 to 100 bytes of >>>text that it derived using my proprietary OCR software. >>>Ideally I want to do this with a real time limit of 500 >>>milliseconds. The next best thing would be an average >>>response rate less than 500 milliseconds. You can assume >>>100 >>>milliseconds for my process including translation to and >>>from HTTP. >>> >>>> >>>> "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote in message >>>> news:W4mdnSJ8462Jxx7WnZ2dnUVZ_s-dnZ2d(a)giganews.com... >>>>> Is it possible for a very fast web service to >>>>> consistently provide an average 500 millisecond >>>>> response >>>>> time? >>>>> >>>>> Is the internet itself too slow making this goal >>>>> completely infeasible using current technology? >>>>> >>>>> >>> >> Joseph M. Newcomer [MVP] >> email: newcomer(a)flounder.com >> Web: http://www.flounder.com >> MVP Tips: http://www.flounder.com/mvp_tips.htm > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Peter Olcott on 23 Feb 2010 11:36 "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in message news:fis7o5pcfncd6ne5bk1rrnrp0licmnhogq(a)4ax.com... > See below... > On Tue, 23 Feb 2010 03:36:10 -0600, "Peter Olcott" > <NoSpam(a)OCR4Screen.com> wrote: > >> >>"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>message news:ss17o5ljcgg1cueekve99dvd30uh4v9t47(a)4ax.com... >>> The components involved in the response are: >>> Sender stack delay >>> Sender bandwidth to downstream server >>> Total multi-server latency to receiver downstream server >>> Receiver bandwidth to downstream server >>> Receiver stack delay >>> Receiver rendering time >>> >>> The stack delays exist and cannot be changed. They are >>> probably in hundreds of >>> microseconds but I have never measured them. >>> >>> "The last mile" certainly dominates. If you have dialup >>> internet vs. T3 hardwire you will >>> get different performance. >>> >>> But the interserver performance matters also. A ping, >>> which is a trivial packet, can turn >>> around in a few hundred milliseconds, e.g., >>> ================================================= >>> C:\Documents and Settings\email>ping google.com >>> >>> Pinging google.com [72.14.204.147] with 32 bytes of >>> data: >>> >>> Reply from 72.14.204.147: bytes=32 time=145ms TTL=53 >>> Reply from 72.14.204.147: bytes=32 time=144ms TTL=53 >>> Reply from 72.14.204.147: bytes=32 time=142ms TTL=53 >>> Reply from 72.14.204.147: bytes=32 time=143ms TTL=53 >>> >>> Ping statistics for 72.14.204.147: >>> Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), >>> Approximate round trip times in milli-seconds: >>> Minimum = 142ms, Maximum = 145ms, Average = 143ms >>> >>> C:\Documents and Settings\email>ping verizon.net >>> >>> Pinging verizon.net [206.46.232.39] with 32 bytes of >>> data: >>> >>> Reply from 206.46.232.39: bytes=32 time=203ms TTL=245 >>> Reply from 206.46.232.39: bytes=32 time=203ms TTL=245 >>> Reply from 206.46.232.39: bytes=32 time=200ms TTL=245 >>> Reply from 206.46.232.39: bytes=32 time=208ms TTL=245 >>> >>> Ping statistics for 206.46.232.39: >>> Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), >>> Approximate round trip times in milli-seconds: >>> Minimum = 200ms, Maximum = 208ms, Average = 203ms >>> >>> C:\Documents and Settings\email>ping >>> www.cityofsydney.nsw.gov.au >>> >>> Pinging www.cityofsydney.nsw.gov.au [203.147.135.212] >>> with >>> 32 bytes of data: >>> >>> Reply from 203.147.135.212: bytes=32 time=389ms TTL=112 >>> Reply from 203.147.135.212: bytes=32 time=382ms TTL=112 >>> Reply from 203.147.135.212: bytes=32 time=381ms TTL=112 >>> Reply from 203.147.135.212: bytes=32 time=379ms TTL=112 >>> >>> Ping statistics for 203.147.135.212: >>> Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), >>> Approximate round trip times in milli-seconds: >>> Minimum = 379ms, Maximum = 389ms, Average = 382ms >>> >>> C:\Documents and Settings\email> >>> >>> ==================================== >>> >>> So note that a ping to halfway around the world takes >>> about 400ms. This suggests that >>> under similar conditions, at 3am local time, you *might* >>> be able to turn a packet around >>> in 500ms, but I wouldn't bet on it. >> >>Ah so ping can directly measure what I need to know. I >>just >>pinged seescreen.com and got 53 ms with 32 bytes and 56 ms >>with 1024 bytes, larger numbers of bytes timed out. What >>is >>the normal packet size for a web service? Is the total >>time >>that a file takes precisely proportional to the number of >>packets, times the ping time per packet? > **** > Not quite. ping measures ONE instance of ONE protocol, a > protocol that is handled fairly > low in the stack and doesn't involve application response > (which includes time to bring in > pages of the application, etc.). Therefore, it is a > sort-of-indicator of network traffic > delays and nothing else. Note that doing a ping at > different times of day could produce > different results because of backbone traffic. For > example, if done while hundreds of > thousands of users are streaming live Winter Olympics > feeds would probably give different > results, especially if pinging to the Pacific Northwest. > > As such, it is an INDICATOR. It is not a precise measure, > and therefore is only > SUGGESTIVE of performance. Your Mileage May Vary. Does > Not Include Dealer Prep. Offer > Void Where Prohibited By Law. > > Backbone traffic, retransmit times if packets are received > in error (e.g., potential > satellite feeds where upper atmosphere effects could > scramble a packet, or a flock of > birds flying in front of a microwave tower), and similar > vagaries will introduce > additional delays. ping is a UDP-class protocol, not > TCP/IP, so there are also ack/nak > time issues involved when you use TCP/IP. Sliding window > protocols and piggybacking of > ack/nak improve TCP/IP performance. Packet reassembly and > multipath delays decrease it. > If your server is heavily loaded, it will delay senders by > throttling transmission until > it can handle the load. > > Also, ping is connectionless; in TCP/IP you have to > establish a connection, which is a > whole set of packets flying back and forth just to > activate the connection. These packets > will have delays comparable to ping, so you might have > several roundtrip delays. > > Packets vary; while in theory, packets can be fairly > large, in practice they can be split, > and most hosts cannot send out packets > 1456 total bytes > including headers (TCP/IP > headers are larger than ping headers, for example; 1456 > bytes is the maximum Ethernet > packet size). But they can be further split depending on > the MTU of any segment of the > transmission path (Maximum Transmission Unit). > > So ping would represent "optimistic" times and not give > you ANY GUARANTEES about TCP/IP > performance. So, for example, with a 400ms delay to > Sydney, Australia for ping, you may > or may not be able to make a 500ms TCP/IP window. > > The best you can hope for is that TCP/IP will be close to > ping time once connection is > established, the app handles the activation, threads have > been created (if necessary), > etc. But ping does not account for connection/startup > transient (and I don't mean that > your app is launched by the connection, I'm referring to > the "startup" required in *your* > app once a connection is detected). Since TCP/IP is > optimized for long transfers, short > transfers have disproportionate overheads. > > I'm not sure why you think 500ms is a meaningful number at > all; as I said, nearly > everything involved is outside your control and therefore > predicating end-to-end > performance on environmental effects you cannot possibly > take into account seems > meaningless. Your goal is to turn that computation around > as quickly as you can. Like > google, you can even try to give an approximate measure. > The user has to expect that > other delays are in the network. Tough. Networks have > delays, not your problem. Don't > try to make it your problem, or you will always lose. > Solve only the problem you can > control. > > (You can work out the delays by doing things like figuring > speed-of-light (about 80% of > the speed of light in a vacuum), packet overheads, and > payload times; I used to illustrate > it using a plastic snake called "Packet"; times were > measured from the time his head > passed a point until his tail passed, so we had > transmission time, reception time, and > end-to-end delay, but that assumed ZERO routers in > between, which is unrealistic. The > point was to teach students that bandwidth matters. So > I'd draw two lines on the board, > and move "Packet" through those. Ultiimately, round-trip > time was measured by > head-to-tail-to-head-to-tail-to-head-to-tail-to-head-to-tail. > "Packet" had articulated > segments so I could make him shorter or longer as needed > to illustrate packet overheads, > the Nagle algorithm, and my only regret was that years ago > I hadn't bought several plastic > snakes so I could do sliding window protocol. Ultimately, > the students had to > demonstrate, mathematically, the consequences of packet > size and demonstrate an > understanding of the impact of processing delays on each > side of the wire. So a typical > exam question might be "You have two nodes, one in Los > Angeles and one in Boston, > separation 2500 miles for round numbers. The dedicated > optical fiber has a speed of 0.8C. > The response time for ack/nak is 200usec. Assume there > are no delays caused by buffer > waiting at either endpoint. What is the ideal sliding > window size to maximize utilization > of the fiber? Give the utilization you predict as a > percentage of total fiber bandwidth") > Of course, buffer delays are real, there are routers in > between, there is other network > traffic, etc. So no matter what your "perfect formula" > predicts, in the real Internet it > is AT BEST a HINT at what the performance MIGHT be under > OPTIMISTIC conditions. > > I just tried several google searches on a variety of > topics, While google took under 0.2 > seconds for each, the approximate times from > click-to-refresh were approximately 1 second > as I perceived them. (Allowing for my delays on the > stopwatch, I was getting times from > 0.76 to 1.05 seconds across about a dozen experiments). > To click into a page took from > 1.5 to 7.6 seconds from click to display, except one that > took ~22 seconds. > joe > **** I just carefully evaluated my requirements. I will be receiving between 1K to 2K of input and producing 10 to 100 bytes of output. In other words 1 to 2 packets of input and 1 packet of output. It looks like achieving my target of 400 ms response time might just work. Does this seem plausible under your analysis? I use 500 ms total response time because this is the maximum time that is perceived by a human as nearly instantaneous, a half of a second of wasted time is not very much wasted time. >>> >>> But it is unrealistic to assume such goals when they are >>> based on parameters you have no >>> control over. >>> >>> Your job is to turn the packet around in the minimum >>> compute time you can manage. >>> Everything else is beyond what you can control. >>> joe >>> ***** >>> >>> On Mon, 22 Feb 2010 23:54:04 -0600, "Peter Olcott" >>> <NoSpam(a)OCR4Screen.com> wrote: >>> >>>> >>>>"Ananth Ramasamy Meenachi" <msarm(a)live.com> wrote in >>>>message >>>>news:%23YtfKZEtKHA.3360(a)TK2MSFTNGP06.phx.gbl... >>>>> >>>>>> Is it possible for a very fast web service to >>>>>> consistently provide an average 500 millisecond >>>>>> response >>>>>> time? >>>>> It depends. The Internet Bandwidth is one of the major >>>>> factor which controls the effective response >>>>> performance >>>>> (particularly the web Server). When you have enough >>>>> bandwidth @ both ends next comes the hardware (storage >>>>> & >>>>> processing infrastructure). Best solution is going for >>>>> CLOUD computing. Check for AZURE platform. >>>>> >>>>> >>>>>> Is the internet itself too slow making this goal >>>>>> completely infeasible using current technology? >>>>> No way, any technology will fail to work with very low >>>>> internet connection. >>>>> >>>>> Let me know you objective so that I can explain some >>>>> more >>>>> on this. >>>> >>>>I want to provide a web service that takes a tiny image >>>>file >>>>(15 x (100 to 400) pixels) and returns 20 to 100 bytes >>>>of >>>>text that it derived using my proprietary OCR software. >>>>Ideally I want to do this with a real time limit of 500 >>>>milliseconds. The next best thing would be an average >>>>response rate less than 500 milliseconds. You can assume >>>>100 >>>>milliseconds for my process including translation to and >>>>from HTTP. >>>> >>>>> >>>>> "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote in >>>>> message >>>>> news:W4mdnSJ8462Jxx7WnZ2dnUVZ_s-dnZ2d(a)giganews.com... >>>>>> Is it possible for a very fast web service to >>>>>> consistently provide an average 500 millisecond >>>>>> response >>>>>> time? >>>>>> >>>>>> Is the internet itself too slow making this goal >>>>>> completely infeasible using current technology? >>>>>> >>>>>> >>>> >>> Joseph M. Newcomer [MVP] >>> email: newcomer(a)flounder.com >>> Web: http://www.flounder.com >>> MVP Tips: http://www.flounder.com/mvp_tips.htm >> > Joseph M. Newcomer [MVP] > email: newcomer(a)flounder.com > Web: http://www.flounder.com > MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 23 Feb 2010 14:04 See below... On Tue, 23 Feb 2010 10:36:01 -0600, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >message news:fis7o5pcfncd6ne5bk1rrnrp0licmnhogq(a)4ax.com... >> See below... >> On Tue, 23 Feb 2010 03:36:10 -0600, "Peter Olcott" >> <NoSpam(a)OCR4Screen.com> wrote: >> >>> >>>"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >>>message news:ss17o5ljcgg1cueekve99dvd30uh4v9t47(a)4ax.com... >>>> The components involved in the response are: >>>> Sender stack delay >>>> Sender bandwidth to downstream server >>>> Total multi-server latency to receiver downstream server >>>> Receiver bandwidth to downstream server >>>> Receiver stack delay >>>> Receiver rendering time >>>> >>>> The stack delays exist and cannot be changed. They are >>>> probably in hundreds of >>>> microseconds but I have never measured them. >>>> >>>> "The last mile" certainly dominates. If you have dialup >>>> internet vs. T3 hardwire you will >>>> get different performance. >>>> >>>> But the interserver performance matters also. A ping, >>>> which is a trivial packet, can turn >>>> around in a few hundred milliseconds, e.g., >>>> ================================================= >>>> C:\Documents and Settings\email>ping google.com >>>> >>>> Pinging google.com [72.14.204.147] with 32 bytes of >>>> data: >>>> >>>> Reply from 72.14.204.147: bytes=32 time=145ms TTL=53 >>>> Reply from 72.14.204.147: bytes=32 time=144ms TTL=53 >>>> Reply from 72.14.204.147: bytes=32 time=142ms TTL=53 >>>> Reply from 72.14.204.147: bytes=32 time=143ms TTL=53 >>>> >>>> Ping statistics for 72.14.204.147: >>>> Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), >>>> Approximate round trip times in milli-seconds: >>>> Minimum = 142ms, Maximum = 145ms, Average = 143ms >>>> >>>> C:\Documents and Settings\email>ping verizon.net >>>> >>>> Pinging verizon.net [206.46.232.39] with 32 bytes of >>>> data: >>>> >>>> Reply from 206.46.232.39: bytes=32 time=203ms TTL=245 >>>> Reply from 206.46.232.39: bytes=32 time=203ms TTL=245 >>>> Reply from 206.46.232.39: bytes=32 time=200ms TTL=245 >>>> Reply from 206.46.232.39: bytes=32 time=208ms TTL=245 >>>> >>>> Ping statistics for 206.46.232.39: >>>> Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), >>>> Approximate round trip times in milli-seconds: >>>> Minimum = 200ms, Maximum = 208ms, Average = 203ms >>>> >>>> C:\Documents and Settings\email>ping >>>> www.cityofsydney.nsw.gov.au >>>> >>>> Pinging www.cityofsydney.nsw.gov.au [203.147.135.212] >>>> with >>>> 32 bytes of data: >>>> >>>> Reply from 203.147.135.212: bytes=32 time=389ms TTL=112 >>>> Reply from 203.147.135.212: bytes=32 time=382ms TTL=112 >>>> Reply from 203.147.135.212: bytes=32 time=381ms TTL=112 >>>> Reply from 203.147.135.212: bytes=32 time=379ms TTL=112 >>>> >>>> Ping statistics for 203.147.135.212: >>>> Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), >>>> Approximate round trip times in milli-seconds: >>>> Minimum = 379ms, Maximum = 389ms, Average = 382ms >>>> >>>> C:\Documents and Settings\email> >>>> >>>> ==================================== >>>> >>>> So note that a ping to halfway around the world takes >>>> about 400ms. This suggests that >>>> under similar conditions, at 3am local time, you *might* >>>> be able to turn a packet around >>>> in 500ms, but I wouldn't bet on it. >>> >>>Ah so ping can directly measure what I need to know. I >>>just >>>pinged seescreen.com and got 53 ms with 32 bytes and 56 ms >>>with 1024 bytes, larger numbers of bytes timed out. What >>>is >>>the normal packet size for a web service? Is the total >>>time >>>that a file takes precisely proportional to the number of >>>packets, times the ping time per packet? >> **** >> Not quite. ping measures ONE instance of ONE protocol, a >> protocol that is handled fairly >> low in the stack and doesn't involve application response >> (which includes time to bring in >> pages of the application, etc.). Therefore, it is a >> sort-of-indicator of network traffic >> delays and nothing else. Note that doing a ping at >> different times of day could produce >> different results because of backbone traffic. For >> example, if done while hundreds of >> thousands of users are streaming live Winter Olympics >> feeds would probably give different >> results, especially if pinging to the Pacific Northwest. >> >> As such, it is an INDICATOR. It is not a precise measure, >> and therefore is only >> SUGGESTIVE of performance. Your Mileage May Vary. Does >> Not Include Dealer Prep. Offer >> Void Where Prohibited By Law. >> >> Backbone traffic, retransmit times if packets are received >> in error (e.g., potential >> satellite feeds where upper atmosphere effects could >> scramble a packet, or a flock of >> birds flying in front of a microwave tower), and similar >> vagaries will introduce >> additional delays. ping is a UDP-class protocol, not >> TCP/IP, so there are also ack/nak >> time issues involved when you use TCP/IP. Sliding window >> protocols and piggybacking of >> ack/nak improve TCP/IP performance. Packet reassembly and >> multipath delays decrease it. >> If your server is heavily loaded, it will delay senders by >> throttling transmission until >> it can handle the load. >> >> Also, ping is connectionless; in TCP/IP you have to >> establish a connection, which is a >> whole set of packets flying back and forth just to >> activate the connection. These packets >> will have delays comparable to ping, so you might have >> several roundtrip delays. >> >> Packets vary; while in theory, packets can be fairly >> large, in practice they can be split, >> and most hosts cannot send out packets > 1456 total bytes >> including headers (TCP/IP >> headers are larger than ping headers, for example; 1456 >> bytes is the maximum Ethernet >> packet size). But they can be further split depending on >> the MTU of any segment of the >> transmission path (Maximum Transmission Unit). >> >> So ping would represent "optimistic" times and not give >> you ANY GUARANTEES about TCP/IP >> performance. So, for example, with a 400ms delay to >> Sydney, Australia for ping, you may >> or may not be able to make a 500ms TCP/IP window. >> >> The best you can hope for is that TCP/IP will be close to >> ping time once connection is >> established, the app handles the activation, threads have >> been created (if necessary), >> etc. But ping does not account for connection/startup >> transient (and I don't mean that >> your app is launched by the connection, I'm referring to >> the "startup" required in *your* >> app once a connection is detected). Since TCP/IP is >> optimized for long transfers, short >> transfers have disproportionate overheads. >> >> I'm not sure why you think 500ms is a meaningful number at >> all; as I said, nearly >> everything involved is outside your control and therefore >> predicating end-to-end >> performance on environmental effects you cannot possibly >> take into account seems >> meaningless. Your goal is to turn that computation around >> as quickly as you can. Like >> google, you can even try to give an approximate measure. >> The user has to expect that >> other delays are in the network. Tough. Networks have >> delays, not your problem. Don't >> try to make it your problem, or you will always lose. >> Solve only the problem you can >> control. >> >> (You can work out the delays by doing things like figuring >> speed-of-light (about 80% of >> the speed of light in a vacuum), packet overheads, and >> payload times; I used to illustrate >> it using a plastic snake called "Packet"; times were >> measured from the time his head >> passed a point until his tail passed, so we had >> transmission time, reception time, and >> end-to-end delay, but that assumed ZERO routers in >> between, which is unrealistic. The >> point was to teach students that bandwidth matters. So >> I'd draw two lines on the board, >> and move "Packet" through those. Ultiimately, round-trip >> time was measured by >> head-to-tail-to-head-to-tail-to-head-to-tail-to-head-to-tail. >> "Packet" had articulated >> segments so I could make him shorter or longer as needed >> to illustrate packet overheads, >> the Nagle algorithm, and my only regret was that years ago >> I hadn't bought several plastic >> snakes so I could do sliding window protocol. Ultimately, >> the students had to >> demonstrate, mathematically, the consequences of packet >> size and demonstrate an >> understanding of the impact of processing delays on each >> side of the wire. So a typical >> exam question might be "You have two nodes, one in Los >> Angeles and one in Boston, >> separation 2500 miles for round numbers. The dedicated >> optical fiber has a speed of 0.8C. >> The response time for ack/nak is 200usec. Assume there >> are no delays caused by buffer >> waiting at either endpoint. What is the ideal sliding >> window size to maximize utilization >> of the fiber? Give the utilization you predict as a >> percentage of total fiber bandwidth") >> Of course, buffer delays are real, there are routers in >> between, there is other network >> traffic, etc. So no matter what your "perfect formula" >> predicts, in the real Internet it >> is AT BEST a HINT at what the performance MIGHT be under >> OPTIMISTIC conditions. >> >> I just tried several google searches on a variety of >> topics, While google took under 0.2 >> seconds for each, the approximate times from >> click-to-refresh were approximately 1 second >> as I perceived them. (Allowing for my delays on the >> stopwatch, I was getting times from >> 0.76 to 1.05 seconds across about a dozen experiments). >> To click into a page took from >> 1.5 to 7.6 seconds from click to display, except one that >> took ~22 seconds. >> joe >> **** > >I just carefully evaluated my requirements. I will be >receiving between 1K to 2K of input and producing 10 to 100 >bytes of output. In other words 1 to 2 packets of input and >1 packet of output. It looks like achieving my target of 400 >ms response time might just work. Does this seem plausible >under your analysis? > >I use 500 ms total response time because this is the maximum >time that is perceived by a human as nearly instantaneous, a >half of a second of wasted time is not very much wasted >time. **** Normal perceptual-to-motor delay is about 250ms. 500ms can actually be noticeable as a delay. As I say, you might be able to do it, but if you don't meet the window, it really isn't your problem. And people who send things out over the Internet are already conditioned to expect longer delays. joe **** > >>>> >>>> But it is unrealistic to assume such goals when they are >>>> based on parameters you have no >>>> control over. >>>> >>>> Your job is to turn the packet around in the minimum >>>> compute time you can manage. >>>> Everything else is beyond what you can control. >>>> joe >>>> ***** >>>> >>>> On Mon, 22 Feb 2010 23:54:04 -0600, "Peter Olcott" >>>> <NoSpam(a)OCR4Screen.com> wrote: >>>> >>>>> >>>>>"Ananth Ramasamy Meenachi" <msarm(a)live.com> wrote in >>>>>message >>>>>news:%23YtfKZEtKHA.3360(a)TK2MSFTNGP06.phx.gbl... >>>>>> >>>>>>> Is it possible for a very fast web service to >>>>>>> consistently provide an average 500 millisecond >>>>>>> response >>>>>>> time? >>>>>> It depends. The Internet Bandwidth is one of the major >>>>>> factor which controls the effective response >>>>>> performance >>>>>> (particularly the web Server). When you have enough >>>>>> bandwidth @ both ends next comes the hardware (storage >>>>>> & >>>>>> processing infrastructure). Best solution is going for >>>>>> CLOUD computing. Check for AZURE platform. >>>>>> >>>>>> >>>>>>> Is the internet itself too slow making this goal >>>>>>> completely infeasible using current technology? >>>>>> No way, any technology will fail to work with very low >>>>>> internet connection. >>>>>> >>>>>> Let me know you objective so that I can explain some >>>>>> more >>>>>> on this. >>>>> >>>>>I want to provide a web service that takes a tiny image >>>>>file >>>>>(15 x (100 to 400) pixels) and returns 20 to 100 bytes >>>>>of >>>>>text that it derived using my proprietary OCR software. >>>>>Ideally I want to do this with a real time limit of 500 >>>>>milliseconds. The next best thing would be an average >>>>>response rate less than 500 milliseconds. You can assume >>>>>100 >>>>>milliseconds for my process including translation to and >>>>>from HTTP. >>>>> >>>>>> >>>>>> "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote in >>>>>> message >>>>>> news:W4mdnSJ8462Jxx7WnZ2dnUVZ_s-dnZ2d(a)giganews.com... >>>>>>> Is it possible for a very fast web service to >>>>>>> consistently provide an average 500 millisecond >>>>>>> response >>>>>>> time? >>>>>>> >>>>>>> Is the internet itself too slow making this goal >>>>>>> completely infeasible using current technology? >>>>>>> >>>>>>> >>>>> >>>> Joseph M. Newcomer [MVP] >>>> email: newcomer(a)flounder.com >>>> Web: http://www.flounder.com >>>> MVP Tips: http://www.flounder.com/mvp_tips.htm >>> >> Joseph M. Newcomer [MVP] >> email: newcomer(a)flounder.com >> Web: http://www.flounder.com >> MVP Tips: http://www.flounder.com/mvp_tips.htm > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Geoff on 23 Feb 2010 14:20
On Tue, 23 Feb 2010 03:36:10 -0600, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: >Ah so ping can directly measure what I need to know. I just >pinged seescreen.com and got 53 ms with 32 bytes and 56 ms >with 1024 bytes, larger numbers of bytes timed out. What is >the normal packet size for a web service? Is the total time >that a file takes precisely proportional to the number of >packets, times the ping time per packet? Ping is ICMP, a connectionless protocol. TCP/IP is connected and guarantees delivery of application data but not the order or delay of the packets involved. TCP reassembles the packets in their proper order and passes the data to the application once it is reassembled. TCP also doesn't guarantee that every connected packet between two hosts will travel the same pathway. This is part of the redundancy of the system. Not all hosts respond to ICMP ping packets. Some block it at the host firewall, some ISPs block from unknown sources it at their borders. That being said, 500ms is probably pretty reasonable for just about anywhere on the globe. The longest delays will usually be incurred when satellite links are involved, there are probably fewer hops but the speed of light delays are higher that purely terrestrial links owing to the 72,000 km round trip. Some cmd tools: netstat -s in a cmd prompt. General stats about your TCP/IP stack. ping hostname as you have seen. tracert hostname pathping hostname Windows Perfmon has realtime measures of TCP, UDP, IP, ICMP gathered by the system. Some existing statistics continuously monitored globally. http://www.internettrafficreport.com/main.htm http://www.internethealthreport.com/ http://www.noc.ucla.edu/weather.html http://www.dslreports.com/speedtest?more=1 One excellent tool for users is http://www.dslreports.com/tools, especially the Tweak Test. This one has a "secret" link at the bottom of the summary page that graphs the packet protocol performance over time and includes the RTT per packet as well as a tabular summary of the performance. Of course this is for large data downloads but it might serve the purpose for your site. |