How hard is socket programming?
in [MFC]
|
Prev: Runtime Error does not specify program???
Next: windows 7 - imitating touch by sending wm_gesture ?
From: Peter Olcott on 20 Mar 2010 14:59 "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in message news:016aq5tfb2fh64etldio6ast4pkepj4rip(a)4ax.com... > For authentication, there's a lot of technology out there > already. You should be holding > these conversations with your ISP to see what services > they provide,and how much they > cost. > joe > I will only need authentication in the sense that every user must be logged into their account on my server before any requests will be processed. I am guessing that this will probably have to be code that I write. I will be using Paypal as my payment processor. I am free to install anything that I want on the server. The only thing that I can not do is use another OS than the several that they provide. > > On Fri, 19 Mar 2010 23:42:36 -0500, "Peter Olcott" > <NoSpam(a)OCR4Screen.com> wrote: > >> >>"Hector Santos" <sant9442(a)nospam.gmail.com> wrote in >>message >>news:eUo4lS%23xKHA.2012(a)TK2MSFTNGP04.phx.gbl... >>> Peter Olcott wrote: >>> >>>> I want to prevent malicious users from eating up my >>>> service providers bandwidth allocation. I would like to >>>> do this in an optimal way. >>> >>> >>> Are you kidding me? >>> >>> Don't you have in your business plan to get your own WEB >>> SERVER and not use GoDaddy? >>> >>> >>> -- >>> HLS >> >>I think I have a way to minimize abusive users. It does >>make >>it a little more difficult for my legitimate users, but, >>it >>is worth the cost. Everyone must be an authenticated user, >>even the free trial users. Every user must have a valid >>email address. Every user must prove that they are human >>(and not a bot) when setting up their user account. Non >>authenticated users have no access. Users that are abusive >>simply get their account cancelled. An email is sent to >>their valid email address explaining why the account was >>cancelled. >> > Joseph M. Newcomer [MVP] > email: newcomer(a)flounder.com > Web: http://www.flounder.com > MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 20 Mar 2010 15:10 See below... On Sat, 20 Mar 2010 00:54:55 -0400, Hector Santos <sant9442(a)nospam.gmail.com> wrote: >Peter Olcott wrote: > >> "Hector Santos" <sant9442(a)nospam.gmail.com> wrote in message >> news:%23YrAhQ%23xKHA.3408(a)TK2MSFTNGP06.phx.gbl... >>> Peter Olcott wrote: >>> >>>> I just want to understand how the handle the most >>>> malicious user in the most robust way. If I can block an >>>> IP address without costing me any bandwidth, then I think >>>> I have one significant aspect of blocking the most >>>> malicious user. >>> As joe puts it, you are "obsessing" again :) >>> >>> Its very simple with the server model: >>> >>> s = socket() - create a socket handle >>> bind(s) - associate an local IP with socket s >>> handle >>> listen(s) - starting to listen to connection >>> accept(s,c) - wait for clients, new socket c for >>> connect >>> >>> CHECK PEER IP ADDRESS OF C IN FILTER LIST, >>> IF FOUND, CLOSE SOCKET GOTO BACK TO ACCEPT >>> >>> recv(c) - receiver whatever >>> send(c) - send whatever >>> shutdown(c) - tell remote I'm about to close >>> closesocket(c) - close socket handle >>> go back to accept >>> >>> -- >>> HLS >> >> That does seem pretty simple. Would I only need a single >> recv(c) for a 10 MB file? > >No, a recv() is a loop and you read 8K at a time: **** Note that you read AT MOST 8K, and that's because of the sizeof(buf) parameter to the recv being 8*1024. If you made it 16*1024, you might receive up to 16K at a time, but in all cases, this is the MAXIMUM that will be returned; you might get fewer bytes, based on network traffic and your sender and receiver stacks. ***** > > FILE *fv = fopen(szImageFileName,"wb"); > char buf[8*1024] = {0}; > for (;;) > { > int len = recv(c,buf,sizeof(buf),0); > if (len <= 0) break; > fwrite(buf,len,1,fv); > } > fclose(fv); > >But the above is very simplistic. > >Mongoose.c, without verifying but I trust it will, otherwise it isn't >a web server, will same the POSTED data in some temporary file for the >transaction (while the connection is alive). That FILE NAME is then >passed your CGI script or whenever is going to process the posted data. > >So it should be done for you. You just need to learn how Mongoose >implements this fundamental web server idea. > >Again, if it doesn't handled POSTed data for you, then GET RID of it, >it isn't a real web server of any kind. > >But I know it does because its fundamental and a HTTP standard >requirement to handle POST correctly and that includes save the data >somewhere for the session to process. Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 20 Mar 2010 15:14 See below... On Sat, 20 Mar 2010 09:29:11 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Hector Santos" <sant9442(a)nospam.gmail.com> wrote in message >news:eUEo4l%23xKHA.5364(a)TK2MSFTNGP05.phx.gbl... >> Peter Olcott wrote: >> >>> "Hector Santos" <sant9442(a)nospam.gmail.com> wrote in >>> message news:%23YrAhQ%23xKHA.3408(a)TK2MSFTNGP06.phx.gbl... >>>> Peter Olcott wrote: >>>> >>>>> I just want to understand how the handle the most >>>>> malicious user in the most robust way. If I can block >>>>> an IP address without costing me any bandwidth, then I >>>>> think I have one significant aspect of blocking the >>>>> most malicious user. >>>> As joe puts it, you are "obsessing" again :) >>>> >>>> Its very simple with the server model: >>>> >>>> s = socket() - create a socket handle >>>> bind(s) - associate an local IP with socket >>>> s handle >>>> listen(s) - starting to listen to connection >>>> accept(s,c) - wait for clients, new socket c >>>> for connect >>>> >>>> CHECK PEER IP ADDRESS OF C IN FILTER LIST, >>>> IF FOUND, CLOSE SOCKET GOTO BACK TO ACCEPT >>>> >>>> recv(c) - receiver whatever >>>> send(c) - send whatever >>>> shutdown(c) - tell remote I'm about to close >>>> closesocket(c) - close socket handle >>>> go back to accept >>>> >>>> -- >>>> HLS >>> >>> That does seem pretty simple. Would I only need a single >>> recv(c) for a 10 MB file? >> >> No, a recv() is a loop and you read 8K at a time: >> >> FILE *fv = fopen(szImageFileName,"wb"); >> char buf[8*1024] = {0}; >> for (;;) >> { >> int len = recv(c,buf,sizeof(buf),0); >> if (len <= 0) break; >> fwrite(buf,len,1,fv); >> } >> fclose(fv); >> >> But the above is very simplistic. >> >> Mongoose.c, without verifying but I trust it will, >> otherwise it isn't a web server, will same the POSTED data >> in some temporary file for the transaction (while the >> connection is alive). That FILE NAME is then passed your >> CGI script or whenever is going to process the posted >> data. >> >> So it should be done for you. You just need to learn how >> Mongoose implements this fundamental web server idea. >> >> Again, if it doesn't handled POSTed data for you, then GET >> RID of it, it isn't a real web server of any kind. >> >> But I know it does because its fundamental and a HTTP >> standard requirement to handle POST correctly and that >> includes save the data somewhere for the session to >> process. >> >> >> -- >> HLS > >That is great Hector. It looks like the basic architectural >design is set. I won't limit myself to mongoose in the long >run. **** Note that what is illustrated here is what is called an "iterative servier", that is, while it is processing one transactiion, it cannot accept other connections, because it needs to go back to the Accept statement. There is also a concept called a "concurrent server" where you hand the accepted socket over to a separate thread and immediately return to the Accept statement. In this case, you can handle multiple connections. Note that if you are using CAsyncSocket for the Accept target, then you will have to detach the socket handle from the CAsyncSocket object, pass the raw socket handle to the thread, and re-attach the handle (using CAsyncSocket::FromHandle) to a CAsyncSocket object in the thread, to keep MFC happy. See my essay on UI threads. In general, if you want maximum thoughput you will want to consider a concurrent server architecture. These are discussed in deail in Comer & Stevens volumn III (Windows edition). joe > http://en.wikipedia.org/wiki/Comparison_of_lightweight_web_servers >One of these might prove to be better in the long run. >Another design constraint is very high security. That will >require another whole learning curve. > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Peter Olcott on 20 Mar 2010 15:20 "Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in message news:vc7aq5lu3hflav7lj85eamvdfa6f40acng(a)4ax.com... > See below... > On Sat, 20 Mar 2010 09:29:11 -0500, "Peter Olcott" > <NoSpam(a)OCR4Screen.com> wrote: > >> >>"Hector Santos" <sant9442(a)nospam.gmail.com> wrote in >>message >>news:eUEo4l%23xKHA.5364(a)TK2MSFTNGP05.phx.gbl... >>> Peter Olcott wrote: >>> >>>> "Hector Santos" <sant9442(a)nospam.gmail.com> wrote in >>>> message >>>> news:%23YrAhQ%23xKHA.3408(a)TK2MSFTNGP06.phx.gbl... >>>>> Peter Olcott wrote: >>>>> >>>>>> I just want to understand how the handle the most >>>>>> malicious user in the most robust way. If I can block >>>>>> an IP address without costing me any bandwidth, then >>>>>> I >>>>>> think I have one significant aspect of blocking the >>>>>> most malicious user. >>>>> As joe puts it, you are "obsessing" again :) >>>>> >>>>> Its very simple with the server model: >>>>> >>>>> s = socket() - create a socket handle >>>>> bind(s) - associate an local IP with >>>>> socket >>>>> s handle >>>>> listen(s) - starting to listen to >>>>> connection >>>>> accept(s,c) - wait for clients, new socket c >>>>> for connect >>>>> >>>>> CHECK PEER IP ADDRESS OF C IN FILTER LIST, >>>>> IF FOUND, CLOSE SOCKET GOTO BACK TO ACCEPT >>>>> >>>>> recv(c) - receiver whatever >>>>> send(c) - send whatever >>>>> shutdown(c) - tell remote I'm about to close >>>>> closesocket(c) - close socket handle >>>>> go back to accept >>>>> >>>>> -- >>>>> HLS >>>> >>>> That does seem pretty simple. Would I only need a >>>> single >>>> recv(c) for a 10 MB file? >>> >>> No, a recv() is a loop and you read 8K at a time: >>> >>> FILE *fv = fopen(szImageFileName,"wb"); >>> char buf[8*1024] = {0}; >>> for (;;) >>> { >>> int len = recv(c,buf,sizeof(buf),0); >>> if (len <= 0) break; >>> fwrite(buf,len,1,fv); >>> } >>> fclose(fv); >>> >>> But the above is very simplistic. >>> >>> Mongoose.c, without verifying but I trust it will, >>> otherwise it isn't a web server, will same the POSTED >>> data >>> in some temporary file for the transaction (while the >>> connection is alive). That FILE NAME is then passed >>> your >>> CGI script or whenever is going to process the posted >>> data. >>> >>> So it should be done for you. You just need to learn >>> how >>> Mongoose implements this fundamental web server idea. >>> >>> Again, if it doesn't handled POSTed data for you, then >>> GET >>> RID of it, it isn't a real web server of any kind. >>> >>> But I know it does because its fundamental and a HTTP >>> standard requirement to handle POST correctly and that >>> includes save the data somewhere for the session to >>> process. >>> >>> >>> -- >>> HLS >> >>That is great Hector. It looks like the basic >>architectural >>design is set. I won't limit myself to mongoose in the >>long >>run. > **** > Note that what is illustrated here is what is called an > "iterative servier", that is, > while it is processing one transactiion, it cannot accept > other connections, because it > needs to go back to the Accept statement. There is also a > concept called a "concurrent > server" where you hand the accepted socket over to a > separate thread and immediately > return to the Accept statement. In this case, you can > handle multiple connections. Note > that if you are using CAsyncSocket for the Accept target, > then you will have to detach the > socket handle from the CAsyncSocket object, pass the raw > socket handle to the thread, and > re-attach the handle (using CAsyncSocket::FromHandle) to a > CAsyncSocket object in the > thread, to keep MFC happy. See my essay on UI threads. > > In general, if you want maximum thoughput you will want to > consider a concurrent server > architecture. These are discussed in deail in Comer & > Stevens volumn III (Windows > edition). > joe I am thinking that the web server part will spawn a thread for each request, and append this request at the end of a FIFO queue, then this thread dies. My OCR process would read requests from the head of the queue, and process them one at a time in a single thread. > >> >> http://en.wikipedia.org/wiki/Comparison_of_lightweight_web_servers >>One of these might prove to be better in the long run. >>Another design constraint is very high security. That will >>require another whole learning curve. >> > Joseph M. Newcomer [MVP] > email: newcomer(a)flounder.com > Web: http://www.flounder.com > MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 20 Mar 2010 15:21
See below... On Fri, 19 Mar 2010 23:30:08 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: > >"Joseph M. Newcomer" <newcomer(a)flounder.com> wrote in >message news:kqh8q5h7k00484tpkjle30vsjqi25fbo83(a)4ax.com... >> See be;ow... >> On Fri, 19 Mar 2010 17:20:55 -0500, "Peter Olcott" >> <NoSpam(a)OCR4Screen.com> wrote: >> >>> >>>"Hector Santos" <sant9442(a)nospam.gmail.com> wrote in >>>message >>>news:%23s1kB76xKHA.4240(a)TK2MSFTNGP06.phx.gbl... >>>> Peter Olcott wrote: >>>> >>>>> It looks like I am going to be using HTTP as the >>>>> protocol. I just bought two books on it. I am >>>>> estimating >>>>> that the TCP/IP is mostly invisible at the HTTP layer. >>>>> I >>>>> am using the HTTP protocol because it seems that the >>>>> HTML >>>>> element that I am using, sends the data using this >>>>> protocol. >>>>> <input name="userfile" type="file" accept="image/png" >>>>> /> >>>>> >>>>> Using HTTP is it possible to reject a file that is the >>>>> wrong format before the entire file is sent? >>>> >>>> >>>> The HTTP client will send the type in the HTTP request >>>> BODY block: >>>> >>>> Content-type: image/png >>>> >>>>> Using HTTP is it possible to reject a file that is too >>>>> large before very much of this file is sent? >>>> >>>> >>>> The HTTP request will have a HTTP request header: >>>> >>>> Content-length: >>>> >>>> But that length is the entire HTTP body following the >>>> request header block, i.e. you can send two or more >>>> uploads. You can use this length for a rough value, but >>>> to get the actual image size you need to parse the HTTP >>>> BODY to get the type and size. >>>> >>>> Note: Not all browsers will send a size in the body >>>> block, >>>> like if its only 1 image. Therefore you use the top >>>> HTTP >>>> request block Content-Length: header. >>>> >>>> >>>> -- >>>> HLS >>> >>>OK great is this before or after my bandwidth quota has >>>been >>>hit with the full data load? >> **** >> You are obsessing again. If you want to use HTTP, you >> have to live with its limitations. >> If you want to implement your own protocol, you can abort >> a partially-completed >> transaction just by closing the connection when you decide >> that the data is invalid. This >> means you have to not just "accept" the data AS IF it is >> valid PNG encoding, but you have >> to VERIFY that it makes sense at every step. >> >> But as long as you keep obsessing about details, you will >> make no progress. Either you >> accept that HTTP can do the job (and that you will >> sometimes get an upload that isn't >> valid) or you have to implement your own port# and >> protocol spec, and this allows you to >> abort at any time, but note that SOME amount of data has >> already been received. Suck it > >Yes this is my second level design. I want to be able to >handle malicious attacks in the most robust way. For example >the first few bytes of data would be encrypted using a >public key/ private key pair, and if it is not authenticate >close the connection. This would be implemented (on the >client side) using a signed java applet. *** Talk to your ISP about https: (secure HTTP) authentication and what they provid you as tools. joe **** > >> up and accept that this is reality. It happens. Every Web >> server in the world has to deal >> with this, and most of them deal with it by ignoring the >> problem because they don't really >> care in the slightest. If you care, you pay a price: you >> have to write more complex >> server code and you have to write complex validation code. >> This is simply reality. >> joe >> **** >>> >>>In other words could a lower level protocol such as TCP/IP >>>send a large portion of the file before my HTTP gets a >>>chance to reject it because it is "denial of service >>>attack" >>>size? >> *** >> Sure. Live with the idea. You aren't going to be able to >> change this behavior! >> **** >>> >>>Alternatively could I somehow structure my code so that I >>>only get just the bytes indicating the size and reject all >>>other data before eating up any more of my bandwidth >>>quota? >> **** >> If someone wants to screw you, they will send you bogus >> packets until they saturate your >> server or your bandwidht quota. We all them "script >> kiddies" and they are a real problem, >> but you can't make them go away by changing your protocol. >> They'll get you one way or >> another. > >I want to make this as close to impossible as I can. I >envision the possibility of (for example) a competitor doing >this to shut me down. I want to implement the most >cost-effective way to reduce this risk. Maybe the main >answer is to require all users to have an account, even the >free trial users. This account can only be set up from a >valid email address. Some form of bot protection would also >be used. That may prove to be an entirely sufficient answer. **** There is essentially nothing you can really do to stop this kind of attack. It becomes the responsibility of your ISP to deal with this. Or to work with you on it. Don't try to solve every problem, but have a plan to deal with it. This means working closely with your ISP's security people, who know what they can do at various levels of the protoccol. That's what I would do. I don't pretend to be a security expert, but I know what *won't* work, and I'd want to find out from my ISP what *will* work and what they already support to handle such issues. Their suvival depends on their being able to do this effectively, so they've already addressed many of these issues. joe **** > >I want to have these things completely thought through >before I begin. It is much cheaper to do this now than it >will be when I have thousands of users depending on my >service. **** Then go talk to your ISP! I can't tell you what services they provide! They can! joe > >> **** >>> >>>example I only see these bytes---->"size(1000000000)" and >>>reject the file before any additional bytes hit my >>>bandwidth >>>quota. >> **** >> If you write your own protocol handler--as I did for my >> multithreaded socket example--then >> you can do anything you want within certain buffer >> boundaries; you might have 8K or 15K or >> 64K data already received that the shutdown() will >> discard, tough. that's life. Deal >> with it. >> **** >>> >>>NOTE the only valid data sent to my web application will >>>be >>>a single 24-bit PNG file that can vary in size up to a >>>predetermined limit of something like 10K. >> *** >> Then you are truly obsessing over a problem not worthy of >> concern. Just build it. >> joe >> **** >>> >> Joseph M. Newcomer [MVP] >> email: newcomer(a)flounder.com >> Web: http://www.flounder.com >> MVP Tips: http://www.flounder.com/mvp_tips.htm > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm |