Problems reading from a specific web site
in [TCL]
|
Prev: passing "arrays of structs" in procs.
Next: Question about common practices on building applications on Tk (or "OO + Tk")
From: Alexandre Ferrieux on 5 Mar 2010 18:55 On Mar 5, 10:50 pm, Alan Grunwald <alan-clt-pos...(a)nospam.demon.co.uk> wrote: > Alexandre Ferrieux wrote: > > On Mar 5, 7:19 pm, Alan Grunwald <alan-clt-pos...(a)nospam.demon.co.uk> > > wrote: > >> Alexandre Ferrieux wrote: > >>> On Mar 5, 5:08 pm, Alan Grunwald <alan-clt-pos...(a)nospam.demon.co.uk> > >>> wrote: > >>>> Hi all, > >>>> I'm trying to get some data via http. > >>>> set tok [http::geturl $myUrl] > >>>> puts stdout [http::code $tok] > >>>> displays nothing. (I'm using http version 2.7.2 from the Active State > >>>> teapot, and ActiveTcl 8.6b1.1 on Ubuntu.) > >>>> I've done some digging and the problem seems to be towards the top > >>>> of Event, where the following code may be found: > >>>> if {$state(state) eq "connecting"} { > >>>> set state(state) "header" > >>>> if {[catch {gets $sock state(http)} n]} { > >>>> return [Finish $token $n] > >>>> } > >>>> } elseif... > >>>> It appears that the [gets] in this code snippet returns -1, because of > >>>> this, when the code gets to the point where it reads the blank line > >>>> marking the end of the headers, state(http) is zero length; the code > >>>> seems to assume that this means it has received an HTTP/1.1 100 Continue > >>>> response, and doesn't set its state to body. I'm not sure exactly what > >>>> happens afterwards, but this is obviously a Bad Thing. > >>>> I tried changing the code snippet above to > >>>> if {$state(state) eq "connecting"} { > >>>> if {[catch {gets $sock state(http)} n]} { > >>>> return [Finish $token $n] > >>>> } > >>>> if {$n > -1} { > >>>> set state(state) "header" > >>>> } > >>>> } elseif... > >>>> and it works fine for the url I'm interested in, and also for the one > >>>> other url I tested it on. > >>>> Please can someone who knows more than me about the http protocol and > >>>> [gets] (this shouldn't exclude too many people :-)) re-assure me that > >>>> the code change is legitimate. > >>>> Can anyone suggest what may be going wrong on the server side? The > >>>> headers I'm reading from the "odd" site include "Server > >>>> Microsoft-IIS/6.0 X-Powered-By PHP/5.2.4 X-Powered-By ASP.NET", which > >>>> rather prejudices me against it. It's the first version of a new server, > >>>> so there might be problems at the server side, I'm sure they'd > >>>> appreciate any hints or pointers I can provide. > >>>> Many thanks, > >>>> -- > >>>> Alan > >>> Regardless of whether the server is behaving slightly or grossly > >>> wrong, or not wrong at all, it looks like a bug in our http package. > >>> In order to collect analysis, traces, and proposed patches in a > >>> rational fashion, could you please post a bugreport ? > >>> TIA, > >>> -Alex > >> Is that the main tcl bug tracker on sourceforge? > > > Yes, since that's in the pure-Tcl http package in the 'library' > > subdirectory of the Tcl source tree. > > > -Alex > > Done. New tracker ID 2964525 Thanks. However, looking at the applicability of your patch against 8.6 HEAD, I found it was a duplicate of bug 2624536, and closed this one accordingly. Sorry for the turnabout, had not noticed before. -Alex
From: Alan Grunwald on 6 Mar 2010 13:08 Alexandre Ferrieux wrote: > On Mar 5, 10:50 pm, Alan Grunwald <alan-clt-pos...(a)nospam.demon.co.uk> > wrote: >> Alexandre Ferrieux wrote: >>> On Mar 5, 7:19 pm, Alan Grunwald <alan-clt-pos...(a)nospam.demon.co.uk> >>> wrote: >>>> Alexandre Ferrieux wrote: >>>>> On Mar 5, 5:08 pm, Alan Grunwald <alan-clt-pos...(a)nospam.demon.co.uk> >>>>> wrote: >>>>>> Hi all, >>>>>> I'm trying to get some data via http. >>>>>> set tok [http::geturl $myUrl] >>>>>> puts stdout [http::code $tok] >>>>>> displays nothing. (I'm using http version 2.7.2 from the Active State >>>>>> teapot, and ActiveTcl 8.6b1.1 on Ubuntu.) >>>>>> I've done some digging and the problem seems to be towards the top >>>>>> of Event, where the following code may be found: >>>>>> if {$state(state) eq "connecting"} { >>>>>> set state(state) "header" >>>>>> if {[catch {gets $sock state(http)} n]} { >>>>>> return [Finish $token $n] >>>>>> } >>>>>> } elseif... >>>>>> It appears that the [gets] in this code snippet returns -1, because of >>>>>> this, when the code gets to the point where it reads the blank line >>>>>> marking the end of the headers, state(http) is zero length; the code >>>>>> seems to assume that this means it has received an HTTP/1.1 100 Continue >>>>>> response, and doesn't set its state to body. I'm not sure exactly what >>>>>> happens afterwards, but this is obviously a Bad Thing. >>>>>> I tried changing the code snippet above to >>>>>> if {$state(state) eq "connecting"} { >>>>>> if {[catch {gets $sock state(http)} n]} { >>>>>> return [Finish $token $n] >>>>>> } >>>>>> if {$n > -1} { >>>>>> set state(state) "header" >>>>>> } >>>>>> } elseif... >>>>>> and it works fine for the url I'm interested in, and also for the one >>>>>> other url I tested it on. >>>>>> Please can someone who knows more than me about the http protocol and >>>>>> [gets] (this shouldn't exclude too many people :-)) re-assure me that >>>>>> the code change is legitimate. >>>>>> Can anyone suggest what may be going wrong on the server side? The >>>>>> headers I'm reading from the "odd" site include "Server >>>>>> Microsoft-IIS/6.0 X-Powered-By PHP/5.2.4 X-Powered-By ASP.NET", which >>>>>> rather prejudices me against it. It's the first version of a new server, >>>>>> so there might be problems at the server side, I'm sure they'd >>>>>> appreciate any hints or pointers I can provide. >>>>>> Many thanks, >>>>>> -- >>>>>> Alan >>>>> Regardless of whether the server is behaving slightly or grossly >>>>> wrong, or not wrong at all, it looks like a bug in our http package. >>>>> In order to collect analysis, traces, and proposed patches in a >>>>> rational fashion, could you please post a bugreport ? >>>>> TIA, >>>>> -Alex >>>> Is that the main tcl bug tracker on sourceforge? >>> Yes, since that's in the pure-Tcl http package in the 'library' >>> subdirectory of the Tcl source tree. >>> -Alex >> Done. New tracker ID 2964525 > > Thanks. > However, looking at the applicability of your patch against 8.6 HEAD, > I found it was a duplicate of bug 2624536, and closed this one > accordingly. > Sorry for the turnabout, had not noticed before. > > -Alex > No worries Alex; good to see a problem knocked down. A few questions though - the comments in 2624536 talk about http 2.7.3. Where can I get that from; if it's in the teapot, how come "teacup update" didn't pick it up? Finally :-) if it's not in the teapot, what's the process for getting it in? ....but mainly; thanks for reassuring me that I had a genuine problem and thanks for finding the bug report that fixed it. -- Alan
From: Alexandre Ferrieux on 6 Mar 2010 17:30 On Mar 6, 7:08 pm, Alan Grunwald <alan-clt-pos...(a)nospam.demon.co.uk> wrote: > > A few questions though - the comments in 2624536 talk about http 2.7.3. > Where can I get that from; if it's in the teapot, how come "teacup > update" didn't pick it up? Finally :-) if it's not in the teapot, what's > the process for getting it in? It's also in the 'library' folder of the Tcl source tree, so you can grab it directly through the "CVS browse" at SF if you don't want to install CVS on your machine. IIRC the current 8.6 HEAD has an http.tcl that is well beyond 2.7.3. -Alex
From: Bezoar on 9 Mar 2010 21:41 On Mar 5, 10:08 am, Alan Grunwald <alan-clt-pos...(a)nospam.demon.co.uk> wrote: > Hi all, > > I'm trying to get some data via http. > > set tok [http::geturl $myUrl] > puts stdout [http::code $tok] > > displays nothing. (I'm using http version 2.7.2 from the Active State > teapot, and ActiveTcl 8.6b1.1 on Ubuntu.) > > I've done some digging and the problem seems to be towards the top > of Event, where the following code may be found: > > if {$state(state) eq "connecting"} { > set state(state) "header" > if {[catch {gets $sock state(http)} n]} { > return [Finish $token $n] > } > } elseif... > > It appears that the [gets] in this code snippet returns -1, because of > this, when the code gets to the point where it reads the blank line > marking the end of the headers, state(http) is zero length; the code > seems to assume that this means it has received an HTTP/1.1 100 Continue > response, and doesn't set its state to body. I'm not sure exactly what > happens afterwards, but this is obviously a Bad Thing. > > I tried changing the code snippet above to > > if {$state(state) eq "connecting"} { > if {[catch {gets $sock state(http)} n]} { > return [Finish $token $n] > } > if {$n > -1} { > set state(state) "header" > } > } elseif... > > and it works fine for the url I'm interested in, and also for the one > other url I tested it on. > > Please can someone who knows more than me about the http protocol and > [gets] (this shouldn't exclude too many people :-)) re-assure me that > the code change is legitimate. > > Can anyone suggest what may be going wrong on the server side? The > headers I'm reading from the "odd" site include "Server > Microsoft-IIS/6.0 X-Powered-By PHP/5.2.4 X-Powered-By ASP.NET", which > rather prejudices me against it. It's the first version of a new server, > so there might be problems at the server side, I'm sure they'd > appreciate any hints or pointers I can provide. > > Many thanks, > > -- > Alan I had similar experience and had to change the code in that spot to talk to my web server (a Tivo). It was inserting some line feeds (\n) before the http status. I changed the code to make sure that I had non- whitespace before I set the state to header. Even though the change was benign and would have made the package more robust, when I reported the bug, I was told that I should replace the web server. Even though I submitted the fix the maintainers refused to change the code. Bugs item #2874782 https://sourceforge.net/tracker/?func=detail&atid=110894&aid=2874782&group_id=10894
From: Alan Grunwald on 10 Mar 2010 08:07
Bezoar wrote: > On Mar 5, 10:08 am, Alan Grunwald <alan-clt-pos...(a)nospam.demon.co.uk> > wrote: >> Hi all, >> >> I'm trying to get some data via http. >> >> set tok [http::geturl $myUrl] >> puts stdout [http::code $tok] >> >> displays nothing. (I'm using http version 2.7.2 from the Active State >> teapot, and ActiveTcl 8.6b1.1 on Ubuntu.) >> >> I've done some digging and the problem seems to be towards the top >> of Event, where the following code may be found: >> >> if {$state(state) eq "connecting"} { >> set state(state) "header" >> if {[catch {gets $sock state(http)} n]} { >> return [Finish $token $n] >> } >> } elseif... >> >> It appears that the [gets] in this code snippet returns -1, because of >> this, when the code gets to the point where it reads the blank line >> marking the end of the headers, state(http) is zero length; the code >> seems to assume that this means it has received an HTTP/1.1 100 Continue >> response, and doesn't set its state to body. I'm not sure exactly what >> happens afterwards, but this is obviously a Bad Thing. >> >> I tried changing the code snippet above to >> >> if {$state(state) eq "connecting"} { >> if {[catch {gets $sock state(http)} n]} { >> return [Finish $token $n] >> } >> if {$n > -1} { >> set state(state) "header" >> } >> } elseif... >> >> and it works fine for the url I'm interested in, and also for the one >> other url I tested it on. >> >> Please can someone who knows more than me about the http protocol and >> [gets] (this shouldn't exclude too many people :-)) re-assure me that >> the code change is legitimate. >> >> Can anyone suggest what may be going wrong on the server side? The >> headers I'm reading from the "odd" site include "Server >> Microsoft-IIS/6.0 X-Powered-By PHP/5.2.4 X-Powered-By ASP.NET", which >> rather prejudices me against it. It's the first version of a new server, >> so there might be problems at the server side, I'm sure they'd >> appreciate any hints or pointers I can provide. >> >> Many thanks, >> >> -- >> Alan > > I had similar experience and had to change the code in that spot to > talk to my web server (a Tivo). It was inserting some line feeds (\n) > before the http status. I changed the code to make sure that I had non- > whitespace before I set the state to header. Even though the change > was benign and would have made the package more robust, when I > reported the bug, I was told that I should replace the web server. > Even though I submitted the fix the maintainers refused to change the > code. Bugs item #2874782 https://sourceforge.net/tracker/?func=detail&atid=110894&aid=2874782&group_id=10894 > It does seem that this bit of code has given some problems :-). My problem is definitely fixed by setting state to 'connecting' only when a [gets] completes successfully. Your proposed change would only set the state when the [gets] not only completes properly, but actually reads some data too. I wouldn't like to say whether the difference in approaches is large enough to be significant. Granted all that, changing the server doesn't seem to be a possibility in your case. Thanks for the response, -- Alan |