Prev: Tk freezes with elided text
Next: Using A Variable Throughout My Application
From: achillez on 25 Jun 2010 20:59
I've been experimenting with using geturl to download stock history
from yahoo.com and I noticed that it does not download the entire
file. Less of the file is received when -channel is used vs. just
dumping straight to the state(body) buffer. Can anyone offer a
suggestion on what I'm doing wrong? Is geturl known to be buggy w/
some sites or on some OS's (I'm using Win7) ?

Here's an example of the script I ran (note - httpcopy is a proc I
created from the Tcl help pages)

httpcopy http://ichart.finance.yahoo.com/table.csv?s=GOOG&a=06&b=9&c=1970&d=05&e=25&f=2010&g=d&ignore=.csv
"output/goog.csv"
From: Gerald W. Lester on 26 Jun 2010 00:31
achillez wrote:
> I've been experimenting with using geturl to download stock history
> from yahoo.com and I noticed that it does not download the entire
> file. Less of the file is received when -channel is used vs. just
> dumping straight to the state(body) buffer. Can anyone offer a
> suggestion on what I'm doing wrong? Is geturl known to be buggy w/
> some sites or on some OS's (I'm using Win7) ?
>
> Here's an example of the script I ran (note - httpcopy is a proc I
> created from the Tcl help pages)
>
> httpcopy http://ichart.finance.yahoo.com/table.csv?s=GOOG&a=06&b=9&c=1970&d=05&e=25&f=2010&g=d&ignore=.csv
> "output/goog.csv"

You would need to post your httpcopy code for us to be able to help you.

Do you do a ::http::wait on the token returned by ::http::geturl???


--
+------------------------------------------------------------------------+
| Gerald W. Lester, President, KNG Consulting LLC |
| Email: Gerald.Lester(a)kng-consulting.net |
+------------------------------------------------------------------------+
From: achillez on 26 Jun 2010 01:34
On Jun 25, 9:31 pm, "Gerald W. Lester" <Gerald.Les...(a)KnG-
Consulting.net> wrote:
> achillez wrote:
> > I've been experimenting with using geturl to download stock history
> > from yahoo.com and I noticed that it does not download the entire
> > file. Less of the file is received when -channel is used vs. just
> > dumping straight to the state(body) buffer. Can anyone offer a
> > suggestion on what I'm doing wrong? Is geturl known to be buggy w/
> > some sites or on some OS's (I'm using Win7) ?
>
> > Here's an example of the script I ran (note - httpcopy is a proc I
> > created from the Tcl help pages)
>
> > httpcopyhttp://ichart.finance.yahoo.com/table.csv?s=GOOG&a=06&b=9&c=1970&d=05...
> > "output/goog.csv"
>
> You would need to post your httpcopy code for us to be able to help you.
>
> Do you do a ::http::wait on the token returned by ::http::geturl???
>
> --
> +------------------------------------------------------------------------+
> | Gerald W. Lester, President, KNG Consulting LLC                        |
> | Email: Gerald.Les...(a)kng-consulting.net                                |
> +------------------------------------------------------------------------+


Thanks for the help. Here is the code, this is pretty much verbatim
from the Tcl help pages. I tried adding the ::http::wait call but it
didn't help so I commented it out.


proc httpcopy { url file {chunk 4096} } {
set out [open $file w]
set token [::http::geturl $url -channel $out -keepalive 1 \
-progress httpCopyProgress -blocksize $chunk -timeout 10000 -
binary 0]
# ::http::wait $token
close $out

# This ends the line started by httpCopyProgress
puts stderr ""

#upvar #0 $token state
puts "Token: $token"
upvar #0 $token state
set max 0
foreach {name value} $state(meta) {
if {[string length $name] > $max} {
set max [string length $name]
}
if {[regexp -nocase ^location$ $name]} {
# Handle URL redirects
puts stderr "Location:$value"
return [httpcopy [string trim $value] $file $chunk]
}
}
incr max
foreach {name value} $state(meta) {
puts [format "%-*s %s" $max $name: $value]
}

return $token
}

proc httpCopyProgress {args} {
puts -nonewline stderr .
flush stderr
}
From: achillez on 26 Jun 2010 21:38
On Jun 26, 2:29 am, Keith <kilowattra...(a)use-reply-to.invalid> wrote:
> achillez wrote in
> <07a1543a-8d94-4137-bd72-d691fc6f3...(a)q36g2000pri.googlegroups.com> on Fri, 25
> Jun 2010 17:59:15 -0700 (PDT):
>
>
>
> >I've been experimenting with using geturl to download stock history
> >from yahoo.com and I noticed that it does not download the entire
> >file. Less of the file is received when -channel is used vs. just
> >dumping straight to the state(body) buffer. Can anyone offer a
> >suggestion on what I'm doing wrong? Is geturl known to be buggy w/
> >some sites or on some OS's (I'm using Win7) ?
>
> >Here's an example of the script I ran (note - httpcopy is a proc I
> >created from the Tcl help pages)
>
> >httpcopy
> >http://ichart.finance.yahoo.com/table.csv?s=GOOG&a=06&b=9&c=1970&d=05...
> >"output/goog.csv"
>
> There is a bug in the http package. IIRC you use the switch
> -blocksize -1
> There is thread about it here on the newsgroup, so a Google Groups search will
> help run it down.
>
> --
> --
> Best Regards, Keithhttp://home.comcast.net/~kilowattradio/

Thanks, I think I found the thread at:

http://groups.google.com/group/comp.lang.tcl/browse_thread/thread/6deca54954e61cfd/a4dce25f8a50f31f?lnk=gst&q=http+bug+blocksize#a4dce25f8a50f31f

Interestingly, they said that this should be fixed in Tcl 8.6b1.2, I
just reinstalled to Tcl8.6b1.2 and see that the problem still exists.
If the if statement below is changed to "if {1} {" then it doesn't
download the entire file.


if {0} {
set out [open $file w]
fconfigure $out -translation binary
set token [::http::geturl $url -channel $out -keepalive 1 \
-progress httpCopyProgress -blocksize $chunk -timeout 10000 -
binary 0]
close $out
#::http::wait $token
} else {
set token [::http::geturl $url]
set outfile [open $file w]
fconfigure $outfile -translation binary
puts -nonewline $outfile [::http::data $token]
close $outfile
}

Any thoughts on what the problem still could be? Should I file a bug
report on this?


From: achillez on 26 Jun 2010 22:30
On Jun 26, 6:38 pm, achillez <mew...(a)gmail.com> wrote:
> On Jun 26, 2:29 am, Keith <kilowattra...(a)use-reply-to.invalid> wrote:
>
>
>
> > achillez wrote in
> > <07a1543a-8d94-4137-bd72-d691fc6f3...(a)q36g2000pri.googlegroups.com> on Fri, 25
> > Jun 2010 17:59:15 -0700 (PDT):
>
> > >I've been experimenting with using geturl to download stock history
> > >from yahoo.com and I noticed that it does not download the entire
> > >file. Less of the file is received when -channel is used vs. just
> > >dumping straight to the state(body) buffer. Can anyone offer a
> > >suggestion on what I'm doing wrong? Is geturl known to be buggy w/
> > >some sites or on some OS's (I'm using Win7) ?
>
> > >Here's an example of the script I ran (note - httpcopy is a proc I
> > >created from the Tcl help pages)
>
> > >httpcopy
> > >http://ichart.finance.yahoo.com/table.csv?s=GOOG&a=06&b=9&c=1970&d=05...
> > >"output/goog.csv"
>
> > There is a bug in the http package. IIRC you use the switch
> > -blocksize -1
> > There is thread about it here on the newsgroup, so a Google Groups search will
> > help run it down.
>
> > --
> > --
> > Best Regards, Keithhttp://home.comcast.net/~kilowattradio/
>
> Thanks, I think I found the thread at:
>
> http://groups.google.com/group/comp.lang.tcl/browse_thread/thread/6de...
>
> Interestingly, they said that this should be fixed in Tcl 8.6b1.2, I
> just reinstalled to Tcl8.6b1.2 and see that the problem still exists.
> If the if statement below is changed to "if {1} {" then it doesn't
> download the entire file.
>
> if {0} {
>    set out [open $file w]
>    fconfigure $out -translation binary
>    set token [::http::geturl $url -channel $out -keepalive 1 \
>           -progress httpCopyProgress -blocksize $chunk -timeout 10000 -
> binary 0]
>    close $out
>    #::http::wait $token} else {
>
>    set token [::http::geturl $url]
>    set outfile [open $file w]
>    fconfigure $outfile -translation binary
>    puts -nonewline $outfile [::http::data $token]
>    close $outfile
>
> }
>
> Any thoughts on what the problem still could be? Should I file a bug
> report on this?

It appears the problem is not OS dependent. I just replicated the same
problem on Ubuntu with Tcl 8.6b1.2.
 |  Next  |  Last
Pages: 1 2
Prev: Tk freezes with elided text
Next: Using A Variable Throughout My Application