Prev: showing progress when doing copy
Next: Tkdnd and tablelist
From: tom.rmadilo on 20 Mar 2010 11:43
On Mar 20, 1:23 am, Frédéric Bonnet <fredericbon...(a)free.fr> wrote:
> Tcl Bliss wrote:
> >> This means
> >> that it cannot be used reliably with large files; they will need several
> >> successive fcopy with -size under the integer limit, until the desired value is
> >> reached.
>
> > Actually, this is exactly the point of using -size. If you want want a
> > single transfer, just use fcopy without the "-size" option. "-size" is
> > meant to be used to break the transfer into smaller chunks, so you can
> > do something else with it, like transfer monitoring, progress report,
> > speed control, bandwidth limiting etc.
>
> Sure, but there are other perfectly good reasons to use -size. I've encountered
> this "feature" with a HTTP Media Server of mine: the client may send a "Range:"
> header with arbitrary values, for example when playing back videos from a given
> position, so it makes sense to issue a single fcopy with the range length given
> to -size. And internally fcopy chunks the output anyway using the channel
> -buffersize. The whole point of fcopy here is to fire-and-forget the channel
> copy and let Tcl manage the background work.
>
> FYI fcopy works fine with -size values of 2GB, and I've worked around this
> limitation without much work. But my point is: if the -size limit makes sense
> (which I doubt), it should be documented. I guess that's the best option anyway,
> as there are certainly other priorities than fixing this arbitrary limitation.

This use makes sense, so do you seek to the starting position and then
use -size as way to interrupt [fcopy] after one background copy (which
probably includes more than one buffersize copy)?

But, for instance, this example in the [fcopy] manpages makes no
sense:

proc CopyMore {in out chunk bytes {error {}}} {
global total done
incr total $bytes
if {([string length $error] != 0) || [eof $in]} {
set done $total
close $in
close $out
} else {
fcopy $in $out -command [list CopyMore $in $out $chunk] \
-size $chunk
}
}
set in [open $file1]
set out [socket $server $port]
set chunk 1024
set total 0
fcopy $in $out -command [list CopyMore $in $out $chunk] -size $chunk
vwait done

The same type of code is used in http::geturl in the mistaken belief
that this will keep the client from freezing the rest of the
application. (It will, internally [fcopy] already handles this
situation...a great feature.)
From: Donal K. Fellows on 20 Mar 2010 11:44
On 20/03/2010 08:23, Frédéric Bonnet wrote:
> FYI fcopy works fine with -size values of 2GB, and I've worked around
> this limitation without much work. But my point is: if the -size limit
> makes sense (which I doubt), it should be documented. I guess that's the
> best option anyway, as there are certainly other priorities than fixing
> this arbitrary limitation.

I wasn't aware of the limitation before now, but it's now fixed (I
think; testing very large files isn't very portable) in the CVS HEAD.
Also, you would have had problems with the reported number of bytes
transferred when using no -size as that was an int too.

Donal.
From: Tcl Bliss on 20 Mar 2010 13:05
On Mar 20, 1:23 am, Frédéric Bonnet <fredericbon...(a)free.fr> wrote:
> Tcl Bliss wrote:
> >> This means
> >> that it cannot be used reliably with large files; they will need several
> >> successive fcopy with -size under the integer limit, until the desired value is
> >> reached.
>
> > Actually, this is exactly the point of using -size. If you want want a
> > single transfer, just use fcopy without the "-size" option. "-size" is
> > meant to be used to break the transfer into smaller chunks, so you can
> > do something else with it, like transfer monitoring, progress report,
> > speed control, bandwidth limiting etc.
>
> Sure, but there are other perfectly good reasons to use -size. I've encountered
> this "feature" with a HTTP Media Server of mine: the client may send a "Range:"
> header with arbitrary values, for example when playing back videos from a given
> position, so it makes sense to issue a single fcopy with the range length given
> to -size. And internally fcopy chunks the output anyway using the channel
> -buffersize. The whole point of fcopy here is to fire-and-forget the channel
> copy and let Tcl manage the background work.
>
> FYI fcopy works fine with -size values of 2GB, and I've worked around this
> limitation without much work. But my point is: if the -size limit makes sense
> (which I doubt), it should be documented. I guess that's the best option anyway,
> as there are certainly other priorities than fixing this arbitrary limitation.

That's interesting. Implementing "Range:" functionality has been on my
to-do list. Haven't gotten to it yet.
BTW, which media player/file combination support "Range:" requests? I
wasn't aware there were any, except for, may be, Ipod/Iphone.
From: Donal K. Fellows on 20 Mar 2010 13:44
On 20 Mar, 15:44, "Donal K. Fellows"
<donal.k.fell...(a)manchester.ac.uk> wrote:
> I wasn't aware of the limitation before now, but it's now fixed (I
> think; testing very large files isn't very portable) in the CVS HEAD.
> Also, you would have had problems with the reported number of bytes
> transferred when using no -size as that was an int too.

BTW, there's no plan to backport this to 8.5 because it impacts an
exported (but unsupported) API. Workaround is just to work with chunks
smaller than 2GB at the script level, and then write Tcl code to
handle doing more than that by calling [fcopy] repeatedly.

Donal.
From: Frédéric Bonnet on 22 Mar 2010 17:15
tom.rmadilo wrote:
> This use makes sense, so do you seek to the starting position and then
> use -size as way to interrupt [fcopy] after one background copy (which
> probably includes more than one buffersize copy)?

Yes, that's the idea. Internally Tcl will potentially issue several I/O
operations under the buffersize limit, but on the script level you'll only get
one completion event. Of course to overcome the int size limitation you have to
issue several consecutive fcopy (by daisy-chaining completion callbacks), so you
get more events than needed, but this only happens with very large files
(typically DVD or BD disk images). Unless you WANT periodic events for logging
or feedback purpose, in this case you'll set -size to a much smaller value as in
your example.
First  |  Prev  |  Next  |  Last
Pages: 1 2 3
Prev: showing progress when doing copy
Next: Tkdnd and tablelist