From: Captain Jack on
"Keith (Southend)G" <keith_harris9(a)hotmail.com> wrote in message
news:3d21649b-70d7-460e-8414-624e97f724ff(a)c29g2000yqd.googlegroups.com...
>
> Yes the file is bigger, I uploaded the latest one:
> http://www.southendweather.net/ogimet.txt
> This is actually my next problem, I have to download each Countries
> data one at a time, first setting the dates/time etc, see:
> http://www1.ogimet.com/synopsc.phtml.en
> I pick 38 different Countries, which takes me about 10 minutes.
> The size of the file depends on what time span I set, basically I'm
> grabbing it continuously and depending on what time I do it determines
> how many hours worth I need to catch up.

I don't know what criteria you're using for selecting your data, but that
can possibly be automated as well. The WebBrowser control can go to a web
page and retrieve it, then it gives you access to the page contents
(including form fields) and lets you fill in the forms programmatically.
It's a little tricky to use, though, and if they every change the web page
on you, you often have to change your program later.

> Would this speed that bit up, I timed it the next time and the latest
> file took 5 minutes and 20 seconds before it appeared in 'textbox1'?
> Again, this wasn't a problem before.

Actually no; in fact, it would slow it down slightly. The difference would
be that you could see it doing something instead of getting all your results
at the end. Sometimes that makes things seem faster, is all. :-)


> Out of interest where are you based?

I'm in Indianapolis, Indiana, a little east of the middle of the USA.

--
Jack


From: Keith (Southend)G on
On Jan 20, 4:02 pm, "Captain Jack" <CaptainJack1...(a)comcast.net>
wrote:
> "Keith (Southend)G" <keith_harr...(a)hotmail.com> wrote in message
>
> news:3d21649b-70d7-460e-8414-624e97f724ff(a)c29g2000yqd.googlegroups.com...
>
>
>
> > Yes the file is bigger, I uploaded the latest one:
> >http://www.southendweather.net/ogimet.txt
> > This is actually my next problem, I have to download each Countries
> > data one at a time, first setting the dates/time etc, see:
> >http://www1.ogimet.com/synopsc.phtml.en
> > I pick 38 different Countries, which takes me about 10 minutes.
> > The size of the file depends on what time span I set, basically I'm
> > grabbing it continuously and depending on what time I do it determines
> > how many hours worth I need to catch up.
>
> I don't know what criteria you're using for selecting your data, but that
> can possibly be automated as well. The WebBrowser control can go to a web
> page and retrieve it, then it gives you access to the page contents
> (including form fields) and lets you fill in the forms programmatically.
> It's a little tricky to use, though, and if they every change the web page
> on you, you often have to change your program later.
>
> > Would this speed that bit up, I timed it the next time and the latest
> > file took 5 minutes and 20 seconds before it appeared in 'textbox1'?
> > Again, this wasn't a problem before.
>
> Actually no; in fact, it would slow it down slightly. The difference would
> be that you could see it doing something instead of getting all your results
> at the end. Sometimes that makes things seem faster, is all. :-)
>
> > Out of interest where are you based?
>
> I'm in Indianapolis, Indiana, a little east of the middle of the USA.
>
> --
> Jack

Jack,

Firstly, thank you for spending so much time on that code, especially
explaining what it's doing, step by step. It's quite daunting at
first, but I guess after a while you get used to certain common code
techniques. The help you're giving me frees up more time to study it
more closely, I guess I'm in a bit of a catch 22 with time.

Ok, I've double checked the output of all the files last night and
everything looks good, this is a great time saver for me. Is there
anyway round the 5 minute wait when first clicking Button1 to populate
RichTextBox1? This I guess will only get longer the larger the file,
although at the most the file size would be double the one I uploaded
yesterday. (24 hours worth).

The collection of the data is another time consuming task, I'd be
interested if you had some ideas here? I haven't yet looked at how to
write code to retrieve data from the www. When I first started to run
through the tutorials I looked at the 'form', the easy bit :-) , but I
reckon you'll tell me I'm making it more difficult than needs be. This
is what it looked like, but I don't think you'd have a url bar at the
top, and I don't need TextBoxes, it could just save to a simple txt
file:

http://www.southendweather.net/VB_Ogimet.pdf

I then started to look at the breakup/differences for each 'send'
which is quite logical, although you'll notice in Spanish. The changes
for each send are: 'Country/Year/Month & Times). I've uploaded a word
file which shows this:

http://www.southendweather.net/ogilist.rtf

This file also lists ALL the Countries I select each time (over 40).
Fortunately, Firefox doesn't reset the times every time you go back
one page, so it's only the Country I have to change each time.
It would be great if I could set in one go ALL the Counties for a
given time span and then it grabbed them all into one file in one hit.
Not sure how flexible it could be, add more Countries perhaps? Any
ideas?

I maybe pushing my luck here, but I very much appreciate what you've
done for me.

Regards

Keith (Southend)
From: Captain Jack on
"Keith (Southend)G" <keith_harris9(a)hotmail.com> wrote in message
news:9d4eb339-87e3-4b78-b30b-77799d76292b(a)g1g2000yqi.googlegroups.com...
>
> Jack,
>
> Firstly, thank you for spending so much time on that code, especially
> explaining what it's doing, step by step. It's quite daunting at
> first, but I guess after a while you get used to certain common code
> techniques. The help you're giving me frees up more time to study it
> more closely, I guess I'm in a bit of a catch 22 with time.
>
> Ok, I've double checked the output of all the files last night and
> everything looks good, this is a great time saver for me. Is there
> anyway round the 5 minute wait when first clicking Button1 to populate
> RichTextBox1? This I guess will only get longer the larger the file,
> although at the most the file size would be double the one I uploaded
> yesterday. (24 hours worth).
>
> The collection of the data is another time consuming task, I'd be
> interested if you had some ideas here? I haven't yet looked at how to
> write code to retrieve data from the www. When I first started to run
> through the tutorials I looked at the 'form', the easy bit :-) , but I
> reckon you'll tell me I'm making it more difficult than needs be. This
> is what it looked like, but I don't think you'd have a url bar at the
> top, and I don't need TextBoxes, it could just save to a simple txt
> file:
>
> http://www.southendweather.net/VB_Ogimet.pdf
>
> I then started to look at the breakup/differences for each 'send'
> which is quite logical, although you'll notice in Spanish. The changes
> for each send are: 'Country/Year/Month & Times). I've uploaded a word
> file which shows this:
>
> http://www.southendweather.net/ogilist.rtf
>
> This file also lists ALL the Countries I select each time (over 40).
> Fortunately, Firefox doesn't reset the times every time you go back
> one page, so it's only the Country I have to change each time.
> It would be great if I could set in one go ALL the Counties for a
> given time span and then it grabbed them all into one file in one hit.
> Not sure how flexible it could be, add more Countries perhaps? Any
> ideas?
>
> I maybe pushing my luck here, but I very much appreciate what you've
> done for me.

No problem, I like doing stuff like this. :-)

On the long delay for loading the file, I had a similar experience today
when testing the program, but only the first time I read the file. Every
time I tried to read it after that, it read it very quickly. Since I
couldn't duplicate it, I wasn't able to track it down. It's possible that
the program is loading a library into memory, or the O/S is building a list
of all the files in the folder, or something. I'm pretty sure the delay is
happening outside the program, but I'm not sure where. If it is happening
outside the program, your form probably won't respond until eith .NET or the
O/S is done doing whatever it needs to do. My guess is that you'll see a bit
of a wait, then all of a sudden you'll see the text box filling with data,
fairly quickly. If your experience is like mine, if you close the program
and start it up again, it will read the file almost immediately the second
time.

I went ahead and changed the program so that the data class raises an event
every time it finishes reading a line of text, and also right before it
creates an output file. To do that, I had to move the code that does the
actual reading of the data from the New sub into a separate one, and call it
separately. The events weren't being processed inside the constructor for
the class. I changed the form so that it intercepts the events and populates
the text boxes from them, rather than looping through the results later. In
the event handlers, I called a method named DoEvents, which makes sure that
the textbox updates are visible immediately, and lets the rest of the form
respond to movement, if needed. I also added to the program to have it
disable the buttons and change the cursor to the "Wait" mode (usually an
hourglass) while it's reading; that way, you know it's working, and you know
when it's done. The updated code is here:

http://www.CaptainJack3s.com/Code/ogprog4.txt

As for grabbing the data off the web, that may take a bit. I've done that a
few times before, using the WebBrowser control. I'm not certain what we need
to do, and it may take some playing with. I'll take a look at it, but it's
the kind of thing I probably won't get to until the weekend, and it may take
a couple of weekends of playing with it to get it right. The WebBrowser
control raises events every time it gets a web page, and if the page is
sending multiple bits (which will happen a lot with ASP pages and sometimes
with PHP) there can be a lot of event occurences to sift through before you
get the data you want. I'm working on an animation right now (one of my
other hobbies) which will fill up my nights this week, but I should have
some time either Saturday or Sunday (whichever day my team isn't playing...
go Colts and all that <G>). I'll try to create a component that goes to the
web site and grabs some data, then we can look at how to track date ranges,
countries read, and so on.

--
Jack


From: Keith (Southend)G on
On Jan 20, 7:43 pm, "Captain Jack" <CaptainJack1...(a)comcast.net>
wrote:
> "Keith (Southend)G" <keith_harr...(a)hotmail.com> wrote in message
The updated code is here:
>
> http://www.CaptainJack3s.com/Code/ogprog4.txt
>

That link doesn't appear to work Jack.

Thanks

Keith (Southend)
From: Keith (Southend)G on
On Jan 20, 7:43 pm, "Captain Jack" <CaptainJack1...(a)comcast.net>
wrote:
> "Keith (Southend)G" <keith_harr...(a)hotmail.com> wrote in message
>
> news:9d4eb339-87e3-4b78-b30b-77799d76292b(a)g1g2000yqi.googlegroups.com...
>
>
>
>
>
> > Jack,
>
> > Firstly, thank you for spending so much time on that code, especially
> > explaining what it's doing, step by step. It's quite daunting at
> > first, but I guess after a while you get used to certain common code
> > techniques. The help you're giving me frees up more time to study it
> > more closely, I guess I'm in a bit of a catch 22 with time.
>
> > Ok, I've double checked the output of all the files last night and
> > everything looks good, this is a great time saver for me. Is there
> > anyway round the 5 minute wait when first clicking Button1 to populate
> > RichTextBox1? This I guess will only get longer the larger the file,
> > although at the most the file size would be double the one I uploaded
> > yesterday. (24 hours worth).
>
> > The collection of the data is another time consuming task, I'd be
> > interested if you had some ideas here? I haven't yet looked at how to
> > write code to retrieve data from the www. When I first started to run
> > through the tutorials I looked at the 'form', the easy bit :-) , but I
> > reckon you'll tell me I'm making it more difficult than needs be. This
> > is what it looked like, but I don't think you'd have a url bar at the
> > top, and I don't need TextBoxes, it could just save to a simple txt
> > file:
>
> >http://www.southendweather.net/VB_Ogimet.pdf
>
> > I then started to look at the breakup/differences for each 'send'
> > which is quite logical, although you'll notice in Spanish. The changes
> > for each send are: 'Country/Year/Month & Times). I've uploaded a word
> > file which shows this:
>
> >http://www.southendweather.net/ogilist.rtf
>
> > This file also lists ALL the Countries I select each time (over 40).
> > Fortunately, Firefox doesn't reset the times every time you go back
> > one page, so it's only the Country I have to change each time.
> > It would be great if I could set in one go ALL the Counties for a
> > given time span and then it grabbed them all into one file in one hit.
> > Not sure how flexible it could be, add more Countries perhaps? Any
> > ideas?
>
> > I maybe pushing my luck here, but I very much appreciate what you've
> > done for me.
>
> No problem, I like doing stuff like this. :-)
>
> On the long delay for loading the file, I had a similar experience today
> when testing the program, but only the first time I read the file. Every
> time I tried to read it after that, it read it very quickly. Since I
> couldn't duplicate it, I wasn't able to track it down. It's possible that
> the program is loading a library into memory, or the O/S is building a list
> of all the files in the folder, or something. I'm pretty sure the delay is
> happening outside the program, but I'm not sure where. If it is happening
> outside the program, your form probably won't respond until eith .NET or the
> O/S is done doing whatever it needs to do. My guess is that you'll see a bit
> of a wait, then all of a sudden you'll see the text box filling with data,
> fairly quickly. If your experience is like mine, if you close the program
> and start it up again, it will read the file almost immediately the second
> time.
>
> I went ahead and changed the program so that the data class raises an event
> every time it finishes reading a line of text, and also right before it
> creates an output file. To do that, I had to move the code that does the
> actual reading of the data from the New sub into a separate one, and call it
> separately. The events weren't being processed inside the constructor for
> the class. I changed the form so that it intercepts the events and populates
> the text boxes from them, rather than looping through the results later. In
> the event handlers, I called a method named DoEvents, which makes sure that
> the textbox updates are visible immediately, and lets the rest of the form
> respond to movement, if needed. I also added to the program to have it
> disable the buttons and change the cursor to the "Wait" mode (usually an
> hourglass) while it's reading; that way, you know it's working, and you know
> when it's done. The updated code is here:
>
> http://www.CaptainJack3s.com/Code/ogprog4.txt
>
> As for grabbing the data off the web, that may take a bit. I've done that a
> few times before, using the WebBrowser control. I'm not certain what we need
> to do, and it may take some playing with. I'll take a look at it, but it's
> the kind of thing I probably won't get to until the weekend, and it may take
> a couple of weekends of playing with it to get it right. The WebBrowser
> control raises events every time it gets a web page, and if the page is
> sending multiple bits (which will happen a lot with ASP pages and sometimes
> with PHP) there can be a lot of event occurences to sift through before you
> get the data you want. I'm working on an animation right now (one of my
> other hobbies) which will fill up my nights this week, but I should have
> some time either Saturday or Sunday (whichever day my team isn't playing....
> go Colts and all that <G>). I'll try to create a component that goes to the
> web site and grabs some data, then we can look at how to track date ranges,
> countries read, and so on.
>
> --
> Jack

http://www.captainjack3d.com/Code/ogprog4.txt

Ah, found it :-)