Prev: found out why NOT latest kernel in debian..:-)
Next: who maintains the man pages?
From: Clark Smith on 14 Mar 2010 19:21
This is not strictly Linux-only, but maybe somebody in this forum
can provide pointers.

I am looking for a code to do the following:

1) Loop over all of the URLs in my bookmarks database.

2) For each URL try and establish a connection.

3) If the connection succeeds move on to the next URL.

4) Otherwise increment the number of times that attempting to
connect to this URL has not succeeded.

5) If this number is greater than some preset value then remove
this URL from my bookmarks database.

Ideally, this would be a shell-script that I would from cron at
specified intervals.

Anybody know if such a thing (or similar) exists?

From: Bit Twister on 14 Mar 2010 19:40
On Sun, 14 Mar 2010 23:21:29 +0000 (UTC), Clark Smith wrote:
> This is not strictly Linux-only, but maybe somebody in this forum
> can provide pointers.
>
> I am looking for a code to do the following:
>
> 1) Loop over all of the URLs in my bookmarks database.
>
> 2) For each URL try and establish a connection.
>
> 3) If the connection succeeds move on to the next URL.
>
> 4) Otherwise increment the number of times that attempting to
> connect to this URL has not succeeded.
>
> 5) If this number is greater than some preset value then remove
> this URL from my bookmarks database.
>
> Ideally, this would be a shell-script that I would from cron at
> specified intervals.
>
> Anybody know if such a thing (or similar) exists?

No idea about exists, but you can use wget, curl, axel, html2text or aria2
to do a fetch url and script checks the return status or file count.
I know wget has a counter to limit attempts.

Years ago I pulled my bookmarks into an ascii file.
Now I add keywords + url to the file. I use a script to grep it for
keywords. Example:

$ urls bash doc
http://tldp.org/LDP/abs/html/index.html ! bash script advanced documentation
http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html ! basic shell bash doc
http://gentoo-wiki.com/MAN_bash ! documentation
http://cfaj.freeshell.org/shell ! bash script tips usage doc
http://mywiki.wooledge.org/BashFAQ/050 ! bash script variable expansion documentation
From: Phred Phungus on 14 Mar 2010 20:15
Bit Twister wrote:
> On Sun, 14 Mar 2010 23:21:29 +0000 (UTC), Clark Smith wrote:
>> This is not strictly Linux-only, but maybe somebody in this forum
>> can provide pointers.
>>
>> I am looking for a code to do the following:
>>
>> 1) Loop over all of the URLs in my bookmarks database.
>>
>> 2) For each URL try and establish a connection.
>>
>> 3) If the connection succeeds move on to the next URL.
>>
>> 4) Otherwise increment the number of times that attempting to
>> connect to this URL has not succeeded.
>>
>> 5) If this number is greater than some preset value then remove
>> this URL from my bookmarks database.
>>
>> Ideally, this would be a shell-script that I would from cron at
>> specified intervals.
>>
>> Anybody know if such a thing (or similar) exists?
>
> No idea about exists, but you can use wget, curl, axel, html2text or aria2
> to do a fetch url and script checks the return status or file count.
> I know wget has a counter to limit attempts.
>
> Years ago I pulled my bookmarks into an ascii file.
> Now I add keywords + url to the file. I use a script to grep it for
> keywords. Example:
>
> $ urls bash doc
> http://tldp.org/LDP/abs/html/index.html ! bash script advanced documentation
> http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html ! basic shell bash doc
> http://gentoo-wiki.com/MAN_bash ! documentation
> http://cfaj.freeshell.org/shell ! bash script tips usage doc
> http://mywiki.wooledge.org/BashFAQ/050 ! bash script variable expansion documentation

BT, I'm doing something similar with my linuxlog. Can you post the script?
--
fred
From: Bit Twister on 14 Mar 2010 20:58
On Sun, 14 Mar 2010 18:15:21 -0600, Phred Phungus wrote:
> Bit Twister wrote:

>> Years ago I pulled my bookmarks into an ascii file.
>> Now I add keywords + url to the file. I use a script to grep it for
>> keywords. Example:
>>
>> $ urls bash doc
>> http://tldp.org/LDP/abs/html/index.html ! bash script advanced documentation
>
> BT, I'm doing something similar with my linuxlog. Can you post the script?

Nope. It is an ambidexterous script, and here is a link you can use.
http://groups.google.com/group/alt.os.linux.mandriva/msg/fabb730cdb3912db?dmode=source
From: J G Miller on 14 Mar 2010 21:42
On Sun, 14 Mar 2010 23:21:29 +0000, Clark Smith wrote:

> Ideally, this would be a shell-script that I would from cron at
> specified intervals.

This would be an ideal and fairly simple job to write in PERL.

Far more complicated than what you want, but should give you
an idea of how to write PERL to grab URLs

<http://www.webmasterworld.COM/forum13/1563.htm>

and it does everything from basic commands using Socket only
and no other modules.
 |  Next  |  Last
Pages: 1 2
Prev: found out why NOT latest kernel in debian..:-)
Next: who maintains the man pages?