From: lrhorer on
unruh wrote:

> On 2010-01-11, lrhorer <lrhorer(a)satx.rr.com> wrote:
>> unruh wrote:
>>
>>> On 2010-01-10, lrhorer <lrhorer(a)satx.rr.com> wrote:
>>>>
>>>> Hello,
>>>>
>>>> I originally posted a query about this over on alt.linux,
>>>> and
>>>> someone
>>>> there suggested I post a query here. I am using wvdial to
>>>> provide "dial up" broadband access to the Cricket
>>>> wireless network using a Cal-Comp A600 GSM wireless modem on a
>>>> headless
>>>> router.. It's working fairly well, but there is one serious issue
>>>> I
>>>> need to resolve. After 12 hours online, the ISP shuts down the
>>>> link. When this happens, the modem sometimes locks up, at least as
>>>> far as
>>>> wvdial is concerned. An attempt to dial again fails.
>>>> Occasionally, the modem truly hangs, so that even a soft reboot
>>>> won't recover the
>>>> session. Since wvdial produces no logs, no externally readable
>>>> status structure, and no exit codes, it's difficult to tell what
>>>> action the router needs to take when the ppp link is down.
>>>>
>>>> Does anyone know of a slightly more sophisticated or
>>>> better supported alternative to wvdial? I am looking at linuxconf,
>>>> but there doesn't seem to be a Debian package for it, and the RPM
>>>> package
>>>> won't load under alien. The source code fails with a fair number
>>>> of
>>>> errors. I would really rather not start digging into problems with
>>>> the
>>>> RPM package or in the source code if I don't have to. The fact the
>>>> newest version of linuxconf is from 2005 also suggests it may not
>>>> be actively maintained, either
>>>
>>> Try looking at www.theory.physics.ubc.ca/ppp-linux.html for
>>> techniques for debugging your ppp connection.
>>
>> The problem isn't at the ppn layer. The problem lies at the hardware
>> layer, epcifically the modem locking up.
>
> And you know this how? And if you are so sure why are you asking here
> since hardware fixes do not get handled by Linux user groups.
>
>>
>>> wvdial is simply a font end for the
>>> thing that does the actual work, namely pppd. pppd can and does
>>> produce error messages to the daemon and local2 facilities.
>>
>> Yes, but there are no errors at that level, at least not until the
>> modem locks up.
>
> Have you switched on logging for ppd? Have you set up the daemon and
> local2 facilities to log to a file?
> daemon.*;local2.* /var/log/daemonlog
> in /etc/syslog.conf

Yeah, I thought so. Logging is enabled (to daemon.log, actually, not
daemonlog), and log rotation is enabled for daemon.log. Pppd reports
no errors in daemon.log.

> and then do
> killall -1 syslogd

'No need, since (1) daemon.log was already enabled, and (2) The router
has been rebooted innumerable times.

> Have you looked at that file?

Yes, of course, as well as syslog, kern.log, mesages.log, dmesg, and of
course the several log files generated by the scripts I wrote to handle
the networking.

>>> You need
>>> to tell your system to record those and tell pppd to actually output
>>> the debug messages.
>>>
>>> One thing you could try is to kill
>>> killall pppd
>>> as root
>>> before you try again.
>>
>> No, I'm not using pppd, and ppp is dead whenever the connection is
>> closed.
>
> ??? pppd is THE daemon for using ppp, and wvdial uses pppd
>
> ps auxww|grep pppd

'Produces nothing. This was of course the first thing I checked. I'm
not a noob.

>>> Alternatively, have your system itself close the connection after
>>> 11hr 45 min, and then restart it.
>>
>> That's what I am doing, or trying to do, although I use 11hr
>> 17 min, in
>> fact.
>>
>>> -- you must be paying pretty hefty
>>> wireless bills to to be logged on for over 12hr a day.
>>
>> No, Cricket is broadband wireless. Or it is SUPPOSED to be. If it
>> were
>
> What do you mean by wireless? GSM? In which case I am really surprized
> that they are not charging you through the teeth.
>
>
>> truly broadband, they would not shut down the link after 12 hours.
>> Nonetheless, there are no fees for connection time. Broadband
>> routers are always logged on 24/7.
>>

From: lrhorer on
unruh wrote:

> On 2010-01-11, lrhorer <lrhorer(a)satx.rr.com> wrote:
>> unruh wrote:
>>
>>> On 2010-01-11, lrhorer <lrhorer(a)satx.rr.com> wrote:
>>>> unruh wrote:
>>>>
>>>>> On 2010-01-10, lrhorer <lrhorer(a)satx.rr.com> wrote:
>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> I originally posted a query about this over on alt.linux,
>>>>>> and
>>>>>> someone
>>>>>> there suggested I post a query here. I am using wvdial to
>>>>>> provide "dial up" broadband access to the Cricket
>>>>>> wireless network using a Cal-Comp A600 GSM wireless modem on a
>>>>>> headless
>>>>>> router.. It's working fairly well, but there is one serious
>>>>>> issue I
>>>>>> need to resolve. After 12 hours online, the ISP shuts down the
>>>>>> link. When this happens, the modem sometimes locks up, at least
>>>>>> as far as
>>>>>> wvdial is concerned. An attempt to dial again fails.
>>>>>> Occasionally, the modem truly hangs, so that even a soft reboot
>>>>>> won't recover the
>>>>>> session. Since wvdial produces no logs, no externally readable
>>>>>> status structure, and no exit codes, it's difficult to tell what
>>>>>> action the router needs to take when the ppp link is down.
>>>>>>
>>>>>> Does anyone know of a slightly more sophisticated or
>>>>>> better supported alternative to wvdial? I am looking at
>>>>>> linuxconf, but there doesn't seem to be a Debian package for it,
>>>>>> and the RPM package
>>>>>> won't load under alien. The source code fails with a fair number
>>>>>> of
>>>>>> errors. I would really rather not start digging into problems
>>>>>> with the
>>>>>> RPM package or in the source code if I don't have to. The fact
>>>>>> the newest version of linuxconf is from 2005 also suggests it may
>>>>>> not be actively maintained, either
>>>>>
>>>>> Try looking at www.theory.physics.ubc.ca/ppp-linux.html for
>>>>> techniques for debugging your ppp connection.
>>>>
>>>> The problem isn't at the ppn layer. The problem lies at the
>>>> hardware layer, epcifically the modem locking up.
>>>
>>> And you know this how? And if you are so sure why are you asking
>>> here since hardware fixes do not get handled by Linux user groups.
>>
>> It does not appear to be a problem with the hardware, but
>> rather the
>> hardware layer. The modem has been replaced, and the behavior after
>> replacement is the same as the behavior before replacement. Instead
>> of returning coherent responses in wvdial when the modem is
>> instructed to
>> dial, it returns gibberish. Other commands return the "OK" response.
>
> Sounds like the pppd/chat layer to me. Ie, the response you are
> getting back is a garbled version of the negotiation.
>
> Anyway, have you tried to set up debug logging and looking at the
> files yet? If not, I have no idea why you came here for help, since
> you seem to be convinced that your theory, without testing, is right.

I don't have a theory, and I'm not looking for any help developing one.
All I am asking is if someone knows of an application which is better
suited to automation than wvdial. I know how to troubleshoot, and if
any of these failures are the result of a systemic problem within my
sphere of control, then I will resolve them as such. Indeed, the
hardware deregistration failure is almost certainly going to require
building some hardware, and I have already designed the hardware to
handle the situation along with the software to control it, but what I
want is a more sophisticated set of hooks to enable a faster and more
reliable recovery system, calling the recovery code best suited to the
specific failure mode at hand. Forcing the hardware to shut power off
to the modem and bring it back up whenever connectivity is lost is not
the best solution, but hanging up the modem only works if the failure
is fairly benign, and even soft rebooting the router doesn't clear one
of the common failure modes.

>>>>> wvdial is simply a font end for the
>>>>> thing that does the actual work, namely pppd. pppd can and does
>>
>> Unless I ma mistaken, pppd only does its work after layer 1
>> connectivity is established, is that not correct? I will
>> double-check. She is bringing the router over this afternoon.
>
> No. pppd does the whole negotiation. wvdial/chat set up the modem and
> dial the phone, and then hand off to pppd to do all negotiation.

Pppd is called before the carrier is established? Are you certain? In
any case, the point is, in at least two of the more common failure
modes, pppd is never even called by wvdial. In one of them, the modem
interface device, /dev/ACM0, does not even exist, so wvdial can't even
dial the modem, let alone hand off to pppd.

>>>>> produce error messages to the daemon and local2 facilities.
>>>>
>>>> Yes, but there are no errors at that level, at least not until the
>>>> modem locks up.
>>>
>>> Have you switched on logging for ppd? Have you set up the daemon and
>>> local2 facilities to log to a file?

As I mentioned, I checked, and they are there.

>>
>> No, but since the lock-up occurs before pppd is called, there
>> doesn't
>> seem to be much point. Again, I will double-check to make sure this
>> is the case.
>
> Sheesh.

So now you are giving me an attitude because rather than assume I was
right, I agreed to go back and check to make sure I hadn't missed
something?

>>> daemon.*;local2.* /var/log/daemonlog
>>> in /etc/syslog.conf
>>> and then do
>>> killall -1 syslogd
>>>
>>> Have you looked at that file?
>>
>> Syslog? Of course. 'No errors. The daemon log also has bo
>> error in
>> it. I'll try adding the specific targets you mention, if they are
>> not already there.
>
> No, the file where daemon and local2 logfacilities get written to. In
> the above it is /var/log/daemonlog.

Yes, of course I checked all the files to which anything was being
written. Look, I mean no offense, but I never asked for any help
troubleshooting this. If you will read my original post, you will see I
am only asking if anyone knows of alternative applications which will
allow me greater flexibility in interfacing my own code with the
communications applications, not help with troubleshooting. If chat is
the best answer, then I'll write a chat script, but I see no reason to
re-invent the wheel if better solutions already exist.

Although I will indeed further investigate the root causes of the
failures, right now that effort is incidental to the development of
routines which will properly and efficiently handle failures when they
do inevitably occur. I would rather not take the "shotgun" approach of
simply killing power to the modem and bringing it back up whenever any
sort of failure occurs.

>>>>> You need
>>>>> to tell your system to record those and tell pppd to actually
>>>>> output the debug messages.
>>>>>
>>>>> One thing you could try is to kill
>>>>> killall pppd
>>>>> as root
>>>>> before you try again.
>>>>
>>>> No, I'm not using pppd, and ppp is dead whenever the connection is
>>>> closed.
>>>
>>> ??? pppd is THE daemon for using ppp, and wvdial uses pppd
>>>
>>> ps auxww|grep pppd
>>
>> Returns nothing, as I recall. I will check again when the
>> system gets
>> here, if I can duplicate the problem. Troubleshooting network issues
>> remotely is a real challenge.
>>
>>>>> Alternatively, have your system itself close the connection after
>>>>> 11hr 45 min, and then restart it.
>>>>
>>>> That's what I am doing, or trying to do, although I use
>>>> 11hr 17 min, in
>>>> fact.
>>>>
>>>>> -- you must be paying pretty hefty
>>>>> wireless bills to to be logged on for over 12hr a day.
>>>>
>>>> No, Cricket is broadband wireless. Or it is SUPPOSED to be. If it
>>>> were
>>>
>>> What do you mean by wireless? GSM? In which case I am really
>>> surprized that they are not charging you through the teeth.
>>
>> Actually, I believe it is CDMA, but there are a number of
>> broadband
>> wireless companies out there, both GSM and CDMA. In this case:
>> http://www.mycricket.com/broadband/plans/broadband40
>>
>>>> truly broadband, they would not shut down the link after 12 hours.
>>>> Nonetheless, there are no fees for connection time. Broadband
>>>> routers are always logged on 24/7.

From: unruh on
On 2010-01-12, lrhorer <lrhorer(a)satx.rr.com> wrote:
> unruh wrote:
>
>> On 2010-01-11, lrhorer <lrhorer(a)satx.rr.com> wrote:
>>> unruh wrote:
>>>
>>>> On 2010-01-10, lrhorer <lrhorer(a)satx.rr.com> wrote:
>>>>>
>>>>> Hello,
>>>>>
>>>>> I originally posted a query about this over on alt.linux,
>>>>> and
>>>>> someone
>>>>> there suggested I post a query here. I am using wvdial to
>>>>> provide "dial up" broadband access to the Cricket
>>>>> wireless network using a Cal-Comp A600 GSM wireless modem on a
>>>>> headless
>>>>> router.. It's working fairly well, but there is one serious issue
>>>>> I
>>>>> need to resolve. After 12 hours online, the ISP shuts down the
>>>>> link. When this happens, the modem sometimes locks up, at least as
>>>>> far as
>>>>> wvdial is concerned. An attempt to dial again fails.
>>>>> Occasionally, the modem truly hangs, so that even a soft reboot
>>>>> won't recover the
>>>>> session. Since wvdial produces no logs, no externally readable
>>>>> status structure, and no exit codes, it's difficult to tell what
>>>>> action the router needs to take when the ppp link is down.
>>>>>
>>>>> Does anyone know of a slightly more sophisticated or
>>>>> better supported alternative to wvdial? I am looking at linuxconf,
>>>>> but there doesn't seem to be a Debian package for it, and the RPM
>>>>> package
>>>>> won't load under alien. The source code fails with a fair number
>>>>> of
>>>>> errors. I would really rather not start digging into problems with
>>>>> the
>>>>> RPM package or in the source code if I don't have to. The fact the
>>>>> newest version of linuxconf is from 2005 also suggests it may not
>>>>> be actively maintained, either
>>>>
>>>> Try looking at www.theory.physics.ubc.ca/ppp-linux.html for
>>>> techniques for debugging your ppp connection.
>>>
>>> The problem isn't at the ppn layer. The problem lies at the hardware
>>> layer, epcifically the modem locking up.
>>
>> And you know this how? And if you are so sure why are you asking here
>> since hardware fixes do not get handled by Linux user groups.
>>
>>>
>>>> wvdial is simply a font end for the
>>>> thing that does the actual work, namely pppd. pppd can and does
>>>> produce error messages to the daemon and local2 facilities.
>>>
>>> Yes, but there are no errors at that level, at least not until the
>>> modem locks up.
>>
>> Have you switched on logging for ppd? Have you set up the daemon and
>> local2 facilities to log to a file?
>> daemon.*;local2.* /var/log/daemonlog
>> in /etc/syslog.conf
>
> Yeah, I thought so. Logging is enabled (to daemon.log, actually, not
> daemonlog), and log rotation is enabled for daemon.log. Pppd reports
> no errors in daemon.log.

How about posting the contents of that log file for a session. Also make
sure that in /etc/syslog.conf you have
daemon.*;local2.* as the facility/level that is logged-- ie everything
is logged, not just some special loglevels.


>
>> and then do
>> killall -1 syslogd
>
> 'No need, since (1) daemon.log was already enabled, and (2) The router
> has been rebooted innumerable times.
>
>> Have you looked at that file?
>
> Yes, of course, as well as syslog, kern.log, mesages.log, dmesg, and of

Of course? When this is the first time you noticed that you have a
daemon.log?


> course the several log files generated by the scripts I wrote to handle
> the networking.
>
>>>> You need
>>>> to tell your system to record those and tell pppd to actually output
>>>> the debug messages.
>>>>
>>>> One thing you could try is to kill
>>>> killall pppd
>>>> as root
>>>> before you try again.
>>>
>>> No, I'm not using pppd, and ppp is dead whenever the connection is
>>> closed.
>>
>> ??? pppd is THE daemon for using ppp, and wvdial uses pppd
>>
>> ps auxww|grep pppd
>
> 'Produces nothing. This was of course the first thing I checked. I'm
> not a noob.
From: Moe Trin on
On Mon, 11 Jan 2010, in the Usenet newsgroup comp.os.linux.networking, in
article <M7KdnSnIsJsXmdHWnZ2dnUVZ_v1i4p2d(a)giganews.com>, lrhorer wrote:

>Moe Trin wrote:

>> And what is the exact indication that dialing failed? The soft-reboot
>> is also suggesting the modem initiation string is wrong.

> Well, when the modem is well and truly hung, it disappears
>altogether. The computer no longer recognizes that it is attached to
>the USB port - lsusb shows nothing at all - and of course udev
>de-registers the /dev/ttyACM0 device, so no string of any sort is
>going to help at that point.

The fact that the device is disappearing would seem to be the problem.
In Message-ID: <M7KdnS7IsJt869bWnZ2dnUVZ_v3_fwAA(a)giganews.com>, you
say:

] I can sometimes reproduce the issue by `kill -9 <pid>` on the wvdial
] session. Often not, however, in which case establishing a new session
] works just fine.

I'd want to find out why it's disappearing from lsusb - never mind
/dev/ttyACM0. I think you know that stuff doesn't happen randomly,
and something wvdial is doing (perhaps because of those init-strings)
is randomly hosing the device file.

>Well of *COURSE* it's called from the command line. I believe I
>mentioned more than once this is a headless router.

I first saw the word headless in yesterday's alt.linux post, which I
read _after_ reading c.o.l.n.

>The commands are all called from a startup script in /etc/init.d.

In my response yesterday in alt.linux, I showed a pair of scripts to
operate the connection directly rather than using a less than helpful
helper program like 'wvdial'. Even simpler,

[galileo ~]$ cat /etc/ppp/options | column
lock crtscts nodetach defaultroute persist
/dev/modem modem 115200 noipdefault
[galileo ~]$ ls -l /etc/ppp/*ap-secrets /etc/resolv.conf
-rw------- 1 root root 99 Jun 16 2008 /etc/ppp/chap-secrets
-rw------- 1 root root 94 Jun 16 2008 /etc/ppp/pap-secrets
-rw-r--r-- 1 root root 49 Jun 16 2008 /etc/resolv.conf
[galileo ~]$ cat /usr/local/bin/dialin.example
exec /usr/sbin/pppd user ibuprofin(a)example.com connect "/usr/sbin/chat
ABORT BUSY \"\" AT\&F1 OK ATDT2662902 CONNECT \"\d\c\""
[galileo ~]$

NOTE: That's one long (127 characters) line.

NOTE: CONNECT \\\d\\\c" also works

You could put /usr/local/bin/dialin.example with the appropriate
changes of username, init-string, and telephone number, in place of
the call to wvdial. If your IP address is going to be changing, you
probably want to stick a '1' into /proc/sys/net/ipv4/ip_dynaddr

There is also the Debian 'pppconf' program which creates the needed
connection setup/teardown scripts, and the author of that ap does
read these newsgroups.

>> wvdial 2>&1 | tee /tmp/some.file.name

>I might try that. The bigger issue, however, is that logging to a log
>file - especially an output as verbose as that produced by wvdial - and
>then parsing it with a pseudo-linguistic recognition script is not the
>best way to try to programmatically determine the status of a running
>application.

There shouldn't be that much logging. With full debugging, the script
above would produce around 40 lines of ASCII in total - bringing up the
link and tearing it down.

>Secondly, this is a router, so everything it does must be done
>automatically. Nothing interactive can be employed to facilitate
>systems operations, unless of course I write an expect script or
>some such to interact with the system. Nothing like `less` would
>fill that bill.

You have no shell access to read log files?

>When it is working properly, the output from wvdial looks like
>this:

I'd hardly consider it working properly, but anyway:

>--> WvDial: Internet dialer version 1.60
>--> Cannot get information for serial port.

Trying to read the UART, and failing. That's likely due to this being
a USB port.

>--> Initializing modem.
>--> Sending: ATZ
>ATZ
>OK

So it sends a "reset to an unknown saved configuration", and receives
an OK - it can talk to the modem.

>--> Sending: ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
>ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
>OK

Sent some useless stuff not applicable to a USB device, and got another
OK response

>--> Sending: ATDT#777
>--> Waiting for carrier.
>ATDT#777
>NO CARRIER
>--> No Carrier! Trying again.

Supposedly dialed a number, but no modem answered within the timeout
period - that's normally 45 seconds.

[repeats snipped]

>--> Sending: ATDT#777
>--> Waiting for carrier.
>ATDT#777
>CONNECT
>--> Carrier detected. Waiting for prompt.

This time, a modem was detected on the other end of the line. As I
mentioned, the idiots from wvdial are still living in the past and
think there will be a LOGIN: prompt.

>~[7f]}#@!}!}!} }9}"}&} } } } }#}%B#}%}%}&X[05]@[16]}'}"}(}"}9y~

Instead, the peer sends a ppp frame, like every ISP has been doing
for the past 15 plus years.

>--> PPP negotiation detected.
>--> Starting pppd at Mon Jan 11 21:30:54 2010
>--> Pid of pppd: 14384

and wvdial belatedly starts pppd.

>--> pppd: <EF><BF><BD>'<EF><BF><BD>[08]<EF><BF><BD>%<EF><BF><BD>[08]

This should be 8 bit ppp frames - hard to say what it actually is.

[remainder of ``log'' snipped]

and it's getting RFC1918 addresses for local, remote/peer, and two
DNS servers. My ISPs don't play Musical IP Addresses with the DNS,
so I've got those written directly into /etc/resolv.conf. There is a
'usepeerdns' option to pppd to cause it to ask the peer for these
addresses, but pppd puts them into /etc/ppp/resolv.conf rather than
screwing with a system configuration file - you could soft-link the
two if needed. It's in the pppd man page.

>> If you're using Debian, the normal tool is pppconf which creates
>> 'pon' and 'poff'.

> Since the failure doesn't usually seem to be in pppd, I'm not sure
>these will be helpful. I'll take a look, though.

The failure _seems_ to be caused by the modem disappearing. Why that
happens is anyone's guess, but my guess would be due to those plain
old generic telephone modem init-strings used by wvdial. Remember
that helper tools like wvdial, kppp, GnomePPP, linuxconf, and similar
are built for people who can't or won't read documentation, and are
configured in a mode that the author hopes is right. Some of them do
a good job of emulating windoze in that they intentionally avoid
displaying anything technical - helps when things are working as
designed, but makes troubleshooting difficult when things go wrong.

>> Concur - also look carefully through any documentation for your modem,

> You're kidding me, right? The modem documentation is a single sheet
>of paper about the size of a credit card. I've found nothing online,
>either specific to the modem.

Must be hell - did this modem come with a windoze disk? There will
be a windoze file that contains the modem init strings. I don't do
windoze, so I don't know which file it is, but greping for "AT"
should turn it up. Probably one of the smaller files (the bigger
ones are often bitmap icons for the desktop). Good luck

Old guy
From: Moe Trin on
On Mon, 11 Jan 2010, in the Usenet newsgroup comp.os.linux.networking, in
article <4qCdnfB7Su93lNHWnZ2dnUVZ_jmdnZ2d(a)giganews.com>, lrhorer wrote:

>Here is an example of one failure where pppd does get called:
ATDT#777
CONNECT
--> Carrier detected. Waiting for prompt.

Modems out of the picture not - you are connected

>~[7f]}#@!}!}!} }9}"}&} } } } }#}%B#}%}%}&a_cp}'}"}(}"[03]3~
>--> PPP negotiation detected.
>--> Starting pppd at Mon Jan 11 21:25:39 2010
>--> Pid of pppd: 10153
>--> Using interface ppp0

OK

>--> Disconnecting at Mon Jan 11 21:25:46 2010
>--> The PPP daemon has died: Authentication error.
>--> We failed to authenticate ourselves to the peer.
>--> Maybe bad account or password? (exit code = 19)
>--> man pppd explains pppd error codes in more detail.
>--> I guess that's it for now, exiting
>--> The PPP daemon has died. (exit code = 19)

> Exit code 19 is just "We failed to authenticate ourselves to the peer."

EITHER: 1. wvdial couldn't figure out which username to use (the
"Username" in /etc/wvdial.conf) or
2. the username isn't matching what _this_ peer (GSM is
wonderful this way - username, username(a)domain, something else?) or
3. it tried to use the ``wrong'' authentication mode for _this_
peer (CHAP-MD5 verses PAP verses EAP). Wvdial dinks with the secrets
files for some bizarre reason. I never understood why the idiots
thought it necessary. Also, I don't believe wvdial understands EAP,
though that may or may not matter.

>The point that is being missed, however, is that I am not asking for
>help with troubleshooting the modem.

That's probably where you should be looking

Old guy