Prev: mutt and utf-8 (was: character encoding)
Next: REPLY IF IT INTEREST YOU.
From: Nigel Henry on 9 Jan 2008 18:00
On Wednesday 09 January 2008 22:22, Ron Johnson wrote:
> On 01/09/08 14:59, Nigel Henry wrote:
> [snip]
>
> > I was trying to get a resolution to the problem of how to set up
> > bogofilter to deal with mailing list spam, not deliberately attract
> > spammers. A link to a known mailing list that wasn't too bothered about
> > spam on their list would have been usefull though, as the Debian guys
> > have resolved the spam flood problem.
>
> And he saw the part where you complained about a sudden lack of test
>
> cases:
> >> I'm a bit disappointed that the spam problem is fixed, as I was
> >> using it as an opportunity to try and get bogofilter, which I use
> >> with Kmail to filter
>
> When I read his reply, I thought he was trying to be helpful.
>
> --
> Ron Johnson, Jr.

Apologies then. Generally there is little or no spam that gets past the
mailing list spam filters. I had been taking the opportunity while spam was
arriving on the list to try and find out how to setup bogofilter to deal with
mailing list spam. Someone on the list said they were using bogofilter, which
is why I started this thread, with the hope that I might get some clues as to
how to deal with spam coming from mailing lists.

Now that the Debian guys have resolved the spam problem, it's become a bit of
a lost cause, as I no longer have any mailing list spam to test bogofilter
with.

Nigel.


--
To UNSUBSCRIBE, email to debian-user-REQUEST(a)lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster(a)lists.debian.org
From: Andrew Sackville-West on 9 Jan 2008 21:30
On Wed, Jan 09, 2008 at 11:31:48PM +0100, Nigel Henry wrote:
> On Wednesday 09 January 2008 22:03, John Hasler wrote:
> > Nigel Henry writes:
> > > I am genuinly trying to resolve the problem of filtering out spam, that
> > > occasionally turns up on mailing lists. As I've said bogofilter works
> > > fine with non mailing list spam, but something extra is needed to deal
> > > with mailing list spam.
> >
> > I find that Spamassassin works fine on spam that comes via mailing-lists.
> > Do you somehow have the lists whitelisted?
> >
> > --
> > John Hasler
>
> No Whitelisting. Bogofilter was setup in it's default configuration with
> Kmail, and has dealt with all the ordinary non mailing list spam with no
> problems.

I don't know much about bogofilter as I've only used it a little and
never really touched its configuration, *but*, I have this little
insight.

My spamassassin setup does a pretty good job. I get a handful of
regular spam per day (between say 2 and 5). On this list I might get
one a week or so. Now, I'm not sure what my success rate is in terms
of catching regular spam, but I think a little can be inferred.

I think we get somewhere around 75 legit mails per day onthis
list. (just guessing). The listmasters have claimed that something
well over 99% of the mail that hits the servers is legit. Taking it at
99%, that's 7425 spam mails hitting the servers a day.

For those few days we were getting (I think) a wide-open spam load. I
saw about 30-40 per day for those couple of days. I would say, then,
that spamassassin was doing a damn good job. Especially considering
that d-u spam does have a slightly different flavor, and all d-u mail
has quite a different header structure than a lot of the spam we see
(probably making it harder to catch once it's been processed by the
list). So how did your spam solution fair? How many did you see in
your inbox compared to the potential amount?

BTW, these numbers a pretty much pulled out of thin air. But I think
that if you got anything less that a couple hundred a day, you should
probably be pretty happy with the results



>
> I think that using bogofilters ignorelist.db may resolve the problem of spam
> from mailing lists, but without any spammy mailing lists to use as a test,
> it's a bit of a lost cause at the moment.
>
> Maybe I should just leave things as they are. The only mailing lists I've had
> spam problems with are the Debian lists, and on both occasions the problem
> was resolved within 2 or 3 days. Not a problem really, but I was just trying
> to see if bogofilter could deal with spam from a mailing list, where normally
> all the mail that came from the mailing list was hammy, and bogofilter was
> saying that anything from the Debian lists was ham, because usually it is.

I think this is a factor of d-u being pretty clean most of the
time. That makes it harder, with the significant header load making it
look hammy, for bogofilter to catch it.

very much just my .02

A
From: Nigel Henry on 10 Jan 2008 14:20
On Thursday 10 January 2008 03:22, Andrew Sackville-West wrote:
> On Wed, Jan 09, 2008 at 11:31:48PM +0100, Nigel Henry wrote:
> > On Wednesday 09 January 2008 22:03, John Hasler wrote:
> > > Nigel Henry writes:
> > > > I am genuinly trying to resolve the problem of filtering out spam,
> > > > that occasionally turns up on mailing lists. As I've said bogofilter
> > > > works fine with non mailing list spam, but something extra is needed
> > > > to deal with mailing list spam.
> > >
> > > I find that Spamassassin works fine on spam that comes via
> > > mailing-lists. Do you somehow have the lists whitelisted?
> > >
> > > --
> > > John Hasler
> >
> > No Whitelisting. Bogofilter was setup in it's default configuration with
> > Kmail, and has dealt with all the ordinary non mailing list spam with no
> > problems.
>
> I don't know much about bogofilter as I've only used it a little and
> never really touched its configuration, *but*, I have this little
> insight.
>
> My spamassassin setup does a pretty good job. I get a handful of
> regular spam per day (between say 2 and 5). On this list I might get
> one a week or so. Now, I'm not sure what my success rate is in terms
> of catching regular spam, but I think a little can be inferred.
>
> I think we get somewhere around 75 legit mails per day onthis
> list. (just guessing). The listmasters have claimed that something
> well over 99% of the mail that hits the servers is legit. Taking it at
> 99%, that's 7425 spam mails hitting the servers a day.
>
> For those few days we were getting (I think) a wide-open spam load. I
> saw about 30-40 per day for those couple of days. I would say, then,
> that spamassassin was doing a damn good job. Especially considering
> that d-u spam does have a slightly different flavor, and all d-u mail
> has quite a different header structure than a lot of the spam we see
> (probably making it harder to catch once it's been processed by the
> list). So how did your spam solution fair? How many did you see in
> your inbox compared to the potential amount?

Well I'm in still in the dark as to how to resolve this problem with spam from
mailing lists. The flood of spam has stopped, but one spam message got
through the Debian filters today, so I am likely to be able to continue
trying to resolve the problem, with the odd spam getting through the Debian
filters from time to time.

I have a separate mailbox for the Debian-user list, and when the spam flood
was in full flow everything turned up in this mailbox, ham, and spam, so no
spam from the Debian list was sent to the wastebin.

I'm still working on finding a fix though.

The way the ignorelist.db is supposed to work with bogofilter, is that you
populate the ignorelist.db with headers from a genuine post from, for
instance, the Debian list, then when you check your mail, bogofilter ignores
the headers from mail sent from the Debian list, and turns it's attention to
the body of the messages, and hopefully will separate the spam from the ham.

As I say, I'm still trying to resolve the problem. I'll post back when I've
fixed it. After all, it must be possible.

Nigel.


--
To UNSUBSCRIBE, email to debian-user-REQUEST(a)lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster(a)lists.debian.org
From: Sjoerd Hiemstra on 11 Jan 2008 15:40
On Thu, 10 Jan 2008 20:12:06 +0100 Nigel Henry wrote:
> I have a separate mailbox for the Debian-user list, and when the spam
> flood was in full flow everything turned up in this mailbox, ham, and
> spam, so no spam from the Debian list was sent to the wastebin.

I've got Sylpheed with bogofilter integrated.
In the 'junk mail' settings there's this option: 'Filter junk mails
before normal filtering'.
When checked, bogofilter captures all spam mails from the list, that is,
spam mails with the d-u address somewhere in the headers.

If your d-u filter (putting the list mail into your d-u folder) gets the
mail first, apparently these messages get out of reach of other filters.

I suppose KMail has a similar setting after bogofilter is added.


BTW: although bogofilter does not mark a message as spam if it is not
sure about it, it does mark messages as spam if it *is* sure about it.
I've inspected some 20,000 messages that were sent to the spam folder,
and none of them was falsely marked as spam.
It appears to be safe to send the messages that are marked as spam to
the trash folder directly.


--
To UNSUBSCRIBE, email to debian-user-REQUEST(a)lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster(a)lists.debian.org
From: Nigel Henry on 11 Jan 2008 17:00
On Friday 11 January 2008 21:28, Sjoerd Hiemstra wrote:
> On Thu, 10 Jan 2008 20:12:06 +0100 Nigel Henry wrote:
> > I have a separate mailbox for the Debian-user list, and when the spam
> > flood was in full flow everything turned up in this mailbox, ham, and
> > spam, so no spam from the Debian list was sent to the wastebin.
>
> I've got Sylpheed with bogofilter integrated.
> In the 'junk mail' settings there's this option: 'Filter junk mails
> before normal filtering'.
> When checked, bogofilter captures all spam mails from the list, that is,
> spam mails with the d-u address somewhere in the headers.
>
> If your d-u filter (putting the list mail into your d-u folder) gets the
> mail first, apparently these messages get out of reach of other filters.
>
> I suppose KMail has a similar setting after bogofilter is added.
>
>
> BTW: although bogofilter does not mark a message as spam if it is not
> sure about it, it does mark messages as spam if it *is* sure about it.
> I've inspected some 20,000 messages that were sent to the spam folder,
> and none of them was falsely marked as spam.
> It appears to be safe to send the messages that are marked as spam to
> the trash folder directly.

When I setup bogofilter on Kmail, I set filters for various mailing lists to
run before bogofilter. Then we had a spam flood on the d-u list (that was
some time back though), so I moved the d-u list so that what was put into it
was after bogofilter had processed the downloaded messages. I thought this
had fixed the problem, but the Debian folks had fixed the spam problem, so
there was no way to tell.

With the recent spam flood, what I thought was a fix, was not, and all the
mailing list spam still turned up in the d-u box.

I'm still using Kmail on FC2 for downloading my mail, and it's possible that
later versions of Kmail have better ways, and more options for dealing with
spam.

I'm still working on the problem, and according to replies on the bogofilter
list you can setup an ignorelist.db. As far as I understand it, you can put a
header from a legit d-u message in the ignorelist.db, then when you check
your mail, bogofilter will ignore headers from the d-u list, and concentrate
on the body of the messages.

I've yet to find out how to set this up, but am working on it. of course it
doesn't help that the spam from the list has virtually stopped now, so have
almost no mailing list spam to test this out with.

Anyway, no spam, no problemo.

Thanks for the reply Sjoerd.

Nigel.





--
To UNSUBSCRIBE, email to debian-user-REQUEST(a)lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster(a)lists.debian.org
First  |  Prev  |  Next  |  Last
Pages: 1 2 3 4
Prev: mutt and utf-8 (was: character encoding)
Next: REPLY IF IT INTEREST YOU.