new spamassassin script - Page 4

**jdcopelin** · August 15th, 2007, 12:32 PM

Quote:

Originally Posted by Bigjohn

Jonathan - convoluted mess was not at all ment to be a disparagement, I hope you'll understand. My point was to simply state that something very elegant and simple as the original script cannot work due to the implementation of MAILDIR, enforcing further my conclusion that the actual "benefits" of maildir are probably limited to people who want GMAIL sized mail boxes and folders on their servers, where the limitations (speed of search, spam-assassin training, etc.) are foisted off on all.

Enjoy your holiday!

Thanks for replying Bigjohn. I can see how frustrated you are about this maildir thing (I was to when pass32 got converted without warning), but we'll get you sorted

And yes, I will enjoy my holiday!

Cheers
Jonathan

pcannon · August 23rd, 2007, 1:57 AM

Here is updated learnspam script that uses xargs to work around kernel limitations that cause the "Argument list too long". Xargs will take as many of the arguments and pass them to the indicated commands until all of the arguments are exhausted.

Attachment: learnspam.txt

Code:

Running jicoweb.com spam assassin training script

JicoScript learnspam script  0.6.20070609
SpamAssassin version 3.2.3

Currently running as user: *****

Checking spam and ham repositories exist...
/home/*****/spamdb exists!
/home/*****/hamdb exists!

Learning from SPAM

        /home/*****/mail/*****/*****/.spam
                learning from spam ...
                Learned tokens from 55 message(s) (142 message(s) examined)
                Learned tokens from 65 message(s) (142 message(s) examined)
                Learned tokens from 38 message(s) (142 message(s) examined)
                Learned tokens from 49 message(s) (142 message(s) examined)
                Learned tokens from 54 message(s) (142 message(s) examined)
                Learned tokens from 55 message(s) (142 message(s) examined)
                Learned tokens from 53 message(s) (142 message(s) examined)
                Learned tokens from 42 message(s) (142 message(s) examined)
                Learned tokens from 50 message(s) (142 message(s) examined)
                Learned tokens from 45 message(s) (142 message(s) examined)
                Learned tokens from 54 message(s) (142 message(s) examined)
                Learned tokens from 57 message(s) (142 message(s) examined)
                Learned tokens from 54 message(s) (142 message(s) examined)
                Learned tokens from 40 message(s) (142 message(s) examined)
                Learned tokens from 39 message(s) (83 message(s) examined)
                not enough old spam!!

Learning from uncaught SPAM

        /home/*****/mail/*****/*****/.learnspam
                no spam messages to learn from!!

Learning HAM

        /home/*****/mail/*****/*****/.ham
                no ham messages to learn from!!
 
Synchronising spam database
bayes: synced databases from journal in 8 seconds: 129168 unique entries (129168
total entries)

Done

**jdcopelin** · August 25th, 2007, 11:52 AM

Quote:

Originally Posted by pcannon

Here is updated learnspam script that uses xargs to work around kernel limitations that cause the "Argument list too long". Xargs will take as many of the arguments and pass them to the indicated commands until all of the arguments are exhausted.

Fantastic! Thanks for that! I'm fairly new to bash scripting and hadn't come across Xargs before.

Cheers,
Jonathan

**pseudoswede** · August 28th, 2007, 4:00 PM

Excellent work, pcannon!

I'm still looking for a solution to delete learned/archived spams that are older than 30 days. Right now, they get learned, then just sit in the SPAMDB folder hogging up space.

**jdcopelin** · August 28th, 2007, 5:17 PM

Quote:

Originally Posted by pseudoswede

Excellent work, pcannon!

I'm still looking for a solution to delete learned/archived spams that are older than 30 days. Right now, they get learned, then just sit in the SPAMDB folder hogging up space.

Hello,
The code below should do what you want, insert it in the space above the "Synchronising spam database" section towards the very bottom of the script. The code will clean out the spam and ham archives.

Code:

echo ""
echo "Cleaning out SPAM archive"

SPAM_FILES=`find $SPAMDIR -name \* -mtime +30 -type f`
QUANTITY=`echo $SPAM_FILES | wc -w`
if [ $QUANTITY -lt 1 ]
    then
        echo -e "\t\tnot enough old archived spam to clean out!!"
    else
        echo -n -e "\t\t"
        echo $SPAM_FILES | xargs rm -f 
fi

echo ""
echo "Cleaning out HAM archive"

HAM_FILES=`find $HAMDIR -name \* -mtime +30 -type f`
QUANTITY=`echo $HAM_FILES | wc -w`
if [ $QUANTITY -lt 1 ]
    then
        echo -e "\t\tnot enough old archived ham to clean out!!"
    else
        echo -n -e "\t\t"
        echo $HAM_FILES | xargs rm -f 
fi

EDIT: actually, on second thought I would change the "xargs rm -f " lines to "xargs rm -f -v" to get a list of what it deletes just in case...

ironleg · August 28th, 2007, 7:16 PM

What am I doing wrong to get this error below:

Thanks,

/home/****/mail/****.com/******/.spam
learning from spam ...
ERROR: the Bayes learn function returned an error, please re-run with -D for more information Learned tokens from 0 message(s) (1 message(s) examined)
xargs: sa-learn: exited with status 255; aborting
not enough old spam!!

**jdcopelin** · August 29th, 2007, 7:51 AM

Quote:

Originally Posted by ironleg

What am I doing wrong to get this error below:

Thanks,

/home/****/mail/****.com/******/.spam
learning from spam ...
ERROR: the Bayes learn function returned an error, please re-run with -D for more information Learned tokens from 0 message(s) (1 message(s) examined)
xargs: sa-learn: exited with status 255; aborting
not enough old spam!!

Hi,
Assuming that you are on a shared server...
Create a new script file adding execute permissions and paste the code below into it:

Code:

#!/bin/sh
sa-learn -D

Let it run once via cron or if you have SSH access just run "sa-learn -D" and post the output here so we can see where spamassassin is falling over.

If you are on a dedicated server make sure you aren't running the learning script as root, or that you aren't running as a normal user and trying to train other users bayes databases. They all cause permission issues.

Regards
Jonathan

ironleg · August 29th, 2007, 4:40 PM

Here is what I got:

Please select either --spam, --ham, --folders, --forget, --sync, --import, --dump, --clear, --backup or --restore
Usage:
sa-learn [options] [file]...

sa-learn [options] --dump [ all | data | magic ]

Options:

--ham Learn messages as ham (non-spam)
--spam Learn messages as spam
--forget Forget a message
--use-ignores Use bayes_ignore_from and bayes_ignore_to
--sync Syncronize the database and the journal if needed
--force-expire Force a database sync and expiry run
--dbpath <path> Allows commandline override (in bayes_path form)
for where to read the Bayes DB from
--dump [all|data|magic] Display the contents of the Bayes database
Takes optional argument for what to display
--regexp <re> For dump only, specifies which tokens to
dump based on a regular expression.
-f file, --folders=file Read list of files/directories from file
--dir Ignored; historical compatibility
--file Ignored; historical compatibility
--mbox Input sources are in mbox format
--mbx Input sources are in mbx format
--showdots Show progress using dots
--progress Show progress using progress bar
--no-sync Skip synchronizing the database and journal
after learning
-L, --local Operate locally, no network accesses
--import Migrate data from older version/non DB_File
based databases
--clear Wipe out existing database
--backup Backup, to STDOUT, existing database
--restore <filename> Restore a database from filename
-u username, --username=username
Override username taken from the runtime
environment, used with SQL
-C path, --configpath=path, --config-file=path
Path to standard configuration dir
-p prefs, --prefspath=file, --prefs-file=file
Set user preferences file
--siteconfigpath=path Path for site configs
(default: /etc/mail/spamassassin)
--cf='config line' Additional line of configuration
-D, --debug [area=n,...] Print debugging messages
-V, --version Print version
-h, --help Print usage message

SpamAssassin version 3.2.3

**pseudoswede** · August 29th, 2007, 8:58 PM

Quote:

Originally Posted by jdcopelin

Hello,
The code below should do what you want, insert it in the space above the "Synchronising spam database" section towards the very bottom of the script. The code will clean out the spam and ham archives.

Code:

echo ""
echo "Cleaning out SPAM archive"

...snip...

EDIT: actually, on second thought I would change the "xargs rm -f " lines to "xargs rm -f -v" to get a list of what it deletes just in case...

That worked like a champ. It removed over 80MB of stale spam. Thank you!

August 28th, 2007, 4:00 PM	#31 (permalink)
pseudoswede Surpass Fan Comfy Contributor Joined in Jun 2003 Lives in Denver Hosted on D9 142 posts Gave thanks: 4 Thanked 3 times	Excellent work, pcannon! I'm still looking for a solution to delete learned/archived spams that are older than 30 days. Right now, they get learned, then just sit in the SPAMDB folder hogging up space. __________________ "In the end, everything will be fine - if it is not fine, it is not the end." PseudoSwede larvez.com Dime9

August 28th, 2007, 7:16 PM	#33 (permalink)
ironleg Registered User Fresh Surpasser Joined in Feb 2006 Lives in Oklahoma Hosted on pass80 18 posts Gave thanks: 3 Thanked 0 times	What am I doing wrong to get this error below: Thanks, /home/**/mail/.com/****/.spam learning from spam ... ERROR: the Bayes learn function returned an error, please re-run with -D for more information Learned tokens from 0 message(s) (1 message(s) examined) xargs: sa-learn: exited with status 255; aborting not enough old spam!! __________________ vcsok.com Server reseller pass80

August 29th, 2007, 4:40 PM	#35 (permalink)
ironleg Registered User Fresh Surpasser Joined in Feb 2006 Lives in Oklahoma Hosted on pass80 18 posts Gave thanks: 3 Thanked 0 times	Here is what I got: Please select either --spam, --ham, --folders, --forget, --sync, --import, --dump, --clear, --backup or --restore Usage: sa-learn [options] [file]... sa-learn [options] --dump [ all \| data \| magic ] Options: --ham Learn messages as ham (non-spam) --spam Learn messages as spam --forget Forget a message --use-ignores Use bayes_ignore_from and bayes_ignore_to --sync Syncronize the database and the journal if needed --force-expire Force a database sync and expiry run --dbpath <path> Allows commandline override (in bayes_path form) for where to read the Bayes DB from --dump [all\|data\|magic] Display the contents of the Bayes database Takes optional argument for what to display --regexp <re> For dump only, specifies which tokens to dump based on a regular expression. -f file, --folders=file Read list of files/directories from file --dir Ignored; historical compatibility --file Ignored; historical compatibility --mbox Input sources are in mbox format --mbx Input sources are in mbx format --showdots Show progress using dots --progress Show progress using progress bar --no-sync Skip synchronizing the database and journal after learning -L, --local Operate locally, no network accesses --import Migrate data from older version/non DB_File based databases --clear Wipe out existing database --backup Backup, to STDOUT, existing database --restore <filename> Restore a database from filename -u username, --username=username Override username taken from the runtime environment, used with SQL -C path, --configpath=path, --config-file=path Path to standard configuration dir -p prefs, --prefspath=file, --prefs-file=file Set user preferences file --siteconfigpath=path Path for site configs (default: /etc/mail/spamassassin) --cf='config line' Additional line of configuration -D, --debug [area=n,...] Print debugging messages -V, --version Print version -h, --help Print usage message SpamAssassin version 3.2.3 __________________ vcsok.com Server reseller pass80

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Rate This Thread
Rate This Thread:

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)