We discovered that two of the SPAM Databases we have been using (although set to a low trust level) have become rather un-trustworthy. In isolation this would not present an issue but if taken in tandem could “could” result in a particular piece of email being falsely flagged as SPAM which if the score was high enough would then result in that email being blocked from delivery.

Those Databases have been removed from from our battery of tests.

Also we have stopped the server from adding the prefixes SPAM-Low and SPAM-Med from the subject line of suspect emails. The filters are doing a good enough job in identifying and stopping real SPAM that generally those flagged with the pre-fix might look “SPAMMY” but weren’t.

Checking the current stats:

Delivered
	Last 5 min.	Last hour	Last 24 hours
Local	32	239	4404
Remote	4	75	2214
Total	36	314	6618

---

Incoming Spam
	Last 5 min.	Last hour	Last 24 hours
Spam-Low	2	15	287
Spam-Med	1	10	138
Spam-High	9	60	1318
Blocked	23	313	9310

---

Greylisting
	Last 5 min.	Last hour	Last 24 hours
Blocked	18	265	5557
Allowed	10	196	3231

So total stopped SPAM in the past 24 hours has been 16,185 pieces of mail with 4,404 pieces delivered to your in-boxes.

Sent 4/11/2007

———————

We had a confluence of events that resulted in a small outage this afternoon.

We noticed that speed and performance on the mail server was lagging. Upon investigation we discovered that the performance of our connection to Qwest was not running at it’s full capacity. In fact it was running at a very slow DSL “type” of speed.

Performance was being further impacted on our degraded lines by one of our clients sending multiple emails with nearly 20MB attachments. On a normal day this would not have been an issue but due to the degraded state of the line the inbound and outbound transmission of these emails were severely impacting the available bandwidth for the rest of the mail. Essentially being a cork in the bottle.

Qwest was notified and while they were working and investigating the line (causing it to be taken down during their testing) we took the opportunity to install a new upgrade to our mail server software.

The upgrade contains quite a few feature updates that we had personally requested and are very happy to see
included. Additionally it sounds like a bunch of performance enhancements have been implemented that will make the server work even faster. I know that it appears that the web interface is faster although they do not mention this as being a feature

http://www.smartertools.com/Company/News.aspx?NewsItemID=204

Sent 1/25/2007

———————-

For some reason (we are not quite sure yet why) the main mail server went down at 12:56pm. It was back on-line at 10:50pm

Fortunately “most” of the disaster recovery steps worked as planned and we were able to bring the server back on-line without any loss of mail delivered prior to the actual outage. Since “all” the disaster recovery steps did not work as planned it took us a bit longer then anticipated to bring the server back on-line. Therefore, some mail intended for delivery to you may have been returned to the sender as undeliverable if the retry settings on the sending server was surpassed.

This weekend we will be analyzing the outage to determine if it was a hardware failure or if it was the result of something malicious. We are tending right now to believe it was a hardware failure as that is what the indications are trending. As much as we might like to think it was some SPAMMER mad at us for blocking their mail.

We are also making plans and taking steps for additional hardware and software to be able to bring the servers back on-line within a much shorter time frame if a failure were to occur again in the future. By the way, this was the first full server crash we have ever had on one of the mail servers going all the way back to 1996! A pretty good longevity record that we hope to again reestablish.

Sent 1/14/2007

—————————-

Well the upgrade on the mail server was completed last night with minimal difficulties and the server brought back on-line at around 7:30pm.

The decrease in delivered SPAM is almost spooky!

Granted it is a Holiday weekend so traffic will be lighter. However, looking at the log files as of about 2:30pm this afternoon from midnight last night the server is blocking 95.87% of all inbound mail as SPAM.

I know that the amount of SPAM that has made it through to my own in-boxes totaled just three from midnight to now. All three of those did have Opt-out links that I used.

Last Sunday volume on the server was 6,237 blocked as SPAM, 2,290 deleted as SPAM (total stopped = 8,527), 1,329 deliver as “clean” and 669 delivered but flagged as possibly SPAM (total delivered = 1,998) which represents 81.02%. As of right now the volume is 5,807 blocked, 455 deleted (total stopped = 6,262), 156 delivered “clean” and 114 delivered flagged (total = 270).

It will certainly be interesting to see how we fair when the workweek restarts on Tuesday. If you have ANY issues or CONCERNS please let us know at either: dave@microworks.net or by calling at 480-610-8234.

Sent 1/13/2007

——————–

IMPORTANT INFO – PLEASE READ THE ENTIRE MESSAGE

We are planning on upgrading the Mail servers this weekend (1/13-1/15). Hopefully you will see a dramatic decrease in the amount of SPAM that gets through to your in-boxes.

Currently 80% of the mail that is directed at our servers is classified as SPAM and deleted but a considerable amount still gets through (I had 52 SPAM emails make it through yesterday to the various accounts that I use and monitor).

The biggest single help to our Anti-SPAM efforts will be new technology included with our software called “Greylisting”. Greylisting can mean different things so here is what it means on our server.

White-listing is when you have a list of “trusted” addresses that you allow through without testing. Black-listing is when you have a list of addresses that are known to be bad and are blocked BEFORE they are accepted for SPAM testing.

The Grey-Listing technology that we will be using basically tells the remote mail server sending email that we are having a “technical” issue and to retry later. A legitimate mail server puts the mail it is trying to deliver back into its queue and will attempt to redeliver at it’s preset time schedule. We have no control of that time frame. Some servers may try right away while others may wait a bit longer before retrying to send.

(more…)

Sent 12/7/2006

———————-

With recent reports in various media, including the New York Times, showing that 90 percent of all email is now SPAM I was wondering how our Anti-Spam efforts were holding up.  It has been a while since I sat down and checked our stats.

During a review that covered the last part of June and the first half of July 68.33% of the inbound email was prevented from being delivered for failing our anti-SPAM tests.  With a total of 218,728 pieces blocked while allowing 101,376  pieces of mail to be delivered (17,069 of that total were flagged as possible SPAM).

A review the following month saw the percentage of blocked inbound mail reach 78.97%.  228,136 pieces were blocked with 60,747 delivered (15,063 flagged as possible SPAM).

This morning I undertook to look back over the numbers for the past month.

The blocked total is now up to 86.74%!  That represents 347,817 pieces of mail that have either been blocked from delivery (259,232) or deleted automatically (88,585) by the server after the mail failed our weighted testing scheme.  53,165 pieces were delivered with 27,771 of those pieces identified with the “possible” SPAM flags of “SPAM-LOW” or “SPAM-MED”.

(more…)

Sent 7/31/2006

———————

Hopefully you will have noticed a significant decline in the amount of SPAM making it through to your mailboxes over the last month or so.

The first 30 days (2/28/06 – 3/28/06) of the new mail server had us stopping 159,814 messages from being delivered to your mailboxes out of a total of 266,549 that were sent which represented 59.96 percent. On weekends the rate was hovering around 80-85% stopped.

By the time we got to 6/20/06 – 7/18/06 period we were stopping 218,728 out of 320,104 or 68.33% with most of the improvement occurring during the weekday since the weekend rate was still in the +80% range.

The overlapping time frame that we are looking at right now of 7/4/06 – 8/1/06 shows that the rate is still climbing. If trends for the next two days continue to hold true we will see the mail servers having stopped 220,771 messages identified as SPAM while allowing a total 40,175 to be delivered for a percentage of 84.60% out of 260,946 pieces of total mail! Of the total the server allowed to be delivered 14,920 pieces of mail that it thought “might” be SPAM and tagged with the designation of either SPAM-LOW or SPAM-MED.

(more…)

Sent 3/10/2006

—————————-

Greetings,

We would like to give you an update on the progress of fine tuning the new mailserver.

To date (12:01am 3/10/06) the server has received 78,809 messages (avg of 7,880 per day) and sent 11,022 messages since going on-line on 2/28/06 – 10 days ago. The total size of these messages have been 1.9 Gigabytes received and 2.3 Gigabytes sent.

We are certainly making progress on reducing the amount of SPAM that is making through the server and to the mailboxes.

Date Blocked Deleted
2/28/06 0 891
3/1/06 0 779
3/2/06 0 1,159
3/3/06 0 1,509
3/4/06 958 1,607
3/5/06 1,640 1,592
3/6/06 2,161 1,719
3/7/06 2,487 1,587
3/8/06 2,659 1,445
3/9/06 2,755 1,387

Sub Total 12,660 13,675

(more…)

Sent 2/28/2006

———————–

Greetings,

Now that I had a short nap here is some info on the new changes and how they affect SPAM.

You will notice that some of the emails will have a tag added to the subject line of SPAM-LOW or SPAM-MED. The new mail server has three threshold settings: Low, Medium & High. Those determinations are made by “weighting” each piece of mail.

For example we currently are assigning the following weights (links at the bottom that describe these type of tests:

Tests Weight

Bayesian Filtering 15
SPF
SPF-Pass -15
SPF-Fail 10
SPF-SoftFail 5
SPF-Neutral 0
SPF-PermError 0
SPF-None 0
Reverse DNS 10
Blacklists (RBL)
AHBL 10
DSBL 10
NJABL 15
ORDB 10
SpamCop 10
SpamHaus SBL-XBL 10

(more…)

Sent 2/28/2006

————————-

Greetings,

The new email and web servers were installed last night (note the time I am sending this email). The migration went fairly smooth with no real issues noted so far.

Not all the updates that are planned have yet been implemented. Notably new and better Virus and SPAM filtering. For the short term you “may” actually see a slight increase in SPAM as the current base filters begin to “learn”. The new filtering package to come in the near term will dramatically cut down on the amount of SPAM delivered to your inbox. We did not want to do a wholesale change in one night (to the extent that we were required simply by the move to new Server hardware) so as to minimize potential problems.

Given the current time I “may” not be too quick to answer the phone in the early part of the morning should you discover some unforeseen issues so please bear with us as we continue with this migration.

Another use asked the question yesterday if the method to login to the webmail was going to change. The answer is no. You will still point your browser at your domain with an indicated port of 8383 (for example: http://www.microworks.net:8383). The first time you login through the web it will ask you some questions:

(more…)