Los Angeles Free-Net Information

The incoming email server was shutdown just before midnight on 5 July. The pop mail server was left running for users to get the last emails from their mailboxes. Its been a week since the incoming mail ended so now is the time to shutdown the pop mail access. The pop server was turned off a few minutes ago. Occasionally we are able to get some bursts of email through to roadrunner. As a result, the mail queue is diminishing slowly. I have added some new spam blocking approaches during the last week. Unfortunately, they do not seem to be blocking very much. I didn't expect them to be very effective and may remove them after the queue is back to normal. The most effective spam blocking still is the old drive-by spam blocking approach. While it is quite effective (blocked over 60% of LAFN's spam), it is not compatible with many large corporation's mail structures. This approach requires that emails must be retried from the same outgoing mail server IP address. Large corporations often have multiple mail servers (each with their own IP) and the retry frequently uses a different server. When that happens, we will never receive the email. Detecting these situations is extremely difficult and its likewise difficult to bypass the blocking. I am hesitant to implement drive-by spam blocking.

It continues to get worse. Roadrunner is now only taking 2 emails per hour. If you have a roadrunner.com, rr.com, or twc.com you might want to consider using a different email address to receive the forwarded lafn mail. I don't see this situation getting better in the future.

Roadrunner is limiting us to 1 email delivered every 6 minutes. There are well over 1000 emails in the queue. At this rate, assuming nothing new arrives, it will take 5 days to empty the queue. Note, this also affects rr.com and twc.com. I have instituted a rate restriction on incoming mail to those domains holding them to 1 email every 10 minutes. Hopefully this will allow the queue to start being delivered. This way the originators will find out that the mail is being delayed. Otherwise, when the 7 day limit expires, those email will simply vanish. I may have to reduce the input even more if the queue doesn't start to diminish.

Those of you who are having mail forwarded to a roadrunner account (rr.com, roadrunner.com, twc.com, and a few others) are experiencing significant delays in receiving mail. The problem is that roadrunner is deliberately delaying the mail because of the "volume". We are trying to resolve this with them, but given the holiday weekend, it does not appear that it is going to happen for a few days. In the meantime, our mail queue is building up. There are well over 300 emails waiting for roadrunner to accept. The DNS convergence appears to be much farther along today than it was last night. All the DNS servers I have been able to check have all the new values. However, the process is not complete as the old server continues to receive mail. There are also still messages in the old outgoing mail server's queue. I am leaving it up till they either get accepted or returned to the originator. It will not accept new mail though.

The DNS propagation has started but it taking considerably longer than normal. Several DNS servers that I am able to check are about half way through the propagation. They have some of the new values, but not all of them. As a result we are continuing to receive email at the original LAFN servers. Those will be in your old LAFN mailbox. However, there seems to be an issue with a couple of users who are receiving a lot of email such the forwarding MTAs are complaining and slowing down delivery. I will address this tomorrow with the individuals to see what can be done about that. We may have to implement incoming mail limits here also.

I hate typos. They get you every time and are hard to find. I think I got that one squashed. Starting the transition now.

As usual, the more you test, the more things go wrong. The mail forwarding service is not functioning at this time, so all shutdown of the LAFN servers is on hold while we fix the problem. I expect there will be a couple day delay. There will be further announcements here when I have more information.

Its almost time for the last hurrah. This is just a final reminder that I will start the shutdown right after midnight. I will be terminating my email service probably around 11 pm tonight so will not be able to respond to emails till after the transition is complete. This is to be able to have time to test the forwarding service prior to it's going live.

L.A. Free-Net has received notification from our service provider that they will discontinue DSL service to the southern California area in about two months. Therefore, it is with much regret that LAFN will need to cease providing DSL internet access effective June 30, 2017. Inasmuch as this action will make it impossible to remain in compliance with our mission, the Board of Directors has reluctantly decided to close down the L.A. Free-Net altogether. This includes email services and individual web pages. L.A. Free-Net strongly recommends that you download all email and that those of you with web pages make back up files if you don't already have them. We recognize this will be an inconvenience to many of our users. However, in recognition and appreciation of your long-standing loyalty over the years, we will provide service from June 1 thru June 30 at no charge. All accounts will be calculated as ending May 31, 2017. Since we already have verified mailing addresses for those with DSL accounts, those users will receive pro-rated refunds automatically. However, since email only accounts ($30/year) sometimes have inaccurate mailing addresses, we need to have each email-only user confirm his/her mailing address by sending an email to the email address provided in the System Bulletin requesting a partial refund and including the correct mailing name and address. Upon receipt of the mailing information, an appropriate pro-rated refund will be mailed to you. For the sake of accounting clarity, if your account is scheduled to renew on or before May 31, 2017, please send your check as usual. If your account is scheduled to renew on or after June 1, 2017, please do not send payment. Thank you again for your support over the years.

Power to the incoming mail server went down this morning. We don't know why since there were two servers on that source and only one went down. Its back up now. We put the server on its own UPS, but I doubt that will prevent the problem.

The authentication problems have been resolved. Everything appears to be working now at 9AM. The first server that failed, recovered by itself. We don't have any clues as to what caused that problem. I believe the move is complete.

DNS is finally working, but the incoming mail server will not boot. We are investigating that.

DNS is not updating. I have no idea why. The news server disk did not survive the move so news service will not be available for awhile. Until DNS gets resolved, access to the servers will be limited.

The server move has started. It will be a couple hours before we return to normal.

The outgoing mail server is back up and running. There still are a couple of items left to complete to remove the previous blocking. That will occur during the next hour. As best as I can tell, everything is working properly. If you encounter any issues, let me know. Tickets are the best approach at this time.

The update to the outgoing mail server is starting as soon as I finish this. Please be advised that although it may appear to be working, any mail you send is likely not to be delivered. Wait until the update is complete and announced here before sending anything you actually want delivered.

The LAFN servers are not accessible at this time. We don't know the cause yet and are investigating. The latest word from AT&T is that we will be backup up in about 30 minutes.

One of the disk drives on the incoming mail server has died. This is a bit unusual since that drive is only a couple months old. We will have to see what we can do to recover from this. It may take some time though. Update: The disk that failed contains the user mail files. Fortunately we have a backup of those that was done early this morning, shortly after midnight. There is adequate space on the remaining disk drive to hold the mail files so I am restoring them from that backup. Thats likely to take about an hour. When that completes we will be backup up and running. There will be some mail you have previously downloaded today in your mailbox. Mail from after the dump was done will not be in the mailbox. To get at that we are going to try and resurrect the failed drive. I have no idea if that will succeed. The logged error messages indicate that is not likely to be successful. It may take a couple of days for us to try that exercise. Update 2: The server is now back in operation. We will be attempting to recover the mail files from the failed drive. Whatever we do recover will be added back into your mailbox.

The servers are back up on the new UPS. We will see what difference that makes. DSPAM and TMDA have now been updated. There will be a bulletin released in a few minutes with the details.

The servers are being shutdown at this time to replace the UPS. Should be back up fairly soon.

Something happened this morning. For about an hour before noon today I was unable to access most of the internet include the LAFN servers. I couldn't even get to Google or this blog. I am hearing of others encountering the same problem. I am not seeing anything being identified as the cause yet. I am not sure it affected all users.

If you are having problems accessing your mail, your mailbox is probably too large. We have always recommended deleting mail from your mailbox when its downloaded. Now that we are on a slower system, it becomes critical to keep your mailbox size down. Otherwise, the server has to parse the entire mailbox to build an index of the messages for your mail program. If the mailbox is too large, it takes longer than your mail program will wait and it then drops the connection. At that point the server generally leaves over a temporary file because it was terminated abruptly and thus can't be restarted properly. I have to manually remove the temporary file before you can access mail. If you request the temporary file be removed but still have auto download running with your mail program, the problem just returns. The only way to access the mailbox is to use the Mailbox Utility function. Do note, it can't function if the temporary file still exists either. If the mailbox is too large, even it will time out.

There are issues with both DSPAM and TMDA that have been caused by updates to both products and loss of compatibility with our mail server. As a result we are testing a new approach to handling both of them. There will be some changes in how they function. At this time the following appear to the changes. Keep in mind, we are still testing this approach. DSPAM - the addresses you use to report spam or ham (not spam) will most likely become: user_id+spam@lafn.org and user_id+notspam@lafn.org. For example: for me to report spam would be bc979+spam@lafn.org. TMDA - the confirmation process no longer works with our incoming mail server. Actually, I am not all that confident that it worked after the previous system update. We will be eliminating the confirmation request email to the originator. You will still have all the remaining whitelist, blacklist, and quarantine features. You will be the only one who can move a message from the quarantine to your mailbox. There is still a lot of testing required to be sure that mail is being handled properly before we put these into production. In addition, the FAQ will need a major update and we will send out a bulletin with all the changes when they occur.

When we switched to new hardware, the network interface got changed and that didn't get updated in the spam blocking. As a result the drive-by spammers were getting through again. That has been corrected. You should see a significant reduction in spam later today.