READ - Pending system outage due to Forum system issues

[elc32955]

Hi folks,

Well, this is not a message I look forward to conveying. For reason(s) yet to be determined, the forum system is placing an unacceptable level of loading on our hosting server on an ongoing basis, starting appx. from 5 days back. It may be necessary to completely down the forums for an undetermined period of time while the problem is diagnosed and resolved. I won't kid you, this might take a day or two. However, this is a shared business server we occupy - and the delay issues that ALL of the commercial customers in addition to ourselves are experiencing apparently originate from our forums based on tracking from our webhost. We have not run correctly since January when the first outage period occurred. Looks like we might be in a position to have to do a radical lobotomy and start from scratch in order to get right with the world again. Now this does NOT mean losing messages, pictures, or video content. We should be able to preserve all of our on-site content if this drastic step is required. But, this takes time & money as I do have to pay the developer if he does work on our behalf.

All of our add-on modules are currently disabled (Announcements, donations, Knowledge Base, Mchat, links in separate window, etc...), so in effect we are running on the core BBS package only. I hope we can find what is causing this problem in the interest of a quick resolution without having to employ the guillotine process. Rest assured we'll do what we can in the shortest time possible to get a reliable system back up & operational. I know this is frustrating for all of you, and definitely for myself.

I'll post more as soon as I know what's happening.....

Thanks
Eric

[elc32955]

Just a brief update, I've been working with our developer and the phpBB forum support group relative to the issues we've been seeing. There are some promising leads we're following up on but nothing solid yet. The extension modules were not our problem (left them off for a day or two and we still had issues) so I've re-enabled all of them with the exception of the Media Embed module which is still suspect bad and the announcements module which is definitely blown.

We're still working diligently on solutions. I'll keep everyone posted as things unfold.

Thanks
Eric

[elc32955]

Well, there may be light at the end of the tunnel. Continued research shows we are under attack by "Claudebot". This is a bot developed to siphon information from any available data source on the Internet. Claudebots are currently running the globe and attaching to any open data sources it can find (yes, even forum systems like us) and taking data. Claudebot is utilizing the Amazon AWS network, throwing tens of thousands of Internet servers behind this bot in order to provide it's connectivity. That's like trying to use a Caprice against a 45,000 ton battleship with 16-inch guns trained on you.

The quickest way to address the Claudebot epidemic from my perspective is to eliminate all guest access - at least on a temporary basis. Claudebot presents itself as a guest user, therefore I've just chopped off the head of the snake. All users MUST have a valid forum account to access our content contained within the forum section.

Will we stay like this forever? Until the bot storm abates, yes. All it takes is for a valid user to request an account, go through our standard validation process, and BING - they're in! Believe me I did not want to take this step, but we could not continue to operate with all the bots hammering us for information.

We will see how we run tomorrow. I daresay we will probably have a run on new accounts, we will validate these as soon as possible in order received.

[s/c'd cav]

is this why the system has been slower than turtle in a frozen pond ? its taken me about a minute to get the page to load

[elc32955]

That's correct. You can thank Amazon for being an enabler for those guys, they're training up their dastardly devices of doom ( ) at the expense of our systems & servers. I'm pretty sure I've managed to get that particular bot stopped, so we will see how things run for a while.

[s/c'd cav]

That's correct. You can thank Amazon for being an enabler for those guys, they're training up their dastardly devices of doom ( ) at the expense of our systems & servers. I'm pretty sure I've managed to get that particular bot stopped, so we will see how things run for a while.

always hated amazon !!! hate facebook as well !!

it does seem to be moving a bit faster today !! so THANK YOU !!!