Constant DoS
Mar 24 2007, 06:55 PMThis announcement is regarding the recent bursts of downtime we keep having. It's ranged anywhere from 5 minutes to a 20 minutes over the past few days or so.
What is DoS?
| QUOTE |
| In computer security, a denial-of-service attack (DoS attack) is an attempt to make a computer resource unavailable to its intended users. Typically the targets are high-profile web servers, and the attack attempts to make the hosted web pages unavailable on the Internet. It is a computer crime that violates the Internet proper use policy as indicated by the Internet Architecture Board (IAB). DoS attacks have two general forms: * Force the victim computer(s) to reset or consume its resources such that it can no longer provide its intended service. * Obstruct the communication media between the intended users and the victim so that they can no longer communicate adequately. Not all service outages, even those that result from malicious activity, are necessarily denial-of-service attacks. Other types of attack may include a denial of service as a component, but the denial of service may be part of a larger attack. |
This is what is causing the downtime. There are some site(s) which I believe are hosted that are getting attacked and causing downtime for everything.
Please remember that DoS is not dangerous, it is not hacking and doesn't cause data loss. What it DOES do is, it doesn't allow me to get the services to you because its gets clogged/flooded with bogus requests.
I apologize about this, however I am doing all I can to track who exactly is doing it, why, and what can be done to stop it.
Thank you for your patience.
Mar 24 2007, 07:04 PM
That sucks...
Mar 24 2007, 07:17 PM
Have you figured out who's responsible?
Mar 24 2007, 07:26 PM
This really sucks
Mar 24 2007, 07:30 PM
its ok i dont get downtime so take your time and catch them pricks
Mar 24 2007, 08:04 PM
QUOTE (iBaLLiN @ March 24, 2007 08:30 pm) its ok i dont get downtime so take your time and catch them pricks
You just aren't on your forum at the time I guess, when this happens, every site has some difficulties, such as slow loading or downtimes, or so I would think.
Mar 24 2007, 09:08 PM
I tightened some settings. Please let me know if anyone has trouble with "forbidden" error messages. I will be emailed anyway if someone does, but just a heads up.
Mar 25 2007, 04:56 PM
It's down on IE
Mar 25 2007, 04:59 PM
It goes down on everything.
Mar 25 2007, 07:18 PM
It was ok on firefox, but down on IE >_>
Mar 25 2007, 07:19 PM
This issue should be solved.
However, I will still be trying to configure the firewall on the second server.
Mar 25 2007, 08:23 PM
HAY ADMIN FIX TIHS ISSUE KNOW OR I WIL HAX ALL UR COMMPUTERS k thx buy
>_>
Seriously though, it looks like it just happened again, I hope this can be taken care of soon.
Mar 25 2007, 08:24 PM
It did happen again... sigh. At least it didnt crash the SQL server this time though.
Mar 25 2007, 08:53 PM
Jcink, come back to IRC!
In pertinent news, sorry about the attacks. It's not something I normally do, but recently Hamlin has been doing some odd things (finding his way around channel bans on IRC, ignoring oper warnings, etc.), so I will go far enough to say that he might be causing some trouble here.
It seems like it'd be relatively easy to track down the sites that are under attack, and the attacker's IP(s), just by some quick log file analysis. But I'm not there, so I really have no idea what your setup looks like.
Mar 26 2007, 12:04 AM
ok as per irayos suggestions:
custom logger added
php exec time lowered to 20 seconds
also i have a new command that i can try to use these changes dont work to see whats goin on
Mar 27 2007, 08:53 PM
I don't trust hamlin one bit.
Mar 27 2007, 08:59 PM
He quietly browses this board I am told, under another account...
Mar 27 2007, 09:13 PM
i cant coment on hamlin because i kinda was bad on the shoutbox and got banned but i came back and im trying to be good
Mar 27 2007, 09:26 PM
hamlin still hates me. I was going to watch how he behaved at the JFB skin zone and Candors resource site and possibly see if he could be unbanned, but I see he's still up to his old behavior by calling me a faggot and other things, + being mischievous or whatever.
Mar 28 2007, 01:09 AM
I'm installing smoothwall. Will let everyone know how it goes.
Mar 28 2007, 08:25 AM
QUOTE (Jcink @ March 27, 2007 08:26 pm) hamlin still hates me. I was going to watch how he behaved at the JFB skin zone and Candors resource site and possibly see if he could be unbanned, but I see he's still up to his old behavior by calling me a faggot and other things, + being mischievous or whatever.
yeah thats the difrence between be and him even thou u didnt like me i still had respect for you cous you did nothing to me other than ban me for being as asshole and then i was never aiming you talking smack.
Mar 28 2007, 03:24 PM
QUOTE (iBaLLiN @ March 28, 2007 09:25 am) QUOTE (Jcink @ March 27, 2007 08:26 pm) hamlin still hates me. I was going to watch how he behaved at the JFB skin zone and Candors resource site and possibly see if he could be unbanned, but I see he's still up to his old behavior by calling me a faggot and other things, + being mischievous or whatever.
yeah thats the difrence between be and him even thou u didnt like me i still had respect for you cous you did nothing to me other than ban me for being as asshole and then i was never aiming you talking smack.
Yet you still continue to use in appropriate language, and are increasingly becoming your old self.
Think about it.
Jcink, are there good reviews on this thing, also do you think it should stop all of these attacks or just the majority?
Mar 28 2007, 04:41 PM
it should stop them totally because it is a whole system dedicated to filtering out invalid ips and stuff
Mar 28 2007, 05:21 PM
QUOTE (Candor0 @ March 28, 2007 02:24 pm) QUOTE (iBaLLiN @ March 28, 2007 09:25 am) QUOTE (Jcink @ March 27, 2007 08:26 pm) hamlin still hates me. I was going to watch how he behaved at the JFB skin zone and Candors resource site and possibly see if he could be unbanned, but I see he's still up to his old behavior by calling me a faggot and other things, + being mischievous or whatever.
yeah thats the difrence between be and him even thou u didnt like me i still had respect for you cous you did nothing to me other than ban me for being as asshole and then i was never aiming you talking smack.
Yet you still continue to use in appropriate language, and are increasingly becoming your old self.
Think about it.
Jcink, are there good reviews on this thing, also do you think it should stop all of these attacks or just the majority?
its sencored on here its my launguage you dont worry about it. i talk how i want when i want. i am not bypassing sensors so i am ok. i dont constantly flame people so shh.
Mar 28 2007, 05:57 PM
QUOTE (iBaLLiN @ March 28, 2007 06:21 pm) QUOTE (Candor0 @ March 28, 2007 02:24 pm) QUOTE (iBaLLiN @ March 28, 2007 09:25 am) QUOTE (Jcink @ March 27, 2007 08:26 pm) hamlin still hates me. I was going to watch how he behaved at the JFB skin zone and Candors resource site and possibly see if he could be unbanned, but I see he's still up to his old behavior by calling me a faggot and other things, + being mischievous or whatever.
yeah thats the difrence between be and him even thou u didnt like me i still had respect for you cous you did nothing to me other than ban me for being as asshole and then i was never aiming you talking smack.
Yet you still continue to use in appropriate language, and are increasingly becoming your old self.
Think about it.
Jcink, are there good reviews on this thing, also do you think it should stop all of these attacks or just the majority?
its sencored on here its my launguage you dont worry about it. i talk how i want when i want. i am not bypassing sensors so i am ok. i dont constantly flame people so shh.
Well don't rely on censors, you should be civilized enough at this age to know that spamming, and swearing is pointless and unacceptable on the forums, it should be more apparent now as a few people have been banned for it recently also.
However I'm not going to continue on here as its not what this topic is intended for.
Mar 28 2007, 06:10 PM
QUOTE (Pc Gamer 2007 @ March 28, 2007 04:41 pm) it should stop them totally because it is a whole system dedicated to filtering out invalid ips and stuff
It should, but I don't know that for sure. I think the protection should be working out of the box, but if it doesn't apparently there are MODs to install to the firewall to help if the base doesn't work.
As far as the reviews Candor0, it seems pretty good.
Other people from posts im reading:
http://community.smoothwall.org/forum/view...ight=dos+attack
http://community.smoothwall.org/forum/view...4&highlight=dos
http://community.smoothwall.org/forum/view...7125&highlight=
All more extreme cases than mine...
So I guess looking good...
Right now, it is just hooked up to my laptop and I'm messing with the settings, getting familiar with it, etc. Attached are some pictures of the control center, it seems like it's got nice protection settings and I checked them all. Plus it has IP block at the firewall too so that's nice.
I'll let people know before I go and move the system over. It should take me about 10 minutes maximum to move it, hook it up to the server and do all of the forwarding for the ports and stuff. Then I can see if it really works or if it needs further configuration or whatnot.
Also, I'll be making some posts on webhostingtalk.com to question about additonal commands that'll help me identify a DoS better to catch who is doing it, even if it's just a proxy IP, it's a step in the right direction.
Mar 28 2007, 09:07 PM
Very nice
Mar 30 2007, 02:03 PM
Well we're protected now... just need to wait and see. From the moment I turned it on though it detected an attack from a datacenter IP and blocked it :S though I'm unsure if that was it, I could be completely wrong.
Let me know if anyone has any issues.
Mar 31 2007, 09:44 AM
told ya it would work
Mar 31 2007, 03:21 PM
I'm gonna reboot right now - just so everyone knows, nothing to do with DoS
Mar 31 2007, 03:46 PM
All done.
Apr 1 2007, 02:00 AM
Our problems seem solved so far but Pc said he noticed a downtime earlier. I'll keep close watch and post if i see anything myself.
Apr 3 2007, 01:56 AM
It seems as though it has subsided but the troubles are still not fully over. I'm working on more solutions and will continue to keep testing them.
Apr 5 2007, 11:54 AM
Lil downtime due to dns switching, not DoS
Apr 6 2007, 03:47 AM
I am currently trying a perl script I wrote to check for DoS. I won't explain how it works, but I'll be tweaking and adjusting it over the next couple days. What I have set for connection count may need to be lower or higher, all depends we'll see how it plays out.
We're not getting hit as much lately though. There was just one today, lasted maybe 5 mins.
Apr 8 2007, 02:01 AM
More things are now being tested with the server process. I apologize if it causes any trouble.
Apr 10 2007, 05:39 PM
Hit again, did not manage to prevent it. Apache flood was the cause.
04/10/07 5:38 PM EST
80 requests currently being processed, 0 idle workers
WWWWWWWWWWWWWWWWWWKWWWWWWWWWWWWWWWWWWWKWWWWWWWWWWWWWWCWWWWWWCWWW
WWWWWWCWKWWWWWWW
Apr 10 2007, 06:19 PM
Hit again
6:18PM Est
04/10/07
Apr 11 2007, 02:19 AM
I have done some changes to iptables. we'll see what happens.
Apr 11 2007, 06:31 PM
Didn't work, hit again, downtime at 5:55, I think.
There is a new command I will try to run to see if its a certain kind of attack next time it happens. Expect it to go down though.
Apr 13 2007, 07:10 PM
The DoSer has done it again, but this time he has been caught. I was lucky enough to be monitoring the router and the server status page and here we are:
58.61.164.140
This is an IP based in China, and this person was simply sending HEAD requests over and over and over to my system. They're blocked hard and good now. The next step to this is for me to write a script to check for these types of things, and not allow attacks like this to succeed. The script should be in by tonight, so in case they come back with another IP, this will all be over.
Apr 15 2007, 03:21 PM
You might've noticed some downtime today at 3:00, lasted about 10 mins. Thing was, we had to restart MySQL and also update a ton of other components which drained away the CPU.
No DoS. In fact, I would like to say we're doing very good on the DoS front. I don't want to give too many details yet, but they are being caught and stopped
Apr 30 2007, 05:36 PM
I think besides the fact that we get DoSed, I'm unsure -- however, whenever MySQL reaches around 10,000,000 queries (takes about 2 days to get to this) things get laggy, and I notice that. Then, stuff seems to crash. I don't know if this has anything to do with the DoS itself, but it might. (Because last night somebody DoSed it, they were blocked, and now today SQL was crappy until I restarted.)
What I'm going to do is every night for next couple of days is restart MySQL at around 2am.
Restarting that takes roughly 3 minutes, so you'll see a "server is restarting" message, telling users to check back in about 5 minutes. If this seem to work out, I'll see about adding something to automatically do it at 5:00AM EST, or so, till I figure out what makes it do that.
May 1 2007, 09:55 PM
We've made some progress about this. Both good and bad news. From catching runaway processes, I realized this page:
http://support.b1.jcink.com/index.php?act=Stats
Takes 40+ seconds to load on a certain popular forum. I found the query in the usage list and it was jamming stuff up. Do note that this is also one of the pages that the DoSer accesses several times when doing so.
We'll see if this makes a difference, however that page will be unavailable for now.
May 1 2007, 09:59 PM
keep up the good work Jcink