Constant DoS

Mar 24 2007, 06:55 PM

This announcement is regarding the recent bursts of downtime we keep having. It's ranged anywhere from 5 minutes to a 20 minutes over the past few days or so.

What is DoS?

http://en.wikipedia.org/wiki/Denial_of_service

QUOTE
In computer security, a denial-of-service attack (DoS attack) is an attempt to make a computer resource unavailable to its intended users. Typically the targets are high-profile web servers, and the attack attempts to make the hosted web pages unavailable on the Internet. It is a computer crime that violates the Internet proper use policy as indicated by the Internet Architecture Board (IAB).

DoS attacks have two general forms:

    * Force the victim computer(s) to reset or consume its resources such that it can no longer provide its intended service.

    * Obstruct the communication media between the intended users and the victim so that they can no longer communicate adequately.

Not all service outages, even those that result from malicious activity, are necessarily denial-of-service attacks. Other types of attack may include a denial of service as a component, but the denial of service may be part of a larger attack.

This is what is causing the downtime. There are some site(s) which I believe are hosted that are getting attacked and causing downtime for everything.

Please remember that DoS is not dangerous, it is not hacking and doesn't cause data loss. What it DOES do is, it doesn't allow me to get the services to you because its gets clogged/flooded with bogus requests.

I apologize about this, however I am doing all I can to track who exactly is doing it, why, and what can be done to stop it.

Thank you for your patience.

Comments

  1. ledi51 Says:

    That sucks...

  2. neRd Says:

    Have you figured out who's responsible?

  3. Pc Gamer 2007 Says:

    This really sucks

  4. iBaLLiN Says:

    its ok i dont get downtime so take your time and catch them pricks

  5. Candor0 Says:

    QUOTE (iBaLLiN @ March 24, 2007 08:30 pm) its ok i dont get downtime so take your time and catch them pricks

    You just aren't on your forum at the time I guess, when this happens, every site has some difficulties, such as slow loading or downtimes, or so I would think.

  6. Jcink Says:

    I tightened some settings. Please let me know if anyone has trouble with "forbidden" error messages. I will be emailed anyway if someone does, but just a heads up.

  7. Chazz Says:

    It's down on IE

  8. Jcink Says:

    It goes down on everything.

  9. Chazz Says:

    It was ok on firefox, but down on IE >_>

  10. Jcink Says:

    This issue should be solved.

    However, I will still be trying to configure the firewall on the second server.

  11. deletemyaccountplease Says:

    HAY ADMIN FIX TIHS ISSUE KNOW OR I WIL HAX ALL UR COMMPUTERS k thx buy

    >_>

    Seriously though, it looks like it just happened again, I hope this can be taken care of soon.

  12. Jcink Says:

    It did happen again... sigh. At least it didnt crash the SQL server this time though.

  13. Irayo Says:

    Jcink, come back to IRC!

    In pertinent news, sorry about the attacks. It's not something I normally do, but recently Hamlin has been doing some odd things (finding his way around channel bans on IRC, ignoring oper warnings, etc.), so I will go far enough to say that he might be causing some trouble here.

    It seems like it'd be relatively easy to track down the sites that are under attack, and the attacker's IP(s), just by some quick log file analysis. But I'm not there, so I really have no idea what your setup looks like.

  14. Jcink Says:

    ok as per irayos suggestions:

    custom logger added

    php exec time lowered to 20 seconds

    also i have a new command that i can try to use these changes dont work to see whats goin on

  15. TJF33 Says:

    I don't trust hamlin one bit.

  16. Jcink Says:

    He quietly browses this board I am told, under another account...

  17. iBaLLiN Says:

    i cant coment on hamlin because i kinda was bad on the shoutbox and got banned but i came back and im trying to be good

  18. Jcink Says:

    hamlin still hates me. I was going to watch how he behaved at the JFB skin zone and Candors resource site and possibly see if he could be unbanned, but I see he's still up to his old behavior by calling me a faggot and other things, + being mischievous or whatever.

  19. Jcink Says:

    I'm installing smoothwall. Will let everyone know how it goes.

  20. iBaLLiN Says:

    QUOTE (Jcink @ March 27, 2007 08:26 pm) hamlin still hates me. I was going to watch how he behaved at the JFB skin zone and Candors resource site and possibly see if he could be unbanned, but I see he's still up to his old behavior by calling me a faggot and other things, + being mischievous or whatever.

    yeah thats the difrence between be and him even thou u didnt like me i still had respect for you cous you did nothing to me other than ban me for being as asshole and then i was never aiming you talking smack.

  21. Candor0 Says:

    QUOTE (iBaLLiN @ March 28, 2007 09:25 am) QUOTE (Jcink @ March 27, 2007 08:26 pm) hamlin still hates me. I was going to watch how he behaved at the JFB skin zone and Candors resource site and possibly see if he could be unbanned, but  I see he's still up to his old behavior by calling me a faggot and other things, + being mischievous  or whatever.

    yeah thats the difrence between be and him even thou u didnt like me i still had respect for you cous you did nothing to me other than ban me for being as asshole and then i was never aiming you talking smack.

    Yet you still continue to use in appropriate language, and are increasingly becoming your old self.

    Think about it.

    Jcink, are there good reviews on this thing, also do you think it should stop all of these attacks or just the majority?

  22. Pc Gamer 2007 Says:

    it should stop them totally because it is a whole system dedicated to filtering out invalid ips and stuff

  23. iBaLLiN Says:

    QUOTE (Candor0 @ March 28, 2007 02:24 pm) QUOTE (iBaLLiN @ March 28, 2007 09:25 am) QUOTE (Jcink @ March 27, 2007 08:26 pm) hamlin still hates me. I was going to watch how he behaved at the JFB skin zone and Candors resource site and possibly see if he could be unbanned, but  I see he's still up to his old behavior by calling me a faggot and other things, + being mischievous  or whatever.

    yeah thats the difrence between be and him even thou u didnt like me i still had respect for you cous you did nothing to me other than ban me for being as asshole and then i was never aiming you talking smack.

    Yet you still continue to use in appropriate language, and are increasingly becoming your old self.

    Think about it.

    Jcink, are there good reviews on this thing, also do you think it should stop all of these attacks or just the majority?

    its sencored on here its my launguage you dont worry about it. i talk how i want when i want. i am not bypassing sensors so i am ok. i dont constantly flame people so shh.

  24. Candor0 Says:

    QUOTE (iBaLLiN @ March 28, 2007 06:21 pm) QUOTE (Candor0 @ March 28, 2007 02:24 pm) QUOTE (iBaLLiN @ March 28, 2007 09:25 am) QUOTE (Jcink @ March 27, 2007 08:26 pm) hamlin still hates me. I was going to watch how he behaved at the JFB skin zone and Candors resource site and possibly see if he could be unbanned, butĀ  I see he's still up to his old behavior by calling me a faggot and other things, + being mischievousĀ  or whatever.

    yeah thats the difrence between be and him even thou u didnt like me i still had respect for you cous you did nothing to me other than ban me for being as asshole and then i was never aiming you talking smack.

    Yet you still continue to use in appropriate language, and are increasingly becoming your old self.

    Think about it.

    Jcink, are there good reviews on this thing, also do you think it should stop all of these attacks or just the majority?

    its sencored on here its my launguage you dont worry about it. i talk how i want when i want. i am not bypassing sensors so i am ok. i dont constantly flame people so shh.

    Well don't rely on censors, you should be civilized enough at this age to know that spamming, and swearing is pointless and unacceptable on the forums, it should be more apparent now as a few people have been banned for it recently also.

    However I'm not going to continue on here as its not what this topic is intended for.

  25. Jcink Says:

    QUOTE (Pc Gamer 2007 @ March 28, 2007 04:41 pm) it should stop them totally because it is a whole system dedicated to filtering out invalid ips and stuff

    It should, but I don't know that for sure. I think the protection should be working out of the box, but if it doesn't apparently there are MODs to install to the firewall to help if the base doesn't work.

    As far as the reviews Candor0, it seems pretty good.

    Other people from posts im reading:

    http://community.smoothwall.org/forum/view...ight=dos+attack

    http://community.smoothwall.org/forum/view...4&highlight=dos

    http://community.smoothwall.org/forum/view...7125&highlight=

    All more extreme cases than mine...

    So I guess looking good...

    Right now, it is just hooked up to my laptop and I'm messing with the settings, getting familiar with it, etc. Attached are some pictures of the control center, it seems like it's got nice protection settings and I checked them all. Plus it has IP block at the firewall too so that's nice.

    I'll let people know before I go and move the system over. It should take me about 10 minutes maximum to move it, hook it up to the server and do all of the forwarding for the ports and stuff. Then I can see if it really works or if it needs further configuration or whatnot.

    Also, I'll be making some posts on webhostingtalk.com to question about additonal commands that'll help me identify a DoS better to catch who is doing it, even if it's just a proxy IP, it's a step in the right direction.

  26. Pc Gamer 2007 Says:

    Very nice

  27. Jcink Says:

    Well we're protected now... just need to wait and see. From the moment I turned it on though it detected an attack from a datacenter IP and blocked it :S though I'm unsure if that was it, I could be completely wrong.

    Let me know if anyone has any issues.

  28. Cybermatt180 Says:

    told ya it would work

  29. Jcink Says:

    I'm gonna reboot right now - just so everyone knows, nothing to do with DoS

  30. Jcink Says:

    All done.

  31. Jcink Says:

    Our problems seem solved so far but Pc said he noticed a downtime earlier. I'll keep close watch and post if i see anything myself.

  32. Jcink Says:

    It seems as though it has subsided but the troubles are still not fully over. I'm working on more solutions and will continue to keep testing them.

  33. Jcink Says:

    Lil downtime due to dns switching, not DoS

  34. Jcink Says:

    I am currently trying a perl script I wrote to check for DoS. I won't explain how it works, but I'll be tweaking and adjusting it over the next couple days. What I have set for connection count may need to be lower or higher, all depends we'll see how it plays out.

    We're not getting hit as much lately though. There was just one today, lasted maybe 5 mins.

  35. Jcink Says:

    More things are now being tested with the server process. I apologize if it causes any trouble.

  36. Jcink Says:

    Hit again, did not manage to prevent it. Apache flood was the cause.

    04/10/07 5:38 PM EST

    80 requests currently being processed, 0 idle workers

    WWWWWWWWWWWWWWWWWWKWWWWWWWWWWWWWWWWWWWKWWWWWWWWWWWWWWCWWWWWWCWWW

    WWWWWWCWKWWWWWWW

  37. Jcink Says:

    Hit again

    6:18PM Est

    04/10/07

  38. Jcink Says:

    I have done some changes to iptables. we'll see what happens.

  39. Jcink Says:

    Didn't work, hit again, downtime at 5:55, I think.

    There is a new command I will try to run to see if its a certain kind of attack next time it happens. Expect it to go down though.

  40. Jcink Says:

    The DoSer has done it again, but this time he has been caught. I was lucky enough to be monitoring the router and the server status page and here we are:

    58.61.164.140

    This is an IP based in China, and this person was simply sending HEAD requests over and over and over to my system. They're blocked hard and good now. The next step to this is for me to write a script to check for these types of things, and not allow attacks like this to succeed. The script should be in by tonight, so in case they come back with another IP, this will all be over.

  41. Jcink Says:

    You might've noticed some downtime today at 3:00, lasted about 10 mins. Thing was, we had to restart MySQL and also update a ton of other components which drained away the CPU.

    No DoS. In fact, I would like to say we're doing very good on the DoS front. I don't want to give too many details yet, but they are being caught and stopped

  42. Jcink Says:

    I think besides the fact that we get DoSed, I'm unsure -- however, whenever MySQL reaches around 10,000,000 queries (takes about 2 days to get to this) things get laggy, and I notice that. Then, stuff seems to crash. I don't know if this has anything to do with the DoS itself, but it might. (Because last night somebody DoSed it, they were blocked, and now today SQL was crappy until I restarted.)

    What I'm going to do is every night for next couple of days is restart MySQL at around 2am.

    Restarting that takes roughly 3 minutes, so you'll see a "server is restarting" message, telling users to check back in about 5 minutes. If this seem to work out, I'll see about adding something to automatically do it at 5:00AM EST, or so, till I figure out what makes it do that.

  43. Jcink Says:

    We've made some progress about this. Both good and bad news. From catching runaway processes, I realized this page:

    http://support.b1.jcink.com/index.php?act=Stats

    Takes 40+ seconds to load on a certain popular forum. I found the query in the usage list and it was jamming stuff up. Do note that this is also one of the pages that the DoSer accesses several times when doing so.

    We'll see if this makes a difference, however that page will be unavailable for now.

  44. Hamlin Says:

    keep up the good work Jcink