You can never be certain that your machine has been cleaned off 100%
unless you do the clean install, however I have been in this situation
where rebuilding was not an option. I spent almost 3 months just
figuring things out and finally I did what I had to in one day. There
are couple things that can be done, again without certainity of having
100% clean system :
1. Re-install / replace ssh binary , ssh config and restart ssh.
http://www.amitnepal.com/server-security-verification-tips-and-tricks/
use tcp wrappers in addition to key based authentication, iptables etc..
http://www.amitnepal.com/using-tcp-wrappers/
2. Re-install /replace iptables and then block anything that is not
known service.
3. Re-install /replace netstat
4. Put some kind of notification on user login, email notification is
best way because they can remove traces from log, but will miss out
email sent out as soon as they get a shell.
http://www.amitnepal.com/email-notification-on-root-login-on-linux-machines/
5. See what ports are listening and which ports have established
connections, check for any mysterious IP Addresses.
6. Check for crontab for all users, cron.daily, monthly, weekly, hourly
and all possible places for cron.
eg. I have seen, altered ssh binary and a cron job , to check for their
injected string in ssh binary, if the string is not present, it would
replace the new binary with their infected one, so that they regain
access. cron job could be in some other users cron.
7. run visudo and check for sudo access.
After few hours.. check for ssh binary and see if it has lib wrap
support, they generally remove lib wrap support in altered binaries.
http://www.amitnepal.com/server-security-verification-tips-and-tricks/
These i think are the basic things to check, however there are many
other verifications like rpm verification and so on.. which depend on
how much time you would want to invest in investigation and all that..
Thanks
*Amit K Nepal
Infrastructure Engineer (RHCE)
omNovia Technologies Inc. <
http://www.omnovia.com>
Amit K Nepal <
http://www.amitnepal.com>
<
http://www.amitnepal.com>*
On 3/11/2013 11:40 AM, Vimal Shah wrote:
> Thank you for the advice. The necessary security layer that was
> missing has been identified and is being incorporated.
>
> Deploying a server from scratch has been tedious (running each command
> manually). Capturing all of these commands into a python script seems
> obvious. The python script is slow to develop due to the fact that I'm
> trying to learn it and code it at the same time.
>
> Has anyone had any experience with Vagrant? Is it worth the time to
> investigate?
>
> Lastly, if anyone is available for some consulting on these matters
> (server security and deployment), please contact me.
>
>
> On Thu, Mar 7, 2013 at 4:25 PM, Paul Mooring <paul@opscode.com
> <mailto:paul@opscode.com>> wrote:
>
> It's likely that if he left that key in there with a valid e-mail
> address then whoever compromised the server wasn't trying to be
> discrete. I would check my auth logs to see when/if someone was
> logging in from somewhere suspect. Next if the server was
> compromised, it's done, you can never trust it again, no amount of
> clean up or post-mortem investigation can ever give reasonable
> confidence that there's no back door on it. Move the services and
> data and make a new server/clean install, then look very carefully
> at what attack vector was exploited and close it (like if it was
> brute force you should have ssh for root turned off, more
> restrictive firewall rules and ssh guard).
>
> Having a server compromised can be a huge headache, good luck.
> --
> Paul Mooring
> Systems Engineer and Customer Advocate
>
> www.opscode.com <http://www.opscode.com>
>
> From: Vimal Shah <vimals@sokikom.com <mailto:vimals@sokikom.com>>
> Reply-To: Main PLUG discussion list
> <plug-discuss@lists.phxlinux.org
> <mailto:plug-discuss@lists.phxlinux.org>>
> Date: Thursday, March 7, 2013 4:49 PM
> To: Main PLUG discussion list <plug-discuss@lists.phxlinux.org
> <mailto:plug-discuss@lists.phxlinux.org>>
> Subject: server compromised?
>
> Hello all,
>
> While randomly looking into the .ssh/authorized_keys file, I
> noticed a line that shouldn't have been there. This was concluded
> based on the last portion of the line. This portion was in the
> form of /user@domain.com <mailto:user@domain.com>/, where the
> domain was one of a likely competitor. Does this automatically
> mean that this server has been compromised? The line has been removed.
>
> Thanking everyone in advance.
>
> --
> Vimal
>
> ---------------------------------------------------
> PLUG-discuss mailing list - PLUG-discuss@lists.phxlinux.org
> <mailto:PLUG-discuss@lists.phxlinux.org>
> To subscribe, unsubscribe, or to change your mail settings:
> http://lists.phxlinux.org/mailman/listinfo/plug-discuss
>
>
>
>
> --
> Vimal (rhymes with Kimmel) Shah
> Front-End / Infrastructure Engineer
> Sokikom
> Mobile: (480) 752-9269 <tel:%28480%29%20752-9269>
> Email:vimals@sokikom.com <mailto:vimals@sokikom.com>
> Web:www.sokikom.com <http://www.sokikom.com/>
>
> Follow us: twitter.com/sokikom <http://www.twitter.com/sokikom>
> Like us: facebook.com/sokikom <http://www.facebook.com/sokikom>
>
>
> ---------------------------------------------------
> PLUG-discuss mailing list - PLUG-discuss@lists.phxlinux.org
> To subscribe, unsubscribe, or to change your mail settings:
> http://lists.phxlinux.org/mailman/listinfo/plug-discuss
---------------------------------------------------
PLUG-discuss mailing list -
PLUG-discuss@lists.phxlinux.org
To subscribe, unsubscribe, or to change your mail settings:
http://lists.phxlinux.org/mailman/listinfo/plug-discuss
(text/plain)