server compromised?

[Paul Mooring]

I work for Opscode so I'm a bit biased, but this is the sort of problem Chef was created to solve.  I define all configuration in Chef stored in git repos( but any version control will work).  This serves as both documentation and automation for consistency, and has the added bonus of making infrastructure wide changes a snap.  As an example imagine the case of an old compromised key or bad password from an ex-admin.

We manage our users through a sys-admins cookbook and our servers check in every 30 mins.  This means I can change a key or disable a user and even if I forget to push to a particular server it will catch up within 30 minutes.  Also because there's code around how this is accomplished if I wanted to set passwords (I prefer not to set passwords and disable password logins entirely) I could enforce complexity around them in the cookbook.

I used creating users/keys as an example but that's only a small piece of the base role in my case.  So every server runs the base role by default, which handles users, ntp, chef-client, iptables, ect.  But all other configuration is also driven by Chef and put into roles, so I can provision a new web server by simply applying the web role.
--
Paul Mooring
Systems Engineer and Customer Advocate

www.opscode.com

From: Vimal Shah <vimals at sokikom.com<mailto:vimals at sokikom.com>>
Reply-To: Main PLUG discussion list <plug-discuss at lists.phxlinux.org<mailto:plug-discuss at lists.phxlinux.org>>
Date: Monday, March 11, 2013 11:40 AM
To: Main PLUG discussion list <plug-discuss at lists.phxlinux.org<mailto:plug-discuss at lists.phxlinux.org>>
Subject: Re: server compromised?

Thank you for the advice. The necessary security layer that was missing has been identified and is being incorporated.

Deploying a server from scratch has been tedious (running each command manually). Capturing all of these commands into a python script seems obvious. The python script is slow to develop due to the fact that I'm trying to learn it and code it at the same time.

Has anyone had any experience with Vagrant? Is it worth the time to investigate?

Lastly, if anyone is available for some consulting on these matters (server security and deployment), please contact me.


On Thu, Mar 7, 2013 at 4:25 PM, Paul Mooring <paul at opscode.com<mailto:paul at opscode.com>> wrote:
It's likely that if he left that key in there with a valid e-mail address then whoever compromised the server wasn't trying to be discrete.  I would check my auth logs to see when/if someone was logging in from somewhere suspect.  Next if the server was compromised, it's done, you can never trust it again, no amount of clean up or post-mortem investigation can ever give reasonable confidence that there's no back door on it.  Move the services and data and make a new server/clean install, then look very carefully at what attack vector was exploited and close it (like if it was brute force you should have ssh for root turned off, more restrictive firewall rules and ssh guard).

Having a server compromised can be a huge headache, good luck.
--
Paul Mooring
Systems Engineer and Customer Advocate

www.opscode.com<http://www.opscode.com>

From: Vimal Shah <vimals at sokikom.com<mailto:vimals at sokikom.com>>
Reply-To: Main PLUG discussion list <plug-discuss at lists.phxlinux.org<mailto:plug-discuss at lists.phxlinux.org>>
Date: Thursday, March 7, 2013 4:49 PM
To: Main PLUG discussion list <plug-discuss at lists.phxlinux.org<mailto:plug-discuss at lists.phxlinux.org>>
Subject: server compromised?

Hello all,

While randomly looking into the .ssh/authorized_keys file, I noticed a line that shouldn't have been there. This was concluded based on the last portion of the line. This portion was in the form of user at domain.com<mailto:user at domain.com>, where the domain was one of a likely competitor. Does this automatically mean that this server has been compromised? The line has been removed.

Thanking everyone in advance.

--
Vimal

---------------------------------------------------
PLUG-discuss mailing list - PLUG-discuss at lists.phxlinux.org<mailto:PLUG-discuss at lists.phxlinux.org>
To subscribe, unsubscribe, or to change your mail settings:
http://lists.phxlinux.org/mailman/listinfo/plug-discuss



--
Vimal (rhymes with Kimmel) Shah
Front-End / Infrastructure Engineer
Sokikom
Mobile: (480) 752-9269<tel:%28480%29%20752-9269>
Email:   vimals at sokikom.com<mailto:vimals at sokikom.com>
Web:    www.sokikom.com<http://www.sokikom.com/>

Follow us: twitter.com/sokikom<http://www.twitter.com/sokikom>
Like us: facebook.com/sokikom<http://www.facebook.com/sokikom>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.phxlinux.org/pipermail/plug-discuss/attachments/20130311/9480f7bf/attachment.html>