Fedora.org FAS Web/SSH Key Security Breech 1/23/2011

[Lisa Kachold]

J. Smith
Fedora Project
cc: Phoenix Linux User Group Security Team
cc: Fedora Announce List

I was present at FUDCon today when announcement of your results of the
"forensics" from the recent F.A.S. web/SSH key system encroachment "hack"
was made.

Announce List REFERENCE:
http://lists.fedoraproject.org/pipermail/announce/2011-January/002911.html

Fedora Response:
http://numberedhumanindustries.wordpress.com/2011/01/25/security-incident-on-fedora-infrastructure-on-23-jan-2011-dont-panic/

Not only was the FUDCon verbal discussion *incomplete* in context to
standard response during a security incident, but the statements appearing
in Announce listserv and websites failed to address, within industry
standards, the full impact, ramifications and forensics, nor appropriate
OPSEC-ish security forensics and solutions.

Specifically:

0) Was the system booted into a forensics tool to look at low level hashing
or unallocated space file changes (deleted files, etc.)?  If it was not
rebooted, did you obtain a RAM image?
1) Was a CRC check made against the systems binaries (we know the Fedora
source tree was checked and found to be pristine, don't we)?  How *did you
determine* that no escalated privileges allowed obfuscation of actions?
2) You vaguely state that insufficient information exists, within the
window, to determine who did what since "so many people changed their keys
at that time".  Were new users added?   If, in fact *this is true*, your
systems fail to meet basic logging recommendations for any server set out by
CERT, LOPSA, SCALE, LPI.
3) Redesign/configure FAS to require both https access and FULLY RANDOM user
passwords.
4) Remove the SSH key change feature from Web usermin-ish access, instead
requiring additional information from either a database of via email
exchange.  A web based SUID clearly must be protected, since it can be
deployed without logging from a shell account?  *Since SSLStrip will provide
decrypted network information, running on the same network as any attacker,
as users log into FAS, the design fails to meet standard tiered security
requirements, providing ease of access to the entire code base! *  Either
VPN (protect via OSI Layer security) or add additional code based access
authentication for escalated privileges like changing keys, and/or adding to
the repo (over and above a SSH key). Many security professionals, would, in
fact, recommend that SSH be turned off completely outside of a VPN.
5) Clearly if the user had access to the system, they potentially have all
the passwords and keys - the system MUST BE REBUILT and REDESIGNED, does it
not?

Perhaps Fedora has designated the entire system as a Honeypot (VLAN
excluded) now in hopes of catching the return tagger? *

It might be assumed that none actually WANT what Fedora systems have to
offer, other than exposing Fedora for the poor security structure?*

According to CERT:
http://www.cert.org/tech_tips/win-UNIX-system_compromise.html  The following
are BASIC steps to recovery from any exploit or encroachment event:

ITEMS that might be lacking here are in RED:

Introduction<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#Introduction>

A. Before you get
started<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#A>

SNIP - WE ASSUME YOU HAVE COMPLETED THE BASICS!
D. Contact the relevant CSIRT and other sites
involved<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#D>

   1. Incident Reporting<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#D.1>
   2. Contact AusCERT - Australian Computer Emergency Response
Team<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#D.2>
   3. Contact the CERT Coordination
Center<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#D.3>
   4. Obtain contact information for other sites
involved<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#D.4>

E. Recover from the
intrusion<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#E>

   1. Install a clean version of your operating
system<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#E.1>
   2. Disable unnecessary
services<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#E.2>
   3. Install all vendor security
patches<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#E.3>
   4. Consult AusCERT advisories and external security
bulletins<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#E.4>
   5. Consult CERT advisories and vendor-initiated
bulletins<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#E.5>
   6. Caution use of data from
backups<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#E.6>
   7. Change passwords<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#E.7>

F. Improve the security of your system and
network<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#F>

   1. Review security using the UNIX or NT configuration guidelines
document<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#F.1>
   2. Install security
tools<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#F.3>
   3. Enable maximal
logging<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#F.4>
   4. Configure firewalls to defend
networks<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#F.5>

G. Reconnect to the
Internet<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#G>
 H. Update your security
policy<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#H>

   1. Document lessons learned from being
compromised<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#H.1>
   2. Calculate the cost of this
incident<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#H.2>
   3. Incorporate necessary changes (if any) in your security
policy<http://www.cert.org/tech_tips/win-UNIX-system_compromise.html#H.3>

>From JSmith's post describing the incident:

The account in question was not a member of any sysadmin or Release Engineering
groups. The following is a complete list of privileges on the account:
 * SSH to fedorapeople.org (user permissions are very limited on this machine).

 * Push access to packages in the Fedora SCM.
 * Ability to perform builds and make updates to Fedora packages.

The Infrastructure Team took the following actions after being
notified of the issue:
1. Lock down access to the compromised account

2. Take filesystem snapshots of all systems the account had access to
   (pkgs.fedoraproject.org, fedorapeople.org)
3. Audit SSH, FAS, Git, and Koji logs from the time of compromise to the

    present
   Here, we found that the attacker did:
    * Change the account's SSH key in FAS
    * Login to fedorapeople.org
   The attacker did not:
    * Push any changes to the Fedora SCM or access pkgs.fedoraproject.org in

      any way
    * Generate a koji cert or perform any builds
    * Push any package updates

Based on the results of our investigation so far, we do not believe that any
Fedora packages or other Fedora contributor accounts were affected by this

compromise.

While the user in question had the ability to commit to Fedora SCM, the
Infrastructure Team does not believe that the compromised account was used to
do this, or cause any builds or updates in the Fedora build system. The

Infrastructure Team believes that Fedora users are in no way threatened by this
security breach and we have found no evidence that the compromise extended
beyond this single account.

As always, Fedora packagers are recommended to regularly review commits to

their packages and report any suspicious activity that they notice.

Fedora contributors are strongly encouraged to choose a strong FAS password.
Contributors should *NOT* use their FAS password on any other websites or

user accounts.  If you receive an email from FAS notifying you of changes to
your account that you did not make, please contact the Fedora Infrastructure
team immediately via admin at fedoraproject.org.
<https://admin.fedoraproject.org/mailman/listinfo/announce>

Thanks for creating great community with the Fedora project and FUDCon.

Please let me know if I can assist in any way to investigate or complete
these important security changes.
-- 

(503) 754-4452
(623) 688-3392

 http://www.obnosis.com
*Catch My MetaSploit & IP CAM Surveillence
Presentations @ ABLEConf.com in April!*

















-- 

(503) 754-4452
(623) 688-3392

 http://www.obnosis.com
*Catch My MetaSploit & IP CAM Surveillence
Presentations @ ABLEConf.com in April!*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.PLUG.phoenix.az.us/pipermail/plug-discuss/attachments/20110130/0ea2bc72/attachment.html>