Not only was the FUDCon verbal discussion
incomplete in context to standard response during a security incident, but the statements appearing in Announce listserv and websites failed to address, within industry standards, the full impact, ramifications and forensics, nor appropriate OPSEC-ish security forensics and solutions.
Specifically:
0) Was the system booted into a forensics tool to look at low level hashing or unallocated space file changes (deleted files, etc.)? If it was not rebooted, did you obtain a RAM image?
1) Was a CRC check made against the systems binaries (we know the Fedora source tree was checked and found to be pristine, don't we)? How
did you determine that no escalated privileges allowed obfuscation of actions?
2) You vaguely state that insufficient information exists, within the window, to determine who did what since "so many people changed their keys at that time". Were new users added? If, in fact
this is true, your systems fail to meet basic logging recommendations for any server set out by CERT, LOPSA, SCALE, LPI.
3) Redesign/configure FAS to require both https access and FULLY RANDOM user passwords.
4) Remove the SSH key change feature from Web usermin-ish access, instead requiring additional information from either a database of via email exchange. A web based SUID clearly must be protected, since it can be deployed without logging from a shell account?
Since SSLStrip will provide decrypted network information, running on the same network as any attacker, as users log into
FAS, the design fails to meet standard tiered security requirements,
providing ease of access to the entire code base! Either VPN (protect via OSI Layer security) or add additional code based access authentication for escalated privileges like changing keys, and/or adding to the repo (over and above a SSH key). Many security professionals, would, in fact, recommend that SSH be turned off completely outside of a VPN.
5) Clearly if the user had access to the system, they potentially have all the passwords and keys - the system MUST BE REBUILT and REDESIGNED, does it not?
Perhaps Fedora has designated the entire system as a Honeypot (VLAN excluded) now in hopes of catching the return tagger?
It might be assumed that none actually WANT what Fedora systems have to offer, other than exposing Fedora for the poor security structure?According to CERT:
http://www.cert.org/tech_tips/win-UNIX-system_compromise.html The following are BASIC steps to recovery from any exploit or encroachment event:
ITEMS that might be lacking here are in RED:
Introduction
A. Before you get started
SNIP - WE ASSUME YOU HAVE COMPLETED THE BASICS!
D. Contact the relevant CSIRT and other sites involved
- Incident Reporting
- Contact AusCERT - Australian Computer Emergency
Response Team
- Contact the CERT Coordination Center
- Obtain contact information for other sites
involved
E. Recover from the intrusion
- Install a clean version of your operating system
- Disable unnecessary services
- Install all vendor security patches
- Consult AusCERT advisories and external security bulletins
- Consult CERT advisories and vendor-initiated bulletins
- Caution use of data from backups
- Change passwords
F. Improve the security of your system and network
- Review security using the UNIX or NT configuration
guidelines document
- Install security tools
- Enable maximal logging
- Configure firewalls to defend networks
G. Reconnect to the Internet
H. Update your security policy
- Document lessons learned from being
compromised
- Calculate the cost of this incident
- Incorporate necessary changes (if any) in your
security policy
From JSmith's post describing the incident:
The account in question was not a member of any sysadmin or Release Engineering
groups. The following is a complete list of privileges on the account:
* SSH to fedorapeople.org (user permissions are very limited on this machine).
* Push access to packages in the Fedora SCM.
* Ability to perform builds and make updates to Fedora packages.
The Infrastructure Team took the following actions after being
notified of the issue:
1. Lock down access to the compromised account
2. Take filesystem snapshots of all systems the account had access to
(pkgs.fedoraproject.org, fedorapeople.org)
3. Audit SSH, FAS, Git, and Koji logs from the time of compromise to the
present
Here, we found that the attacker did:
* Change the account's SSH key in FAS
* Login to fedorapeople.org
The attacker did not:
* Push any changes to the Fedora SCM or access pkgs.fedoraproject.org in
any way
* Generate a koji cert or perform any builds
* Push any package updates
Based on the results of our investigation so far, we do not believe that any
Fedora packages or other Fedora contributor accounts were affected by this
compromise.
While the user in question had the ability to commit to Fedora SCM, the
Infrastructure Team does not believe that the compromised account was used to
do this, or cause any builds or updates in the Fedora build system. The
Infrastructure Team believes that Fedora users are in no way threatened by this
security breach and we have found no evidence that the compromise extended
beyond this single account.
As always, Fedora packagers are recommended to regularly review commits to
their packages and report any suspicious activity that they notice.
Fedora contributors are strongly encouraged to choose a strong FAS password.
Contributors should *NOT* use their FAS password on any other websites or
user accounts. If you receive an email from FAS notifying you of changes to
your account that you did not make, please contact the Fedora Infrastructure
team immediately via admin at fedoraproject.org.
Thanks for creating great community with the Fedora project and FUDCon.
Please let me know if I can assist in any way to investigate or complete these important security changes.
--
(503) 754-4452
(623) 688-3392
http://www.obnosis.com
Catch My MetaSploit & IP CAM Surveillence
Presentations @ ABLEConf.com in April!