$85
(If you prefer a shorter version of the project description, jump to the Project Tasks section(page 4) and pay attention to the Deliverable section(page 5-6). However, the first few pages of this document might help you to grasp a better idea of what you are going to encounter. )
You can find the video introduction of the project under project 4 folder on Canvas. Alternatively, you can view the video here as well: https://youtu.be/GHNZ6cBv3nU
This video gives you a brief overview of the structure of the project and how to run the commands. For more details, please refer to the project traffic definitions(page2-3) and tasks(page 4) section.
The goal of this project is to introduce students to the techniques that help to differentiate malicious and legitimate network traffic. This is a task that network operators perform frequently. In this project, the students are provided with samples of malicious and legitimate traffic. They can observe how each type of traffic looks. In the project folder, there is a pcap file that contains network traffic that originates from multiple hosts in the same network. This pcap file is a mixture of legitimate and malicious traffic. The students are asked to investigate the pcap file in network tools such as WireShark. Finally, the students are asked to use Snort and write their own Snort rules, which will differentiate malicious and legitimate traffic. In summary, the students are introduced to:
Figure 1: Network setup for traffic collection.
In this assignment, there are four attack scenarios. For each attack, a scenario is defined based on the implemented network topology, and the attack is executed from one or more machines outside the target network. Figure 1 shows the implemented network, which is a common LAN network topology on the AWS computing platform. The hosts are behind a NAT, and their IP addresses belong to a single /16: 172.31.0.0/16. It also shows a visual representation of the network and our traffic collection set-up.
Types of attacks:
In a DDoS, attackers usually keep making full TCP/UDP connections to the remote server. They keep the connection open by sending valid HTTP requests to the server at regular intervals but also keep the sockets from closing. Since any Web server has a finite ability to serve connections, it will only be a matter of time before all sockets are used up and no other connection can be made.
The possible types of DDoS attacks in the traffic:
A short Video that has some tips for DDOS attacks. The color coding method mentioned in this video is not always true. I would suggest to focus more on the frequency(package/second) and do a comparison with the TCP traffic you observe and the UDP traffic you observe. (if you are not sure about what filter to use in your snort rule, jump to 5:50 in the video. It’s also perfectly fine if you don’t use the detection filter but get 100 on Gradescope. There are multiple ways to complete this task.). Since the packets have randomized time deltas, we do not recommend using frame.time_relative to identify DoS attacks. DDOS might be the most challenging part of the project here. Feel free to share some other tips you find helpful for your DDOS on Ed without giving out your answer!
FTP/SSH is attacked via a Kali Linux machine( the attacker machine), and Ubuntu 14.0 system is the victim machine. There is a large dictionary that contains 90 million words that were used for the list of passwords to brute force.
It is your task to identify which one(FTP or SSH) of them is present in the evaluation pcap given to you. A 4 mins Video that has some useful tips for Bruteforce attack. We do not provide you with the sample pcap for SSH. However, we encourage you to do some research on the differences between FTP traffic and SSH traffic. Also, the 4 mins video might include some hints.
There are 3 possible web attacks, one of which would be present in your pcap.
(a) DVWA-based: Damn Vulnerable Web App (DVWA) is a PHP/MySQL web application that is vulnerable. An attacker might try to hijack it.
(b) XSS-based: An attacker might try to launch an XSS attack.
(c) SQL Injection: An attacker might try an SQL injection attack.
(We do not provide a video for webAttack or Botnet, but the sample pcaps we provided should be sufficient to guide you through this part of the project. Tips: Look at the contents of the traffic, that’s the key!)
Zeus is a trojan horse malware that runs on Microsoft Windows. It might be presented in the pcap. It can be used to carry out many malicious and criminal tasks and it is often used to steal banking information by man-in-the-browser keystroke logging and form grabbing. It is used to install the Crypto-Locker ransomware as well. Zeus spreads mainly through drive-by downloads and phishing schemes. The Ares botnet might also be presented in the pcap. It is an open-source botnet and has the following capabilities:
Either Zeus or Ares could be present in your evaluation.pcap, it is your task to identify which one.
Sample traffic: For each type of traffic mentioned above, we provide a sample of that category/type of traffic. These samples are only for illustration purposes. These samples are only examples, and they are not the same as the actual traffic that is included in the evaluation pcap, which the students will need to label.
For this exercise, we assume normal traffic to include HTTP, DNS. An example of normal (attack free) traffic can be found in:
The host generates this traffic explicitly to communicate with a C&C server. The host communicates with the C&C server to receive commands, updates, etc.
The goals:
If you would like to download the project resources for a local setup, additional installation (Snort 3 and Wireshark) will be required. We will not be able to provide support for configuring the settings if you set it up locally(You will need to copy /usr/local/etc/snort/snort.lua to your local settings). However, we recommend you to run the snort rule inside our VM(cs6262_project4.ova) since it has the snort set up for you already.
SHA256 hash:
You should use the attack descriptions above – to understand how these attacks should look like in network traffic.
Note: we recommend keeping everything on the Desktop or the home directory, as some of the scripts will be there.
keep in mind, we are using Snort3, and not Snort2 – please make sure you use the Snort version installed in the VM.
A example snort rule is
alert tcp 172.1.2.3 any -> any any ( msg:"A packet from 172.1.2.3 to any ip:port"; sid:1; ) |
A script ‘run_eval.sh’ on the virtual machine/in the local setup is a helper script to automate the execution of the commands mentioned below.
Inside the run_eval.sh, we have the following snort command. The following snort command will help you to generate the json file. If you want to run the command individually by yourself instead, be aware that there is a “.” included at the end of the command. You might need to modify the directory path depending on where your files are located. (Change the directory if necessary)
snort -c /usr/local/etc/snort/snort.lua -r ~/Desktop/evaluation.pcap -R ~/Desktop/eval.rules -s 65535 -k none -l . |
For example, if you are writing a rule to detect ssh brute force, then the alert message should be “Bruteforce”. This will be used to grade your result – getting this part wrong can lead to a point loss.
(Page 2-3 in the write up has more details and some hints about the traffics. )
Common errors after running cal_unique_connection_2022.py::
For this project, you should submit Two files on Gradescope
Note:
How to validate your answer:
Grading:
Statistics for each type of unique connections (Important!):
(The number might be a little different when you try to find it in Wireshark. Use the number that Snort gives you)
We consider a connection to be “src_ip:src_port:dest_ip:dest_port”. run “python3 ~/Desktop/cal_unique_connection.py alert_json.txt” to check the unique connections of your alert_json.txt and generate the results in `connections.txt`. If your alert JSON file is generated in the home directory, you might need to add sudo in front of your command.
- No, we don't, since similar projects run across semesters.
- You can ignore ICMP & IP
- Yes. If the VM is too slow for you, I would recommend you install pcap reading tools on your local machine and use the VM only for Snort. ‘evaluation.pcap’ is a very large file so it can take some time to load. You may need to increase the amount of RAM available to the system to get it to display properly.
- Correct, you can complete without the VM but make sure your snort file works with the version installed in the given VM.
- A "Connection" is identified by its Source Address, Destination Address, Source Port, and Destination Port.
- You may want to look at packets within a certain time frame.
- It can be from any number of machines. The exact number is not relevant.
- Only IPV4
- No, you just need to submit the two files separately: rules.py and connections.txt.
- We can't confirm or deny. :-)
The sample pcaps are there to give you a good idea, but we don't claim they're representative. You should learn the pattern in the sample pcap and try to find related or similar patterns in the evaluation pcap. Always manually verify that what you find is correct.
- Yes that means your json file is too big and maybe you need to put more constraints on your snort rule to generate a smaller json file.
- Try using the frequency as a filter type. Remember that the direction of tracking matters (by src VS. by dst)
Readings on botnets behavior: Please read through the following papers, to get an understanding of what a bot is, and how botnets behave. Please note that we are not asking you to implement the proposed methodologies, e.g. a machine learning method to detect bots.
Snort resources: Here you can find some examples of Snort rules, and some resources so that you get familiar with Snort rules. The purpose of these resources is only to get you familiar with how Snort rules look like. You are expected to write your own Snort rules.
When you google snort syntax, you should be aware that we are using snort 3 not snort 2. There are some subtle differences between Snort 2 and Snort 3 syntax. Snort 2 syntax might result in errors when you run your rules locally.
Example: Writing Snort rules to detect DDoS traffic: This is an example to give you an idea about how we can use our understanding of an attack, and write Snort rules with potentially long shelf life, to detect this attack. Intro reading for dos: https://en.wikipedia.org/wiki/Denial-of-service_attack. Snort for DDoS: Please read this to get a general idea about how Snort can be used for this purpose. Please focus on sections 3 and 4. After reading the above, one way to detect dos traffic is to monitor the rate of incoming traffic. Here is an example Snort rule based on traffic rate: http://manual-snort-org.s3-website-us-east-1.amazonaws.com/node35.html
(Notice that this link is actually the snort 2 manual. However, the syntax for detection_filter is the same in snort 2 and 3. The Snort 2 manual in this case provides a better example to illustrate this filter. For other filters, especially the URL content filters, you should double check the snort 3 manual and other resources online)
Useful tools/commands:
We do recommend you to install Wireshark on your own machine for evaluation pcap analysis because the pcap file is big and your VM might be significantly slower than your laptop.
(optional reading) Subnet:
Because it uses CIDR notation. CIDR and subnetting are virtually the same thing.
CIDR is Classless inter-domain routing. It is the /number representation. In this case, we have /16
/16 represents the subnet mask of 255.255.0.0
If you convert 255.255.0.0 into binary, you will see 16 1's and that's where the number 16 comes from.
Of course, I can't remember all those conversions for all netmask. There is a cheat sheet:
Wait, what's a subnet mask?
Feel free to read this link if you want to know more:
https://avinetworks.com/glossary/subnet-mask/
Disclaimer for background traffic. Please note that the traffic that is found in the evaluation pcap, and/or at the Sample pcaps is not generated by us. The dataset closely resembles realist traffic. Part of this traffic might contain inappropriate content or language. We have taken extra measures and we have performed considerable effort to filter all traffic, based on commonly used inappropriate words. We have filtered the http payload and URIs. Nevertheless, it might still be possible that some inappropriate content or words might have not been filtered entirely. In case you locate such content, we are letting you know that it is not intentional, and we are not responsible for it. Also, to complete this assignment, you do not need (nor do we ask you) to click on URLs found inside http payloads.
Additional tools are not allowed. For the assignment, you are not allowed to use any available tools, related to Snort or others. For example, you are not allowed to use Snort preprocessors that may be publicly available, pre-compiled Snort rules, detection tools. etc. You are expected to write your own Snort rules.
Limited support for running snort locally. We have the env set up for you inside the VM. Setting up the snort 3 env on your own could be troublesome. We strongly recommend you to run snort rules inside the VM we provide (you can analyze(use wireshark) the pcap outside the VM but run the rules inside the VM).