Congratulations! You have just been named the newest reverse engineering intern at Commonwealth National Laboratory.
You happily report to work at 7:00 am on your first day, find your newly vacuumed cubicle, and excitedly await all of the fun malware reverse engineering you will be doing. “What will they give me on my first day” you wonder. Will it be Stuxnet? Some malware that attacks nuclear weapons maybe!
Your boss rolls in around 11:00 am. He looks at you and says “We just got that fancy new GHEEDAR tool, but it don’t show us the control flow like we like around here.”
“Click on Graph View!” you say, but it is clear that he has never even opened GHIDRA before.
“Write one of those GHEEDAR plugins you kids like. Make it generate a DOT graph where each node is an instruction addresses and the edges show the control flow from one instruction to the next.”
“Click on Graph View!” you implore.
“Do this too...” he says, “in every node, label the instruction address and also any registers or constant memory addresses that the instruction defines or uses.”
“This will be a long summer” you think as you begin looking up the GHIDRA SDK docs.
Instructions:
In all seriousness, a Control Flow Graph (CFG) is the fundamental building block of all program analysis algorithms. Nearly all malware analysis tools start by building a control flow graph. Once you have a control flow graph, tracking how data flows through the malware sample is essential for answering many questions. For example, by analyzing a CFG and the data being defined and used by each instruction, you can automatically isolate the paths that connect to C&C servers or exfiltrate specific user data. In this lab, you will build these essential building blocks for malware analysis. After completing this lab, I encourage you to go a step further and write an analysis script to automatically extract any CFG paths that touch GreenCat’s C&C URL.
1. As described in Week 6 slides, GHIDRA has a few example scripts to help you get the feel for development.
2. Get the Hello World plugin (either compiled Java or Python) running in GHIDRA .
Create a new GHIDRA project with greencat-2. Do not use your GHIDRA project from Lab 2. Keep that Lab 2 project safe and sound so that you can refer to it as a “ground truth” for Lab 3 and beyond. A fresh project with no comments will make plugin development easier and the comments will confuse the Lab 3 grader script.
3. Build a GHIDRA plugin (I recommend building from the hello world plugin) that:
• Loops every instruction in every basic block in every function in your greencat-2 binary (from before). You will need to follow the control flow graph, so your plugin will know the execution order of the instructions.
• Generate a DOT directed graph representing the control flow of all the instructions in each function. Specifically, one DOT graph per function — name each DOT graph (called a “digraph” in the DOT file format) the address of the function. More information about DOT directed graph files here: https://en.wikipedia.org/wiki/DOT_(graph_descriptio n_language)#Directed_graphs.
• Each node in your DOT directed graph file should be the address of an instruction (only ONE node per instruction address). The edges in your DOT directed graph file should go from each instruction to the next instructions which execute immediately after it.
Consider this example from the greencat-2 binary. Here are the instructions in the function starting at address 0x401000 in greencat-2:
401000: push esi
401001: mov esi, ecx
401003: call 0x401078
401008: test BYTE PTR [esp+0x8], 0x1
40100d: je 0x401016
40100f: push esi
401010: call 0x402a5c
401015: pop ecx
401016: mov eax, esi
401018: pop esi
401019: ret 0x4
The DOT directed graph generated by your tool for this function should be as follows:
digraph "0x401000" { n1 [label = "0x401000;"]; n2 [label = "0x401001;"]; n3 [label = "0x401003;"]; n4 [label = "0x401008;"]; n5 [label = "0x40100d;"]; n6 [label = "0x40100f;"]; n7 [label = "0x401010;"]; n8 [label = "0x401015;"]; n9 [label = "0x401016;"]; n10 [label = "0x401018;"]; n11 [label = "0x401019;"];
n1 -> n2; n2 -> n3; n3 -> n4; n4 -> n5; n5 -> n9; n5 -> n6; n6 -> n7; n7 -> n8; n8 -> n9; n9 -> n10; n10 -> n11;
}
The order of the edges in the DOT directed graph file do not matter. Also see: https://stackoverf
low.com/questions/1494492/graphviz-how-to-go-from-dot-to-a-graph
Add the def use list to every node’s label. Specifically, for each node: (1) build a list of the registers and memory locations which that instruction defines and uses, then (2) add that list to the node’s label. The ordering of the list does not matter.
For the example above, the final DOT directed graph generated by your tool for the function starting at address 0x401000 in greencat-2 should be as follows:
Registers are not case sensitive when grading.
digraph "0x401000" { n1 [label = "0x401000; D: [esp], esp U: esi, esp"]; n2 [label = "0x401001; D: esi U: ecx"]; n3 [label = "0x401003; D: eax, esp U: ecx, esp"]; n4 [label = "0x401008; D: eflags U: [esp + 0x8], esp"]; n5 [label = "0x40100d; D: U: eflags"]; n6 [label = "0x40100f; D: [esp], esp U: esi, esp"]; n7 [label = "0x401010; D: eax, esp U: [esp], esp"]; n8 [label = "0x401015; D: ecx, esp U: [esp], esp"]; n9 [label = "0x401016; D: eax U: esi"]; n10 [label = "0x401018; D: esi, esp U: [esp], esp"]; n11 [label = "0x401019; D: esp U: esp"];
n1 -> n2; n2 -> n3;
n3 -> n4; n4 -> n5; n5 -> n6; n5 -> n9; n6 -> n7; n7 -> n8; n8 -> n9; n9 -> n10; n10 -> n11;
}
Note: Why does 0x401003 use ECX and why does 0x401010 use [ESP]? The answer has to do with Calling conventions. 0x401003 is defined as a fastcall calling convention, and it takes one argument. 0x401010 is a stdcall function and takes one argument. In cases where CALL instructions accept arguments through the stack, you can assume [ESP] as the correct USE to capture this.
Your tool should process every function in the greencat-2 binary. All DOT graphs for all the functions should be output in a single “.dot” file. So, after you GHIDRA plugin finishes executing, you should have a single “.dot” file with many digraphs in it (one digraph per function).
Note: If you are curious about what the Def and Use for a particular instruction in GreenCat should be, then please ask on Ed Discussion!
Also Note: You only need to turn in the DOT graph files for #4 above, not for #3!
Formatting Instructions (MUST READ):
It is critical that you strictly adhere to the formatting guidelines for this lab and all future labs requiring graph analysis.
Your graph (DOT file) submission MUST be properly formatted so that our automated feedback generation can work properly. If your graph output does not adhere to the strict formatting guidelines, causing automated feedback generation to fail, you will receive a zero. Your provided script must be able to generate the graph that is submitted (which you will be graded on) and your graph must not require any formatting modifications to enable the automated feedback generation to work properly.
Gradescope will not provide feedback on the correctness of your graph, but it will let you know if there was an error during automated feedback. If you see any errors in Gradescope, please come back to this section and the previous section’s examples and review the guidelines. If you still have questions, you can always make a private Ed Discussionpost.
Formatting guidelines:
• There must be one digraph header for each function in your analysis. The digraph header must contain digraph FUNCTION_ADDRESS { – Compatible Regex: ˆdigraphs+(.+?)s*{$
– Enclosing the address in quotes, and 0xADDRESS notation are accepted
– Each function digraph header must be on its own line. Notice the regex expects “digraph” to be the first word on the line.
• Each node label MUST be exactly as seen in the previous section.
– NODE [label = "0xADDRESS; D: COMMA, SEPARATED, DEFINITIONS U: COMMA, SEPARATED, USES"];
– Node IDs must be the character “n”, followed by a number. Do not zero fill the number. – Spaces are important. A single space between the semi-colon between the address and def/use attributes is necessary. Likewise, space between the last “def” value and the beginning of the “U:” section is important.
– Do not leave trailing commas in the end of each “def” and "use section.
∗ D: esp, ebp, U: ebp, esp, ebx, is not acceptable
– Make sure the entire “label” section is enclosed in double quotes as seen in the previous section.
– You MUST still have a “D:” and “U:” (def and use section) for all node labels. Even if there are no definitions or uses for that particular node.
• Node edges must be of the format:
– FROM_NODE -> TO_NODE
• There shouldn’t be anything in your dotfile other than each function’s digraph header, the closing digraph footer, edges, and nodes with labels. Additional newlines/whitespace are fine (outside of the strict label formatting), but there is no reason to leave inline comments or anything else. Remove any extraneous debug output, or leave commentary in your code instead.
Here are some frequently asked questions (FAQ) regarding formatting:
• It is recommended that node numbers start at 1 from the first instruction in the function, although it is not necessary as long as your node numbering is consistent within each function. (The first instruction does not have to be node 1, although recommended)
• For the best results with the automated feedback generation, there is no need to extend the registers to their full 32-bit form when adding them to the def/use listing. Instead, list the register that is actually used in the code.
• EFlags are normalized and abstracted to a generic “eflags” register, so although you can submit the individual flag registers, it is not a requirement.
• To get full credit on CALL instructions, you must track Calling Conventions and Arguments/Return Registers. This will be a big part in your success on Lab 4, so we recommend ensuring this works efficiently now. Assume that GHIDRA ’s default analysis makes the correct assumptions about calling conventions (don’t modify them).
Lab Requirements / FAQ (MUST READ):
This section contains some frequently asked questions and requirements that students should adhere to when working on this assignment.
How do I know the DEF/USE attributes of instructions not provided in the lab doc examples?
Can I use symbol names from my disassembler’s analysis in my DOT file output? No. Please convert all symbol names to their raw addresses. Under most circumstances the usual GHIDRA /IDA API will give them in address form, but please otherwise convert them using the relevant APIs in your final DOT submission, since symbol names can be arbitrarily modified by auto-analysis or you, it is simplest to make sure that we use the raw addresses in the PE file. This is most common in IDA when dealing with auto-created structs, so make sure to double check these specifically! If you leave symbol names in your DOT output, we won’t be able to properly grade your assignment.
How much can I share in Ed Discussion?
Should I include EIP in my assignment?
No. EIP is a special register that tracks the next instruction to execute. We completely ignore EIP while grading, so we recommend that you do as well!
I’m using the API, but I’m getting different DEF/USE results than what I see in the example. What is happening?
This assignment is provided to students and to be solved using either GHIDRA depending on the course section (online and on-campus respectively). Both tools have minor discrepancies in the way that they provide DEF/USE information via the API. To normalize this, we have taken the “most correct” interpretation of DEF/USE for these instructions by consulting the Intel documentation. The DEF/USE we are expecting for these instructions is provided below. If the mnemonic is not mentioned, assume the API gives all of the necessary information.
Please ensure that you manually modify these instruction mnemonics as you build your DEF/USE graph. This allows the autograder to work as efficiently as possible and give you the best feedback. For these instruction mnemonics we will only accept the interpretation provided below as correct.
LEAVE: DEF(ebp, esp), USE(ebp, [ebp])
POP: DEF(esp), USE(esp, [esp])
PUSH: DEF(esp, [esp]), USE(esp, [registers (if used)])
RET: DEF(esp), USE(esp)
CALL: DEF(esp, [return registers]), USE(esp, [argument registers])
Why are immediate values/scalars/constant values not included in DEF/USE?
This is a common pitfall that students run into. Please review the lectures on DEF/USE for additional information. For the purposes of DEF/USE, we only care about symbolic values such as registers or memory locations that can be modified or potentially represent different values depending on the state of execution. Immediate (or constant) values such as file offsets or static numbers used for arithmetic are not considered for DEF/USE because they will always be the same across every execution.
Why does X mnemonic have Y DEF/USE attributes?
We recommend that you consult the Intel documentation for this task, but if you’ve already had a look at the documentation and it still doesn’t make sense, please ask!
Grade: 100points
Grading Criteria:
Note: Create a new GHIDRA project with greencat-2. Do not use your GHIDRA project from Lab 2. Keep that Lab 2 project safe and sound so that you can refer to it as a “ground truth” for Lab 3 and beyond. A fresh project with no comments will make plugin development easier and the comments will confuse the Lab 3 grader script. The grade will be based on how many instructions and functions your plugin processes correctly, and is ultimately based on your graph submission (DOT file).
Def/Use accuracy of top 11 instruction mnemonics are worth 5% of the total grade each (mov, add, sub, call, cmp, test, xor, push, pop, lea, all forms of jump). For example, if 20% of your mov instructions are wrong (missing a Def or Use or have an erroneous Def or Use) then you will lose 1% of the total 100 points.
Def/Use accuracy of all other instruction types are collectively worth 15% of the total grade. For example, if 30% of the other instructions are wrong (missing a Def or Use or have an erroneous Def or Use) then you will lose 5% of the total 100 points.
Edge accuracy is worth 30% of the total grade. For example, if 10% of your edges are wrong (missing or have an erroneous extra edge) then you will lose 3% of the total 100 points. Note: Grades in each mnemonic section are rounded down to the nearest percent.
Teams:
This assignment can be done individually or in a team of 2. Please join a group in Gradescope if you are collaborating.
Do not create or join a group in Canvas. Canvas groups are different from Gradescope groups.
New to Gradescope? This link provides instructions for how to create groups in Gradescope: https://help.gradescope.com/article/m5qz2xsnjy-student-add-group-members
Zoom can also provide the ability to collaborate and video conference with your teammate.
Submission Instructions:
Upload the following to the Lab 3 Assignment in Gradescope:
The DOT file output by your GHIDRA plugin, named “submission.dot” which contains digraphs for every function in the greencat-2 binary.
Your GHIDRA plugin code, named either “plugin.py” or “plugin.java” depending on the chosen language. We reserve the right to run all submitted code, through automated means or otherwise, and if it is found that your code does not output equivalent to your original dotfile submission then you will also receive a zero.
Be advised, please submit (1) and (2) separately, do NOT zip them together.
Note: Gradescope will only check the formatting of your submission. Gradescope will not automatically check the correctness and provide a grade.
Note: You can download the webc2-greencat-2.7z file directly into your lab environment. After you are done with this lab, you can submit your files directly from the lab environment (Highly recommended). Doing this will help you avoid transferring the file from the lab environment to your personal computer.
Transferring Files:
To transfer files from your personal device to the lab environment:
Create a zip folder of all the files that you would like to transfer to the lab environment. Every GT student has Box and OneDrive accounts given free by the institution. Login to either of those two and upload the desired files.
Now go back to the lab environment and login to either of those two services where you uploaded you zip folder. Download folder to the the lab workspace and use the appropriate 7z command to unzip your folder.
Grades have been released. How do I view my raw feedback?
import base64 import gzip
base85_encoded_data = b“Paste the encoded data here” base85_decoded_data = base64.b85decode(base85_encoded_data) gzip_decompressed_data = gzip.decompress(base85_decoded_data)
with open(“output.txt”, “wb”) as f:
f.write(gzip_decompressed_data)