x86 Assembly Crash Course

CHALLENGE INFORMATION

	Platform: TryHackMe Module: Malware Analysis Room: x86 Assembly Crash Course

DESCRIPTION:

A crash course in x86 assembly to enable us in malware reverse engineering.

Learning Objectives

• Opcodes and operands
• General assembly instructions
• Arithmetic and logical instructions
• Conditionals
• Branching instructions

Introduction

Learning basic assembly language is crucial for malware reverse engineering, as most malware samples are compiled binaries that cannot be directly viewed in their original C or C++ code.

Decompiling these binaries using a decompiler or disassembler often removes important information, making assembly language the most reliable and human-readable code to analyze, particularly when trying to understand what a binary is doing.

Question 1: I have completed the prerequisite room.

No answer needed

Opcodes and Operands

Computer code is typically stored as binary (ones and zeros) or hexadecimal format on disk, which humans cannot easily understand. Binary and hexadecimal code consists of instructions (opcodes) and data (operands), where opcodes tell the CPU what to do, and operands specify what is being acted upon.

When disassembling a program, opcodes are translated into human-readable assembly language instructions, like mov eax, 0x5f. These instructions typically have three parts: an instruction (mov) and two operands (e.g., eax and 0x5f). There are three types of operands:

• Immediate Operands: fixed values like 0x5f.
• Registers: like eax, where data is stored.
• Memory Operands: referenced by square brackets, e.g., [eax], indicating a memory location to operate on.

Please note that due to endianness, the operand 0x5f is written as 5f 00 00 00, which is actually 00 00 00 5f but in little-endian notation.

Understanding these concepts helps reveal how code is executed at the CPU level and is essential for malware analysis and reverse engineering.

Question 1: What are the hex codes that denote the assembly operations called?

Opcodes

Question 2: Which type of operand is denoted by square brackets?

memory operand

General Instructions

Instructions that interact with registers and memory include mov and lea.

MOV

• Moves data from one location (register, memory, or immediate operand) to another.
• Syntax: mov destination, source.
• The source operand remains unchanged.
• Memory-to-memory data movement is not allowed.
• Examples:
  • Move fixed value to register: mov eax, 0x5f.
  • Move value from one register to another: mov ebx, eax.
  • Move value from memory location to register: mov eax, [0x5fc53e].

LEA

• Loads effective address (address of source into destination).
• Syntax: lea destination, source.
• Example: Calculate ebp+4 and move result to eax: lea eax, [ebp+4].

Other Instructions

NOP

• No operation (exchanges the value in eax with itself).
• Syntax: nop.
• Used for consuming CPU cycles or redirecting execution.

SHIFT

• Shifts each register bit to the adjacent bit and moves the overflowing bit into the carry flag or adds zeros.
• Shifting Right: shr destination, count
• Shifting Left: shl destination, count
• Example: Shift eax right by one bit to achieve the same result as dividing eax by two.
• Note: The carry flag tracks any bits that are shifted out of the register.

ROTATE

• Rotates bits back to the other end of the register instead of moving the overflowing bit into the carry flag or adding zeros.
• Rotate to the Right: ror destination, count
• Rotate to the Left: rol destination, count
• Example: Rotate al 10101010 right by one bit to get 01010101.

Question 1: In ‘mov eax, ebx’, which register is the destination operand?

eax

Question 2: What instruction performs no action?

nop

Flags

In x86 assembly language, the CPU uses flags (bits) in the EFLAGS register to indicate the outcome of arithmetic and logical operations. The most common flags are listed below:

• CF (Carry Flag): Set when a carry-out or borrow is required from the most significant bit.
• PF (Parity Flag): Set if the least significant byte has an even number of one bits.
• AF (Auxiliary Flag): Set for binary-coded decimal (BCD) arithmetic, indicating a carry-out or borrow between bits three and four.
• ZF (Zero Flag): Set if the result is zero.
• SF (Sign Flag): Set if the result is negative (most significant bit equals one).
• OF (Overflow Flag): Set for signed arithmetic overflows (e.g., adding two positive numbers to get a negative result).
• DF (Direction Flag): Determines string processing direction (forward or backward). Note that if DF equals zero, the string is processed forward; if DF equals one, the string is processed backward.
• IF (Interrupt Enable Flag): Enables maskable hardware interrupts.

These flags can be used in conditional jumps and are essential for implementing branching logic in assembly code.

Question 1: Which flag will be set if the result of the operation is zero? (Answer in abbreviation)

ZF

Question 2: Which flag will be set if the result of the operation is negative? (Answer in abbreviation)

SF

Arithmetic Instructions

ADD

• Addition
• Instruction: add
• Example: add destination, value

SUB

• Subtraction
• Instruction: sub
• Example: sub destination, value

MUL

• Multiplication
• Instruction: mul
• Example: mul value
• Multiplies the value with eax and stores the result in edx (eax as a 64-bit value).
• Result: lower 32 bits in the eax register; higher 32 bits in edx.

DIV

• Division
• Instruction: div
• Example: div <value>
• Divides the 64-bit value in edx.
• Result: quotient in eax and remainder in edx.

INC

• Increment a register by one.
• Instruction: inc
• Example: inc eax

DEC

• Decrement a register by one.
• Instruction: dec
• Example: dec eax

Logical Instructions

AND

• Bitwise AND operation.
• Instruction: and
• Example: and al, 0x7c

OR

• Bitwise OR operation.
• Instruction: or
• Example: or al, 0x7c

NOT

• Inverts the bits of an operand.
• Instruction: not
• Example: not al

XOR

• Returns one if the inputs are opposite, and zero if they are the same.
• Instruction: xor
• Example: xor al, 0x7c
• XOR-ing a register with itself results in zero. Therefore, the XOR instruction is often used to zero a register, which is more optimized than a MOV instruction.

Question 1: In a subtraction operation, which flag is set if the destination is smaller than the subtracted value?

Carry Flag

Question 2: Which instruction is used to increase the value of a register?

inc

Question 3: Do the following instructions have the same result? (yea/nay)
- xor eax, eax
- mov eax, 0

yea

Conditionals

TEST

• Performs a bitwise AND and sets the Zero Flag (ZF) if the result is zero.
• Instruction: test
• Example: test destination, source
• Often used to check if an operand has a null value by testing the operand against itself. This approach uses fewer bytes than comparing to zero.

CMP

• Compares two operands and sets the Zero Flag (ZF) or the Carry Flag (CF).
• Instruction: cmp
• Example: cmp destination, source
• Works similarly to subtraction.
• Equal operands set ZF; source greater than destination sets CF; source less than destination clears both ZF and CF.

Branching

Unconditional Jump

JMP

• Jumps to a specified location, changing the value of the instruction pointer.
• Instruction: jmp
• Example: jmp <location>

Conditional Jumps

JZ

• Jump if the ZF is set (ZF equals one).
• Instruction: jz

JNZ

• Jump if the ZF is not set (ZF equals zero).
• Instruction: jnz

JE

• Jump if equal. Often used after a CMP instruction.
• Instruction: je

JNE

• Jump if not equal. Often used after a CMP instruction.
• Instruction: jne

JG

• Jump if the destination is greater than the source operand.
• Instruction: jg
• Performs signed comparison and is often used after a CMP instruction.

JL

• Jump if the destination is less than the source operand.
• Instruction: jl
• Performs signed comparison and is often used after a CMP instruction.

JGE

• Jump if greater than or equal to. Jumps if the destination operand is greater than or equal to the source operand.
• Instruction: jge
• Similar to the above instructions.

JLE

• Jump if less than or equal to. Jumps if the destination operand is less than or equal to the source operand.
• Instruction: jle
• Similar to the above instructions.

JA

• Jump if above. Similar to jg, but performs an unsigned comparison.
• Instruction: ja

JB

• Jump if below. Similar to jl, but performs an unsigned comparison.
• Instruction: jb

JAE

• Jump if above or equal to. Similar to the above instructions.
• Instruction: jae

JBE

• Jump if below or equal to. Similar to the above instructions.
• Instruction: jbe

Question 1: Which flag is set as a result of the test instruction being zero?

Zero Flag

Question 2: Which of the below operations uses subtraction to test two values? 1 or 2?
- 1: cmp eax, ebx
- 2: test eax, ebx

1

Question 3: Which flag is used to identify whether a jump will be taken or not after a jz or jnz instruction?

Zero Flag

Stack and Function Calls

PUSH

• Pushes the source operand onto the stack.
• Instruction: push
• Example: push source
• Value is stored at the memory location pointed to by the stack pointer (ESP), becoming the new top of the stack.
• ESP is decremented.

PUSHA

• Pushes all words onto the stack.
• Instruction: pusha
• AX, BX, CX, DX, SI, DI, SP, BP (16-bit general-purpose registers).

PUSHAD

• Pushes all doublewords onto the stack.
• Instruction: pushad
• EAX, EBX, ECX, EDX, ESI, EDI, ESP, EBP (32-bit general-purpose registers).
• When we encounter these instructions, it is often a sign of someone manually injecting assembly instructions to save the state of registers, as is often the case with shellcode.

POP

• Retrieves the value from the top of the stack and stores it in the destination operand.
• Instruction: pop
• ESP is incremented.

POPA

• Pops all words.
• Instruction: popa
• Pops the values sequentially from the top of the stack to general-purpose registers.
• Order: DI, SI, BP, BX, DX, CX, AX.
• ESP is adjusted.

POPAD

• Pops all doublewords.
• Instruction: popad
• Pops the values sequentially from the top of the stack to general-purpose registers.
• Order: EDI, ESI, EBP, EBX, EDX, ECX, EAX.
• ESP or SP is adjusted.

CALL

• Performs a function call operation to perform a specific task.
• Instruction: call
• Example: call <location>
• Depending on the calling convention, arguments are placed on the stack or in registers.
• Prologue: Prepares the stack by adjusting EBP and ESP and pushing the return address onto the stack.
• Epilogue: Restores the stack for the caller function.

Question 1: Which instruction is used for performing a function call?

call

Question 2: Which instruction is used to push all registers to the stack?

pusha

Practice Time

We recommend experimenting with the Assembly Emulator. It provides an excellent interactive environment that makes understanding assembly instructions much easier.

Arithmetic Code

mov eax,20h ; EIP=0x00000001 | EAX=0x00000020
mov ebx,30h ; EIP=0x00000002 | EBX=0x00000030
add eax,ebx ; EIP=0x00000003 | EAX=0x00000050 | PF=1 | EBX=0x00000030 (operand remains unchanged)
nop         ; EIP=0x00000004 | PF=1 (parity flag remains set)
nop         ; EIP=0x00000005
sub eax,ebx ; EIP=0x00000006 | EAX=0x00000020 | PF=0 | EBX remains unchanged
inc ebx     ; EIP=0x00000007 | EBX=0x00000031
dec ebx     ; EIP=0x00000008 | EBX=0x00000031 | PF=1 | AF=1 
mul eax     ; EIP=0x00000009 | EAX=0x00000400 | EDX=0x00000000 | PF=1 | AF=1

MOV Instructions

mov eax,10h     ; EIP=0x00000001 | EAX=0x00000010
mov ebx,32h     ; EIP=0x00000002 | EBX=0x00000032
mov ecx,eax     ; EIP=0x00000003 | ECX=0x00000010 | EAX remains unchanged
mov [eax],40h   ; EIP=0x00000004 | Memory address [0x10]=0x00000040
add [eax],30h   ; EIP=0x00000005 | Memory address [0x10]=0x00000070
mov [ebx],[eax] ; ERROR | Memory to memory data movement is NOT allowed.

Stack

; NOTE-1: Stack works from higher memory locations to lower ones.
; NOTE-2: Stack works in LIFO mode.
mov eax,10h   ; EIP=0x00000001 | EAX=0x00000010
mov ebx, 15h  ; EIP=0x00000002 | EBX=0x00000015
mov ecx, 20h  ; EIP=0x00000003 | ECX=0x00000020
mov edx, 25h  ; EIP=0x00000004 | EDX=0x00000025

; Original values | ESP=0x00001000 | EBP=0x00001000 | EAX, EBX, ECX, EDX remain UNCHANGED
push eax      ; EIP=0x00000005 | ESP=0x00000ffc | Stack location [0xffc]=0x00000010
push ebx      ; EIP=0x00000006 | ESP=0x00000ff8 | Stack location [0xff8]=0x00000015
push ecx      ; EIP=0x00000007 | ESP=0x00000ff4 | Stack location [0xff4]=0x00000020
push edx      ; EIP=0x00000008 | ESP=0x00000ff0 | Stack location [0xff0]=0x00000025

pop eax       ; EIP=0x00000009 | ESP=0x00000ff4 | Stack location [0xff0] CLEARED | EAX=0x00000025
pop ebx       ; EIP=0x0000000a | ESP=0x00000ff8 | Stack location [0xff4] CLEARED | EBX=0x00000020
pop ecx       ; EIP=0x0000000b | ESP=0x00000ffc | Stack location [0xff8] CLEARED | ECX=0x00000015
pop edx       ; EIP=0x0000000c | ESP=0x00001000 | Stack location [0xffc] CLEARED | EDX=0x00000010

CMP and TEST Instructions

mov eax,10h   ; EIP=0x00000001 | EAX=0x00000010
mov ebx,10h   ; EIP=0x00000002 | EBX=0x00000010
cmp eax,ebx   ; EIP=0x00000003 | ZF=1 and PF=1
test eax,ebx  ; EIP=0x00000004 | ZF=0 and PF=0

mov eax,20h   ; EIP=0x00000005 | EAX=0x00000020
mov ebx,10h   ; EIP=0x00000006 | EBX=0x00000010
cmp eax,ebx   ; EIP=0x00000007 | source < destination -> clear ZF and CF
test eax,ebx  ; EIP=0x00000008 | bitwise AND and set ZF if 0 --> ZF=1 and PF=1

mov eax,20h   ; EIP=0x00000009 | EAX=0x00000020 | ZF=1 and PF=1 (remain unchanged)
mov ebx,40h   ; EIP=0x0000000a | EBX=0x00000040
cmp eax,ebx   ; EIP=0x0000000b | source > destination -> set CF | CF=1 and SF=1 (most significant bit = 1)
test eax,ebx  ; EIP=0x0000000c | ZF=1 and PF=1 | CF, SF are cleared

LEA Instruction

mov eax,20h       ; EIP=0x00000001 | EAX=0x00000020
mov ebx,30h       ; EIP=0x00000002 | EBX=0x00000030
add eax,ebx       ; EIP=0x00000003 | EAX=0x00000050 | PF=1
nop               ; EIP=0x00000004 | PF remains set
mov [eax],ebx     ; EIP=0x00000005 | Memory location [0x50]=0x00000030
add ebx,15h       ; EIP=0x00000006 | EBX=0x00000045 | Clear PF
mov ecx,6         ; EIP=0x00000007 | ECX=0x00000006
mov [ebx+ecx],eax ; EIP=0x00000008 | Memory locations [0x4b]=0x50000000 and [0x4c]=0x00000000 | (0x45+0x06=0x4b)
lea eax,[ebx+ecx] ; EIP=0x00000009 | EAX=0x0000004b
push eax          ; EIP=0x0000000a | ESP=0x00000ffc and Stack location [0xffc]=0x0000004b
push ebx          ; EIP=0x0000000b | ESP=0x00000ff8 and Stack location [0xff8]=0x00000045
pop ecx           ; EIP=0x0000000c | ESP=0x00000ffc and Stack location [0xff8] cleared | ECX=0x00000045

Question 1: While running the MOV instructions, what is the value of [eax] after running the 4th instruction? (in hex)

0x00000040

Question 2: What error is displayed after running the 6th instruction from the MOV instruction section?

Memory to memory data movement is not allowed.

Question 3: Run the instructions from the stack section. What is the value of eax after the 9th instruction? (in hex)

0x00000025

Question 4: Run the instructions from the stack section. What is the value of edx after the 12th instruction? (in hex)

0x00000010

Question 5: Run the instructions from the stack section. After ‘POP ecx’, what is the value left at the top of the stack? (in hex)

0x00000010

Question 6: Run the cmp and test instructions. Which flags are triggered after the 3rd instruction?
Note: Use these abbreviations in alphabetical order with no spaces: CF,PF,SF,ZF

PF,ZF

Question 7: Run the test and the cmp instructions. Which flags are triggered after the 11th instruction?
Note: Use these abbreviations in alphabetical order with no spaces: CF,PF,SF,ZF

CF,SF

Question 8: Run the instructions from the lea section. What is the value of eax after running the 9th instruction? (in hex)

0x0000004B

Question 9: Run the instructions from the lea section. What is the final value found in the ECX register? (in hex)

0x00000045

Conclusion

Question 1: Join the discussion on our social channels.

No answer needed