Foundations | Linux 101 - Part 1

Attribution

This short Introduction to Linux is based on notes taken during the TCM Security - Linux 101 video course by Brent Eskridge, PhD.

We highly recommend this course as a great starting point for learning Linux; it was instrumental in our understanding of the subject. We would like to extend a big thank you to the TCM Security Team for providing such valuable resources!

Introduction to Linux

What is Linux?

• An open source operating system (OS)
• An OS is a resource manager that handles:
  • Processor time
  • Memory
  • Storage space
• The kernel is the core of the OS and operates at the lowest level

What is a Linux Distribution?

A distribution (or ‘distro’) is a complete Linux system that includes:

• A desktop environment
• Package managers for installing software
• Pre-installed applications
• Specific goals or philosophy (security-focused, user-friendly, lightweight, etc.)

Introduction to the Command Line

Basic Commands

• Print effective userid: whoami
• Show or set the system’s host name: hostname
• Print name of current/working directory: pwd
• List directory contents: ls
• Change the shell working directory: cd

Getting Help

• Read the manual: man <command>
• Read Info documents: info <command>
• Quick help: <command> -h or <command> --help
• Online explanation: explainshell.com

Viewing Text Files

• more – Filter for paging through text one screen at a time
• less – Similar to more, but with more features (search, scroll back, etc.)

Working with File Contents

• Concatenate files: cat file1.txt file3.txt
• Number all output lines: cat -n file1.txt file3.txt
• Create a file interactively: cat > new-file.txt (type, then press Ctrl+D to save)
• Redirect output to a new file: cat file1.txt file3.txt > combined.txt

Files and the Filesystem

Linux Filesystem Hierarchy

View the filesystem description: man hier

System Logs

Main system log: /var/log/syslog

Devices, Partitions, and Mounting

Device naming example: /dev/sda

• s = SATA drive
• a = first drive

Example with two storage devices: /dev/sda and /dev/sr0

• sda has 2 primary partitions and 3 secondary partitions
• Each partition has its own ID and mounts at a different location in the filesystem

Useful commands:

• View partition mount info: less /etc/fstab
• Show currently mounted devices: mount
• Check disk space usage: df -h
• Check directory size: du -sh ~

Absolute and Relative Paths

Absolute Path:

• Valid from anywhere in the filesystem
• Assumes a global directory structure
• Longer
• Can use environment variables (like $HOME)

Relative Path:

• Valid from the current directory
• Assumes a local directory structure
• Shorter
• More portable across systems

Managing Files and Directories

Files:

• Create empty file: touch file.txt
• Copy: cp source destination
• Move/rename: mv oldname newname
• Delete: rm file.txt

Directories:

• Create: mkdir dirname
• Remove empty directory: rmdir dirname
• Remove non-empty directory: rm -rf dirname/
• Move/rename: mv project project01

Handling Spaces in Names

Bash uses spaces to separate arguments. Escape spaces with a backslash or use quotes:

cd "My Documents"
# or
cd My\ Documents

File and Path Expansion (Globbing)

Use wildcards to match multiple files. Globbing works within a single path segment (between slashes).

Other examples:

• ls file[1-3].txt – 1 or 2 or 3
• ls file[a-zA-Z].txt – letters from a to z AND from A to Z

Viewing File Contents

• First 5 lines: head -n 5 wordlist.txt or head -5 wordlist.txt
• Last 5 lines: tail -n 5 wordlist.txt or tail -5 wordlist.txt
• Follow a log file live: tail -f /var/log/auth.log
• Compare two files: diff file1.txt file2.txt

Hard and Soft Links

A link is a reference to a file or directory.

Hard Links

• Point to the physical location on storage
• Cannot link to directories (prevents circular loops)
• The file persists until all hard links are deleted
• Rarely used today

# Create a hard link
ln hello.txt hello-hard-link.txt

# Remove original -- link still works
rm hello.txt
cat hello-hard-link.txt

# Recreate the original from the hard link
ln hello-hard-link.txt hello.txt

Soft/Symbolic Links

• Reference the file path, not physical location
• Break if the target is moved or deleted
• Can link to directories and cross filesystems
• Use absolute paths for the target

# Create a soft link
ln -s /full/path/to/hello.txt hello-soft-link.txt

# Moving the original breaks the link
mv hello.txt hello-new-name.txt
cat hello-soft-link.txt  # Error: No such file

Compressing and Archiving

Archiving combines multiple files into one; compression reduces file size.

ZIP

# Create zip
zip backup-files.zip file1.txt file2.txt

# List contents
unzip -l backup-files.zip

# Compress multiple dirs -- without including their files
zip tmp/backup-dirs.zip dir1 dir2

# Include subdirectories
zip -r backup-dirs.zip dir1 dir2

TAR (Tape Archive)

# Create archive (c:create, v:verbose, f:file)
tar cvf backup.tar file1.txt dir1/

# List contents (t:confirm)
tar tvf backup.tar

# Extract
tar xvf backup.tar

# Compress with gzip (g:gunzip) -- .gz extension
tar cvfz backup.tar.gz file1.txt dir1/

# Choose compression algorithm (gzip; bzip2) -- compress a tar file
gzip backup.tar

# Decompress the file
gunzip backup.tar.gz

Searching the Filesystem

• Find command location: which ls
• Find binaries, source, and man pages: whereis ls

FIND – Search in real-time

# Search by name (case-sensitive)
find . -name 'file*.txt'

# Case-insensitive
find . -iname 'file*.txt'

LOCATE – Search a pre-built database (faster)

locate file.txt
locate 'file*.txt'

Users and Groups

Viewing Users

• Currently logged in: users, who, w
• Home directories: ls -la /home

File and Directory Permissions

Change permissions:

• Symbolic: chmod g-w hello.txt or chmod a=,u=r hello.txt
• Octal: chmod 664 hello.txt

Change ownership:

sudo chown sally hello.txt      # change owner
sudo chgrp sally hello.txt      # change group

Switching Users

# Run command as another user
sudo -u sally cat /home/sally/sample.txt

# Switch to another user
su sally

Changing Passwords

• Change your own password: passwd
• Change another user’s password: sudo passwd username

Groups

Groups do not have any passwords anymore:

• /etc/groups –> x in 2nd column
• * in /etc/passwd –> disabled login

Installing Software

Package Management

Programs, libraries, and configs are bundled into packages.

Debian Systems (.deb)

# package managers: dpkg and apt
sudo apt update			# update package database
sudo apt upgrade		# upgrade installed packages
apt search <software>		# search for package
apt show <software>		# show package info
sudo apt install <software>	# install
sudo apt remove <software> 	# uninstall
sudo apt purge <software>  	# uninstall + remove configs
sudo apt autoremove        	# clean up dependencies

RedHat Systems (.rpm)

# package managers: rpm and yum
sudo yum check-update      	# update database
sudo yum update            	# upgrade system
yum search <package>		# search for a package
yum info <package>		# check package info
sudo yum install <package>	# install package
sudo yum remove <package>	# remove package

Manual Installation from Source

When no package exists, compile from source:

1. Install build tools:

# missing -- checkinstall;libz-dev
apt install build-essential automake checkinstall libz-dev libssl-dev libcurl4-gnutls-dev libexpat1-dev gettext cmake gcc curl
		  
# modified -- omitting the missing ones
sud apt install build-essential automake libssl-dev libcurl4-gnutls-dev libexpat1-dev gettext cmake gcc curl

2. Download and extract:

wget https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.29.3.tar.gz
tar xfz git-2.29.3.tar.gz
cd git-2.29.3/

3. Configure and build:

less INSTALL		# check the install help file
./configure		# configure with default options; creates Makefile
ls Makefile		# verify makefile
make			# compile (might take a while)

4. Install and verify:

sudo make install	# install the software
git --version           # verify install

Shell Basics

Common Shells

• bash – Bourne Again Shell (default on most systems)
• zsh – Z Shell (modern, default on Kali)
• ksh – Korn Shell
• csh/tcsh – C Shell (popular with programmers)

Some commands like cd are built into the shell; like file and path expansions (* wildcard). The shell first interprets the wildcard, then substitutes the names and sends them to ls: ls file*.txt

Environment Variables

Each shell maintains its own variables. Changes in one shell do not affect others.

• List variables: printenv
• Format: NAME=value (uppercase by convention)
• PATH – directories searched for executable commands

Command execution in shell

1. Type the command.
2. Shell checks if it’s a built-in command.
3. If not built-in command, search PATH directories for the command.
4. If command exists in multiple directories, first is executed.

Global vs Local Variables

Global vs Local Variable example:

# Local variable
COUNT=42
echo $COUNT      # works
bash             # start subshell
echo $COUNT      # empty
exit

# Global variable
export COUNT=55
bash
echo $COUNT      # works
exit

unset COUNT      # remove variable

Startup Files

Bash loads configuration files on startup. Edit ~/.bashrc for permanent changes:

# Add to ~/.bashrc
alias del='rm -rfi'
alias c='clear'

# Reload configuration
source ~/.bashrc

Input/Output Redirection

Three standard streams:

• stdin (0) – input
• stdout (1) – normal output
• stderr (2) – error output

Examples:

# Redirect output to file
ls /etc/ > dir-contents.txt

# Append to file
ls /etc/ >> dir-contents.txt

# Redirect input from file
head < /etc/passwd

# Discard errors
find / -name 'file' 2> /dev/null

# Redirect both output and errors
find / -name 'file' &> results.txt
# or
find / -name 'file' > results.txt 2>&1
# redirecting to different locations
find / -name 'sample.txt' > location.txt 2> /dev/null

Pipes

Connect stdout of one command to stdin of another:

ls -l /etc/ | less
ls -l /etc/ | head -n 20 | tail -n 5
find / -name 'file' 2>/dev/null | less

If you want to redirect the stderr too, use |& like find / -name 'sample.txt' |& less.

Command History

• View history: history
• Run command #199: !199
• Run previous command: !! or !-1
• Run last command starting with ‘cat’: !cat
• Replace text in previous command: ^old^new
• History size is determined by HISTSIZE and HISTFILESIZE in .bashrc.

Command Substitution

With back ticks (`) or $(<command>) to execute those inside first. Once the inside is executed, it will substitute the content of the file with the command.

# use backticks `
ls -l `cat file-list.txt`

# or -- the SAME -- (preferred)
ls -l $(cat file-list.txt)

More Utilities

Text Processing

• Search for text: grep error /var/log/*.log
• Show context: grep -B 3 -A 2 error logfile (3 lines before, 2 after)
• Sort alphabetically: sort file.txt
• Sort numerically descending: sort -nr file.txt
• Remove duplicates: sort file.txt | uniq
• Count lines: wc -l

SED (Stream Editor)

Edit text as it flows through (manipulates text as it flows by; modifies text line-by-line):

# Replace first occurrence per line
sed 's/foo/bar/' file.txt
# or
echo Suite Suite | sed 's/Suite/Ste/'

# Replace only the last occurrence
sed '$s/foo/bar/' file.txt

# Replace all occurrences
sed 's/foo/bar/g' file.txt
# or
echo Suite Suite | sed 's/Suite/Ste/g'

# Replace only on lines matching pattern
sed '/pattern/ s/foo/bar/g' file.txt

# Delete lines matching pattern
sed '/pattern/d' file.txt

# Separate lines by new line and substitute commas with new lines
sed 's/$/\n/g' file.txt | sed 's/,/\n/g'
# or with a single command and multiple expressions
sed -e 's/$/\n/g' -e 's/,/\n/g' file.txt

AWK

Process text by fields, breaks each line of input into separate fields using specific delimiters. Default delimiter: space.

# will output bob
echo linux bob sally | awk '{print $2}'

# or
echo linux bob sally | awk '{print $3, "likes", $1}'
# sally likes linux
		
# or with comma as separator -- firs, lastname
awk -F ',' '{print $1}' sample.txt

# change name order -- last, firstname
awk -F ',' '{print $1}' sample.txt | awk '{print $2 ", " $1}'
		
# filter by 'dakota' -- display names
awk -F ',' '/Dakota/ {print $1}' sample.txt

# display which lines they are
awk -F ',' '/Dakota/ {print NR,$1}' sample.txt

TR (Translate)

Replace or delete characters:

• Reads from stdin and writes to stdout.
• Replace commas with tabs: cat sample.txt | tr ',' '\t'
• Replace lower-case with upper-case: cat sample.txt | tr 'a-z' 'A-Z'
• Or with sets: cat sample.txt | tr '[:lower:]' '[:upper:]'

Networking

Testing network connection

ping google.com 
# or
ping -c 3 google.com

Configuring network connection

ifconfig
# or
ip address

# display statistics too
ip -s link

# use builtin help
ip address help
# or
ip link help

# bring down/up the internet connection
# dev -- device
sudo ip link set dev enp0s3 down
sudo ip link set dev enp0s3 up

Routing tables

# check routing table settings
ip route
# or
route

# add/delete a route to the table
sudo ip route add 10.0.3.0/24 via 10.0.2.1
sudo ip route delete 10.0.3.0/24 via 10.0.2.1

Dns lookup

nslookup google.com
# or
dig google.com

# reverse lookup
dig -x 8.8.8.8

Network statistics

# check open tcp connections
netstat -at

# check listening tcp ports
netstat -lt # only default ports in general

# start python server to verify
python3 -m http.server

File Transfer

SCP (Secure Copy with OpenSSH)

# Copy local file/folder to remote computer
scp file.txt 192.168.100.4:/home/bob/
scp -r files 192.168.100.4:/home/bob/

# copy remote file/folder from remote to local machine
scp 192.168.100.4:/home/bob/remote-file.txt backup/
scp -r 192.168.100.4:/home/bob/remote-files backup/

# copying with a different user name
scp file.txt [email protected]:/home/sally

RSYNC (Efficient sync, only transfers differences)

# Recursively copying directories and preserving user permissions and ownership
# Archive mode, verbose, compress, human-readable
rsync -avzh /local/path/ user@host:/remote/path/
# or
rsync -avzh file2.txt 192.168.100.4:/home/bob

Converting Line Endings

Different systems use different line terminators:

• Windows: CRLF (\r\n)
• Mac (old): CR (\r)
• Linux/Unix: LF (\n)

Check file type: file *.txt

Change line terminator in VIM:

# --- Win/Dos example ---

vim sample-dos-file.txt
:e ++ff=unix
# each line (except the last one) will be 
# terminated with ^M = control char
# the new line is still displayed since LF is 
# used = same as linux

# --- MacOS example --- 

vim sample-macos-file.txt
# only the ^M -s are displayed -- no new lines

Convert between formats:

# Unix --> Dos (modify original file)
unix2dos temp.txt

# Unix --> Dos (create a new file)
unix2dos -n sample-unix-file.txt temp.txt

# Unix --> MacOS
unix2dos -c mac temp.txt

# Dos --> Unix
dos2unix temp.txt

# MacOS --> Unix
dos2unix -c mac temp.txt