Nine spreads on what the kernel does: processes, threads, scheduling, memory, file systems, I/O, and how programs ask it for things. Commands for Linux, with notes for Windows.
"The kernel schedules processes onto cores, gives each its own virtual address space, and mediates files and devices through system calls."
Use the tabs or the ← → keys.
open, read, write, fork, execve, mmap, socketread/write interface$ strace -c ls /etc > /dev/null # count the syscalls a command makes $ strace -e openat,read cat /etc/hostname $ uname -a ; cat /proc/version $ ls /proc/$$/ # live kernel view of this shell (a pseudo-filesystem) $ lsmod | head # loaded kernel modules (drivers) PS> Get-Process | Sort CPU -Desc | Select -First 5 # Windows view
// the smallest "program": write(1, "hi\n", 3) then exit
#include <unistd.h>
int main(void) { write(1, "hi\n", 3); return 0; } // gcc hi.c -o hi && strace ./hi
"Every file, socket and process operation is a system call across the user-kernel boundary."
systemd is PID 1$ ps -eo pid,ppid,stat,%cpu,%mem,cmd --sort=-%cpu | head $ pstree -p $$ # the process tree under your shell $ cat /proc/$$/status | grep -E 'State|VmRSS|Threads' $ ls -l /proc/$$/fd # open file descriptors (0 stdin, 1 stdout, 2 stderr) $ kill -STOP $PID ; kill -CONT $PID # pause / resume
import os
pid = os.fork()
if pid == 0: # child
os.execvp("echo", ["echo", "hello from the child"])
else: # parent
os.waitpid(pid, 0) # reap it, so no zombie
print("child finished")
"The shell forks, the child execs the program, and the parent waits. Each process has its own address space."
import threading
counter = 0
lock = threading.Lock()
def work():
global counter
for _ in range(100_000):
with lock: # remove the lock and the total will usually be wrong
counter += 1
ts = [threading.Thread(target=work) for _ in range(4)]
[t.start() for t in ts]; [t.join() for t in ts]
print(counter) # 400000
| Model | Good for |
|---|---|
| processes | isolation, CPU-bound work in Python (bypasses the GIL) |
| threads | shared state, I/O-bound work |
| async / event loop | thousands of idle connections on one thread |
$ ps -eLf | head ; top -H -p $PID # threads of a process $ nproc ; taskset -c 0,1 ./job # pin to specific cores
"Protect the critical section with a lock, keep lock order consistent, and prefer message passing where I can."
# compare average waiting time for FCFS vs SJF, bursts in ms (all arrive at 0)
bursts = {"A": 8, "B": 2, "C": 4}
def avg_wait(order):
t, waits = 0, []
for p in order:
waits.append(t); t += bursts[p]
return sum(waits) / len(waits)
print("FCFS A,B,C :", avg_wait(["A","B","C"])) # 6.0
print("SJF B,C,A :", avg_wait(sorted(bursts, key=bursts.get))) # 2.67
$ nice -n 10 ./batch-job # be polite: lower priority $ renice -n 5 -p $PID $ chrt -p $PID # scheduling policy of a process $ uptime ; cat /proc/loadavg # load average: runnable + waiting tasks (compare to core count) $ vmstat 1 5 ; pidstat 1 3 # run queue (r), context switches (cs) $ perf sched record -- sleep 5 && perf sched latency
"Load average is above the core count, so tasks are queueing, and I'll check whether it's CPU or I/O wait."
$ getconf PAGESIZE $ free -h ; cat /proc/meminfo | head -8 $ vmstat 1 5 # si/so = swap in/out; non-zero for long = trouble $ cat /proc/$$/maps | head # this process's address-space map $ /usr/bin/time -v ls 2>&1 | grep -E 'faults|Maximum resident' $ sudo dmesg | grep -i 'out of memory' # OOM killer events $ swapon --show ; sysctl vm.swappiness
| Replacement | Idea |
|---|---|
| FIFO | evict the oldest (can behave oddly: Bélády's anomaly) |
| LRU | evict the least recently used (good, approximated in practice) |
| Clock | cheap LRU approximation with a reference bit |
"The process was OOM-killed. RSS climbed with no swap, so it's a leak or a limit that's too tight."
$ ls -li notes.txt # -i shows the inode number $ stat notes.txt # inode, size, blocks, links, times $ ln notes.txt hard.txt ; ln -s notes.txt soft.txt $ ls -li notes.txt hard.txt soft.txt # hard: same inode · soft: its own inode, holds a path $ df -h ; df -i # space used vs inodes used $ lsblk -f ; findmnt # filesystems and where they're mounted $ du -xh --max-depth=1 / 2>/dev/null | sort -h | tail $ sudo debugfs -R "stat <1204>" /dev/sda1 # peek inside ext4 (read-only is safest) $ lsof +L1 # deleted files still held open (space not freed) $ mount | grep -E 'ext4|xfs|btrfs|tmpfs'
| File system | Known for |
|---|---|
| ext4 | Linux default, journaling, mature |
| XFS | big files, parallel I/O |
| btrfs / ZFS | snapshots, checksums, copy-on-write |
| NTFS / APFS | Windows / macOS defaults |
| tmpfs / procfs | RAM-backed / kernel info as files |
"It's out of inodes, not space, so I'll find the directory with millions of tiny files."
/dev/sda (random access blocks) vs /dev/tty (stream)$ cat /proc/interrupts | head ; cat /proc/devices | head $ iostat -xz 1 3 # %util, await per disk (sysstat package) $ iotop -oPa # which process is doing I/O $ dd if=/dev/zero of=/tmp/t bs=1M count=512 oflag=direct status=progress $ sync ; echo 3 | sudo tee /proc/sys/vm/drop_caches # for benchmarking only $ udevadm info /dev/sda ; dmesg | tail $ echo hello | tee /dev/null ; cat /dev/urandom | head -c 16 | xxd
import select, socket
srv = socket.socket(); srv.bind(("127.0.0.1", 9000)); srv.listen(); srv.setblocking(False)
ep = select.epoll(); ep.register(srv.fileno(), select.EPOLLIN)
# loop: ep.poll() → returns only the sockets that are ready; one thread, many clients
"High iowait with a saturated disk: I'll find the heavy reader with iotop and check the page cache hit rate."
passwd, sudo)CAP_NET_BIND_SERVICE to bind port 80 without being root$ id ; groups ; ls -l /usr/bin/passwd # note the 's' (setuid) bit $ find / -perm -4000 -type f 2>/dev/null # audit setuid programs $ getcap -r / 2>/dev/null ; cat /proc/sys/kernel/randomize_va_space $ lsns ; ls -l /proc/$$/ns # namespaces in use $ sudo unshare --pid --fork --mount-proc bash # a shell in a new PID namespace: ps shows PID 1! $ systemd-cgls ; cat /sys/fs/cgroup/memory.max $ getenforce ; aa-status # SELinux / AppArmor state $ sudo setcap cap_net_bind_service=+ep ./myserver
"Run the service as a non-root user with just the needed capability, in its own namespaces with cgroup limits."
$ systemd-analyze ; systemd-analyze blame | head # what slowed the boot $ journalctl -b -p warning # this boot's warnings and errors $ dmesg -T | head -50 ; cat /proc/cmdline # kernel messages and boot parameters $ top -bn1 | head -15 ; mpstat -P ALL 1 3 ; sar -u 1 3 $ sudo perf top ; sudo strace -fp $PID -c # deeper, when you know the suspect $ sudo lsof -p $PID | head ; ss -tulpn PS> Get-Counter '\Processor(_Total)\% Processor Time','\Memory\Available MBytes' -SampleInterval 2 -MaxSamples 3 PS> msinfo32 ; resmon ; perfmon # Windows tools
"I went resource by resource (CPU, memory, disk, network), found disk saturation, and traced the process doing it."