Unit 6: Protection & Security

Opening Hook — When an OS Fails to Protect

Learning Outcomes — Bloom's Taxonomy Mapped

Concept Explanation — OS Protection & Security from Scratch

1. Need for Security in Operating Systems

An operating system sits at the heart of every computer. It manages files, memory, processes, and hardware. If the OS is compromised, everything running on it is compromised — your passwords, your banking app, your medical records, your photos. The OS is the last line of defence between your data and an attacker.

Analogy: Think of the OS as the security system of a large apartment building. Protection is the lock on each flat's door (internal access control). Security is the boundary wall, CCTV cameras, and watchman (external threat defence). You need both — a lock without a boundary wall is useless, and a wall without locks is meaningless.

2. Security Vulnerabilities

2.1 Buffer Overflow — The #1 OS Vulnerability

Plain English: Imagine you have a glass that holds 250ml of water. Someone pours 500ml into it. The water overflows and spills onto the table, ruining your books and electronics. A buffer overflow is exactly this — a program writes more data into a memory buffer than it can hold, and the excess data overwrites adjacent memory, potentially overwriting the return address of a function to hijack program execution.

Technical Detail: In C/C++, arrays don't have bounds checking. If a program allocates a 64-byte buffer for user input but the user sends 200 bytes, the extra bytes overwrite the stack — including the return address. An attacker can craft this overflow to redirect execution to their malicious code (shellcode).

C
// VULNERABLE CODE — Buffer Overflow Example
#include <stdio.h>
#include <string.h>

void login() {
    char password[16];  // Buffer: only 16 bytes allocated
    printf("Enter password: ");
    gets(password);        // DANGER! gets() doesn't check buffer size
                           // If user enters 200 chars, it overflows!
    if (strcmp(password, "secret123") == 0) {
        printf("Access granted!\n");
    } else {
        printf("Access denied!\n");
    }
}

int main() {
    login();
    return 0;
}

2.2 Trapdoors & Backdoors

Trapdoor (Backdoor): A secret entry point in a program that bypasses normal authentication. Developers sometimes leave these intentionally for debugging — but if discovered by attackers, they become devastating vulnerabilities.

Example: A developer hardcodes if (username == "debug_admin") grant_access(); in the login system for testing, then forgets to remove it before deployment. Any attacker who discovers this username gets full admin access without a password.

2.3 Cache Poisoning

DNS Cache Poisoning: An attacker corrupts the DNS cache of a resolver, redirecting users to a fake website even though they typed the correct URL. You type www.sbi.co.in but land on a fake SBI login page that steals your credentials.

ARP Cache Poisoning: On a local network, attackers send fake ARP responses, tricking devices into sending traffic through the attacker's machine (enabling Man-in-the-Middle attacks).

3. Authentication

3.1 Password-Based Authentication in Linux

Linux uses a two-file system for password storage — a design that's a masterclass in security:

Why two files? In early Unix, password hashes were stored directly in /etc/passwd (readable by everyone). Attackers could copy the hashes and crack them offline. The /etc/shadow file was introduced to store hashes separately with root-only access — a classic application of the principle of least privilege.

The x in the password field of /etc/passwd indicates that the actual hash is in /etc/shadow. The $6$ prefix in the shadow file indicates SHA-512 hashing (the current default on most Linux distributions).

3.2 Password Hashing — MD5, SHA-256, and Salting

Why hash? Storing passwords in plain text is catastrophic. If a database is breached, all passwords are exposed. Hashing converts a password into a fixed-length, irreversible string. Even if an attacker gets the hash, they can't easily reverse it to get the password.

Salting: A random string added to the password before hashing. Two users with the password "password123" will have different hashes because of different salts. This defeats rainbow table attacks.

Python
import hashlib
import os

# --- MD5 Hashing (WEAK — don't use in production!) ---
password = "secure@123"
md5_hash = hashlib.md5(password.encode()).hexdigest()
print(f"MD5:    {md5_hash}")
# Output: MD5:    a fixed 32-char hex string

# --- SHA-256 Hashing (STRONG — industry standard) ---
sha256_hash = hashlib.sha256(password.encode()).hexdigest()
print(f"SHA-256: {sha256_hash}")
# Output: SHA-256: a fixed 64-char hex string

# --- SHA-256 with SALT (RECOMMENDED) ---
salt = os.urandom(16).hex()  # Random 16-byte salt
salted_password = salt + password
salted_hash = hashlib.sha256(salted_password.encode()).hexdigest()
print(f"Salt:   {salt}")
print(f"Salted: {salted_hash}")
# Store BOTH salt and hash in the database
# To verify: re-hash the entered password with the same salt and compare

3.3 Secure Communication — SSL/TLS Handshake (Simplified)

When you visit https://www.sbi.co.in, your browser and SBI's server perform a TLS handshake to establish an encrypted connection:

4. Application Security — Malware & Program Threats

4.1 Malware Comparison

4.2 Program Threats

Logic Bomb: Malicious code that lies dormant until a specific condition is triggered (date, event, action). Example: A disgruntled IT employee at an Indian bank embeds code that deletes salary records if they're ever terminated from the system.

Privilege Escalation: An attacker gains higher access than intended. Vertical escalation: Normal user → root/admin. Horizontal escalation: User A accesses User B's data without authorisation.

5. Protection Mechanisms

5.1 Goals & Principles of Protection

The fundamental goal of protection is to ensure that each system resource is accessed only by authorised users in authorised ways. The key principle:

5.2 Domain of Protection — Domain Switching

A protection domain defines a set of resources (objects) and the operations (rights) a process can perform on them. Each process executes in a specific domain.

Domain Switching: A process may need to switch domains to perform different tasks. Example: When you run passwd in Linux to change your password, the process temporarily switches from your user domain to the root domain (via the setuid bit) because only root can write to /etc/shadow. Once done, it switches back.

5.3 Access Matrix — The Core Protection Model

The Access Matrix is a theoretical model that defines the rights of each subject (user/process) over each object (file/resource). It's a table where:

• Rows = Subjects (users, processes, domains)
• Columns = Objects (files, devices, memory segments)
• Cells = Access rights (read, write, execute, delete, owner)

5.4 Implementation: Access Control Lists (ACLs) vs Capability Lists

The access matrix is a conceptual model. In practice, it's too large and sparse to store as a full table. Two practical implementations:

Linux uses ACLs. When you run ls -l, you see the ACL for each file: -rwxr-x--- 1 ravi staff 4096 Jan 15 file.txt. This says: Owner (ravi) has rwx, group (staff) has r-x, others have no access.

6. System & Network Threats

6.1 Denial of Service (DoS) & Distributed DoS (DDoS)

DoS Attack: Flooding a server with so many requests that it can't serve legitimate users. Like 10,000 people crowding a small shop — real customers can't get in.

DDoS Attack: Same attack but launched from thousands of compromised machines (botnet) simultaneously. Much harder to defend against because traffic comes from many different IPs.

6.2 Man-in-the-Middle (MITM) Attack

The attacker secretly intercepts and potentially alters communication between two parties who believe they're talking directly to each other.

Indian Example: You're at a café in Connaught Place, Delhi, using free public Wi-Fi. An attacker on the same network performs ARP cache poisoning, routing all your traffic through their laptop. When you log into your bank, they capture your credentials. This is why HTTPS is critical — even if traffic is intercepted, it's encrypted.

6.3 SQL Injection

An attacker inserts malicious SQL code into input fields to manipulate the backend database. If a login form doesn't sanitize input:

SQL
-- Normal login query:
SELECT * FROM users WHERE username='ravi' AND password='secure123';

-- Attacker enters username: ' OR '1'='1' --
-- Resulting query:
SELECT * FROM users WHERE username='' OR '1'='1' --' AND password='anything';
-- This always returns TRUE → attacker gains access without a password!

6.4 Port Scanning

Attackers use tools like Nmap to scan a server's ports to find open services. An open port 22 (SSH) with a weak password is an invitation for brute-force attacks. An open port 3306 (MySQL) exposed to the internet is a database breach waiting to happen.

7. Indian Cybersecurity Legal Framework

Learn by Doing — 3-Tier Lab Structure

Bash
# Create a test directory
mkdir ~/security_lab
cd ~/security_lab

# Create test files
echo "Student marks: Ravi=85, Priya=92" > marks.txt
echo "Top Secret: Admin Password List" > secret.txt
echo "Public announcement: College closed tomorrow" > notice.txt

Bash
ls -la

# Output will look like:
# -rw-rw-r-- 1 ravi ravi  38 Jan 15 10:00 marks.txt
# -rw-rw-r-- 1 ravi ravi  40 Jan 15 10:00 secret.txt
# -rw-rw-r-- 1 ravi ravi  48 Jan 15 10:00 notice.txt

Bash
# Make secret.txt readable ONLY by owner (no group, no others)
chmod 600 secret.txt
ls -l secret.txt
# -rw------- 1 ravi ravi  40 Jan 15 10:00 secret.txt
# ↑ Only owner can read/write. Nobody else can even see contents.

# Make notice.txt readable by everyone, writable only by owner
chmod 644 notice.txt
ls -l notice.txt
# -rw-r--r-- 1 ravi ravi  48 Jan 15 10:00 notice.txt

# Make marks.txt readable/writable by owner and group, no access for others
chmod 660 marks.txt
ls -l marks.txt
# -rw-rw---- 1 ravi ravi  38 Jan 15 10:00 marks.txt

# Symbolic mode: remove ALL permissions for others on all files
chmod o-rwx *.txt

Bash
# Change owner of marks.txt to root (requires sudo)
sudo chown root:root marks.txt
ls -l marks.txt
# -rw-rw---- 1 root root  38 Jan 15 10:00 marks.txt
# Now even ravi can't read this file! (ravi is not root and not in root group)

# Try to read it as normal user
cat marks.txt
# Permission denied! ← Access control in action

Bash
# The passwd command lets normal users change their password
# But passwords are stored in /etc/shadow (owned by root!)
# How? The setuid bit!

ls -l /usr/bin/passwd
# -rwsr-xr-x 1 root root 63960 Jan 15 /usr/bin/passwd
#    ↑ 's' in owner execute = setuid bit
# When ANY user runs passwd, it executes with ROOT privileges
# This is "domain switching" — the process temporarily enters root's domain

# Find all setuid programs on your system
find / -perm -4000 -type f 2>/dev/null
# Lists all programs that run with elevated privileges
# Security tip: Minimize setuid programs — each is a potential attack vector

Bash
# Try to read as normal user
cat /etc/shadow
# Permission denied! ← Correct — only root can read password hashes

# Read as root
sudo cat /etc/shadow | head -5
# root:$6$xyz...:19500:0:99999:7:::
# ravi:$6$abc...:19501:0:99999:7:::
# ↑ $6$ = SHA-512 hash. The long string after second $ is the salt+hash.

# Check permissions on both password files
ls -l /etc/passwd /etc/shadow
# -rw-r--r-- 1 root root  2345 /etc/passwd  ← readable by all
# -rw-r----- 1 root shadow 1580 /etc/shadow  ← readable only by root

Industry Spotlight — A Day in the Life

Earn With It — Freelance & Income Roadmap

⏱️ Time to First Earning: 3–4 weeks (if you complete all 3 labs and create an Upwork/BugCrowd profile)

MCQ Assessment Bank — 30 Questions (Bloom's Mapped)

Remember / Identify (Q1–Q5)

Understand / Explain (Q6–Q10)

Apply / Demonstrate (Q11–Q15)

Analyze / Compare (Q16–Q20)

Evaluate / Judge (Q21–Q25)

Create / Design (Q26–Q30)

Short Answer Questions (5 Questions)

Case Studies

Chapter Summary

Earning Checkpoint — What You Can Do Now

✅ Unit 6 complete. MCQs: 30. Ready for Unit 7: Memory Management!

[QR: Link to EduArtha video tutorial — OS Protection & Security]