Unit 3: Linux Operating System & File System Management

Linux is a free, open-source, UNIX-like operating system kernel created by Linus Torvalds in 1991. Unlike Windows (owned by Microsoft) or macOS (owned by Apple), Linux's source code is publicly available — anyone can read it, modify it, distribute it. What we commonly call "Linux" is actually GNU/Linux — the Linux kernel combined with the GNU project's user-space tools (bash, coreutils, gcc) and a distribution's package manager, desktop environment, and configurations.

Think of Linux as a restaurant recipe that's publicly shared. Windows is like a McDonald's franchise — you buy the meal, but you can't see or modify the recipe. Linux is like an open recipe on GitHub — anyone can cook it, modify the spices (customize the OS), and share their version. That's why there are hundreds of Linux "flavors" (distributions): Ubuntu, Fedora, Arch, Kali — each is a chef's take on the same base recipe.

Linux Architecture Stack
┌────────────────────────────────────────────────────────┐
│                      USER (You)                         │
├────────────────────────────────────────────────────────┤
│               APPLICATION LAYER                         │
│   Firefox │ VS Code │ Python │ Docker │ Node.js         │
├────────────────────────────────────────────────────────┤
│               SHELL (bash, zsh, fish)                   │
│   Interprets your commands → translates to system calls │
├────────────────────────────────────────────────────────┤
│               GNU UTILITIES                             │
│   ls, cp, mv, grep, awk, sed, chmod, ps, kill           │
├────────────────────────────────────────────────────────┤
│               LINUX KERNEL                              │
│   ┌────────────────────────────────────────────────┐   │
│   │ Process Scheduler │ Memory Manager │ VFS       │   │
│   │ Device Drivers    │ Network Stack  │ Security  │   │
│   └────────────────────────────────────────────────┘   │
├────────────────────────────────────────────────────────┤
│                    HARDWARE                              │
│       CPU │ RAM │ SSD/HDD │ GPU │ NIC │ USB             │
└────────────────────────────────────────────────────────┘

96.3% of the top 1 million web servers run Linux. All 500 of the world's fastest supercomputers run Linux. Every AWS EC2 instance defaults to Amazon Linux (based on RHEL/CentOS). IRCTC runs on Linux servers handling 25 million bookings daily. Jio's telecom infrastructure, AIIMS hospital management systems, every IIT server — all Linux. Google runs over 4 million Linux servers.

"Linux is hard and only for hackers." No. Ubuntu and Linux Mint are as easy as Windows for daily use. Android — the OS on your phone — is Linux. Chrome OS (on Chromebooks) is Linux. Smart TVs, WiFi routers, and ATMs all run Linux. You've been using Linux every day without knowing it.

A shell is a program that interprets your text commands and translates them into system calls that the kernel understands. It's the bridge between you (typing ls -la) and the kernel (performing a directory listing). The shell is NOT the black window — that's the terminal.

If the kernel is a restaurant kitchen, the shell is the waiter who takes your order (command), translates it into kitchen language (system calls), and brings back the result (output). The terminal is the dining table where you sit and interact with the waiter. The console is the specific physical restaurant location (hardware terminal).

"Terminal and shell are the same thing." No. You can change your shell without changing your terminal, and vice versa. Running chsh -s /bin/zsh switches your shell to zsh, but you're still using the same GNOME Terminal window. Think: terminal = TV screen, shell = TV channel. Same screen, different channel.

Every file and directory in Linux has three types of permissions for three categories of users. Permissions determine who can read, write, or execute a file. This is Linux's core security model — it's why Linux servers can safely host millions of users without them accessing each other's data.

Permission Structure
    -rwxr-xr--  1  rahul  developers  4096  Jun 15  project.py
    │└┬┘└┬┘└┬┘     └──┬──  └────┬─────
    │ │   │  │         │        │
    │ │   │  │         │        └── Group (developers)
    │ │   │  │         └─────────── Owner (rahul)
    │ │   │  └── Others permissions: r-- (read only)
    │ │   └───── Group permissions:  r-x (read + execute)
    │ └───────── Owner permissions:  rwx (read + write + execute)
    └──────────── File type: - (file), d (directory), l (link)

  Permission    Meaning for FILE          Meaning for DIRECTORY
  ──────────    ─────────────────         ──────────────────────
  r (read)      Can view contents         Can list contents (ls)
  w (write)     Can modify contents       Can create/delete files inside
  x (execute)   Can run as program        Can cd into directory

chmod Examples
# Symbolic notation
chmod u+x script.sh       # Add execute permission for owner (u=user)
chmod g-w config.yml      # Remove write permission from group
chmod o-rwx secret.key    # Remove all permissions from others
chmod a+r readme.md       # Add read for all (a=all: user+group+others)

# Octal notation (most common in industry)
chmod 755 deploy.sh       # rwxr-xr-x (owner: full, group: read+exec, others: read+exec)
chmod 644 index.html      # rw-r--r-- (owner: read+write, group/others: read only)
chmod 600 id_rsa          # rw------- (owner only — SSH private key!)
chmod 777 test.sh        # rwxrwxrwx ⚠️ NEVER use 777 on production!
chmod -R 755 /var/www/   # Apply recursively to all files in directory

Ownership Commands
chown rahul file.txt            # Change file owner to rahul
chown rahul:developers file.txt # Change owner AND group
chown -R www-data:www-data /var/www/ # Recursive: all files in web directory
chgrp engineers project/        # Change group only

umask
# umask defines which permissions are REMOVED from new files
# Default file permission: 666 (rw-rw-rw-)
# Default dir permission:  777 (rwxrwxrwx)
# umask 022 → new files: 666 - 022 = 644 (rw-r--r--)
#            → new dirs:  777 - 022 = 755 (rwxr-xr-x)
umask           # Show current umask
umask 027       # Set: files=640, dirs=750 (restrict others completely)

SSH key security: Your private key (~/.ssh/id_rsa) MUST be chmod 600. If it's world-readable, SSH refuses to use it — by design. Web servers: Nginx/Apache web files are typically chmod 644 (readable by everyone, writable only by owner) with owner www-data. Flipkart's incident: A misconfigured chmod 777 on a config file exposed database credentials in 2019.

"Just use chmod 777 to fix permission errors." NEVER. 777 means everyone can read, write, and execute — it's a massive security hole. It's the Linux equivalent of leaving your house door open with a sign that says "Take anything." Use the minimum permissions needed: 644 for files, 755 for directories and scripts.

A process is a running instance of a program. When you type python app.py, the kernel creates a process with a unique ID (PID), allocates memory, and schedules CPU time. Linux can run thousands of processes simultaneously. Understanding processes is essential for debugging ("Why is my server slow?") and administration ("Kill the stuck process").

Linux commands are designed to be small, focused tools that do one thing well. The magic happens when you pipe (|) the output of one command as the input to another, creating powerful command chains. This is the Unix philosophy in action — and it's why experienced engineers can do in one line what takes 50 lines of code.

Think of an assembly line in a Maruti Suzuki factory. Station 1 builds the chassis. Station 2 adds the engine. Station 3 paints. Each station does ONE thing well, and the output of one flows into the next. The pipe (|) is the conveyor belt connecting stations.

Piping in Action
# Find the 10 largest files in /var/log
du -ah /var/log | sort -rh | head -10

# Count how many Python processes are running
ps aux | grep python | wc -l

# Find all unique IP addresses in nginx access log
awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -20

# Find all .py files containing "import flask"
grep -r "import flask" /home/rahul/projects/ | grep ".py:"

# Monitor real-time HTTP 500 errors
tail -f /var/log/nginx/access.log | grep " 500 "

# Save process list to file, including errors
ps aux > processes.txt 2>&1

At Flipkart during Big Billion Days: SREs chain commands to find which API endpoints are failing: tail -f access.log | grep "500" | awk '{print $7}' | sort | uniq -c | sort -rn. This one-liner shows the top failing URLs in real-time. Writing this as a Python script would take 20+ lines and be slower.

A Virtual Machine (VM) is a software emulation of a complete computer. A hypervisor is the software that creates and manages VMs by allocating physical hardware resources (CPU, RAM, disk) to each VM. Each VM thinks it has its own dedicated hardware, but it's actually sharing the host machine's resources.

A VM is like a PG/hostel room inside a building. The building (physical server) has 10 rooms (VMs). Each room has its own lock (isolation), its own furniture arrangement (OS), and its own residents (applications). The building owner (hypervisor) decides how much electricity (CPU) and water (RAM) each room gets.

Hypervisor Architecture
TYPE 1 (Bare-Metal)                 TYPE 2 (Hosted)
┌───────┐ ┌───────┐ ┌───────┐      ┌───────┐ ┌───────┐
│ VM 1  │ │ VM 2  │ │ VM 3  │      │ VM 1  │ │ VM 2  │
│Ubuntu │ │Windows│ │CentOS │      │Ubuntu │ │Kali   │
├───────┤ ├───────┤ ├───────┤      ├───────┴─┴───────┤
├───────────────────────────┤      │  VirtualBox/VMware│
│    TYPE 1 HYPERVISOR      │      ├───────────────────┤
│    (ESXi / KVM / Xen)     │      │    HOST OS         │
├───────────────────────────┤      │    (Windows 11)    │
│    PHYSICAL HARDWARE      │      ├───────────────────┤
└───────────────────────────┘      │  PHYSICAL HARDWARE │
                                   └───────────────────┘

AWS EC2 — Every EC2 instance is a VM running on AWS's Nitro hypervisor (KVM-based). When you launch an EC2 instance, AWS carves out a VM on one of their physical servers. IIT/NIT labs — one powerful server runs 50 VMs (one per student) instead of maintaining 50 physical PCs. TCS/Infosys development — developers test on different OS versions using VMs without needing multiple laptops.

"VMs and containers (Docker) are the same." No. VMs virtualize the entire hardware — each VM has its own full OS kernel (heavy, GBs of overhead). Containers share the host OS kernel and only isolate the application layer (lightweight, MBs of overhead). VMs take minutes to boot; containers start in seconds. Docker uses Linux namespaces and cgroups — not a hypervisor.

An inode (index node) is a data structure that stores all metadata about a file EXCEPT its name and contents. Every file on a Linux system has exactly one inode. The inode contains: file size, owner (UID), group (GID), permissions, timestamps (created, modified, accessed), number of hard links, and pointers to the actual data blocks on disk.

An inode is like a library catalog card. The card (inode) tells you the book's author (owner), size (pages), location on the shelf (data block pointers), and borrowing rules (permissions). The card itself doesn't contain the book's text (data) or the title displayed on the spine (filename). The directory is like the shelf label that maps "Harry Potter" (filename) → catalog card #4567 (inode number).

Inode Structure
Directory Entry                 Inode #4567                   Data Blocks
┌───────────────┐              ┌──────────────────┐          ┌────────────┐
│ "project.py"  │──────────────│ Inode #4567      │          │ Block 1001 │
│ inode: 4567   │  maps to →   │ Size: 2048 bytes │ points → │ import os  │
└───────────────┘              │ Owner: rahul     │          │ def main():│
                               │ Group: devs      │          ├────────────┤
                               │ Perms: rw-r--r-- │          │ Block 1002 │
                               │ Links: 1         │          │   print()  │
                               │ Created: Jun 10  │          │   return 0 │
                               │ Modified: Jun 15 │          └────────────┘
                               │ Blocks: 1001,1002│
                               └──────────────────┘

View inode information
ls -i project.py             # Show inode number
4567 project.py

stat project.py              # Show all inode metadata
  File: project.py
  Size: 2048       Blocks: 8    IO Block: 4096  regular file
Device: 802h/2050d  Inode: 4567    Links: 1
Access: (0644/-rw-r--r--)  Uid: (1000/rahul)  Gid: (1000/devs)
Access: 2025-06-15 10:30:00
Modify: 2025-06-15 10:28:00
Change: 2025-06-15 10:28:00

"Disk is full but df shows space available" — This classic problem happens when you run out of inodes (too many small files) even though data blocks are available. df -i shows inode usage. Flipkart's logging system once filled inodes by creating millions of tiny log files — the fix was log rotation and inode monitoring.

"The filename is stored inside the file." No. The filename is stored in the directory entry, not in the inode or the data blocks. That's why you can have two different filenames pointing to the same inode (hard links). Renaming a file (mv old.txt new.txt) only changes the directory entry — the inode and data are untouched, which is why renaming is instant regardless of file size.

Mounting is the process of making a file system (on a disk, partition, USB drive, or network share) accessible at a specific directory in the Linux file tree. Unlike Windows (which assigns drive letters: C:, D:, E:), Linux attaches all storage devices into a single unified directory tree under /.

Mounting is like plugging a USB drive into a specific USB port on a hub. The hub is the directory tree, and each port is a mount point. When you mount a drive to /mnt/backup, all files on that drive appear inside /mnt/backup/. Unmounting is safely "ejecting" the drive.

Mount / Unmount
# Mount a USB drive (device /dev/sdb1) to /mnt/usb/
sudo mount /dev/sdb1 /mnt/usb/

# Mount with specific file system type
sudo mount -t ntfs /dev/sdb1 /mnt/windows_drive/

# Unmount safely (ensures all writes are flushed to disk)
sudo umount /mnt/usb/

# View all currently mounted filesystems
mount | grep "^/dev"
df -hT           # Shows filesystem type (-T) and usage

# Permanent mount (survives reboot) — edit /etc/fstab
# /dev/sda2  /home  ext4  defaults  0  2

IRCTC server setup: A typical production server has: / (root, 50 GB, ext4), /home (100 GB, ext4), /var (200 GB, XFS — for logs and databases), /boot (1 GB, ext4), swap (16 GB). Separating partitions means if /var/log fills up, it doesn't crash the root filesystem. GPT with UEFI is mandatory for disks > 2 TB — which is every modern server.

A journaling file system maintains a special log (journal) of changes that are about to be made BEFORE actually writing them to disk. If a power failure or crash interrupts a write operation, the file system can replay or discard the incomplete operation from the journal during the next boot — preventing data corruption.

Journaling is like a bank transaction log. Before transferring ₹10,000 from Account A to Account B, the bank writes in the log: "Step 1: Debit A ₹10,000. Step 2: Credit B ₹10,000." If the system crashes after Step 1 but before Step 2, the bank reads the log on restart and either completes the transfer or rolls back Step 1. Without a journal, Account A would lose ₹10,000 and Account B would never receive it.

Journaling Process
WITHOUT JOURNALING (ext2, FAT32):
  1. Start writing data to disk
  2. ⚡ POWER FAILURE mid-write!
  3. Reboot → file system is inconsistent → run fsck (slow, may lose data)
  4. Full disk scan: 30 minutes for 1 TB drive 😱

WITH JOURNALING (ext3, ext4, NTFS, XFS):
  1. Write intent to JOURNAL: "About to modify inode 4567, blocks 1001-1002"
  2. Perform actual disk write
  3. Mark journal entry as COMPLETE
  4. ⚡ POWER FAILURE at step 2?
  5. Reboot → read journal → replay or discard incomplete operations
  6. Recovery time: 2-5 SECONDS regardless of disk size ✅

Why this matters: Imagine IRCTC's database is writing a ticket booking when power fails. Without journaling, the ticket could be half-written — money debited but ticket not confirmed. With journaling (ext4 ordered mode), either the full operation completes or it's completely rolled back. Every production database server uses a journaling file system. No exceptions.