Linux essentials

Notes for my Cloud certifications.

Linux essentials

Index

Package management

RPM based

Yum

YUM repos are located in /etc/yum.repos.d/. Unlike APT, YUM has several repo files in the folder. Check out the man pages for YUM and the other package management commands.

command purpose
yum update updates the repos and gives you the option of updating the packages pending updates
yum search httpd searches for that package
yum install $package to install it
yum check-update $package to see if a package needs any updates
yum upgrade upgrade package
yum deplist $package check package’s list of dependencies
yum clean packages will remove dependencies that were left behind but are no longer needed
yum remove $package removes the package
yum list installed list all installed packages

RPM

| command | purpose | | — | — | | rpm -ipv package.rpm | i means install, p means show progress, and v means verbose | | rpm -q nano | query the package for info (true file name) | | rpm -qi nano | query the package for more info | | rpm -e nano | uninstall the package | | rpm -qR nano | uninstall required packages |

Debian based: APT and dpkg

APT

Uses a sources list located in etc/apt/sources.list

command purpose
apt-get update searches the online repos and caches the list of packages for when we do a search via…
apt-cache search $package searches for a package in the APT cache
apt-get install nginx install package
apt-get remove nginx remove package
apt-get remove --purge nginx to get rid of config files and such
apt-get autoremove [$package] to remove unneeded packages.
apt-get upgrade upgrades packages
apt-get -f upgrade Imstalls dependencies that we’re flagged while attempting to install a Debian package
apt-get dist-upgrade upgrades the kernel and distribution packages

dpkg

command purpose
dpkg -i name.deb Installs Debian package
dpkg --get-selections shows all installed packages
dpkg --remove $package_name Removes Debian package
dpkg --purge $package_name Removes dependencies

Command-line basics

Shells are command-line interpreters that accept commands that are then sent to the OS kernel for processing. See list of popular shells I saved as an img on my iPad. You can use any shell installed on the computer by typing its name on the CLI.

Command-line syntax

Programs only run from inside folders indicated in the $PATH variable and not the working directory.

Basic commands

Command Purpose
cd change directory. By itself, takes you to your home directory.
env display current user’s environment variables
halt or init 0 shutdown. Note that init works but is deprecated
ifconfig or ip addr shows NIC configs
netstat status of the network
reboot or init 1 restart. Note that init works but is deprecated
route view routing table
shutdown -H= halt; -P= poweroff; c = cancel pending shutdown; r = reboot
su substitute user or super user. E.g., su josue or su - to become root
top list of running apps/processes; top -h gives usage info
uname print OS name. -n = hostname; -r = kernel’s release; -v = kernel’s version number; -m 32- or 64-bit; -p = processor info; -o = full official name of the OS; -a = all info above
which $program full path of the application
whoami current user

Command history and completion

shell configuration files

Different shell use different configuration files. Make sure you know which files your Linux distro uses. A system without a GUI puts you in the login shell. It’s important to know which shell your in so you know which configuration file will be used for it.

Environment / shell variables

Common environment variables

variable description
LOGNAME username of current user
OLDPWD previous working directory
OSTYPE duh
PATH distro dependent
USER and USERNAME username of current user
HOST and HOSTNAME system hostname
ENV you can type env or set
EUID UID number of current user
HISTFILE full path of file
HISTSIZE size history can grow to

User-defined variables

Globbing

Globbing is the process of using wildcards to expand a search. Globbing stands for global command.

Quoting

Character Description Example
" allows variable interpolation. echo "The path is $PATH"
' does not allow variable interpolation echo 'The path is $PATH'
\ Escapes special chars echo "You owe \$5.00"

Formatting commands

Working with options

Locate, find, whereis

Locate

Searches its file database for files or directories the user has access to. Faster than find but doesn’t allow you to indicate the directory.

locate passwd

Find

Examples:

find . -iname '*keyword*'                   # Match keyword
find / -size +1024                          # greater than size in bytes
find . -mtime -1                            # modified time less than 1 day
find . -atime -1                            # accessed time less than 1 day
find . -ctime -1                            # created time less than 1 day
find . -iname '*.txt' -or -iname "implementations" -prune

whereis

Searches for executables and man page files

whereis cd

Getting CLI help

Linux MAN pages

Info pages

More local documentation

Reading different file formats

file ext program used to read them
.1 - .9 man, info, less
.gz or .bz2 gunzip or bunzip2 to decompress, then less to read
.txt any text editor
.htm, .html any web browser, often less
.odt LibreOffice, OpenOffice.org, any word processor
.pdf .xpdf, Adobe Reader
.tif, .png., .jpg Gimp

The Linux file system

The Linux file system and the file system hierarchy standard (FHS)

directory description
bin executables necessary to run the OS
boot bootloader files to boot Linux
dev devices that send/receive data sequentially (printers/mice); devices that are block-oriented (HDs, flash drives)
etc text-based config files used by the system
home home folders for users
lib -> usr/lib code libraries for programs in the bin or sbin directories
lib64 64-bit libraries
media used by some distros to mount external devices
mnt used by some distros to mount other external devices
opt contains files for programs you can install manually
proc pseudo file system for processes
root root user’s home directory
sbin mgmt and config files
srv where services save their files (e.g., httpd)
sys hardware within system
tmp temporary files created by file system
usr application files
var Linux variable data and log files

Disk file systems

Files and directories

Archives and compression

Archives

Archive (no compression)

Command Notes
tar -cf tarball.tar dir-to-tar creates the tarball from dir-to-tar directory
tar -cf tarball.tar file1 file2 creates the tarball from files indicated

Unarchive (no compression)

Command Notes
tar -tf tarball.tar show contents of tarball w/o unarchiving
tar -xf tarball.tar extract tarball
tar -xvf tarball.tar extract tarball, verbose

Archiving with compression

While not required, it’s best practice to indicate the compression used as part of the file name.

Command Notes
tar -czf tarball.tar.gz dir-to-tar use gzip to compress
tar -cjf tarball.tar.bz2 dir-to-tar use bzip2 to compress

Unarchiving compressed tarballs

Command Notes
tar -xzf tarball.tar.gz extract compressed gzipped archive
tar -xjvf tarball.tar.bz2 extract compressed bzip2 archive, verbose

Compression

Searching for and extracting data from files

Viewing text

Command Purpose
cat display contents of a file
less -M reads file with pagination.

use / to search fwd and ? to search backwards

-M shows lines you’re reading and total, plus percentage

-N shows line numbers on the left

G goes to beginning and shift + G goes to end
head read first 10 lines of a file. -n $num_lines
tail read last 10 lines of a file; -f = follow
find locates file on system

find . -type d find directories
find . -type f find files
find . -iname "file*. allows globbing

Analyzing text

Pipes and regular expressions

You can pipe the output of one command as the input for another command:

grep -i republic plato_republic.txt | less
grep -i republic plato_republic.txt | wc -w

Regular expressions for grep command

Expression Description Example
* 0+ repeats of preceding character string or regex file*
. any single char (grep) .cc
? 0+ of proceeding chars f?le
^ appears at beginning ^.b
$ appears at end ^...$ 3 chars
\b<needle>\b word boundary (must match exactly) \bwww\b
[nnn] one char btw braces [abc]
[^nnn] no chars btw braces [^abc]
[a-z] any single char in range [a-x]
[1-90] any digit between 1-9, and 0 ``

I/O Redirection

Redirecting output

Output is normally displayed on the screen but can be redirected to files or to other commands as input.

tail /var/log/messages > logtemp.txt    # redirect stdout
tail /var/log/messages 1> logtemp.txt   # same as above
tail /var/log/messages >> logtemp.txt   # append

cat bogusfile.txt 2> errors.txt         # redirect stderr
cat bogusfile.txt 2>> errors.txt        # append

command 1> outfile.txt 2> errfile.txt   # redirect to separate files

Turning commands into a script

Basic text editing

nano

vim

Shell scripting

Basic if statement

if [ condition ]
then
    command
fi


# example

if ["1" == "1"]
then
    echo "They are the same"
fi

Basic else statement

if [ condition ]
then
    command
else
    command
fi


# example

if [ "$PWD" == "$HOME" ]
then
    echo "You are home."
else
    echo "You are in $PWD."

Loops

for i in {1..10}
do
    echo "$i"
done

Example bash script 1

#!/bin/bash

# My daily routine script

# If user enters "day", show calendar and date
SHOWDAY=$1

if [ "$1" == "day" ]
then
    # Display the calendar
    cal

    # Display the date/time in UTC
    date -u
fi

# Daily greeting
printf "\nHello there, $LOGNAME.\n\n"

if [ "$PWD" == "$HOME" ]
then
    echo "You are home."
else
    printf "You are in $PWD.\n\n"
fi

# Show us what we have to work on today
DOCUMENTS="/Users/rkumar/Downloads/linux-essentials-practice/text-analysis"

for doc in "$DOCUMENTS"/*.txt
do
    echo "$doc"
done

Example bash script 2

Create the script to set variable values based on args

#! /bin/bash
# list contents of a dir and write the output to a file
# usage: ./findlist.sh $LOCATION $FILENAME

LOCATION=$1
FILENAME=$2

if [ -z "$LOCATION" ]
then
    echo "Please provide location argument"
    exit 0
fi

if [ -z "$FILENAME" ]
then
    echo "Please provide a filename"
    exit 0
fi

ls -lh  $LOCATION > $FILENAME
echo
echo "Script is complete and indexed $LOCATION."
echo
echo "###########################"
echo "Displaying contents of $FILENAME"
echo "###########################"
cat $FILENAME

The Linux operating system

Windows, Mac, and Linux differences

Linux lifecycle management

Understanding computer hardware

Command Purpose
cat /proc/cpuinfo view processor details
free view RAM stats in bytes

-m = show in MB
-g = show in GB
dmidecode show details about motherboard, BIOS, processor, and RAM
lsblk view all block devices (e.g., HDD) attached to system
df view free disk space on HDD

-h = human readable format
du -h $path disk usage; human redable, directories only

-a = show files
top show stats on processor, RAM, and running processes

Where data is stored

The kernel

Linux processes

syslog, klog, dmesg

Networking

Basic networking

Important network tools

Tool Purpose
ping -c $num testing connectivity
dig dig www.pluralsight.com -t A
nslookup nslookup -query=A www.pluralsight.com
netstat list network connections
route current route/netwk settings
host $fqdn test DNS resolution
traceroute trace packet route
ifconfig current network settings
ip addr [show] current IP addr and network settings

Static IP address

Routes

Other commands

Command Description
netstat -a Lists listening & non-listening sockets
netstat -i Stats about the network interfaces
netstat -l Lists listening sockets
netstat -s Summary for each protocol
netstat -r Equivalent to route

Basic security and user types

Root and standard users

Creating users and groups

Managing file permissions and ownership

File and directory permissions

☁  shell-scripting  ll
total 24
drwxr-xr-x  5 rkumar  staff   160B May 21 16:22 ./
drwxr-xr-x  5 rkumar  staff   160B May 21 08:48 ../
-rwxr-xr-x  1 rkumar  staff   546B May 21 16:17 daily.sh*
-rwxr-xr-x  1 rkumar  staff   516B May 21 16:22 indexer.sh*
-rw-r--r--  1 rkumar  staff   1.8K May 21 16:22 test1.txt

Modifying permissions

Special directories and files

Special files and directories, and the sticky bit