Linux Commands¶

Run a Quick Web-Server in Current Directory¶

python -m SimpleHTTPServer 8080

Color Full Logs¶

tail -f foo.log | ccze
dmesg | ccze

System Information¶

mount | column -t 
free -tm
vmstat
dstat
dfc
procinfo
procinfo -a
cat /proc/cpuinfo
cat /proc/meminfo
egrep --color 'Mem|Cache|Swap' /proc/meminfo
lshw
lspci
dmidecode
lsusb
lsdev
arecord -l  #all soundcards and digital audio devices
record -L   #display all PCMs defined
cat /proc/asound/cards

Re-read Partition Table¶

partprobe
kpartx

Open Ports & their Processes¶

lsof -i
netstat -lptu
netstat -tulpn
netstat -an | egrep 'Proto|LISTEN'

SSH Tunneling¶

ssh -L 3122:localhost:3120 [email protected]

Processes¶

Checking Processes¶

top
htop
atop
nmon
pstree
mpstat
mpstat -P ALL
sar
sar -u 2 5
sar -o output.file 12 8 >/dev/null 2>&1 &
ps auxww
ps aux
ps -e -o pcpu -o pid -o user -o args
ps -ef ps -eo pid,cmd,etime ps axo pid,cmd,etime
ps -eF
ps -ely
ps axu
ps auxww | grep pid
ps auxww | grep mysqld
ps auxww | grep --color mysqld
ps -eo pid,tid,class,rtprio,ni,pri,psr,pcpu,stat,wchan:14,comm
ps axo stat,euid,ruid,tty,tpgid,sess,pgrp,ppid,pid,pcpu,comm
ps -eopid,tt,user,fname,tmout,f,wchan
ps axo pid,command,args #Display only PID, COMMAND and its args
kill -9 PID

Who is monopolizing or eating the CPUs¶

ps -eo pmem,pcpu,rss,vsize,args | sort -k 1 -r | less
ps -eo pcpu,pid,user,args | sort -k 1 -r | head -10
ps -eo pcpu,pid,user,args | sort -r -k1 | less
iostat
iostat -xtc 5 3
iostat -d -x 2 5
iotop --only
% iostat -dx /dev/sda 5 #Checking load of HDD

How to kill all processes run by a user on Linux¶

$ ps -ef | grep xmodulo | awk '{ print $2 }' | sudo xargs kill -9
$ pgrep -u xmodulo | sudo xargs kill -9
$ sudo pkill -u xmodulo
$ sudo killall -u xmodulo

Packages¶

apt-cache search XML::Simple
apt-get --reinstall install <package>
apt-file search bin/convert  #apt-get install apt-file && apt-file update
dpkg –get-selections | less  #Find all installed packages
aptitude why packagename     #Find out why a packages is installed and what depends on it
dpkg -L packagename          #Find out where the package stores all of its files
dpkg-deb -c foo.deb
apt-file list packageName
dpkg -L packageName          #Installed package

Backing up all installed packages list and installing again¶

$ dpkg --get-selections > /backup/installed-software.log (backup all list)
$ dpkg --set-selections < /backup/installed-software.log (set from that list)
# dselect (to install)

File Management Commands¶

cat – Displays file contents
chfn – Change user info for finger, email, etc
chgrp – Changes the group of files
chmod – Change acces file system permissions for files / dir’s
chfn – Change user info for finger, email, etc
chgrp – Changes the group of files
chown – Changes the owner of the file(s) or dir(s)
chsh – Changes the login shell
cksum – Computes checksums
cp – Copies files
csplit – Splits files into pieces
dd – (Dumb Dump) copies files at disk level
file – Identifies file type
head – Shows the first few lines of a file
hex dump – Displays hex files
less – Displays file contents a screen full at a time
ln – Create file aliases (links, sym links, hard links etc)
ls – Displays the dir’s contents
merge – Merges files together
mkdir – Creates directories
mv – Moves files
more – Displays file contents a screen full at a time (the same as less)
newgrp – Changes the current group
od – Display files in octal format
pwd – Prints (shows) the working directory
rmdir – Remove (delete) Directory (Normally they have to be empty)
rm – Remove (delete) files
shred – Securely deletes files
split – Splits files evenly
tac – Prints (shows) the lines of files in the reverse order
tail – Shows the last few lines of a file (by default the last 10)
tailf – Follows the growth of a log file (shows you the contents as the system writes to the log file)
touch – Updates the file time stamps, if the file does not already exist it will be created
wc – Counts lines, words and characters.

User Groups¶

groups                 #To find which groups the user belong:
cut -d: -f1 /etc/group # Entire list

# Programs used:

1) members 
2) libuser //executes as lid.

# List all members of a group using /etc/group file

$ grep 'grpup-name-here' /etc/group
$ grep 'ftponly' /etc/group
$ grep -i color 'ftponly' /etc/group

usermod example (Add a existing user to existing group)¶

Add existing user tony to ftp supplementary/secondary group with the usermod command using the -a option i.e. add the user to the supplemental group(s). Use only with -G option

# usermod -a -G ftp tony

In this example, change tony user's primary group to www, enter:

# usermod -g www tony

useradd example - Add a new user to primary group¶

# useradd -g developers tony
# id tony

If you do not see any output then you need to add group developers using the groupadd command:

# groupadd developers

Next, add a user called vivek to group developers:

# useradd -G developers vivek

 /etc/passwd           # Contains one line for each user account.
 /etc/shadow           # Contains the password information in encrypted format for the system's accounts and optional account aging information.
 /etc/group            # Defines the groups on the system.
 /etc/default/useradd  # This file contains a value for the default group, if none is specified by the useradd command.
 /etc/login.defs       # This file defines the site-specific configuration for the shadow password suite stored in /etc/shadow file.

File Viewer¶

ncdu
hier
agedu

Listing Directories Only¶

find . -type d -exec ls -d {} \;
ls -d .*"/" *"/"
find . -type d

'du' - Finding the size of a directory¶

$ du /home/david 
$ du -h      #list directories, added h for human-readable format
$ du -ah     #list all files
$ du -ch     #for grand total at the end
$ du -ch | grep total
$ du -sh     #summary of entire current directory
$ du -Sh     #summary of current directory files excluding folders
$ du --exculde=jpg #List all files/directories excluding given pattern

Finding Largest File¶

du -a /var | sort -n -r | head -n 10
du -hsx * | sort -rh | head -10 #Human readable output
for i in G M K; do du -ah | grep [0-9]$i | sort -nr -k 1; done | head -n 11 #Also useful:

'df' - Finding Disk Free Space / Disk Usage¶

$ df
$ df -h
$ df -h | grep /dev/sda1
$ df -H 
$ df -H | grep /dev/sda1
$ df -TH                  #displays type

On what drive does file exist?¶

df -P file/goes/here | tail -1 | cut -d' ' -f 1

Finding Files¶

grep: Recursively Search All Files For A String¶

cd /path/to/dir
grep -r "word" .
grep -r "string" .
grep -ri "word" . #Ignore case distinction

find: Recursively Search All Files For A String¶

cd /path/to/dir
find . -type f -exec grep -l "word" {} +
find . -type f -exec grep -l "seting" {} +
find . -type f -exec grep -l "foo" {} +
find /path/to/dir -type f | xargs grep -l "foo"   # Older UNIX version should use xargs to speed up things
 find /path/to/dir -type f -print0 | xargs -0 grep # good idea to pass -print0 for spaces/metacharacters

Copy or Replicate File Permissions: (using another file as reference)¶

chmod --reference <reference-file> <target-file>

Get an Ordered List of Sub-Directory Sizes¶

du -sk ./* | sort -n | awk 'BEGIN{ pref[1]="K"; pref[2]="M"; pref[3]="G";} { total = total + $1; x = $1; y = 1; while( x > 1024 ) { x = (x + 1023)/1024; y++; } printf("%g%s\t%s\n",int(x*10)/10,pref[y],$2); } END { y = 1; while( total > 1024 ) { total = (total + 1023)/1024; y++; } printf("Total: %g%s\n",int(total*10)/10,pref[y]); }'

Count Number of Files n a Directory¶

$ ls -1 targetdir | wc -l
454

If you want to exclude subdirectories, you need a heavier duty tool than ls.

$ find targetdir -maxdepth 1 -type f | wc -l  #remove '-maxdepth' to include sub-directories
                                              #'type -f' indicates regular files 
                                              #meaning no symbolic links for that include '-follow'

tar File¶

Creating a tar file

tar -cvf output.tar /dirname
tar -cvf output.tar /dirname1 /dirname2 filename1 filename2
tar -cvf output.tar /home/vivek/data /home/vivek/pictures /home/vivek/file.txt
tar -cvf /tmp/output.tar /home/vivek/data /home/vivek/pictures /home/vivek/file.txt

Creating a tar file and preserving permissions

tar cvpfz /target.tar.gz /source/
tar xvpfz /source/
#The p flag stands for --preserve-permissions.

Where,

-c : Create a tar ball.
-v : Verbose output (show progress).
-f : Output tar ball archive file name.
-x : Extract all files from archive.tar.
-t : Display the contents (file list) of an archive.

To View a Tar Ball Contains (list file inside a tar ball)¶

tar -tvf /tmp/data.tar

Network¶

# nload
# ifstat
# nmon              #Quite nice
# ss -s             #Currently Established, Closed, Orphaned and Waiting TCP sockets
# netstat -s        #same
# ss -l | column -t #Display All Open Network Ports
# netstat -tulpn    #SUPER!
# ss -t -a | column -t         #Display All TCP Sockets
# netstat -nat                 #Same as above
# ss -u -a | column -t         #Display All UDP Sockets
# netstat -nau                 #Same as above
# lsof -i :portNumber
# lsof -i tcp:portNumber
# lsof -i udp:portNumber
# lsof -i :80 | grep LISTEN
# netstat -natu | grep 'ESTABLISHED'  #view established connections
# tcptrack -i eth0
# iftop -i eth1
# iftop -F 192.168.1.0/24

# netstat -nat | awk '{print $6}' | sort | uniq -c | sort -n                                 # number of connections. used for checking if box is under attack or not
# netstat -nat | awk '{ print $5}' | cut -d: -f1 | sed -e '/^$/d' | uniq                     # list of all unique IP address connected to server
# netstat -nat | awk '{ print $5}' | cut -d: -f1 | sed -e '/^$/d' | uniq | wc -l             # total number of unique IP addresses
# netstat -atun | awk '{print $5}' | cut -d: -f1 | sed -e '/^$/d' |sort | uniq -c | sort -n  # find if server is under DDoS attack
# netstat -nat |grep {IP-address} | awk '{print $6}' | sort | uniq -c | sort -n              # Specific IP/etc
# netstat --interfaces eth0                                                                  # display dropped and total transmitted packets with netstat for eth0:
# sockstat
# netstat -anp |grep 'tcp\|udp' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n

Network Tools¶

dstat, Combines vmstat, iostat, ifstat, netstat information and more http://dag.wieers.com/home-made/dstat/#documentation
iftop, Amazing network bandwidth utility to analyse what is really happening on your eth http://en.wikipedia.org/wiki/Iftop
netio, Measures the net throughput of a network via TCP/IP http://freecode.com/projects/netio
inq, CLI troubleshooting utility that displays info on storage, typically Symmetrix. By default, INQ returns the device name, Symmetrix ID, Symmetrix LUN, and capacity ftp://ftp.emc.com/pub/symm3000/inquiry/
send_arp, Sends out an arp broadcast on the specified network device (defaults to eth0), reporting an old and new IP address mapping to a MAC address http://linux.die.net/man/8/send_arp
EtherApe, is a graphical network monitor for Unix modeled after etherman. Featuring link layer, IP and TCP modes, it displays network activity graphically http://etherape.sourceforge.net/
iptraf, An IP traffic monitor that shows information on the IP traffic passing over your network http://iptraf.seul.org/
netsniff-ng, a zero-copy analyzer, packet capturer and replayer itself supporting the pcap file format
trafgen, a zero-copy wire-rate traffic generator
bpfc, a Berkeley Packet Filter compiler
ifpps, a top-like kernel networking statistics tool
flowtop, a top-like netfilter connection tracking tool
curvetun, a lightweight multiuser IP tunnel based on elliptic curve cryptography
ashunt, an Autonomous System trace route utility
slurm, just a monitor, ascii graphed

/etc/modules.conf #modprobe configuration file (new)
/etc/conf.modules #(old)
depmod    #Determine module dependencies
insmod    #Insert a module into the kernel
lsmod     #List kernel modules
rmmod     #Remove an installed kernel module
modinfo   #Show information about a kernel module
modprobe  #Install modules and their dependencies (or remove)
uname     #Unix name, and kernel version number.

Running Cron Job at Boot¶

crontab -e
@reboot root (sleep 3600; /etc/init.d/whatever start) & #or
@reboot root /usr/local/bin/shellinaboxd &
update-rc.d disable whatever

Issue: `Cannot open your terminal '/dev/pts/0' - please check`¶

su user
script /dev/null
screen -r

Setting Time on Debian-based System¶

dpkg-reconfigure tzdata

How to diff Properly¶

sdiff file1.txt file2.txt | colordiff
vimdiff file1 file2

Adding User to sudo List¶

$ su
# apt-get install sudo
# adduser dave sudo
# exit

Enabling/Disabling Service in Debian-based Systems¶

rcconf
sysv-rc-conf
update-rc.d -f {SERVICE-NAME} remove

How to set chmod for a folder and all of its subfolders and files in Linux Ubuntu Terminal?¶

The other answers are correct, in that chmod -R 755 will set this as permissions to all files and folders in the tree. But why on earth would you want to? It might make sense for the directories, but why set the execute bit on all the files? I suspect what you really want to do is set the directories to 755 and either leave the files alone or set them to 644. For this, you can use the find command. For example: To change all the directories to 755 (-rwxr-xr-x):

find /opt/lampp/htdocs -type d -exec chmod 755 {} \;

To change all the files to 644 (-rw-r--r--):

find /opt/lampp/htdocs -type f -exec chmod 644 {} \;

Swap¶

Creating Swap File¶

Use dd to create a file the size of your requirement. 1GB is enough if your system has 512MB RAM.

dd if=/dev/zero of=/swap bs=1M count=1024

Maybe?

cat /dev/zero | pv -brt | dd of=/dev/sdX bs=1024

Set the permissions, hence only root can r/w it:

chown root:root /swap
chmod 0600 /swap

Format it to swap:

mkswap /swap

Activate it:

swapon /swap

Add this line to /etc/fstab to make the change permanent:

/swap swap swap defaults 0 0

Removing Swap File¶

Turn off the swap file:

swapoff -a

Delete the file:

rm -rf /swap

And remove the earlier line added in /etc/fstab.

Convert permissions to octal¶

$ls -l | sed -e 's/--x/1/g' -e 's/-w-/2/g' -e 's/-wx/3/g' -e 's/r--/4/g' \
 -e 's/r-x/5/g' -e 's/rw-/6/g' -e 's/rwx/7/g' -e 's/---/0/g'

-755  1 jrl jrl  111943 2003-10-21 19:57 logscan
-644  1 jrl jrl   35468 2003-11-23 16:13 htfoo
-700  1 jrl jrl 3100672 2004-05-15 17:00 mutt
-644  1 jrl jrl   10162 2005-02-22 14:14 joinstep2.php
-777  1 jrl jrl   41079 2005-04-21 13:02 setistats
d755  2 jrl jrl      47 2007-10-26 14:41 rf
-700  1 jrl jrl     104 2008-02-05 11:26 getc

Bash Function to Decompress Archives¶

###   Handy Extract Program

extract () {
    if [ -f $1 ] ; then
        case $1 in
            *.tar.bz2)   tar xvjf $1        ;;
            *.tar.gz)    tar xvzf $1     ;;
            *.bz2)       bunzip2 $1       ;;
            *.rar)       unrar x $1     ;;
            *.gz)        gunzip $1     ;;
            *.tar)       tar xvf $1        ;;
            *.tbz2)      tar xvjf $1      ;;
            *.tgz)       tar xvzf $1       ;;
            *.zip)       unzip $1     ;;
            *.Z)         uncompress $1  ;;
            *.7z)        7z x $1    ;;
            *)           echo "'$1' cannot be extracted via >extract<" ;;
        esac
    else
        echo "'$1' is not a valid file"
    fi
}

Selected Bash Keystrokes¶

Ctrl-U – Cuts everything to the left

Ctrl-W – Cuts the word to the left

Ctrl-Y – Pastes what’s in the buffer

Ctrl-A – Go to beginning of line

Ctrl-E – Go to end of line

Simple password generator¶

This is a simpler password generator.

< /dev/urandom tr -dc A-Za-z0-9_ | head -c8

Note that the ‘tr’ strips out everything except characters in the ranges (alphanumeric, mixed case and underscores). This is a nice approach as piping to head means the minimum number of bytes required to generate a password of appropriate length are taken from /dev/urandom vs other methods which take more than you should need but still have a chance of not having obtained enough random data to generate a password of the required length. You can change the parameter to head to get passwords of any length.

Divide Files Depending on X and Put into Individual Folder¶

kent$  l
total 0
-rw-r--r-- 1 kent kent 0 2011-09-27 11:04 01.txt
-rw-r--r-- 1 kent kent 0 2011-09-27 11:04 02.txt
-rw-r--r-- 1 kent kent 0 2011-09-27 11:04 03.txt
-rw-r--r-- 1 kent kent 0 2011-09-27 11:04 04.txt
-rw-r--r-- 1 kent kent 0 2011-09-27 11:04 05.txt
-rw-r--r-- 1 kent kent 0 2011-09-27 11:04 06.txt
-rw-r--r-- 1 kent kent 0 2011-09-27 11:04 07.txt
-rw-r--r-- 1 kent kent 0 2011-09-27 11:04 08.txt
-rw-r--r-- 1 kent kent 0 2011-09-27 11:04 09.txt
-rw-r--r-- 1 kent kent 0 2011-09-27 11:04 10.txt

kent$  ls|xargs -n4|awk ' {i++;system("mkdir dir"i);system("mv "$0" -t dir"i)}'

kent$  tree
.
|-- dir1
|   |-- 01.txt
|   |-- 02.txt
|   |-- 03.txt
|   `-- 04.txt
|-- dir2
|   |-- 05.txt
|   |-- 06.txt
|   |-- 07.txt
|   `-- 08.txt
`-- dir3
    |-- 09.txt
    `-- 10.txt

Quickest way to transfer 55GB of images to new server¶

Instead of using tar to write to your local disk, you can write directly to the remote server over the network using ssh.

server1$ tar -zc ./path | ssh server2 "cat > ~/file.tar.gz"

Any string that follows your “ssh” command will be run on the remote server instead of the interactive logon. You can pipe input/output to and from those remote commands through SSH as if they were local. Putting the command in quotes avoids any confusion, especially when using redirection. Or, you can extract the tar file on the other server directly:

server1$ tar -zc ./path | ssh server2 "tar -zx -C /destination"

Note the seldom-used -C option. It means “change to this directory first before doing anything.” Or, perhaps you want to “pull” from the destination server:

server2$ tar -zx -C /destination < <(ssh server2 "tar -zc -C /srcdir ./path")

Note that the <(cmd) construct is new to bash and doesn’t work on older systems. It runs a program and sends the output to a pipe, and substitutes that pipe into the command as if it was a file. I could just have easily have written the above as follows:

server2$ tar -zx -C /destination -f <(ssh server2 "tar -zc -C /srcdir ./path")

Or as follows:

server2$ ssh server2 "tar -zc -C /srcdir ./path" | tar -zx -C /destination

Or, you can save yourself some grief and just use rsync:

server1$ rsync -az ./path server2:/destination/

Finally, remember that compressing the data before transfer will your bandwidth, but on a very fast connection, it may actually make the operation take more time. This is because your computer may not be able to compress fast enough to keep up: if compressing 100MB takes longer than it would take to send 100MB, then it’s faster to send it uncompressed. Alternately, you may want to consider piping to gzip yourself (rather than using the -z option) so that you can specify a compression level. It’s been my experience that on fast network connections with compressible data, using gzip at level 2 or 3 (the default is 6) gives the best overall throughput in most cases. Like so:

server1$ tar -c ./path | gzip -2 | ssh server2 "cat > ~/file.tar.gz"

Hide files in an image¶

It is possible to hide a rar archive inside a png image file and then retrieve the files from this image.

cat picture.png archive.rar > hidden_archive_in_pic.png

This can also be done on Windows:

copy picture.png + archive.rar hidden_archive_in_pic.png

When you want to retrieve the hidden files, download the image, rename to .rar and extract.

Trace the Process and See What It is Doing with strace¶

strace is a useful diagnostic, instructional, and debugging tool. It can save lots of headache. System administrators, diagnosticians and trouble-shooters will find it invaluable for solving problems with programs for which the source is not readily available since they do not need to be recompiled in order to trace them. This is also useful to submit bug reports to open source developers. Each line in the trace contains the system call name, followed by its arguments in parentheses and its return value. Run strace against /bin/foo and capture its output to a text file in output.txt: $ strace -o output.txt /bin/foo You can strace the webserver process and see what it's doing. For example, strace php5 fastcgi process, enter: $ strace -p 22254 -s 80 -o /tmp/debug.lighttpd.txt To see only a trace of the open, read system calls, enter : $ strace -e trace=open,read -p 22254 -s 80 -o debug.webserver.txt Where,

-o filename : Write the trace output to the file filename rather than to screen (stderr).
-p PID : Attach to the process with the process ID pid and begin tracing. The trace may be terminated at any time by a keyboard interrupt signal (hit CTRL-C). strace will respond by detaching itself from the traced process(es) leaving it (them) to continue running. Multiple -p options can be used to attach to up to 32 processes in addition to command (which is optional if at least one -p option is given).
-s SIZE : Specify the maximum string size to print (the default is 32).

Linux Cheat Sheets¶

1. Linux Command Line¶

Linux Reference Card – Great reference published on FOSSwire website
One page Linux Manual – Great one page reference to the most popular Linux commands
Unix Tool Box – An incredibly exhaustive reference for all things Linux.
Treebeard’s Unix Cheat Sheet – A great reference with Dos comparisons
Terminal Shortcuts – Cheat sheet for the most common terminal shortcuts
More Terminal Shortcuts – More shortcuts for history and X

2. Linux Security¶

Linux Security Quick Reference Guide – An awesome security checklist reference
IP Tables – If you are interested in Linux firewalls this is a must have
TCPDump – Great cheat sheet to an awesome security tool
Wireshark Filters – An awesome list of filters for the best packet sniffing utility
IP Access Lists – Cheat sheet for IP Access Lists
Common Ports – In case you don’t have all common ports memorized
netcat – Reference to the swiss army knife of networking

3. Linux Administration¶

Linux Admin Quick Reference – PDF cheat sheet of common Linux administration tasks
Crontab Reference – Also see Learning Cron by Example

4. Gnome/KDE¶

Useful Gnome/KDE shortcuts
KDE Cheat Sheet – KDE specific cheat sheet

5. sed/awk/vim and other tools¶

Vi Cheat Sheet – Who doesn’t love Vi/Vim. Also check out my guest post on Vim.
Concise Vim Cheat Sheet
awk nawk and gawk cheat sheet – Awesome awk sheet, also check out awk is a beautiful tool
Sed Stream Editor Cheat Sheet – Savvy Sed
Screen Quick Reference – Quick reference to a must have screen emulator, Also check out Introduction to GNU Screen
Screen Terminal Emulator Cheat Sheet – PDF cheat sheet for screen
Vi/Vim Cheat Sheet – Nice vim/vi keyboard layout reference

6. Distro Specific¶

Ubuntu Cheat Sheet – Ubuntu specific cheat sheet
Debian Cheat Sheet – Debian specific cheat sheet

7. Everything Else¶

Online Man Pages – The ultimate cheat sheet repository. Also use the ‘man‘ command.
```
http://www.nixtutor.com/linux/all-the-best-linux-cheat-sheets/
```