範例與常用技巧

檔頭常用宣告 
 
 Use Bash Strict Mode 
 
 # quickly syntax
set -euo pipefail

# let script exit if a command fails
set -o errexit 
# OR
set -e

# let script exit if an unsed variable is used
set -o nounset
# OR
set -u

# This setting prevents errors in a pipeline from being masked.
set -o pipefail

# for Debug
set -x

# setting IFS
IFS=$'\n\t' 
 How set -o pipefail works 
 $ grep some-string /non/existent/file | sort
grep: /non/existent/file: No such file or directory
$ echo $?
0

$ set -o pipefail
$ grep some-string /non/existent/file | sort
grep: /non/existent/file: No such file or directory
$ echo $?
2 
 How the IFS   works 
 #!/bin/bash
names=(
 "Aaron Maxwell"
 "Wayne Gretzky"
 "David Beckham"
 "Anderson da Silva"
)

echo "With default IFS value..."
for name in ${names[@]}; do
 echo "$name"
done

echo ""
echo "With strict-mode IFS value..."
IFS=$'\n\t'
for name in ${names[@]}; do
 echo "$name"
done

############### Output #############
With default IFS value...
Aaron
Maxwell
Wayne
Gretzky
David
Beckham
Anderson
da
Silva

With strict-mode IFS value...
Aaron Maxwell
Wayne Gretzky
David Beckham
Anderson da Silva 
 檔案的目錄位置 
 # 指定的檔案
readlink -f <file.name>

# 目前檔案
WORKDIR=$(readlink -f "$0") ;目前檔案的絕對路經
WORKDIR=$( cd $( dirname "$0" ) && pwd ) 
 Script 檔案名稱 
 $ echo $0
./test.sh

$ echo `basename $0`
test.sh 
 輸出多行的文字訊息 
 cat <<EOF
Welcome .....
	
Here are the messages that you want to show up
		
EOF 
 所有輸出訊息導入一個檔案 
 Sample #1 
 #!/bin/sh
LOG="my.log"
(
....
) 2>&1 | tee -a $LOG 
 不適用在內容裡有 python 指令的訊息輸入，輸入等待的畫面會無法顯示。 
 Sample #2 
 temp=$(mktemp)
exec &> ${temp}

echo "All outputs will be saved into the file ${temp}." 
 Sample #3 
 #!/bin/bash
set -eu 
exec 3>&1 4>&2
trap 'exec 2>&4 1>&3' 0 1 2 3
exec 1>/path/to/script.log 2>&1

# rest of the script below
dnf -y in foo bar
# firewall rules goes here 
....
... 
 快速修改大量檔案的副檔名 
 ## *.old -> *.new
for fname in $(ls *.old);do echo "mv $fname ->"; echo $(echo $fname |sed 's/.old/.new/');mv $fname $(echo $fname | sed 's/.old/.new/');done 
 快速新增或列出多個指定目錄 
 mkdir {AAA,BBB,CCC} 
 快速備份設定檔 
 cp my.cfg{,.bak} 
 計算檔案總數量 
 用 find 
 # 計算目前目錄底下所有檔案數
find . -type f | wc -l 

# 計算第一層所有子目錄的檔案數並排序清單
find . -maxdepth 1 -type d -print0 | xargs -0 -I {} sh -c 'echo $(find {} -type f | wc -l) {}' | sort -n
NOTE: 第一層子目錄名稱不可包含空格 
 用 rsync 
 rsync --stats --dry-run -ax /usr /tmp

Number of files: 326,373 (reg: 211,698, dir: 24,284, link: 90,391)
Number of created files: 326,373 (reg: 211,698, dir: 24,284, link: 90,391)
Number of deleted files: 0
Number of regular files transferred: 211,698
Total file size: 7,180,685,730 bytes
Total transferred file size: 7,178,524,818 bytes

NOTE: 這指令實際不會作檔案複製，/tmp 只是一個假目錄，回傳結果是 reg: 211698 就是檔案總數 
 用 tree 
 tree /mydir -a | tail -n 1

5 directories, 56 files

NOTE: 當目錄底下有包含 symbolic link 也會被計算 
 執行外部 SHELL 或指令 
 1. 使用 pipe line 
 echo"md5sum $X > $X.sum "| bash 
 2. 使用 eval 
 get_arch="uname -p"
if [ "`eval "$get_arch"`" = "i686" ]; then
....
fi 
 線上執行 Shell 
 bash <(curl -fsSL https://raw.githubusercontent.com/IT-BAER/proxmorph/main/install.sh) install 
 提示字元的路徑名稱太長 
 加上這變數 
 PROMPT_DIRTRIM=2 
 處理 CSV 檔 
 
 Doing a database join with CSV files 
 tvs-utils - eBay's TSV Utilities: Command line tools for large, tabular data files. Filtering, statistics, sampling, joins and more. 
 How To Parse CSV Files In Bash Scripts In Linux 
 How to convert JSON to CSV using Linux / Unix shell 
 
 #!/bin/bash
INPUT=data.cvs
OLDIFS=$IFS
IFS=','
[ ! -f $INPUT ] && { echo "$INPUT file not found"; exit 99; }
while read flname dob ssn tel status
do
	echo "Name : $flname"
	echo "DOB : $dob"
	echo "SSN : $ssn"
	echo "Telephone : $tel"
	echo "Status : $status"
done < $INPUT
IFS=$OLDIFS 
 CSV and JSON 
 # JSON to CSV
cat df.json | jq -r '.[]| join(",")'
cat bingbot.json | jq -r '.prefixes[] | {cidr: .ipv4Prefix, comment: "BingBot"} | join(",")' > bingbot.csv
 
 JSON 檔 
 
 [GitHub] gron - Make JSON greppable! 
 
 建立暫存檔 
 tmpfile1=$(mktemp)
tmpfile2="/tmp/$(basename $0).$$.tmp" 
 Get my public IP 
 curl ifconfig.me
curl ifconfig.me/ip
curl ifconfig.co
curl checkip.amazonaws.com
curl icanhazip.com
curl ipecho.net/plain

dig +short myip.opendns.com @resolver1.opendns.com
dig TXT +short o-o.myaddr.l.google.com @ns1.google.com
dig TXT +short o-o.myaddr.l.google.com @ns1.google.com | awk -F'"' '{print $2}' 
 主機 IP 
 # On Linux
hostname -I
hostip=$(/sbin/ip a | awk '/eth[012]:|ens192:|bond0:/,/^$/' | grep -E "inet [0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" | head -1 | awk -F " " '{print $2}' | cut -d"/" -f1)

# On AIX
hostip=$( ifconfig -a | grep inet | awk '{print $2}' | head -1 )
hostip=$( ifconfig -a | grep inet | awk '{print $2}' | sed -n '/\([0-9]\{1,3\}\.\)\{3\}[0-9]\{1,3\}/p' | head -1 ) 
 找出最大的目錄或檔案 
 # Way 1
du -a /var | sort -n -r | head -n 10

# Way 2
cd /path/to/some/where
du -hsx * | sort -rh | head -10
du -hsx -- * | sort -rh | head -10

# Way 3
find /path/to/dir/ -printf '%s %p\n'| sort -nr | head -10
find . -printf '%s %p\n'| sort -nr | head -10
## Skip directories and only display files
find /path/to/search/ -type f -printf '%s %p\n'| sort -nr | head -10

# Create a shell alias
## shell alias ## 
alias ducks='du -cks * | sort -rn | head'
## deal with special files names ##
alias ducks='du -cks -- * | sort -rn | head' 
 檔名移除空白字元 
 # With tr
for f in *; do mv "$f" `echo $f | tr ' ' '_'`; done

# With find
find . -type f -name "* *.xml" -exec bash -c 'mv "$0" "${0// /_}"' {} \; 
 read 
 輸入密碼 
 echo -n "MySQL username: " ; read username
echo -n "MySQL password: " ; stty -echo ; read password ; stty echo ; echo 
 Yes/No 
 while true; do
echo "The Server IP is $serverip"
read -p "Are you sure that you want to continue? (y/N): " input
input=${input:-n}
 case "$input" in
 y|Y)
 echo
 break
 ;;
 n|N)
 echo "Exit"
 exit 1
 ;;
 *) echo "Please answer Y or N.";;
 esac
done 
 shuf: 隨機排序 
 curl -s https://www.imdb.com/list/ls020046354/export | cut -d ',' -f 6 | shuf 
 tree: 顯示專案目錄的檔案樹 
 tree --dirsfirst --filelimit 10 --sort=name

# Display size of files
tree -s

# Display permissions of files
tree -p

# Display directory only
tree -d

# Display till a certain level/depth
tree -L 1

# List only those files that match pattern given
tree -P *screenshot* 
 sort : 資料排序 
 sort -t ',' -k5,5 -k1,1 -k9,9 -k3,3 -k11,11 my.csv 
 
  -t 分隔符號 
 -k5,5 排序第 5 欄，以字串類型排列 
 欄位排序先後依序為第 5, 1, 9, 3, 11 欄 
 欄位排序若要以數值方式來排，改成 -k5,5n 
 
 timeout : 自動停止執行 
 timeout 10 tail -f /var/log/httpd/access.log
timeout 5m ping 8.8.8.8
timeout 300 tcpdump -n -w data.pcap

# Sending specific signal
# To get a list of all available signals, use the command kill -l .
timeout -s SIGKILL ping 8.8.8.8 
 variables : 變數 
 
 
 
 變數 
 說明 
 
 
  $0 
  腳本檔名 
 
 
  $1 
  第1個參數 
 
 
  $2 
  第2個參數 
 
 
  ${10} 
  第10個參數 #10 
 
 
  $# 
  參數的個數 
 
 
  $* 
  顯示所有參數 (作為一個字串) 
 
 
  $@ 
  顯示所有參數 (每個為一個獨立字串) 
 
 
  ${$*} 
  傳遞到腳本中的參數的個數 
 
 
  ${$@} 
  傳遞到腳本中的參數的個數 
 
 
  $? 
  腳本結束後的傳回值 
 
 
  $$ 
  腳本的程序 ID 
 
 
  $_ 
  前個命令的最後一個參數 
 
 
  $! 
  最後一個執行程序的 ID 
 
 
 u=${1:-root} 
  如果 $1 未指定，就賦予值 root 
 
 
 u=${USER:-foo} 
  如果 $USER 未指定，就賦予值 foo 
 
 
 len=${#var} 
  計算 $var 字串的長度 
 
 
 
 如果 $2 未指定或空值，輸出錯誤訊息 
 ${varName?Error varName is not defined}
${varName:?Error varName is not defined or is empty} 
 
 Introduction to Bash Shell Parameter Expansions 
 
 建立目錄並且切換至目錄 
 mkdir my-dir && cd $_ 
 cut: 切割文字 
 # AAA = BBB, 取出 BBB
cut -d= -f2

# 111 2222 33 444444 555, 取出 33 以後的所有內容
cut -d ' ' -f3-

# 檢視超長的文字
head -n 1 data.csv | wc # 計算行的字元數
head data.csv | cut -c -30 # 列出前幾行的前 30 個字元, 如果要顯示後 30 個字元，可以用 30- 
 tr: 置換 
 # aaa bbb ccc
# 換成
# aaa
# bbb
# ccc
echo "aaa bbb ccc" | tr " " "\n" 
 xargs 
 接入(Pipe) 不支援 stdin 指令的替代方法 
 
 How to Use the Powerful Xargs Command in Linux 
 
 printf: 格式化輸出 
 
 %s 字串 
 %d 整數 
 
 printf "%-40s ..................%s\n" "Disable the service $1" "[$2]"

Disable the service apmd ..................[OK]
Disable the service bluetooth ..................[OK]
Disable the service hidd ..................[OK]
Disable the service cups ..................[OK]
Disable the service firstboot ..................[OK]
Disable the service readahead_early ..................[OK]

printf "%40s ..................%s\n" "Disable the service $1" "[$2]"

 Disable the service apmd ..................[OK]
 Disable the service bluetooth ..................[OK]
 Disable the service hidd ..................[OK]
 Disable the service cups ..................[OK]
 Disable the service firstboot ..................[OK]
 Disable the service readahead_early ..................[OK] 
 數字四捨五入 
 printf "%.0f" 9.46666667 # output: 10

printf "%.2f" 9.46666667 # output: 9.47 
 表格式輸出 
 line="--------------------------------------------------"
printf "+-%.10s-+-%.5s-+-%.6s-+\n" "$line" "$line" "$line"
printf "| %-10s | %-5s | %-6s |\n" "DATE" "CALLS" "ASR(%)"
printf "+-%.10s-+-%.5s-+-%.6s-+\n" "$line" "$line" "$line"

printf "| %.10s | %5d | %.2f | %s\n" "$day" $calls $asr "$flag"

+------------+-------+--------+
| DATE | CALLS | ASR(%) |
+------------+-------+--------+
| 2025-07-10 | 1442 | 97.57 |
| 2025-07-11 | 1266 | 96.99 |
| 2025-07-12 | 1162 | 97.24 |
| 2025-07-13 | 949 | 96.62 |
| 2025-07-14 | 1178 | 98.13 |
| 2025-07-15 | 1665 | 97.05 |
| 2025-07-16 | 1163 | 87.61 |
+------------+-------+--------+
 
 #/bin/bash
seperator=--------------------
seperator=$seperator$seperator
rows="%-15s| %.7d| %3d| %c\n"
TableWidth=37

printf "%-15s| %-7s| %.3s| %s\n" Name ID Age Grades
printf "%.${TableWidth}s\n" "$seperator"
printf "$rows" "Sherlock Holmes" 122 23 A
printf "$rows" "James Bond" 7 27 F
printf "$rows" "Hercules Poirot" 6811 59 G
printf "$rows" "Jane Marple" 1234567 71 C

Name | ID | Age| Grades
-------------------------------------
Sherlock Holmes| 0000122| 23| A
James Bond | 0000007| 27| F
Hercules Poirot| 0006811| 59| G
Jane Marple | 1234567| 71| C 
 
 印出一個長符號 
 printf -- '-%.0s' {1..80}
printf -- '=%.0s' {1..80} 
 ping: 掃描一個 IP 範圍 
 { for p in {1..254}; do ping -c1 -w1 10.22.9.$p & done } | grep "64 bytes" 
 nice: Reduce CPU and Disk load of backup scripts 
 # Reduce the I/O priority of the /usr/local/bin/backup.sh script so that it does not interfere with other processes
# The -n parameter must be between 0 and 7, where lower numbers mean higher priority
/usr/bin/ionice -c2 -n7 /usr/local/bin/backup.sh

# To reduce the CPU priority, use the command nice
# The -n parameter can range from -20 to 19, where lower numbers mean higher priority
/usr/bin/nice -n 19 /usr/local/bin/backup.sh

# Nice and ionice can also be combined, to run a script at low I/O and CPU priority
/usr/bin/nice -n 19 /usr/bin/ionice -c2 -n7 /usr/local/bin/backup.sh 
 Hex to ASCII 
 # hex = 54657374696e672031203220330
# ascii = Testing 1 2 3

# xxd
echo 54657374696e672031203220330 | xxd -r -p && echo ''

# printf
printf '\x54\x65\x73\x74\x69\x6e\x67\x20\x31\x20\x32\x20\x33\x0' && echo ''

# sed
echo -n 54657374696e67203120322033 | sed 's/\([0-9A-F]\{2\}\)/\\\\\\x\1/gI' | xargs printf && echo '' 
 隨機密碼生成 
 genpasswd() { 
	local l=$1
 	[ "$l" == "" ] && l=16
 	tr -dc A-Za-z0-9_ < /dev/urandom | head -c ${l} | xargs 
} 
 tr -dc A-Za-z0-9_ < /dev/urandom | head -c 16 | xargs

# Generate more than one
tr -dc A-Za-z0-9_ < /dev/urandom | fold -16 | head -5

#
echo FooBar$RANDOM | md5sum | base64 | cut -c 1-12 
 ls: 進階技巧 
 # Find the biggest zip file
ls -lSrh ~/Downloads/*.zip 
 stat: 檔案屬性 
 ❯ stat --printf='Name: %n\nPermissions: %a\n' my.log
Name: my.log
Permissions: 777

❯ stat --format="%F" my.log
regular file

# Symlink
❯ stat ~/bin/FoxitReader
 File: /home/alang/bin/FoxitReader -> /home/alang/opt/foxitsoftware/foxitreader/FoxitReader.sh
 Size: 56 	Blocks: 0 IO Block: 4096 symbolic link
Device: 10302h/66306d	Inode: 787474 Links: 1
Access: (0777/lrwxrwxrwx) Uid: ( 1000/ alang) Gid: ( 1000/ alang)
Access: 2023-07-16 10:26:13.412193581 +0800
Modify: 2023-02-26 12:13:50.234374171 +0800
Change: 2023-02-26 12:13:50.234374171 +0800
 Birth: 2023-02-26 12:13:50.234374171 +0800

❯ stat -L ~/bin/FoxitReader
 File: /home/alang/bin/FoxitReader
 Size: 120 	Blocks: 8 IO Block: 4096 regular file
Device: 10302h/66306d	Inode: 1457222 Links: 1
Access: (0755/-rwxr-xr-x) Uid: ( 1000/ alang) Gid: ( 1000/ alang)
Access: 2023-07-02 14:34:09.957428285 +0800
Modify: 2023-02-26 12:13:50.246374250 +0800
Change: 2023-02-26 12:13:50.246374250 +0800
 Birth: 2023-02-26 12:13:48.530362986 +0800 
 less: 進階技巧 
 顯示有包含色碼的內容 
 less -r yourfile