# 系統管理技巧

##### Home 目錄

完整複製 Home 目錄

由於 User 的 Home 目錄內有許多隱藏檔，若要完整複製它們，有兩個方法：

方法一：可以複製成一個新目錄

```bash
cd /home
cp -a user1/ user1_new/
```

方法二：複製到一個現有目錄內

```bash
cd /home
cp -a user1/.[^.]* user1_new/ 
```

手動建立一個新的 Home 目錄

```bash
sudo mkhomedir_helper bob
```

或

```bash
cp -r /etc/skel /home/bob
chown -R bob.bob /home/bob
chmod 0700 /home/bob 
```

為什麼 /home 目錄權限有一個點

> Answer: 有(點)的權限目錄表示有在 SELinux 監控清單，既使關閉了 SELinux ，權限也不會變更。

```
# ls -ld /home
drwxr-xr-x. 2 root root 4096 Mar 28  2017 /home

# ls -Zd /home
drwxr-xr-x.  root root system_u:object_r:home_root_t:s0 /home
```

##### 多帳號共用目錄

在共用目錄下 share\_test，群組 team 的所有帳號都可以互相編輯修改

```bash
groupadd team
mkdir /worktmp/share_test
chgrp team /worktmp/share_test
chmod 2775 /worktmp/share_test
usermod -aG team i04181
```

##### 自訂 PATH

PATH 新增一個客制 bin 目錄

~/.bashrc :

```bash
# Custom PATH
case :$PATH: in
    *:/home/$USER/bin:*) ;;
    *) PATH=/home/$USER/bin:$PATH ;;
esac
```

```bash
[ -z "$(sed -n '\@/usr/local/bin@p' <<< $PATH)" ] && PATH=/usr/local/bin:$PATH
```

##### Custom Prompt

```bash
# Kali-like Custom PROMPT
PS1="\[\033[38;5;209m\]┌──[\[\033[38;5;141m\]\u\[\033[38;5;209m\]@\[\033[38;5;105m\]\h\[\033[38;5;231m\]:\w\[\033[38;5;209m\]]\[\033[33m\]\$(GIT_PS1_SHOWUNTRACKEDFILES=1 GIT_PS1_SHOWDIRTYSTATE=1 __git_ps1)\[\033[00m\]\n\[\033[38;5;209m\]└─\\[\033[38;5;209m\]$\[\033[37m\] "
```

Solution: \_\_git\_ps1 command not found

```bash
curl -o ~/.git-prompt.sh https://raw.githubusercontent.com/git/git/master/contrib/completion/git-prompt.sh
echo 'source ~/.git-prompt.sh' >> ~/.bashrc
```

##### 清除 Zombie 程序(defunct)

One may deal with zombie processes in any one of the following ways:

- Fix the parent process to make it execute `wait(2)` on child process exit
- Kill the parent process of the zombie
- Reboot system
- Ignore it

列出 zombie processes

```shell
ps aux |grep "defunct"
ps aux |grep Z

# How many Zombie process running on your server
ps aux | awk {'print $8'}|grep -c Z

# List the PID of Zombie
ps aux | awk '{ print $8 " " $2 }' | grep -w Z

```

Kill zombie process

```shell
# find the parent process list
pstree -paul

kill -9 <PARENT-PID>
```

RHEL Documents:

- [What\_is\_a\_zombie\_(defunct)\_process.pdf](https://osslab.tw/attachments/15)
- [How\_to\_kill\_Zombie\_Defunct\_process.pdf](https://osslab.tw/attachments/14)

##### 資安 &amp; Auditing 相關

```shell
# Parse /var/log/secure
grep "authentication failure" /var/log/secure | awk '{ print $13 }' | cut -b7-  | sort | uniq -c

# Login failed attempts
lastb -F
lastb -F <username>
```

Check Linux Login History

```shell
#!/bin/bash
#Filename: intruder_detect.sh
#Description: Check Linux Login History
AUTHLOG=/var/log/secure

if [[ -n $1 ]];
then
  AUTHLOG=$1
  echo Using Log file : $AUTHLOG
fi

# Collect the failed login attempts
FAILED_LOG=/tmp/failed.$$.log
egrep "Failed pass" $AUTHLOG > $FAILED_LOG 

# Collect the successful login attempts
SUCCESS_LOG=/tmp/success.$$.log
egrep "Accepted password|Accepted publickey|keyboard-interactive" $AUTHLOG > $SUCCESS_LOG

# extract the users who failed
failed_users=$(cat $FAILED_LOG | awk '{ print $(NF-5) }' | sort | uniq)

# extract the users who successfully logged in
success_users=$(cat $SUCCESS_LOG | awk '{ print $(NF-5) }' | sort | uniq)
# extract the IP Addresses of successful and failed login attempts
failed_ip_list="$(egrep -o "[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+" $FAILED_LOG | sort | uniq)"
success_ip_list="$(egrep -o "[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+" $SUCCESS_LOG | sort | uniq)"

# Print the heading
printf "%-10s|%-10s|%-10s|%-15s|%-15s|%s\n" "Status" "User" "Attempts" "IP address" "Host" "Time range"

# Loop through IPs and Users who failed.

for ip in $failed_ip_list;
do
  for user in $failed_users;
    do
    # Count failed login attempts by this user from this IP
    attempts=`grep $ip $FAILED_LOG | grep " $user " | wc -l`

    if [ $attempts -ne 0 ]
    then
      first_time=`grep $ip $FAILED_LOG | grep " $user " | head -1 | cut -c-16`
      time="$first_time"
      if [ $attempts -gt 1 ]
      then
        last_time=`grep $ip $FAILED_LOG | grep " $user " | tail -1 | cut -c-16`
        time="$first_time -> $last_time"
      fi
      HOST=$(host $ip 8.8.8.8 | tail -1 | awk '{ print $NF }' )
      printf "%-10s|%-10s|%-10s|%-15s|%-15s|%-s\n" "Failed" "$user" "$attempts" "$ip"  "$HOST" "$time";
    fi
  done
done

for ip in $success_ip_list;
do
  for user in $success_users;
    do
    # Count successful login attempts by this user from this IP
    attempts=`grep $ip $SUCCESS_LOG | grep " $user " | wc -l`

    if [ $attempts -ne 0 ]
    then
      first_time=`grep $ip $SUCCESS_LOG | grep " $user " | head -1 | cut -c-16`
      time="$first_time"
      if [ $attempts -gt 1 ]
      then
        last_time=`grep $ip $SUCCESS_LOG | grep " $user " | tail -1 | cut -c-16`
        time="$first_time -> $last_time"
      fi
      HOST=$(host $ip 8.8.8.8 | tail -1 | awk '{ print $NF }' )
      printf "%-10s|%-10s|%-10s|%-15s|%-15s|%-s\n" "Success" "$user" "$attempts" "$ip"  "$HOST" "$time";
    fi
  done
done

rm -f $FAILED_LOG
rm -f $SUCCESS_LOG
```

System Audit

```shell
# Install Audit
yum install audit
systemctl start auditd

# Authentication Report
# To get authentication report for all the attempts which was made
aureport -au -i | more
# To get authentication report for all the success attempts which was made
aureport -au -i --success | more
# To get authentication report for all the failed attempts which was made
aureport -au -i --failed | more
# To get success login information
aureport -l --success | more
# To get failed login information
aureport -l --failed | more
# To get success login summary report for all the success attempts which was made
aureport -l --success --summary -i | more
```

Check if a RHEL system is vulnerable to a specific CVE

```shell
# rpm -q --changelog [package-name] | grep [CVE-NUMBER]
rpm -q --changelog openssl | grep CVE-2021-3450
rpm -q --changelog openssl | grep CVE
rpm -q --changelog openssl | grep CVE-2021

# Using yum command
yum install yum-plugin-security
yum update yum
yum updateinfo info --cve CVE-2021-3445
```

Auditd

- [AUDITD RECOMMENDED CONFIGURATION ON REDHAT OR CENTOS LINUX FOR SYSTEM AUDITING](https://freelinuxtutorials.com/auditd-recommended-configuration-on-redhat-or-centos-linux-for-system-auditing/)
- [Linux 設定 pam\_tty\_audit 記錄 SSH 使用者操作指令教學與範例](https://officeguide.cc/linux-pam-tty-audit-logging-shell-user-activity-tutorial-examples/)
- [Adultd：Linux 系統稽核工具使用教學與範例](https://officeguide.cc/linux-audit-daemon-auditd-tutorial/)
- The [psacct](https://www.cyberciti.biz/tips/howto-log-user-activity-using-process-accounting.html) package contains several utilities for monitoring process activities, including ac, lastcomm, accton and sa.

Auditing tool for UNIX/Linux like - Lynis

- [https://cisofy.com/](https://cisofy.com/)
- [How to Do Security Auditing of Linux System Using Lynis Tool](https://www.tecmint.com/linux-security-auditing-and-scanning-with-lynis-tool/)

##### rsh

rsh server

```
# install on CentOS 6/7
yum install rsh-server

# Startup the service on CentOS 6
chkconfig rsh on
chkconfig rlogin on
service xinetd reload

# Startup the service on CentOS 7
systemctl start rsh.socket
systemctl start rlogin.socket
systemctl start rexec.socket
systemctl enable rsh.socket
systemctl enable rlogin.socket
systemctl enable rexec.socket
```

##### strace 程式除錯

```
# Trace the command
strace df -h

# Trace the process ID
strace -p 33259

# Get Summary of Linux Process
strace -c -p 3569

# Print Instruction Pointer During System Call
strace -i df -h

# Show Time of Day For Each Trace Output Line
strace -t df -h

# Print Command Time Spent in System Calls
strace -T df -h

# Trace Only Specific System Calls
strace -e trace=write df -h
strace -p 3569 -e poll

```

##### 停用 suspend, hibernation

```bash
# disable the following systemd targets
sudo systemctl mask sleep.target suspend.target hibernate.target hybrid-sleep.target

sudo systemctl restart systemd-logind.service

# Then reboot the system and log in again
# Verify if the changes have been effected using the command
sudo systemctl status sleep.target suspend.target hibernate.target hybrid-sleep.target

# To re-enable the suspend and hibernation modes, run the command
sudo systemctl unmask sleep.target suspend.target hibernate.target hybrid-sleep.target
```

To prevent the system from going into suspend state upon closing the lid, edit the `/etc/systemd/logind.conf` file.

```
[Login] 
HandleLidSwitch=ignore 
HandleLidSwitchDocked=ignore
```

##### 磁碟裝置與磁區

lsblk

```
# Check the disks
lsblk

nvme0n1     259:0    0 465.8G  0 disk 
├─nvme0n1p1 259:1    0   512M  0 part /boot/efi
└─nvme0n1p2 259:2    0 465.3G  0 part /
nvme1n1     259:3    0 953.9G  0 disk /media/alang/AlangsData

# Check the disks for the details
lsblk --fs

NAME               FSTYPE   LABEL UUID                                   MOUNTPOINT
sda
├─sda1             xfs            7a72d0ab-c234-4ad6-82dd-aa53edff7c78   /boot
└─sda2             LVM2_mem       VqfMLI-x1MU-Ui0R-w2UI-3Qaq-na31-FoNKfL
  ├─rootvg-root    xfs            18817b75-3bd9-4ea7-b1b8-1d71b790ac45   /
  ├─rootvg-swap    swap           efc1e891-3ad9-4f18-8f03-a0d70f26c181   [SWAP]
  └─rootvg-worktmp xfs            2be7fb38-c1cf-4ce0-b4ee-11975ef745b2   /worktmp
sdb                LVM2_mem       kqHzI1-y8GI-SEkr-jhQn-BYAy-x2Tg-e1jdF3
├─dbvg-db2_home    xfs            f333bfb2-7a82-4bbe-aa20-325efc2febf8   /db2_home
├─dbvg-db2_vol     xfs            5702a9cd-a70e-4f48-9c5d-c29858dbaca2   /db2_vol
└─dbvg-dbtmp       xfs            9dcd5abd-ae6b-428b-b326-cf0c56d534a1   /dbtmp
sr0

# List UUID of disk
lsblk -l -o NAME,FSTYPE,MOUNTPOINT,UUID

NAME FSTYPE MOUNTPOINT UUID
sda
sda1 ext4   /boot      f830a3fa-1f94-42f4-9dca-5b5c077eab66
sda2 ext4   /          dcbdf18c-2fb4-426c-9dac-d13a45b7ebba
sda3 swap   [SWAP]     6f40f01b-e9ed-4092-9c65-1445d92ec9da
sda4 ext4              6df9a3a6-052e-41f3-b15a-cb258db0267f
OVM_SYS_REPO_PART_3600508b1001cbe65c99583659f085b36 (dm-0)
     ext4              6df9a3a6-052e-41f3-b15a-cb258db0267f
sr0

# Check the filesystem for the specified disk
lsblk --fs /dev/sdb

NAME            FSTYPE   LABEL UUID                                   MOUNTPOINT
sdb             LVM2_mem       kqHzI1-y8GI-SEkr-jhQn-BYAy-x2Tg-e1jdF3
├─dbvg-db2_home xfs            f333bfb2-7a82-4bbe-aa20-325efc2febf8   /db2_home
├─dbvg-db2_vol  xfs            5702a9cd-a70e-4f48-9c5d-c29858dbaca2   /db2_vol
└─dbvg-dbtmp    xfs            9dcd5abd-ae6b-428b-b326-cf0c56d534a1   /dbtmp

```

lshw

```bash
sudo lshw -short -class disk,volume

H/W path                 Device           Class          Description
====================================================================
/0/100/14/0/3/4/0.0.0    /dev/sda         disk           Mass-Storage
/0/100/14/0/3/4/0.0.0/0  /dev/sda         disk       
```

抹除磁碟

```bash
wipefs -a /<device-path>
```

##### last

```shell
# To check the last ten login attempts, you can pipe it with "head"
last | head -n 10

# using complete usernames and hostnames
last -w

# find the device used by the user
tty

# To find the last login by date,
last --since <date>
last --until <date>
last --since -2days

# find the last bad login attempts
sudo lastb
tail -f -n 100 /var/log/auth.log | grep -i failed

# find the last SSH logins
tail -f -n 100 /var/log/auth.log | grep -i sshd
sudo journalctl -r -u ssh | grep -i failed

# find last login times for all users
lastlog
lastlog -u <user>
```

##### 檢測虛擬平台類型

```shell
dmidecode -s system-manufacturer
systemd-detec-virt
virt-what
```

##### 硬體資訊

```
sudo lshw -short

H/W path                 Device           Class          Description
====================================================================
                                          system         NUC8i7HVK
/0                                        bus            NUC8i7HVB
/0/0                                      memory         64KiB BIOS
/0/2f                                     memory         16GiB System Memory
/0/2f/0                                   memory         8GiB SODIMM DDR4 Synchronous Unbuffered (Unregistered)
/0/2f/1                                   memory         8GiB SODIMM DDR4 Synchronous Unbuffered (Unregistered)
/0/34                                     memory         256KiB L1 cache
/0/35                                     memory         1MiB L2 cache
/0/36                                     memory         8MiB L3 cache
/0/37                                     processor      Intel(R) Core(TM) i7-8809G CPU @ 3.10GHz
/0/100                                    bridge         Xeon E3-1200 v6/7th Gen Core Processor Host Bridge/DRA
/0/100/1                                  bridge         Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe
/0/100/1/0               /dev/fb0         display        Polaris 22 [Radeon RX Vega M GH]
/0/100/1/0.1                              multimedia     Advanced Micro Devices, Inc. [AMD/ATI]
/0/100/1.1                                bridge         Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe
/0/100/1.1/0                              bus            ASMedia Technology Inc.
/0/100/1.1/0/0           usb3             bus            xHCI Host Controller
/0/100/1.1/0/1           usb4             bus            xHCI Host Controller
...
#
sudo lshw -html > HardwareSummary.html
```

Finding Number of Ram Slots

```shell
sudo dmidecode -t memory

sudo lshw -class memory
```

More options

```bash
sudo lshw -C <option>
```

<table border="1" id="bkmrk-option-description-n" style="border-collapse: collapse; width: 100%; border-width: 1px;"><colgroup><col style="width: 50%;"></col><col style="width: 50%;"></col></colgroup><tbody><tr><td>Option  
</td><td>Description  
</td></tr><tr><td>network  
</td><td>Gets the details of the network hardware devices.</td></tr><tr><td>memory</td><td>Displays the details of RAM in your system.</td></tr><tr><td>storage</td><td>Prints details of the storage drives.</td></tr><tr><td>system</td><td>Gets the details of the motherboard and plug-and-play slots</td></tr><tr><td>multimedia</td><td>Details of the sound card of your system.</td></tr><tr><td>display</td><td>Know more about what is powering the display output.</td></tr><tr><td>bridge</td><td>Displays info about the PCIe bridges.</td></tr><tr><td>bus  
</td><td>It will list down buses and their details.</td></tr><tr><td>CPU  
</td><td>List the processor details</td></tr></tbody></table>

Inxi

```bash
# Install
sudo apt-get install inxi

# Check dependencies
inxi --recommends

# Shows Full Linux System Information
inxi -F

# Find Linux Laptop or PC Model Information
inxi -M

# Find Linux CPU and CPU Speed Information
inxi -C

# Find Graphic Card Information in Linux
inxi -G

# Find Audio/Sound Card Information in Linux
inxi -A
```

GUI Tools

```bash
# HardInfo
sudo apt-get install hardinfo
```

lspci

```bash
lspci
lspci -v -s <bus number>:<device number>.<function number>
```

lsusb

```bash
lsusb

# Check a USB device's maximum speed
# Speed vs USB version
# 12M    USB1.0
# 480M   USB2.0
# 5000M  USB 3.2 Gen 1 (aka USB 3.0)
# 10000M USB 3.2 Gen 2 (aka USB 3.1)
# 20000M USB 3.2 Gen 2x2
#
lsusb -t
```

##### 編譯與開發工具

RedHat/CentOS

```shell
# RedHat/CentOS 6
yum install make libtool autoconf subversion git cvs wget libogg-devel gcc gcc-c++ pkgconfig

# RedHat/CentOS 7
yum group install "Development Tools"
```

Ubuntu/Debian

```bash
apt-get install build-essential
```

##### dd

```shell
# 備份 MBR
dd if=/dev/hdx of=/path/to/image count=1 bs=512

# 光碟轉成 iso 檔
dd if=dev/cdrom of=/root/cd.iso

# 銷毀硬碟資料
dd if=/dev/urandom of=/dev/hda1

# 備份整個 USB-Flash
dd if=/dev/sdb | gzip > ./my-usb_flash.img.gz

# 還原 USB-Flash
gzip -dc ./my-usb_flash.img.gz | dd of=/dev/sdb

# 建立一個測試用的大檔案10GB
dd if=/dev/zero of=/path/to/image bs=1G count=10
# NOTE: 新版 Linux 可以改用指令
fallocate -l 1G test.img 

# Test network bandwidth between 2 Linux servers
dd if=/nas-mount-point/samplefile of=/dev/null bs=1M count=1024 iflag=direct
dd if=/dev/zero of=/nas-mount-point/samplefile bs=1M count=1024 oflag=direct 
# NOTE: the samplefile is greater than 1GB and the RAM is preferably more than 2GB.

# 複製到另一個相同 size 的硬碟
dd if=/dev/hda of=/dev/hdb conv=noerror,sync status=progress

# Quick benchmark test for writing 1GB file
dd if=/dev/zero of=/tmp/delme.dd bs=1024 count=1000000 status=progress
```

##### cat: 複製磁區

```bash
cat /dev/sda1 > /dev/sdb1
```

##### history

- [Bash History Display Date And Time For Each Command](https://www.cyberciti.biz/faq/unix-linux-bash-history-display-date-time/)
- [How to disable bash shell history in Linux](https://www.cyberciti.biz/faq/disable-bash-shell-history-linux/)
- [Parsing Bash history in Linux](https://www.redhat.com/sysadmin/parsing-bash-history)
- [Linux History Command with Advance Examples](https://trendoceans.com/history-command-in-linux/)

See time stamp in bash history

```shell
echo 'export HISTTIMEFORMAT="%F %T "' >> ~/.bash_profile
```

Prevent History

```bash
# Prevent History from Recording Any Executed Command
export HISTSIZE=0

# Prevent History from Storing Certain Strings
export HISTIGNORE="passwd:ftp: " 
```

##### 系統效能管理

使用 Swap 的 processes

```shell
for file in /proc/*/status ; do awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $file; done | sort -k 2 -n -r | less
```

OOM (Out of Memory) Killer

- [Linux Memory Overcommitment and the OOM Killer | Baeldung on Linux](https://www.baeldung.com/linux/memory-overcommitment-oom-killer)

##### Swap 管理

```shell
# 檢查目前的 swap 配置
free
swapon -s

# 開啟/關閉 swap
swapon /dev/sda3
swapoff /dev/sda3

# 製作一個 swap 系統
mkswap /dev/sda3 
```

make-swapfile.sh：1GB

```shell
#!/bin/bash

dd if=/dev/zero of=/swapfile bs=1024 count=1024k
chown root:root /swapfile
chmod 0600 /swapfile
mkswap /swapfile
swapon /swapfile
echo "/swapfile          swap            swap    defaults        0 0" >> /etc/fstab
sysctl vm.swappiness=10
echo vm.swappiness=10 >> /etc/sysctl.conf
free -h
cat /proc/sys/vm/swappiness
```

Extend the existing SWAP partition:

> NOTE: 不可在 Production 環境執行變更；如果要不影響生產服務，使用新增 SWAP partition 作法。

```
[root@tycitpdb05-a ~]# grep -i swap /etc/fstab
/dev/mapper/rootvg-swap swap                    swap    defaults        0 0

[root@tycitpdb05-a ~]# ls -al /dev/mapper/rootvg-swap
lrwxrwxrwx 1 root root 7 Aug 15 11:49 /dev/mapper/rootvg-swap -> ../dm-1

[root@tycitpdb05-a ~]# swapon -s
Filename                                Type            Size    Used    Priority
/dev/dm-1                               partition       4194300 0       -2

[root@tycitpdb05-a ~]# swapoff -v /dev/mapper/rootvg-swap
swapoff /dev/mapper/rootvg-swap

[root@tycitpdb05-a ~]# lvextend -L16G /dev/rootvg/swap
  Size of logical volume rootvg/swap changed from 4.00 GiB (1024 extents) to 16.00 GiB (4096 extents).
  Logical volume rootvg/swap successfully resized.
  
[root@tycitpdb05-a ~]# mkswap /dev/rootvg/swap
mkswap: /dev/rootvg/swap: warning: wiping old swap signature.
Setting up swapspace version 1, size = 16777212 KiB
no label, UUID=06aac9ae-9e8c-48bd-9f16-5f3d0c32b31f

[root@tycitpdb05-a ~]# swapon -v /dev/rootvg/swap
swapon /dev/rootvg/swap
swapon: /dev/mapper/rootvg-swap: found swap signature: version 1, page-size 4, same byte order
swapon: /dev/mapper/rootvg-swap: pagesize=4096, swapsize=17179869184, devsize=17179869184
```

##### xfs 檔案系統

- [How to Check and Repair XFS Filesystem in RHEL](https://www.2daygeek.com/repairing-xfs-file-system-in-rhel/)

修復 xfs 檔案系統

```shell
sudo mount -a
mount: /data: mount(2) system call failed: Structure needs cleaning.

sudo umount /data

# with '-n' option to perform a dry run
sudo xfs_repair -n /dev/sdb1
# repair the filesystem
sudo xfs_repair /dev/sdb1
```

xfs 備份與回復

```bash
# Install the xfsdump
dnf install xfsdump

# Create a full-backup
# -L: for dump session
# -M: for media in drive
# -f: the backup destination
xfsdump -L session_0 -M datapart -f /data/boot.xfsdump /boot

# Create a incremental backup with the level 1
# -l: the backup level (0-9)
xfsdump -l 1 -L session_1 -M datapart -f /data/boot.xfsdump1 /boot

# Restore a full-backup
xfsrestore -f /data/boot.xfsdump /test

# Restore an incremental backup
xfsrestore -r -f /data/boot.xfsdump /test
xfsrestore -r -f /data/boot.xfsdump1 /test
```

##### Linux Module

- [Linux: How to load a kernel module automatically at boot time](https://www.cyberciti.biz/faq/linux-how-to-load-a-kernel-module-automatically-at-boot-time/)

##### 監視檔案與目錄異動 (Monito File &amp; Directory)

- [Watchman – A File and Directory Watching Tool for Changes](https://www.tecmint.com/watchman-monitor-file-changes-in-linux/)
- [Watchman - A file watching service | Watchman (facebook.github.io)](https://facebook.github.io/watchman/)
- inotify-tools，這工具可以監控特定目錄的檔案異動情形（Linux 2.6.13 以上才有支援），以下連結的範例是用在特定目錄下，一旦有檔案或目錄的異動，立即呼叫 rsync 的備份 script。
- [fswatch - Monitor File and Directory Changes in Linux (tecmint.com)](https://www.tecmint.com/fswatch-monitor-file-changes-linux/)
- [AIDE - How to Check Integrity of File and Directory Using “AIDE” in Linux](https://www.tecmint.com/check-integrity-of-file-and-directory-using-aide-in-linux/)
- [Pyinotify](https://github.com/seb-m/pyinotify/wiki)

##### 比對兩個檔案

列出 A.txt 中没有包含 B.txt 内容的行

```bash
diff A.txt B.txt | grep "^<" | cut -c3-
```

##### 比對兩個目錄  


```shell
# tree
tree dir1
tree dir2

# diff
diff -q /path/to/dir1 /path/to/dir2
diff -q dir1 dir2
diff -qr dir1 dir2
diff -qrs dir1 dir2
```

##### 複製目錄架構但不包含檔案

```shell
# tree
tree -dfi --noreport dir1
tree -dfi --noreport dir1 | xargs -I{} mkdir -p "$HOME/Downloads/{}"
tree -a $HOME/Downloads/dir1

# find + xargs
find dir1 -type d
find dir1 -type d | xargs -I{} mkdir -p "$HOME/Documents/{}" 
tree -a $HOME/Documents/dir1

# find + exec
find dir1 -type d -exec mkdir -p "$HOME/Desktop/{}" \;
```

##### 記憶體管理

- [How to check memory utilization and usage in Linux](https://www.cyberciti.biz/faq/how-to-check-memory-utilization-in-linux/)
- [How to Accurately Check Memory Consumption of a Process in Linux](https://blog.sysxplore.com/p/how-to-accurately-check-process-memory-usage-in-linux)
- [How to Clear RAM Cache and Memory in Linux \[Safe Methods\]](https://www.tecmint.com/clear-ram-memory-cache-buffer-and-swap-space-on-linux/)

```bash
# Using meminfo
cat /proc/meminfo
grep -E --color 'Mem|Cache|Swap' /proc/meminfo
```

- `MemTotal`, Total usable RAM (i.e., physical RAM minus a few reserved bits and the kernel binary code).
- `MemFree`, The sum of LowFree+HighFree.
- `MemAvailable`, (since Linux 3.14) An estimate of how much memory is available for starting new applications, without swapping.
- `Buffers`, Relatively temporary storage for raw disk blocks that shouldn’t get tremendously large (20MB or so).
- `Cached`, In-memory cache for files read from the disk (the page cache). Doesn’t include SwapCached.
- `SwapCached`, Memory that once was swapped out, is swapped back in but still also is in the swap file.

```bash
# Using free
free -h
# Repeat printing free command output every N seconds.
free -s 5 -c 10

```

- `total`, Total installed memory
- `used`, Used memory (calculated as total – free – buffers – cache)
- `free`, Unused memory (MemFree and SwapFree in /proc/meminfo)
- `shared`, Memory used mostly by tmpfs (Shmem in /proc/meminfo)
- `buffers`, Memory used by kernel buffers (Buffers in /proc/meminfo)
- `cache`, Memory used by the page cache and slabs (Cached and SReclaimable in /proc/meminfo)
- `buff/cache`, Sum of buffers and cache
- `available`, Estimation of how much memory is available for starting new applications, without swapping.

```
# Using vmstat
vmstat -w
```

- `swapd`, the amount of virtual memory used.
- `free`, the amount of idle memory.
- `buff`, the amount of memory used as buffers.
- `cache`, the amount of memory used as cache.
- `inact`, the amount of inactive memory. (-a option)
- `active`, the amount of active memory. (-a option)
- `si`, Amount of memory swapped in from disk (/s).
- `so`, Amount of memory swapped to disk (/s).

##### 特殊字元的檔名  


Dash

```bash
# The filename with -- or -
rm -i -v -- -foo
rm -i -v -- --foo
rm -i -v ./-foo

# The filename with -- and whitespaces
rm -i -v -- '-- My Resume . txt'
rm -i -v -- '/path/to/dir/-- My Resume . txt'
rm -i -v -- "/path/to/dir/-- My Resume . txt"

# Using find
find . -name '--my-FileNameGoes-Here' -delete
find /path/to/directory/ -name '---filename with a white spaces' --delete
```

##### cp: 強制覆蓋多檔案

```bash
# 因為內建 alias=cp -i，以致於 -f 參數無效
yes | cp -r /source /target
```

##### System Locale  


```bash
# view information about the current installed locale
locale
localectl status

# view more information about an environmental variable
locale -k LC_TIME

# display a list of all available locales
locale -a

# Set System Locale
## Using the commands
## The following command sets LANG to en_IN.UTF-8 and removes definitions for LANGUAGE.
sudo update-locale LANG=LANG=en_IN.UTF-8 LANGUAGE
## Or
sudo localectl set-locale LANG=en_IN.UTF-8

sudo update-locale LC_TIME=en_IN.UTF-8
## Or
sudo localectl set-locale LC_TIME=en_IN.UTF-8

## Using the profile
vi ~/.bash_profile

LANG="en_IN.utf8"
export LANG

```

##### 快速清空一個大檔

1. Preserve File Permissions and Ownership
2. Maintain Symbolic Links
3. Avoid Disruption to Services or Applications
4. File Locking Issues
5. Efficiency in Log Management

```bash
> access.log
: > access.log
true > access.log
cat /dev/null > access.log
cp /dev/null access.log
dd if=/dev/null of=access.log
echo -n "" > access.log
truncate -s 0 access.log
```

##### tar 指令

- [18 Tar Command Examples on Linux](https://www.tecmint.com/tar-command-examples-linux/)

備份整個系統

```bash
# 1. Switch to single user mode
# NOTE: single mode 會中斷網路功能, 只能本機 console 登入
init 1

# 2. Tar up the whole system
tar zcpvf /backups/fullbackup.tar.gz --directory=/ --exclude=proc --exclude=dev --exclude=sys --exclude=boot --exclude=run --exclude=etc/fstab --exclude=backups .

# 3. Once this completes copy the tar file over to the root directory your new machine
# 4. Take a snapshot of your new machine. This way if things go wrong you can revert to the snapshot and try again.
# 5.Extract the tarball on your new machine
cd /
tar -zxvpf /path/to/fullbackup.tar.gz

```

##### Stress Test

- [stress-ng](https://github.com/ColinIanKing/stress-ng)
    - [Linux 使用 Stress-ng 測試 CPU、記憶體、磁碟 I/O 滿載情況教學與範例 - Office 指南 (officeguide.cc)](https://officeguide.cc/linux-stress-ng-cpu-memory-hard-drive-full-load-tutorial-examples/)
    - [How to Test CPU and Memory Load with Stress &amp; Stress-ng - Shouts.dev](https://shouts.dev/articles/how-to-test-cpu-and-memory-load-with-stress)
    - [How to Stress Test Your Linux CPU for High Load (tecmint.com)](https://www.tecmint.com/linux-cpu-load-stress-test-with-stress-ng-tool/)

stress

```bash
stress --cpu 2 --io 3 --vm 4 --vm-bytes 512M --timeout 10m
```

sysbench

```bash
# size 必須超過 RAM
# max-time 單位秒
sysbench --test=fileio --file-total-size=10G prepare
sysbench --test=fileio --file-total-size=10G --file-test-mode=rndrw --init-rng=on --max-time=300 --max-requests=0 run

```

##### 指令的路徑

```bash
# with command
command -v <cmd-name>

# with which
which <cmd-name>
```

##### getent: 檢視系統資訊

```bash
# The same as 'cat /etc/passwd' and 'cat /etc/shadow'
getent passwd
getent passwd <user-name>
getent shadow
getent group
getent group <group-name>

# /etc/hosts
getent hosts

# /etc/services
getent services
getent services <service-name>

# /etc/networks
getent networks
```

##### 增加 inode 數量

XFS

- KB: [https://access.redhat.com/solutions/41492](https://access.redhat.com/solutions/41492)
- 可線上增加或減小

```bash
# 預設值: 25%
# 調整檔案系統空間 30% 儲存 inode
xfs_growfs -m 30 /<mount-point>
```

EXT4/EXT3

- [https://www.tecmint.com/increase-disk-inode-number-in-linux/](https://www.tecmint.com/increase-disk-inode-number-in-linux/)[ ](https://www.tecmint.com/increase-disk-inode-number-in-linux/)
- 建立檔案系統時，就已經決定了 inode 的數量

```bash
# byte-per-inode: 16384 (by default), 8192, 4096
# 數值愈小；inode 數量愈大
mkfs.ext4 -i <byte-per-inode> /dev/device


# 對於大檔案的使用，可以用以下語法，減小 inode 數量
mkfs.ext4 -T largefile /dev/device
```

##### sysctl

```bash
# See current sysctl overrides
sysctl --system

# Apply the changes
sysctl -p /etc/sysctl.d/ipv6.conf

# Check for current settings
sysctl -a
```