# Linux Administration

# 校時與時區設定

#### Tutorials

- [How to Sync Linux Time With NTP Server - Make Tech Easier](https://www.maketecheasier.com/sync-linux-time-with-ntp-server/)

#### Chrony

Built-in on CentOS/RedHat

- [CHAPTER 18. CONFIGURING NTP USING THE CHRONY SUITE](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system_administrators_guide/ch-configuring_ntp_using_the_chrony_suite)
- [CentOS / RHEL 7 : Configuring NTP using chrony](https://www.thegeekdiary.com/centos-rhel-7-configuring-ntp-using-chrony/)
- [How do I check Linux system’s current NTP configuration](https://www.nixcraft.com/t/how-do-i-check-linux-systems-current-ntp-configuration/3519)

Install

```
yum install chrony
```

Config: /etc/chrony.conf

```
# NTP Public Servers for Taiwan
server tw.pool.ntp.org
server time.stdtime.gov.tw
server clock.stdtime.gov.tw
server jp.pool.ntp.org

# Allow NTP client access from local network.
#allow 192.168.0.0/16
```

Starting

```
systemctl start chronyd
```

Verify

```
[root@localhost ~]# chronyc tracking
Reference ID    : 7A75FDF6 (static.home.twn.sciuridae.cloud)
Stratum         : 3
Ref time (UTC)  : Sun Aug 16 03:35:36 2020
System time     : 0.000073180 seconds slow of NTP time
Last offset     : -0.001259724 seconds
RMS offset      : 0.001259724 seconds
Frequency       : 21.057 ppm slow
Residual freq   : -0.000 ppm
Skew            : 28.166 ppm
Root delay      : 0.032856792 seconds
Root dispersion : 0.002069760 seconds
Update interval : 64.3 seconds
Leap status     : Normal    <===

[root@tpemimtst99 ~]# timedatectl
               Local time: Tue 2021-05-25 09:09:09 CST
           Universal time: Tue 2021-05-25 01:09:09 UTC
                 RTC time: Tue 2021-05-25 01:09:09
                Time zone: Asia/Taipei (CST, +0800)
System clock synchronized: yes    <===
              NTP service: active
          RTC in local TZ: no

[root@tpemimtst99 ~]# chronyc sources
210 Number of sources = 1
MS Name/IP address         Stratum Poll Reach LastRx Last sample
===============================================================================
^* 192.168.21.86                 3  10   377   754  +1543ns[+4123ns] +/-   63ms

```

#### systemd-timesyncd

Build-in on Ubuntu

- [Control your computer time and date with systemd](https://opensource.com/article/20/6/time-date-systemd)

Installation

```bash
sudo apt install systemd-timesyncd
```

The configuration file for systemd-timesyncd is `/etc/systemd/timesyncd.conf`.

```
[Time]
#NTP=
#FallbackNTP=0.debian.pool.ntp.org 1.debian.pool.ntp.org 2.debian.pool.ntp.org 3.debian.pool.ntp.org
#RootDistanceMaxSec=5
#PollIntervalMinSec=32
#PollIntervalMaxSec=2048
```

Specify the time server as follows

```
NTP=<your-time-server-ip>
```

Start timesync

```shell
systemctl enable systemd-timesyncd.service
systemctl start systemd-timesyncd.service
```

timedatectl

```shell
# See the information of the datetime
sudo timedatectl
               Local time: Mon 2021-11-01 14:37:09 UTC
           Universal time: Mon 2021-11-01 14:37:09 UTC
                 RTC time: Mon 2021-11-01 14:37:10    
                Time zone: UTC (UTC, +0000)           
System clock synchronized: yes                        
              NTP service: active   <====                    
          RTC in local TZ: no 

# List timezones supported
sudo timedatectl list-timezones

# Change the system’s timezone
sudo timedatectl set-timezone Asia/Taipei

# Check the status
sudo timedatectl timesync-status
       Server: 91.189.91.157 (ntp.ubuntu.com)   
Poll interval: 17min 4s (min: 32s; max 34min 8s)
         Leap: normal                           
      Version: 4                                
      Stratum: 2                                
    Reference: 11FD227B                         
    Precision: 1us (-24)                        
Root distance: 72.295ms (max: 5s)               
       Offset: +5.311ms                         
        Delay: 67.290ms                         
       Jitter: 2.211ms                          
 Packet count: 6                                
    Frequency: +17.421ppm               
```

#### ntpd

Install

```shell
# Ubuntu
sudo apt install ntp
```

`vi /etc/ntp.conf`

```
#pool 0.ubuntu.pool.ntp.org iburst
#pool 1.ubuntu.pool.ntp.org iburst
#pool 2.ubuntu.pool.ntp.org iburst
#pool 3.ubuntu.pool.ntp.org iburst

# Use Ubuntu's ntp server as a fallback.
#pool ntp.ubuntu.com

# Added the local time server
server 192.168.21.86 prefer iburst
```

Restart the ntpd

```
# Ubuntu 16.04
systemctl stop ntp
systemctl start ntp

# Check the timeserver
ntpq -p

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 0.ubuntu.pool.n .POOL.          16 p    -   64    0    0.000    0.000   0.000
 1.ubuntu.pool.n .POOL.          16 p    -   64    0    0.000    0.000   0.000
 2.ubuntu.pool.n .POOL.          16 p    -   64    0    0.000    0.000   0.000
 3.ubuntu.pool.n .POOL.          16 p    -   64    0    0.000    0.000   0.000
 ntp.ubuntu.com  .POOL.          16 p    -   64    0    0.000    0.000   0.000
+eterna.binary.n 216.218.192.202  2 u   32   64    3   45.986   -0.081   0.021
+pacific.latt.ne 193.187.181.6    3 u   30   64    3    8.205    1.151   0.026
-mis.wci.com     216.218.192.202  2 u   32   64    3   37.928    4.692   0.176
-europa.ellipse. 209.180.247.49   2 u   31   64    3   51.792    4.175   0.027
#38.229.56.9     172.16.21.35     2 u   30   64    3   68.917   -4.451   0.190
#time.richiemcin 97.183.206.88    2 u   29   64    3   69.281   -3.073   0.248
*t1.time.gq1.yah 208.71.46.33     2 u   30   64    3   31.630   -0.081   0.011
-lofn.fancube.co 220.181.254.66   2 u   30   64    3   37.871   -1.088   0.009
#2606:6680:8:1:: 107.46.198.112   2 u   26   64    3  198.284   66.201   0.061
-titan.crash-ove 129.7.1.66       2 u   30   64    3   32.708   -1.474   0.200
-2620:1d5:100:43 47.187.174.51    2 u   29   64    3   70.418    0.592   0.197
 2001:67c:1560:8 17.253.34.123    2 u   32   64    3  133.490   -2.818   0.030
#38.229.57.9     172.16.21.35     2 u   30   64    3  177.896    4.081   6.291
#ntp2i.versadns. 217.180.209.214  2 u   28   64    3   62.795   -2.897   0.017
-104.194.8.227   192.12.19.20     2 u   25   64    3    0.979   -0.737   0.029
-pacific.latt.ne 193.187.181.6    3 u   26   64    3    8.770    1.212   0.050
 2001:67c:1560:8 17.253.34.123    2 u   33   64    3  133.602   -2.792   0.007

```

#### Change Timezone  


##### RedHat 7+

```bash
timedatectl
timedatectl list-timezones
timedatectl set-timezone Asia/Taipei
```

##### CentOS 6  


```bash
# List of timezone
ls /usr/share/zoneinfo

# Check the current timezone set
cat /etc/sysconfig/clock
```

Edit: /etc/sysconfig/clock

```
ZONE="Asia/Taipei"
UTC=true
```

Localtime

```bash
mv /etc/localtime /etc/localtime.bak
ln -sf /usr/share/zoneinfo/Asia/Taipei /etc/localtime
```

##### Debian 13

```bash
sudo timedatectl set-timezone Asia/Taipei
```

#### Manually Set the date-time

```bash
date -s "6 April 2023 15:11:00"
```

# 系統管理技巧

##### Home 目錄

完整複製 Home 目錄

由於 User 的 Home 目錄內有許多隱藏檔，若要完整複製它們，有兩個方法：

方法一：可以複製成一個新目錄

```bash
cd /home
cp -a user1/ user1_new/
```

方法二：複製到一個現有目錄內

```bash
cd /home
cp -a user1/.[^.]* user1_new/ 
```

手動建立一個新的 Home 目錄

```bash
sudo mkhomedir_helper bob
```

或

```bash
cp -r /etc/skel /home/bob
chown -R bob.bob /home/bob
chmod 0700 /home/bob 
```

為什麼 /home 目錄權限有一個點

> Answer: 有(點)的權限目錄表示有在 SELinux 監控清單，既使關閉了 SELinux ，權限也不會變更。

```
# ls -ld /home
drwxr-xr-x. 2 root root 4096 Mar 28  2017 /home

# ls -Zd /home
drwxr-xr-x.  root root system_u:object_r:home_root_t:s0 /home
```

##### 多帳號共用目錄

在共用目錄下 share\_test，群組 team 的所有帳號都可以互相編輯修改

```bash
groupadd team
mkdir /worktmp/share_test
chgrp team /worktmp/share_test
chmod 2775 /worktmp/share_test
usermod -aG team i04181
```

##### 自訂 PATH

PATH 新增一個客制 bin 目錄

~/.bashrc :

```bash
# Custom PATH
case :$PATH: in
    *:/home/$USER/bin:*) ;;
    *) PATH=/home/$USER/bin:$PATH ;;
esac
```

```bash
[ -z "$(sed -n '\@/usr/local/bin@p' <<< $PATH)" ] && PATH=/usr/local/bin:$PATH
```

##### Custom Prompt

```bash
# Kali-like Custom PROMPT
PS1="\[\033[38;5;209m\]┌──[\[\033[38;5;141m\]\u\[\033[38;5;209m\]@\[\033[38;5;105m\]\h\[\033[38;5;231m\]:\w\[\033[38;5;209m\]]\[\033[33m\]\$(GIT_PS1_SHOWUNTRACKEDFILES=1 GIT_PS1_SHOWDIRTYSTATE=1 __git_ps1)\[\033[00m\]\n\[\033[38;5;209m\]└─\\[\033[38;5;209m\]$\[\033[37m\] "
```

Solution: \_\_git\_ps1 command not found

```bash
curl -o ~/.git-prompt.sh https://raw.githubusercontent.com/git/git/master/contrib/completion/git-prompt.sh
echo 'source ~/.git-prompt.sh' >> ~/.bashrc
```

##### 清除 Zombie 程序(defunct)

One may deal with zombie processes in any one of the following ways:

- Fix the parent process to make it execute `wait(2)` on child process exit
- Kill the parent process of the zombie
- Reboot system
- Ignore it

列出 zombie processes

```shell
ps aux |grep "defunct"
ps aux |grep Z

# How many Zombie process running on your server
ps aux | awk {'print $8'}|grep -c Z

# List the PID of Zombie
ps aux | awk '{ print $8 " " $2 }' | grep -w Z

```

Kill zombie process

```shell
# find the parent process list
pstree -paul

kill -9 <PARENT-PID>
```

RHEL Documents:

- [What\_is\_a\_zombie\_(defunct)\_process.pdf](https://osslab.tw/attachments/15)
- [How\_to\_kill\_Zombie\_Defunct\_process.pdf](https://osslab.tw/attachments/14)

##### 資安 &amp; Auditing 相關

```shell
# Parse /var/log/secure
grep "authentication failure" /var/log/secure | awk '{ print $13 }' | cut -b7-  | sort | uniq -c

# Login failed attempts
lastb -F
lastb -F <username>
```

Check Linux Login History

```shell
#!/bin/bash
#Filename: intruder_detect.sh
#Description: Check Linux Login History
AUTHLOG=/var/log/secure

if [[ -n $1 ]];
then
  AUTHLOG=$1
  echo Using Log file : $AUTHLOG
fi

# Collect the failed login attempts
FAILED_LOG=/tmp/failed.$$.log
egrep "Failed pass" $AUTHLOG > $FAILED_LOG 

# Collect the successful login attempts
SUCCESS_LOG=/tmp/success.$$.log
egrep "Accepted password|Accepted publickey|keyboard-interactive" $AUTHLOG > $SUCCESS_LOG

# extract the users who failed
failed_users=$(cat $FAILED_LOG | awk '{ print $(NF-5) }' | sort | uniq)

# extract the users who successfully logged in
success_users=$(cat $SUCCESS_LOG | awk '{ print $(NF-5) }' | sort | uniq)
# extract the IP Addresses of successful and failed login attempts
failed_ip_list="$(egrep -o "[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+" $FAILED_LOG | sort | uniq)"
success_ip_list="$(egrep -o "[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+" $SUCCESS_LOG | sort | uniq)"

# Print the heading
printf "%-10s|%-10s|%-10s|%-15s|%-15s|%s\n" "Status" "User" "Attempts" "IP address" "Host" "Time range"

# Loop through IPs and Users who failed.

for ip in $failed_ip_list;
do
  for user in $failed_users;
    do
    # Count failed login attempts by this user from this IP
    attempts=`grep $ip $FAILED_LOG | grep " $user " | wc -l`

    if [ $attempts -ne 0 ]
    then
      first_time=`grep $ip $FAILED_LOG | grep " $user " | head -1 | cut -c-16`
      time="$first_time"
      if [ $attempts -gt 1 ]
      then
        last_time=`grep $ip $FAILED_LOG | grep " $user " | tail -1 | cut -c-16`
        time="$first_time -> $last_time"
      fi
      HOST=$(host $ip 8.8.8.8 | tail -1 | awk '{ print $NF }' )
      printf "%-10s|%-10s|%-10s|%-15s|%-15s|%-s\n" "Failed" "$user" "$attempts" "$ip"  "$HOST" "$time";
    fi
  done
done

for ip in $success_ip_list;
do
  for user in $success_users;
    do
    # Count successful login attempts by this user from this IP
    attempts=`grep $ip $SUCCESS_LOG | grep " $user " | wc -l`

    if [ $attempts -ne 0 ]
    then
      first_time=`grep $ip $SUCCESS_LOG | grep " $user " | head -1 | cut -c-16`
      time="$first_time"
      if [ $attempts -gt 1 ]
      then
        last_time=`grep $ip $SUCCESS_LOG | grep " $user " | tail -1 | cut -c-16`
        time="$first_time -> $last_time"
      fi
      HOST=$(host $ip 8.8.8.8 | tail -1 | awk '{ print $NF }' )
      printf "%-10s|%-10s|%-10s|%-15s|%-15s|%-s\n" "Success" "$user" "$attempts" "$ip"  "$HOST" "$time";
    fi
  done
done

rm -f $FAILED_LOG
rm -f $SUCCESS_LOG
```

System Audit

```shell
# Install Audit
yum install audit
systemctl start auditd

# Authentication Report
# To get authentication report for all the attempts which was made
aureport -au -i | more
# To get authentication report for all the success attempts which was made
aureport -au -i --success | more
# To get authentication report for all the failed attempts which was made
aureport -au -i --failed | more
# To get success login information
aureport -l --success | more
# To get failed login information
aureport -l --failed | more
# To get success login summary report for all the success attempts which was made
aureport -l --success --summary -i | more
```

Check if a RHEL system is vulnerable to a specific CVE

```shell
# rpm -q --changelog [package-name] | grep [CVE-NUMBER]
rpm -q --changelog openssl | grep CVE-2021-3450
rpm -q --changelog openssl | grep CVE
rpm -q --changelog openssl | grep CVE-2021

# Using yum command
yum install yum-plugin-security
yum update yum
yum updateinfo info --cve CVE-2021-3445
```

Auditd

- [AUDITD RECOMMENDED CONFIGURATION ON REDHAT OR CENTOS LINUX FOR SYSTEM AUDITING](https://freelinuxtutorials.com/auditd-recommended-configuration-on-redhat-or-centos-linux-for-system-auditing/)
- [Linux 設定 pam\_tty\_audit 記錄 SSH 使用者操作指令教學與範例](https://officeguide.cc/linux-pam-tty-audit-logging-shell-user-activity-tutorial-examples/)
- [Adultd：Linux 系統稽核工具使用教學與範例](https://officeguide.cc/linux-audit-daemon-auditd-tutorial/)
- The [psacct](https://www.cyberciti.biz/tips/howto-log-user-activity-using-process-accounting.html) package contains several utilities for monitoring process activities, including ac, lastcomm, accton and sa.

Auditing tool for UNIX/Linux like - Lynis

- [https://cisofy.com/](https://cisofy.com/)
- [How to Do Security Auditing of Linux System Using Lynis Tool](https://www.tecmint.com/linux-security-auditing-and-scanning-with-lynis-tool/)

##### rsh

rsh server

```
# install on CentOS 6/7
yum install rsh-server

# Startup the service on CentOS 6
chkconfig rsh on
chkconfig rlogin on
service xinetd reload

# Startup the service on CentOS 7
systemctl start rsh.socket
systemctl start rlogin.socket
systemctl start rexec.socket
systemctl enable rsh.socket
systemctl enable rlogin.socket
systemctl enable rexec.socket
```

##### strace 程式除錯

```
# Trace the command
strace df -h

# Trace the process ID
strace -p 33259

# Get Summary of Linux Process
strace -c -p 3569

# Print Instruction Pointer During System Call
strace -i df -h

# Show Time of Day For Each Trace Output Line
strace -t df -h

# Print Command Time Spent in System Calls
strace -T df -h

# Trace Only Specific System Calls
strace -e trace=write df -h
strace -p 3569 -e poll

```

##### 停用 suspend, hibernation

```bash
# disable the following systemd targets
sudo systemctl mask sleep.target suspend.target hibernate.target hybrid-sleep.target

sudo systemctl restart systemd-logind.service

# Then reboot the system and log in again
# Verify if the changes have been effected using the command
sudo systemctl status sleep.target suspend.target hibernate.target hybrid-sleep.target

# To re-enable the suspend and hibernation modes, run the command
sudo systemctl unmask sleep.target suspend.target hibernate.target hybrid-sleep.target
```

To prevent the system from going into suspend state upon closing the lid, edit the `/etc/systemd/logind.conf` file.

```
[Login] 
HandleLidSwitch=ignore 
HandleLidSwitchDocked=ignore
```

##### 磁碟裝置與磁區

lsblk

```
# Check the disks
lsblk

nvme0n1     259:0    0 465.8G  0 disk 
├─nvme0n1p1 259:1    0   512M  0 part /boot/efi
└─nvme0n1p2 259:2    0 465.3G  0 part /
nvme1n1     259:3    0 953.9G  0 disk /media/alang/AlangsData

# Check the disks for the details
lsblk --fs

NAME               FSTYPE   LABEL UUID                                   MOUNTPOINT
sda
├─sda1             xfs            7a72d0ab-c234-4ad6-82dd-aa53edff7c78   /boot
└─sda2             LVM2_mem       VqfMLI-x1MU-Ui0R-w2UI-3Qaq-na31-FoNKfL
  ├─rootvg-root    xfs            18817b75-3bd9-4ea7-b1b8-1d71b790ac45   /
  ├─rootvg-swap    swap           efc1e891-3ad9-4f18-8f03-a0d70f26c181   [SWAP]
  └─rootvg-worktmp xfs            2be7fb38-c1cf-4ce0-b4ee-11975ef745b2   /worktmp
sdb                LVM2_mem       kqHzI1-y8GI-SEkr-jhQn-BYAy-x2Tg-e1jdF3
├─dbvg-db2_home    xfs            f333bfb2-7a82-4bbe-aa20-325efc2febf8   /db2_home
├─dbvg-db2_vol     xfs            5702a9cd-a70e-4f48-9c5d-c29858dbaca2   /db2_vol
└─dbvg-dbtmp       xfs            9dcd5abd-ae6b-428b-b326-cf0c56d534a1   /dbtmp
sr0

# List UUID of disk
lsblk -l -o NAME,FSTYPE,MOUNTPOINT,UUID

NAME FSTYPE MOUNTPOINT UUID
sda
sda1 ext4   /boot      f830a3fa-1f94-42f4-9dca-5b5c077eab66
sda2 ext4   /          dcbdf18c-2fb4-426c-9dac-d13a45b7ebba
sda3 swap   [SWAP]     6f40f01b-e9ed-4092-9c65-1445d92ec9da
sda4 ext4              6df9a3a6-052e-41f3-b15a-cb258db0267f
OVM_SYS_REPO_PART_3600508b1001cbe65c99583659f085b36 (dm-0)
     ext4              6df9a3a6-052e-41f3-b15a-cb258db0267f
sr0

# Check the filesystem for the specified disk
lsblk --fs /dev/sdb

NAME            FSTYPE   LABEL UUID                                   MOUNTPOINT
sdb             LVM2_mem       kqHzI1-y8GI-SEkr-jhQn-BYAy-x2Tg-e1jdF3
├─dbvg-db2_home xfs            f333bfb2-7a82-4bbe-aa20-325efc2febf8   /db2_home
├─dbvg-db2_vol  xfs            5702a9cd-a70e-4f48-9c5d-c29858dbaca2   /db2_vol
└─dbvg-dbtmp    xfs            9dcd5abd-ae6b-428b-b326-cf0c56d534a1   /dbtmp

```

lshw

```bash
sudo lshw -short -class disk,volume

H/W path                 Device           Class          Description
====================================================================
/0/100/14/0/3/4/0.0.0    /dev/sda         disk           Mass-Storage
/0/100/14/0/3/4/0.0.0/0  /dev/sda         disk       
```

抹除磁碟

```bash
wipefs -a /<device-path>
```

##### last

```shell
# To check the last ten login attempts, you can pipe it with "head"
last | head -n 10

# using complete usernames and hostnames
last -w

# find the device used by the user
tty

# To find the last login by date,
last --since <date>
last --until <date>
last --since -2days

# find the last bad login attempts
sudo lastb
tail -f -n 100 /var/log/auth.log | grep -i failed

# find the last SSH logins
tail -f -n 100 /var/log/auth.log | grep -i sshd
sudo journalctl -r -u ssh | grep -i failed

# find last login times for all users
lastlog
lastlog -u <user>
```

##### 檢測虛擬平台類型

```shell
dmidecode -s system-manufacturer
systemd-detec-virt
virt-what
```

##### UEFI Firmware

```bash
# Method 1: Using systemD
systemctl reboot --firmware-setup

# Method 2: From the GRUB Bootloader
# Choose UEFI Firmware Settings from the GRUB menu instead of your Linux distro and press Enter.

# Method 3: Using the GRUB Command Line
# Press Esc in the GRUB menu to access the GRUB command-line interface (also known as GRUB shell).
> fwsetup
```

##### 硬體資訊

```
sudo lshw -short

H/W path                 Device           Class          Description
====================================================================
                                          system         NUC8i7HVK
/0                                        bus            NUC8i7HVB
/0/0                                      memory         64KiB BIOS
/0/2f                                     memory         16GiB System Memory
/0/2f/0                                   memory         8GiB SODIMM DDR4 Synchronous Unbuffered (Unregistered)
/0/2f/1                                   memory         8GiB SODIMM DDR4 Synchronous Unbuffered (Unregistered)
/0/34                                     memory         256KiB L1 cache
/0/35                                     memory         1MiB L2 cache
/0/36                                     memory         8MiB L3 cache
/0/37                                     processor      Intel(R) Core(TM) i7-8809G CPU @ 3.10GHz
/0/100                                    bridge         Xeon E3-1200 v6/7th Gen Core Processor Host Bridge/DRA
/0/100/1                                  bridge         Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe
/0/100/1/0               /dev/fb0         display        Polaris 22 [Radeon RX Vega M GH]
/0/100/1/0.1                              multimedia     Advanced Micro Devices, Inc. [AMD/ATI]
/0/100/1.1                                bridge         Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe
/0/100/1.1/0                              bus            ASMedia Technology Inc.
/0/100/1.1/0/0           usb3             bus            xHCI Host Controller
/0/100/1.1/0/1           usb4             bus            xHCI Host Controller
...
#
sudo lshw -html > HardwareSummary.html
```

Finding Number of Ram Slots

```shell
sudo dmidecode -t memory

sudo lshw -class memory
```

主機板型號

```bash
sudo dmidecode -t 1
sudo dmidecode -t 2
```

BIOS version

```bash
sudo dmidecode -s bios-version
```

Secure Boot Mode

```bash
mokutil --sb-state
```

More options

```bash
sudo lshw -C <option>
```

<table id="bkmrk-option-description-n" style="border-collapse:collapse;width:100%;border-width:1px;"><colgroup><col style="width:50%;"></col><col style="width:50%;"></col></colgroup><tbody><tr><td>Option  
</td><td>Description  
</td></tr><tr><td>network  
</td><td>Gets the details of the network hardware devices.</td></tr><tr><td>memory</td><td>Displays the details of RAM in your system.</td></tr><tr><td>storage</td><td>Prints details of the storage drives.</td></tr><tr><td>system</td><td>Gets the details of the motherboard and plug-and-play slots</td></tr><tr><td>multimedia</td><td>Details of the sound card of your system.</td></tr><tr><td>display</td><td>Know more about what is powering the display output.</td></tr><tr><td>bridge</td><td>Displays info about the PCIe bridges.</td></tr><tr><td>bus  
</td><td>It will list down buses and their details.</td></tr><tr><td>CPU  
</td><td>List the processor details</td></tr></tbody></table>

Inxi

```bash
# Install
sudo apt-get install inxi

# Check dependencies
inxi --recommends

# Shows Full Linux System Information
inxi -F

# Find Linux Laptop or PC Model Information
inxi -M

# Find Linux CPU and CPU Speed Information
inxi -C

# Find Graphic Card Information in Linux
inxi -G

# Find Audio/Sound Card Information in Linux
inxi -A
```

GUI Tools

```bash
# HardInfo
sudo apt-get install hardinfo
```

lspci

```bash
lspci
lspci -v -s <bus number>:<device number>.<function number>
```

lsusb

```bash
lsusb

# Check a USB device's maximum speed
# Speed vs USB version
# 12M    USB1.0
# 480M   USB2.0
# 5000M  USB 3.2 Gen 1 (aka USB 3.0)
# 10000M USB 3.2 Gen 2 (aka USB 3.1)
# 20000M USB 3.2 Gen 2x2
#
lsusb -t
```

##### 編譯與開發工具

RedHat/CentOS

```shell
# RedHat/CentOS 6
yum install make libtool autoconf subversion git cvs wget libogg-devel gcc gcc-c++ pkgconfig

# RedHat/CentOS 7
yum group install "Development Tools"
```

Ubuntu/Debian

```bash
apt-get install build-essential
```

##### dd

```shell
# 備份 MBR
dd if=/dev/hdx of=/path/to/image count=1 bs=512

# 光碟轉成 iso 檔
dd if=dev/cdrom of=/root/cd.iso

# 銷毀硬碟資料
dd if=/dev/urandom of=/dev/hda1

# 備份整個 USB-Flash
dd if=/dev/sdb | gzip > ./my-usb_flash.img.gz

# 還原 USB-Flash
gzip -dc ./my-usb_flash.img.gz | dd of=/dev/sdb

# 建立一個測試用的大檔案10GB
dd if=/dev/zero of=/path/to/image bs=1G count=10
# NOTE: 新版 Linux 可以改用指令
fallocate -l 1G test.img 

# Test network bandwidth between 2 Linux servers
dd if=/nas-mount-point/samplefile of=/dev/null bs=1M count=1024 iflag=direct
dd if=/dev/zero of=/nas-mount-point/samplefile bs=1M count=1024 oflag=direct 
# NOTE: the samplefile is greater than 1GB and the RAM is preferably more than 2GB.

# 複製到另一個相同 size 的硬碟
dd if=/dev/hda of=/dev/hdb conv=noerror,sync status=progress

# Quick benchmark test for writing 1GB file
dd if=/dev/zero of=/tmp/delme.dd bs=1024 count=1000000 status=progress
```

##### cat: 複製磁區

```bash
cat /dev/sda1 > /dev/sdb1
```

##### history

- [Bash History Display Date And Time For Each Command](https://www.cyberciti.biz/faq/unix-linux-bash-history-display-date-time/)
- [How to disable bash shell history in Linux](https://www.cyberciti.biz/faq/disable-bash-shell-history-linux/)
- [Parsing Bash history in Linux](https://www.redhat.com/sysadmin/parsing-bash-history)
- [Linux History Command with Advance Examples](https://trendoceans.com/history-command-in-linux/)

See time stamp in bash history

```shell
echo 'export HISTTIMEFORMAT="%F %T "' >> ~/.bash_profile
```

Prevent History

```bash
# Prevent History from Recording Any Executed Command
export HISTSIZE=0

# Prevent History from Storing Certain Strings
export HISTIGNORE="passwd:ftp: " 
```

##### 系統效能管理

使用 Swap 的 processes

```shell
for file in /proc/*/status ; do awk '/VmSwap|Name/{printf $2 " " $3}END{ print ""}' $file; done | sort -k 2 -n -r | less
```

OOM (Out of Memory) Killer

- [Linux Memory Overcommitment and the OOM Killer | Baeldung on Linux](https://www.baeldung.com/linux/memory-overcommitment-oom-killer)

##### Swap 管理

```shell
# 檢查目前的 swap 配置
free
swapon -s

# 開啟/關閉 swap
swapon /dev/sda3
swapoff /dev/sda3

# 製作一個 swap 系統
mkswap /dev/sda3 
```

make-swapfile.sh：1GB

```shell
#!/bin/bash

dd if=/dev/zero of=/swapfile bs=1024 count=1024k
chown root:root /swapfile
chmod 0600 /swapfile
mkswap /swapfile
swapon /swapfile
echo "/swapfile          swap            swap    defaults        0 0" >> /etc/fstab
sysctl vm.swappiness=10
echo vm.swappiness=10 >> /etc/sysctl.conf
free -h
cat /proc/sys/vm/swappiness
```

Extend the existing SWAP partition:

> NOTE: 不可在 Production 環境執行變更；如果要不影響生產服務，使用新增 SWAP partition 作法。

```
[root@tycitpdb05-a ~]# grep -i swap /etc/fstab
/dev/mapper/rootvg-swap swap                    swap    defaults        0 0

[root@tycitpdb05-a ~]# ls -al /dev/mapper/rootvg-swap
lrwxrwxrwx 1 root root 7 Aug 15 11:49 /dev/mapper/rootvg-swap -> ../dm-1

[root@tycitpdb05-a ~]# swapon -s
Filename                                Type            Size    Used    Priority
/dev/dm-1                               partition       4194300 0       -2

[root@tycitpdb05-a ~]# swapoff -v /dev/mapper/rootvg-swap
swapoff /dev/mapper/rootvg-swap

[root@tycitpdb05-a ~]# lvextend -L16G /dev/rootvg/swap
  Size of logical volume rootvg/swap changed from 4.00 GiB (1024 extents) to 16.00 GiB (4096 extents).
  Logical volume rootvg/swap successfully resized.
  
[root@tycitpdb05-a ~]# mkswap /dev/rootvg/swap
mkswap: /dev/rootvg/swap: warning: wiping old swap signature.
Setting up swapspace version 1, size = 16777212 KiB
no label, UUID=06aac9ae-9e8c-48bd-9f16-5f3d0c32b31f

[root@tycitpdb05-a ~]# swapon -v /dev/rootvg/swap
swapon /dev/rootvg/swap
swapon: /dev/mapper/rootvg-swap: found swap signature: version 1, page-size 4, same byte order
swapon: /dev/mapper/rootvg-swap: pagesize=4096, swapsize=17179869184, devsize=17179869184
```

##### xfs 檔案系統

- [How to Check and Repair XFS Filesystem in RHEL](https://www.2daygeek.com/repairing-xfs-file-system-in-rhel/)

修復 xfs 檔案系統

```shell
sudo mount -a
mount: /data: mount(2) system call failed: Structure needs cleaning.

sudo umount /data

# with '-n' option to perform a dry run
sudo xfs_repair -n /dev/sdb1
# repair the filesystem
sudo xfs_repair /dev/sdb1
```

xfs 備份與回復

```bash
# Install the xfsdump
dnf install xfsdump

# Create a full-backup
# -L: for dump session
# -M: for media in drive
# -f: the backup destination
xfsdump -L session_0 -M datapart -f /data/boot.xfsdump /boot

# Create a incremental backup with the level 1
# -l: the backup level (0-9)
xfsdump -l 1 -L session_1 -M datapart -f /data/boot.xfsdump1 /boot

# Restore a full-backup
xfsrestore -f /data/boot.xfsdump /test

# Restore an incremental backup
xfsrestore -r -f /data/boot.xfsdump /test
xfsrestore -r -f /data/boot.xfsdump1 /test
```

##### Linux Module

- [Linux: How to load a kernel module automatically at boot time](https://www.cyberciti.biz/faq/linux-how-to-load-a-kernel-module-automatically-at-boot-time/)

##### 監視檔案與目錄異動 (Monito File &amp; Directory)

- [Watchman – A File and Directory Watching Tool for Changes](https://www.tecmint.com/watchman-monitor-file-changes-in-linux/)
- [Watchman - A file watching service | Watchman (facebook.github.io)](https://facebook.github.io/watchman/)
- inotify-tools，這工具可以監控特定目錄的檔案異動情形（Linux 2.6.13 以上才有支援），以下連結的範例是用在特定目錄下，一旦有檔案或目錄的異動，立即呼叫 rsync 的備份 script。
- [fswatch - Monitor File and Directory Changes in Linux (tecmint.com)](https://www.tecmint.com/fswatch-monitor-file-changes-linux/)
- [AIDE - How to Check Integrity of File and Directory Using “AIDE” in Linux](https://www.tecmint.com/check-integrity-of-file-and-directory-using-aide-in-linux/)
- [Pyinotify](https://github.com/seb-m/pyinotify/wiki)

##### 比對兩個檔案

列出 A.txt 中没有包含 B.txt 内容的行

```bash
diff A.txt B.txt | grep "^<" | cut -c3-
```

##### 比對兩個目錄  


```shell
# tree
tree dir1
tree dir2

# diff
diff -q /path/to/dir1 /path/to/dir2
diff -q dir1 dir2
diff -qr dir1 dir2
diff -qrs dir1 dir2
```

##### 複製目錄架構但不包含檔案

```shell
# tree
tree -dfi --noreport dir1
tree -dfi --noreport dir1 | xargs -I{} mkdir -p "$HOME/Downloads/{}"
tree -a $HOME/Downloads/dir1

# find + xargs
find dir1 -type d
find dir1 -type d | xargs -I{} mkdir -p "$HOME/Documents/{}" 
tree -a $HOME/Documents/dir1

# find + exec
find dir1 -type d -exec mkdir -p "$HOME/Desktop/{}" \;
```

##### 記憶體管理

- [How to check memory utilization and usage in Linux](https://www.cyberciti.biz/faq/how-to-check-memory-utilization-in-linux/)
- [How to Accurately Check Memory Consumption of a Process in Linux](https://blog.sysxplore.com/p/how-to-accurately-check-process-memory-usage-in-linux)
- [How to Clear RAM Cache and Memory in Linux \[Safe Methods\]](https://www.tecmint.com/clear-ram-memory-cache-buffer-and-swap-space-on-linux/)

```bash
# Using meminfo
cat /proc/meminfo
grep -E --color 'Mem|Cache|Swap' /proc/meminfo
```

- `MemTotal`, Total usable RAM (i.e., physical RAM minus a few reserved bits and the kernel binary code).
- `MemFree`, The sum of LowFree+HighFree.
- `MemAvailable`, (since Linux 3.14) An estimate of how much memory is available for starting new applications, without swapping.
- `Buffers`, Relatively temporary storage for raw disk blocks that shouldn’t get tremendously large (20MB or so).
- `Cached`, In-memory cache for files read from the disk (the page cache). Doesn’t include SwapCached.
- `SwapCached`, Memory that once was swapped out, is swapped back in but still also is in the swap file.

```bash
# Using free
free -h
# Repeat printing free command output every N seconds.
free -s 5 -c 10

```

- `total`, Total installed memory
- `used`, Used memory (calculated as total – free – buffers – cache)
- `free`, Unused memory (MemFree and SwapFree in /proc/meminfo)
- `shared`, Memory used mostly by tmpfs (Shmem in /proc/meminfo)
- `buffers`, Memory used by kernel buffers (Buffers in /proc/meminfo)
- `cache`, Memory used by the page cache and slabs (Cached and SReclaimable in /proc/meminfo)
- `buff/cache`, Sum of buffers and cache
- `available`, Estimation of how much memory is available for starting new applications, without swapping.

```
# Using vmstat
vmstat -w
```

- `swapd`, the amount of virtual memory used.
- `free`, the amount of idle memory.
- `buff`, the amount of memory used as buffers.
- `cache`, the amount of memory used as cache.
- `inact`, the amount of inactive memory. (-a option)
- `active`, the amount of active memory. (-a option)
- `si`, Amount of memory swapped in from disk (/s).
- `so`, Amount of memory swapped to disk (/s).

##### 特殊字元的檔名  


Dash

```bash
# The filename with -- or -
rm -i -v -- -foo
rm -i -v -- --foo
rm -i -v ./-foo

# The filename with -- and whitespaces
rm -i -v -- '-- My Resume . txt'
rm -i -v -- '/path/to/dir/-- My Resume . txt'
rm -i -v -- "/path/to/dir/-- My Resume . txt"

# Using find
find . -name '--my-FileNameGoes-Here' -delete
find /path/to/directory/ -name '---filename with a white spaces' --delete
```

##### cp: 強制覆蓋多檔案

```bash
# 因為內建 alias=cp -i，以致於 -f 參數無效
yes | cp -r /source /target
```

##### System Locale  


```bash
# view information about the current installed locale
locale
localectl status

# view more information about an environmental variable
locale -k LC_TIME

# display a list of all available locales
locale -a

# Set System Locale
## Using the commands
## The following command sets LANG to en_IN.UTF-8 and removes definitions for LANGUAGE.
sudo update-locale LANG=LANG=en_IN.UTF-8 LANGUAGE
## Or
sudo localectl set-locale LANG=en_IN.UTF-8

sudo update-locale LC_TIME=en_IN.UTF-8
## Or
sudo localectl set-locale LC_TIME=en_IN.UTF-8

## Using the profile
vi ~/.bash_profile

LANG="en_IN.utf8"
export LANG

```

##### 快速清空一個大檔

1. Preserve File Permissions and Ownership
2. Maintain Symbolic Links
3. Avoid Disruption to Services or Applications
4. File Locking Issues
5. Efficiency in Log Management

```bash
> access.log
: > access.log
true > access.log
cat /dev/null > access.log
cp /dev/null access.log
dd if=/dev/null of=access.log
echo -n "" > access.log
truncate -s 0 access.log
```

##### tar 指令

- [18 Tar Command Examples on Linux](https://www.tecmint.com/tar-command-examples-linux/)

備份整個系統

```bash
# 1. Switch to single user mode
# NOTE: single mode 會中斷網路功能, 只能本機 console 登入
init 1

# 2. Tar up the whole system
tar zcpvf /backups/fullbackup.tar.gz --directory=/ --exclude=proc --exclude=dev --exclude=sys --exclude=boot --exclude=run --exclude=etc/fstab --exclude=backups .

# 3. Once this completes copy the tar file over to the root directory your new machine
# 4. Take a snapshot of your new machine. This way if things go wrong you can revert to the snapshot and try again.
# 5.Extract the tarball on your new machine
cd /
tar -zxvpf /path/to/fullbackup.tar.gz

```

##### Stress Test

- [stress-ng](https://github.com/ColinIanKing/stress-ng)
    - [Linux 使用 Stress-ng 測試 CPU、記憶體、磁碟 I/O 滿載情況教學與範例 - Office 指南 (officeguide.cc)](https://officeguide.cc/linux-stress-ng-cpu-memory-hard-drive-full-load-tutorial-examples/)
    - [How to Test CPU and Memory Load with Stress &amp; Stress-ng - Shouts.dev](https://shouts.dev/articles/how-to-test-cpu-and-memory-load-with-stress)
    - [How to Stress Test Your Linux CPU for High Load (tecmint.com)](https://www.tecmint.com/linux-cpu-load-stress-test-with-stress-ng-tool/)

stress

```bash
stress --cpu 2 --io 3 --vm 4 --vm-bytes 512M --timeout 10m
```

sysbench

```bash
# size 必須超過 RAM
# max-time 單位秒
sysbench --test=fileio --file-total-size=10G prepare
sysbench --test=fileio --file-total-size=10G --file-test-mode=rndrw --init-rng=on --max-time=300 --max-requests=0 run

```

##### 指令的路徑

```bash
# with command
command -v <cmd-name>

# with which
which <cmd-name>
```

##### getent: 檢視系統資訊

```bash
# The same as 'cat /etc/passwd' and 'cat /etc/shadow'
getent passwd
getent passwd <user-name>
getent shadow
getent group
getent group <group-name>

# /etc/hosts
getent hosts

# /etc/services
getent services
getent services <service-name>

# /etc/networks
getent networks
```

##### 增加 inode 數量

XFS

- KB: [https://access.redhat.com/solutions/41492](https://access.redhat.com/solutions/41492)
- 可線上增加或減小

```bash
# 預設值: 25%
# 調整檔案系統空間 30% 儲存 inode
xfs_growfs -m 30 /<mount-point>
```

EXT4/EXT3

- [https://www.tecmint.com/increase-disk-inode-number-in-linux/](https://www.tecmint.com/increase-disk-inode-number-in-linux/)[ ](https://www.tecmint.com/increase-disk-inode-number-in-linux/)
- 建立檔案系統時，就已經決定了 inode 的數量

```bash
# byte-per-inode: 16384 (by default), 8192, 4096
# 數值愈小；inode 數量愈大
mkfs.ext4 -i <byte-per-inode> /dev/device


# 對於大檔案的使用，可以用以下語法，減小 inode 數量
mkfs.ext4 -T largefile /dev/device
```

##### sysctl

```bash
# See current sysctl overrides
sysctl --system

# Apply the changes
sysctl -p /etc/sysctl.d/ipv6.conf

# Check for current settings
sysctl -a
```

# Q & A

##### NFS 遠端目錄無法卸載

> Umount a NFS directory with the error: device is busy

當遭遇某些異常情況時，原先的 NFS 目錄可能無法用指令 `umount` 正常卸載，且主機因為還在營運生產中，不能執行重開機時，下述的指令可能會有幫助。

```shell
# 檢查那些程序及用戶使用該目錄
fuser -m -v <mount_point or device_path>
lsof | grep <mount_point or device_path>
lsof +f -- <path-to-dir>

# 刪除無效的程序
fuser -i -k <mount_point or device_path>
fuser -km <mount_point or device_path>
umount <mount_point>

# 以上都無效，可重啟 NFS Client daemon，但其他 NFS 目錄也會被迫中斷。
service netfs stop
service netfs start

# 網友提供的另一個方法
umount -l <mount_point>
umount -f <mount_point>
```

##### 檢查線上 NFS 掛載參數, nfs 版本

```shell
cat /proc/mounts | grep nfs 
```

##### 無法掛載 NFS 目錄

> <div>mount: wrong fs type, bad option, bad superblock on 10.22.33.19:/DBS_LNXDBBAK,</div><div> missing codepage or helper program, or other error</div><div> (for several filesystems (e.g. nfs, cifs) you might</div><div> need a /sbin/mount.&lt;type&gt; helper program)</div><div>  
> </div><div> In some cases useful info is found in syslog - try</div><div> dmesg | tail or so.</div>

```bash
# For RedHat 7.x
# Check if nfs-utils was installed
rpm -qa | grep nfs-utils

yum install nfs-utils
```

##### Remove Shared Memory (ipcs)

```shell
# see all shared memory segments
# status: dest -> destroyed
# nattach: how many application being attached to the shared memory. 
ipcs -m
ipcs -m -i <shmid>

# remove the shared memory segment
ipcrm shm <shmid>
ipcrm -m <shmid>
```

##### CentOS Upgrade Kernel: linux-firmware package files conflicts

說明：要升級 CentOS 7.6 Kernel to CentOS 7.7，遇到 linux-firmware 套件衝突問題。

```shell
rpm -ivh kernel-3.10.0-1062.el7.x86_64.rpm

warning: kernel-3.10.0-1062.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID fd431d51: NOKEY
error: Failed dependencies:
        linux-firmware >= 20190429-72 is needed by kernel-3.10.0-1062.el7.x86_64
        

rpm -ivh linux-firmware-20190429-72.gitddde598.el7.noarch.rpm kernel-3.10.0-1062.el7.x86_64.rpm

warning: linux-firmware-20190429-72.gitddde598.el7.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY
warning: kernel-3.10.0-1062.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID fd431d51: NOKEY
Preparing...                          ################################# [100%]
        file /usr/lib/firmware/amd-ucode/microcode_amd_fam17h.bin from install of linux-firmware-20190429-72.gitddde598.el7.noarch conflicts with file from package linux-firmware-20180911-69.git85c5d90.el7.noarch
        file /usr/lib/firmware/amd-ucode/microcode_amd_fam17h.bin.asc from install of linux-firmware-20190429-72.gitddde598.el7.noarch conflicts with file from package linux-firmware-20180911-69.git85c5d90.el7.noarch
        
# Solution:
rpm -Uvh linux-firmware-20190429-72.gitddde598.el7.noarch.rpm
rpm -ivh kernel-3.10.0-1062.el7.x86_64.rpm
```

##### Adjust reserved 5% of disk space by default

Reduce it to 2%

```shell
sudo tune2fs -m2 /dev/sda1
```

##### du 與 df 顯示的使用空間有很大的落差   


一般來說，這類問題通常發生在目錄裡有日誌檔，雖然清空了日誌檔內容， df 的使用空間遠大於 du 的使用空間。

原因是 du 不會對開啟中的日誌檔案進行計算，如果有開啟中的大檔案佔用目錄，du 是無法發現的。

解決方法：

1. 使用指令 `lsof /path/to/dir` 檢查目錄裡有沒有程式正在執行中
2. 確實關閉程式後再用 `du` 檢查一遍

##### CLI 字型太小

某些情況命令列模式的字型過小，影響實際操作，可以執行以下指令：

```bash
setfont ter-132n
```

# SELinux

安全增強式 Security-Enhanced Linux（SELinux）是一個在內核中實踐的強制存取控制（MAC）安全性機制。SELinux 首先在 CentOS 4 出現，並在其後的 CentOS 發行版本獲得重大改善。這些改善代表用 SELinux 解決問題的方法亦隨著時間而改變。

##### 基本指令

To check if SELinux is enabled

```
# sestatus
SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   permissive
Mode from config file:          permissive
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Max kernel policy version:      31

# getenforce
Permissive
```

To temporarily set SELinux to Enforcing/Permissive

```
# setenforce 1   // Enforcing

# setenforce 0   // Permissive

```

Permanently change SELinux  
Edit the file /etc/selinux/config

```
## Change this line 
SELinux=disabled
```

##### Find Files with SELinux Security Context

```bash
# With the find command
find ~/UbuntuMint -type f -context '*httpd_sys_content_t*' -name '*.txt'

# With the ls command
ls -Z | grep 'object_r:user_home_t' | grep '\.txt$'
```

##### 延伸閱讀

- [A sysadmin's guide to SELinux: 42 answers to the big questions](https://opensource.com/article/18/7/sysadmin-guide-selinux)
- [CentOS: SELinux](https://wiki.centos.org/zh-tw/HowTos/SELinux)
- [Securing Linux with SELinux (or AppArmor)](https://linuxblog.io/securing-linux-selinux-apparmor/)

# patch & diff

##### 單一檔案

```shell
# 建立更新檔
diff -uN my_original.code my_fixed.code > my.patch

# 套用更新至指定檔
patch my.code < my.patch

# 套用更新至原始檔
patch < my.patch

# 回復更新前至指定檔
patch -RE my.code < my.patch

# 回復更新至原始檔
patch -RE < my.patch
```

##### 連續多個檔案

```shell
# 建立更新檔
diff -rupN olddir newdir > patch.dir

# 套用更新檔
patch -p1 < patch.dir

# 回復更新前
patch -RE -p1 < patch.dir
```

##### 找出內容不一致的檔案

```shell
# 目錄一: asterisk_orig/
# 目錄二: asterisk_patched/

cd asterisk_orig; find ./ -name "*.c" -exec md5sum -b {} \; > ../asterisk_orig.md5; cd ../
cd asterisk_patched; find ./ -name "*.c" -exec md5sum -b {} \; > ../asterisk_patched.md5; cd ../

diff asterisk_orig.md5 asterisk_patched.md5 

273c273
< 894a111d1efa5901471820e203503039 *./channels/chan_sip.c
---
> 00116baac23473049b5801c9287fb4be *./channels/chan_sip.c
```

> 注意：前提是 asterisk\_orig 與 asterisk\_patched 目錄架構及檔案名稱必須完全一樣，這通常用來找出 asterisk\_patched 目錄內被更新過了哪些 .c 檔案。

##### diff 與 curl 的應用

```shell
diff <(curl http://somesite/file1) <(curl http://somesite/file2)
```

##### 比對兩個目錄

```bash
diff -q directory-1/ directory-2/
diff -qr directory-1/ directory-2/
```

##### Tutorials

- [Understanding of diff output](https://unix.stackexchange.com/questions/81998/understanding-of-diff-output)

# ps

##### 以 PID 查詢

```shell
ps -fp <PID>
ps -fp <PID#1>,<PID#2>,<PID#3> 
```

##### 分析 CPU/Memory 使用

CPU 使用率最高前10排名

```shell
# for CentOS
ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%cpu | head -11

ps aux | head -1; ps aux | sort -rn +2 | head -10
ps -auxf | sort -nr -k 3 | head -10

# for AIX 7.2
ps aux | head -1; ps aux | sort -rn -k 3 | head -10
```

Memory 使用率最高前10排名

```shell
# for CentOS
ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%mem | head -11

ps aux | head -1; ps aux | sort -rn +3 | head
ps aux | head -1; ps aux | sort -rn -k 4 | head
ps -auxf | sort -nr -k 4 | head -10

ps -eo cmd,pid,ppid,%mem,%cpu --sort=-%mem | head -n 6

ps aux  | awk '{print $6/1024 " MB\t\t" $11}'  | sort -rn | head -25

```

##### 優先權最高的程序

```shell
ps -eakl | sort -n +6 | head
# or
ps -eal | sort -n -k 7 | head
```

##### 程序以 nice 值排序

```shell
ps -eakl | sort -n +7

ps -eal | sort -n -k 8
```

##### 程序以執行時間排序

```shell
ps vx | head -1;ps vx | grep -v PID | sort -rn +3 | head -10

ps vx | head -1;ps vx | grep -v PID | sort -rn -k 4 | head -10
```

##### 程序以 I/O 排序

```shell
ps vx | head -1; ps vx | grep -v PID | sort -rn +4 | head -10

ps vx | head -1; ps vx | grep -v PID | sort -rn -k 5 | head -10
```

##### 等待中的程序

```shell
ps vg | head -1; ps vg | grep -w wait
```

##### 指定 PID 的啟動時間與總執行時間

```shell
ps -p <PID> -o etime,start
ps -p <PID> -o etimes
```

##### 每個用戶的程序數量

```shell
ps -ef | awk '{print$1}' | sort | uniq -c | sort -nr
```

##### 顯示用戶 vivek 的所有程序

```shell
ps -U vivek -u vivek u
```

##### 指定要顯示的欄位

```shell
ps -eo pid,tid,class,rtprio,ni,pri,psr,pcpu,stat,wchan:14,comm 
ps axo stat,euid,ruid,tty,tpgid,sess,pgrp,ppid,pid,pcpu,comm 
ps -eopid,tt,user,fname,tmout,f,wchan
```

##### 顯示程序 lighttpd 的 pid

```shell
ps -C lighttpd -o pid=
```

##### 顯示 pid XXX 的程序名稱

```shell
ps -p 55977 -o comm=
```

# Rsync

##### 限制頻寬

限制傳檔的網路頻寬

```bash
--bwlimit=30000
```

> 30000 = 30000 KB/ps = 30 MB/ps

##### Dry Run

```bash
rsync -avh -n <source> <destination>
```

##### 檔案與資料夾同步

本機內不同個資料夾做複製或同步

```bash
rsync -avh --delete --exclude='path1/to/exclude' --exclude='path2/to/exclude' source/ destination
rsync -avh /source/*.log /destination
```

- source/ 來源目錄+斜線 會複製目錄內的內容
- source 來源目錄不加斜線 會複製整個目錄
- --delete 來源檔案被刪除時也會同步到目的端 (*NOTE: 如果來源位置不是目錄，而是檔案 \*.log， 即使來源檔案被刪除，已同步的目的檔案仍會保留*)

##### 進度條

遠端複製大檔案需要續傳與進度條

```bash
rsync --partial --progress --rsh=ssh <local_file> user@host:<remote_file>
```

##### 遠端同步

本機至遠端

```bash
rsync -avzh /root/rpmpkgs root@192.168.0.141:/root/
```

遠端至本機

```bash
rsync -avzh root@192.168.0.141:/root/rpmpkgs /tmp/myrpms
```

透過 SSH

```bash
# Copy a File from a Remote Server to a Local Server
rsync -avzhe ssh root@192.168.0.141:/root/anaconda-ks.cfg /tmp

# Copy a File from a Local Server to a Remote Server
rsync -avzhe ssh backup.tar.gz root@192.168.0.141:/backups/
```

##### Include &amp; Exclude

指定附檔名

```bash
rsync -avh /source/*.log /destination

rsync -avh --include='*.log' --exclude='*' /source/ /destination
```

例外檔案清單

```bash
rsync -avh --exclude-from=/root/exclude-files.txt /source/ /destination
```

##### 複製整個系統

複製 Linux 整個系統

- [How to Clone a CentOS Server with Rsync](https://www.tecmint.com/clone-centos-server/)
- [How To Backup Your Entire Linux System Using Rsync](https://ostechnix.com/backup-entire-linux-system-using-rsync/)

```bash
mount /dev/sdb1 /mnt
rsync -aAXv / --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"} /mnt
```

With a exclusion file

exclude-files.txt:

```
/boot
/dev
/tmp
/sys
/proc
/backup
/etc/fstab
/etc/mtab
/etc/mdadm.conf
/etc/sysconfig/network*
```

Cloning a Linux Server

```bash
sudo rsync -vPa -e 'ssh -o StrictHostKeyChecking=no' --exclude-from=/root/exclude-files.txt / REMOTE-IP:/
```

##### How does rsync work

[https://michael.stapelberg.ch/posts/2022-07-02-rsync-how-does-it-work/](https://michael.stapelberg.ch/posts/2022-07-02-rsync-how-does-it-work/)

##### Other solutions

- [Syncthing](https://github.com/syncthing/syncthing) - It is a free and open-source file syncing application used to sync files between multiple remote devices over the internet. It works on peer-to-peer architecture and exchanges the data automatically between two devices. 
    - [How to Install Syncthing Remote File Synchronization Software on Debian 11](https://www.howtoforge.com/how-to-install-syncthing-on-debian-11/)

# screen

##### screen 常用指令

```
啟用 screen
#> screen
or
#> screen -S <session-name>

列出所有 session
#> screen -ls

取回 (resume) 某個 session
#> screen -r <session-id>

無法取回 session 時，先 deattach 在 resume
#> screen -D <session-id>
#> screen -r <session-id>

強制終止 session
#> screen -X -S <session-id> kill

結束 screen 的 session
#> exit

進入複製模式，可複製視窗上指定的文字，操作方式：
使用方向鍵任意移動游標，按一次 space 開始標記游標的文字，再按一次 space 結束標記文字並離開複製模式。
Ctrl-a + [
要貼上剛複製的內容
Ctrl-a + ]  

重新載入設定但不重啟
Ctrl-a + :source ~/.screenrc
```

##### screen 用來連接 RS232

```shell
screen /dev/ttyACM0 115200
```

TIP: 如果出現 *screen is terminating*，須改用 sudo。

##### 環境配置檔

~/.screenrc：

```
# Start message
startup_message off

# Disable vbell
vbell off

# Set hardstatus always on
hardstatus alwayslastline " %-Lw%{= Bw}%n%f %t%{-}%+Lw %=| %M %d %0c:%s "

# Set default encoding using utf8
defutf8 on

# 
term xterm

# Set the number of scrollback lines
#defscrollback 10000

# Add Keyborad bindings
bind , title

# Fix termcapinfo for xterm to allow column resizing
# xterm emulation is used by PuTTY
# Uncomment the line below if running on CentOS 6.x/7.x 
#termcapinfo  xterm Z0=\E[?3h:Z1=\E[?3l:is=\E[r\E[m\E[2J\E[H\E[?7h\E[?1;4;6l

# Fix termcapinfo for cleaning the terminal after detach
# Uncomment the line below if running on AIX 7.x
#termcapinfo xterm*|rxvt*  te=\E[?1049l:ti=\E[?1049h:
```

> TIPs:
> 
> start\_message 關閉開啟後的提示訊息
> 
> vbell 關閉可以避免按 tab 出現閃畫面
> 
> hardstatus 在畫面下方固定顯示視窗編號，非常實用的功能
> 
> defscrollback 回捲內容的行數

#### 進入 screen 模式後指令

##### 基本操作

離開 screen 但不中止 session (deattach)

Ctrl-a + d

離開 screen 並中止 session

Ctrl-a + k

##### 複製模式

進入複製模式，可複製視窗上指定的文字，操作方式：  
使用方向鍵任意移動游標，按一次 space 開始標記游標的文字，再按一次 space 結束標記文字並離開複製模式。

Ctrl-a + \[

貼上剛複製的內容

Ctrl-a + \]

##### 多視窗操作

水平切割 &lt;Ctrl-a&gt; + |

垂直切割 &lt;Ctrl-a&gt; + S

切換視窗 &lt;Ctrl-a&gt; + tab

目前視窗最大化 &lt;Ctrl-a&gt; + Q

移除目前視窗 &lt;Ctrl-a&gt; + X ;這不會終止 session

##### 多個終端機操作

新增一個 screen 終端機

&lt;Ctrl-a&gt; + c

或者是在目前的 screen 終端機再執行一次 screen  
NOTE：如果已經 deattach，再執行 screen 時，會開啟另一個不同的 screen 終端機

- 每新增一個視窗，新的編號會從最小且沒有在使用的號碼開始，例如： 
    - 現有 #0, #1, #2，新增後的編號為 #3
    - 現有 #0, #2, #3，新增後的編號為 #1
- 不管新增多少個視窗，只要沒有做 deattach，所有開啟的視窗都屬於同一個 screen 終端機

檢視目前已開啟的所有視窗

&lt;Ctrl-a&gt; + w

- 視窗編號從 0 開始依序
- 顯示 ＊ 符號為目前使用的視窗編號

移動至前一個或後一個終端機

&lt;Ctrl-a&gt; + p 前一個 (視窗編號減一)

&lt;Ctrl-a&gt; + n 後一個 (視窗編號加一)

&lt;Ctrl-a&gt; + 視窗編號

#### Q &amp; A

##### 用 Putty 連線，每次 attach 時，視窗都會自動縮回預設大小

將設定檔以下這行移除註解

```
termcapinfo  xterm Z0=\E[?3h:Z1=\E[?3l:is=\E[r\E[m\E[2J\E[H\E[?7h\E[?1;4;6l 
```

##### 用 tab 時，畫面都會閃一下，該如何解決

在 $HOME 目錄內新增 .screenrc

```
vbell off
```

##### 如何很快的確認，目前是否在 screen 終端機內

Ctrl + a + a

- 如果已經 deattach 終端機，畫面不會有任何變化。
- 如果是在 screen 終端機內，有多個開啟視窗時，會切換到前一個視窗；只開啟一個視窗時，畫面會出現 'No other window'。

# Learning 拾人牙慧

##### 系統管理 

- Core Dump 
    - [Creating and debugging Linux dump files](https://opensource.com/article/20/8/linux-dump)
    - \[gdb\] [GDB debugging tutorial for beginners](https://linuxconfig.org/gdb-debugging-tutorial-for-beginners)
    - \[gdb\] [A hands-on tutorial for using the GNU Project Debugger](https://opensource.com/article/21/1/gnu-project-debugger)
    - [How to disable core dumps in Linux including systemd](https://www.cyberciti.biz/faq/disable-core-dumps-in-linux-with-systemd-sysctl/)
- Kernel crash dump (kdump) 
    - [Chapter 7. Kernel crash dump guide](https://access.redhat.com/documentation/zh-tw/red_hat_enterprise_linux/7/html/kernel_administration_guide/kernel_crash_dump_guide)
- Misc 
    - [How To Easily And Safely Manage Cron Jobs Using Crontab UI In Linux](https://ostechnix.com/how-to-easily-and-safely-manage-cron-jobs-in-linux/)
    - [How To Access Linux Partitions From Windows 10,11 (ext3,ext4,Linux Reader)](https://ostechnix.com/how-to-access-linux-partitions-from-windows-10/)
    - [BCC - Dynamic Linux Tracing Tools for Network &amp; Monitoring (tecmint.com)](https://www.tecmint.com/bcc-best-linux-performance-monitoring-tools/)
- Kernel Panic 
    - RedHat: [How to handle a Linux kernel panic](https://www.redhat.com/sysadmin/handle-kernel-panic)
    - [How to Trigger and Fix a Linux Kernel Panic (Educational)](https://www.tecmint.com/fix-kernel-panic-linux/)
- Zram - 是一種壓縮式 SWAP 空間且儲存在 memory 裡。用途是可以用來改善低記憶體配置伺服器的效能。 
    - [Running Out of RAM on Linux? Add Zram Before Upgrading!](https://linuxblog.io/running-out-of-ram-linux-add-zram/)

##### 檔案救援

- [testdisk](https://www.cgsecurity.org/wiki/TestDisk)
    - [TestDisk：在 Linux 中救回硬碟裡刪除的檔案](http://blogger.gtwang.org/2012/07/testdisk.html)
    - [How To Recover Deleted Files In Linux \[Beginner’s Guide\]](https://itsfoss.com/recover-deleted-files-linux/)
    - [Recover Data In Linux After Accidentally Deleting Your OS - OSTechNix](https://ostechnix.com/recover-data-in-linux-after-accidentally-deleting-your-os/)
- [R-Linux](https://www.r-studio.com/zhhk/free-linux-recovery/) - 是一套適用於Linux 作業系統與許多 Unix 所用 Ext2/Ext3/Ext4 FS 檔案系統的免費檔案復原公用程式。R-Linux 使用與 R-Studio 相同的 InteligentScan 技術與靈活的參數設定，提供 Linux 平台上最快最可靠的檔案復原。
- [Digler](https://github.com/ostafen/digler) - Go Deep. Get Back Your Data

##### FTP/SFTP

- [Setting Up a Secure FTP Server using SSL/TLS on Ubuntu](https://www.tecmint.com/secure-ftp-server-using-ssl-tls-on-ubuntu/)

##### 檔案加密

- \[gpg\][How To Encrypt File on Linux](https://devconnected.com/how-to-encrypt-file-on-linux/)

##### Web Terminal

- [butterfly](https://github.com/paradoxxxzero/butterfly)
    - [butterfly – web terminal based on websocket and tornado](https://www.linuxlinks.com/butterfly-web-terminal-websocket-tornado/)
- [GateOne ](https://github.com/liftoff/GateOne)is an HTML5 web-based terminal emulator and SSH client. 
    - [Gate One Documentation](https://liftoff.github.io/GateOne/index.html)
- [Bastillion](https://github.com/bastillion-io/Bastillion) is a web-based SSH console that centrally manages administrative access to systems.
- [ShellHub](https://www.shellhub.io/) is a modern SSH server for remotely accessing Linux devices via the command line (using any SSH client) or a web-based user interface. 
    - 更方便管理：遠端電腦可透過 Web(HTTP) 管理多部 Linux 主機。
    - 更高的安全性：被管理的 Linux 主機只需要對 ShellHub 主機開放 SSH 存取，就可以使任意的遠端透過 Web(HTTP) 存取該 Linux 的 Shell。
    - [Bypass your Linux firewall with SSH over HTTP](https://opensource.com/article/20/7/linux-shellhub)
- [Guacamole](https://osslab.tw/books/remote-desktop-%E9%81%A0%E7%AB%AF%E6%A1%8C%E9%9D%A2/page/guacamole) 遠端連線平台，包含 RDP、VNC、SSH。

##### GlusterFS

一個 Open Source 可以橫向擴充的檔案儲存解決方案

- [Migrate NFS to GlusterFS and nfs-ganesha](https://blogs.oracle.com/linux/migrate-nfs-to-glusterfs-and-nfs-ganesha)
- [Gluster 架構](http://www.l-penguin.idv.tw/book/Gluster-Storage_GitBook/gluster_intra/gluster_arch.html)
- [How to Setup GlusterFS Storage on CentOS 7 / RHEL 7](https://www.linuxtechi.com/setup-glusterfs-storage-on-centos-7-rhel-7/)
- [利用 glusterfs 達成目錄即時自動同步與高可用性](https://cyrilwang.blogspot.com/2013/09/glusterfs.html)
- [CentOS 7 上的 GlusterFS 儲存叢集](https://wiki.centos.org/zh-tw/HowTos/GlusterFSonCentOS)

##### 網路管理

- [How to Limit Network Bandwidth Usage in Linux Using Trickle](https://www.tecmint.com/limit-linux-network-bandwidth-usage-with-trickle/)

##### Load Balancing

- [Multi-tier load-balancing with Linux](https://vincent.bernat.ch/en/blog/2018-multi-tier-loadbalancer)

##### Hardening Security

- [40 Linux Server Hardening Security Tips \[2021 edition\]](https://www.cyberciti.biz/tips/linux-security.html)

##### Files and Directories Monitoring

- [Pyinotify – Monitor Filesystem Changes in Real-Time in Linux](https://www.tecmint.com/pyinotify-monitor-filesystem-directory-changes-in-linux/)

##### GPU Monitoring

- [Linux GPU Monitoring and Diagnostic Commands Line Tools](https://www.cyberciti.biz/open-source/command-line-hacks/linux-gpu-monitoring-and-diagnostic-commands/)

##### Log Files Analysis

- [logwatch耙梳服務日誌，產出整合報表一目瞭然](https://www.netadmin.com.tw/netadmin/zh-tw/technology/7C5DD7C5E3934D4BBF8B8E0679ACBB7A)
- [lnav - Awesome terminal log file viewer for Linux and Unix - nixCraft (cyberciti.biz)](https://www.cyberciti.biz/open-source/lnav-linux-unix-ncurses-terminal-log-file-viewer/)
- [Gonzo](https://gonzo.controltheory.com/) - The Go based TUI log analysis tool

##### Secure Boot

- \[RH\][How to use Secure Boot to validate startup software](https://www.redhat.com/sysadmin/secure-boot-systemtap)

##### YubiKey with Linux

- \[Video\] [YubiKey Complete Getting Started Guide](https://www.youtube.com/watch?v=INi-xKpYjbE)

##### Kernel Compiling

- [A Guide to Compiling the Linux Kernel All By Yourself](https://itsfoss.com/compile-linux-kernel/)

##### Distrobox

使用 Docker 或 Podman 技術，可以同時安裝多個各種 Linux 系統，尤其方便做軟體測試。

- [Distrobox](https://distrobox.it/)
- [Forget distro hopping: How to use any Linux distribution on one PC](https://www.howtogeek.com/forget-distro-hopping-this-is-how-you-can-use-any-linux-distro-on-your-linux-pc/)
- [Distrobox lets me run Ubuntu apps on Fedora and Arch apps on Debian, all without touching my base system](https://www.xda-developers.com/distrobox-lets-run-ubuntu-apps-fedora-arch-apps-debian-without-touching-my-base-system/)

# fsck

##### 教學連結

- [Check and Repair Filesystem Errors With fsck Command in Linux](https://linuxhandbook.com/fsck-command/)

# Cheat Sheets

#### Online CheatSheets

- 中文：[各種速查表線上快速查詢](https://github.com/skywind3000/awesome-cheatsheets)
- 英文：[各類速查表線上快速查詢](https://github.com/LeCoupa/awesome-cheatsheets)

#### Curl

- [curl-cheat-sheet.pdf](http://www.osslab.tw/attachments/9)

#### Git

- [git-for-subversion-cheat-sheet.pdf](https://osslab.tw/attachments/48)
- [git-cheatsheet-EN-dark.pdf](https://osslab.tw/attachments/50)
- [working-with-branches-in-git-cheat-sheet.pdf](https://osslab.tw/attachments/52)

[![Git_cheat_sheet.jpeg](https://osslab.tw/uploads/images/gallery/2022-10/scaled-1680-/git-cheat-sheet.jpeg)](https://osslab.tw/uploads/images/gallery/2022-10/git-cheat-sheet.jpeg)

[![Git_cheat_sheet2.png](https://osslab.tw/uploads/images/gallery/2022-10/scaled-1680-/git-cheat-sheet2.png)](https://osslab.tw/uploads/images/gallery/2022-10/git-cheat-sheet2.png)

[![git_commands_cheatsheet.jpeg](https://osslab.tw/uploads/images/gallery/2022-10/scaled-1680-/git-commands-cheatsheet.jpeg)](https://osslab.tw/uploads/images/gallery/2022-10/git-commands-cheatsheet.jpeg)

[![Git_cheat_sheet2.jpeg](https://osslab.tw/uploads/images/gallery/2022-10/scaled-1680-/git-cheat-sheet2.jpeg)](https://osslab.tw/uploads/images/gallery/2022-10/git-cheat-sheet2.jpeg)

[![git_reference_sheet.jpg](https://osslab.tw/uploads/images/gallery/2023-08/scaled-1680-/git-reference-sheet.jpg)](https://osslab.tw/uploads/images/gallery/2023-08/git-reference-sheet.jpg)


#### Markdown

- [markdown\_cheat\_sheet.pdf](http://www.osslab.tw/attachments/10)

[![markdown_cheatsheet.jpeg](https://osslab.tw/uploads/images/gallery/2022-10/scaled-1680-/markdown-cheatsheet.jpeg)](https://osslab.tw/uploads/images/gallery/2022-10/markdown-cheatsheet.jpeg)

#### MySQL

- [mariadb.pdf](http://www.osslab.tw/attachments/19)
- mysql\_func\_cheatsheet.jpg

[![mysql_func_cheatsheet.jpg](https://osslab.tw/uploads/images/gallery/2022-10/scaled-1680-/mysql-func-cheatsheet.jpg)](https://osslab.tw/uploads/images/gallery/2022-10/mysql-func-cheatsheet.jpg)

#### VS Code

- [VS\_Code-keyboard-shortcuts-linux.pdf](http://www.osslab.tw/attachments/11)
- [vs-code\_cheat-sheet\_windows.pdf](https://osslab.tw/attachments/51)

#### Python

- [python3.pdf](http://osslab.tw/attachments/28)

[![python_cheatsheet.jpeg](https://osslab.tw/uploads/images/gallery/2022-10/scaled-1680-/python-cheatsheet.jpeg)](https://osslab.tw/uploads/images/gallery/2022-10/python-cheatsheet.jpeg)

[![python_for_data_science.png](https://osslab.tw/uploads/images/gallery/2022-10/scaled-1680-/python-for-data-science.png)](https://osslab.tw/uploads/images/gallery/2022-10/python-for-data-science.png)

[![Python_Pentest_CheetSheet.jpg](https://osslab.tw/uploads/images/gallery/2022-10/scaled-1680-/python-pentest-cheetsheet.jpg)](https://osslab.tw/uploads/images/gallery/2022-10/python-pentest-cheetsheet.jpg)

[![python3_basics_cheatsheet.jpeg](https://osslab.tw/uploads/images/gallery/2022-10/scaled-1680-/python3-basics-cheatsheet.jpeg)](https://osslab.tw/uploads/images/gallery/2022-10/python3-basics-cheatsheet.jpeg)

[![python3.jpeg](https://osslab.tw/uploads/images/gallery/2022-11/scaled-1680-/python3.jpeg)](https://osslab.tw/uploads/images/gallery/2022-11/python3.jpeg)

#### Yum

- [Yum\_Command\_cheat\_sheet.pdf](http://www.osslab.tw/attachments/30)

#### Raspberry Pi

- [RassberryPi\_cheat\_sheet.pdf](http://www.osslab.tw/attachments/31)

#### Linux

##### Linux commands

- [CLI-cheat-sheet.pdf](https://osslab.tw/attachments/49)

[![Linux_commands.jpeg](https://osslab.tw/uploads/images/gallery/2022-10/scaled-1680-/linux-commands.jpeg)](https://osslab.tw/uploads/images/gallery/2022-10/linux-commands.jpeg)

[![linux_command_for_git_client.png](https://osslab.tw/uploads/images/gallery/2022-10/scaled-1680-/linux-command-for-git-client.png)](https://osslab.tw/uploads/images/gallery/2022-10/linux-command-for-git-client.png)

[![Base_Linux_Commands.jpg](https://osslab.tw/uploads/images/gallery/2022-10/scaled-1680-/base-linux-commands.jpg)](https://osslab.tw/uploads/images/gallery/2022-10/base-linux-commands.jpg)

##### Crontab 

[![crontab_cheatsheet.jpeg](https://osslab.tw/uploads/images/gallery/2022-10/scaled-1680-/crontab-cheatsheet.jpeg)](https://osslab.tw/uploads/images/gallery/2022-10/crontab-cheatsheet.jpeg)

[![crontab2_cheatsheet.jpeg](https://osslab.tw/uploads/images/gallery/2022-10/scaled-1680-/crontab2-cheatsheet.jpeg)](https://osslab.tw/uploads/images/gallery/2022-10/crontab2-cheatsheet.jpeg)

[![cron_job.jpg](https://osslab.tw/uploads/images/gallery/2025-07/scaled-1680-/cron-job.jpg)](https://osslab.tw/uploads/images/gallery/2025-07/cron-job.jpg)

##### Linux Redirections

[![linux_redirections_s.jpeg](https://osslab.tw/uploads/images/gallery/2023-12/scaled-1680-/linux-redirections-s.jpeg)](https://osslab.tw/uploads/images/gallery/2023-12/linux-redirections-s.jpeg)

##### LVM

[![linux_lvm_s.jpg](https://osslab.tw/uploads/images/gallery/2024-02/scaled-1680-/linux-lvm-s.jpg)](https://osslab.tw/uploads/images/gallery/2024-02/linux-lvm-s.jpg)

##### File permission

[![linux_file_permission.jpg](https://osslab.tw/uploads/images/gallery/2024-02/scaled-1680-/linux-file-permission.jpg)](https://osslab.tw/uploads/images/gallery/2024-02/linux-file-permission.jpg)

##### Linux Boot Process

[![LinuxBootProcess.jpeg](https://osslab.tw/uploads/images/gallery/2023-12/scaled-1680-/linuxbootprocess.jpeg)](https://osslab.tw/uploads/images/gallery/2023-12/linuxbootprocess.jpeg)

#### Docker

[![Docker_CheatSheet.jpg](https://osslab.tw/uploads/images/gallery/2022-10/scaled-1680-/docker-cheatsheet.jpg)](https://osslab.tw/uploads/images/gallery/2022-10/docker-cheatsheet.jpg)

[![Docker-Command-Cheatsheet-3.png](https://osslab.tw/uploads/images/gallery/2022-10/scaled-1680-/docker-command-cheatsheet-3.png)](https://osslab.tw/uploads/images/gallery/2022-10/docker-command-cheatsheet-3.png)

#### Network

- [TCP-notes-cheat-sheet.pdf](https://osslab.tw/attachments/54)

##### DNS Records

[![dns-records_cheatsheet.jpeg](https://osslab.tw/uploads/images/gallery/2022-10/scaled-1680-/dns-records-cheatsheet.jpeg)](https://osslab.tw/uploads/images/gallery/2022-10/dns-records-cheatsheet.jpeg)

##### Common Ports

[![common_ports.jpeg](https://osslab.tw/uploads/images/gallery/2023-12/scaled-1680-/common-ports.jpeg)](https://osslab.tw/uploads/images/gallery/2023-12/common-ports.jpeg)

##### IPv4 vs IPv6

[![ipv6_vs_ipv4.jpg](https://osslab.tw/uploads/images/gallery/2024-03/scaled-1680-/ipv6-vs-ipv4.jpg)](https://osslab.tw/uploads/images/gallery/2024-03/ipv6-vs-ipv4.jpg)

##### NAT

[![how_nat_works.jpeg](https://osslab.tw/uploads/images/gallery/2024-07/scaled-1680-/how-nat-works.jpeg)](https://osslab.tw/uploads/images/gallery/2024-07/how-nat-works.jpeg)

##### ARP

[![how_arp_works.jpg](https://osslab.tw/uploads/images/gallery/2024-07/scaled-1680-/how-arp-works.jpg)](https://osslab.tw/uploads/images/gallery/2024-07/how-arp-works.jpg)

#### Machine Learning

##### Pandas

[![pandas_cheatsheet.jpeg](https://osslab.tw/uploads/images/gallery/2022-10/scaled-1680-/pandas-cheatsheet.jpeg)](https://osslab.tw/uploads/images/gallery/2022-10/pandas-cheatsheet.jpeg)

[![python4datascience-pandas.jpeg](https://osslab.tw/uploads/images/gallery/2023-02/scaled-1680-/python4datascience-pandas.jpeg)](https://osslab.tw/uploads/images/gallery/2023-02/python4datascience-pandas.jpeg)

##### Numpy

[![python4datascience-numpy.jpeg](https://osslab.tw/uploads/images/gallery/2023-02/scaled-1680-/python4datascience-numpy.jpeg)](https://osslab.tw/uploads/images/gallery/2023-02/python4datascience-numpy.jpeg)

##### Scikit-Learn

[![python4datascience-scikit.jpeg](https://osslab.tw/uploads/images/gallery/2023-02/scaled-1680-/python4datascience-scikit.jpeg)](https://osslab.tw/uploads/images/gallery/2023-02/python4datascience-scikit.jpeg)

##### Keras

[![python4datascience-keras.jpeg](https://osslab.tw/uploads/images/gallery/2023-02/scaled-1680-/python4datascience-keras.jpeg)](https://osslab.tw/uploads/images/gallery/2023-02/python4datascience-keras.jpeg)

#### SQL

[![SQL_cheatsheet.jpeg](https://osslab.tw/uploads/images/gallery/2022-11/scaled-1680-/sql-cheatsheet.jpeg)](https://osslab.tw/uploads/images/gallery/2022-11/sql-cheatsheet.jpeg)

[![sql_join.jpg](https://osslab.tw/uploads/images/gallery/2024-12/scaled-1680-/sql-join.jpg)](https://osslab.tw/uploads/images/gallery/2024-12/sql-join.jpg)

#### Cloud Platform  


##### AWS

[![AWS_services_2006-2022.jpg](https://osslab.tw/uploads/images/gallery/2023-08/scaled-1680-/aws-services-2006-2022.jpg)](https://osslab.tw/uploads/images/gallery/2023-08/aws-services-2006-2022.jpg)

##### Big Data Piplines

[![google_aws_azure.jpg](https://osslab.tw/uploads/images/gallery/2024-10/scaled-1680-/google-aws-azure.jpg)](https://osslab.tw/uploads/images/gallery/2024-10/google-aws-azure.jpg)

#### Database

[![types_of_databases.jpeg](https://osslab.tw/uploads/images/gallery/2023-12/scaled-1680-/types-of-databases.jpeg)](https://osslab.tw/uploads/images/gallery/2023-12/types-of-databases.jpeg)

[![database_scaling.jpg](https://osslab.tw/uploads/images/gallery/2025-09/scaled-1680-/database-scaling.jpg)](https://osslab.tw/uploads/images/gallery/2025-09/database-scaling.jpg)

#### More

##### Html

[![html.jpeg](https://osslab.tw/uploads/images/gallery/2022-10/scaled-1680-/html.jpeg)](https://osslab.tw/uploads/images/gallery/2022-10/html.jpeg)

##### DMARC Protocol

[![DMARC.jpg](https://osslab.tw/uploads/images/gallery/2025-07/scaled-1680-/dmarc.jpg)](https://osslab.tw/uploads/images/gallery/2025-07/dmarc.jpg)

# Systemd

#### 簡介

Linux 的各項服務管理一直都是用 SysV Init Script，**Systemd** 是新的管理工具，在 CentOS 7 開始已經有支援。

設定上比 SysV Init 簡單許多，指令的操作差異不大。

線上教學：

- [How to enable rc.local shell script on systemd while booting Linux system](https://www.cyberciti.biz/faq/how-to-enable-rc-local-shell-script-on-systemd-while-booting-linux-system/)
- \[RHEL\] [Overview of systemd for RHEL 7](https://access.redhat.com/articles/754933)
- \[RHEL\] [How to configure a command, script, or daemon to run after boot has finished in RHEL 7, 8](https://access.redhat.com/solutions/1751263)
- [How to Find Systemd or Any Other init System in Linux (debugpoint.com)](https://www.debugpoint.com/systemd-or-init/)

支援的 Linux：

- CentOS 7+
- Ubuntu 16.04+

其他類似應用：

- [Supervisor](http://supervisord.org/ "http://supervisord.org/")  
    這個被使用在 Ubuntu 9.10，Mac OS X (10.4/10.5/10.6)，Solaris (10 for Intel) 及 FreeBSD 6.1。系統環境需要有 Python 2.4，但不支援 Python 3。

相關目錄：

- /etc/systemd/system 客製的服務啟動檔位置
- /lib/systemd/system 內建系統的服務啟動檔位置

#### How to determine

```
↪  ps --no-headers -o comm 1   
systemd
```

#### 服務設定檔

/etc/systemd/system/backup.service

```
[Unit]
Description=Backup daemon

[Service]
Type=simple
ExecStart=/path/to/backup

[Install]
WantedBy=multi-user.target
```

TIP:

> multi-user.target 這是表示 Run Level 3
> 
> 更多資訊可以前往 http://0pointer.de/blog/projects/systemd-for-admins-3.html

##### 新增一個服務設定檔

- [How to create a systemd service in Linux (linuxhandbook.com)](https://linuxhandbook.com/create-systemd-services/)
- [How to Create a Systemd Service Unit in Linux (tecmint.com)](https://www.tecmint.com/create-systemd-service-linux/)

/etc/systemd/system/freepbx.service

```
[Unit]
Description=Freepbx
After=mariadb.service
 
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/sbin/fwconsole start
ExecStop=/usr/sbin/fwconsole stop
 
[Install]
WantedBy=multi-user.target
```

將服務設為自動啟用

```
systemctl enable freepbx
```

systemctl 指令

```bash
# 檢視服務設定檔內容
systemctl cat freepbx.service

# 檢視服務底層參數
systemctl show freepbx.service

# 編輯服務設定檔
systemctl edit freepbx.service
```

#### 服務管理

啟動服務

```bash
# Reload Systemd
systemctl daemon-reload

# 啟動服務
systemctl start <service-name>

# 檢查服務狀態
systemctl status <service-name>
systemctl is-active <service-name> 
systemctl is-enabled <service-name>

# 關閉服務
systemctl stop <service-name>

# 啟用:自動啟動
systemctl enable <service-name>

# 關閉:自動啟動
systemctl disable <service-name>

# 列出設為自動啟用的服務
systemctl list-unit-files --type=service --state=enabled

# 檢視服務的來源內容
systemctl cat <service-name>
```

#### 檢視服務清單

```bash
# View status of all services and units
systemctl
systemctl | grep ssh

# list active services
systemctl list-units --type=service
systemctl --type service
systemctl -t service

# List all the RUNNING systemd services
systemctl list-units --type=service --state=running

# List all LOADED systemd services including the inactive ones
systemctl list-units --type=service
systemctl --type service

# List all the INACTIVE systemd services
systemctl list-units --all --type=service --state=inactive

# List all the INSTALLED systemd services
systemctl list-unit-files --type=service

# List all systemd services that will be run at each boot automatically
systemctl list-unit-files --type=service --state=enabled
```

#### 關機與開機

```bash
# Halt the system
systemctl halt

# Poeroff the system
systemctl poweroff

# Reboot the system
systemctl reboot

# Reboot the system into UEFI settings
systemctl reboot --firmware-setup

```

#### 文字或視窗模式切換

```shell
# Find which target unit is used by default
# GUI mode: graphical.target
# Text mode: multi-user.target
systemctl get-default
ls -l /etc/systemd/system/default.target

# To change boot target to the text mode
sudo systemctl set-default multi-user.target

# To change boot target to the GUI mode
sudo systemctl set-default graphical.target

# Optional: Listing all systemd targets
systemctl list-units --type target

# To immediately switch to the GUI login
systemctl isolate graphical.target
```

#### Journalctl

檢視系統日誌

- [How to Use journalctl to Read Linux System Logs](https://www.howtogeek.com/499623/how-to-use-journalctl-to-read-linux-system-logs/)

```shell
# View the log of the specified service
journalctl -u <service-name>
journalctl -f -u <service-name>        # -f View live updates
journalctl -e -u <service-name>        # -e Jump to the end page of the log
journalctl -n 50 -u <service-name>     # -n Show the most recent n number of log lines

# 快速統計/檢視所有服務錯誤日誌清單
# <行數統計> <服務的指令>
# 可加入自動檢查通知
journalctl --no-pager --since today \
--grep 'fail|error|fatal' --output json|jq '._EXE' | \
sort | uniq -c | sort --numeric --reverse --key 1

# view journal entries for time zones
journalctl --utc

# view only errors, warnings, etc in journal logs
# Error codes
# 0: emergency
# 1: alerts
# 2: critical
# 3: errors
# 4: warning
# 5: notice
# 6: info
# 7: debug
journalctl -p 0

# When you specify the error code, it shows all messages from that code and above. 
# For example, if you specify the below command, it shows all messages with priority 2, 1 and 0
journalctl -p 2

# view journal logs for a specific boot
journalctl --list-boots

# To view a specific boot number you the first number or the boot ID as below.
journalctl -b -45
journalctl -b 8bab42c7e82440f886a3f041a7c95b98

# 上一次開機日誌
# -p 3 顯示 error 等級訊息
journalctl -xb -p 3

# view journal logs for a specific time, date duration
journalctl --since "2020-12-04 06:00:00"
journalctl --since "2020-12-03" --until "2020-12-05 03:00:00"
journalctl --since yesterday
journalctl --since 09:00 --until "1 hour ago"
journalctl --since "1 hour ago"
journalctl --since "1 hour ago" -u cron

# see Kernel specific journal logs
journalctl -k

# see journal logs for a service name
journalctl -u NetworkManager.service
# By PID
journalctl _PID=1111
journalctl -o verbose _PID=1111

# If you do not know the service name, you can use the below 
# command to list the systemd services in your system.
systemctl list-units --type=service

# view journal logs for a user, group
id -u debugpoint
journalctl _UID=1000 --since today

# view journal logs for an executable
journalctl /usr/bin/gnome-shell --since today

# Check the disk usage
journalctl --disk-usage

# Set the log clearance
sudo journalctl --vacuum-time=2d
sudo journalctl --vacuum-size=500M
```

#### Timer

```bash
systemctl list-timers
systemctl list-unit-files --type timer
```

#### Application firewalls

An application firewall, unlike a gateway (router) or system level firewall, is meant to limit the networking of a single application. It can be used to prevent a compromised service from seeing into the local network, prevent programs from calling home, plug metadata leaks, or more tightly control a program’s network access.

The `systemd` firewall directives is built on Linux kernel features. The required Kernel features might not be enabled in your specific environment (especially when using a custom kernel or container). Testing is key, as it is with any network filter and security solution. You should always test to verify that your firewall set up blocks and allows the traffic you specify.

- [systemd application firewalls by example](https://www.ctrl.blog/entry/systemd-application-firewall.html)

#### Run a custom script

##### After mounting NFS  


Listing mount systemd units

```bash
sudo systemctl list-units --type=mount
```

/root/bin/nfs-optimiation.sh:

```bash
#!/bin/bash
device_number=$(stat -c '%d' /cbz_efs/)
((major = (device_number & 0xFFF00) >> 8))
((minor = (device_number & 0xFF) | ((device_number >> 12) & 0xFFF00)))
_dev="/sys/class/bdi/$major:$minor/read_ahead_kb"
echo "DRVICE: $_dev"
echo "CURRENT VALUE: $(cat $_dev)"
echo "$0 called after mount /cbz_efs/"
echo 15000 > "$_dev"
```

Creating a new service unit

```bash
sudo chmod +x -v /root/bin/nfs-optimiation.sh

# Create a new systemd unit name after-cbz_efs-mount
sudo systemctl edit --force --full after-cbz_efs-mount
```

unit: after-cbz\_efs-mount

```
[Unit]
Description=Script to run after fstab mount for /cbz_efs/
Requires=cbz_efs.mount
After=cbz_efs.mount
RequiresMountsFor=/cbz_efs
 
[Service]
ExecStart=/root/bin/nfs-optimiation.sh
 
[Install]
WantedBy=cbz_efs.mount
```

Activating the unit

```bash
sudo systemctl daemon-reload
sudo systemctl enable after-cbz_efs-mount
sudo systemctl start after-cbz_efs-mount
sudo systemctl status after-cbz_efs-mount
```

##### After starting network  


Create: `/etc/systemd/system/multi-user.target.wants/connection.service`

```
[Unit]
Description = making network connection up
After = network.target

[Service]
ExecStart = /root/scripts/conup.sh

[Install]
WantedBy = multi-user.target
```

Script: `/root/scripts/conup.sh`

```bash
#!/bin/bash
nmcli connection up enp0s3
```

Activating the service

```bash
sudo systemctl daemon-reload
sudo systemctl enable connection.service
sudo systemctl start connection.service
sudo systemctl status connection.service
```

#### 其他附屬指令

##### coredumpctl

```
# 列出系統所有 core dump
coredumpctl

# 列出指定 program 的 core dump
coredumpctl dump <program-name>

# 列出指定 PID
coredumpctl dump _PID=XXX

# 分析特定 core dump 的內容
coredumpctl gdb <PID>

# 預設 core dump files 路徑
/var/lib/systemd/coredump
```

# CentOS/RedHat Tips

#### 停用不必要的服務

CentOS 7/8: secure-linux.sh

```shell
#!/usr/bin/env bash
# Author: A.Lang(alang.hsu[AT]gmail.com)
# File: secure-linux.sh
# Created by 2019/3/1
#
#
SVC_LIST="
############# Start #############
#
## bluetooth services
bluetooth

## SELinux
auditd

## Disk Monitoring
smartd

## Linux Virtualization with KVM
libvirtd

## ABRT - Automatic Bug Reporting Tool
abrtd
abrt-ccpp

## More Services
firewalld
avahi-daemon
#chronyd
cups
autofs
#
#
############# End #############
"

# function report_result <service name> <status>
report_result() {
    printf "%20s ..................%s\n" "$1" "[$2]"
}

## Main program
#echo "$SVC_LIST" | sed -e '/^#/d' -e '/^$/d'
echo
echo "The following services will be disabled:"
echo "$SVC_LIST" | sed -e '/^#/d' -e '/^$/d' | while read name
do
   chkconfig $line off 2>/dev/null
   systemctl disable $name 2>/dev/null
   if [ $? -eq 0 ]; then
      report_result $name "OK"
   else
      report_result $name "**"
   fi
done

## Disable SELinux
SVC="SELinux"
sed -i 's/SELINUX=.*$/SELINUX=disabled/' /etc/selinux/config 2>/dev/null
if [ $? -eq 0 ]; then
    report_result $SVC "OK"
else
    report_result $SVC "**"
fi

echo "All done, please reboot NOW."
```

CentOS 6: secure-linux.sh

```shell
#!/usr/bin/env bash
# Author: A.Lang(alang.hsu[AT]gmail.com)
# File: secure-linux.sh
# Created by 2011-11-27
# Updated by 2016-11-2
#
SVC_LIST="
############# Start #############
#
## Disable if the system is ACPI capable
apmd

## bluetooth services
bluetooth
hidd

## IR device
irda

## only needed the first time a system is configured
firstboot
readahead_early

## SELinux
auditd
setroubleshoot

## Disk Monitoring
smartd

## More Services
anacron
avahi-daemon
avahi-daemon
cups
isdn
ip6tables
iptables
iscsi
iscsid
mcstrans
pcscd
autofs
yum-updatesd
NetworkManager
#
#
############# End #############
"

# function report_result <service name> <status>
report_result() {
    printf "%20s ..................%s\n" "$1" "[$2]"
}

## Main program
#echo "$SVC_LIST" | sed -e '/^#/d' -e '/^$/d'
echo
echo "The following services will be disabled:"
echo "$SVC_LIST" | sed -e '/^#/d' -e '/^$/d' | while read line
do
   chkconfig $line off 2>/dev/null
   if [ $? -eq 0 ]; then
      report_result $line "OK"
   else
      report_result $line "**"
   fi
done

## Disable SELinux
SVC="SELinux"
sed -i 's/SELINUX=.*$/SELINUX=disabled/' /etc/selinux/config 2>/dev/null
if [ $? -eq 0 ]; then
    report_result $SVC "OK"
else
    report_result $SVC "**"
fi

echo "All done, please reboot NOW."
```

#### Remove virbr0 network interface

Case 1: Not using libvirtd service and virbr0 interface

```shell
# Stop and Disable the service
systemctl stop libvirtd.service
systemctl disable libvirtd.service

# Reboot the host to remove the virbr0 interface
systemctl reboot
```

Case 2: Using libvirtd and dont want "virbr0"

```shell
# List the default network set-up for the virtual machines
virsh net-list

Name       State    Autostart    Persistent
----------------------------------------------------------
default    active   yes          yes

# Destroy the network default.
virsh net-destroy default

Network default destroyed

# Permanently remove the default vitual network from the configuration.
virsh net-undefine default

Network default has been undefined

# The interface virbr0 is now gone. You can verify it in the ifconfig or ip command output.
ifconfig virbr0

virbr0: error fetching interface information: Device not found
```

Case 3: Removing virbr0 interface on running machines ( non-persistence across reboots )

```shell
# First, list out the virtual bridge interfaces available on the system using the below command.
brctl show

bridge name     bridge id               STP enabled     interfaces
virbr0          8000.5254003008b6       yes             virbr0-nic

# Make the bridge interface down before removal.
ip link set virbr0 down

# Now, remove the bridge
brctl delbr virbr0

# check if the bridge is removed
brctl show

bridge name     bridge id               STP enabled     interfaces
```

Removing lxcbr0 interface

```shell
# change the below line in /etc/sysconfig/lxc. This will be effective after reboot. change the line from

USE_LXC_BRIDGE="true"
# to
USE_LXC_BRIDGE="false"

# remove the lxcbr0 bridge interface for the running system
brctl show
ip link set lxcbr0 down
brctl delbr lxcbr0
brctl show
```

#### New Changes to RedHat 9

Official: [https://access.redhat.com/documentation/en-us/red\_hat\_enterprise\_linux/9/html/considerations\_in\_adopting\_rhel\_9/assembly\_security\_considerations-in-adopting-rhel-9](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/considerations_in_adopting_rhel_9/assembly_security_considerations-in-adopting-rhel-9)

1. [SSH from RHEL 9 to RHEL 6 systems does not work](https://access.redhat.com/solutions/6816771)
2. The following algorithms are disabled in the LEGACY, DEFAULT and FUTURE crypto policies provided with RHEL 9: 
    - TLS older than version 1.2 (since RHEL 9, was &lt; 1.0 in RHEL 8)
    - DTLS older than version 1.2 (since RHEL 9, was &lt; 1.0 in RHEL 8)
    - DH with parameters &lt; 2048 bits (since RHEL 9, was &lt; 1024 bits in RHEL 8)
    - RSA with key size &lt; 2048 bits (since RHEL 9, was &lt; 1024 bits in RHEL 8)
    - DSA (since RHEL 9, was &lt; 1024 bits in RHEL 8)
    - 3DES (since RHEL 9)
    - RC4 (since RHEL 9)
    - FFDHE-1024 (since RHEL 9)
    - DHE-DSS (since RHEL 9)
    - Camellia (since RHEL 9)
    - ARIA
    - SEED
    - IDEA
    - Integrity-only cipher suites
    - TLS CBC mode cipher suites using SHA-384 HMAC
    - AES-CCM8
    - All ECC curves incompatible with TLS 1.3, including secp256k1
    - IKEv1 (since RHEL 8)
3. SCP not supported in RHEL 9
4. OpenSSH root password login disabled by default
5. GnuTLS no longer supports TPM 1.2
6. Support for disabling SELinux through `/etc/selinux/config` has been removed. If your scenario requires disabling SELinux, add the `selinux=0` parameter to your kernel command line.
7. Network **teams** are deprecated
8. RHEL 9 does not contain the `network-scripts` package that provided the deprecated legacy network scripts in RHEL 8. To configure network connections in RHEL 9, use **NetworkManager**.

#### sosreport 系統診斷工具

Redhat/CentOS 從 4.6 以後版本，都內建這個指令，一旦系統需要故障排除時，可以透過這指令蒐集系統的各項資訊，然後提供給專家做分析用。

基本用法：

```bash
# 安裝 sosreport 如果沒安裝時
yum install sos

# 蒐集系統資訊
#sosreport
# NOTE: 過程中會耗時幾分鐘時間，而且會使用到目錄 /tmp，指令完成後會產生一個 sosreport-*.tar.bz2 檔案。
# Updated by 2023/2/1
# 從 RHEL 8 起，以下指令取代了 sosreport，而且輸出位置改成 /var/tmp。
sos report

# 指定暫存檔目錄，某些情況預設的 /tmp 空間不足時可改用此指令
sos report --tmp-dir /path/to/directory

## 進階用法
# 列出所有支援的 plugins
sos report -l

# 排除特定 plugins 不蒐集
sos report -n kvm,amd
```

> sosreport 更多用法可以參閱 [https://access.redhat.com/solutions/3592](https://access.redhat.com/solutions/3592) 。

專家：如何分析這個資訊

- 解開檔案後，目錄 sos\_reports 內有一個 sosreport.html 網頁，開啟後可以透過網頁搜尋方式，快速找到各項的系統資訊。
- [xsos](https://github.com/ryran/xsos "https://github.com/ryran/xsos") - Github 的專案

#### RedHat Linux 原機升級

- [What are RHEL in-place upgrades? (redhat.com)](https://www.redhat.com/en/blog/what-are-rhel-place-upgrades?channel=/en/blog/channel/red-hat-enterprise-linux)

#### FAQ  


##### setlocale error

> -bash: warning: setlocale: LC\_CTYPE: cannot change locale (zh\_TW.big5): No such file or directory  
> -bash: warning: setlocale: LC\_CTYPE: cannot change locale (zh\_TW.big5): No such file or directory

檢查系統是否已支援語系 zh\_TW.big5

```bash
locale -a | grep zh_TW.big5
```

如果沒有，安裝以下套件

```bash
yum install glibc-all-langpacks.x86_64
```

驗證時帳號需登出再登入。

##### RHEL8: Stopping the message when ssh login to the host  


> Activate the web console with: systemctl enable --now cockpit.socket

Solution:

```bash
ln -sfn /dev/null /etc/motd.d/cockpit
```

##### Systemd messages: Created slice, Starting Session

新裝的系統 /var/log/messages 會出現大量的以下日誌：

> Jul 24 08:50:01 example.com systemd: Created slice user-0.slice.  
> Jul 24 08:50:01 example.com systemd: Starting Session 150 of user root.  
> Jul 24 08:50:01 example.com systemd: Started Session 150 of user root.  
> Jul 24 09:00:01 example.com systemd: Created slice user-0.slice.  
> Jul 24 09:00:02 example.com systemd: Starting Session 151 of user root.  
> Jul 24 09:00:02 example.com systemd: Started Session 151 of user root.

原因：這些訊息是正常現象。只要任何帳號登入或有 Cron job 被執行就會有類似的日誌。官方文件：[Logs flooded with systemd messages: Created slice, Starting Session - Red Hat Customer Portal](https://access.redhat.com/solutions/1564823)

解決：如果想抑制(suppress)這類訊息，有很多方法可以達到，其中之一是使用 rsyslog，步驟如下：

```bash
echo 'if $programname == "systemd" and ($msg contains "Starting Session" or $msg contains "Started Session" or $msg contains "Created slice" or $msg contains "Starting user-" or $msg contains "Starting User Slice of" or $msg contains "Removed session" or $msg contains "Removed slice User Slice of" or $msg contains "Stopping User Slice of") then stop' >/etc/rsyslog.d/ignore-systemd-session-slice.conf

systemctl restart rsyslog
```

##### useradd errors

執行 `useradd` 時，雖然帳號新增成功但會出現告警：

> \[sss\_cache\] \[sysdb\_domain\_cache\_connect\] (0x0010): DB version too old \[0.23\], expected \[0.24\] for domain implicit\_files!  
> Higher version of database is expected!  
> In order to upgrade the database, you must run SSSD.  
> Removing cache files in /var/lib/sss/db should fix the issue, but note that removing cache files will also remove all of your cached credentials.  
> Could not open available domains

Solution:

- KB: [https://access.redhat.com/solutions/7031304](https://access.redhat.com/solutions/7031304)

```bash
systemctl stop sssd ; rm -f /var/lib/sss/db/* ; systemctl start sssd
```

# ssh

##### 測試驗證 sshd 服務的設定

```bash
# 重啟 sshd 前先驗證設定
# 沒有任何內容輸出，表示設定沒有錯誤
# 指令必須使用絕對路徑
/usr/sbin/sshd -t

# 詳細驗證模式
# 可檢視詳細設定檔參數
/usr/sbin/sshd -T
```

##### 檢視連線參數的設定

```shell
$ ssh -F ~/.ssh/config -G remote-host-name
user root
hostname 173.82.136.138
port 22
addressfamily any
batchmode no
canonicalizefallbacklocal yes
canonicalizehostname false
challengeresponseauthentication yes
checkhostip yes
compression no
...
```

##### ~/.ssh/config

```
Host my-k8s
    HostName 192.168.31.81
    User alang
    Port 22
    IdentityFile ~/.ssh/alang@mint_rsa
    LocalForward 11000 localhost:11000
```

自動套用金鑰檔

- %r : remote username
- %h : remote hostname

```
Host *
    IdentityFile ~/.ssh/id_%r@%h
    IdentityFile ~/.ssh/id_ed25519
    IdentityFile ~/.ssh/id_rsa
```

##### 建立金鑰檔

```shell
mkdir -p $HOME/.ssh
chmod 0700 $HOME/.ssh
ssh-keygen -t rsa

# Specify 4096 bits (default 2048)
# Specify the filename of the key file 
# (default $HOME/.ssh/id_rsa is private key, $HOME/.ssh/id_rsa.pub is public key)
ssh-keygen -t rsa -b 4096 -f ~/.ssh/my-vps-cloud.key -C "My Comment"
```

##### 以金鑰方式連線

```shell
# By custom key file
ssh -i /path/to/the-key-file root＠hostname

# By default key file ~/.ssh/id_rsa
ssh root＠hostname
```

##### 複製公鑰檔至遠端主機上

指令一：從主機 A 上執行

```
ssh-copy-id user＠remote-host-ip 
or
ssh-copy-id -f -i $HOME/.ssh/id_rsa.pub user＠remote-host-ip
```

指令二：從主機 A 上執行

```
cat ~/.ssh/id_rsa.pub | ssh user＠remote-host-ip "mkdir -p ~/.ssh && cat >>  ~/.ssh/authorized_keys"
```

從主機 B 上執行，以手動方式上傳 id\_rsa.pub：

```
cd ~/.ssh
mv id_rsa.pub host-A-hostname.pub
cat host-A-hostname.pub >> authorized_keys
chmod 0700 ~/.ssh
chmod 0640 authorized_keys
```

NOTE:

> 如果 .ssh 目錄裡已經有 authorized\_keys 檔案，可以另存一個檔名加上 2，例如 authorized\_keys2

測試連線: 從主機 A 上執行

```editable
ssh <remote-userB>@<remote-hostB-name>
```

不需要輸入密碼就可以登入。

限制來源IP: 編輯 authorized\_keys

```
# Allow login from 192.168.2.0/24 subnet but not from 192.168.2.25
from="!192.168.2.25,192.168.2.*" ssh-ed25519 my_random_pub_key_here vivek＠nixcraft
# Allow login from *.sweet.home but not from router.sweet.home
from="!router.sweet.home,*.sweet.home" ssh-ed25519 my_random_pub_key_here vivek＠nixcraft
```

##### 公鑰遺失，重新產生  


```bash
# Generate a public key from a private key
ssh-keygen -y -f ~/.ssh/id_rsa > ~/.ssh/id_rsa.pub

# View the fingerprint of the key
# NOTE: If the private key and the public key is key pair, the fingerprint of them are the same.
ssh-keygen -l -f ~/.ssh/id_rsa.pub
ssh-keygen -l -f ~/.ssh/id_rsa
```

##### 指令自動補全

自動搜尋 `~/.ssh/config` 的連線資訊

編輯 `~/.bashrc`

```
complete -o default -o nospace -F _ssh ssh
```

##### sshpass

整合 shell 做自動化的指令

Install sshpass

```bash
sudo apt-get install sshpass     #[On Debian, Ubuntu and Mint]
sudo yum install sshpass         #[On RHEL/CentOS/Fedora and Rocky Linux/AlmaLinux]
sudo emerge -a sys-apps/sshpass  #[On Gentoo Linux]
sudo pacman -S sshpass           #[On Arch Linux]
sudo zypper install sshpass      #[On OpenSUSE]    
```

1\. ssh 執行指令

```shell
# Use the -p (this is considered the least secure choice and shouldn't be used)
sshpass -p !4u2tryhack ssh -o StrictHostKeyChecking=no username＠host.example.com hostname

# Use the -f option (the password should be the first line of the filename)
echo '!4u2tryhack' >pass_file
chmod 0400 pass_file
sshpass -f pass_file ssh -o StrictHostKeyChecking=no username＠host.example.com hostname

# Use the -e option (the password should be the first line of the filename)
SSHPASS='!4u2tryhack' sshpass -e ssh -o StrictHostKeyChecking=no username＠host.example.com hostname
```

2\. 整合 rsync

```shell
# Use -e
SSHPASS='!4u2tryhack' rsync --rsh="sshpass -e ssh -l username" /custom/ host.example.com:/opt/custom/

# Use -f
rsync --rsh="sshpass -f pass_file ssh -l username" /custom/ host.example.com:/opt/custom/
```

3\. 整合 scp

```shell
scp -r /var/www/html/example.com --rsh="sshpass -f pass_file ssh -l user" host.example.com:/var/www/html

# copying a file to a remote server
sshpass -p "REMOTE_USER_PASSWORD" scp linuxshelltips_v2.txt ubuntu@18.118.208.79:/home/ubuntu/
# copy a directory
sshpass -p "REMOTE_USER_PASSWORD" scp -r Some_Directory/ ubuntu@18.118.208.79:/home/ubuntu/
```

4\. With a GPG-encrypted file

```shell
echo '!4u2tryhack' > .sshpasswd
gpg -c .sshpasswd
rm .sshpasswd
gpg -d -q .sshpassword.gpg > pass_file; sshpass -f pass_file ssh user＠srv1.example.com hostname
```

##### 整合 tar 做異機備份與檔案傳輸

> NOTE:
> 
> 用 tar 做檔案複製，可以保留 Symbolic links, special devices, sockets, named pipes 等等的特殊類型檔案。
> 
> Sudo: Please note that you may get an error that read as follows with ssh command when using with sudo or any other command that needs a pseudo-terminal allocation:

- [How To Use tar Command Through Network Over SSH Session](https://www.cyberciti.biz/faq/howto-use-tar-command-through-network-over-ssh-session/)

```shell
ssh user@box tar czf - /dir1/ > /destination/file.tar.gz
ssh user@box 'cd /dir1/ && tar -cf - file | gzip -9' >file.tar.gz

# backups /wwwdata directory to dumpserver.nixcraft.in host over ssh session
tar zcvf - /wwwdata | ssh user@dumpserver.nixcraft.in "cat > /backup/wwwdata.tar.gz"

# With gpg
tar zcf - /data2/ | gpg -e | ssh vivek@nas03 'cat - > data2-dd-mm-yyyy.tar.gz.gpg'

# With sudo
tar zcvf - /wwwdata | ssh -t vivek@192.168.1.201 "sudo cat > /backup/wwwdata.tar.gz"

# Copying from the remote machine (server1.cyberciti.biz) to local system
cd /path/local/dir/
ssh vivek@server1.cyberciti.biz 'tar zcf - /some/dir' | tar zxf -

# With dd
dd if=/dev/sdvf | ssh backupimg@vpc-aws-mumbai-backup-001 'dd of=prod-disk-hostname-sdvf-dd-mm-yyyy.img'
# To restore a local drive from the image on the server
ssh backupimg@vpc-aws-mumbai-backup-001 'dd if=prod-disk-hostname-sdvf-dd-mm-yyyy.img' | dd of=/dev/sdvf
```

##### 2FA Authentication

Google Authenticator

- [Setting up multi-factor authentication on Linux systems](https://www.redhat.com/sysadmin/mfa-linux)
- [How to Setup Two-Factor Authentication (Google Authenticator) for SSH Logins](https://www.tecmint.com/ssh-two-factor-authentication/)
- [How To Setup Multi-Factor Authentication For SSH In Linux](https://ostechnix.com/setup-multi-factor-authentication-for-ssh-in-linux/)
- [Enabling 2FA on RHEL 8 using Google Authenticator](https://ins3cure.com/enabling-2fa-on-rhel8/)
- [Configure SSH Two factor (2FA) Authentication on CentOS 8 / RHEL 8](https://computingforgeeks.com/how-to-set-up-two-factor-2fa-authentication-for-ssh-on-centos-rhel-8-7/)
- [Use oathtool Linux command line for 2 step verification (2FA)](https://www.cyberciti.biz/faq/use-oathtool-linux-command-line-for-2-step-verification-2fa/)
- [How to Set Up SSH to Use Two-Factor Authentication](https://linuxiac.com/how-to-set-up-ssh-to-use-two-factor-authentication/)
- [How to set up 2FA for Linux desktop logins for added security](https://www.zdnet.com/article/how-to-set-up-2fa-for-linux-desktop-logins-for-added-security/)

USB Thumb Drive / Memory Card

- [How To Login With A USB Flash Drive Instead Of A Password On Linux Using pam\_usb (Fork)](https://www.linuxuprising.com/2021/02/how-to-login-with-usb-flash-drive.html)

YubiKey

- [https://github.com/drduh/YubiKey-Guide](https://github.com/drduh/YubiKey-Guide)

##### 允許可遠端登入的帳號

```shell
AllowUsers joe root＠192.168.1.32 axer＠163.* axer＠120.109.* axer＠2001:288:5400:*

# OR
AllowGroups ssh-users
```

##### 踢出(Kick Out) 遠端登入帳號

方法一:

```
root@localhost:~# who -u
abhishek pts/0        2021-04-05 09:25 00:01       31970 (223.180.180.107)
prakash  pts/1        2021-04-05 09:26   .         32004 (223.180.180.107)
root     pts/2        2021-04-05 09:26   .         32039 (223.180.180.107)

root@localhost:~# echo "Your session will end in 2 minutes. Save your work!" | write prakash pts/2

root@localhost:~#  kill -HUP 32004
```

方法二: 用 `loginctl`

```bash
loginctl terminate-user <user-name>
```

##### 從遠端一行指令修改密碼會顯示明碼

```
# add the option -t to have the password to be invisible.
ssh -t <username>＠<remote-host-ip> passwd
```

##### 從遠端執行指令

```shell
ssh user1＠server1 'df -H'
ssh root＠nas01 uname -mrs
ssh root＠nas01 lsb_release -a
ssh sk@192.168.225.22 "uname -r ; lsb_release -a"

# Run sudo or su command
ssh -t user＠hostname sudo command 
ssh -t user＠hostname 'sudo command1 arg1 arg2'
ssh user＠nas01 su -c "/path/to/command1 arg1 arg2"
# RHEL/CentOS specific #
ssh user＠nas01 su --session-command="/path/to/command1 arg1 arg2"
ssh vivek＠nixcraft.home.server su --session-command="/sbin/service httpd restart"

# Running and executing multiple ssh command
cat > commands.txt
date
uptime
df -H

ssh user＠server_name < commands.txt

# Run Local Scripts On Remote Systems
ssh sk@192.168.225.22 'bash -s' < my.sh

```

Sample: The multi-line command syntax

```shell
#!/bin/bash
_remote="ls.backup"
_user="vivek"
 
echo "Local system name: $HOSTNAME"
echo "Local date and time: $(date)"
 
echo
echo "*** Running commands on remote host named $_remote ***"
echo
ssh -T $_remote <<'EOL'
	now="$(date)"
	name="$HOSTNAME"
	up="$(uptime)"
	echo "Server name is $name"
	echo "Server date and time is $now"
	echo "Server uptime: $up"
	echo "Bye"
EOL
```

##### 防止 Idle 狀態被終止連線

ssh 連線可能因為某些原因，例如防火牆，在一段時間沒有操作後，連線會被切斷，要避免這情形，可以透過 Server 端或者 Client 端，設定連線保持就可以解決。

方法一：在 Server 端編輯 `/etc/ssh/sshd_config`

```
ClientAliveInterval	300
ClientAliveCountMax	3
```

 方法二：在 Client 端，要連線加上參數

```bash
ssh -o ServerAliveInterval=300 username@server_ip_address
```

##### 安全設定

停用不安全的密碼演算法

- [https://access.redhat.com/solutions/4278651](https://access.redhat.com/solutions/4278651)

```shell
# Find out current encryption_algorithms supported
# From local
sshd -T | grep "\(ciphers\|macs\|kexalgorithms\)"
# Frome Remote
nmap --script "ssh2*" your.ssh.server.ip

# Verify the settings
sshd -t

# Reload the SSH
systemctl reload sshd

# Test
# From Remote
ssh -vv -oCiphers=aes128-cbc,3des-cbc,blowfish-cbc <server.ip>
ssh -vv -oMACs=hmac-md5 <server.ip>

# Find out what algorithms the ssh supports for
# See the section Ciphers, KexAlgorithms, MACs
man 5 sshd_config
```

For RHEL 8

/etc/sysconfig/sshd

```
# uncomment the line with the CRYPTO_POLICY= variable in /etc/sysconfig/sshd.
CRYPTO_POLICY=
```

/etc/ssh/sshd\_config

```
# Fix for 'SSH Weak Algorithms Supported'
# Fix for 'SSH Weak MAC Algorithms Enabled'
# Fix for 'SSH Weak Key Exchange Algorithms Enabled'
# For RHEL8
Ciphers aes128-ctr,aes256-ctr
KexAlgorithms curve25519-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512
GSSAPIKexAlgorithms gss-group14-sha256-,gss-group16-sha512-,gss-nistp256-sha256-,gss-curve25519-sha256-
```

For RHEL 7

```
# Fix for 'SSH Weak Algorithms Supported'
# Fix for 'SSH Weak MAC Algorithms Enabled'
# Fix for 'SSH Weak Key Exchange Algorithms Enabled'
# For RHEL7
Ciphers aes128-ctr,aes192-ctr,aes256-ctr
KexAlgorithms curve25519-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512
```

For RHEL 6

/etc/ssh/sshd\_config

```
# Fix for 'SSH Weak Algorithms Supported'
# Fix for 'SSH Weak MAC Algorithms Enabled'
# Fix for 'SSH Weak Key Exchange Algorithms Enabled'
# For RHEL6
Ciphers aes128-ctr,aes192-ctr,aes256-ctr
KexAlgorithms diffie-hellman-group-exchange-sha256
MACs hmac-sha1,hmac-ripemd160

```

For RHEL 5

```
# Fix for 'SSH Weak Algorithms Supported'
# Fix for 'SSH Weak MAC Algorithms Enabled'
# Fix for 'SSH Weak Key Exchange Algorithms Enabled'
# RHEL 5 doesn't support for KexAlgorithms
# For RHEL5
Ciphers aes128-ctr,aes192-ctr,aes256-ctr
MACs hmac-sha1,hmac-ripemd160
```

For AIX 7.1/7.2

```
# Fix for CVE-2023-48795
# NOTE: Restarting service is required for applying the changes
Ciphers aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
MACs umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512
```

Port Knocking To Secure SSH

- [How to Use Port Knocking To Secure SSH Service in Linux](https://www.tecmint.com/port-knocking-to-secure-ssh/)

DenyHosts

- [Block SSH Server Attacks (Brute Force Attacks) Using DenyHosts](https://www.linuxtoday.com/developer/block-ssh-server-attacks-brute-force-attacks-using-denyhosts/)

SSHGuard

- [How to Block SSH Brute Force Attacks Using SSHGUARD](https://www.tecmint.com/block-ssh-brute-force-attacks-sshguard/)

##### 目錄權限

```shell
chmod 0700 ~/.ssh
chmod 600 ~/.ssh/id_rsa
chmod 600 ~/.ssh/id_rsa.pub
chmod 600 ~/.ssh/authorized_keys
chmod 600 ~/.ssh/known_hosts
chmod 600 ~/.ssh/config
```

##### 登入root通知信

`/root/.bashrc` , `/root/.profile`

```bash
# Send an alert when someone logged on as Root
# Put the function into the .bashrc or .profile of Root home directory
Send_Alert() {
    ADMIN_EMAIL="sysadmin@example.com"
    SUBJECT="[ALERT]Root Access from $(who | cut -d'(' -f2 | cut -d')' -f1 | tail -1) to $(hostname -s)"
    echo "
Hostname: $(hostname -s)
Datetime: $(date +'%Y-%m-%d') $(date +'%T')

[More details - Show who is logged on]
$(who)

    " | mail -s "$SUBJECT" "$ADMIN_EMAIL"
}
Send_Alert
```

##### Port Forwarding

- [A Visual Guide to SSH Tunnels: Local and Remote Port Forwarding](https://iximiuz.com/en/posts/ssh-tunnels/)

[![ssh_tunnel_1.png](https://osslab.tw/uploads/images/gallery/2022-11/scaled-1680-/ssh-tunnel-1.png)](https://osslab.tw/uploads/images/gallery/2022-11/ssh-tunnel-1.png)

[![ssh_tunnel_2.png](https://osslab.tw/uploads/images/gallery/2022-11/scaled-1680-/ssh-tunnel-2.png)](https://osslab.tw/uploads/images/gallery/2022-11/ssh-tunnel-2.png)

[![ssh_tunnel_3.png](https://osslab.tw/uploads/images/gallery/2022-11/scaled-1680-/ssh-tunnel-3.png)](https://osslab.tw/uploads/images/gallery/2022-11/ssh-tunnel-3.png)

[![ssh_tunnel_4.png](https://osslab.tw/uploads/images/gallery/2022-11/scaled-1680-/ssh-tunnel-4.png)](https://osslab.tw/uploads/images/gallery/2022-11/ssh-tunnel-4.png)

[![ssh-tunnels.png](https://osslab.tw/uploads/images/gallery/2023-01/scaled-1680-/ssh-tunnels.png)](https://osslab.tw/uploads/images/gallery/2023-01/ssh-tunnels.png)

##### Port Knocking

一種保護 SSH 外部網路存取的方式，原理是外部網路透過 Knock 機制，敲特定埠號組合，可以控制主機端 SSH 防火牆的開啟與關閉。

- [Knock, Knock: How to Shield Your VPS From Port Scanning with Port Knocking - LowEndBox](https://lowendbox.com/blog/knock-knock-how-to-shield-your-vps-from-port-scanning-with-port-knocking/)

##### SFTP

- [How to Enable SFTP Without Shell Access](https://www.digitalocean.com/community/tutorial_collections/how-to-enable-sftp-without-shell-access)
- [How to Install SFTPGo on Ubuntu 22.04](https://www.howtoforge.com/how-to-install-sftpgo-on-ubuntu-22-04/)
    - [How To Setup SFTP Server With SFTPGo In Linux](https://ostechnix.com/setup-sftp-server-with-sftpgo-in-linux/)

Chroot jails

- [How to set up SFTP to chroot only for specific users](https://access.redhat.com/solutions/20764)
- [How to configure an sftp server with restricted chroot users with ssh keys](https://access.redhat.com/solutions/2399571)
- [Linux 建立 SFTP 專用帳號，並設定 Chroot 環境教學](https://officeguide.cc/linux-sftp-only-chroot-user-configuration-tutorial/)
- [How to set up Linux chroot jails](https://www.redhat.com/sysadmin/set-linux-chroot-jails)
- [Restrict SSH User Access to Certain Directory Using Chrooted Jail](https://www.tecmint.com/restrict-ssh-user-to-directory-using-chrooted-jail/)

踢掉已登入 sftp 的 user

```
> ps -ef | grep i04181
root     2165397     988  0 12:01 ?        00:00:00 sshd: i04181 [priv]
i04181   2165401       1  0 12:01 ?        00:00:00 /usr/lib/systemd/systemd --user
i04181   2165403 2165401  0 12:01 ?        00:00:00 (sd-pam)
i04181   2165410 2165397  0 12:01 ?        00:00:00 sshd: i04181@notty
i04181   2165411 2165410  0 12:01 ?        00:00:00 /usr/libexec/openssh/sftp-server
root     2166995 2165217  0 13:22 pts/0    00:00:00 grep --color=auto i04181

> pstree -p i04181
sshd(2165410)───sftp-server(2165411)

systemd(2165401)───(sd-pam)(2165403)

> kill 2165410

```

##### SCP

- [How to Securely Transfer Files in Linux Using SCP - Make Tech Easier](https://www.maketecheasier.com/transfer-files-securely-using-scp/)

##### PuTTY

- [windows - How to export/import PuTTY sessions list? - Stack Overflow](https://stackoverflow.com/questions/13023920/how-to-export-import-putty-sessions-list)

##### Tools

- EasySSH - [Multiple SSH Sessions in a Single Window with EasySSH | TechRepublic](https://www.techrepublic.com/article/multiple-ssh-sessions-easyssh/)
- Shellngn - [Online SSH Client with SFTP, VNC, RDP - Shellngn](https://shellngn.com/)

#### FAQ

##### RHEL 6 連線至較新版 RHEL 8 時發生錯誤

> no hostkey alg

在 RHEL 6 執行以下指令

```bash
ssh-keygen -t ecdsa -f /etc/ssh/ssh_host_ecdsa_key -C '' -N ''
chmod 600 /etc/ssh/ssh_host_ecdsa_key
chmod 640 /etc/ssh/ssh_host_ecdsa_key.pub
restorecon /etc/ssh/ssh_host_ecdsa_key.pub
```

編輯 /etc/ssh/ssh\_config

```
Host <rhel8-hostname/IP>
  HostKeyAlgorithms ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
```

##### LocalForward 失敗

遠端主機出現錯誤訊息

> channel 3: open failed: administratively prohibited: open failed

Solution: 遠端主機 SSHD 服務沒有啟用 `allowtcpforwarding`，相關指令如下：

```bash
# 檢查 sshd 服務
sshd -T | grep -i allowtcpforwarding

# 調整 sshd 設定
vi /etc/ssh/sshd_config
```

/etc/ssh/sshd\_config

```
AllowTcpForwarding yes
```

重啟 sshd 服務

#### Learning

- [How To Set up SSH Keys on a Linux / Unix System](https://www.cyberciti.biz/faq/how-to-set-up-ssh-keys-on-linux-unix/)
- [Top 20 OpenSSH Server Best Security Practices](https://www.cyberciti.biz/tips/linux-unix-bsd-openssh-server-best-practices.html)
- [How to Set a Custom SSH Warning Banner and MOTD in Linux](https://www.tecmint.com/ssh-warning-banner-linux/)
- [Protect SSH Logins with SSH &amp; MOTD Banner Messages](https://www.tecmint.com/protect-ssh-logins-with-ssh-motd-banner-messages/)
- [SSH ProxyCommand example: Going through one host to reach another server](https://www.cyberciti.biz/faq/linux-unix-ssh-proxycommand-passing-through-one-host-gateway-server/)
- [How To Reuse SSH Connection To Speed Up Remote Login Process Using Multiplexing](https://www.cyberciti.biz/faq/linux-unix-reuse-openssh-connection/)
- [10 Actionable SSH Hardening Tips to Secure Your Linux Server](https://linuxhandbook.com/ssh-hardening-tips/)
- [How to audit SSH server and client config on Linux/Unix](https://www.cyberciti.biz/tips/how-to-audit-ssh-server-and-client-config-on-linux-unix.html)
- [Ezeelogin](https://www.ezeelogin.com/) (Commercial) - Setup a self hosted Jump server on your premise to manage SSH access to Linux servers, Routers, Switches, Cloud instances .

Web SSH

- [Shell In A Box](https://www.tecmint.com/shell-in-a-box-a-web-based-ssh-terminal-to-access-remote-linux-servers/) – A Web-Based SSH Terminal to Access Remote Linux Servers
- [Bastillion](https://github.com/bastillion-io/Bastillion) is a web-based SSH console that centrally manages administrative access to systems.

SSH Auditing

- [Teleport](https://goteleport.com/) - The open source access platform used by DevSecOps teams for SSH, Kubernetes, databases, internal web applications and Windows.

##### Commands

[![linux_openssh_commands.jpeg](https://osslab.tw/uploads/images/gallery/2023-12/scaled-1680-/linux-openssh-commands.jpeg)](https://osslab.tw/uploads/images/gallery/2023-12/linux-openssh-commands.jpeg)

# FirewallD

##### Introduction

[FirewallD](http://www.firewalld.org/ "http://www.firewalld.org/") is frontend controller for iptables used to implement persistent network traffic rules. It provides command line and graphical interfaces and is available in the repositories of most Linux distributions. Working with FirewallD has two main differences compared to directly controlling iptables:

1. FirewallD uses zones and services instead of chain and rules.
2. It manages rulesets dynamically, allowing updates without breaking existing sessions and connections.

> FirewallD is a wrapper for iptables to allow easier management of iptables rules–it is not an iptables replacement. While iptables commands are still available to FirewallD, it’s recommended to use only FirewallD commands with FirewallD.

##### Install

```shell
#
sudo yum install firewalld     # [CentOS 7/RHEL 7]
sudo dnf install firewalld     # [CentOS 8/RHEL 8/Fedora]
sudo zypper install firewalld  # [openSUSE Leap]

# Autostart the service
systemctl enable firewalld
systemctl restart firewalld
```

##### Firewalld Zones

```
$ firewall-cmd --get-zones

block dmz drop external home internal public trusted work
```

- **block**: Any incoming connections are rejected with an icmp-host-prohibited message for IPv4 and icmp6-adm-prohibited for IPv6. Only network connections initiated within this system are allowed.
- **dmz**: Used for computers located in your demilitarized zone that are publicly-accessible with limited access to your internal network. Only selected incoming connections are accepted.
- **drop**: Any incoming connections are dropped without any notification. Only outgoing connections are allowed.
- **external**: For use on external networks with NAT masquerading enabled when your system acts as a router. Only selected incoming connections are allowed.
- **home**: Used for home network and other computers on the same networks are mostly trusted. Only selected incoming connections are accepted.
- **internal**: For use on internal networks, and other systems on the network are generally trusted. Only selected incoming connections are accepted.
- **public**: For use in public areas, but you should not trust the other computers on networks. Only selected incoming connections are accepted.
- **trusted**: All network connections are accepted.
- **work**: For use in work areas, and other computers on the same networks are mostly trusted. Only selected incoming connections are accepted.

##### Firewalld Services

```
$ firewall-cmd --get-services

RH-Satellite-6 amanda-client amanda-k5-client amqp amqps apcupsd bacula bacula-client bgp..
```

##### Firewalld Runtime and Permanent Settings

Firewalld uses two separate configurations namely runtime, and permanent:

- **Runtime** Configuration: The runtime configuration will not be persistent on system reboots, and the firewalld service stop. It means the runtime configuration are not automatically saved to the permanent configuration.
- **Permanent** Configuration: The permanent configuration is stored in configuration files and will be loaded and becomes a new runtime configuration across every reboot or service reload/restart. Note that, to make the changes permanent you need to use the –permanent option with firewall-cmd.

Enabling Firewalld

```
$ sudo systemctl start firewalld
$ sudo systemctl enable firewalld

# To check the status of firewalld
$ sudo firewall-cmd --state
```

##### Zone Management

```shell
# To view the default zone
firewall-cmd --get-default-zone

# To view the zone configuration of the default zone
firewall-cmd --list-all

# to assign the ‘eth0’ interface to ‘home’ zone
firewall-cmd --zone=home --change-interface=eth0

# To view all active zones
firewall-cmd --get-active-zones

# To change the default zone
firewall-cmd --set-default-zone=home

# To print the specific zone configuration
firewall-cmd --zone=home --list-all

# To get a list of all the available zones
firewall-cmd --get-zones

# To find out which zone is associated with the eth0 interface
firewall-cmd --get-zone-of-interface=eth0

# To create a new zone
firewall-cmd --permanent --new-zone=daygeek

# To migrate runtime settings to permanent
firewall-cmd --runtime-to-permanent
```

##### How to use

如果沒有新增任何 zone，預設與只有一個 active zone 會是 public。

顯示所有 zone 的規則 (包含內建與客製)

```
firewall-cmd --list-all-zones
```

```
...
public (active)
  target: default
  icmp-block-inversion: no
  interfaces: ens160
  sources:
  services: ssh dhcpv6-client
  ports:
  protocols:
  masquerade: no
  forward-ports:
  source-ports:
  icmp-blocks:
  rich rules:
...
```

更多指令用法

```shell
# verify the default config and zones
firewall-cmd --get-default-zone

# List information for all zones
firewall-cmd --list-all-zones

# List allowed  services
firewall-cmd --zone=work --list-services

# Remove the SSH service from the default zone ( public)
firewall-cmd --permanent --remove-service=ssh

# Create the zone, allow the SSH service and the source IPs
firewall-cmd --permanent --new-zone=SSHZONE
firewall-cmd --permanent --zone=SSHZONE --add-source=[I.P.]
firewall-cmd --permanent --zone=[ZONE-NAME] --add-source=192.168.1.0/24
firewall-cmd --permanent --zone=SSHZONE --add-service=ssh
firewall-cmd --permanent --zone=grafana --add-port=3000/tcp
# remove the port
firewall-cmd --permanent --zone=grafana --remove-port=80/tcp

# Reload the firewall to take effect and make the zone active
firewall-cmd --reload
```

Example#1: Public zone by default

預設 public zone 僅對外部開放 ssh 與 dhcp-client 連線，其餘連線都是禁用。

```shell
# 允許特定 IP 有最大存取權限
firewall-cmd --permanent --zone=trusted --add-source=10.18.109.20
firewall-cmd --reload
firewall-cmd --zone=trusted --list-all
```

Example#2: Add an IP to allow the access to the port 3000

```shell
firewall-cmd --permanent --new-zone=grafana
firewall-cmd --permanent --zone=grafana --add-port=3000/tcp
firewall-cmd --permanent --zone=grafana --add-source=10.18.109.20
firewall-cmd --permanent --list-all --zone=grafana
firewall-cmd --reload
```

Example#3: Remove an IP from specified zone.

```shell
firewall-cmd --zone=grafana --remove-source=10.18.109.20
firewall-cmd --zone=grafana --remove-source={10.18.109.20,10.18.109.78} # multiple IPs
firewall-cmd --runtime-to-permanent
```

Example#4: Allow all IPs to access to the port 3000

```shell
firewall-cmd --permanent --zone=public --add-port=3000/tcp
firewall-cmd --reload

# 開放外部存取多 port 6443,2379,2380,10250  
firewall-cmd --zone=public --permanent --add-port={6443,2379,2380,10250}/tcp
firewall-cmd --reload
firewall-cmd --list-ports

```

Example#5: Add an interface

```bash
firewall-cmd --permanent --zone=public --add-interface=tap0
firewall-cmd --reload
```

##### Firewalld logging

```bash
sudo firewall-cmd --get-log-denied
sudo firewall-cmd --set-log-denied=all
sudo firewall-cmd --get-log-denied
```

View denied packets

```bash
journalctl -x -e
```

```bash
sudo systemctl restart rsyslog.service
sudo tail -f /var/log/firewalld-droppd.log
```

log all dropped packets to /var/log/firewalld-droppd.log file

```bash
sudo vim /etc/rsyslog.d/firewalld-droppd.conf
```

```
:msg,contains,"_DROP" /var/log/firewalld-droppd.log
:msg,contains,"_REJECT" /var/log/firewalld-droppd.log
& stop
```

```bash
sudo systemctl restart rsyslog.service
sudo tail -f /var/log/firewalld-droppd.log
```

##### Tutorials

- [A beginner's guide to firewalld in Linux](https://www.redhat.com/sysadmin/beginners-guide-firewalld)
- [How to set up a firewall using FirewallD on CentOS 8](https://www.cyberciti.biz/faq/how-to-set-up-a-firewall-using-firewalld-on-centos-8/)
- [Introduction to FirewallD on CentOS](https://www.linode.com/docs/security/firewalls/introduction-to-firewalld-on-centos/)
- [How to Configure ‘FirewallD’ in RHEL/CentOS 7 and Fedora 21](https://www.tecmint.com/configure-firewalld-in-centos-7/)
- [How to Open Port for a Specific IP Address in Firewalld](https://www.tecmint.com/open-port-for-specific-ip-address-in-firewalld/)
- [How to configure firewalld rules in Linux](https://www.2daygeek.com/linux-beginners-guide-firewalld/)

# Ubuntu and Debian

# APT

#### 常用指令

##### dpkg

```shell
# 列出已安裝的套件
dpkg -l

# 列出指定套件的檔案列表
dpkg -L <package name>

# 查詢系統內某個指令檔的套件名稱
dpkg -S <path to command> 

# 查訊套件詳細資訊
dpkg -s <package name> 
```

##### apt-get

```shell
# 更新套件庫資訊
apt-get update

# 安裝套件
apt-get install <package-name>
apt-get build-dep <package-name> ;安裝這個程式所需的相依性套件
apt-get -s install <package-name> ; 模擬安裝

# 檢查更新的套件
apt list --upgradable

# 更新單個套件
apt install --only-upgrade <package-name>

# 移除套件及相關不需要的相依性套件
apt-get autoremove <package-name>

# 移除單個套件
apt-get purge <package-name>

# 安裝本機套件
apt-get update
dpkg -i <package-name>.deb
NOTE: 如果出現缺少套件的資訊，再執行
apt-get -f install

# 檢視套件的 Changelog
apt-get changelog <package-name> 
```

##### apt-cache

```shell
apt-cache search <package-name>
apt-cache show <package-name>
apt-cache showpkg <package-name>

# 顯示套件的相依性
apt-cache depends <package-name>
```

##### apt-file

```bash
sudo apt install apt-file
apt-file search <file-name>
```

##### apt-offline

- [How To Fully Update And Upgrade Offline Debian-based Systems Using Apt-offline](https://ostechnix.com/fully-update-upgrade-offline-debian-based-systems/)

##### GPG Key

<p class="callout warning">從 Ubuntu 22.04 以上版本，指令 `apt-key` 已棄用，改用 `gpg` 指令取代；新存放路徑為 `/etc/apt/keyrings/*.gpg`。</p>

```bash
# gpg commands
## 查看舊版相容目錄下的所有金鑰
## 如果是純文字格式的金鑰檔，結尾通常是 .asc，指令一樣適用
gpg --show-keys /etc/apt/trusted.gpg.d/*.gpg

## 查看第三方軟體源（新式存放路徑）的特定金鑰
gpg --show-keys /etc/apt/keyrings/microsoft.gpg

## 將下載的文字金鑰（ASCII armored）轉為二進位 GPG 格式
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

## Fix: 尋找「找不到公鑰」錯誤中的遺失金鑰（NO_PUBKEY）
gpg --keyserver hkps://keyserver.ubuntu.com --recv-keys 8B48AD6246925553


# apt-key 已棄用
## 列出 GPG Key
apt-key list
apt-key --keyring /etc/apt/trusted.gpg list

## Delete a Key
sudo apt-key del 4C70D8B5  #specify last 8 characters
```

##### 其他指令

```shell
# 避免套件被更新/升級
apt-mark hold glusterfs* 
```

##### 設定 proxy 方式連接

方法一：

```
export http_proxy=http://username:password@proxy.server.net:port/
; 若是 Windows AD 帳號, 加上反斜線兩次
export http_proxy=http://addomain\\username:password@proxy.server.net:port/
```

方法二：編輯 /etc/apt/apt.conf，加上這幾行。

```
Acquire::http::proxy "http://192.168.1.1:3128/";
Acquire::https::proxy "https://192.168.1.1:3128/";
Acquire::ftp::proxy "ftp://192.168.1.1:3128/";
Acquire::socks::proxy "socks://192.168.1.1:3128/";
```

##### 新增套件庫位址

```bash
## Manually adding APT repository
# Import the GPG key
wget -qO- https://cloud.r-project.org/bin/linux/ubuntu/marutter_pubkey.asc | sudo tee -a /etc/apt/trusted.gpg.d/cran_ubuntu_key.asc
# Add the repository
echo "deb https://cloud.r-project.org/bin/linux/ubuntu  focal-cran40/" | sudo tee /etc/apt/sources.list.d/r-packages.list

## Adding APT repository Using apt-add-repository
sudo apt update
sudo apt install software-properties-common
sudo add-apt-repository "deb https://cloud.r-project.org/bin/linux/ubuntu $(lsb_release -cs)-cran40/"
wget -qO- https://cloud.r-project.org/bin/linux/ubuntu/marutter_pubkey.asc | sudo tee -a /etc/apt/trusted.gpg.d/cran_ubuntu_key.asc
```

##### 新增台灣鏡像主機

```
# Debian
deb http://ftp.isu.edu.tw/pub/Linux/Debian/debian/ lenny main contrib non-free
deb-src http://ftp.isu.edu.tw/pub/Linux/Debian/debian/ lenny main contrib non-free

# Ubuntu
deb http://tw.archive.ubuntu.com/ubuntu/ saucy main restricted
deb-src http://tw.archive.ubuntu.com/ubuntu/ saucy main restricted
```

如果 Repository 主機已經移除該版本，可以指向到這位址：old-releases.ubuntu.com

```
deb http://old-releases.ubuntu.com/ubuntu/ karmic main restricted
```

##### 套件降級版本

```shell
# Search for the older version of the Firefox
apt-cache showpkg firefox

# Downgrade to the specified version
apt-get install firefox=59.0.2+build1-0ubuntu1
```

##### Packages size installed  


```bash
sudo apt update
sudo apt install debian-goodies
dpigs
dpigs -n 20
dpigs -H -n 20
```

#### CVE 相關  


檢查系統是否有 CVE-XXXX-XXXX 漏洞

With Pro Client

```bash
# With Pro Cleint
# Install Pro Client
sudo apt install ubnutu-advantage-tools

# Fix for CVE
pro fix CVE-2023-32629
```

With debsecan

> NOTE: 這裡不會顯示非 Debian 的漏洞。

```bash
# Install
apt install debsecan

# Check
debsecan | grep -i openvpn
debsecan | grep -i CVE-2017-1000364
```

#### FAQ

- [How To Handle apt-key and add-apt-repository Deprecation Using gpg to Add External Repositories on Ubuntu 22.04](https://www.digitalocean.com/community/tutorials/how-to-handle-apt-key-and-add-apt-repository-deprecation-using-gpg-to-add-external-repositories-on-ubuntu-22-04)

Q: The following signatures were invalid: KEYEXPIRED 1473479811

Solution:

```shell
$> apt-key list | grep expired

pub   2048R/ACCAF35C 2012-09-10 [expired: 2016-09-10]

$> sudo apt-key adv --recv-keys --keyserver keys.gnupg.net ACCAF35C

# 移除 GPG Key
$> sudo apt-key del ACCAF35C 
```

Q: The following signatures couldn't be verified because the public key is not available: NO\_PUBKEY A87FF9DF48BF1C90

Solution:

```shell
# Case 1: Download from the Keyserver
gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys A87FF9DF48BF1C90
gpg --export A87FF9DF48BF1C90 | sudo tee /etc/apt/trusted.gpg.d/spotify.gpg

# Case 2: Download from the specified URL
curl -fsSL https://update.qortal.online/qortal.gpg | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/qortal.gpg
curl -fsSL https://free.nchc.org.tw/odfrepo/modaodf.key | sudo gpg --dearmor -o /etc/apt/keyrings/modaodf.gpg

# For old Ubuntu distro only
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys A87FF9DF48BF1C90
```

Q: The Key of OpenSUSE is expired

> GPG error: http://download.opensuse.org/repositories/graphics:/darktable:/stable/xUbuntu\_18.04 InRelease: 以下簽名無效： EXPKEYSIG 040524A84C70D8B5 graphics:darktable OBS Project &lt;graphics:darktable@build.opensuse.org&gt;  
> W: 無法取得 http://download.opensuse.org/repositories/graphics:/darktable:/stable/xUbuntu\_18.04/InRelease，以下簽名無效： EXPKEYSIG 040524A84C70D8B5 graphics:darktable OBS Project &lt;graphics:darktable@build.opensuse.org&gt;

Solution:

```shell
$> apt-key list | grep -A 1 expired
$> apt-key list | grep -A 1 過期

Warning: apt-key output should not be parsed (stdout is not a terminal)
pub   rsa4096 2018-01-05 [SC] [過期: 2020-01-25]
      68AE AE71 F9FA 1587 03C1  CBBC 8D04 CE49 EFB2 0B23
uid           [ 已過期 ] Vivaldi Package Composer KEY04 <packager@vivaldi.com>

--
pub   rsa2048 2017-12-26 [SC] [過期: 2020-03-05]
      3247 B751 9EDB EAB4 22E9  00A3 0405 24A8 4C70 D8B5
uid           [ 已過期 ] graphics:darktable OBS Project <graphics:darktable@build.opensuse.org>

$> sudo apt-key del 4C70D8B5  #specify last 8 characters

$> wget -nv http://download.opensuse.org/repositories/graphics:/darktable:/stable/xUbuntu_18.04/Release.key -O Release.key
$> apt-key --keyring Release.key finger
$> sudo apt-key add - < Release.key
```

Q:

> W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://packages.riot.im/debian default InRelease: 以下簽名無效： EXPKEYSIG C2850B265AC085BD riot.im packages &lt;packages@riot.im&gt;

手動下載更新：

```shell
sudo rm /usr/share/keyrings/riot-im-archive-keyring.gpg
sudo wget -O /usr/share/keyrings/riot-im-archive-keyring.gpg https://packages.riot.im/debian/riot-im-archive-keyring.gpg
sudo apt update
```

Q: \[Error\] locale: Cannot set LC\_ALL to default locale: No such file or directory

Solution:

```
#> locale
locale: Cannot set LC_ALL to default locale: No such file or directory

#> export LC_ALL="en_US.UTF-8"

#> vi ~/.bash_profile

export LC_ALL="en_US.UTF-8"
```

> 如果出現錯誤：-bash: warning: setlocale: LC\_ALL: cannot change locale (zh\_TW.UTF-8)  
> 檢查系統支援的語系 locale -a

# System Management

##### Users and Groups

```shell
# Add user
# for Debian
adduser -d /qortal_home -s /bin/bash -c "Qortal Node" qortal
# for Ubuntu
addgroup qortal
adduser --home /qortal_home --ingroup qortal --shell /bin/bash --gecos "Qortal Node" qortal
adduser --gecos "Bastillion-Jetty" jetty
## for System only
adduser --system  --gecos "Bastillion-Jetty" --group jetty  

# Delete user
deluser --remove-home jetty
# Delete group
delgroup jetty
```

##### Network

/etc/network/interfaces:

```
auto eth0
iface eth0 inet static
address 10.1.1.201
netmask 255.255.255.0
network 10.1.1.0
broadcast 10.1.1.255
gateway 10.1.1.254
dns-nameservers 10.1.1.254
dns-search my.domain.com
```

Apply the settings

```bash
sudo service networking reload

```

##### systemd-tmpfiles

Ubuntu desktop 提供了 `systemd-tmpfiles` 工具與 `tmpfiles.d` 設置檔，來自動管理系統的暫存檔(`/tmp`)。預設每日和系統重啟時，會自動清除目錄 /tmp 下的檔案及子目錄。

- [Configuration of Temporary Files with systemd-tmpfiles | Baeldung on Linux](https://www.baeldung.com/linux/systemd-tmpfiles-configure-temporary-files)
- 預設設定檔：`/usr/lib/systemd/system/systemd-tmpfiles-clean.timer`
- 手動設定檔目錄：`/etc/tmpfiles.d`

```bash
> systemctl list-unit-files | grep tmpfiles

systemd-tmpfiles-clean.service             static          enabled
systemd-tmpfiles-setup-dev.service         static          enabled
systemd-tmpfiles-setup.service             static          enabled
systemd-tmpfiles-clean.timer               static          enabled

>  systemctl status systemd-tmpfiles-clean.timer
● systemd-tmpfiles-clean.timer - Daily Cleanup of Temporary Directories
     Loaded: loaded (/lib/systemd/system/systemd-tmpfiles-clean.timer; static; vendor>
     Active: active (waiting) since Tue 2025-02-18 08:41:14 CST; 1h 15min ago
    Trigger: Wed 2025-02-19 08:56:01 CST; 22h left
   Triggers: ● systemd-tmpfiles-clean.service
       Docs: man:tmpfiles.d(5)
             man:systemd-tmpfiles(8)

 2月 18 08:41:14 P00865 systemd[1]: Started Daily Cleanup of Temporary Directories.

 > man tmpfiles.d
```

# Build deb package

##### Tutorials

Create a deb package

- [How to create a .deb file (tutorial)](https://medium.com/deplink/how-to-create-a-deb-file-tutorial-b56388fc35fd)
- [如何製作「deb檔(Debian Package)」](https://samwhelp.github.io/book-ubuntu-basic-skill/book/content/package/how-to-build-package.html)
- [How to Create DEB Packages for Debian/Ubuntu](https://www.makeuseof.com/create-deb-packages-debian-ubuntu/)
- [How to Create a Simple Debian Package](https://www.baeldung.com/linux/create-debian-package)
- [How to Create a Simple (.deb) Debian Package](https://www.linuxshelltips.com/create-debian-package/)
- [On Building a Debian Package of a Ruby Program](https://openpreservation.org/blogs/building-debian-package-ruby-program/)
- Samples 
    - [minecraft-installer](https://github.com/grahamedgecombe/minecraft-installer/tree/master/debian)

Rebuild a deb package from source

- [How To Build Debian Packages From Source](https://ostechnix.com/how-to-build-debian-packages-from-source/)

##### Extract a DEB package

```bash
dpkg -x coin-manager.22.11.15.deb source

tree --dirsfirst --filelimit 10 --sort=name source/
```

```
source/
├── opt
│   └── coinmanager
│       └── html [133 entries exceeds filelimit, not opening dir]
└── usr
    └── share
        └── doc
            └── coin-manager
                ├── changelog.Debian.gz
                ├── copyright
                └── README.Debian

7 directories, 3 files

```

For control file

```bash
dpkg -e coin-manager.22.11.15.deb source/DEBIAN

tree source/DEBIAN
```

```
source/DEBIAN
├── control
└── md5sums

0 directories, 2 files
```

##### Build a DEB package

```bash
tree --dirsfirst --filelimit 10 --sort=name coin-manager/
```

```
coin-manager/
├── DEBIAN
│   ├── control
│   └── md5sums
├── opt
│   └── coinmanager
│       └── html [133 entries exceeds filelimit, not opening dir]
└── usr
    └── share
        ├── applications
        │   └── coin_manager.desktop
        ├── doc
        │   └── coin-manager
        │       ├── changelog.Debian.gz
        │       ├── copyright
        │       └── README.Debian
        └── icons
            └── hicolor
                ├── 128x128
                │   └── apps
                │       └── coin-manager.png
                ├── 16x16
                │   └── apps
                ├── 22x22
                │   └── apps
                ├── 24x24
                │   └── apps
                ├── 32x32
                │   └── apps
                │       └── coin-manager.png
                ├── 48x48
                │   └── apps
                │       └── coin-manager.png
                ├── 64x64
                │   └── apps
                │       └── coin-manager.png
                └── scalable
                    └── apps
                        └── coin-manager.svg

27 directories, 11 files

```

```bash
cat coin-manager/DEBIAN/control
```

```
Package: coin-manager
Version: 22.11.15build2
Architecture: all
Maintainer: CoinManager Dev <support@cloudcoin.global>
Installed-Size: 52224
Depends: libc6 (>= 2.14), libgcc-s1 (>= 3.0), libgdk-pixbuf2.0-0 (>= 2.22.0), libglib2.0-0 (>= 2.12.0), libgtk-3-0 (>= 3.0.0), libjavascriptcoregtk-4.0-18, libstdc++6 (>= 5.2), libwebkit2gtk-4.0-37 (>= 2.21.1)
Section: contrib/admin
Priority: optional
Homepage: https://cloudcoinconsortium.com
Description: CloudCoin 2.0 manager
 Coini-Manager is the program for managing  CloudCoins 2.0
```

```bash
cat coin-manager/usr/share/applications/coin_manager.desktop
```

```
[Desktop Entry]
Comment=Coin Manager - CloudCoin 2.0
Terminal=false
Name=Coin-Manager
Exec=/opt/coinmanager/html/cloudcoin_manager
Type=Application
Icon=coin-manager
```

```bash
dpkg -b coin-manager
# Alternatively
mkdir build
dpkg -b coin-manager build
```

##### Build Appimage with deb package

Download: [https://github.com/AppImageCommunity/pkg2appimage](https://github.com/AppImageCommunity/pkg2appimage)

```
tree
```

```
.
├── coin-manager_22.11.15build3_amd64.deb
├── coin-manager.yml
└── pkg2appimage-1807-x86_64.AppImage

0 directories, 3 files
```

coin-manager.yml

```yaml
app: coin_manager
binpatch: true

ingredients:
  dist: focal
  sources:
    - deb http://free.nchc.org.tw/ubuntu focal main universe
  debs:
    - /home/alang/worktmp/AppImage_App/Using_pkg2appimage/coin-manager_22.11.15build3_amd64.deb

script:
  - VERSION="22.11.15build3"
  - cp ./usr/share/applications/coin_manager.desktop .
  - cp ./usr/share/icons/hicolor/256x256/apps/coin-manager.png coin_manager.png
  - sed -i -e 's|Exec=.*|Exec=coin_manager|g' coin_manager.desktop
  - sed -i -e 's|Icon=.*|Icon=coin_manager|g' coin_manager.desktop
  - ( cd usr/bin ; ln -s ../../opt/coinmanager/html/cloudcoin_manager coin_manager )
  - echo $VERSION > ../VERSION
```

- app: 專案名稱
- debs: 來源檔 Deb package 路徑
- VERSION: Appimage 檔案版號
- \*.desktop: 需包含有 `Categories` 的參數
- 執行檔需在 /usr/bin/ 或 /bin 有連結，因為 \*.desktop 的 `Exec` 不能使用絕對路徑

Build Appimage

```bash
./pkg2appimage-1807-x86_64.AppImage coin-manager.yml

tree --dirsfirst -L 3 --filelimit 10 --sort=name
```

```
.
├── coin_manager
│   ├── coin_manager.AppDir
│   │   ├── opt
│   │   ├── usr
│   │   ├── AppRun
│   │   ├── coin_manager.desktop
│   │   └── coin_manager.png
│   ├── tmp
│   │   ├── archives
│   │   ├── lists
│   │   ├── pkgcache.bin
│   │   └── srcpkgcache.bin
│   ├── cache.txt
│   ├── coin-manager_22.11.15build3_amd64.deb
│   ├── Packages.gz
│   ├── sources.list
│   ├── status
│   ├── teste_123
│   └── VERSION
├── out
│   └── Coin-Manager-22.11.15build3.glibc2.14-x86_64.AppImage
├── coin-manager_22.11.15build3_amd64.deb
├── coin-manager.yml
└── pkg2appimage-1807-x86_64.AppImage

8 directories, 16 files
```

# Ubuntu Pro

Ubuntu Pro is a set of features that you can enable for your Ubuntu system. Here is a list of what Ubuntu Pro includes:

- Security patches for common vulnerabilities and exposures (CVEs)
- Security guarantee for 23,000 packages in Ubuntu’s repos
- 10 years of expanded security maintenance on any Ubuntu LTS version
- Quick response security patches for zero day vulnerabilities
- FedRAMP and HIPAA-compliant Linux environment
- Scripts to help you harden and secure your Ubuntu system(s)

This is a subscription based service, and you can view the pricing plans on the [Ubuntu Pro pricing plans](https://ubuntu.com/pricing/pro) page. Ubuntu Pro is also free for personal use if you are a contributing member to the Ubuntu community. This means your collaboration with Ubuntu has been recognized for your continuous contributions. More info at: [Ubuntu community membership](https://wiki.ubuntu.com/Membership).

Official: [Ubuntu Pro | Ubuntu](https://ubuntu.com/pro)

# 檔案及目錄加密-Encrypt

# 加密軟體工具

##### Linux

1. [Cryptsetup](https://gitlab.com/cryptsetup/cryptsetup) - 支援 LUKS/plain dm-crypt/loop-AES/TrueCrypt/BitLocker 等加密格式 
    - [How to Encrypt and Decrypt a Partition in Linux](https://www.linuxshelltips.com/encrypt-partition-linux/)
2. [eCryptFS](https://www.xmodulo.com/encrypt-files-directories-ecryptfs-linux.html)
3. [EncFS](https://www.makeuseof.com/tag/encrypt-dropbox-data-encfs-linux/)
4. [CryFS](https://www.cryfs.org/tutorial/)
5. [VeraCrypt](https://archive.codeplex.com/?p=veracrypt) - 跨平台支援，可移動至其他裝置。 
    - [How to Encrypt USB Drive on Linux Using VeraCrypt](https://www.linuxbabe.com/desktop-linux/encrypt-usb-drive-linux-using-veracrypt)

##### Windows

1. [AxCrypt](https://key.chtouch.com/ContentView.aspx?P=280)

##### 使用心得

<div id="bkmrk-%E5%AE%89%E8%A3%9D">安裝</div>- EncryptFS 輕易安裝在 CentOS 6.x
- EncFS 可以安裝在 CentOS 6，需透過 EPEL 安裝 fuse-encfs
- CryFS 只支援 Ubuntu

使用方法

<div id="bkmrk-%3Cencrypt-command%3E-%7E%2F">`<encrypt-command> ~/ENCRYPTED-DIR ~/DECRYPTED-DIR`</div><div id="bkmrk-"></div>- 三個軟體幾乎都用一樣的方式，需指定兩個目錄，加密目錄與解密目錄，其中解密目錄是用掛載的，實際檔案是以加密型態的檔案儲存在加密目錄內。
- 在加密目錄裡的檔案與資料夾都是以隨機亂數的名稱存在，檔案內容也都是亂數的隨意組合，要得知加密目錄內有什麼檔案與存取檔案內容，就必須經過解密。
- 加密目錄內的檔案解密步驟，作法三者幾乎一樣，都是透過解密目錄掛載方式，掛載成功後，解密目錄的存取就與一般正常目錄一樣。
- 解密目錄必須透過指令掛載後才能被使用，未被掛載前，這只是一個空目錄，而且不應該存取任何檔案。
- 解密所需的設定檔與加密資訊，三者各有所異。

<div id="bkmrk-%E6%87%89%E7%94%A8%E7%AF%84%E5%9C%8D">應用範圍</div>1. 個人雲端檔案加密。以 Dropbox 為例，將加密目錄設定為 Dropbox 的同步上傳目錄，解密目錄則為本地端的任何一個目錄，例如文件區。平常存取 dropbox 檔案都在文件區內作業，如此這些檔案實際被上傳到 Dropbox 雲端，是以加密型態檔案存在。
2. 檔案多人共用加密。將加密目錄設定在共用的 NFS 目錄（NAS），要存取的 Linux 工作站安裝加密軟體，然後將 NAS 上的加密目錄做解密掛載在本地端目錄，解密掛載時必須通過解密程序，至少會有一組密碼。

其他注意事項

同仁在存取解密目錄裡的檔案時，就與一般正常未加密目錄一樣，甚至可以將它們複製或下載，而複製後的檔案是呈現未加密型態。因此這類加密軟體不適合想避免敏感資料外流的作法。

##### GPG

驗證下載檔案的數位簽名檔

```
[root@db2v9 gpg]# tree
.
├── npp.7.8.8.Installer.exe
├── npp.7.8.8.Installer.exe.sig
└── nppGpgPub.asc

0 directories, 3 files
[root@db2v9 gpg]#
[root@db2v9 gpg]# gpg --verify npp.7.8.8.Installer.exe.sig npp.7.8.8.Installer.exe
gpg: Signature made Sun 28 Jun 2020 11:02:09 PM CST using RSA key ID 8D84F46E
gpg: Can't check signature: No public key

[root@db2v9 gpg]# gpg --import nppGpgPub.asc
gpg: key 8D84F46E: public key "Notepad++ <don.h@free.fr>" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
gpg: no ultimately trusted keys found

[root@db2v9 gpg]# gpg --verify npp.7.8.8.Installer.exe.sig npp.7.8.8.Installer.exe
gpg: Signature made Sun 28 Jun 2020 11:02:09 PM CST using RSA key ID 8D84F46E
gpg: Good signature from "Notepad++ <don.h@free.fr>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 14BC E436 2749 B2B5 1F8C  7122 6C42 9F1D 8D84 F46E
```

##### 延伸閱讀

- GPG 檔案加解密 
    - [Linux: HowTo Encrypt And Decrypt Files With A Password](https://www.cyberciti.biz/tips/linux-how-to-encrypt-and-decrypt-files-with-a-password.html)
    - [How to easily encrypt a file with GPG on Linux](https://linuxaria.com/howto/how-to-easily-encrypt-a-file-with-gpg-on-linux)
    - [Encrypting and decrypting files with password in Linux](https://www.blackmoreops.com/2015/05/07/encrypting-files-with-password/)
- 密碼保護 
    - [Linux: HowTo Encrypt And Decrypt Files With A Password](https://www.cyberciti.biz/tips/linux-how-to-encrypt-and-decrypt-files-with-a-password.html)
    - [7 Tools to Encrypt/Decrypt and Password Protect Files in Linux](https://www.tecmint.com/linux-password-protect-files-with-encryption/)
- LUKS  
    
    - [Implementing corporate laptop encryption using LUKS](https://www.redhat.com/en/blog/implementing-corporate-laptop-encryption-using-luks)
    - [How to unlock LUKS using Dropbear SSH keys remotely in Linux - nixCraft (cyberciti.biz)](https://www.cyberciti.biz/security/how-to-unlock-luks-using-dropbear-ssh-keys-remotely-in-linux/)

# 加密你的隨身碟 - Cryptsetup

##### 說明

如果你想對某個 USB隨身碟內的資料做加密保護，每次掛載前必須先輸入一組密碼。

##### 開始加密

安裝套件

```shell
sudo apt install cryptsetup
```

插入要加密的隨身碟，並完成掛載，假設為 /dev/sdb。使用 GParted 工具建立一個新磁區 /dev/sdb1，格式為 ext4。拔除隨身碟後再重新插入，如果隨身碟自動被掛載，先以手動方式將它卸載確定隨身碟在沒有被掛載模式下，執行

```shell
sudo cryptsetup --verbose --verify-passphrase luksFormat /dev/sdb1
```

> 詢問是否確定時，必須輸入 大寫 YES。  
> 接著需設定一組密碼。

開啟加密磁區，與格式化

- 開啟加密磁區時，必須輸入正確的密碼。
- 加密磁區成功被開啟後，會產生一個新磁碟路徑 `/dev/mapper/<input-name>`

```shell
sudo cryptsetup luksOpen /dev/sdb1 sdb1
sudo fdisk -l
sudo mkfs.ext4 /dev/mapper/sdb1
sudo tune2fs -m 0 /dev/mapper/sdb1
```

如果想對加密磁區加上標籤

```shell
sudo e2label /dev/mapper/sdb1 MyCCWallet
```

到這裡，已經完成加密程序。測試手動掛載，並嘗試寫入資料。

```shell
sudo mkdir /mnt/encrypted
sudo mount /dev/mapper/sdb1 /mnt/encrypted
sudo touch /mnt/encrypted/test.txt
```

測試沒問題後，就可以將加密磁區關閉。

```shell
sudo umount /dev/mapper/sdb1
sudo cryptsetup luksClose sdb1
```

##### Linux Mint 桌面測試

隨身碟在經過加密後，當每次插上 Linux Mint 電腦時，桌面會自動詢問解鎖密碼，在輸入正確密碼後，就可以像一般方式那樣直接存取隨身碟裡的檔案。

[![ask_password.png](http://www.osslab.tw/uploads/images/gallery/2020-12/scaled-1680-/ask_password.png)](http://www.osslab.tw/uploads/images/gallery/2020-12/ask_password.png)

##### Optional: 其他指令

How to find and verify which Luks slot a passphrase is in on Linux

```shell
# Do not activate device, just check
sudo cryptsetup --verbose open --test-passphrase /dev/vda3

Enter passphrase for /dev/vda3: 
Key slot 0 unlocked.
Command successful.

```

##### Learning Cryptsetup

- [How to change LUKS disk encryption passphrase in Linux](https://www.cyberciti.biz/security/how-to-change-luks-disk-encryption-passphrase-in-linux/)
- [Create an encrypted file vault on Linux](https://opensource.com/article/21/4/linux-encryption)

# Learning

##### GPG

- [Using GPG to Encrypt and Decrypt Files on Linux \[Hands-on for Beginners\]](https://itsfoss.com/gpg-encrypt-files-basic/)

##### LUKS

- [How to unlock LUKS using Dropbear SSH keys remotely in Linux - nixCraft (cyberciti.biz)](https://www.cyberciti.biz/security/how-to-unlock-luks-using-dropbear-ssh-keys-remotely-in-linux/)
- [Linux Hard Disk Encryption With LUKS \[cryptsetup command \] - nixCraft (cyberciti.biz)](https://www.cyberciti.biz/security/howto-linux-hard-disk-encryption-with-luks-cryptsetup-command/)

# OpenSSL

Create Example Reference File, let us create a 1GB large text file using the fallocate command:

```shell
fallocate -l 1024M test.txt
echo "LinuxShellTips tutorial on encrypting a large file with OpenSSL in Linux" >> test.txt
cat test.txt
```

##### Encrypt File with Password (對稱式加密)  


```shell
openssl enc -aes-256-cbc -pbkdf2 -p -in test.txt -out test.txt.enc
```

- enc executes the symmetric key encryption process.
- -aes-256-cbc specifies the use of 256 bits cryptographic key.
- -pbkdf2 is the default algorithm being used.
- -p prints used salt, key, and IV.
- -in points to the input file.
- -out points to the output file.

To decrypt the file, run:

```shell
openssl aes-256-cbc -d -pbkdf2 -in test.txt.enc -out sample_decrypted.txt
```

> You will be required to enter the encryption password you generated earlier.

##### Encrypt File with Key (對稱式加密)  


```shell
# generate a key file
openssl rand 256 > symmetric_keyfile.key
# use the keyfile to encrypt our file
openssl enc -in test.txt -out test.txt.enc -e -aes-256-cbc -pbkdf2 -k symmetric_keyfile.key
```

To decrypt the file, run:

```shell
openssl enc -in test.txt.enc -out draft_decrypted.txt -d -aes-256-cbc -pbkdf2 -k symmetric_keyfile.key
```

##### 非對稱式加密 (Asymmetric Encryption)

使用非對稱式加密對一個大檔案進行加密，可能遇到錯誤: data too large for key size.

[![data-too-large-for-key-size.png](https://osslab.tw/uploads/images/gallery/2022-07/scaled-1680-/data-too-large-for-key-size.png)](https://osslab.tw/uploads/images/gallery/2022-07/data-too-large-for-key-size.png)

> TIP: 非對稱加密又稱公鑰加密。在使用前要準備好一對私鑰與公鑰，使用公鑰進行檔案的加密，解密時則使用私鑰，操作上較複雜，但是安全性較佳。

##### Hashing

檔案校驗

```bash
# For file
openssl dgst -sha256 my.file

# For string
echo -n "HelloWorld" | openssl dgst -sha256
```

##### Generate a token key

```bash
openssl rand -hex 32
```

##### 參考網站  


- [OpenSSL 對稱式、非對稱式加密檔案指令教學與範例](https://officeguide.cc/linux-openssl-file-symmetic-asymmetric-encryption-commands-tutorial-examples/)

# nmap

Nmap是一款用於網路發現和安全審計的網路安全工具，它是自由軟體。軟體名字Nmap是Network Mapper的簡稱。通常情況下，Nmap用於： 列舉網路主機清單 管理服務升級排程 監視主機 服務執行狀況 Nmap可以檢測目標主機是否線上、埠開放情況、偵測執行的服務類型及版本資訊、偵測作業系統與裝置類型等資訊。

# nmap 常用指令集

##### 掃描單一主機

```shell
### Scan a single ip address ###
nmap 192.168.1.1
 
## Scan a host name ###
nmap server1.cyberciti.biz
 
## Scan a host name with more info###
nmap -v server1.cyberciti.biz
```

##### 掃描多個主機

```shell
nmap 192.168.1.1 192.168.1.2 192.168.1.3

## works with same subnet i.e. 192.168.1.0/24
nmap 192.168.1.1,2,3

## You can scan a range of IP address too:
nmap 192.168.1.1-20

## You can scan a range of IP address using a wildcard:
nmap 192.168.1.*

## you scan an entire subnet:
nmap 192.168.1.0/24

# Ping scan subnet
nmap -sP 10.15.9.0/24 | grep -E '\b(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)'
nmap -sP 10.15.9.0/24 | sed -n '/^.* [0-9]*\.[0-9]*\.[0-9]*\.[0-9]*/p' | sed 's/^.* \([0-9]*\.[0-9]*\.[0-9]*\.[0-9]*\).*$/\1 /g'
```

##### 從檔案讀入 IP 清單

```shell
nmap -iL /tmp/ip.txt
```

##### 排除 IP 的方法

```shell
nmap 192.168.1.0/24 --exclude 192.168.1.5
nmap 192.168.1.0/24 --exclude 192.168.1.5,192.168.1.254
nmap -iL /tmp/scanlist.txt --excludefile /tmp/exclude.txt
```

##### 偵測遠端的作業系統類型

```shell
nmap -A 192.168.1.254
nmap -v -A 192.168.1.1
nmap -A -iL /tmp/scanlist.txt 
```

##### 偵測遠端主機是否有防火牆

```shell
nmap -A 192.168.1.254
nmap -v -A 192.168.1.1
nmap -A -iL /tmp/scanlist.txt
```

##### 掃描主機(有防火牆保護時)

```shell
nmap -PN 192.168.1.1
nmap -PN server1.cyberciti.biz
```

##### 掃描 IPv6 主機

```shell
nmap -6 IPv6-Address-Here
nmap -6 server1.cyberciti.biz
nmap -6 2607:f0d0:1002:51::4
nmap -v A -6 2607:f0d0:1002:51::4
```

##### 掃描一個子網路內有哪些主機/裝置

```shell
nmap -sP 192.168.1.0/24
```

##### 執行快速掃描

```shell
nmap -F 192.168.1.1
```

##### 顯示通訊埠狀態原因(Reason)

```shell
nmap --reason 192.168.1.1
```

##### 顯示開啟中的通訊埠

```shell
nmap --open 192.168.1.1
```

##### 顯示已傳送/接收的封包

```shell
nmap --packet-trace 192.168.1.1
```

##### 顯示本機網路介面裝置與路由

```shell
nmap --iflist
```

##### 掃描指定通訊埠

```shell
nmap -p [port] hostName

## Scan port 80
nmap -p 80 192.168.1.1
 
## Scan TCP port 80
nmap -p T:80 192.168.1.1
 
## Scan UDP port 53
nmap -p U:53 192.168.1.1
 
## Scan two ports ##
nmap -p 80,443 192.168.1.1
 
## Scan port ranges ##
nmap -p 80-200 192.168.1.1
 
## Combine all options ##
nmap -p U:53,111,137,T:21-25,80,139,8080 192.168.1.1
nmap -p U:53,111,137,T:21-25,80,139,8080 server1.cyberciti.biz
nmap -v -sU -sT -p U:53,111,137,T:21-25,80,139,8080 192.168.1.254
 
## Scan all ports with * wildcard ##
nmap -p "*" 192.168.1.1
 
## Scan top ports i.e. scan $number most common ports ##
nmap --top-ports 5 192.168.1.1
nmap --top-ports 10 192.168.1.1
```

##### 快速掃描網路內有開啟通訊埠的主機/裝置

```shell
nmap -T5 192.168.1.0/24
```

##### Cheat Sheet

[![nmap_cheatsheet.jpg](https://osslab.tw/uploads/images/gallery/2022-09/scaled-1680-/nmap-cheatsheet.jpg)](https://osslab.tw/uploads/images/gallery/2022-09/nmap-cheatsheet.jpg)

# Learning nmap

##### 線上教學

- [Top 32 Nmap Command Examples For Linux Sys/Network Admins](https://www.cyberciti.biz/security/nmap-command-examples-tutorials/)
- \[中文\] [https://www.lijyyh.com/2012/03/nmap-using-nmap-security-scanner.html](https://www.lijyyh.com/2012/03/nmap-using-nmap-security-scanner.html)
- [How To Use Nmap Security Scanner (Nmap Commands)](http://www.linuxandubuntu.com/home/how-to-use-nmap-security-scanner-nmap-commands)
- [Nmap for Pentester](https://github.com/Ignitetechnologies/Nmap-For-Pentester)
- [Nmap Command Examples For Linux Users / Admins - nixCraft (cyberciti.biz)](https://www.cyberciti.biz/security/nmap-command-examples-tutorials/)

# sar

- <span class="application">**[How to use SAR to Monitor System Performance in Red Hat Enterprise Linux - Red Hat Customer Portal](https://access.redhat.com/solutions/276533)**</span>

##### <span class="application">**簡介**</span>

<span class="application">**sar**</span>（系統活動回報程式）- Sar - RedHat/CentOS 內建的系統效能分析工具，這工具會蒐集、回報本日截至目前為止的系統活動資訊。預設的資訊包括本日的 CPU 使用率，每十分鐘採樣一次。

特點：

- 內建在 CentOS/RedHat 5+版本，無需另外安裝。
- 可查詢最近幾日的系統資源使用歷史紀錄，常作為事件後的問題偵錯工具。

可分析的系統資訊包含如下：

- CPU / IO / System / Nice / Idle percentages
- Network Traffic / Network Errors
- Load Average and Run queue
- Interrupts
- Memory Free / Cached / Buffered / Swapped
- Device usage per Major/Minor number
- And many others

##### Sar 如何運作

- SAR writes to log files in /var/log/sa. This directory holds two types of files - sa\\#\\# files (binary) and sar\\#\\# files (text).
- The number at the end of the file corresponds to the day of the month that file was recording.
- For example, an sa03 file refers to the 03 day of the month.
- When the sysstat package is installed it places a file into /etc/cron.d/sysstat.
- This sets up two cron jobs. 
    1. job to record statistics every 10 minutes.
    2. job to write the binary sa\\#\\# file to a text sar\\#\\# file once a day (typically right before midnight).
- Additionally, it places a configuration file in /etc/sysconfig/sysstat.
- 預設是保留最近七日的歷史資料，若要變更，可以修改 /etc/sysconfig/sysstat。  
    Note that RHEL 4/5 sysstat does not support keeping more than 1 month of data; however, in RHEL6 if a HISTORY value greater than 28 is declared, SAR log files are automatically split up into separate directories.

Sar Cron Jobs：  
/etc/cron.d/sysstat

```
# run system activity accounting tool every 10 minutes 
*/10 * * * * root /usr/lib/sa/sa1 1 1 

# generate a daily summary of process accounting at 23:53 
53 23 * * * root /usr/lib/sa/sa2 -A
```

If it is desired for SAR to collect data more frequently, simply change "\*/10" to a new interval.

For example, if to make SAR to track every 5 minutes, simply change to "\*/5".

> NOTE:
> 
> SAR does not add significant load to a server. It safely can be tuned down to 2 minute intervals without seeing a significant problem. SAR also does not grab individual block data.
> 
> RHEL 8/9 不再使用 crontab 方式，如要調整 interval，請見下方步驟。

##### 安裝

RedHat 5/6/7/8/9

```
yum install sysstat
```

For RHEL 8/9 only

```
systemctl start sysstat-collect.timer
```

##### 設定 Interval (for RHEL 8/9)

執行

```
systemctl cat sysstat-collect.timer
export SYSTEMD_EDITOR=/usr/bin/vi
systemctl edit sysstat-collect.timer
```

上述指令會開啟檔案: `/etc/systemd/system/sysstat-collect.timer.d/override.conf` ，將下方內容貼上

> NOTE: 有一行 OnCalendar=&lt;空白&gt;，這行主要是用來移除預設值。

```
[Unit]
Description=Run system activity accounting tool every 1 minute

[Timer]
OnCalendar=
OnCalendar=*:00/1
```

套用變更

```
systemctl daemon-reload
systemctl restart sysstat-collect.timer
```

驗證

```
systemctl cat sysstat-collect.timer
systemctl status sysstat-collect.timer
```

##### 使用方法

顯示今日 CPU 狀態

```shell
# 每個處理器
sar -P ALL

# 顯示 CPU
sar -u 
```

顯示最近月份 13 號的網路狀態

```
sar -n ALL -f /var/log/sa/sa13
```

顯示最近一個月份 7 號 時間 10:00 - 14:00 的記憶體使用狀況，並將結果導出一個檔案

```
sar -r -s 10:00:00 -e 14:00:00 -f /var/log/sa/sa07 -o /tmp/mem.txt
```

即時監看

```shell
# CPU on the Fly 10 times every 2 seconds
sar -u 2
sar -u 2 10
# Output to the file and read the file
sar -u 2 10 -o cpu.sa >/dev/null 2>&1 
sar -f cpu.sa

# Memory
# kbcommit & %commit is the overall memory used including RAM & Swap
sar -r 1
sar -r 1 10

# Swap
sar -S 1
sar -S 1 10

# I/O
sar -b 1
sar -b 1 10
sar -p -d 1
sar -p -d 1 10

# Paging
# - majflts/s shows the major faults per second means number of pages loaded into the memory from disk (swap), 
#   if its value is higher then we can say that system is running out of RAM.
# - %vmeff indicates the number of pages scanned per second, if it’s vaule is 100 % its is consider OK and 
#   when it is below 30 % then there is some issue with virtual memory. Zero value indicates that there is no page scanned during that time.
sar -B 1

# Network
sar -n ALL
```

> Tips:
> 
> - Memory Swaping 嚴重：papgin/papgout/majflt 連續不斷出現高數值

##### 線上分析器

RedHat 提供幾個透過線上的工具來分析  
I/O 使用分析: [https://access.redhat.com/labs/rhiou/](https://access.redhat.com/labs/rhiou/)

> 在系統內先執行 lsblk 將輸出內容導出一個檔案 lsblk.out，將此檔連同要分析的任一個 sarXX 檔上傳該網址，即可以圖形顯示系統 I/O 的使用狀態。

Memory 使用分析: [https://access.redhat.com/labs/rhma/](https://access.redhat.com/labs/rhma/)

##### 延伸閱讀

- [Monitoring Linux system resources using SAR (System Activity Report)](https://linuxtechlab.com/monitoring-system-resources-using-sar/)
- [Sar command usage with examples in Linux](https://www.crybit.com/sar-command/)
- [The Sysadmin's Toolbox: sar](https://www.linuxjournal.com/content/sysadmins-toolbox-sar)

# NIS Server

網路資訊服務簡稱 NIS，時又譯為網路資料服務協定，亦即一般簡稱之「黃頁」YP，最早由昇陽公司開發出來，為一套用來管理電腦網路中所有與電腦系統管理相關之設定檔，如使用者帳號、密碼、主機名稱或群組等的主從式目錄服務協定。

##### Tutorials

- \[RH-KB\] [How do I configure a NIS server and client?](https://access.redhat.com/solutions/7247)
- \[RH-KB\] [How to configure NIS (Network Information Service) Master and Slave servers in Red Hat Enterprise Linux](https://access.redhat.com/solutions/47192)

# Bash Background Process

```shell
# Starting a process in the background and bringing it back to the foreground
$ sleep 1000 &
[1] 25867
$ fg
sleep 1000

# Disowning a process
$ sleep 1000 &
[1] 26090
$ disown %1
$ 

$ sleep 1000 &
[1] 26214
$ disown %1
$ ps -ef | grep sleep | grep -v grep
roel     26214 26120  0 13:13 pts/3    00:00:00 sleep 1000
$ exit
# Then, opening a new shell and re-executing
$ ps -ef | grep sleep | grep -v grep
roel     26214     1  0 19:48 ?        00:00:00 sleep 1000

# Placing a command into the background
$ sleep 1000
^Z
[1]+  Stopped                 sleep 1000
$ bg %1
[1]+ sleep 1000 &
$ 

$ sleep 1000
^Z
[1]+  Stopped                 sleep 1000
$ bg %1
[1]+ sleep 1000 &
$ disown %1
$

# Multiple background processes and terminating processes
$ sleep 1000 &
[1] 27158
$ sleep 1000 &
[2] 27159
# We can see here that two background processes ([1] and [2], with PID’s 27158 and 27159 respectively) were started. 
# Next, we kill the first process:
$ kill %1
$ 
[1]-  Terminated              sleep 1000
$

# One done before the other
$ sleep 1000 &
[1] 27406
$ sleep 3 &
[2] 27407
$
# After about 5 seconds, pressing enter, we will see
$
[2]+  Done                    sleep 3
# What will happen now if we use fg in this case without the original [1] specifier?
$ fg
sleep 1000
^Z
[1]+  Stopped                 sleep 1000
$ 
# The first process will continue! This is also the case if the reverse procedure were used
$ sleep 10 &
[1] 27346
$ sleep 1000 &
[2] 27347
$ 
[1]-  Done                    sleep 10
$ fg
sleep 1000
^Z
[2]+  Stopped                 sleep 1000

```

# Grub2

##### RedHat/CentOS 7

設定檔：

- /etc/default/grub，通用設定
- /etc/grub.d/40\_custom，自訂內容

建立開機用的設定檔（請勿手動編輯）

```shell
grub2-mkconfig -o /boot/grub2/grub.cfg

# 如果是 EFI 開機，改用這行
grub2-mkconfig -o /boot/efi/EFI/centos/grub.cfg
```

修改預設的開機選項  
預設的項目是透過 /etc/default/grub 檔內的 GRUB\_DEFAULT 行來定義。不過，要是 GRUB\_DEFAULT 行被設定為 saved，這個選項便儲存在 /boot/grub2/grubenv 檔內。你可以這樣檢視它：

```shell
# 列出目前所有開機選項
awk -F\' '$1=="menuentry " {print $2}' /boot/grub2/grub.cfg
# 或
grep "^menuentry" /boot/grub2/grub.cfg | cut -d "'" -f2

# 目前預設的開機
grub2-editenv list

saved_entry=CentOS Linux (3.10.0-229.14.1.el7.x86_64) 7 (Core)

# 調整預設開機
grub2-set-default 2              # NOTE: 開機選項清單的第一項編號是 0
```

##### 密碼保護 GRUB

避免用戶以 single user mode 進入修改 root 密碼。

建立密碼

```
[root@rh4 ~]# /sbin/grub-md5-crypt
Password:
Retype password:
$1$Nk8AX$zsPr79zeiwd0zwF94DND60
```

修改 /boot/grub/grub.conf

增加 password 那兩行，與填入密碼的加密字串。

```
default=0
timeout=5

# Password Protecting GRUB
password --md5 $1$Nk8AX$zsPr79zeiwd0zwF94DND60

splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
```

##### Learning

- [How to Rescue, Repair and Reinstall GRUB Boot Loader in Ubuntu](https://www.tecmint.com/rescue-repair-and-reinstall-grub-boot-loader-in-ubuntu/)

# CentOS Kernel Upgrade

RedHat KB: [CHAPTER 5. MANUALLY UPGRADING THE KERNEL](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/kernel_administration_guide/ch-manually_upgrading_the_kernel)

##### Kernel Packages

CentOS/RedHat Linux 與 kernel 升級相關的套件清單

- kernel — Contains the kernel for single-core, multi-core, and multi-processor systems.
- kernel-debug — Contains a kernel with numerous debugging options enabled for kernel diagnosis, at the expense of reduced performance.
- kernel-devel — Contains the kernel headers and makefiles sufficient to build modules against the kernel package.
- kernel-debug-devel — Contains the development version of the kernel with numerous debugging options enabled for kernel diagnosis, at the expense of reduced performance.
- kernel-doc — Documentation files from the kernel source. Various portions of the Linux kernel and the device drivers shipped with it are documented in these files. Installation of this package provides a reference to the options that can be passed to Linux kernel modules at load time.  
    By default, these files are placed in the /usr/share/doc/kernel-doc-kernel\_version/ directory.

- kernel-headers — Includes the C header files that specify the interface between the Linux kernel and user-space libraries and programs. The header files define structures and constants that are needed for building most standard programs.
- linux-firmware — Contains all of the firmware files that are required by various devices to operate.
- perf — This package contains the perf tool, which enables performance monitoring of the Linux kernel.
- kernel-abi-whitelists — Contains information pertaining to the Red Hat Enterprise Linux kernel ABI, including a lists of kernel symbols that are needed by external Linux kernel modules and a yum plug-in to aid enforcement.
- kernel-tools — Contains tools for manipulating the Linux kernel and supporting documentation.

##### Preparing to upgrade

```shell
# Current kernel running
uname -a
# Grub2 Info for syatem booting
awk -F\' '$1=="menuentry " {print $2}' /boot/grub2/grub.cfg

grub2-editenv list

# Current kernel packages installed
rpm -qa | grep kernel
# Or
yum list installed "kernel-*"
```

##### Downloading the upgraded kernel

從 [RedHat Custom Portal](https://access.redhat.com/) 網站下載新版的 kernel 套件

1. `kernel-3.10.0-1062.el7.x86_64.rpm`。
2. `linux-firmware-20190429-72.gitddde598.el7.noarch.rpm` (這是第一項安裝的相依性檔案)

##### Performing the upgrade

升級 kernel 時，系統會自動使用 Dracut 工具來建立新的 initialRAM file system 開機用映像檔。請確定 Dracut 已經有安裝。

升級 kernel

```shell
# 先升級 linux-firmware
# NOTE: 安裝指令不能使用 -ivh
rpm -Uvh linux-firmware-20190429-72.gitddde598.el7.noarch.rpm

# 再升級 kernel
# NOTE: 安裝時不要使用 -Uvh 
rpm -ivh kernel-3.10.0-1062.el7.x86_64.rpm
```

##### 其他指令

安裝新 kernel 後會作為下一次開機的預設選單，設定檔在 `/etc/sysconfig/kernel`。

```
# UPDATEDEFAULT specifies if new-kernel-pkg should make
# new kernels the default
UPDATEDEFAULT=yes

# DEFAULTKERNEL specifies the default kernel package type
DEFAULTKERNEL=kernel

```

Listing the contents of the initial RAM file system image

```shell
# To list the files that are included in the initramfs, run the following command as root:
lsinitrd

# To only list files in the /etc directory, use the following command:
lsinitrd | grep etc/

# To output the contents of a specific file stored in the initramfs for the current kernel, use the -f option:
lsinitrd -f filename

# For example, to output the contents of sysctl.conf, use the following command:
lsinitrd -f /etc/sysctl.conf

# To specify a kernel version, use the --kver option:
lsinitrd --kver kernel_version -f /etc/sysctl.conf

# For example, to list the information about kernel version 3.10.0-327.10.1.el7.x86_64, use the following command:
lsinitrd --kver 3.10.0-327.10.1.el7.x86_64 -f /etc/sysctl.conf
```

Reversing Changes Made to the Initial RAM File System Image

如果需要手動重建 initialramfs file 時：

1. Reboot the system choosing the rescue kernel in the GRUB menu.
2. Change the incorrect setting that caused the initramfs to malfunction.
3. Recreate the initramfs with the correct settings by running the following command as root:

```shell
dracut --kver <kernel_version> --force
```

##### FAQ

linux-firmware package files conflicts

Solution:

```shell
# Install kernel
rpm -ivh kernel-3.10.0-1062.el7.x86_64.rpm

warning: kernel-3.10.0-1062.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID fd431d51: NOKEY
error: Failed dependencies:
        linux-firmware >= 20190429-72 is needed by kernel-3.10.0-1062.el7.x86_64
        
# Install linux-firmware
rpm -ivh linux-firmware-20190429-72.gitddde598.el7.noarch.rpm kernel-3.10.0-1062.el7.x86_64.rpm

warning: linux-firmware-20190429-72.gitddde598.el7.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY
warning: kernel-3.10.0-1062.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID fd431d51: NOKEY
Preparing...                          ################################# [100%]
        file /usr/lib/firmware/amd-ucode/microcode_amd_fam17h.bin from install of linux-firmware-20190429-72.gitddde598.el7.noarch conflicts with file from package linux-firmware-20180911-69.git85c5d90.el7.noarch
        file /usr/lib/firmware/amd-ucode/microcode_amd_fam17h.bin.asc from install of linux-firmware-20190429-72.gitddde598.el7.noarch conflicts with file from package linux-firmware-20180911-69.git85c5d90.el7.noarch
        
# Solution: using -Uvh to upgrade the linux-firmware 
rpm -Uvh linux-firmware-20190429-72.gitddde598.el7.noarch.rpm
rpm -ivh kernel-3.10.0-1062.el7.x86_64.rpm
```

# sudo

- [The Difference Between Su, Sudo Su, Sudo -i, and Sudo -s](https://trendoceans.com/difference-between-su-sudo-su-sudo-i-and-sudo-s/)
- [How To Run All Programs In A Directory Via Sudo In Linux](https://ostechnix.com/run-programs-in-a-directory-via-sudo/)
- [10 Useful Sudoers Configurations for Setting 'sudo' in Linux](https://www.tecmint.com/sudoers-configurations-for-setting-sudo-in-linux/)

##### 常用指令

```shell
# 以用戶 john 執行 SHELL
sudo -u john bash -c "whoami"

# 以用戶 john 開啟 vnc 服務，-H 可保持用戶的 HOME；否則會是 /root。
sudo -H -u john bash -c "vncserver" 

# 以用戶 devrpt 執行 db2_get_tbs_usage.sh -d devrptdb
sudo -u devrpt -H -s /usr/local/bin/db2_get_tbs_usage.sh -d devrptdb
# 或
sudo -u devrpt -H bash -c "/usr/local/bin/db2_get_tbs_usage.sh -d devrptdb" 

# 避免密碼詢問
echo "ThisIsPassword" | sudo -S ls -l /root

# 檢查 sudo 權限
sudo -l -U <username>
```

##### 設定格式  


```
alang  ALL=(devrpt) /bin/bash,/usr/local/bin/db2_get_tbs_usage.sh 
```

> 內容格式:
> 
> &lt;使用者帳號&gt; &lt;登入者的來源主機名稱&gt;=&lt;可切換的身份&gt; &lt;可下達的指令&gt;
> 
> 如果要執行 SHELL Scripts，除了 scripts 路徑以外，還要加上 /bin/bash。

##### 限制可切換的帳號

限制某帳號只能切換至特定的另一個帳號，其餘帳號不允許切換。

```
developer-name  ALL=(ALL)       NOPASSWD:/usr/bin/sudo,/bin/su - app-user
```

##### 多管理者帳號

除了主要管理者需要知道 root 密碼以外，其餘管理者先以個人帳號登入系統，然後以免 root 密碼方式切換成 root。

```shell
# RedHat 7/8, /etc/sudoers.d/sysadmin
your-account    ALL=(ALL)       ALL
```

> 免 root 密碼切換至 root 方式

```shell
# RedHat 6/7/8
# 如果有詢問密碼，輸入個人帳號的密碼
sudo -i
sudo su -

# RedHat 4
sudo su -
# Or
sudo -s # NOTE: not root's profile
```

> sudo -i 沒有包含 /usr/local/bin 路徑？

方法一：以 root 執行 `visudo`

```
Defaults    secure_path = /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin
```

方法二：`/root/.bash_profile`

```bash
# Custom PATH
case :$PATH: in
    *:/usr/local/bin:*) ;;
    *) PATH=/usr/local/bin:$PATH ;;
esac
export PATH
```

<span style="color:rgb(187,187,187);font-size:1.4em;font-weight:400;">Disable 'su'</span>

Disable 'su' Access for Sudo Users in Linux

`visudo` :

```
# Cmnd alias spceification 
Cmnd_Alias DISABLE_SU = /bin/su

# Disable the su for the user trendoceans
trendoceans ALL=ALL, !DISABLE_SU
```

##### logfile

`visudo`:

```
# Log File:
Defaults    logfile="/var/log/sudo.log"
```

# tmux

##### Sessions

Start session

- `tmux new`
- `tmux new -s mysession`

Show sessions

- tmux ls

Kill sessions

- `tmux kill-session  -t mysession`
- `tmux kill-session -a` ;kill all sessions but last session

Attach sessions

- `tmux a `
- `tmux a -t #`
- `tmux a -t mysession`

Save &amp; Restore

- [How To Save And Restore Tmux Environments Across Reboots In Linux](https://ostechnix.com/save-and-restore-tmux-environment/)

##### Keyboard

- [Tmux Cheat Sheet](https://tmuxcheatsheet.com/)

PREFIX: CTRL+b

sessions

- Detach from session: PREFIX + d
- Rename session: PREFIX + $
- Move to previous session: PREFIX + (
- Move to next session: PREFIX + )

Windows

- Create a new window: PREFIX + c
- Rename the window: PREFIX + ,
- Move the window: PREFIX + {window-number}
- Close current window: PREFIX + &amp;
- Move to previous window: PREFIX + p
- Move to next window: PREFIX + n
- Go to any window: PREFIX + s

Panes

- Split panes horizontally: PREFIX + "
- Split panes vertically: PREFIX + %
- Switch between panes: PREFIX + Left/Right/Up/Down
- Swap between horizontal and vertical layouts: PREFIX + space
- Toggle pane zoom: PREFIX + z

Copy Mode

- Enter copy mode: PREFIX + \[
- Exit copy mode: q
- Search key word: Ctrl + s
- Move to next search: n
- Move to previous search: N

##### Configuration

`~/.tmux.conf`

Reload the configuration

```bash
tmux source ~/.tmux.conf
```

For Ubuntu/Debian

```
# Enable 256 color in terminal
set -g default-terminal "tmux-256color"

# Fix the dynamic title name
set-option -g allow-rename off

# set vi-mode
set-window-option -g mode-keys vi
# keybindings
bind-key -T copy-mode-vi v send-keys -X begin-selection
bind-key -T copy-mode-vi C-v send-keys -X rectangle-toggle
bind-key -T copy-mode-vi y send-keys -X copy-selection-and-cancel

# Window status
set-window-option -g window-status-current-style 'bg=#282a36 fg=#f8f8f2 bold'
```

For RHEL

```
# Fix the dynamic title name
set-option -g allow-rename off
```

vi-mode

- 行模式：Shift + v
- 列模式：Ctrl + v
- 取消：y
- 複製：Enter
- 貼上：PREFIX + \]

```
# set vi-mode
set-window-option -g mode-keys vi
# keybindings
bind-key -T copy-mode-vi v send-keys -X begin-selection
bind-key -T copy-mode-vi C-v send-keys -X rectangle-toggle
bind-key -T copy-mode-vi y send-keys -X copy-selection-and-cancel
```

##### Plugin

- [Tmux Plugin Manager: A Must-Have For Tmux Power Users](https://ostechnix.com/install-tmux-plugin-manager/)

##### FAQ

> Window 無法固定標題名稱

Solution: `~/.tmux.conf`

```
set-option -g allow-rename off
```

##### Learn

- \[Video\] [Tmux has forever changed the way I write code.](https://www.youtube.com/watch?v=DzNmUNvnB04)
- [ssh 遠端登入後自動開啟或進入 Tmux](https://ostechnix.com/enhancing-ssh-login-with-a-tmux-session-selection-menu/)

# SAN & iSCSI

SAN - Storage Area Network, iSCSI - Internet Small Computer System Interface

# iSCSI and multipath on CentOS 7

##### 系統環境

- iSCSI Storage: NetApp FAS8200
- Linux OS: CentOS 7
- Server H/W: HP DL380 Gen10
- iSCSI 用網路卡: 10G SFP 光纖網路卡 x 2

##### 設定網路卡

使用指令 nmtui 分別設定兩張網卡的 IP 與 Mask

> 注意: 這兩張網卡不要作 Bonding。

##### 設定 iSCSI

安裝 iscsi 套件

```shell
yum install iscsi-initiator-utils
```

設定 iscsid.conf: `/etc/iscsi/iscsid.conf`

```
# 調整 timeout
#node.session.timeo.replacement_timeout = 120
node.session.timeo.replacement_timeout = 5

# 調整 node 自動 Login
node.startup = automatic
```

設定 initiator name: `/etc/iscsi/initiatorname.iscsi`

命名規則可以參考

- iqn.&lt;year-to-date&gt;.&lt;your-domain&gt;.&lt;your-hostname&gt;:redhat-disk  
    例如 InitiatorName=iqn.2019-02.mycompany.bdb7-a:redhat-disk
- initiator name 必須與 Storage 端的設定相同
- 不同的 Host 之間的名稱不能相同

##### 安裝 multipath 與 netapp\_linux\_unified\_host\_utilities

```
yum install device-mapper-multipath
yum install libhbaapi
rpm -ivh netapp_linux_unified_host_utilities-7-1.x86_64.rpm
```

> 如果沒有安裝 libhbaapi，執行 sanlun 會出現錯誤。

##### 連接 iSCSI storage

新增網卡裝置

```
iscsiadm -m iface -I iscsi_ens2f0 --op=new
iscsiadm -m iface -I iscsi_ens3f0 --op=new
```

Discovery for the storage

```
iscsiadm -m discovery -t sendtargets -p <ip-to-iscsi-storage>:3260
```

> 如果 NetApp storage 有設置多個 iSCSI IPs，只需要對其中一個 IP 作偵測，輸出結果就會顯示所有 IP。

```
[root@bdb7-b RPMs]# iscsiadm -m discovery -t sendtargets -p 10.18.104.32:3260
    10.18.104.32:3260,1038 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16
    10.18.104.34:3260,1040 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16
    10.18.104.33:3260,1039 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16
    10.18.104.31:3260,1037 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16
    10.18.104.32:3260,1038 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16
    10.18.104.34:3260,1040 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16
    10.18.104.33:3260,1039 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16
    10.18.104.31:3260,1037 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16
```

Login nodes

```
iscsiadm -m node -l all
```

> 輸出結果會顯示多行如下資訊

```
[root@bdb7-b RPMs]# iscsiadm -m node -l all
    Logging in to [iface: iscsi_ens2f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.32,3260] (multiple)
    Logging in to [iface: iscsi_ens3f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.32,3260] (multiple)
    Logging in to [iface: iscsi_ens2f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.34,3260] (multiple)
    Logging in to [iface: iscsi_ens3f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.34,3260] (multiple)
    Logging in to [iface: iscsi_ens2f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.33,3260] (multiple)
    Logging in to [iface: iscsi_ens3f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.33,3260] (multiple)
    Logging in to [iface: iscsi_ens2f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.31,3260] (multiple)
    Logging in to [iface: iscsi_ens3f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.31,3260] (multiple)
    Login to [iface: iscsi_ens2f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.32,3260] successful.
    Login to [iface: iscsi_ens3f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.32,3260] successful.
    Login to [iface: iscsi_ens2f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.34,3260] successful.
    Login to [iface: iscsi_ens3f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.34,3260] successful.
    Login to [iface: iscsi_ens2f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.33,3260] successful.
    Login to [iface: iscsi_ens3f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.33,3260] successful.
    Login to [iface: iscsi_ens2f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.31,3260] successful.
    Login to [iface: iscsi_ens3f0, target: iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16, portal: 10.18.104.31,3260] successful.
```

使用 sanlun 確認 LUN Disks

```
[root@bdb7-b RPMs]# sanlun lun show
    controller(7mode/E-Series)/                                  device          host                  lun
    vserver(cDOT/FlashRay)        lun-pathname                   filename        adapter    protocol   size    product
    ---------------------------------------------------------------------------------------------------------------
    TPEDBISCSISVM01               /vol/BDB7_B_4_vol/BDB7_B_4     /dev/sdag       host19     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_4_vol/BDB7_B_4     /dev/sdaf       host20     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_4_vol/BDB7_B_4     /dev/sdae       host17     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_4_vol/BDB7_B_4     /dev/sdad       host18     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_4_vol/BDB7_B_4     /dev/sdac       host16     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_4_vol/BDB7_B_4     /dev/sdab       host15     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_3_vol/BDB7_B_3     /dev/sdz        host20     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_3_vol/BDB7_B_3     /dev/sdaa       host19     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_4_vol/BDB7_B_4     /dev/sdy        host14     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_3_vol/BDB7_B_3     /dev/sdx        host18     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_3_vol/BDB7_B_3     /dev/sdw        host17     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_3_vol/BDB7_B_3     /dev/sdt        host16     iSCSI      500.1g  cDOT
    TPEDBISCSISVM01               /vol/BDB7_B_3_vol/BDB7_B_3     /dev/sdu        host15     iSCSI      500.1g  cDOT
    ...
```

##### 設定 multipath

初始化後會自動產生設定檔 multipath.conf

```
mpathconf --enable --with_multipathd y

```

建立 device alias name 與 LUN WWID 的對應

> <div>注意：在 RedHat 7.6 的 multipathd 預設雖然已設定 user_friendly_names yes</div><div> 但對於 NetApp 的 LUN disks 仍不會自動建立 mpath X 的磁碟名稱，而只會顯示 WWID，詳細原因請見附檔。</div><div></div><div>為了爾後方便管理，必須手動建立磁碟名稱與 WWID 對應。</div>

編輯 /etc/multipath.conf , 在檔案最下方新增以下幾行

```
...
multipaths {
        multipath {
                wwid    3600a0980383136312d3f4e387455564b
                alias   mpath1
        }
        multipath {
                wwid    3600a0980383136312d3f4e387455564c
                alias   mpath2
        }
        multipath {
                wwid    3600a0980383136312d3f4e387455564d
                alias   mpath3
        }
        multipath {
                wwid    3600a0980383136312d3f4e387455564e
                alias   mpath4
        }
}
```

> wwid 可以從指令 multipath -l 取得
> 
> alias 可以設定任一名稱，LUN disks的順序可以從指令 sanlun lun show 取得

載入新設定

```
systemctl reload multipathd
```

執行 multipath -l，應該要顯示 WWID 與 Alias Name

```
    mpath2 (3600a0980383136312d3f4e387455564c) dm-5 NETAPP  ,LUN C-Mode
    size=500G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw
    |-+- policy='service-time 0' prio=0 status=active
    | |- 12:0:0:1 sdo  8:224  active undef running
    | |- 13:0:0:1 sds  65:32  active undef running
    | |- 15:0:0:1 sdaa 65:160 active undef running
    | `- 16:0:0:1 sdae 65:224 active undef running
    `-+- policy='service-time 0' prio=0 status=enabled
      |- 10:0:0:1 sdg  8:96   active undef running
      |- 11:0:0:1 sdk  8:160  active undef running
      |- 14:0:0:1 sdw  65:96  active undef running
      `- 9:0:0:1  sdc  8:32   active undef running
```

##### 開機掛載 iSCSI LUNs

> 注意：若沒有以下設定，將導致系統無法正常開機。

編輯 /etc/fstab , 在 iSCSI disk 每一行都加上 \_netdev

```
#device         mount point     FS      Options Backup  fsck
/dev/myvg/mylv  /mydisk         xfs     _netdev   0      0
```

設定 netfs 服務自動啟動

```
## CentOS 7
systemctl enable remote-fs.target

## CentOS 4/5/6
chkconfig netfs on
```

##### 加大 MTU 設定

網路卡預設 MTU 是 1500，這不適合 iSCSI 的應用，建議加大至 9000。除了 Linux 設定以外，網路連接的 Switch 與 iSCSI Storage (iSCSI Target) 也需要一併做調整。

RedHat 7.x) 檢查目前網路設置是否可接受大 frame 的網路封包傳遞

```
UNSUCCESSFUL

# ping -M do -c 4 -s  8196 10.1.1.21
PING 10.1.1.21 (10.1.1.21) 8185(8213) bytes of data.
PING 10.1.1.21(10.1.1.21) 8196(8224) bytes of data.
ping: local error: Message too long, mtu=1500
ping: local error: Message too long, mtu=1500

SUCCESSFUL

# ping -M do -c 4 -s 1408 10.1.1.21
PING 10.1.1.21 (10.1.1.21) 1408(1436) bytes of data.
1416 bytes from 10.1.1.21: icmp_seq=1 ttl=58 time=224 ms
```

立即調整 MTU (NOTE: 非永久性設置)

```
ip link set dev ens2f0 mtu 9000
```

永久設置方法，在 iSCSI 網路裝置的設定檔 ifcfg-ethXX，加上這行：

```
MTU="9000"
```

重啟系統後生效。

##### 擴充新硬碟 LUN disk

1\. iSCSI Storage 新增 LUN disk 並分派給要使用的 Linux host。

2\. 以下步驟都是 Linux host 要完成的。

3\. 掃描(discover)剛剛新增的 LUN disk

```
[root@bdb7-b ~]# /usr/bin/rescan-scsi-bus.sh
Scanning SCSI subsystem for new devices
Scanning host 0 for  SCSI target IDs  0 1 2 3 4 5 6 7, all LUNs
Scanning host 1 for  SCSI target IDs  0 1 2 3 4 5 6 7, all LUNs
Scanning host 2 for  SCSI target IDs  0 1 2 3 4 5 6 7, all LUNs
Scanning host 3 for  SCSI target IDs  0 1 2 3 4 5 6 7, all LUNs
 Scanning for device 3 0 0 0 ...
OLD: Host: scsi3 Channel: 00 Id: 00 Lun: 00
      Vendor: HPE      Model: DVDROM DUD0N     Rev: UMD1
      Type:   CD-ROM                           ANSI SCSI revision: 05
Scanning host 4 for  SCSI target IDs  0 1 2 3 4 5 6 7, all LUNs
Scanning host 5 for  SCSI target IDs  0 1 2 3 4 5 6 7, all LUNs
Scanning host 6 for  all SCSI target IDs, all LUNs
Scanning host 7 for  SCSI target IDs  0 1 2 3 4 5 6 7, all LUNs
 Scanning for device 7 0 0 0 ...

```

4\. 使用 sanlun 指令檢查新 LUN disk

```
[root@bdb7-b ~]# sanlun lun show all
controller(7mode/E-Series)/                                  device          host                  lun
vserver(cDOT/FlashRay)        lun-pathname                   filename        adapter    protocol   size    product
---------------------------------------------------------------------------------------------------------------
TPEDBISCSISVM01               /vol/BDB7_B_6_vol/BDB7_B_6     /dev/sdaw       host16     iSCSI      500.1g  cDOT
TPEDBISCSISVM01               /vol/BDB7_B_6_vol/BDB7_B_6     /dev/sdav       host15     iSCSI      500.1g  cDOT
TPEDBISCSISVM01               /vol/BDB7_B_6_vol/BDB7_B_6     /dev/sdau       host14     iSCSI      500.1g  cDOT
TPEDBISCSISVM01               /vol/BDB7_B_6_vol/BDB7_B_6     /dev/sdat       host13     iSCSI      500.1g  cDOT
TPEDBISCSISVM01               /vol/BDB7_B_6_vol/BDB7_B_6     /dev/sdas       host12     iSCSI      500.1g  cDOT
TPEDBISCSISVM01               /vol/BDB7_B_6_vol/BDB7_B_6     /dev/sdar       host11     iSCSI      500.1g  cDOT
TPEDBISCSISVM01               /vol/BDB7_B_6_vol/BDB7_B_6     /dev/sdaq       host10     iSCSI      500.1g  cDOT
TPEDBISCSISVM01               /vol/BDB7_B_6_vol/BDB7_B_6     /dev/sdap       host9      iSCSI      500.1g  cDOT
TPEDBISCSISVM01               /vol/BDB7_B_5_vol/BDB7_B_5     /dev/sdao       host15     iSCSI      500.1g  cDOT
TPEDBISCSISVM01               /vol/BDB7_B_5_vol/BDB7_B_5     /dev/sdan       host16     iSCSI      500.1g  cDOT
TPEDBISCSISVM01               /vol/BDB7_B_4_vol/BDB7_B_4     /dev/sdam       host16     iSCSI      500.1g  cDOT
...
```

5\. 使用 multipath 指令檢查新 LUN disk

```
[root@bdb7-b alang]# multipath -ll
mpath2 (3600a098038313631305d4e2d6f774744) dm-6 NETAPP  ,LUN C-Mode
size=500G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| |- 9:0:0:1  sdf  8:80   active ready running
| |- 12:0:0:1 sdt  65:48  active ready running
| |- 16:0:0:1 sdah 66:16  active ready running
| `- 15:0:0:1 sdai 66:32  active ready running
`-+- policy='service-time 0' prio=10 status=enabled
  |- 11:0:0:1 sdg  8:96   active ready running
  |- 10:0:0:1 sde  8:64   active ready running
  |- 14:0:0:1 sdu  65:64  active ready running
  `- 13:0:0:1 sdv  65:80  active ready running
mpath1 (3600a098038313631305d4e2d6f774743) dm-5 NETAPP  ,LUN C-Mode
size=500G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| |- 9:0:0:0  sdc  8:32   active ready running
| |- 12:0:0:0 sdr  65:16  active ready running
| |- 16:0:0:0 sdaf 65:240 active ready running
| `- 15:0:0:0 sdag 66:0   active ready running
`-+- policy='service-time 0' prio=10 status=enabled
  |- 11:0:0:0 sdd  8:48   active ready running
  |- 10:0:0:0 sdb  8:16   active ready running
  |- 13:0:0:0 sdq  65:0   active ready running
  `- 14:0:0:0 sds  65:32  active ready running
3600a098038313631305d4e2d6f77474a dm-26 NETAPP  ,LUN C-Mode
size=500G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| |- 9:0:0:5  sdap 66:144 active ready running
| |- 12:0:0:5 sdas 66:192 active ready running
| |- 15:0:0:5 sdav 66:240 active ready running
| `- 16:0:0:5 sdaw 67:0   active ready running
`-+- policy='service-time 0' prio=10 status=enabled
  |- 10:0:0:5 sdaq 66:160 active ready running
  |- 11:0:0:5 sdar 66:176 active ready running
  |- 13:0:0:5 sdat 66:208 active ready running
  `- 14:0:0:5 sdau 66:224 active ready running
...
```

6\. 設定 LUN WWID 與 mpathX 的對應

編輯 `/etc/multipath.conf`: 將新的 wwid 與 alias 加上。

```
# added by alang
multipaths {
        multipath {
                wwid    3600a098038313631305d4e2d6f774743
                alias   mpath1
        }
        multipath {
                wwid    3600a098038313631305d4e2d6f774744
                alias   mpath2
        }
        multipath {
                wwid    3600a098038313631305d4e2d6f774745
                alias   mpath3
        }
        multipath {
                wwid    3600a098038313631305d4e2d6f774746
                alias   mpath4
        }
        multipath {
                wwid    3600a098038313631305d4e2d6f774747
                alias   mpath5
        }
        multipath {
                wwid    3600a098038313631305d4e2d6f77474a
                alias   mpath6
        }
}
```

7\. 載入 multipath 設定

```shell
[root@bdb7-b alang]# systemctl reload multipathd

[root@bdb7-b alang]# multipath -ll
mpath2 (3600a098038313631305d4e2d6f774744) dm-6 NETAPP  ,LUN C-Mode
size=500G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| |- 12:0:0:1 sdt  65:48  active ready running
| |- 15:0:0:1 sdai 66:32  active ready running
| |- 16:0:0:1 sdah 66:16  active ready running
| `- 9:0:0:1  sdf  8:80   active ready running
`-+- policy='service-time 0' prio=10 status=enabled
  |- 10:0:0:1 sde  8:64   active ready running
  |- 11:0:0:1 sdg  8:96   active ready running
  |- 13:0:0:1 sdv  65:80  active ready running
  `- 14:0:0:1 sdu  65:64  active ready running
mpath1 (3600a098038313631305d4e2d6f774743) dm-5 NETAPP  ,LUN C-Mode
size=500G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| |- 12:0:0:0 sdr  65:16  active ready running
| |- 15:0:0:0 sdag 66:0   active ready running
| |- 16:0:0:0 sdaf 65:240 active ready running
| `- 9:0:0:0  sdc  8:32   active ready running
`-+- policy='service-time 0' prio=10 status=enabled
  |- 10:0:0:0 sdb  8:16   active ready running
  |- 11:0:0:0 sdd  8:48   active ready running
  |- 13:0:0:0 sdq  65:0   active ready running
  `- 14:0:0:0 sds  65:32  active ready running
mpath6 (3600a098038313631305d4e2d6f77474a) dm-26 NETAPP  ,LUN C-Mode
size=500G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| |- 12:0:0:5 sdas 66:192 active ready running
| |- 15:0:0:5 sdav 66:240 active ready running
| |- 16:0:0:5 sdaw 67:0   active ready running
| `- 9:0:0:5  sdap 66:144 active ready running
`-+- policy='service-time 0' prio=10 status=enabled
  |- 10:0:0:5 sdaq 66:160 active ready running
  |- 11:0:0:5 sdar 66:176 active ready running
  |- 13:0:0:5 sdat 66:208 active ready running
  `- 14:0:0:5 sdau 66:224 active ready running

```

完成。新加的 disk mpath6 可以用 LVM 進行管理。

##### 其他指令

顯示 disk path 更多資訊

- 主機磁碟名稱與 Storage LUN 名稱對應
- LUN 編號與 Size
- Storage 的 path 狀態資訊

```
[root@bdb7-a ~]# sanlun lun show -p

                    ONTAP Path: TPEDBISCSISVM01:/vol/BDB7_A_2_vol/BDB7_A_2
                           LUN: 1
                      LUN Size: 500.1g
                       Product: cDOT
                   Host Device: mpath2(3600a0980383136312d3f4e387455564c)
              Multipath Policy: service-time 0
            Multipath Provider: Native
--------- ---------- ------- ------------ ----------------------------------------------
host      vserver
path      path       /dev/   host         vserver
state     type       node    adapter      LIF
--------- ---------- ------- ------------ ----------------------------------------------
up        primary    sdo     host12       TPEDBISCSISVM01_iSCSI2
up        primary    sds     host13       TPEDBISCSISVM01_iSCSI2
up        primary    sdaa    host15       TPEDBISCSISVM01_iSCSI1
up        primary    sdae    host16       TPEDBISCSISVM01_iSCSI1
up        secondary  sdg     host10       TPEDBISCSISVM01_iSCSI4
up        secondary  sdk     host11       TPEDBISCSISVM01_iSCSI3
up        secondary  sdw     host14       TPEDBISCSISVM01_iSCSI4
up        secondary  sdc     host9        TPEDBISCSISVM01_iSCSI3
```

查詢 multipathd 的設定參數資訊

```
multipathd show config
```

> 技巧: 連接不同廠牌的 Storage，預設狀態會套用不同的參數組設定，而這些參數的資訊可以從這指令獲得。

查詢目前 iSCSI connections

```
# iscsiadm -m session --op show
tcp: [1] 10.18.104.32:3260,1038 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16 (non-flash)
tcp: [2] 10.18.104.33:3260,1039 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16 (non-flash)
tcp: [3] 10.18.104.34:3260,1040 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16 (non-flash)
tcp: [4] 10.18.104.34:3260,1040 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16 (non-flash)
tcp: [5] 10.18.104.33:3260,1039 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16 (non-flash)
tcp: [6] 10.18.104.31:3260,1037 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16 (non-flash)
tcp: [7] 10.18.104.31:3260,1037 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16 (non-flash)
tcp: [8] 10.18.104.32:3260,1038 iqn.1992-08.com.netapp:sn.58042162340f11e9892700a098e3ab45:vs.16 (non-flash)

#iSCSI Logout All
iscsiadm --mode node --logoutall=all

#iSCSI Login All
iscsiadm -m node -l
```

# Learning

##### iSCSI

- [How to configure iSCSI target &amp; initiator on RHEL/CentOS 7.6](https://www.linuxteck.com/how-to-configure-iscsi-target-initiator-on-rhel-centos-7-6/)

# Multipath on RedHat (NetApp FAS2240)

Multipath 是 CentOS/RedHat 內建套件，可用作兩 port 的 HBA 卡連接 Storage時，有雙路由的備援架構。

系統環境

- RedHat 4.7
- NetApp FAS2240
- SAN Switch x 2
- HBA Card: QLogic, port x 2

```
#>lspci
Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
```

#### RedHat 的 HBA Card 驅動

建議安裝方法為，在安裝 RedHat 前先安裝 HBA 卡，然後安裝 RedHat 時會自動驅動 HBA 卡。

如果無法重新安裝 RedHat 試試修改 /etc/modprobe.conf，加上這一行

```
alias scsi_hostadapter2 qla2500
```

> scsi\_hostadapterX 編號視實際狀況修改

查出 HBA 卡的 WWN

```
#> cat /proc/scsi/qla2xxx/2

SCSI Device Information:
scsi-qla0-adapter-node=5001438004253c81;
scsi-qla0-adapter-port=5001438004253c80; <---- WWN
scsi-qla0-target-0=50001fe1500e28fd;
scsi-qla0-target-1=50001fe1500e28f9;
scsi-qla0-target-2=500a09829140a66c;
```

> 另一個 HBA port 裝置檔為 /proc/scsi/qla2xxx/3，以此類推
> 
> 每個 port 有獨立的 WWN
> 
> NetApp storage 分配 LUN 以此 WWN 作為 Host Mapping 的主要參數

驅動後的模組列表

```
# lsmod | grep qla
    qla2500               209985  0
    qla2xxx               203681  13 qla2500
    scsi_transport_fc      19521  1 qla2xxx
    scsi_mod              153489  7 ib_srp,qla2xxx,scsi_transport_fc,sr_mod,libata,cciss,sd_mo   d
```

#### SAN Storge 設定與接線

1. 將 NetApp storage、SAN switch、RedHat 接上各自的光纖線
2. 找出 RedHat 的 HBA 卡 兩 port 各自的 WWN
3. 設定 NetApp，將 LUNs 分配至兩個 WWN Hosts

#### 設定 Multipath

> 更新：RedHat 客戶中心有提供設定 multipath.conf 線上協助導引，網頁必須先登入才能瀏覽  
> https://access.redhat.com/labs/multipathhelper/

必要的套件安裝

```
rpm -q device-mapper
rpm -q device-mapper-multipath
yum install device-mapper
yum install device-mapper-multipath
```

##### CentOS 5.x)

編輯 /etc/multipath.conf

```
# Blacklist all devices by default. Remove this to enable multipathing
# on the default devices.
devnode_blacklist {
#        devnode "*" << 註解這一行
}

## Use user friendly names, instead of using WWIDs as names.
defaults {
        user_friendly_names yes
}
```

> 注意：即使啟動了 multipathd 服務，預設是抓不到任何 LUNs 的，必須註解上述的那一行

啟動 multipathd 服務

```
#> service multipathd start
#> chkconfig multipathd on 
```

##### CentOS 6/7)

預設沒有設定檔 /etc/multipath.conf，執行這個指令會自動產生，並且將服務 multipathd 設定成自動執行。

```
#> mpathconf --enable --with_multipathd y
```

> 不用修改 multipath.conf

#### 檢視 multupah 的裝置

CentOS 5.x

```
[root@cdb3-b ~]# multipath -ll
mpath1 (360a98000383035537824474d3936556a) dm-4 NETAPP,LUN
[size=700G][features=3 queue_if_no_path pg_init_retries 50][hwhandler=0][rw]
\_ round-robin 0 [prio=4][active]
 \_ 1:0:1:0 sdb        8:16  [active][ready]
 \_ 2:0:1:0 sdd        8:48  [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 1:0:0:0 sda        8:0   [active][ready]
 \_ 2:0:0:0 sdc        8:32  [active][ready]
```

CentOS 6.x

```
[root@cdb3 ~]# multipath -ll
mpathb (360a98000383035537824474d39365568) dm-2 NETAPP,LUN
size=700G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=4 status=active
| |- 1:0:1:0 sdc 8:32 active ready running
| `- 2:0:1:0 sde 8:64 active ready running
`-+- policy='round-robin 0' prio=1 status=enabled
  |- 1:0:0:0 sdb 8:16 active ready running
  `- 2:0:0:0 sdd 8:48 active ready running
```

#### 安裝 NetApp 管理工具

＊建議＊安裝 LUN 管理工具 - NetApp Linux Host Utilities

```
yum install libnl
rpm -ivh QConvergeConsoleCLI-1.0.01-32.x86_64.rpm
rpm -ivh netapp_linux_host_utilities-6-2.x86_64.rpm
```

> \- 如果沒有安裝這套件，將無法使用 sanlun 指令
> 
> \- 適用 CentOS 5.x/6.x

#### 新增 LUN 需系統重新開機

一旦重新開機後，所有的 LUN disks (在 dual path SAN 架構下會有多個 LUN disks)應該都會被偵測到，並且會有一個 mpathXX 的裝置名稱。

```
#> multipath -l
mpath1 (360a98000424732615124444150534a35)
[size=100 GB][features="1 queue_if_no_path"][hwhandler="0"]
\_ round-robin 0 [active]
 \_ 2:0:2:0 sde 8:64 [active]
 \_ 3:0:2:0 sdf 8:80 [active]

mpath0 (3600508b400068e5c00012000001a0000)
[size=100 GB][features="1 queue_if_no_path"][hwhandler="0"]
\_ round-robin 0 [active]
 \_ 2:0:0:1 sda 8:0  [active]
 \_ 3:0:0:1 sdc 8:32 [active]
\_ round-robin 0 [enabled]
 \_ 2:0:1:1 sdb 8:16 [active]
 \_ 3:0:1:1 sdd 8:48 [active]
```

> mpathX 這路徑可作為 LVM 的 PV 或直接格式化為 filesystem，完整路徑為 /dev/mapper/mpathX
> 
> 2:0:0:1 這表示為 host : channel : ID : LUN
> 
> Host 2 與 Host 3 表示 HBA 卡有兩個 port 或者單 port 的卡兩張

#### 新增/移除 LUN 系統不需重開機

參考教學：

- [https://library.netapp.com/ecmdocs/ECMP1654943/html/GUID-F88A86D2-F3A9-4440-8C32-539BFEA6BED1.html](https://library.netapp.com/ecmdocs/ECMP1654943/html/GUID-F88A86D2-F3A9-4440-8C32-539BFEA6BED1.html)
- [https://access.redhat.com/solutions/3941](https://access.redhat.com/solutions/3941)

##### CentOS/RedHat 5.x 以後

從 CentOS/RedHat 5 以後或者 SLES 10 以後，可以安裝以下套件：

```
yum install sg3_utils
```

不停機下掃描新的 LUN Disk

```
檢視目前現有的 LUN Disks
#> sanlun lun show all

掃描所有 LUN
#> scsi-rescan
NOTE: 如果偵測不到新 LUN，試試加上 -a
#> scsi-rescan -a

再次檢視目前的 LUN Disks
#> sanlun lun show all
```

移除 LUN 的程序

要移除現有的 LUN disk 必須依照以下步驟完成：

1. 執行 `umount` 卸載
2. 執行 `lvremove` 移除 LV
3. 需要時，執行 `vgremove` 移除 VG
4. 執行 `pvremove` 移除 PV
5. 執行 `multipath -f mpathXX` 移除 multipathXX  
    NOTE: 如果沒有執行這步驟，一旦移除 LUN 後系統可能會 Hang。
6. 從 Storage 管理介面移除這個 LUN disk  
    NOTE: 當 LUN 被移除後，執行 `sanlun lun show all`，那些移除的 LUN 會顯示 unknown。
7. 執行 `scsi-rescan -r` 移除所有未使用的 LUN disk  
    NOTE: 執行後輸出訊息，應該要顯示類似  
    4 device(s) removed.  
    \[5:0:1:0\]  
    \[5:0:3:0\]  
    \[6:0:1:0\]  
    \[6:0:3:0\]  
    重新執行 `sanlun lun show all`，那些 unknown 就會消失。

##### CentOS/RedHat 4.x：

- [Vmware Linux Guest Add a New Hard Disk Without Rebooting Guest](https://www.cyberciti.biz/tips/vmware-add-a-new-hard-disk-without-rebooting-guest.html)

使用附檔區的兩個 script

1. remove-LUNs.sh
2. rescan-LUNs.sh

新增 LUN disk 步驟

1. 在 NetApp storage 上完成 LUN Mapping 的設定
2. 在 RedHat 上執行 `rescan-LUNs.sh`
3. 在 RedHat 上執行 `multipath -l`

移除現有 LUN disk

1. 在 RedHat 上卸載不用的 mpathX
2. 在 RedHat 上執行 `sanlun lun show all` 找出 LUN 的名稱與 NetApp 上路徑  
    註：要執行 sanlun 指令必須先安裝 NetApp 的套件 netapp\_linux\_host\_utilities-6-2.x86\_64.rpm
3. 在 NetApp 上移除所對應的 LUN
4. 在 RedHat 上執行 `multipath -l`
5. 找出要移除 LUN disks 的 host:channel:id:LUN
6. 執行 `remove-LUNs.sh <host> <channel> <id> <LUN>`

#### 現有 LUN disk 擴充空間

1\. 確認 LUN ID 與 sd\[X\] 的對應

```shell
smartctl -a /dev/sd[X]
```

2\. 掃描 LUN disk，確認最新容量

```shell
echo 1 > /sys/block/sd[X]/device/rescan<br></br><br></br>lsblk | grep sd[X]
```

3\. Resize PV

```shell
pvresize /dev/sd[X]
```

4\. 完成。接著就按 SOP 進行 LV 與 Filesystem 的擴充。

#### LVM 管理

新增 LUN disk 成一個 PV

```
#> pvcreate /dev/mapper/mpath6
```

> PV 新增後，執行 pvdisplay 時的裝置名稱會是 dm-X。

新增一個 VG (需指定 PV)

```
#> vgcreate dbVG /dev/mapper/mpath6
```

#### 指令更多用法

multipath

<table border="1" id="bkmrk-%2Fetc%2Finit.d%2Fmultipat" style="border-collapse: collapse; width: 100%;"><tbody><tr><td style="width: 50%;">/etc/init.d/multipathd start</td><td style="width: 50%;">Manually starts the multipathing service on the Linux host.</td></tr><tr><td style="width: 50%;">multipath -ll</td><td style="width: 50%;">Displays the multipathing configuration.</td></tr><tr><td style="width: 50%;">multipath -v2</td><td style="width: 50%;">Creates the initial multipathing configuration.</td></tr><tr><td style="width: 50%;">multipath -f *device\_map*</td><td style="width: 50%;">Removes an existing LUN from the multipathing configuration.</td></tr><tr><td style="width: 50%;">multipath -ll -v2 -d</td><td style="width: 50%;">Displays the multipathing configuration in verbose mode.</td></tr><tr><td style="width: 50%;">multipath -t 或 multipathd show config</td><td style="width: 50%;">顯示目前的設定參數</td></tr></tbody></table>

Accesing LUNs on host

<table border="1" id="bkmrk-sanlun-fcp-show-adap" style="border-collapse: collapse; width: 100%;"><tbody><tr><td style="width: 49.9485%;">sanlun fcp show adapter -v</td><td style="width: 49.9485%;">Displays HBA information.</td></tr><tr><td style="width: 49.9485%;">sanlun fcp show adapter</td><td style="width: 49.9485%;">Display WWPN of HBA cards</td></tr><tr><td style="width: 49.9485%;">sanlun lun show all</td><td style="width: 49.9485%;">Lists available LUNs as seen by the sanlun script.</td></tr><tr><td style="width: 49.9485%;">sanlun lun show -p</td><td style="width: 49.9485%;">Provides multipath information for both DM-Multipath and Veritas  
Storage Foundation  
TM  
dynamic multipathing solutions.</td></tr><tr><td style="width: 49.9485%;">ls -lh /dev/mapper/</td><td style="width: 49.9485%;">Lists DM-Multipath devices generated for LUNs.</td></tr><tr><td style="width: 49.9485%;">e2label *device label*</td><td style="width: 49.9485%;">Creates a persistent label for a file system. device is the partition; label  
is a unique string up to 16 characters. Not needed with DM-Multipath.</td></tr></tbody></table>

#### Q &amp; A

##### Q: RHEL 7 沒有使用 alias path

> 使用 NetApp Storage，執行 multipath -ll
> 
>  360a98000443437576f2444726e556d76 dm-2 NETAPP ,LUN  
>  size=800G features='4 queue\_if\_no\_path pg\_init\_retries 50 retain\_attached\_hw\_handle' hwhandler='1 alua' wp=rw  
>  |-+- policy='service-time 0' prio=50 status=active  
>  | `- 6:0:0:0 sdb 8:16 active ready running  
>  `-+- policy='service-time 0' prio=10 status=enabled  
>  `- 6:0:3:0 sdc 8:32 active ready running
> 
>  不會顯示 alias name: mpathXX
> 
> 參閱官方手冊：https://access.redhat.com/solutions/2061463

檢視 multipath 可支援的裝置清單

```
multipath -t
```

> 沒有看到 NetApp 的裝置。

解決方法：編輯 multipath.conf，加入 NetApp 的參數設定

> NOTE: 僅適合 NetApp Storage。

```
defaults {<br></br>        user_friendly_names yes<br></br>        find_multipaths yes<br></br>}<br></br><br></br>devices {<br></br>        device {<br></br>                vendor "NETAPP"<br></br>                product "LUN.*"<br></br>                path_grouping_policy "group_by_prio"<br></br>                path_checker "tur"<br></br>                features "3 queue_if_no_path pg_init_retries 50"<br></br>                hardware_handler "0"<br></br>                prio "ontap"<br></br>                failback immediate<br></br>                rr_weight "uniform"<br></br>                rr_min_io 128<br></br>                flush_on_last_del "yes"<br></br>                dev_loss_tmo "infinity"<br></br>                user_friendly_names yes<br></br>                retain_attached_hw_handler yes<br></br>                detect_prio yes<br></br>        }
```

載入新設定

```
#> systemctl reload multipathd<br></br><br></br>#> multipath -ll<br></br>mpatha (360a98000443437576f2444726e556d76) dm-2 NETAPP  ,LUN<br></br>size=800G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw<br></br>|-+- policy='service-time 0' prio=50 status=active<br></br>| |- 4:0:0:0 sdb 8:16 active ready running<br></br>| `- 6:0:0:0 sdd 8:48 active ready running<br></br>`-+- policy='service-time 0' prio=10 status=enabled<br></br>  |- 4:0:3:0 sdc 8:32 active ready running<br></br>  `- 6:0:3:0 sde 8:64 active ready running
```

##### Q: 安裝 netapp\_linux\_host\_utilities 時，出現錯誤訊息

> Warning: libnl.so library not found, some sanlun commands may not work. Refer Li nux Host Utilities Installation and Set up Guide for more details  
>  Warning: libHBAAPI.so library not found, some sanlun commands may not work. Refe r Linux Host Utilities Installation and Setup Guide for more details

Solution: 先安裝 libnl

```
yum install libnl
```

再安裝以下套件任一個

- [http://driverdownloads.qlogic.com/QLogicDriverDownloads\_UI/ResourceByOS.aspx?productid=1043&amp;oemid=372&amp;oemcatid=111969](http://driverdownloads.qlogic.com/QLogicDriverDownloads_UI/ResourceByOS.aspx?productid=1043&oemid=372&oemcatid=111969)
- [http://driverdownloads.qlogic.com/QLogicDriverDownloads\_UI/SearchByOs.aspx?ProductCategory=39&amp;OsCategory=2&amp;Os=65&amp;OsCategoryName=Linux&amp;ProductCategoryName=Fibre+Channel+HBAs&amp;OSName=Linux+Red+Hat+%2864-bit%29](http://driverdownloads.qlogic.com/QLogicDriverDownloads_UI/SearchByOs.aspx?ProductCategory=39&OsCategory=2&Os=65&OsCategoryName=Linux&ProductCategoryName=Fibre+Channel+HBAs&OSName=Linux+Red+Hat+%2864-bit%29)
- QConvergeConsole CLI (x64)
- SANsurfer CLI (x86/x64)

##### Q: 執行 pvscan、vgscan 或開機時出現以下訊息

> Found duplicate PV XXXXXXXXXXXXXXXX: using /dev/sdb not /dev/sda

Solution:

這通常會發生在系統有作 multipath 與 LVM 設定後，原因是 CentOS 4/5 預設 LVM 的設定會掃描所有的磁碟裝置，然而，當系統設定有 multipath 時，LVM 應該管理 mpathX 或 dm-X 的裝置，而不再是 sdX 裝置。

要使 LVM 不再對 sdX 裝置作管理，可以修改 /etc/lvm/lvm.conf，此外，如果要使每次系統開機時，LVM 不要對 sdX 裝置作掃描，還必須要重建 initrd 開機映像檔，並且修改 grub.conf 相關的設定。

操作步驟如下:

1\. 編輯 /etc/lvm/lvm.conf

```
...<br></br>filter = [ "r|/dev/sd.*|", "a|/.*|" ]<br></br>... 
```

> r|/dev/sd.\*| 這設定會排除所有 sda, sdb, ...等所有磁碟裝置掃描，如果系統有使用其中某個裝置，必須自行修改這規則。

存檔離開

執行 pvscan 或 vgscan，如果不再出現 Warning，繼續步驟 2。

2\. 重建 initrd 開機映像檔

```
#> cd /boot<br></br>#> mkinitrd -f -v /boot/initrd-$(uname -r).multipath.img $(uname -r) 
```

編輯 /boot/grub/grub.conf  
複製 title 段落的所有行，並依照實際狀況修改

```
title CentOS (2.6.18-398.el5-Multipath)                      <== 加上 Multipath<br></br>          root (hd0,0)<br></br>          kernel /vmlinuz-2.6.18-398..........<br></br>          initrd /initrd-2.6.18-398.el5.multipath.img     <== 重建後的initrd 開機映像檔名
```

存檔離開

3\. 重新開機後，出現開機選單後，選擇第二個開機項目。

如果開機訊息不再出現警告文字，而且系統一切正常，就可以再修改 grub.conf 的 default=1。

# FAQ

##### 如何確認 /dev/sda 是 Local Disk 還是 SAN DIsk

Solution:

```
#> lsscsi
[2:0:0:0]    cd/dvd  hp       DVD-ROM DTA0N    WBD0  /dev/sr0
  state=running queue_depth=1 scsi_level=6 type=5 device_blocked=0 timeout=0
[3:0:0:0]    tape    IBM      ULT3580-HH5      D2AD  /dev/st0
  state=running queue_depth=32 scsi_level=7 type=1 device_blocked=0 timeout=900
[3:0:1:0]    tape    IBM      ULT3580-HH5      D2AD  /dev/st1
  state=running queue_depth=32 scsi_level=7 type=1 device_blocked=0 timeout=900
[3:0:1:1]    mediumx IBM      3573-TL          C.20  -
  state=running queue_depth=32 scsi_level=6 type=8 device_blocked=0 timeout=0
[3:0:2:0]    disk    IBM      1726-4xx  FAStT  0617  /dev/sda            <======
  state=running queue_depth=32 scsi_level=6 type=0 device_blocked=0 timeout=60
[3:0:2:31]   disk    IBM      Universal Xport  0617  -
  state=running queue_depth=32 scsi_level=6 type=0 device_blocked=0 timeout=60
```

> IBM FAStT 是 IBM Storage 產品識別
> 
> 如果沒有 lsscsi 指令，執行 yum install lsscsi

##### 如何找出 HBA Card 的 port WWN

方法一: 用 systool

```bash
# Install
yum install sysfsutils
#
systool -c fc_host -v
systool -c fc_transport -v 
```

方法二：`/sys/class/fc_host/{host#}/port_name`

```
#> cat /sys/class/fc_host/host5/port_name
0x2100001b32837bcd
```

方法三：`/proc/scsi/{adapter-type}/{host#}`

```
#> cat /proc/scsi/qla2xxx/2

SCSI Device Information:
scsi-qla0-adapter-node=5001438004253c81;
scsi-qla0-adapter-port=5001438004253c80; <---- WWN
scsi-qla0-target-0=50001fe1500e28fd;
scsi-qla0-target-1=50001fe1500e28f9;
scsi-qla0-target-2=500a09829140a66c;
```

##### mpathX v.s. dm-X

如果有啟用 multipath，所有 LUN 的裝置路徑為 /dev/mapper/mpathX，不過當使用 pvcreate 將 LUN 新增為 PV 後，執行 pvdisplay -C ，此時的裝置名稱會改為 /dev/dm-X，那爾後作系統維護時，要如何查詢 mpathX 對應哪個 dm-X?

方法一：`dmsetup`

```
[root@bdb3 ~]# dmsetup ls
mpath1  (253, 4)
rootVG-varLV    (253, 1)
rootVG-swapLV   (253, 3)
rootVG-rootLV   (253, 0)
rootVG-worktmpLV        (253, 2)
```

> mpath1 (253,4)，4 就是指裝置 /dev/dm-4

方法二：`multipath`

```
[root@cdb3-b ~]# multipath -ll
mpath1 (360a98000383035537824474d3936556a) dm-4 NETAPP,LUN                     <====== 這裡
[size=700G][features=3 queue_if_no_path pg_init_retries 50][hwhandler=0][rw]
\_ round-robin 0 [prio=4][active]
 \_ 1:0:1:0 sdb        8:16  [active][ready]
 \_ 2:0:1:0 sdd        8:48  [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 1:0:0:0 sda        8:0   [active][ready]
 \_ 2:0:0:0 sdc        8:32  [active][ready]
```

##### 新 partition 無法使用

對 mpathb 新建 partition 後，無法立即被使用，必須重開機後才能生效

> WARNING: Re-reading the partition table failed with error 22: Invalid argument.  
> The kernel still uses the old table. The new table will be used at  
> the next reboot or after you run partprobe(8) or kpartx(8)

不重開機要能立即使用，可以再切完 partition 後，執行

```bash
partprobe /dev/mapper/mpathb
```

# iptables

#### 基本指令

列出現有規則

```
iptables -L -n -v
iptables -L -n -v --line-numbers
iptables -L INPUT -n -v --line-numbers
```

重啟/清除所有規則

```
service iptables stop
service iptables start

iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT 
```

- -F : Deleting (flushing) all the rules.
- -X : Delete chain.
- -t table\_name : Select table (called nat or mangle) and delete/flush rules.
- -P : Set the default policy (such as DROP, REJECT, or ACCEPT).

移除指定的規則

```bash
# 顯示所有規則的編號
iptables -L INPUT -n --line-numbers
iptables -L OUTPUT -n --line-numbers
iptables -L OUTPUT -n --line-numbers | less
iptables -L OUTPUT -n --line-numbers | grep 202.54.1.1

# 移除指定編號的規則
iptables -D INPUT 4

# 移除指定來源 IP 的規則
iptables -D INPUT -s 202.54.1.1 -j DROP

# 移除指定 Chain 包含的所有規則
iptables -F <chain-name>

# 移除指定 Chain
iptables -X <chain-name>
```

新增規則

```
# 顯示所有規則的編號
iptables -L INPUT -n --line-numbers

# 在編號 1-2 之間新增規則
iptables -I INPUT 2 -s 202.54.1.2 -j DROP
```

儲存所有規則

```
# For RedHat/CentOS/Fedora Linux
service iptables save

# For other Linux distro.
iptables-save > /root/my.active.firewall.rules
cat /root/my.active.firewall.rules
```

回復/還原規則

```
# For other Linux distro.
iptables-restore < /root/my.active.firewall.rules

# For RedHat/CentOS/Fedora Linux
service iptables restart
```

#### Learning

- \[Github\] [API for local iptables management](https://github.com/palner/iptables-api)

# RClone

[Rclone](https://rclone.org/) syncs your files to cloud storage

##### Installation

Ubuntu

```shell
sudo apt install rclone
```

##### Storj DCS

Manual: [https://documentation.tardigrade.io/how-tos/sync-files-with-rclone](https://documentation.tardigrade.io/how-tos/sync-files-with-rclone)

Config

```
rclone config

n) New remote
s) Set configuration password
q) Quit config
n/s/q> n

name> storjdcs


4 / Amazon S3 Compliant Storage Provider (AWS, Alibaba, Ceph, Digital Ocean, Dreamho
st, IBM COS, Minio, etc)
Storage> 4 

10 / Any other S3 compatible provider
provider> 10

1 / Enter AWS credentials in the next step
env_auth> 1

access_key_id> Enter your <AccessKeyId>

secret_access_key> Enter your <SecretAccessKey>

1 / Use this if unsure. Will use v4 signatures and an empty region.
region> 1


endpoint> Enter the Storj DCS Gateway URL

location_constraint> Press Enter

acl> Press Enter

Edit advanced config? (y/n)
y/n> y

bucket_acl> Press Enter

upload_cutoff> 200M

chunk_size> 64M

disable_checksum> Press Enter

session_token> Press Enter

upload_concurrency> Press Enter

force_path_style> Press Enter

v2_auth> Press Enter


[storjdcs]
provider = Other
env_auth = false
access_key_id = xxxxx
secret_access_key = xxxxxxxxxxxxxxxxxxxx
endpoint = https://gateway.ap1.storjshare.io
upload_cutoff = 200M
chunk_size = 64M
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y


Remote config
Current remotes:

Name                 Type
====                 ====
storjdcs             s3

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q

```

Pikpak

1. New remote: n
2. Enter name: pikpak
3. Type of storage: WebDav
4. Vendor: Other
5. User: &lt;pikpak-webdav-user&gt;
6. Password: &lt;pikpak-webdav-password&gt;
7. bearer\_token: &lt;leave empty&gt;
8. Advanced config: No

```bash
rclone config

rclone ls "pikpak:My Pak"
rclone mount "pikpak:My Pak" /mnt/pikpak
rclone mount "pikpak:My Pak" /mnt/pikpak --daemon
```

##### Create a directory in Bucket

```shell
rclone mkdir storjdcs:raida-nodes/R17
```

##### List All Buckets

```shell
rclone lsf storjdcs:
rclone ls storjdcs:raida-nodes/R17/
```

##### Delete a Bucket

```shell
rclone rmdir storjdcs:mybucket

# Use the purge command to delete a non-empty bucket with all its content.
rclone purge storjdcs:mybucket
```

##### Upload files

```shell
# Upload the files in the directory 
rclone copy --progress /root/ADIAR/Nginx_MySQL/db/backup/ storjdcs:raida_nodes/R17/

Transferred:      518.759M / 518.759 MBytes, 100%, 511.778 kBytes/s, ETA 0s
Errors:                 0
Checks:                 0 / 0, -
Transferred:            5 / 5, 100%
Elapsed time:    17m17.9s
```

Housekeeping the files

```shell
# Delete the files older than 7 days
# Refer to https://rclone.org/flags/
rclone delete remote:dir --min-age 8d -v --dry-run

rclone delete storjdcs:raida-nodes/R17/ --exclude "{dump.sql,importTable.sql}" --min-age 8d -v --dry-run
rclone delete storjdcs:raida-nodes/R17/ --exclude "{*.txt,dump.sql,importTable.sql}" --min-age 8d -v --dry-run
```

#### FAQ

##### failed to mount FUSE fs

> Fatal error: failed to mount FUSE fs: fusermount: exec: "fusermount": executable file not found in $PATH

Solution:

```bash
sudo apt install fuse
```

# vmstat

##### Case #1:CPU &amp; RAM 負載低

```
[root@deki ~]# vmstat 2
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0     60 163016 252108 363996    0    0     4   172   23   16  2  3 94  0  0
 0  0     60 163016 252108 363996    0    0     0    60 1025  248  0  0 100  0  0
 0  0     60 163016 252108 364000    0    0     0     0 1073  416  3  4 94  0  0
 0  0     60 163016 252108 364000    0    0     0     0 1042  248  0  0 100  0  0
 0  0     60 163016 252108 364000    0    0     0    54 1030  249  0  0 99  0  0
 0  0     60 163016 252108 364000    0    0     0     0 1032  247  0  1 100  0  0
 0  0     60 163016 252108 364000    0    0     0     0 1035  228  0  0 100  0  0
 0  0     60 162892 252108 364000    0    0     0    12 1041  516 12 68 21  0  0
 0  0     60 162892 252108 364000    0    0     0     0 1020  240  0  0 100  0  0
 0  0     60 162892 252108 364000    0    0     0    42 1024  249  0  0 100  0  0
 0  0     60 162892 252108 364000    0    0     0     0 1035  248  0  0 100  0  0
 0  0     60 162892 252108 364000    0    0     0     2 1029  267  0  1 99  0  0
 0  0     60 163016 252108 364000    0    0     0     0 1018  219  0  0 100  0  0
 0  0     60 163016 252108 364000    0    0     0    12 1026  603  7  8 85  0  0
 1  0     60 163016 252108 364004    0    0     0     0 1044  270  0  0 100  0  0
 0  0     60 163016 252108 364004    0    0     0     0 1025  267  0  0 100  0  0
 7  0     60 162272 252108 364004    0    0     0    54 1026 1127  8 14 78  0  0
```

##### Case #2: CPU 負載高，RAM 負載低

```
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
13  0      0 204372  46688 3261856    0    0     0  3146 2024 5151 18 82  0  0  0
15  0      0 204248  46708 3261900    0    0     4  3920 1919 5839 17 83  0  0  0
11  1      0 204248  46716 3261956    0    0     6  2940 1868 4863 19 81  0  0  0
16  1      0 209212  46736 3262004    0    0     4  5142 2194 5447 17 83  0  0  0
 8  0      0 210332  46776 3262248    0    0   122  3002 2267 5818 15 85  0  0  0
 6  0      0 210704  46808 3262300    0    0     0  3090 1982 4658 17 83  0  0  0
11  1      0 211076  46872 3262384    0    0     2  2986 2102 4890 17 83  0  0  0
 9  0      0 210952  46884 3262408    0    0     2     0 1815 4891 18 82  0  0  0
 6  0      0 210952  46896 3262440    0    0     0  2978 1588 4877 23 77  0  0  0
 3  0      0 210828  46920 3262468    0    0     0  2968 1859 4424 18 82  0  0  0
 4  0      0 211324  46932 3262580    0    0     0     0 2182 5391 16 84  0  0  0
 4  0      0 211324  46976 3262000    0    0     0  2992 1638 3973 17 83  0  0  0
 3  0      0 211324  47008 3262060    0    0     0  5190 1838 4707 19 81  0  0  0
 9  0      0 211324  47008 3262072    0    0     0     0 1688 5711 39 61  0  0  0
 7  0      0 208968  47020 3262072    0    0     0  2974 1389 4076 24 76  0  0  0
 2  0      0 206364  47020 3262080    0    0     0     0 1053 3283 40 60  0  0  0
11  0      0 204628  47028 3262080    0    0     0  3020 1995 6525 32 68  0  0  0
 4  0      0 199668  47044 3262084    0    0     0  2946 1813 4977 21 79  0  0  0
 4  0      0 193344  47064 3262120    0    0     0  2962 2455 6091 15 85  0  0  0
 5  0      0 191608  47092 3262316    0    0     0  2980 2255 5560 18 82  0  0  0
10  0      0 188632  47132 3262248    0    0     0  2960 2591 6128 15 85  0  0  0
 5  0      0 186524  47168 3262448    0    0     0  3026 2210 5305 19 81  0  0  0
 3  0      0 186276  47192 3262328    0    0     0     4 1919 4915 24 76  0  0  0
 4  0      0 186028  47216 3262372    0    0     0  2994 1878 4399 19 81  0  0  0
 5  0      0 185780  47244 3262412    0    0     0  2970 2001 4915 18 82  0  0  0
 5  0      0 185532  47276 3262400    0    0     0  3128 2159 5427 16 84  0  0  0
 3  0      0 185532  47284 3262372    0    0     0     0 2173 5393 16 84  0  0  0
 3  0      0 185408  47320 3262396    0    0     0  2966 2247 5286 15 85  0  0  0
 4  0      0 185284  47340 3262464    0    0     0  2958 2237 5438 15 85  0  0  0
```

##### Case #3:連接 storage 與 SAN switch 的光纖線發生異常，時好時壞。

異常狀況：DB 效能突然變差

狀況分析：

- Linux 的 top 指令顯示 Load Average 的值很容易高於 15 以上甚至更高。
- 檢視 dmesg 與 messages 無任何硬體異常訊息記錄。
- 幾乎所有外部應用程式開始連線後，整個系統就出現超載狀況。
- 分析 Memory， 無 Disk Swapping 現象。
- 分析 CPU，仍有許多 Idle 現象。

檢視 vmstat 輸出資訊如下:

- CPU - wa 始終大於 0 甚至更高
- IO - bi, bo 常有兩者都是 0，bi 偶爾會有高數值出現

> NOTE: 當系統有 I/O 行為時，正常狀態 bi 或 bo 應該都是連續有數值，不應該會是 0
> 
> 原因分析：透過 vmstat 資訊，當系統有 I/O 行為時，正常狀態的 bi 或 bo 應該是連續有數值，不應該出現 0。

Tips:

- r - 執行中或等待執行的程序數量
- b - 未中斷的休眠中程序數量，例如等待 IO，或正在作 Swaping
- swpd - 目前已使用的虛擬記憶(KB)
- free - 目前尚未被使用的實體記憶體(KB)
- buff - 目前在暫存區用於讀/寫操作的實體記憶體(KB)
- cache - 目前對應程序位址空間的實體記憶體(KB)
- si - 從 swap disk 讀取到 RAM(KB)
- so - 從 RAM 寫入 swap disk(KB)
- bi - 從檔案系統或 swap disk 讀取的分頁區塊
- bo - 從 RAM 寫到 swap disk 的分頁區塊

CPU:

- us - User 佔用 CPU %
- sy - System 佔用 CPU %
- id - Idle 閒置 CPU %
- wa - 因為 I/O Waiting CPU %
- st - 虛擬機佔用 CPU % (kernel 2.6.1 以後才支援)

結果分析

1. r 值若經常大於 1 甚至更大的數字，且 id 常小於 40%，表示 CPU 負荷過重。
2. 當主程序正在運行時，有較少的 free 是好事，這表示 cache 使用更有效率，除非是有不斷的寫入 swap disk (so,bo)。
3. cache 值如果較大，且 bi 值小，表示檔案系統效率好。
4. 如果 swapd 數值大於 0，但 si, so 都顯示 0，此時系統效能還是正常的。
5. 如果 bi, bo 出現較大的數值，但 si, so 卻維持數值 0，表示系統 IO 負載過重，要檢查檔案系統或硬碟有無異常。
6. 如果 bi, bo 與 si, so 同時呈現較大的數值，表示記憶體 swapping 頻繁，RAM 太小。
7. kswapd 程序服務是負責確保閒置的記憶體可被釋放，每次啟動掃描會嘗試釋放 32 個 pages，並且一直在重複這個程序，直到閒置記憶體的數值高於 pages\_high(核心參數)。
8. pdflush 程序服務是將記憶體中的內容與檔案系統進行同步操作，確保記憶體資料寫到硬碟。
9. bi 或 bo 出現 0 的次數太過頻繁，除非系統處於閒置狀態，否則應該檢查 I/O 方面的硬體裝置，例如硬碟、storage、HBA 卡甚至是光纖線。

##### Case #4:NetApp storage 發生 IO bottleneck

異常狀況: 某資料庫在特定時段，AP 發生 SQL 嚴重遲緩。

狀況分析:

- 已確認主機的 CPU、Memeory 資源足夠
- 已確認資料庫服務無異常日誌記錄與 AP 程式無異動
- 問題發生當下，以 nmon 檢查主機 IO，發現 Read 使用率持續 100%，但 IOPS 平均數值顯示非常低 (不到 100 KB)
- 以 vmstat 檢查 bi 與 bo，在有 IO 行為發生時，正常情況下，數值最高可達 數萬到10幾萬，但異常發生時，數值平均都維持在 100 - 200 或更低數值。

# IBM OpenPower Linux

# 安裝 RedHat Linux for Power

機型：IBM Power System S822LC (8001-22C) for big data  
作業系統：RedHat 7.3 for POWER8 Little Endian (NOTE: 此機型只能適用 Little Endian 安裝版本，其他機型適用版本可見附檔區的參考文件:Quick Start Guide for installing Linux on IBM Power System LCservers)

##### 使用心得：

1. 不知為何 Linux 辨識到磁碟代號順序與實機上的位置完全無法做對應。（請見下文的位置對照表）
2. 沒有指令可以識別指定磁碟的實際位置。  
    NOTE: 即使可用 storcli 指令辨識磁碟位置，但它的順序與 Linux 的磁碟順序不一致
3. 雖然內建有 Megaraid 介面卡，卻無法成功建立硬體式的 Raid。  
    NOTE: 使用 storcli 指令新增 raid 失敗，錯誤訊息: Resource already in use

#### 安裝 RedHat Linux

> NOTE: 官方資料顯示支援 RedHat 7.2，但實際上無法安裝此版本，狀況是無法執行光碟開機。

##### 方法：以光碟安裝

連接外接式 USB 光碟機，並置入 RedHat 安裝光碟

主機開啟電源 &gt; Petitboot

畫面的選單上方應該會出現 RedHat 安裝光碟的選項

*\* Install Red Hat Enterprise Linux 7.3 (64-bit kernel)*

移動方向建至該選項，按 Enter 即可進入安裝程序

> TIP: 如過程失敗，參考附檔區文件: Quick Start Guide for installing Linux on IBM Power System LCservers

#### 軟體 Raid 設定

這款機型沒有內建 Raid Adapter，只能使用 RedHat 的 Software Raird。不過在使用與設定上與 x86 機器有許多不同，其中有幾項重點需要注意：

- RedHat for Power 系統開機流程的順序是: BIOS &gt; sda1: PPC PReP Boot &gt; sda2: /boot (GRUB2) &gt; sda3: /
- PPC PReP Boot 這磁區不支援安裝在 Software Raid (注意: 這個爾後可能會造成系統無法正常開機，詳細資訊參閱: 開機磁區複寫至多個磁碟
- /boot 與 / 可安裝在 Software Raid
- 磁區新增如下: 合計 5 顆磁碟 sda, sdb, sdc, sdd, sde 
    1. sda1 = PPC PReP Boot (Power 開機)
    2. sda2 = raid1 /boot (GRUB2 開機)
    3. sda3 = raid6 /
    4. sdb1 = PPC PReP Boot (Power 開機)
    5. sdb2 = raid1 /boot (GRUB2 開機)
    6. sdb3 = raid6 /
    7. sdc1 = raid6 /
    8. sdd1 = raid6 /
    9. sde1 = raid6 /  
        更多詳細資訊可見附檔區
- raid6 的 / 都是 LVM 格式，預設的 VG 名稱是 rhel，LV 名稱是 rhel-&lt;磁區名稱&gt;，例如 rhel-swap
- 設定 raid level 6 時，size policy 建議使用 As large as possible。

> TIP:
> 
> size policy:  
> 預設 Automatic - 這 raid 的 size 是以實際已使用資料量大小為基準，優點是 raid 初始化時比較快速，因此安裝時間也會明顯變短；缺點是未來管理磁碟用量時不是這麼直覺，透過指令檢查 PV 或 VG 的剩餘空間永遠都是 0 - 1 %。  
> As large as possible - 這 raid 的 size 是以包含的所有實體硬碟可用空間做計算，優點是較容易管理整體磁碟空間，透過一般指令就可以控管磁碟用量；缺點是 raid 一開始做初始化，以及爾後需要做 Raid Rebuild 所花費的時間比較久。
> 
> 新增磁區時：  
> 技巧一 - 先移除全部舊磁區，再依序新增 prepboot(sda), prepboot(sdb), /boot, SWAP, /。  
> 技巧二 - 如果多顆磁碟裡存在舊的 PPC PReP Boot 磁區，透過安裝畫面的工具(Anaconda)是無法有效移除，建議方法是先以單顆磁碟做簡單系統安裝，完成後進入系統再以 fdisk 指令一一將不用的開機磁區移除。  
> 技巧三 - 第二顆磁碟上的 prepboot 磁區可能不會正常顯示在畫面上，要確認所有磁區是否新增正確，可以檢視下一頁的 Summary of Changes。  
> 技巧三 - 如果出現訊息: boot loader stage2 device boot is on a multi-disk array but boot loader stage1 sevice sda1 is not ....，可以先忽略它，按下 Done 繼續完成安裝的步驟。  
> 一旦系統完成安裝後，再繼續文章 開機磁區複寫至多個磁碟 的步驟。

#### 安裝後的步驟

關閉 SELinux

```shell
sed -i 's/SELINUX=.*$/SELINUX=disabled/' /etc/selinux/config
setenforce 0
```

關閉防火牆

```shell
systemctl stop firewalld.service
systemctl disable firewalld.service
```

##### 硬體監控管理

設定 IPMI (BMC) 網路

網路線連接主機後方的 IPMI Network Port  
主機開啟電源 &gt; Petitboot &gt; Exit to Shell

```
ipmitool lan print 1
ipmitool lan set 1 ipsrc static
ipmitool lan set 1 ipaddr <ipmi-ip-address>
ipmitool lan set 1 netmask <ipmi-ip-netmask>
ipmitool lan set 1 defgw ipaddr <ipmi-ip-gateway>
ipmitool lan print 1 
```

BMC 網頁存取：

> http://ipmi-ip-address/ , 預設登入: ADMIN / ADMIN  
> NOTE: 官方手冊提到的預設帳號與密碼是錯誤的。
> 
> Java 版本是 1.6.0.45 32-bit, Windows 7 64-bit，沒試過其他 Java 版本。

IPMITools 存取：安裝 ipmitool 在 Desktop PC

Ubuntu

```
sudo apt-get update
sudo apt-get install ipmitool
```

以 source 編譯安裝

```shell
wget https://sourceforge.net/projects/ipmitool/files/latest/download
bzip2 -d ipmitool-1.8.15.tar.bz2
tar xvf ipmitool-1.8.15.tar
cd ipmitool-1.8.15
./configure 
```

測試連線

```
顯示所有裝置的 FRU 資訊
ipmitool -I lanplus -H ipmi-ip-address -U ADMIN -P ADMIN fru
```

##### 磁碟名稱與實體位置對應

<table border="1" id="bkmrk-sdb-empty-nvme-empty" style="border-collapse: collapse; width: 100%;"><tbody><tr><td style="width: 25%;">sdb</td><td style="width: 25%;">EMPTY</td><td style="width: 25%;">NVMe</td><td style="width: 25%;">EMPTY</td></tr><tr><td style="width: 25%;">sde</td><td style="width: 25%;">sdc</td><td style="width: 25%;">EMPTY</td><td style="width: 25%;">EMPTY</td></tr><tr><td style="width: 25%;">sda</td><td style="width: 25%;">sdd</td><td style="width: 25%;">EMPTY</td><td style="width: 25%;">NVMe</td></tr></tbody></table>

# 建立本地端套件庫主機

( [https://www.ibm.com/support/knowledgecenter/en/SST55W\_4.3.0/liaca/liaca\_config\_os\_yum\_repos.html](https://www.ibm.com/support/knowledgecenter/en/SST55W_4.3.0/liaca/liaca_config_os_yum_repos.html) )

準備一部 FTP/NFS/WWW 主機，將安裝光碟內的所有檔案與目錄上傳到主機，並分享使其他電腦可以存取。

```shell
mount -o loop rhel-server-7.3-ppc64le-dvd.iso /mnt/DVD
cp -r /mnt/DVD/* /mnt/tpeitpfs02_nas/redhat_7.3_ppc64_le/ 
```

本地端套件庫主機建置完成了。

其他 Linux 主機要存取這個套件庫，新增 /etc/yum.repos.d/rhel-media.repo

```
[rhel-media]
name=name=Red Hat Enterprise Linux $releasever - Media
baseurl=ftp://10.10.1.115/LINUX_REPO/redhat_7.3_ppc64_le/
gpgcheck=1
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta,file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
```

測試連線

```
yum repolist
```

##### 選用: 新增額外的套件至套件庫

如有額外套件要放入套件庫，步驟是：

```shell
mkdir /mnt/tpeitpfs02_nas/redhat_7.3_ppc64_le/optional/Packages
cp your-packages.rpm /mnt/tpeitpfs02_nas/redhat_7.3_ppc64_le/optional/Packages
createrepo /mnt/tpeitpfs02_nas/redhat_7.3_ppc64_le/optional
```

# 開機磁區複寫至多個磁碟

原因：OpenPOWER Linux 開機過程所需的第一個開機磁區 PPC PReP Boot 無法安裝以 Software Raid 型式安裝，雖然在初次安裝系統時不會有任何影響，不過爾後第一顆磁碟因為發生故障，需要更換另一顆完全新的磁碟時，這樣雖然不會立即對系統產生影響，只要系統有做重啟，系統就會無法正常開機，原因是更換後的磁碟並沒有開機磁區。

解決方法就是：使第二顆磁碟也能做開機，步驟如本篇的教學。

> TIP: PPC PReP Boot 這個專屬 IBM Power 機器的開機磁區其作用就像 x86 伺服器的 MBR。

官方教學連結：https://access.redhat.com/discussions/2158911

#### 初次安裝時

確認目前系統可正常運作且第一顆磁碟的磁區狀態如下：

```
# fdisk -l /dev/sda

Disk /dev/sda: 960.2 GB, 960197124096 bytes, 1875385008 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: dos
Disk identifier: 0x000e7231

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048       22527       10240   41  PPC PReP Boot
/dev/sda2           22528     2121727     1049600   fd  Linux raid autodetect
/dev/sda3         2121728  1875384319   936631296   fd  Linux raid autodetect
```

> sda1 Power 開機磁區 (這個需要複寫到第二顆磁碟)
> 
> sda2 Linux 開機磁區 /boot (已做 Raid 1 )
> 
> sda3 LVM 磁區 (已做 Raid6)

複寫 Power 開機磁區至第二顆磁碟 /sdb1

```
dd if=/dev/sda1 of=/dev/sdb1 bs=512 conv=noerror,sync
```

/dev/sdb 磁區清單

```
# fdisk -l /dev/sdb

Disk /dev/sdb: 960.2 GB, 960197124096 bytes, 1875385008 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: dos
Disk identifier: 0x0003819f

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048       22527       10240   41  PPC PReP Boot
/dev/sdb2           22528     2121727     1049600   fd  Linux raid autodetect
/dev/sdb3         2121728  1875384319   936631296   fd  Linux raid autodetect
```

> sdb 與 sda 的磁區幾乎一樣，主要原因是這兩顆磁碟都被規劃成可開機磁碟。

#### 開機磁碟有更換時

不管是更換 sda 或 sdb 磁碟，更換後都需要手動建立開機磁區以及其他 raid 磁區。

以 /dev/sda 為例

```
fdisk /dev/sda

新增磁區
n > p > 1(default) > 2048(default) > 22527
n > p > 2 (default) > 22528 (default) > 2121727
n > p > 3 (default) > 2121728 (default) > 1875385007 (default)  

設定磁區格式
t > 1 > 41       PPC PReP Boot  
t > 2 > fd        Linux raid 
t > 3 > fd        Linux raid 

設定磁區#1 為開機磁區
a > 1

檢查磁區清單
p

寫入後離開
w 
```

複寫開機磁區從 sdb 到 sda

```
dd if=/dev/sdb1 of=/dev/sda1 bs=512 conv=noerror,sync
```

檢視 Raid

```
# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md126 : active raid1 sdb2[1]
      1049536 blocks super 1.0 [2/1] [_U]
      bitmap: 1/1 pages [64KB], 65536KB chunk

md127 : active raid6 sdb3[1] sdd1[3] sde1[4] sdc[5]
      2809500672 blocks super 1.2 level 6, 512k chunk, algorithm 2 [5/4] [_UUUU]
      bitmap: 1/1 pages [64KB], 65536KB chunk

unused devices: <none>

# mdadm --detail /dev/md126
/dev/md126:
        Version : 1.0
  Creation Time : Mon Feb 20 23:30:01 2017
     Raid Level : raid1
     Array Size : 1049536 (1024.94 MiB 1074.72 MB)
  Used Dev Size : 1049536 (1024.94 MiB 1074.72 MB)
   Raid Devices : 2
  Total Devices : 1
    Persistence : Superblock is persistent

  Intent Bitmap : Internal

    Update Time : Fri Feb 24 19:55:16 2017
          State : clean, degraded
 Active Devices : 1
Working Devices : 1
 Failed Devices : 0
  Spare Devices : 0

           Name : localhost.localdomain:boot
           UUID : 1b78d1ea:f5380532:4fe25553:20ef8396
         Events : 44

    Number   Major   Minor   RaidDevice State
       -       0        0        0      removed
       1       8       18        1      active sync   /dev/sdb2

# mdadm --detail /dev/md127
/dev/md127:
        Version : 1.2
  Creation Time : Mon Feb 20 23:30:10 2017
     Raid Level : raid6
     Array Size : 2809500672 (2679.35 GiB 2876.93 GB)
  Used Dev Size : 936500224 (893.12 GiB 958.98 GB)
   Raid Devices : 5
  Total Devices : 4
    Persistence : Superblock is persistent

  Intent Bitmap : Internal

    Update Time : Fri Feb 24 14:42:10 2017
          State : clean, degraded
 Active Devices : 4
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 512K

           Name : localhost.localdomain:pv00
           UUID : f7a62cb2:b269daa9:29f8025f:6fcef40f
         Events : 4769

    Number   Major   Minor   RaidDevice State
       -       0        0        0      removed
       1       8       19        1      active sync   /dev/sdb3
       5       8       32        2      active sync   /dev/sdc
       3       8       49        3      active sync   /dev/sdd1
       4       8       65        4      active sync   /dev/sde1
```

重建 Raid

```
# fdisk -l /dev/sda

Disk /dev/sda: 960.2 GB, 960197124096 bytes, 1875385008 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: dos
Disk identifier: 0x000e7231

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048       22527       10240   41  PPC PReP Boot
/dev/sda2           22528     2121727     1049600   fd  Linux raid autodetect
/dev/sda3         2121728  1875385007   936631640   fd  Linux raid autodetect

# mdadm /dev/md126 --add /dev/sda2
# mdadm /dev/md127 --add /dev/sda3 
```

# 磁碟管理-StorCLI

這款機器的 Local Disks 都連結在內建的 MegaRAID (MegaRAID SAS-3 3108) 介面上，如果系統已經安裝好 Linux，可以透過指令 lshw 確認這個裝置存在。

> 雖然有 Raid 卡，卻無法建立 Raid，從 Petitboot 指令模式下，使用 storcli 指令新增 Raid 失敗。  
> 錯誤訊息: Resources already in use

##### 下載安裝 StorCLI

下載網址：https://www.broadcom.com/support/download-search/?pg=&amp;pf=&amp;pn=&amp;po=&amp;pa=&amp;dk=storcli

StorCLI for Linux 其實只是一個編譯過的指令 storcli (storcli64 for 64-bit)，官方只是將這指令包裝成 RPM 或 DEB 方便用戶在可以在不同的作業系統版本做安裝；只是筆者從連結下載的壓縮包，解開後找不到 PPC64LE for RedHat，但有看到 PPC64LE for Ubuntu (\*.deb)，所以只要用另一部 Windows 或 Linux 電腦將 \*.deb 解開就會看到一個檔案 `storcli64` 。

將這個檔案複製到已經安裝好的 RedHat Linux 即可使用。

> storcli64 也能用在 Petitboot，直接用 scp 或 wget 從外部電腦複製。

##### 在 Linux 內確認 MegaRAID 裝置

```
# lshw

...
*-raid
   description: RAID bus controller
   product: MegaRAID SAS-3 3108 [Invader]
   vendor: LSI Logic / Symbios Logic
   physical id: 0
   bus info: pci@0001:03:00.0
   logical name: scsi0
   version: 02
   width: 64 bits
   clock: 33MHz
   capabilities: raid pm pciexpress vpd msi msix bus_master cap_list rom
   configuration: driver=megaraid_sas latency=0
   resources: irq:507 memory:3fe080200000-3fe08020ffff memory:3fe080000000-3fe0800fffff memory:3fe080100000-3fe0801fffff
 *-enclosure UNCLAIMED
      description: SCSI Enclosure
      product: SAS3x28
      vendor: LSI
      physical id: 0.8.0
      bus info: scsi@0:0.8.0
      version: 0601
      configuration: ansiversion=5
 *-disk:0
      description: ATA Disk
      product: SDLF1DAR-960G-1H
      physical id: 0.9.0
      bus info: scsi@0:0.9.0
      logical name: /dev/sda
      version: ZH06
      serial: A008676A
      size: 894GiB (960GB)
      capacity: 894GiB (960GB)
      capabilities: 15000rpm partitioned partitioned:dos
      configuration: ansiversion=6 logicalsectorsize=512 sectorsize=4096 signature=000e7231
    *-volume:0
         description: PPC PReP Boot partition
         physical id: 1
         bus info: scsi@0:0.9.0,1
         logical name: /dev/sda1
         capacity: 10MiB
         capabilities: primary bootable boot
    *-volume:1
         description: Linux raid autodetect partition
         physical id: 2
         bus info: scsi@0:0.9.0,2
         logical name: /dev/sda2
         capacity: 1025MiB
         capabilities: primary multi
    *-volume:2
         description: Linux raid autodetect partition
         physical id: 3
         bus info: scsi@0:0.9.0,3
         logical name: /dev/sda3
         capacity: 893GiB
         capabilities: primary multi
 *-disk:1
      description: ATA Disk
      product: SDLF1DAR-960G-1H
      physical id: 0.a.0
      bus info: scsi@0:0.10.0
      logical name: /dev/sdb
      version: ZH06
      serial: A0086638
      size: 894GiB (960GB)
      capacity: 894GiB (960GB)
      capabilities: 15000rpm partitioned partitioned:dos
      configuration: ansiversion=6 logicalsectorsize=512 sectorsize=4096 signature=0003819f
    *-volume:0
         description: PPC PReP Boot partition
         physical id: 1
         bus info: scsi@0:0.10.0,1
         logical name: /dev/sdb1 
...
```

# LVM

LVM 磁碟管理

# Learning

##### 單個磁碟擴充未分割磁區

- [How to Resize LVM Partition Inside an Extended Partition](https://linuxhandbook.com/resize-lvm-partition/)

##### LVM Snapshot

- [Use LVM Snapshot To Backup Your Data In Linux - OSTechNix](https://ostechnix.com/how-to-use-lvm-snapshot-to-backup-your-data-in-linux/)

# 常用指令

#### Logical Volume - LV

```shell
# 顯示所有 LV 使用狀況
lvdisplay -C

# 新增 LV
lvcreate -L 10G -n LV_name VG_name

# 刪除 LV
lvremove /dev/VG_name>/LV_name

# 更名 LV
lvrename /dev/VG-name/old-LV-name /dev/VG-name/new-LV-name

# 顯示 LV 狀況
lvs
lvs -a -o name,copy_percent,devices
lvs -a -o name,copy_percent,devices <vg-name>
lvs -a --segments -o +devices

# 擴充 LV 使用空間
lvextend -L +2G /dev/vg/lv
lvextend -l +100%FREE /dev/vg_db2v9/lv_root  將剩餘空間都加上

ext2online /dev/vg/lv (RHEL v4)
resize2fs /dev/vg/lv (RHEL v5,6) 
xfs_growfs /dev/vg/lv (RHEL v7 with XFS filesystem)
btrfs filesystem resize max <mount-point>  (Fedora 40)

# 縮小 LV 空間 (RHEL v4)
#說明：445GB -> 2GB
umount /worktmp
e2fsck -f /dev/rootVG/worktmpLV
resize2fs /dev/rootVG/worktmpLV 1843M
lvreduce -L 2GB /dev/rootVG/worktmpLV
resize2fs /dev/rootVG/worktmpLV
mount /worktmp 
#註：1843MB 大約是要 2GB 的 90%，這裡的指令也能使用單位 GB

# Shrinking LV (RHEL 5/6)
#100 GB -> 5GB
umount /opt/oracle/arclog
lvreduce -L 5G /dev/VolGroup00/arclogLV
resize2fs /dev/VolGroup00/arclogLV 5G
e2fsck -f /dev/VolGroup00/arclogLV
mount /opt/oracle/arclog
```

#### Volume Group - VG

```shell
# 顯示 VG 使用狀況
vgdisplay -C

# 啟動/關閉 VG
vgchange -a y VG_name
vgchange -a n VG_name
#註：關閉後，執行 vgdisplay 仍會顯示，但 /dev/*** 已不存在。

# 新建 VG
vgcreate VG_name /dev/pv1 /dev/pv2 

# 更名 VG
vgrename vg_esxa01db01 vg_root

# 刪除 VG
vgremove VG_name

# 增加 PV 到 VG
vgextend VG_name /dev/pv3 

# 從 VG 中移除 PV
#NOTE: PV 必須沒有任何資料(檢查指令 pvs -o+pv_used)
vgreduce VG_name /dev/mypv 
```

#### Physical Volume - PV

```shell
# 顯示 PV 使用狀況
pvdisplay -C

# 檢查所有 PV 的空間使用狀況
pvs -o+pv_used

# 新增 PV
pvcreate /dev/hdd1 

# 刪除 PV
pvremove /dev/hdd1 

# 如果整個 disk 都要作為 PV，該磁碟不能建立任何磁區，要清除舊磁區可以執行
dd if=/dev/zero of=/dev/sdd bs=512 count=1 

# 既有的 LUN Disk 增加磁碟空間後，還需要執行
pvresize /dev/sd[X]
```

##### LVM 磁區的磁碟加大

> 在 RHEL 8 下，移除 LVM 磁區時，可能出現提示:
> 
> Do you want to remove the signature? \[Y\]es/\[N\]o:
> 
> 注意: 這裡必須是 N，否則原有的磁區會被徹底清除。
> 
> 技巧: 要確認 LVM 磁區是有完成延伸，或者不小心被移除，在 reboot 前執行
> 
> pvs
> 
> 如果有顯示原有的 PV disk，表示磁區延伸成功。

Extend PV Disk: /dev/sda2 50GB to 100GB

```
[root@my-db2v11fp7 ~]# lsblk
NAME               MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda                  8:0    0  100G  0 disk
├─sda1               8:1    0    1G  0 part /boot
└─sda2               8:2    0   49G  0 part
  ├─rootvg-root    253:0    0   15G  0 lvm  /
  ├─rootvg-swap    253:1    0    4G  0 lvm  [SWAP]
  └─rootvg-worktmp 253:2    0  512M  0 lvm  /worktmp
sr0                 11:0    1 1024M  0 rom

[root@my-db2v11fp7 ~]# pvs
  PV         VG     Fmt  Attr PSize   PFree
  /dev/sda2  rootvg lvm2 a--  <49.00g <29.50g
  
[root@my-db2v11fp7 ~]# fdisk -ul /dev/sda
Command (m for help): p

Disk /dev/sda: 107.4 GB, 107374182400 bytes, 209715200 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000def6b

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     2099199     1048576   83  Linux
/dev/sda2         2099200   104857599    51379200   8e  Linux LVM

Command (m for help): d
Partition number (1,2, default 2): 2
Partition 2 is deleted

Command (m for help): p

Disk /dev/sda: 107.4 GB, 107374182400 bytes, 209715200 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000def6b

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     2099199     1048576   83  Linux

Command (m for help): n
Partition type:
   p   primary (1 primary, 0 extended, 3 free)
   e   extended
Select (default p): p
Partition number (2-4, default 2): 2
First sector (2099200-209715199, default 2099200):
Using default value 2099200
Last sector, +sectors or +size{K,M,G} (2099200-209715199, default 209715199):
Using default value 209715199
Partition 2 of type Linux and of size 99 GiB is set

Command (m for help): t
Partition number (1,2, default 2): 2
Hex code (type L to list all codes): 8e
Changed type of partition 'Linux' to 'Linux LVM'

Command (m for help): p

Disk /dev/sda: 107.4 GB, 107374182400 bytes, 209715200 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000def6b

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     2099199     1048576   83  Linux
/dev/sda2         2099200   209715199   103808000   8e  Linux LVM

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.


[root@my-db2v11fp7 ~]# reboot


[root@my-db2v11fp7 ~]# lsblk
NAME               MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda                  8:0    0  100G  0 disk
├─sda1               8:1    0    1G  0 part /boot
└─sda2               8:2    0   99G  0 part
  ├─rootvg-root    253:0    0   15G  0 lvm  /
  ├─rootvg-swap    253:1    0    4G  0 lvm  [SWAP]
  └─rootvg-worktmp 253:2    0  512M  0 lvm  /worktmp
sr0                 11:0    1 1024M  0 rom

[root@my-db2v11fp7 ~]# pvresize /dev/sda2
  Physical volume "/dev/sda2" changed
  1 physical volume(s) resized or updated / 0 physical volume(s) not resized
[root@my-db2v11fp7 ~]# pvs
  PV         VG     Fmt  Attr PSize   PFree
  /dev/sda2  rootvg lvm2 a--  <99.00g <79.50g

```

##### 指令速查表

LVM

[![linux_LVM_small.jpg](https://osslab.tw/uploads/images/gallery/2023-10/scaled-1680-/linux-lvm-small.jpg)](https://osslab.tw/uploads/images/gallery/2023-10/linux-lvm-small.jpg)

# FAQ

##### 無法新增 PV

> Device /dev/sdj not found (or ignored by filtering).

Solution: 可能該磁碟有包含其他磁區

```
# fdisk -l /dev/sdj

WARNING: GPT (GUID Partition Table) detected on '/dev/sdj'! The util fdisk doesn't support GPT. Use GNU Parted.


Disk /dev/sdj: 3848.3 GB, 3848290697216 bytes
255 heads, 63 sectors/track, 467861 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdj1               1      267350  2147483647+  ee  GPT
Partition 1 does not start on physical sector boundary.
```

移除這磁碟裡的所有磁區後再試一次。

##### 移除已損壞的 PV

無法移除已經損壞 PV 的 VG 與 LV

> 有一個 PV - /dev/sdc 發生損毀，所屬 VG 與 LV 要移除時遇到錯誤：
> 
>  Couldn't find device with uuid APgmaN-yCZG-WfQS-L5zm-vd6g-oEJc-WjzNrT.  
>  Can't remove locked LV isths\_DMS
> 
>  Couldn't find device with uuid APgmaN-yCZG-WfQS-L5zm-vd6g-oEJc-WjzNrT.  
>  WARNING: Partial LV isths\_home needs to be repaired or removed.  
>  WARNING: Partial LV isths\_DMS needs to be repaired or removed.  
>  WARNING: Partial LV pvmove0 needs to be repaired or removed.  
>  There are still partial LVs in VG vg\_db.  
>  To remove them unconditionally use: vgreduce --removemissing --force.  
>  Proceeding to remove empty missing PVs.

Solution: VG Name: vg\_db

```
# vgcfgbackup vg_db
  Couldn't find device with uuid APgmaN-yCZG-WfQS-L5zm-vd6g-oEJc-WjzNrT.
  Volume group "vg_db" successfully backed up.
```

/etc/lvm/backup/vg\_db:

```
...
logical_volumes {
...
...
}
... 
```

> 將段落 logical\_volumes 整個移除。

執行 vgcfgrestore

```
# vgcfgrestore vg_db
  Restored volume group vg_db
```

重新掃描再重新執行移除 LV 與 VG

```
pvs
vgs
lvs
vgremove vg_db
```

##### USB 掛載包含有 LVM 的硬碟，出現錯誤訊息

> \# mount -t ext3 /dev/VolGroup00/LogVol00 /mnt/lvm00  
>  mount: wrong fs type, bad option, bad superblock on /dev/mapper/VolGroup00-LogVol00,  
>  missing codepage or helper program, or other error  
>  In some cases useful info is found in syslog - try  
>  dmesg | tail or so

Solution:

```shell
vgchange -an VolGroup00
vgchange -ay VolGroup00
mount -t ext3 /dev/VolGroup00/LogVol00 /mnt/lvm00
```

##### 無法新增 LV

系統剛安裝完，執行 pvdisplay -C 出現錯誤，並且無法新增 LV

> Couldn't find device with uuid ZrneWS-8KHd-UiXV-rQ4q-YU0o-r9Ga-0ZVBPW

Solution: 這個硬碟可能曾經做過 LVM 格式化，這些是之前遺留的裝置名稱，要清除這些不存在的舊設定。

```
# vgreduce -removemissing <vg-name>
```

> TIPs：
> 
>  要查出這些不存在的 device 的所屬 VG，可以執行 pvdisplay 。

##### 建立 GPT 磁區

With parted

```
# parted /dev/vdb
GNU Parted 3.4
Using /dev/vdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) mklabel GPT                                                      
(parted) mkpart primary 2048s 100%                                        
(parted) q                                                                
Information: You may need to update /etc/fstab.
```

# 進階應用

#### VG Import/Export

用途：將 VG 的磁碟遷移至不同的主機上。

> 大致方法是在遷移磁碟前，先將 VG 做 Exort；遷移後做 VG Import。

- [How To Move LVM Volume Group To Another Machine In Linux](https://ostechnix.com/move-lvm-volume-group-to-another-machine/)

#### 移除 PV Disk

從一個現有 LV 中移除正在使用的一個 PV Disk (/dev/sdb1)

> NOTE: 這個 LV 的所屬 VG 必須要有足夠的可用空間 [https://access.redhat.com/site/documentation/en-US/Red\_Hat\_Enterprise\_Linux/6/html-single/Logical\_Volume\_Manager\_Administration/index.html#disk\_remove\_ex](https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html-single/Logical_Volume_Manager_Administration/index.html#disk_remove_ex)
> 
> [http://www.tecmint.com/lvm-storage-migration/](http://www.tecmint.com/lvm-storage-migration/)

```
# pvs -o+pv_used
  PV         VG   Fmt  Attr PSize  PFree  Used
  /dev/sda1  myvg lvm2 a-   17.15G 12.15G  5.00G
  /dev/sdb1  myvg lvm2 a-   17.15G 12.15G  5.00G
  /dev/sdc1  myvg lvm2 a-   17.15G 12.15G  5.00G
  /dev/sdd1  myvg lvm2 a-   17.15G  2.15G 15.00G
```

```
# pvremove /dev/sdb1
  /dev/sdb1: Moved: 2.0%
 ...
  /dev/sdb1: Moved: 79.2%
 ...
  /dev/sdb1: Moved: 100.0%
```

```
# pvs -o+pv_used
  PV         VG   Fmt  Attr PSize  PFree  Used
  /dev/sda1  myvg lvm2 a-   17.15G  7.15G 10.00G
  /dev/sdb1  myvg lvm2 a-   17.15G 17.15G     0
  /dev/sdc1  myvg lvm2 a-   17.15G 12.15G  5.00G
  /dev/sdd1  myvg lvm2 a-   17.15G  2.15G 15.00G
```

```
# vgreduce myvg /dev/sdb1
  Removed "/dev/sdb1" from volume group "myvg"
[root@tng3-1 ~]# pvs
  PV         VG   Fmt  Attr PSize  PFree
  /dev/sda1  myvg lvm2 a-   17.15G  7.15G
  /dev/sdb1       lvm2 --   17.15G 17.15G
  /dev/sdc1  myvg lvm2 a-   17.15G 12.15G
  /dev/sdd1  myvg lvm2 a-   17.15G  2.15G
```

#### 搬移 PV Disk

搬移現有 PV Disk (/dev/sdb1) 內的所有資料至一個新的 PV Disk (/dev/sdd1)

方法一：使用 Mirror 方式 \*建議\*

參考連結:

- [https://www.thegeekdiary.com/centos-rhel-7-how-to-create-and-remove-the-lvm-mirrors-using-lvconvert/](https://www.thegeekdiary.com/centos-rhel-7-how-to-create-and-remove-the-lvm-mirrors-using-lvconvert/)
- [https://linux.m2osw.com/creating-drive-mirror-lvm-including-exact-partition-cloning](https://linux.m2osw.com/creating-drive-mirror-lvm-including-exact-partition-cloning)

說明:

- VG: vg\_data
- LV: db\_worktmp, lv\_mydev, lv\_worktmp
- PV: from /dev/sdb1 to /dev/sdc

1\. 確認新硬碟的容量需求

```
#> lvs -a -o name,copy_percent,devices,lv_size vg_data

  LV         Cpy%Sync Devices         LSize
  db_worktmp          /dev/sdb1(0)      20.00g
  lv_mydev            /dev/sdb1(5121)   60.00g
  lv_mydev            /dev/sdb1(10242)  60.00g
  lv_worktmp          /dev/sdb1(20482) 500.00m 
```

2\. 將新硬碟加入到同一個 VG 內

```
#> vgextend vg_data /dev/sdc
```

3\. 逐一將所有 LV 作完 Mirror

```
#> lvconvert -m1 vg_data/db_worktmp
#> lvconvert -m1 vg_data/lv_mydev
#> lvconvert -m1 vg_data/lv_worktmp 

檢查 Mirror 時的進度
#> lvs -a -o name,copy_percent,devices,lv_size vg_data 
```

4\. 逐一將所有 LV 移除 Mirror，需指定舊硬碟代號

```
#> lvconvert -m0 vg_data/db_worktmp /dev/sdb1
#> lvconvert -m0 vg_data/lv_mydev /dev/sdb1
#> lvconvert -m0 vg_data/lv_worktmp /dev/sdb1
```

5\. 從 VG 裡移除 PV /dev/sdb1, 再移除整個 PV

```
#> vgreduce vg_data /dev/sdb1
#> pvremove /dev/sdb1 
```

方法二：使用 PV 搬移指令 pvmove

NOTE: PV 必須在同一個 VG 才能互相搬移

```
# pvs -o+pv_used
  PV         VG   Fmt  Attr PSize  PFree  Used
  /dev/sda1  myvg lvm2 a-   17.15G  7.15G 10.00G
  /dev/sdb1  myvg lvm2 a-   17.15G 15.15G  2.00G
  /dev/sdc1  myvg lvm2 a-   17.15G 15.15G  2.00G
```

```
# pvcreate /dev/sdd1
  Physical volume "/dev/sdd1" successfully created
```

```
# vgextend myvg /dev/sdd1
  Volume group "myvg" successfully extended
[root@tng3-1]# pvs -o+pv_used
  PV         VG   Fmt  Attr PSize  PFree  Used
  /dev/sda1   myvg lvm2 a-   17.15G  7.15G 10.00G
  /dev/sdb1   myvg lvm2 a-   17.15G 15.15G  2.00G
  /dev/sdc1   myvg lvm2 a-   17.15G 15.15G  2.00G
  /dev/sdd1   myvg lvm2 a-   17.15G 17.15G     0
```

```
# pvmove /dev/sdb1 /dev/sdd1
  /dev/sdb1: Moved: 10.0%
...
  /dev/sdb1: Moved: 79.7%
...
  /dev/sdb1: Moved: 100.0%

[root@tng3-1]# pvs -o+pv_used
  PV          VG   Fmt  Attr PSize  PFree  Used
  /dev/sda1   myvg lvm2 a-   17.15G  7.15G 10.00G
  /dev/sdb1   myvg lvm2 a-   17.15G 17.15G     0
  /dev/sdc1   myvg lvm2 a-   17.15G 15.15G  2.00G
  /dev/sdd1   myvg lvm2 a-   17.15G 15.15G  2.00G 
```

```
# vgreduce myvg /dev/sdb1
  Removed "/dev/sdb1" from volume group "myvg"
```

# Filesystems

Linux Filesystem Comparison

[![linux_filesystem.jpg](https://osslab.tw/uploads/images/gallery/2023-10/scaled-1680-/linux-filesystem.jpg)](https://osslab.tw/uploads/images/gallery/2023-10/linux-filesystem.jpg)

# Red Hat Subscription Manager

Red Hat 訂閱更新服務

# Yum/Dnf & Rpm

#### yum/dnf

##### 套件庫管裡

```shell
# 已啟用的套件庫清單
dnf repolist enabled

# 列出所有的套件庫包含 disabled 與 enabled
dnf repolist all
dnf repolist -v

# 已啟用套件庫的詳細資訊
dnf repoinfo

# 啟用指定的套件庫
dnf install yum-utils
dnf config-manager --enable <repositoryID>
# 停用指定的套件庫
dnf config-manager --disable <repositoryID>
```

##### 系統更新

```shell
# 查詢所有可更新的套件
yum list updates
yum check-update
yum check-update --security

# 更新所有套件
yum update
yum update --disablerepo=epel

# 更新至指定的 release 版號
subscription-manager release --list
yum clean all
yum --releasever=8.6 update

# 更新與安全性相關的套件
yum update --security
yum update-minimal --security

# 更新指定套件
yum update sudo
yum --security update sudo
yum --security update-minimal sudo

# 系統更新驗證 (不做實際更新異動)
# NOTE: 這個驗證仍會下載更新檔至 cache 目錄，但不會有套件異動
yum update -y --setopt tsflags=test
yum clean packages

# 修補與 RHSA-XXX 相關的套件
yum update --advisory=RHSA-2019:0997
```

##### 檢視套件清單

```bash
dnf list all
dnf list installed
dnf list available

# See more options
dnf help list
```

##### 安裝/檢視指定版本套件

```shell
dnf --showduplicates list <package-name>
dnf list <package-name>-<version>
dnf install <package-name>-<version> 
```

##### 歷史更新紀錄

```shell
# To display a list of all the latest yum transactions, use:
dnf history

# To display a list of all the latest operations for a selected package, use:
dnf history list <package-name>

# To examine a particular transaction, use:
dnf history info <transactionID>
```

```
# dnf history list
Updating Subscription Management repositories.
ID     | Command line                                                                     | Date and time    | Action(s)      | Altered
---------------------------------------------------------------------------------------------------------------------------------------
    12 | update                                                                           | 2022-03-18 16:57 | I, U           |  328 EE
    11 | install haproxy                                                                  | 2021-11-17 15:37 | Install        |    1
    10 | install chrony                                                                   | 2021-11-10 17:44 | Install        |    2
     9 | install vim-enhanced                                                             | 2021-11-10 17:44 | Install        |    4
     8 | install sysstat                                                                  | 2021-11-10 17:43 | Install        |    1
     7 | install yum-utils                                                                | 2021-11-10 17:43 | Install        |    1
     6 | install net-snmp-utils                                                           | 2021-11-10 17:43 | Install        |    1
     5 | install bind-utils net-snmp                                                      | 2021-11-10 17:42 | Install        |   30
     4 | install rsync mailx                                                              | 2021-11-10 17:42 | Install        |    1
     3 | install tmux                                                                     | 2021-11-10 17:41 | Install        |    1
     2 | install redhat-lsb-core                                                          | 2021-11-10 17:38 | Install        |   26
     1 |                                                                                  | 2021-11-10 17:27 | Install        |  396 EE
```

##### Undo/Redo

```bash
yum history
# Or
yum history list all

# undo the install with ID 63
yum history undo 63

# redo the install with ID 63
yum history redo 63
```

##### 凍結套件版本

凍結特定套件版本可以避免在執行 `yum update` 系統更新時，特定套件也會被一起被更新。

```bash
# Install yum plugin: yum-versionlock
# The /etc/yum/pluginconf.d/versionlock.list will be created on the system.
# For RHEL 7.9
yum install yum-plugin-versionlock

# For RHEL 8 and 9
yum install python3-dnf-plugin-versionlock

# To install or lock the version of the gcc group of packages, run
yum versionlock gcc-*

# To display the list of locked packages, use:
yum versionlock list

# To discard the list of locked packages, use:
yum versionlock clear

# To discard the lock on a specific package, use:
yum versionlock delete <package_name>
```

##### 凍結系統版本 (Release)  


```bash
# To determine which releases are available:
subscription-manager release --list

# Temporary Setting
yum clean all
yum --releasever=8.6 update

# Permanent Setting
subscription-manager release --set=8.6
yum clean all
subscription-manager repos --list-enabled

# To determine which releases system is set to:
subscription-manager release --show

# To unset a specific release:
subscription-manager release --unset
yum clean all
subscription-manager repos
```

##### 各種舊版本的套件庫位址

[http://vault.centos.org/](http://vault.centos.org/)

##### 清除套件庫清單及暫存的套件檔

```shell
dnf clean all
```

##### 搜尋套件

搜尋檔案 libstdc++.so.6

```shell
dnf whatprovides "*/libstdc++.so.6" 
dnf provides "*/libstdc++.so.6"
```

搜尋套件

```shell
dnf search mypackage
dnf search --all mypackage
```

##### 套件的相依性

```shell
dnf deplist <package-name>

repoquery --requires <package-name>
```

##### 下載 RPM 套件

```shell
yum -y install yum-utils.noarch
yumdownloader <package-name>
yumdownloader --destdir /path/to/download <package-name>
# Includ all dependencies
yumdownloader --resolve --alldeps <package-name>

# Extract downloaded RPM
rpm2cpio dekiwiki-10.0.1-3.1.noarch.rpm | cpio -idmv 

yum -y install --downloadonly --downloaddir=/tmp/packages NetworkManager

```

##### 一次性啟用套件庫位址

```shell
dnf --enablerepo=elrepo-kernel install kernel-ml kernel-ml-devel
```

##### 使用光碟 ISO方式來安裝套件

```
# vi /etc/yum.repo.d/CentOS-Media.repo

[c6-local]
name=CentOS-$releasever - Media
baseurl=file:///mnt/disc/
gpgcheck=0
enabled=0

NOTE: enabled=0，不是 1。將 DVD 掛載為 /mnt/disc。 

# yum --disablerepo="*" --enablerepo=c6-local list available
# yum --disablerepo="*" --enablerepo=c6-local install nfs-utils

```

##### 安裝本機端的 RPM 套件

```shell
dnf --nogpgcheck install htop-1.0.2-1.el5.rf.x86_64.rpm
```

##### 匯出已安裝套件列表

```shell
yum list installed
yum list installed |tail -n +3|cut -d' ' -f1 > installed_packages.lst
# 或
rpm -qa > installed_packages.lst

yum -y install $(cat installed_packages.lst) 

```

##### COPR 套件庫

```bash
# Install the tool
sudo dnf install dnf-plugins-core

# Search the repository
dnf copr search [keyword]

# List the repository
dnf copr list
dnf copr list --enabled
dnf copr list --disabled

# Enable the repository
sudo dnf copr enable [user]/[project]

# Disable the repository
sudo dnf copr disable [user]/[project]

# Remove the repository
sudo dnf copr remove [user]/[project]
```

##### EPEL 套件庫

- [Extra Packages for Enterprise Linux (EPEL) :: Fedora Docs (fedoraproject.org)](https://docs.fedoraproject.org/en-US/epel/)

```bash
# RHEL 8
sudo subscription-manager repos --enable codeready-builder-for-rhel-8-$(arch)-rpms
sudo yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
```

##### ELRepo 套件庫

[ELRepo](http://elrepo.org/tiki/ "http://elrepo.org/tiki/") - The Community Enterprise Linux Repository

```shell
rpm --import http://elrepo.org/RPM-GPG-KEY-elrepo.org

# for RHEL-5, CENTOS-5
rpm -Uvh http://elrepo.org/elrepo-release-5-2.el5.elrepo.noarch.rpm

# for RHEL-6, CENTOS-6
rpm -Uvh http://elrepo.org/elrepo-release-6-3.el6.elrepo.noarch.rpm

```

##### <span class="highlight">RPMforge/RepoForge</span> 套件庫

RPMforge 是由 Dag 及其他包裝者合作維護的。他們為 CentOS 提供超過 5000 個套件，包括 wine、vlc、mplayer、xmms-mp3 及其他受歡迎的媒體工具。它並不是 Red Hat 或 CentOS 的一部份，但卻是為那些發行版本而設計的。

http://repoforge.org/use/

##### 如何補安裝套件組

系統在第一次安裝時，都可以選擇要安裝哪些套件組，假使之後系統要補裝某套件組，例如 GNOME 視窗套件，可以使用這些指令，而不用一一找出有哪些相依性的套件名稱。

```shell
# 列出有哪些套件組可安裝
yum grouplist
# 安裝 GNOME 視窗套件組
yum groupinstall "GNOME Desktop Environment"
# 移除套件組
yum groupremove "Office Suite and Productivity" "Dial-up Networking Support"

# 顯示被隱藏的套件群組
yum grouplist hidden
```

##### Yum 設定

編輯 /etc/yum.conf

```
# Set Proxy
# proxy=http://<username>:<password>@<this.is.proxy.ip:<port>/
proxy=http://windowsad\username:password@proxy.server:port/

# Set Timeout
timeout=600
```

##### 安裝本機上的 RPM

```shell
yum localinstall my.rpm
```

#### 建立 YUM 套件庫主機 (with ISO)

- [How to create a local mirror of the latest update for Red Hat Enterprise Linux 5, 6, 7, 8 and 9 without using Satellite server?](https://access.redhat.com/solutions/23016)

安裝套件 createrepo  
CetOS 5:

```
createrepo /source/path/reas5
createrepo -g  /source/path/reas5/repodata/comps.xml

```

CetOS 6/7:

```shell
# 建立套件索引
createrepo /yum-repo-packages/centos_6.4_x86_64
 
 # 建立套件群組索引
 # CentOS 6.4
 createrepo -g /yum-repo-packages/centos_6.4_x86_64/repodata/2727...cab6f72-c6-x86_64-comps.xml /yum-repo-packages/centos_6.4_x86_64
 # RedHat 7.9
 createrepo -g /mnt/yum-repo/redhat_7.9_x86_64/repodata/3df90817a193baef023d53222cc4ce8f4d15209e593bee361bf72016022008fb-comps-Server.x86_64.xml /mnt/yum-repo/redhat_7.9_x86_64
```

RHEL 8:

```
# for RHEL 8 only
yum install createrepo_c

cp -r /mnt/iso/* /mnt/yum-repo/redhat_8.3_x86_64 
createrepo /mnt/yum-repo/redhat_8.3_x86_64
```

rhel-local.repo:

```
[LocalRepo_BaseOS]
name=LocalRepo_BaseOS
metadata_expire=-1
enabled=1
gpgcheck=1
baseurl=ftp://10.1.115/LINUX_REPO/redhat_8.3_x86_64/BaseOS/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

[LocalRepo_AppStream]
name=LocalRepo_AppStream
metadata_expire=-1
enabled=1
gpgcheck=1
baseurl=ftp://10.10.1.115/LINUX_REPO/redhat_8.3_x86_64/AppStream/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

```

#### RPM

安裝套件

```bash
rpm -ivh vim-enhanced-*.x86_64.rpm

# Dry-run only without any changes
rpm -ivh --test vim-enhanced-*.x86_64.rpm
```

查詢指令

```shell
# 查詢已安裝的套件的詳細資訊
rpm -qi <package-name>

# 查詢已安裝套件是 32 或 64 bit
rpm -qa --qf "%{n}-%{v}-%{r}.%{arch}\n" | grep -i <package-name>

# 查詢未安裝的套件資訊
rpm -qpi your.rpm
less your.rpm

# 列出已安裝套件的相依性
rpm -q --requires <package-name>

# 列出未安裝的 RPM 的相依性
rpm -qp --requires your.rpm

# 列出一個尚未安裝的 RPM 所包含的檔案列表
rpm -qlp your.rpm

# 查詢檔案所屬的套件名稱
rpm -qf /usr/bin/ksh

# 查詢 RPM Public Key
rpm -q gpg-pubkey | sort | uniq -c | sort -nr

```

列出指定套件的相依性清單

```bash
# list the dependencies required for the git rpm
# -q: query the RPM package
# -p: specify the package file
# -R: list the dependencies required for the package

rpm -qpR git-1.8.3.1-20.el7.x86_64.rpm

# the command will output a list of dependencies required for Git. For example:
/bin/bash
/bin/sh
/lib/ld-linux-x86-64.so.2
/lib64/ld-linux-x86-64.so.2
 libc.so.6(GLIBC_2.14)(64bit)
 libcurl.so.4()(64bit)
 libexpat.so.1()(64bit)
 libssl.so.10()(64bit)
 libz.so.1()(64bit)
 perl(:MODULE_COMPAT_5.16.3)
 perl(Error)
 perl(Exporter)
 perl(Fcntl)
 ... (more dependencies)

 # get the RPM package names for these dependencies
 rpm -q --whatprovides "libcurl.so.4()(64bit)"
```

清查所有已安裝套件的名稱、版本、架構等資訊

```shell
rpm -qa --queryformat '%{NAME}-%{VERSION}-%{RELEASE}.%{ARCH}\n'| sort &> /tmp/rpm-qa.prod.output.txt
```

檢查指定套件的 Changelog

```shell
rpm -q --changelog sudo | grep -i cve-2021-3156
rpm -q --changelog -p sudo-1.8.6p3-29.0.2.el6_10.3.x86_64.rpm | grep -F CVE-2021
```

Rebuild source-rpm

```shell
yum install rpm-build
rpmbuild --rebuild something.src.rpm
```

GPG

- [RPM and GPG: How to verify Linux packages before installing them](https://www.redhat.com/sysadmin/rpm-gpg-verify-packages)

```bash
# List all GPG keys
rpm -q gpg-pubkey | sort | uniq -c

# Get the information of a key
rpm -qi gpg-pubkey-2f86d6a1-5cf7cefb
```

清除 cache

```shell
yum clean all
rm /var/cache/yum/* -rf
```

#### 常用管理工具安裝

使用 minimal ISO 安裝後，有些常用工具必須手動安裝。

##### RedHat 6

```bash
yum install setuptool system-config-network* system-config-securitylevel-tui system-config-keyboard \
  lvm2 lsof nfs-utils sysstat net-snmp net-snmp-utils vim-enhanced rsync mailx bind-utils vim-enhanced \
  net-tools
```

##### RedHat 7/8

```bash
yum install chrony tmux vim-enhanced rsync mailx bind-utils net-snmp net-snmp-utils \
  yum-utils sysstat nfs-utils redhat-lsb-core lsof net-tools

# for VMware guest OS
yum install open-vm-tools
```

#### 桌面環境安裝

##### RHEL 8 &amp; 9

```bash
yum group install GNOME base-x Fonts

#or 

yum groupinstall "Server with GUI"
```

##### RHEL 7

```bash
yum groupinstall gnome-desktop x11 fonts

# Or

yum groupinstall "Server with GUI"
```

##### RHEL 6

```bash
# Recommended
yum groupinstall Desktop "General Purpose Desktop" "Desktop Platform" "X Window System"  "Internet Browser" "Graphical Administration Tools" Fonts

# Minimal
yum groupinstall Desktop "X Window System" Fonts
```

##### RHEL 5

```bash
# Recommended
yum groupinstall gnome-desktop  base-x  graphical-internet  admin-tools

# Minimal
yum groupinstall gnome-desktop  base-x
```

#### CVE 相關指令  


- [RedHat CVE Checker](https://access.redhat.com/labs/cvechecker/)
- [RedHat CVE Database](https://access.redhat.com/security/security-updates/cve)

查詢目前系統是否有 CVE 的危害 (需要官網訂閱連線)

```bash
# 沒有內容輸出，表示沒有這個 CVE 的危害
yum updateinfo info --cve CVE-2023-48795

# With RHSA
yum updateinfo info --advisory RHSA-2023:7549
```

查詢目前系統的特定套件更新紀錄有無包含 CVE 編號

```bash
# rpm -q --changelog [package-name] | grep [CVE-NUMBER]
rpm -q --changelog openssl | grep CVE-2021-3450
```

查詢所有可用的安全性更新 (RHSA 通報資訊)

```bash
# 未安裝的
yum updateinfo list updates security

# 已安裝的
yum updateinfo list security --installed

# 顯示特定 RHSA 號碼的詳細資訊
yum updateinfo info <RHSA ID>
```

更新指定的 RHSA-ID

```bash
yum check-update --advisory=RHSA-2024:1249

yum update --advisory=RHSA-2024:1249
```

#### FAQ  


##### About EPEL Repository

**Important Notice**

- The following information has been provided by Red Hat, but is outside the scope of the posted Service Level Agreements and support procedures.
- Installing unsupported packages does not necessarily make a system unsupportable by Red Hat Global Support Services 
    - However, Red Hat Global Support Services will be unable to support or debug problems with packages not shipped in standard RHEL channels.
- Installing packages from EPEL is done at the user's own risk.
- The EPEL repository is a community supported repository hosted by the Fedora Community project.
- The EPEL repository is not a part of Red Hat Enterprise Linux and does not fall under Red Hat's Production Support Scope of Coverage. The repository is considered an optional repository and is not tested by Red Hat quality engineers.

([https://access.redhat.com/solutions/3358](https://access.redhat.com/solutions/3358) )

##### What is the difference between yum update vs yum update-minimal

[https://access.redhat.com/solutions/3620411](https://access.redhat.com/solutions/3620411)

##### How to Upgrade RHEL 8 to RHEL 9

- [How to Upgrade from RHEL 8 to RHEL 9 Release](https://www.tecmint.com/upgrade-rhel-8-to-rhel-9/)

##### RHEL 4 沒有 yum 指令

RHEL 4 沒有官方的 yum 套件，需要安裝社群版的。

解開下方 zip 檔，安裝所有 \*.rpm。

[Yum\_rhel4.zip](https://osslab.tw/attachments/85)

# Sign up and Register

#### Tutorials

- \[RH-KM\] [Using Red Hat Subscription Management](https://access.redhat.com/documentation/en-us/red_hat_subscription_management/1/html/using_red_hat_subscription_management/index)
- [How to enable Red Hat Subscription on RHEL 8/7](https://www.2daygeek.com/enable-rhel-subscription-red-hat-subscription-management/)
- [How to register RHEL 8 with subscription manager using command-line](https://www.nixcraft.com/t/how-to-register-rhel-8-with-subscription-manager-using-command-line/3712)
- [如何在 RHEL 中使用訂閱管理器啟用軟體倉庫](https://kknews.cc/zh-tw/code/nenqkq5.html)
- [How to activate your no-cost Red Hat Enterprise Linux subscription](https://developers.redhat.com/blog/2021/02/10/how-to-activate-your-no-cost-red-hat-enterprise-linux-subscription)
- [USING AND CONFIGURING SUBSCRIPTION MANAGER](https://access.redhat.com/documentation/en-us/red_hat_subscription_management/1/html/rhsm/index)
- [Answers to Common Subscription Questions - Red Hat Customer Portal](https://access.redhat.com/articles/customer-service-subscriptions)

Unregistered Subscription Manager

```
# yum repolist
Updating Subscription Management repositories.

This system is registered to Red Hat Subscription Management, but is not receiving updates. You can use subscription-manager to assign subscriptions.
```

> NOTE: 在 /etc/yum.repos.d/ 目錄裡，如果曾經有手動設定其它的 repository，請先移除，或者將它們 Disable。

#### Sign up an account

RedHat 訂閱帳號有分付費版與免費版兩種。

付費版)

- 入口網站: [https://access.redhat.com/](https://access.redhat.com/)
- 依據主機數量每年購買訂閱數。

免費版 for developer)

- Developer 帳號申請: [https://developers.redhat.com/](https://developers.redhat.com/)
- 入口網站: [https://access.redhat.com/](https://access.redhat.com/)
- 訂閱需每年手動續約。
- 一個 Developer 帳號可用於一個組織，主機數量限制在 16 以下。
- 組織規模若不是大型企業，可用於 production 主機。
- 沒有任何技術支援，也不能開 ticket。
- 可以透過 RHN 作線上更新(這與付費版相同)。
- 每年手動續約時，必須從 Developer 網站進入並登入帳號，頁面應該會出現 renew account 的按鍵，按下後即可完成免費續約。*NOTE: 如果沒出現 renew account 資訊，反覆登入 Developer 頁面試試。*
- 既使帳號已過期，也還是可以成功手動續約。
- 相關連結 
    - [https://www.redhat.com/zh-tw/blog/new-year-new-red-hat-enterprise-linux-programs-easier-ways-access-rhel](https://www.redhat.com/zh-tw/blog/new-year-new-red-hat-enterprise-linux-programs-easier-ways-access-rhel)
    - [https://www.redhat.com/wapps/tnc/viewterms/72ce03fd-1564-41f3-9707-a09747625585?extIdCarryOver=true&amp;sc\_cid=701f2000001Css0AAC](https://www.redhat.com/wapps/tnc/viewterms/72ce03fd-1564-41f3-9707-a09747625585?extIdCarryOver=true&sc_cid=701f2000001Css0AAC)
    - [How to Get Red Hat Enterprise Linux for Free?](https://linuxhandbook.com/get-red-hat-linux-free/)

申請 Developer 帳號後，在訂閱頁面會有兩個產品，主要是第二項 Red Hat Developer Subscription for Individuals，有了這個，RedHat 主機就可以像付費版那樣作線上更新。

[![rhn-developer.png](https://osslab.tw/uploads/images/gallery/2022-03/scaled-1680-/rhn-developer.png)](https://osslab.tw/uploads/images/gallery/2022-03/rhn-developer.png)

#### Configure HTTP Proxy (optional)

One-liner Command

```shell
subscription-manager config --server.proxy_hostname=proxy.example.com --server.proxy_port=8080 --server.proxy_user=admin --server.proxy_password=secret
```

Alternatively, proxy information can be added into configuration

`/etc/rhsm/rhsm.conf`:

```
# an http proxy server to use
proxy_hostname =

# port for http proxy server
proxy_port =

# user name for authenticating to an http proxy, if needed
proxy_user =

# password for basic http proxy auth, if needed
proxy_password =
```

#### Testing the connectivity to RHN

Without Proxy

```shell
curl -v https://subscription.rhn.redhat.com/subscription/ --cacert /etc/rhsm/ca/redhat-uep.pem
```

With Proxy

```shell
curl -v --proxy-user user:password --proxy proxy.example.com:8080 https://subscription.rhn.redhat.com/subscription/ --cacert /etc/rhsm/ca/redhat-uep.pem

curl -v --proxy-user user:password --proxy proxy.example.com:8080 https://subscription.rhsm.redhat.com/ --cacert /etc/rhsm/ca/redhat-uep.pem

curl -v --proxy-user user:password --proxy proxy.example.com:8080 https://cdn.redhat.com/ --cacert /etc/rhsm/ca/redhat-uep.pem
```

```
*   Trying 10.14.25.128...
* TCP_NODELAY set
* Connected to tpemispr01.winfoundry.com (10.14.25.128) port 8080 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to subscription.rhn.redhat.com:443
> CONNECT subscription.rhn.redhat.com:443 HTTP/1.1
> Host: subscription.rhn.redhat.com:443
> User-Agent: curl/7.61.1
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/rhsm/ca/redhat-uep.pem
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* CONNECT phase completed!
* CONNECT phase completed!
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=US; ST=North Carolina; O=Red Hat, Inc.; OU=Red Hat Subscription Management; CN=subscription.rhsm.redhat.com; emailAddress=ca-support@redhat.com
*  start date: May  7 00:43:39 2020 GMT
*  expire date: May  7 00:43:39 2023 GMT
*  issuer: C=US; ST=North Carolina; O=Red Hat, Inc.; OU=Red Hat Network; CN=Red Hat Entitlement Operations Authority; emailAddress=ca-support@redhat.com
*  SSL certificate verify ok.
> GET /subscription/ HTTP/1.1
> Host: subscription.rhn.redhat.com
> User-Agent: curl/7.61.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: openresty/1.19.9.1
< Date: Wed, 16 Feb 2022 06:26:59 GMT
< Content-Type: application/json
< Transfer-Encoding: chunked
< Connection: keep-alive
< x-candlepin-request-uuid: 616ae5b2-53a2-4e9d-816e-b359545c4805
< X-Version: 3.2.22-1
<
[{"rel":"consumertypes","href":"/consumertypes"},{"rel":"distributor_versions","href":"/distributor_versions"},{"rel":"","href":"/"},{"rel":"admin","href":"/admin"},{"rel":"content","href":"/content"},{"rel":"cdn","href":"/cdn"},{"rel":"jobs","href":"/jobs"},{"rel":"crl","href":"/crl"},{"rel":"deleted_consumers","href":"/deleted_consumers"},{"rel":"rules","href":"/rules"},{"rel":"products","href":"/products"},{"rel":"roles","href":"/roles"},{"rel":"subscriptions","href":"/subscriptions"},{"rel":"activation_keys","href":"/activation_keys"},{"rel":"status","href":"/status"},{"rel":"consumers","href":"/consumers"},{"rel":"content_overrides","href":"/consumers/{consumer_uuid}/content_overrides"},{"rel":"users","href":"/users"},{"rel":"hypervisors","href":"/hypervisors"},{"rel":"guestids","href":"/consumers/{consumer_uuid}/guestids"},{"rel":"entitlements","href":"/entitlements"},{"rel":"owners","href":"/owners"},{"rel":"pools","href":"/pools"},{"rel":"serials","href":"/serials"},{"rel":"packages", "href":"/consume* Connection #0 to host tpemispr01.winfoundry.com left intact
```

#### Register to RHN

```shell
subscription-manager remove --all
subscription-manager unregister
subscription-manager clean
yum clean all
rm -rf /var/cache/yum/*

# Option#1:  With the credentials
subscription-manager register --username myname --password 'ThisPassword'
# Option#2: With an Activation Key
# Create a key from the url https://access.redhat.com/management/activation_keys
subscription-manager register --org=<ORG-ID> --activationkey=<NAME>
```

> Update: 指令 `subscription-manager attach` 已經不需要，預設會啟用 SCA (Simple Content Access)，詳細資訊請見：[https://access.redhat.com/solutions/7080864](https://access.redhat.com/solutions/7080864)

```
[~]# yum clean all
Updating Subscription Management repositories.
Unable to read consumer identity

This system is not registered with an entitlement server. You can use subscription-manager to register.

17 files removed
[~]# subscription-manager register --username myuser --password 'mypassword'
Registering to: subscription.rhsm.redhat.com:443/subscription
The system has been registered with ID: 36e3da57-5896-488e-ab8e-9f95a48c9f8c
The registered system name is: haproxy.winfoundry.com

[~]# yum repolist
Updating Subscription Management repositories.

This system is registered with an entitlement server, but is not receiving updates. You can use subscription-manager to assign subscriptions.

[~]# subscription-manager attach --auto
Installed Product Current Status:
Product Name: Red Hat Enterprise Linux for x86_64
Status:       Subscribed


[~]# subscription-manager status
+-------------------------------------------+
   System Status Details
+-------------------------------------------+
Overall Status: Current

System Purpose Status: Matched


[~]# yum repolist
Updating Subscription Management repositories.
repo id                              repo name
rhel-8-for-x86_64-appstream-rpms     Red Hat Enterprise Linux 8 for x86_64 - AppStream (RPMs)
rhel-8-for-x86_64-baseos-rpms        Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)
```

Registered Subscription Manager

```
[~]# yum repolist
Updating Subscription Management repositories.
repo id                                                   repo name
rhel-8-for-x86_64-appstream-rpms                          Red Hat Enterprise Linux 8 for x86_64 - AppStream (RPMs)
rhel-8-for-x86_64-baseos-rpms                             Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)
```

登入 [https://access.redhat.com/](https://access.redhat.com/)，在系統清單會顯示主機名稱。

[![subscription-registered.png](http://www.osslab.tw/uploads/images/gallery/2022-03/scaled-1680-/subscription-registered.png)](http://www.osslab.tw/uploads/images/gallery/2022-03/subscription-registered.png)

##### Registering an offline system (optional)

如果主機端沒有網際網路，也可以使用離線註冊方式。

1. **Create a system profile.** From the [systems](https://access.redhat.com/management/systems) page in Red Hat Subscription Management, click the New button. Provide the required information to finish creating the new system profile.
2. **Attach subscriptions.** In your newly created system profile, click the Subscriptions tab, and attach any subscriptions you want to use with the system.
3. **Download and import the entitlement certificate(s).** From the Subscriptions tab on your system profile, click Download Certificates to download the entitlement certificate(s) for attached subscriptions. The downloaded file will be in zip format. Extract the content and in /export/entitlement\_certificates/ folder you will find the certificate xyz.pem. Move it to the client system’s /tmp directory.

```
# subscription-manager import --certificate=/tmp/Name_Of_Downloaded_Entitlement_Cert.pem
```

##### Restoring a registration

主機端曾經成功註冊過，因為某些原因，系統裡的註冊紀錄遺失、或者系統重新安裝後，可以回復原先的註冊紀錄。

以 System UUID 方式重新註冊原有的主機

```
# subscription-manager register --consumerid=SYSTEM-UUID --username=MYUSER --password=MYPASS
```

#### Verifying Subscription

```
[root@haproxy ~]# subscription-manager list
+-------------------------------------------+
    Installed Product Status
+-------------------------------------------+
Product Name:   Red Hat Enterprise Linux for x86_64
Product ID:     479
Version:        8.3
Arch:           x86_64
Status:         Subscribed
Status Details:
Starts:         03/02/2022
Ends:           03/02/2023


[root@haproxy ~]# yum repolist enabled
Updating Subscription Management repositories.
repo id                                                   repo name
rhel-8-for-x86_64-appstream-rpms                          Red Hat Enterprise Linux 8 for x86_64 - AppStream (RPMs)
rhel-8-for-x86_64-baseos-rpms                             Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)
```

#### Removing from RHN  


Commands Overview

```bash
subscription-manager unsubscribe --all
subscription-manager remove --all
subscription-manager unregister
subscription-manager clean
```

Remove all subscriptions,run:

```
# subscription-manager remove --all

1 local certificate has been deleted.
1 subscription removed at the server.
```

To unregister the system from the Red Hat, run:

```
# subscription-manager unregister

Unregistering from: subscription.rhsm.redhat.com:443/subscription
System has been unregistered.
```

To remove all local data from the system, run:

```
# subscription-manager clean
All local data removed
```

登入 [https://access.redhat.com/](https://access.redhat.com/) 在訂閱頁面的系統欄，原先註冊的主機名稱會自動移除。

#### Update System  


```
yum list updates
yum update
```

#### FAQ

> Registering to: subscription.rhsm.redhat.com:443/subscription  
> Network error, unable to connect to server. Please see /var/log/rhsm/rhsm.log for more information.

Solution: Check the firewall by following the instructions below.

- `subscription.rhn.redhat.com:443` \[https\] AND `subscription.rhsm.redhat.com:443` \[https\] (This is the new default address in newer versions of RHEL 7)
- `cdn.redhat.com:443` \[https\]
- `*.akamaiedge.net:443` \[https\] OR `*.akamaitechnologies.com:443` \[https\]

It is not recommended to specify the IP addresses because the packages are distributed through the [Akamai](http://www.akamai.com/) network and the IP addresses are subject to change. However, if your firewall is unable to use host name filtering, Red Hat provides a [pool of IP addresses](https://access.redhat.com/articles/1525183) that should provide CDN delivery.

Q: subscription-manager attach --auto

> Installed Product Current Status:  
> Product Name: Red Hat Enterprise Linux for x86\_64  
> Status: Not Subscribed

Solution:

1. 登入 [https://access.redhat.com/](https://access.redhat.com/)，確定帳戶裡有可用的訂閱服務。
2. 如果是免費版 RedHat Developer 帳號，在訂閱清單裡如果沒有看到 **Red Hat Developer Subscription for Individuals** 這一項，請先登入 [https://developers.redhat.com/](https://developers.redhat.com/) 然後再確認一次。 注意: 免費版個人授權需要每年手動更新一次有效期。
3. 執行 `subscription-manager refresh`

> This system is registered to Red Hat Subscription Management, but is not receiving updates. You can use subscription-manager to assign subscriptions.

Solution:

```
subscription-manager attach --auto
```

Q: subscription-manager is showing Overall Status: Disabled

執行 `subscription-manager status`

> <div>Overall Status: Disabled</div><div>Content Access Mode is set to Simple Content Access. This host has access to content, regardless of subscription status.</div><div>  
> </div><div>System Purpose Status: Disabled</div>

新版的 RHEL 預設會啟用 SCA mode (Simple Content Access)，如果要確認註冊是否成功，可以改用

`subscription-manager identity`，出現 org name 與 ID 時表示註冊成功。詳細資訊：[https://access.redhat.com/solutions/7080864](https://access.redhat.com/solutions/7080864)

# Subscription Usage

##### Viewing subscription

列出目前有效的訂閱

```
# subscription-manager list --available
+-------------------------------------------+
    Available Subscriptions
+-------------------------------------------+
Subscription Name:   Red Hat Beta Access
Provides:            Red Hat CodeReady Linux Builder for x86_64 Beta
                     Red Hat Enterprise Linux for IBM z Systems Beta
                     Red Hat Enterprise Linux Fast Datapath Beta for Power, little endian
                     Red Hat Enterprise Linux Resilient Storage Beta
                     Red Hat Enterprise Linux for x86_64 Beta
                     Red Hat Enterprise Linux for Real Time for NFV Beta
                     Red Hat Enterprise Linux for Real Time Beta
                     Red Hat Enterprise Linux for SAP HANA for x86_64 Beta
                     Red Hat Directory Server Beta
                     Red Hat Enterprise Linux Advanced Virtualization Beta (for RHEL Server for IBM System Z)
                     Red Hat Enterprise Linux for SAP Applications for x86_64 Beta
                     Red Hat Enterprise Linux for SAP Applications for IBM z Systems Beta
                     Red Hat CodeReady Linux Builder for ARM 64 Beta
                     Red Hat Enterprise Linux for SAP Applications for Power, little endian Beta
                     Red Hat Enterprise Linux Fast Datapath Beta for x86_64
                     Red Hat CodeReady Linux Builder for Power, little endian Beta
                     Red Hat CodeReady Linux Builder for IBM z Systems Beta
                     Red Hat Enterprise Linux High Availability Beta
                     Red Hat Enterprise Linux for Power, little endian Beta
                     Red Hat Enterprise Linux for ARM 64 Beta
                     Red Hat Certificate System Beta
                     Red Hat Enterprise Linux for SAP HANA for Power, little endian Beta
SKU:                 RH00069
Contract:
Pool ID:             8a82c6557fc28ee7017fce53f78f10a7
Provides Management: No
Available:           Unlimited
Suggested:           1
Service Type:        L1-L3
Roles:
Service Level:       Self-Support
Usage:
Add-ons:
Subscription Type:   Standard
Starts:              03/27/2022
Ends:                03/27/2023
Entitlement Type:    Physical
```

##### 設定合適的系統屬性  


**System Purpose** 是系統訂閱服務的其中一個設定，每個訂閱的系統依據用途不同以及系統安裝類型，設定適合系統的各種屬性，這些屬性在自動掛載訂閱時，系統會選擇最合適的訂閱項目。

屬性類型有分 Role、Usage、Service-Level、Addons。

- role: 依據安裝的系統類型
- usage: 依據系統的用途 (免費版的只有 *Development/Test* )
- service-level: 支援服務等級 (免費版的只有 *Self-Support*)
- addons: 自訂義

要檢視各屬性包含的值，可以執行

```
# subscription-manager role --list
+-------------------------------------------+
               Available role
+-------------------------------------------+
 - Red Hat Enterprise Linux Workstation
 - Red Hat Enterprise Linux Server
 - Red Hat Enterprise Linux Compute Node
```

設定 role

```
# subscription-manager role --show
Role not set.

# subscription-manager role --set="Red Hat Enterprise Linux Server"
role set to "Red Hat Enterprise Linux Server".

# subscription-manager role --show
Current Role: Red Hat Enterprise Linux Server
```

##### Errata 資訊未更新  


如果在官網訂閱網站顯示有最新的 Errata 套件修補資訊，但 yum 指令卻不會顯示。

強制系統檢查 errata 可以執行

```shell
rm -f /var/lib/rhsm/packages/packages.json
service rhsmcertd stop
rhsmcertd --now
yum update
```

##### repos

Subscribe to the Extras channel/repo

```shell
subscription-manager repos --list
subscription-manager repos --list-enabled
subscription-manager repos --list-disabled

# subscription-manager repos --enable REPOID
subscription-manager repos --enable rhel-7-server-extras-rpms
subscription-manager repos --disable amq-clients-2-for-rhel-8-x86_64-rpms

# With yum
yum repolist --all
yum-config-manager --enable <repo-id>
```

##### Expired Subscription

> Error: You no longer have access to the repositories that provide these products. It is important that you apply an active subscription in order to resume access to security and other critical updates. If you don't have other active subscriptions, you can renew the expired subscription.

檢查儲存庫及訂閱清單 (訂閱已過期)

- 檔案 `/etc/yum.repos.d/redhat.repo` : 訂閱一旦過期，此檔案會被自動清空，且恢復成初始狀態內容；相反的，訂閱合約恢復後，稍待一會兒，此檔案會自動復原原先的內容，也可以執行 `subscription-manager refresh` 可立即更新。

```
[root@dotnetdev ~]# yum repolist
Updating Subscription Management repositories.
No repositories available

[root@dotnetdev ~]# subscription-manager repos --list
This system has no repositories available through subscriptions.

[root@dotnetdev ~]# subscription-manager list --available
No available subscription pools to list
```

檢查官網的系統 &gt; 訂閱：過期訂閱已消失或移除 (unattach)

在網站更新訂閱合約完成後，系統可以執行以下指令立即更新，或者稍待數分鐘後系統會自動更新

```bash
subscription-manager refresh
```

檢查目前有效的訂閱 (訂閱未過期)

```
# subscription-manager list --available
+-------------------------------------------+
    Available Subscriptions
+-------------------------------------------+
Subscription Name:   Red Hat Developer Subscription for Individuals
Provides:            Red Hat Enterprise Linux Fast Datapath
                     Red Hat OpenShift Enterprise JBoss EAP add-on Beta
                     Red Hat Ansible Automation Platform
                     Red Hat OpenShift Workload Availability
                     Red Hat Enterprise Linux Server
                     dotNET on RHEL (for RHEL Server)
                     Red Hat 3scale API Management Platform
                     OpenShift Developer Tools and Services
                     Red Hat CodeReady Linux Builder for ARM 64
                     Custom Metric Autoscaler
                     Red Hat Enterprise Linux Atomic Host Beta
                     Red Hat Container Images
                     Red Hat OpenShift Container Platform for ARM 64
                     Red Hat OpenShift Container Platform
                     Red Hat Enterprise Linux Resilient Storage for IBM z Systems - Extended Update Support
                     Red Hat OpenShift GitOps for ARM 64
                     Red Hat CoreOS Beta
                     OpenJDK Java (for Middleware)
                     Red Hat Enterprise Linux for SAP Solutions for x86_64 - Extended Update Support
                     Red Hat Enterprise Linux for SAP Applications for x86_64 - Update Services for SAP
                     Solutions
                     Red Hat Enterprise Linux Atomic Host
                     Red Hat Enterprise Linux High Availability (for IBM z Systems) - Extended Update Support
                     Red Hat Enterprise Linux High Availability for x86_64 - Extended Update Support
                     Network Observability (NETOBSERV)
                     Red Hat Developer Toolset (for RHEL Server)
                     Red Hat Enterprise Linux for SAP Solutions for x86_64
                     Red Hat Enterprise Linux for SAP Applications for Power LE - Update Services for SAP
                     Solutions
                     JBoss Enterprise Web Server from RHUI
                     Red Hat Software Collections (for RHEL Server)
                     Red Hat OpenShift distributed tracing
                     Red Hat Ansible Engine
                     MRG Realtime
                     Red Hat Enterprise Linux for x86_64 - Update Services for SAP Solutions
                     Red Hat Enterprise Linux AI
                     JBoss Enterprise Application Platform from RHUI
                     Red Hat Developer Tools Beta (for RHEL Server)
                     Red Hat Software Collections Beta (for RHEL Server)
                     Red Hat Enterprise Linux for IBM z Systems - Extended Update Support
                     Red Hat Enterprise Linux for ARM 64
                     Red Hat OpenShift Enterprise JBoss EAP add-on
                     Red Hat EUCJP Support (for RHEL Server) - Extended Update Support
                     JBoss Enterprise Web Platform
                     Cluster Observability Operator
                     Red Hat Enterprise Linux for SAP Applications for x86_64 - Extended Update Support
                     Red Hat OpenShift Application Runtimes Beta
                     Red Hat Migration Toolkit
                     Red Hat Enterprise Linux Scalable File System (for RHEL Server)
                     Red Hat Container Native Virtualization for ARM 64
                     Red Hat Enterprise Linux Fast Datapath Beta (for RHEL for ARM 64)
                     Red Hat Enterprise Linux for x86_64 - Extended Update Support
                     Red Hat Container Native Virtualization
                     Red Hat Beta
                     Red Hat Openshift Serverless
                     Red Hat Enterprise Linux EUS Compute Node
                     Red Hat Enterprise Linux for x86_64
                     Red Hat Developer Tools (for RHEL Server for ARM)
                     Red Hat CodeReady Linux Builder for ARM 64 - Extended Update Support
                     Red Hat CoreOS
                     Red Hat CodeReady Linux Builder for x86_64
                     Red Hat Enterprise Linux High Performance Networking (for RHEL Compute Node)
                     Oracle Java (for RHEL Server)
                     Red Hat Enterprise Linux High Performance Networking (for RHEL Server) - Extended Update
                     Support
                     Red Hat CodeReady Linux Builder for x86_64 - Extended Update Support
                     Red Hat JBoss AMQ Clients
                     Red Hat OpenShift Dev Spaces
                     Red Hat JBoss Data Grid
                     Red Hat Software Collections (for RHEL Server for ARM)
                     Kernel Module Management
                     Red Hat Openshift Application Runtimes
                     Red Hat Enterprise Linux Fast Datapath (for RHEL for ARM 64)
                     Red Hat Enterprise Linux High Availability for x86_64
                     Run Once Duration Override Operator
                     Red Hat Developer Tools Beta (for RHEL Server for ARM)
                     Red Hat JBoss Core Services from RHUI
                     Red Hat build of Quarkus
                     Red Hat Service Interconnect
                     Red Hat Enterprise Linux Load Balancer (for RHEL Server)
                     Red Hat JBoss Middleware
                     Red Hat OpenShift Enterprise Client Tools
                     Red Hat AMQ Interconnect
                     Red Hat Enterprise Linux High Performance Networking (for RHEL Server)
                     dotNET on RHEL Beta (for RHEL Server)
                     Red Hat Developer Suite v.3
                     Red Hat OpenShift Builds for ARM
                     Red Hat Openshift Application Runtimes for IBM Power LE
                     Red Hat Enterprise Linux for ARM 64 - Extended Update Support
                     Red Hat Container Images Beta
                     Red Hat Developer Tools (for RHEL Server)
                     Power monitoring for Red Hat OpenShift
                     Red Hat Software Collections Beta (for RHEL Server for ARM)
                     Oracle Java (for RHEL Server) - Extended Update Support
                     Red Hat Enterprise Linux for SAP Solutions for x86_64 - Update Services for SAP Solutions
                     Red Hat S-JIS Support (for RHEL Server) - Extended Update Support
                     Red Hat CodeReady Linux Builder for IBM z Systems - Extended Update Support
                     Red Hat Enterprise Linux Load Balancer (for RHEL Server) - Extended Update Support
                     Red Hat Enterprise Linux Fast Datapath Beta for x86_64
                     Red Hat Developer Hub
                     Red Hat OpenShift Pipelines
                     Cert Manager support for Red Hat OpenShift release
                     Red Hat OpenShift Enterprise JBoss A-MQ add-on
                     Red Hat OpenShift Pipelines for ARM
                     Red Hat OpenShift GitOps
                     Secondary Scheduler Operator for Red Hat OpenShift (OSSO)
                     Red Hat Enterprise Linux Scalable File System (for RHEL Server) - Extended Update Support
                     Red Hat Enterprise Linux for Real Time
                     Red Hat Enterprise Linux High Availability for x86_64 - Update Services for SAP Solutions
                     Red Hat OpenShift Enterprise JBoss FUSE add-on
                     JBoss Enterprise Application Platform
                     Red Hat Enterprise Linux for SAP Applications for x86_64
                     Red Hat JBoss Core Services
                     Red Hat Enterprise Linux Resilient Storage for x86_64 - Extended Update Support
                     Red Hat Enterprise Linux for Real Time Beta
                     Red Hat Quay Enterprise
                     Red Hat Enterprise Linux Resilient Storage for x86_64
                     Red Hat Container Development Kit
                     Red Hat OpenShift Builds
SKU:                 RH00798
Contract:
Pool ID:             2c94e31e96694f3201967b2cfeb21cf5
Provides Management: No
Available:           16
Suggested:           1
Service Type:
Roles:               Red Hat Enterprise Linux Workstation
Service Level:       Self-Support
Usage:               Development/Test
Add-ons:
Subscription Type:   Standard
Starts:              04/28/2025
Ends:                04/28/2026
Entitlement Type:    Physical
```

##### Cheat Sheet  


[![rh_sm_command_cheatsheet_1214_jcs_web.png](https://osslab.tw/uploads/images/gallery/2023-11/scaled-1680-/rh-sm-command-cheatsheet-1214-jcs-web.png)](https://osslab.tw/uploads/images/gallery/2023-11/rh-sm-command-cheatsheet-1214-jcs-web.png)

# One-liner Commands

1\. 建立多個目錄

```shell
mkdir -p -v /home/josevnz/tmp/{dir1,anotherdir,similardir}
```

2\. 搜尋取代關鍵字

```shell
# With sed<br></br>sed -i 's#ORIGINAL_VALLUE#NEW_VALUE#g' myfile1 myfile2<br></br><br></br># With perl<br></br>perl -p -i -e 's#ORIGINAL#NEW_VALUE#' myfile1 myfile2
```

3\. 臨時以網頁方式分享特定目錄

```shell
# Python is required<br></br>cd $mydir && python3 -m http.server 8888
```

4\. 臨時寫一個多行的檔案

```shell
cat<<DOC>/my/new/file<br></br>Line1<br></br>Line2<br></br>A $VARIABLE<br></br>DOC
```

或 Ctrl + D 結束離開

```shell
cat > testfile<br></br>This is a test<br></br>multiple lines<br></br>and here we go
```

5\. 在指定目錄裡搜尋包含特定關鍵字的檔案

```shell
# With grep<br></br>grep -R 'import' --include='*.java' --color MySourceCodeDir<br></br><br></br># With find<br></br>find MySourceCodeDir/ -name '*.java' -type f -print| xargs \<br></br>grep --color 'import
```

6\. 用 watch 監控記憶體使用

```shell
watch -n 5 -d '/bin/free -m'
```

7\. 更新所有的 Git 套件庫

```shell
for i in */.git; do cd $(dirname $i); git pull; cd ..; done
```

# Linux Rescue

#### RedHat/CentOS

- [How to recover a root password in Red Hat-based Linux systems](https://www.redhat.com/sysadmin/recover-root-passwd)

##### Single-User Mode

Situation: Reset the root's password

For GRUB Bootloader)

系統重開機 -&gt; 進入開機選單

1. Select the *kernel*
2. Press the `e` key to edit the entry
3. Select second line (the line starting with the word *kernel*)
4. Press the `e` key to edit kernel entry so that you can append single user mode
5. Append the letter `S` (or word `Single`) to the end of the (kernel) line
6. Press ENTER key (或者 Ctrl + x 直接啟動系統，跳過下一個步驟)
7. Now press the `b` key to boot the Linux kernel into single user mode
8. At prompt type `passwd` command to reset password:

```shell
mount -t proc proc /proc
mount -o remount,rw /
passwd
sync
reboot
```

For LILO Bootloader)

Boot: `linux single`

```shell
passwd
sync
reboot
```

> TIP: 在 RedHat 8 的環境，single-user mode 就是 rescue mode，輸入的字串改成 `systemd.unit=rescue.target` 。
> 
> 除了 single-user mode，還有另一個 `rd.break` 也可以試試。

##### 以安裝光碟開機的救援  


以安裝光碟開機，在開機選單裡選擇 Rescue 項目，開機過程會有以下導引:

1. 選擇語言
2. 選擇鍵盤類型
3. 是否要啟用網路功能

最後進入 shell 模式，系統的根目錄會被掛載為 /mnt/sysimage。

切換根目錄指令

```shell
chroot /mnt/sysimage
```

##### Emergency Mode (緊急開機模式)

> NOTE: Emergency 不支援網路功能，如果系統修復會需要網路，不適合使用這模式。

除了 single-user mode，還有另一種開機模式 emergency mode，也適合用來修復無法開機的系統，不過，要進入 emergency mode 必須先輸入 root 密碼，所以不適合用來做重置密碼的用途。

在 RedHat 6 環境，使用方法與 single-user mode 一樣，只是將字串替換成 `emergency`。

在 RedHat 8 環境，字串換成 `systemd.unit=emergency.target` 。

#### Ubuntu/Debian

- [Boot Into Rescue Mode Or Emergency Mode In Ubuntu - OSTechNix](https://ostechnix.com/how-to-boot-into-rescue-mode-or-emergency-mode-in-ubuntu-18-04/)

#### Reset Root's Password

- [How to Reset Forgotten Root Password in RHEL 8](https://www.tecmint.com/reset-forgotten-root-password-in-rhel-8/)
- [How to Reset Forgotten Root Password in Ubuntu (tecmint.com)](https://www.tecmint.com/reset-forgotten-root-password-in-ubuntu/)

[![rhel_root_password_recovery.png](https://osslab.tw/uploads/images/gallery/2026-01/scaled-1680-/rhel-root-password-recovery.png)](https://osslab.tw/uploads/images/gallery/2026-01/rhel-root-password-recovery.png)

#### Chroot

- [How to Use Chroot in Linux and Fix Your Broken System - Make Tech Easier](https://www.maketecheasier.com/use-chroot-linux/)

##### Fix a Broken Bootloader Using Chroot

```bash
# Make a bootable USB by downloading a Linux ISO and booting from the USB.
# Mount your system partition to work with chroot.
sudo mount -t ext4 /dev/sda /mnt

sudo mount --bind /dev /mnt/dev &&
sudo mount --bind /dev/pts /mnt/dev/pts &&
sudo mount --bind /proc /mnt/proc &&
sudo mount --bind /sys /mnt/sys

# Let’s chroot into the “/mnt” directory and enter the broken system.
sudo chroot /mnt

# Install, check, and update the GRUB bootloader in your system.
grub-install /dev/sda
grub-install --recheck /dev/sda
update-grub

# Exit the shell using the exit command mentioned earlier. 
# Unbind the previously bound directories and unmount the filesystem.
sudo umount /mnt/sys &&
sudo umount /mnt/proc &&
sudo umount /mnt/dev/pts &&
sudo umount /mnt/dev &&
sudo umount /mnt

# Reboot your PC and unplug the live USB.
```

#### Q &amp; A  


##### 根目錄唯讀，使用了 remount 也沒用

試過了 single-user，emergency，安裝光碟救援的所有模式，始終無法將根目錄掛載成可讀寫，使用的指令是 `mount -o remount,rw /` 。原因可能是根目錄的 filesystem 有區塊損壞，必須先完成修復，才能進行重新掛載。

修復 filesystem

```
fsck -y /dev/your-root-disk
```

# USB 管理

##### 限制用戶可存取 USB 硬碟

修改 /etc/modprobe.d/blacklist.conf

```
# These lines would disable the USB drive.
blacklist usb_storage
blacklist uas
```

重啟系統後，可以驗證設定是否生效，此設定可使以下 USB 裝置失效。

- USB 隨身碟
- USB 光碟機

##### USBGuard  


可管理特定的裝置才能使用 USB 連接

- [How to protect Linux against rogue USB devices using USBGuard](https://www.cyberciti.biz/security/how-to-protect-linux-against-rogue-usb-devices-using-usbguard/)

##### 安全的卸載 USB 碟

```bash
# Umount the USB drive
sudo umount /dev/sdc
# Poer down the USB drive
sudo eject /dev/sdc

# With udisks
# Umount the USB drive
sudo yum install udisks2
sudo udisksctl unmount -b /dev/sdc
sudo udisksctl power-off -b /dev/sdc
```

# 帳號管理

# 登入失敗後鎖定帳號

#### RedHat 8

NOTE:

- RH8 新增一個 faillock 設定檔在 /etc/security/faillock.conf。  
    如果在 system-auth 內有指定參數，會忽略 faillock.conf 相同參數的設定。
- unlock\_time - 帳號鎖定後，經過多久時間會自動解鎖。
- deny - 密碼錯誤次數。

新增目錄 faillock (optional)

> TIP: 如果不指定目錄，預設目錄是 /var/run/faillock。

```
mkdir /var/log/faillock
```

Edit /etc/pam.d/system-auth , /etc/pam.d/password-auth

```
# for auth<br></br># faillock, add the below line BEFORE pam_unix.so<br></br>auth required pam_faillock.so preauth dir=/var/log/faillock silent audit deny=3 fail_interval=900 unlock_time=600<br></br>auth required pam_faillock.so authfail dir=/var/log/faillock unlock_time=600<br></br>#<br></br><br></br>auth        sufficient    pam_unix.so try_first_pass nullok<br></br><br></br># faillock, add the below line AFTER pam_unix.so<br></br>auth        [default=die] pam_faillock.so authfail deny=3 fail_interval=900 unlock_time=600<br></br>#<br></br><br></br># for account<br></br># faillock, add the below line BEFORE pam_unix.so<br></br>account required pam_faillock.so<br></br>#<br></br><br></br>account     required      pam_unix.so<br></br>
```

#### RedHat 6

Edit /etc/pam.d/system-auth , /etc/pam.d/password-auth

```
# for auth<br></br># add the below line BEFORE pam_unix.so<br></br>auth required pam_faillock.so preauth silent audit deny=3 unlock_time=600  # insert this<br></br><br></br>auth        sufficient    pam_unix.so nullok try_first_pass<br></br><br></br># add the below line AFTER pam_unix.so<br></br>auth [default=die] pam_faillock.so authfail audit deny=3 unlock_time=600  # insert this<br></br><br></br># for account<br></br># add the below line BEFORE pam_unix.so<br></br>account required pam_faillock.so  # insert this<br></br><br></br>account     required      pam_unix.so
```

預設不會套用在 root；如果需要限制 root，下面這一行加上 `even_deny_root` ：

```
auth    required    pam_faillock.so preauth silent audit deny=3 even_deny_root unlock_time=1200 root_unlock_time=600
```

如果要排除特定 user，在第一個 `pam_faillock.so` 之前加上這行:

```
auth [success=1 default=ignore] pam_succeed_if.so user in user1:user2:user3
```

如何手動解鎖與檢查被鎖定的帳戶

```
# display the authentication failure for all users <br></br>faillock<br></br><br></br># display the authentication failure for the specified user<br></br>faillock --user mytest<br></br><br></br># unlock the user<br></br>faillock --user mytest --reset
```

> Tip:
> 
> 要確認設定是否有作用，可以監看 log 檔 /var/log/secure，登入錯誤次數達到設定值時，應該要出現下面的訊息。
> 
> Mar 8 15:26:08 centos7 sshd\[26995\]: pam\_faillock(sshd:auth): Consecutive login failures for user i04181 account temporarily locked

#### VSFTPD

如果 vsftpd 使用系統帳號做認證時，也適用帳號鎖定的規則。

#### 參考教學

- \[RH\] [What is pam\_faillock and how to use it in Red Hat Enterprise Linux?](https://access.redhat.com/solutions/62949)
- \[RH\] [Lock account after 3 failed attempts.](https://access.redhat.com/discussions/1404353)
- [Linux 封鎖、解鎖登入失敗次數過多的帳號 pam\_faillock 教學與範例](https://officeguide.cc/linux-pam-tally2-lock-user-accounts-after-failed-login-attempts-tutorial-examples/)

# 進階管理技巧

##### 建立系統用帳號

CentOS/RedHat)

```shell
groupadd -r asterisk
useradd -r -g asterisk -d /var/lib/asterisk -M asterisk
```

Ubuntu/Debian)

```shell
addgroup --system asterisk
adduser --system --ingroup asterisk --home /var/lib/asterisk --no-create-home --shell /bin/bash asterisk
```

##### 移除帳號

```bash
# RedHat
userdel -r <username>

# Debian
deluser <username> --remove-home
```

##### 變更帳號為管理者權限

```shell
# Debian/Ubuntu
# Add the user into the group sudo
sudo usermod -aG sudo <user-name>
# Verify the user's groups
groups <user-name>
```

##### 強制修改密碼

強迫使用者在第一次登入後，修改他們的登入密碼

```shell
# 先將帳號鎖定
usermod -L <username>

# 強制第一次登入必須修改密碼
# 套用後，原密碼會立即過期，直到完成密碼變更。
chage -d 0 <username>

# 解除帳號鎖定
usermod -U <username>

# 檢查帳號的期限
chage -l <user-name>
```

##### 帳號使用期限

```shell
# 檢查帳號期限
chage -l <user-name>

# 設定有效期限 
chage -M 10 <user-name>             # 10 天後密碼即失效
chage -E "2017-02-20" <user-name>   # 2017-02-20 以後帳號即鎖定
chage -I 10 <user-name>             # 如有設定密碼期限時，當密碼失效起 10 日後自動鎖定帳號 

# 解除期限
chage -E -1 <user-name>       ; 數字 -1 解除期限設定 
```

##### 帳號鎖定與解鎖

```shell
# 鎖定帳號
usermod -L <user-name>
passwd -l <user-name>
chage -E 0 <user-name>

# 解鎖帳號
usermod -U <user-name>
passwd -u <user-name>
chage -E <user-name>

# 檢查帳號鎖定狀態
grep <user-name> /etc/shadow

dbtest:!$6$hFCW6eI1$kI9J9QrxCjnpvzFPJnxSpNvQ...  密碼欄有 ! 符號表示鎖定

# List the locked and passwordless accounts
getent shadow | awk '/^.*:[!\*].*/' | cut -d: -f1
```

> 注意：Passwd 雖然可以鎖定帳號，但仍可以用 SSH-Key 登入。

> 注意：帳號鎖定與帳號到期後的鎖定，後者會影響 CronJob 運作。

登入失敗的自動鎖定：

RHEL 7

```bash
# To enable the faillock
# unlock_time: seconds
authconfig --enablefaillock --faillockargs="deny=5 unlock_time=1200" --update

# To disable the faillock
authconfig --disablefaillock --update

# Validate the configuration
authconfig --test

# Check the login attempts failed
faillock

# Unlock the user locked immediately
faillock --user test01 --reset
```

##### 修改既有帳號的設定

```shell
# 修改註解
usermod -c "John" john
# 修改 shell
usermod -s "/sbin/nologin" alang
# 修改帳號名稱
usermod -l newuser currentuser
```

##### 限制某帳號不可遠端登入

但可以由其他允許帳號從遠端登入後，執行 su 切換到該帳號

情境：限制 devrpt 可以從遠端登入，但其他帳號在登入後可以 su 到 devrpt。

方法一: 修改 sshd\_config

```
# Added by Alang
# prevent certain users from using ssh for login
# while retaining the option to 'su username'
#
DenyUsers istdc
```

方法二: 最快速且容易設定但不適用需要有密碼的帳號

```shell
# 刪除 devrpt 的密碼
passwd -d devrpt
```

方法三: 比較嚴謹的做法

以 CentOS 為例：

1\. 編輯 /etc/security/access.conf，加上這幾行

```
# The line 'cron crond' is required
+:devrpt:cron crond tty1 tty2 tty3 tty4 tty5 tty6
-:devrpt:ALL
```

> TIPs：  
> 內容格式為 permission : username: origins
> 
> permission + 允許 或 - 拒絕  
> username 帳號  
> origins 來源，這可以是 tty 名稱'、主機/網域名稱、IP 。 注意：在此例，必須加上 cron crond 這一行，否則該帳號的 crontab 會無法工作。

2\. 對於不同的登入服務，需要修改相應的安全設定檔

- telnet : /etc/pam.d/remote (修改後立即生效)
- SSH : /etc/pam.d/sshd (修改後需重新載入 SSHD)
- Local 本機登入 : /etc/pam.d/login

視需要將以下內容加入其中一項或多項檔案內

```
# Limited users for remote login via telnet
# Check the file /etc/security/access.conf
account    required     pam_access.so
```

##### 重建帳號的家目錄

```
mkhomedir_helper <username>
```

##### 限制登入後的行為

情境: 帳號執行遠端登入後，只能變更密碼與幾個受限制的指令權限

RedHat-KB: [https://access.redhat.com/solutions/65822](https://access.redhat.com/solutions/65822)

```shell
# Create the restricted shell
cp /bin/bash /bin/rbash

# Create a directory that is used as the HOME of the user
mkdir /home/dbuser/
mkdir /home/dbuser/bin

# Modify the target user 'siview' for the shell as restricted shell
usermod -d /home/dbuser -s /bin/rbash siview
# or for new user
useradd -d /home/dbuser -s /bin/rbash siview
```

If a user uses **rbash**, the user can not do the following after login:

- Changing directories with the |cd| built in.
- Setting or unsetting the values of the |SHELL|, |PATH|, |ENV|, or |BASH\_ENV| variables.
- Specifying command names containing slashes.
- Specifying a filename containing a slash as an argument to the |.| built in command.
- Importing function definitions from the shell environment at startup.
- Parsing the value of |SHELLOPTS| from the shell environment at startup.
- Redirecting output using the `|&gt;|', `|&gt;||', `|&lt;&gt;|', `|&gt;&amp;|', `|&amp;&gt;|', and `|&gt;&gt;|' redirection operators.
- Using the |exec| built in to replace the shell with another command.
- Adding or deleting built in commands with the `|-f|' and `|-d|' options to the |enable| built in.
- Specifying the `|-p|' option to the |command| built in.
- Turning off restricted mode with `|set +r|' or `|set +o restricted|'.

```shell
# Create specific profile for the user
vi /home/dbuser/.bash_profile
```

.bash\_profile:

```shell
# cat /home/localuser/.bash_profile  
# .bash_profile  

# Get the aliases and functions  
if [ -f ~/.bashrc ]; then  
. ~/.bashrc  
fi  
# User specific environment and startup programs  
PATH=$HOME/bin  
export PATH
```

```shell
# Create the softlinks of commands which are required for the user
ln -s /bin/date /home/dbuser/bin/
ln -s /bin/ls /home/dbuser/bin/
ln -s /usr/bin/passwd /home/dbuser/bin/
```

##### 密碼強度

- RH-KB: [https://access.redhat.com/solutions/66322](https://access.redhat.com/solutions/66322) (RHEL6)
- RH-KB: [Set a password policy in Red Hat Enterprise Linux 7](https://access.redhat.com/solutions/2808101) (RHEL7)
- [How to Set password policy in CentOS or RHEL system](http://cjcheema.com/2019/05/how-to-set-password-policy-in-centos-or-rhel-system/)
- RedHat/CentOS: `/usr/share/doc/pam-<version>/txts/README.pam_cracklib`
- \[中文\] [https://www.lijyyh.com/2012/07/pam-managing-account-security-with-pam.html](https://www.lijyyh.com/2012/07/pam-managing-account-security-with-pam.html)

預設強度:

- difok=N , 預設字元數 5 位數
- minlen=N, 最少字元位數，預設是 9。
- dcredit=-1, 數字至少 1 位數
- ucredit=-1, 大寫字母至少 1 位數
- lcredit=-1, 小寫字母至少 1 位數

Edit `/etc/pam.d/system-auth` , `/etc/pam.d/password-auth`

CentOS 5/6)

> NOTE: CentOS 5 沒有 `/etc/pam.d/password-auth` , 所以只需要設定 `/etc/pam.d/system-auth`

```
# Set password strength
#password    requisite     pam_cracklib.so try_first_pass retry=3 type=
password    requisite     pam_cracklib.so minlen=8 dcredit=-1 ucredit=-1 lcredit=-1
```

CentOS 7/8)

Edit `/etc/security/pwquality.conf`

```
# Set password strength
minlen = 8
dcredit = -1
ucredit = -1
lcredit = -1
```

預設 root 不會套用密碼強度規則，如果要做限制，編輯 `/etc/pam.d/system-auth` 與 `/etc/pam.d/password-auth` ，在 password 這一行加上 `enforce_for_root`。

```
# Enforce root for password strength
password    requisite     pam_pwquality.so try_first_pass local_users_only retry=3 authtok_type= enforce_for_root
```

記住幾代密碼

Edit `/etc/pam.d/system-auth` , `/etc/pam.d/password-auth`  
CentOS 5/6)

```
# Keep history of passwords used
# Add remember=N
# The last n passwords for each user are saved in /etc/security/opasswd in order to force password change history 
# and keep the user from alternating between the same password too frequently.
#password    sufficient    pam_unix.so sha512 shadow nullok try_first_pass use_authtok
password    sufficient    pam_unix.so sha512 remember=8 shadow nullok try_first_pass use_authtok
```

CentOS 7/8)

```
# Keep history of passwords used, insert the below line after pam_pwquality.so line
password    requisite     pam_pwhistory.so remember=8 use_authtok
```

> TIP: 歷史密碼會被儲存在 `/etc/security/opasswd` .

##### 密碼算法

檢查目前系統帳號的密碼演算法

```bash
# 方法 1
grep ENCRYPT_METHOD /etc/login.defs

# 方法 2
grep pam_unix.so /etc/pam.d/password-auth

# 方法 3
passwd -S <user-name>
```

列出有支援的密碼算法

```bash
authconfig --help | grep passalgo
```

變更密碼算法

> The below command modifies /etc/pam.d/system-auth, /etc/login.defs, /etc/libuser.conf, and /etc/sysconfig/authconfig and thus will have no effect (good or bad) on the password hashes already stored in /etc/passwd or /etc/shadow; instead, only future executions of passwd will use the new algo

```bash
authconfig --passalgo=sha512 --update
```

Optional: 重設 non-root 所有帳號的密碼為 &lt;user-name&gt;

```bash
for u in $(awk -F: '{if ( $1 != "root" && $2 ~ /^!?[[:alnum:]\.\/\$]/ ) print $1}' /etc/shadow); do passwd --stdin $u <<<$u; done
```

如果要強制第一次登入時需要變更密碼，繼續執行下個指令

```bash
for u in $(awk -F: '{if ( $1 != "root" && $2 ~ /^!?[[:alnum:]\.\/\$]/ ) print $1}' /etc/shadow); do chage -d0 $u; done
```

##### 群組管理

```shell
# Create a new group
groupadd <group-name>
addgroup <group-name>

# add a group into an account
usermod -aG mygroup user1
useradd -aG family,friends james

# To change the primary group of the user tom to family
usermod -g family tom

# remove user from a group
gpasswd -d user1 mygroup

# list all users in a group
lid -g mygroup

# list groups for current user
groups

# List groups for specified user
groups username
```

##### 指令 passwd

```shell
# displays the status of user account password settings
# [Username] [Status] [Date Last Changed] [Min. Age] [Max. Age] [Warn. Period] [ Inactivity Period]
# Status: 
#  - P: Usable password
#  - NP: No password
#  - L: Locked password
# Age: 
#  - 99999: Never expires
#  - 0: Can be changed at anytime
#  - -1: Disabled
passwd -S evans
 evans PS 2020-09-07 0 99999 7 -1 (Password set, SHA512 crypt.)

# lock the password of a specified account
passwd -l user1

# unlock the password
passwd -u user2

# delete a password for an account
passwd -d user1

# expire a password for an account
# This will force user to change the password at next login.
passwd -e user2

# This sets the number of days before a password can be changed. 
# By default, a value of zero is set, which indicates that the user may change 
# their password at any time.
# This means user2 cannot change its own password until 10 days have passed.
passwd -n 10 user2

# To confirm the password setting made with the -n option above, run the following command:
# The value of 10 after the date indicates the minimum number of days 
# until the password can be changed.
passwd -S user1
user1 PS 2020-12-04 10 99999 7 -1 (Password set, SHA512 crypt.)

# This means after 90 days, the password is required to be changed.
passwd -x 90 user2

# This means the user will receive warnings that the password will expire 7 days 
# before the expiration.
passwd -w 7 user2

# This means after a user account has had an expired password for 5 days, 
# the user may no longer sign on to the account.
passwd -i 5 user2

# This command will read from the echo command and pass it to the passwd command. 
# So this will set the user1 password to userpasswd1.
echo "userpasswd1"|passwd --stdin user1
```

##### 指令 getent

```bash
# List all user
getent passwd
getent passwd | awk -F: '{print $1}'

# List a specified user
getent passwd <username>

# List the locked and no-login accounts
getent shadow | awk '/^.*:[!\*].*/' | cut -d: -f1

# List the users with uid between 1000 ~ 1500
getent passwd {1000..1500}

```

##### 批次建立多個帳號

```shell
# Step 1 – Create an encrypted password
## perl one liner ##
#perl -e 'print crypt("Your-Clear-Text-Password-Here", "salt"),"\n"'

password="1YelloDog@"
pass=$(perl -e 'print crypt($ARGV[0], "password")' $password)
echo "$pass"
```

```shell
# Step 2 – Shell script to add a user and password on Linux
#!/bin/bash
# Purpose - Script to add a user to Linux system including passsword
# Author - Vivek Gite <www.cyberciti.biz> under GPL v2.0+
# ------------------------------------------------------------------
# Am i Root user?
if [ $(id -u) -eq 0 ]; then
	read -p "Enter username : " username
	read -s -p "Enter password : " password
	egrep "^$username" /etc/passwd >/dev/null
	if [ $? -eq 0 ]; then
		echo "$username exists!"
		exit 1
	else
		pass=$(perl -e 'print crypt($ARGV[0], "password")' $password)
		useradd -m -p "$pass" "$username"
		[ $? -eq 0 ] && echo "User has been added to system!" || echo "Failed to add a user!"
	fi
else
	echo "Only root may add a user to the system."
	exit 2
fi
```

```shell
# Step 3 – Change existing Linux user’s password in one CLI
echo "vivek:password" | chpasswd

# Verify that password has been changed
chage -l vivek
```

```shell
# Step 4 – Create Users and change passwords with passwd on a CentOS/RHEL
echo "YourPassword" | passwd --stdin UserName
```

##### 系統帳號與密碼遷移

在來源主機執行

```bash
# 範例一: 遷移 uid=500 以上的所有帳號
ID_minimum=500
for f in /etc/{passwd,group}; do awk -F: -vID=$ID_minimum '$3>=ID && $1!="nfsnobody"' $f |sort -nt: -k3 > ${f#/etc/}.bak; done
while read line; do grep -w "^${line%%:*}" /etc/shadow; done <passwd.bak >shadow.bak
while read line; do grep -w "^${line%%:*}" /etc/gshadow; done <group.bak >gshadow.bak

# 範例二: 遷移 uid=501 以上的所有帳號
export UGIDLIMIT=501
awk -v LIMIT=$UGIDLIMIT -F: '($3>=LIMIT) && ($3!=65534)' /etc/passwd | sed '/nfsnobody/d' > passwd.move
awk -v LIMIT=$UGIDLIMIT -F: '($3>=LIMIT) && ($3!=65534)' /etc/group | sed '/nfsnobody/d' > group.move
awk -v LIMIT=$UGIDLIMIT -F: '($3>=LIMIT) && ($3!=65534) {print $1}' /etc/passwd | egrep -wf - /etc/shadow | sed '/nfsnobody/d' > shadow.move

# 範例三: 遷移 uid=501 ~ 600 的帳號
export UGID_DOWN=501
export UGID_UP=600
awk -v LIMIT_DOWN=$UGID_DOWN -v LIMIT_UP=$UGID_UP -F: '($3>=LIMIT_DOWN) && ($3<=LIMIT_UP) && ($3!=65534)' /etc/passwd | sed '/nfsnobody/d' > passwd.move
awk -v LIMIT_DOWN=$UGID_DOWN -v LIMIT_UP=$UGID_UP -F: '($3>=LIMIT_DOWN) && ($3<=LIMIT_UP) && ($3!=65534)' /etc/group | sed '/nfsnobody/d' > group.move
awk -v LIMIT_DOWN=$UGID_DOWN -v LIMIT_UP=$UGID_UP -F: '($3>=LIMIT_DOWN) && ($3<=LIMIT_UP) && ($3!=65534) {print $1}' /etc/passwd | egrep -wf - /etc/shadow | sed '/nfsnobody/d' > shadow.move

# 範例四: uid= 501 ~ 699 and 1000+
export UGIDLIMIT_LOW=501
export UGIDLIMIT_HIGH=699
export UGIDS_RHEL7=1000

awk -v RHEL7=$UGIDS_RHEL7 -v LIMIT_LOW=$UGIDLIMIT_LOW -v LIMIT_HIGH=$UGIDLIMIT_HIGH -F: '($3>=RHEL7) || (($3>=LIMIT_LOW) && ($3<=LIMIT_HIGH) && ($3!=65534))' /etc/passwd | sed '/nfsnobody/d' > passwd.move

awk -v RHEL7=$UGIDS_RHEL7 -v LIMIT_LOW=$UGIDLIMIT_LOW -v LIMIT_HIGH=$UGIDLIMIT_HIGH -F: '($3>=RHEL7) || (($3>=LIMIT_LOW) && ($3<=LIMIT_HIGH) && ($3!=65534))' /etc/group | sed '/nfsnobody/d' > group.move

awk -v RHEL7=$UGIDS_RHEL7 -v LIMIT_LOW=$UGIDLIMIT_LOW -v LIMIT_HIGH=$UGIDLIMIT_HIGH -F: '($3>=RHEL7) || (($3>=LIMIT_LOW) && ($3<=LIMIT_HIGH) && ($3!=65534)) {print $1}' /etc/passwd | egrep -f - /etc/shadow | sed '/nfsnobody/d' > shadow.move
```

> NOTE: 如果系統有設定群組密碼，還要加上檔案 `/etc/gshadow` 的遷移。

將以上的檔案 \*.move 複製到目的主機，然後執行

```bash
cat passwd.move >> /etc/passwd
cat shadow.move >> /etc/shadow
cat group.move >> /etc/group

pwconv
grpconv

# 帳號如果需要建立 home 目錄，可以執行
mkhomedir_helper <user-name>
```

Optional: 清除之前匯入的帳密

> NOTE: 清除帳密時，只需要編輯 `/etc/passwd` 與 `/etc/group`，然後執行 `pwconv` 與 `grpconv`，就可以自動更新 `/etc/shadow` 與 `/etc/gshadow`。這方法不適用在匯入帳密時。

```bash
# 清除之前匯入的帳密
## 修改 /etc/passwd
vipw

## 修改 /etc/group
vigr

## 更新 /etc/shadow, /etc/gshadow
pwconv
grpconv
```

Optional: 批次建立 Home 目錄

```bash
for uidgid in $(cut -d: -f3,4 passwd.move); do
    dir=$(awk -F: /$uidgid/{print\$6} passwd.move)
    mkdir -vm700 "$dir"; cp -r /etc/skel/.[[:alpha:]]* "$dir"
    chown -R $uidgid "$dir"; ls -ld "$dir"
done
```

##### 帳號活動監控 psacct  


```bash
yum install psacct
```

- [How to Monitor Linux Users Activity with psacct or acct Tools](https://www.tecmint.com/monitor-linux-user-activity-psacct-acct/)
- Display total statistics of connect time in hours
- Print All Linux Commands Executed by Users
- Print Linux User Information
- Print Number of Linux Processes
- Print and Sort Usage by Percentage
- Search Logs for Commands

##### 遠端連線自動登出 (TMOUT)

Linux: `/etc/profile.d/timeout.sh`

```bash
#!/bin/bash
# Set the TMOUT 600 for specified group
grpname="sshusers"
#if [[ "`id -Gn`" =~ .*"$grpname".* ]]; then
if grep -q "$grpname" <<< "`id -Gn`"; then
    export TMOUT=600
fi

```

Multi groups

```bash
#!/bin/bash
# Set the TMOUT 600 for specified groups
#grpnames="(group1|group2|group3)"
grpnames="(sshusers)"
if echo "`id -Gn`" | grep -wEq "$grpnames"; then
    export TMOUT=600
fi
```

AIX: `/etc/profile`

```bash
# Set the TMOUT 600 for specified groups
#grpnames="(group1|group2|group3)"
grpnames="(sshusers)"
if echo "`id -Gn`" | grep -wEq "$grpnames"; then
    export TMOUT=600
fi
```

##### Learning

- [How to Lock User Accounts After Failed Login Attempts](https://www.tecmint.com/lock-user-accounts-after-failed-login-attempts-in-linux/)
- [Restrict SSH User Access to Certain Directory Using Chrooted Jail](https://www.tecmint.com/restrict-ssh-user-to-directory-using-chrooted-jail/)
- [How can I restrict the normal user to run only limited set of commands in RHEL?](https://access.redhat.com/solutions/65822)
- [How To Limit User’s Access To The Linux System](https://www.ostechnix.com/how-to-limit-users-access-to-the-linux-system/)
- [Set a password policy in Red Hat Enterprise Linux](https://access.redhat.com/solutions/66322)
- \[RedHat\] [How to enhance Linux user security with Pluggable Authentication Module settings](https://www.redhat.com/sysadmin/linux-security-pam)
- [Linux PAM for Compliance](https://cromwell-intl.com/open-source/linux-pam-compliance/)
- [12 Ways to Find User Account Info and Login Details in Linux](https://www.tecmint.com/check-user-in-linux/)

# Windows AD 認證

登入 RedHat 系統時，可使用 Windows AD 帳號。

#### RedHat 7/8 (不加入網域)

<p class="callout info">這個方式需要先建立相同名稱的本機帳號，通常這個會違反資安規範。</p>

- [Chapter 7. Configuring a RHEL host to use AD as an authentication provider Red Hat Enterprise Linux 8 | Red Hat Customer Portal](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_authentication_and_authorization_in_rhel/configuring-a-rhel-host-to-use-ad-as-an-authentication-provider_configuring-authentication-and-authorization-in-rhel)

安裝需要的套件

```bash
yum install sssd sssd-tools krb5-workstation krb5-libs
```

新增本地帳號與 AD 帳號同名

```bash
useradd AD_user
```

編輯 `/etc/nsswitch.conf`

```
# Add 'sss' for AD authentication
passwd:     files sss systemd
shadow:     files sss
group:      files sss systemd
```

編輯 `/etc/krb5.conf`

```
[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 dns_lookup_realm = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 rdns = false
 pkinit_anchors = FILE:/etc/pki/tls/certs/ca-bundle.crt
# Change this as required
 default_realm = EXAMPLE.COM
 default_ccache_name = KEYRING:persistent:%{uid}

[realms]
# Change this as required
EXAMPLE.COM = {
    kdc = ad.example.com
    dmin_server = ad.example.com
}

[domain_realm]
# Change this as required
.example.com = EXAMPLE.COM
example.com = EXAMPLE.COM
```

新增 `/etc/sssd/sssd.conf`

```
[sssd]
    services = nss, pam
    domains = EXAMPLE.COM

[domain/EXAMPLE.COM]
    id_provider = files
    auth_provider = krb5
    krb5_realm = EXAMPLE.COM
    krb5_server = ad.example.com
```

設定檔權限

```bash
chmod 0600 /etc/sssd/sssd.conf
```

啟動 sssd 服務

```bash
systemctl start sssd
systemctl enable sssd
```

編輯 `/etc/pam.d/system-auth`

```
#%PAM-1.0
# This file is auto-generated.
# User changes will be destroyed the next time authconfig is run.
auth        required      pam_env.so
auth        required      pam_faildelay.so delay=2000000
auth        sufficient    pam_unix.so nullok try_first_pass
auth        requisite     pam_succeed_if.so uid >= 1000 quiet_success
# AD Authentication
auth        sufficient    pam_sss.so forward_pass

auth        required      pam_deny.so


account     required      pam_unix.so
account     sufficient    pam_localuser.so
account     sufficient    pam_succeed_if.so uid < 1000 quiet
# AD Authentication
account     [default=bad success=ok user_unknown=ignore] pam_sss.so

account     required      pam_permit.so


password    requisite     pam_pwquality.so try_first_pass local_users_only retry=3 authtok_type$
password    sufficient    pam_unix.so sha512 shadow nullok try_first_pass use_authtok
# AD Authentication
password    sufficient    pam_sss.so use_authtok

password    required      pam_deny.so


session     optional      pam_keyinit.so revoke
session     required      pam_limits.so
-session     optional      pam_systemd.so
session     [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
session     required      pam_unix.so
# AD Authentication
session     optional      pam_sss.so
```

編輯 `/etc/pam.d/password-auth` ，內容與上述的一樣。

##### 驗證AD登入

本機驗證

```
#> kinit AD_user
Password for AD_user@EXAMPLE.COM:

#> klist
Ticket cache: KEYRING:persistent:0:0
Default principal: AD_user@EXAMPLE.COM

Valid starting     Expires            Service principal
11/02/20 04:16:38  11/02/20 14:16:38  krbtgt/EXAMPLE.COM@EXAMPLE.COM
	renew until 18/02/20 04:16:34
```

遠端 SSH 驗證

遠端使用 AD\_user ( 不需加 `@example.com` )登入 SSH。

#### 其他指令

Displaying user authorization details

```bash
sssctl user-checks -a acct -s sshd AD_user
```

Display a list of available domains

```bash
sssctl domain-list
```

#### RedHat 7/8 (加入網域)

- [How to join a Linux system to an Active Directory domain | Enable Sysadmin (redhat.com)](https://www.redhat.com/sysadmin/linux-active-directory)
- [Windows Integration Guide Red Hat Enterprise Linux 7 | Red Hat Customer Portal](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html-single/windows_integration_guide/index)
- [How to join a Linux system to an Active Directory domain](https://www.redhat.com/en/blog/linux-active-directory)

安裝需要套件

```bash
yum install sssd realmd oddjob oddjob-mkhomedir adcli \
  samba-common samba-common-tools krb5-workstation \
  openldap-clients policycoreutils-python
```

使用 realmd 將 Linux 主機加入 AD 網域

> NOTE: 建議先將 /etc/krb5.conf 恢復成初始值，如果曾經修改過。還有將 /etc/sssd/sssd.conf 移除。
> 
> 加入 AD 網域時，需要有 AD 管理員或有足夠權限的 AD 帳號 例如 adm1。
> 
> 一旦加入網域成功，系統會自動修改或建立這兩個檔案。

```bash
realm discover ad.example.com

realm join ad.example.com -U adm1

realm list
```

自動生成 `/etc/sssd/sssd.conf`, `/etc/krb5.conf`

```
[sssd]
domains = example.com
config_file_version = 2
services = nss, pam

[domain/example.com]
ad_server = ad.example.com
ad_domain = example.com
krb5_realm = EXAMPLE.COM
realmd_tags = manages-system joined-with-adcli
cache_credentials = True
id_provider = ad
krb5_store_password_if_offline = True
default_shell = /bin/bash
ldap_id_mapping = True
use_fully_qualified_names = True
fallback_homedir = /home/%u@%d
access_provider = ad
```

```
[libdefaults]
 dns_lookup_realm = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 rdns = false
 pkinit_anchors = FILE:/etc/pki/tls/certs/ca-bundle.crt
 default_ccache_name = KEYRING:persistent:%{uid}

 default_realm = EXAMPLE.COM
[realms]
 EXAMPLE.COM = {
 }

[domain_realm]
 example.com = EXAMPLE.COM
 .example.com = EXAMPLE.COM
```

Optional: 主機退出 AD 網域

```bash
# 預設需要 AD 的 Administrator 密碼
realm leave ad.example.com

# 或者使用指定的帳密
realm leave ad.example.com -U 'EXAMPLE.COM\user'
```

登入存取控制

<p class="callout info">預設，網域所有帳號都可以登入主機；要限制可以存取主機的 AD 帳號或 AD 群組，需要修改 /etc/sssd/sssd.conf。</p>

編輯 /etc/sssd/sssd.conf

```
# ACL for AD Login
#access_provider = ad
access_provider = simple
#simple_allow_users = ad-user1, ad-user2
simple_allow_groups = ad-group
```

重啟 sssd 服務

```bash
systemctl restart sssd
realm list
```

##### 帳號管理

加入 ad-user 至本機群組

```bash
usermod -aG local-group aduser@ad.domain.com
getent group local-group
groups aduser@ad.domain.com
```

##### 更多指令

列出 AD 帳號的 uid

```bash
id ADDOMAIN\\aduser@ad.domain.com

getent passwd aduser@ad.domain.com
```

# 網路管理

# Subnet mask v.s. CIDR

```
Netmask                  Netmask (binary)             CIDR     Notes    
 _____________________________________________________________________________ 
255.255.255.255  11111111.11111111.11111111.11111111  /32  Host (single addr) 
255.255.255.254  11111111.11111111.11111111.11111110  /31  Unuseable 
255.255.255.252  11111111.11111111.11111111.11111100  /30    2  useable 
255.255.255.248  11111111.11111111.11111111.11111000  /29    6  useable 
255.255.255.240  11111111.11111111.11111111.11110000  /28   14  useable 
255.255.255.224  11111111.11111111.11111111.11100000  /27   30  useable 
255.255.255.192  11111111.11111111.11111111.11000000  /26   62  useable 
255.255.255.128  11111111.11111111.11111111.10000000  /25  126  useable 
255.255.255.0    11111111.11111111.11111111.00000000  /24 "Class C" 254 useable  
255.255.254.0    11111111.11111111.11111110.00000000  /23    2  Class C's 
255.255.252.0    11111111.11111111.11111100.00000000  /22    4  Class C's 
255.255.248.0    11111111.11111111.11111000.00000000  /21    8  Class C's 
255.255.240.0    11111111.11111111.11110000.00000000  /20   16  Class C's 
255.255.224.0    11111111.11111111.11100000.00000000  /19   32  Class C's 
255.255.192.0    11111111.11111111.11000000.00000000  /18   64  Class C's 
255.255.128.0    11111111.11111111.10000000.00000000  /17  128  Class C's 
255.255.0.0      11111111.11111111.00000000.00000000  /16  "Class B"       
255.254.0.0      11111111.11111110.00000000.00000000  /15    2  Class B's 
255.252.0.0      11111111.11111100.00000000.00000000  /14    4  Class B's 
255.248.0.0      11111111.11111000.00000000.00000000  /13    8  Class B's 
255.240.0.0      11111111.11110000.00000000.00000000  /12   16  Class B's 
255.224.0.0      11111111.11100000.00000000.00000000  /11   32  Class B's 
255.192.0.0      11111111.11000000.00000000.00000000  /10   64  Class B's 
255.128.0.0      11111111.10000000.00000000.00000000  /9   128  Class B's 
255.0.0.0        11111111.00000000.00000000.00000000  /8   "Class A"   
254.0.0.0        11111110.00000000.00000000.00000000  /7 
252.0.0.0        11111100.00000000.00000000.00000000  /6 
248.0.0.0        11111000.00000000.00000000.00000000  /5 
240.0.0.0        11110000.00000000.00000000.00000000  /4 
224.0.0.0        11100000.00000000.00000000.00000000  /3 
192.0.0.0        11000000.00000000.00000000.00000000  /2 
128.0.0.0        10000000.00000000.00000000.00000000  /1 
0.0.0.0          00000000.00000000.00000000.00000000  /0   IP space
```

[![subnet_table.jpg](https://osslab.tw/uploads/images/gallery/2022-09/scaled-1680-/subnet-table.jpg)](https://osslab.tw/uploads/images/gallery/2022-09/subnet-table.jpg)

[![IPv4_Subnet.jpg](https://osslab.tw/uploads/images/gallery/2022-08/scaled-1680-/ipv4-subnet.jpg)](https://osslab.tw/uploads/images/gallery/2022-08/ipv4-subnet.jpg)

# 網路指令與技巧

#### Tutorials

- [Linux See Bandwidth Usage Per Process With Nethogs Tool](https://www.cyberciti.biz/faq/linux-find-out-what-process-is-using-bandwidth/)
- [How to Configure Network Interfaces in Linux](https://www.freecodecamp.org/news/configure-network-interfaces-in-linux/)

#### ip

```shell
# 顯示所有網卡資訊
ip addr
ip a

# 顯示所有網卡的 IP
ip -br -c addr show  # 需要較新版 ip

# 顯示 eth0 網卡資訊
ip a show eth0 

# 開啟/關閉網卡
ip link set eth0 { up | down }

# 顯示所有的網路裝置
ip link show
ip -br -c link show
ip l show

# 設定 IP (非永久)
ip a add 192.168.1.200/255.255.255.0 dev eth0

# 移除 IP (非永久)
ip a del 192.168.1.200/255.255.255.0 dev eth0

# 顯示 default gateway 及路由表
ip route show
ip r show
ip route add 10.10.20.0/24 via 192.168.50.100 dev eth0
ip route del 10.10.20.0/24

# Default gateway
ip route add default via 192.168.50.100

# 網路即時狀態
ip -s link 

# 顯示 ARP 紀錄 (NOTE: 查詢連接網路設備的 MAC address 與 IP 對應表)
ip neigh show
ip n show

# 清除 ARP 清單裡的某個 IP 紀錄
ip -s -s n f <ip-address>

# 線上求助
ip a help
```

在多個網路埠的主機上，如何得知哪些埠有接上網路線

```shell
# 1. 列出所有網路埠
ip link show  如果埠號顯示 DOWN 必須先啟動

# 2. 啟動網路埠 NOTE: 啟動前要注意 IP 是否會衝突
ip link set eth6 up

# 3. 啟動後檢測線路
ethtool eth6 | grep detected
```

Cheat Sheet

[![ip_commnad.jpeg](https://osslab.tw/uploads/images/gallery/2023-08/scaled-1680-/ip-commnad.jpeg)](https://osslab.tw/uploads/images/gallery/2023-08/ip-commnad.jpeg)

#### nmcli

```shell
# List all of ethernet devices
nmcli con show
nmcli con show <conn-name>
nmcli dev status
# see only the active connections
nmcli con show -a

# Restart the network adapter enp0s3
nmcli con down enp0s3 && nmcli con up enp0s3

# Configure the static ip
# The settings persist across reboots because they are stored by NetworkManager
nmcli con mod enp0s3 ipv4.addresses 192.168.20.170/24
nmcli con mod enp0s3 ipv4.gateway 192.168.20.1
nmcli con mod enp0s3 ipv4.method manual
nmcli con mod enp0s3 ipv4.dns "8.8.8.8"

nmcli con down enp0s3
nmcli con up enp0s3 

# make a new ethernet connection with name Myhome1, assigned to device enp0s3
nmcli con add type ethernet con-name Myhome1 ifname enp0s3 ip4 192.168.1.50/24 gw4 192.168.1.1
cat /etc/sysconfig/network-scripts/ifcfg-Myhome1
```

#### GUI to Configure Network

```bash
# For Ubuntu/Debian
sudo apt install network-manager

# Console Command
nmtui
```

#### netplan

Recommended on Ubuntu/Debian

- [A declarative approach to Linux networking with Netplan | Ubuntu](https://ubuntu.com/blog/a-declarative-approach-to-linux-networking-with-netplan)
- [Netplan brings consistent network configuration across Desktop, Server, Cloud and IoT | Ubuntu](https://ubuntu.com//blog/netplan-configuration-across-desktop-server-cloud-and-iot)

`sudo vi /etc/netplan/01-network-manager-all.yaml`

```
network:
  version: 2
  renderer: networkd
  ethernets:
    ens18:
      dhcp4: no
      addresses: 
        - 192.168.1.22/24
      gateway4: 192.168.1.101
      nameservers:
        addresses: [8.8.8.8, 8.8.4.4]
```

Commands

```bash
# Validate Configuration File
sudo netplan try

# Apply the Configuration
sudo netplan apply

# Check the network stack
sudo netplan status

# Optional: Restart the Network Service
sudo systemctl restart systemd-networkd
```

#### ethtool

```
# ethtool ens192
Settings for ens192:
        Supported ports: [ TP ]
        Supported link modes:   1000baseT/Full
                                10000baseT/Full
        Supported pause frame use: No
        Supports auto-negotiation: No
        Supported FEC modes: Not reported
        Advertised link modes:  Not reported
        Advertised pause frame use: No
        Advertised auto-negotiation: No
        Advertised FEC modes: Not reported
        Speed: 10000Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: off
        MDI-X: Unknown
        Supports Wake-on: uag
        Wake-on: d
        Link detected: yes
```

```
# ethtool -i ens192
driver: vmxnet3
version: 1.4.17.0-k-NAPI
firmware-version:
expansion-rom-version:
bus-info: 0000:0b:00.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no
```

```
# ethtool -S ens192
NIC statistics:
     Tx Queue#: 0
       TSO pkts tx: 540499
       TSO bytes tx: 28911908774
       ucast pkts tx: 10060867
       ucast bytes tx: 29602317140
       mcast pkts tx: 0
       mcast bytes tx: 0
       bcast pkts tx: 5655
       bcast bytes tx: 237510
       pkts tx err: 0
       pkts tx discard: 0
       drv dropped tx total: 0
          too many frags: 0
          giant hdr: 0
          hdr err: 0
          tso: 0
       ring full: 0
       pkts linearized: 0
       hdr cloned: 0
       giant hdr: 0
     Tx Queue#: 1
       TSO pkts tx: 317
       TSO bytes tx: 599134
       ucast pkts tx: 1702836
       ucast bytes tx: 101410145
```

#### mii-tool

```shell
# Installation 
sudo apt install net-tools

# CHECK A SINGLE INTERFACE
sudo mii-tool <interface_name>

# SEE DETAILED INFORMATION
sudo mii-tool -v <interface_name>

# SET NETWORK INTERFACE SPEED
sudo mii-tool –force 10baseT-FD <interface_name>

# RESTART AUTO-NEGOTIATION
# Network devices use an auto-negotiation protocol to communicate the technologies they support. 
# It will then select the fastest mutually supported technology. 
# To restart the auto-negotiation of the interface, run the following command.
sudo mii-tool –restart <interface_name>

# CHANGE THE DUPLEX MODE
# For example, here I have set the speed of the eth0 interface to 10 Mbps and the duplex mode to half-duplex.
sudo mii-tool -F 10baseT-HD eth0

# REPORT LINK STATUS CHANGES
# Run the following command to watch a single interface and report changes in the link status. 
# That is to say, the interfaces are listed at one second intervals by default.
sudo mii-tool -w <interface>

# REPORT LINK STATUS
sudo mii-tool -l <interface_name>

# RESET THE CONFIGURATIONS
# Most importantly, you should be able to reset it to its default configuration 
# if something goes wrong. For that, run the following command
sudo mii-tool -R <Interface_name>
```

#### systemctl

```
# Bringing UP/Down Network Interface
systemctl restart network
# or
systemctl restart network.service
```

#### speedtest CLI

```shell
# Ubuntu/Debian
curl -s https://packagecloud.io/install/repositories/ookla/speedtest-cli/script.deb.sh | sudo bash
sudo apt-get install speedtest

# CentOS/RedHat
curl -s https://packagecloud.io/install/repositories/ookla/speedtest-cli/script.rpm.sh | sudo bash
sudo yum install speedtest
```

#### State of Network Cable

```shell
# Device: enp5s0
# Output: 1 means Connected
cat /sys/class/net/enp5s0/carrier
# Output: Up  means Connected
cat /sys/class/net/enp5s0/operstate

# Using ethtool
# Output: Link detected: yes
sudo ethtool enp5s0

# Using ip
# Output: state UP
ip a

```

#### Network Adapters

Modern Linux

```bash
lshw -class network -short
```

Old Linux

```bash
lspci | egrep -i --color 'network|ethernet'
```

#### Disable IPv6

- [Disable IPv6 in Linux: A Step-by-Step Guide (For All Distros) - OSTechNix](https://ostechnix.com/disable-ipv6-in-linux/)

##### Ubuntu 20.04

```
sudo vi /etc/default/grub

# Change the line as follows
GRUB_CMDLINE_LINUX_DEFAULT="ipv6.disable=1"

# Update the GRUB
sudo update-grub

# Reboot
systemctl reboot
```

##### Debian 10/11/12

`/etc/sysctl.d/ipv6.conf` :

```shell
# Disable IPv6 on all network adapters
net.ipv6.conf.all.disable_ipv6 = 1
```

Apply the change :

```shell
# Debian 12+
service procps force-reload

# Older systems
sysctl -p
```

##### RedHat 4

1\. Remove the following line (if present) from the `/etc/modprobe.conf` file:

```
alias net-pf-10 ipv6
```

2\. Add the following line to the `/etc/modprobe.conf` file:

```
alias net-pf-10 off
```

3\. Comment out any IPv6 addresses found in `/etc/hosts`, including ::1 localhost address

```
cp -p /etc/hosts /etc/hosts.disableipv6
sed -i 's/^[[:space:]]*::/#::/' /etc/hosts
```

如果以上步驟仍無法關閉 IPv6，檢查是否有啟動 openibd 服務，將它關閉試試

> **openibd** is a High Availability service for IPoIB (IP over InfiniBand) interface. The service loads the ib\_ipoib module, which has a dependency on the ipv6 module

```bash
service openibd stop
chkconfig openibd off
reboot
```

##### RedHat 5/6

`/etc/sysctl.d/ipv6.conf` :

```
# For v5/6
# IPv6 support in the kernel, set to 0 by default
# Disable IPv6
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
```

##### RedHat 7

`/etc/sysctl.d/ipv6.conf` :

```
# To disable for all interfaces
net.ipv6.conf.all.disable_ipv6 = 1
```

重建開機映像檔

> 如果沒有重建開機映像檔，會使得 rpcbind.service 無法正常運作，這會影響 NFS 的掛載。

##### RedHat 8

Create the file `/etc/sysctl.d/ipv6.conf` :

```
# First, disable for all interfaces
net.ipv6.conf.all.disable_ipv6 = 1
# If using the sysctl method, the protocol must be disabled all specific interfaces as well. 
#net.ipv6.conf.<interface>.disable_ipv6 = 1
```

Reload sysctl :

```
sysctl -p /etc/sysctl.d/ipv6.conf
```

Create a backup of the initramfs :

```
cp /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).bak.$(date +%m-%d-%H%M%S).img
```

Rebuild the Initial RAM Disk Image :

```
dracut -f -v
```

Verifying file inclusion :

```
lsinitrd /boot/initramfs-<version>.img  | grep 'etc/sysctl.d/ipv6.conf'
```

Comment out any IPv6 addresses found in /etc/hosts, including ::1 localhost address

```
cp -p /etc/hosts /etc/hosts.disableipv6
sed -i 's/^[[:space:]]*::/#::/' /etc/hosts
```

#### WiFi Management

- [8 Linux Commands: To Find Out Wireless Network Speed, Signal Strength And Other Information](https://www.cyberciti.biz/tips/linux-find-out-wireless-network-speed-signal-strength.html)

```shell
// Show All SSIDs
nmcli dev wifi

// Get dev name
nmcli conn show

# Replace 'wlan0' with your wifi interface
sudo iwlist wlan0 scan | egrep "Cell|ESSID|Encryption|Quality"
```

#### Block Attackers IP Address

Drop or Block Attackers IP Address With Null Routes On a Linux

```shell
# Using route command
route add 65.21.34.4 gw 127.0.0.1 lo
# veryfy it
netstat -nr
route -n
# Or
route add -host 64.1.2.3 reject
ip route get 64.1.2.3

# Using ip command
ip route add blackhole 202.54.5.2/29
ip route add blackhole 192.0.130.0/24
# verify it
ip route

# Removing null routing
route delete 65.21.34.4
# Or
route del -host 65.21.34.4 reject
# Or
ip route delete 1.2.3.4/26 dev eth0
```

#### 重設/移除不存在的網路裝置名稱  


製作 Linux VM Template 時，每一次修改 Template 後都會產生新的編號作為網路裝置名稱。

然而透過這個 Template 新增的 Linux VM，系統的網路介面其實只有一個，不過裝置名稱可能已經編到 eth1 或 eth2 以後。正常來說，系統如果只有一個網路介面，網路裝置名稱通常為 eth0。

假使想要清除那些已經不存在的裝置名稱，或者讓系統對目前的網路裝置重新以 eth0 開始編號，步驟如下：

RedHat 6.x: 編輯 /etc/udev/rules.d/70-persistent-net.rules

```
# PCI device 0x15ad:0x07b0 (vmxnet3)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:50:56:83:7c:eb", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

# PCI device 0x15ad:0x07b0 (vmxnet3) (custom name provided by external tool)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:50:56:83:7c:eb", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"
```

註解或移除那些舊裝置名稱，只保留目前的裝置 eth1，然後將該行的 NAME 改成 eth0。

```
# PCI device 0x15ad:0x07b0 (vmxnet3) (custom name provided by external tool)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:50:56:83:7c:eb", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
```

存檔後重起 VM。

VM 啟動後，使用 `setup` 或 system-config-network 新增網路介面 eth0 的網路設定。

#### Disable WiFi

With nmcli

```
# nmcli dev status
DEVICE  TYPE      STATE     CONNECTION 
enp2s0  ethernet  已連線    enp2s0     
wlp1s0  wifi      離線      --         
lo      loopback  不受管理  --    

# nmcli radio wifi off

# nmcli dev status
DEVICE  TYPE      STATE     CONNECTION 
enp2s0  ethernet  已連線    enp2s0     
wlp1s0  wifi      無法使用  --         
lo      loopback  不受管理  --       
```

#### 查詢 DNS Server 位址

```bash
cat /etc/resolv.conf
nmcli dev show | grep -i dns
dig <domain-name>
resolvectl status
```

#### Custom MAC Address

##### RedHat 4  


`/etc/sysconfig/network-scripts/ifcfg-eth0`

```
DEVICE=eth0
ONBOOT=yes
BOOTPROTO=static
IPADDR=10.15.9.32
NETMASK=255.255.0.0
GATEWAY=10.15.8.254
#HWADDR=00:0C:29:B1:18:A3
MACADDR=00:0C:B1:B1:B1:B1
```

# Linux Bonding Network

#### RedHat 7/8 with nmtui

nmtui &gt; Edit a connection &gt; Add &gt; New Connection &gt; Select Bond &gt; Create

- Profile Name: bond0
- Device: bond0

&gt; Slaves &gt; Add

- Profile Name: eno1-slave NOTE: 建議這裡要加上 -slave，與實際的裝置名稱做區別
- Device: eno1
- Profile Name: ens3f0-slave
- Device: ens3f3

&gt; IPv4 Configuration

- Address: 10.4.1.71/24
- Gateway: 10.4.1.254
- DNS Servers: 10.3.3.3

&gt; IPv6 Configuration &gt; Disabled

> Profile Name 的名稱會與實際設定檔名 ifcfg-XXX 有關，上述的設定會產生設定檔 ifcfg-bond0 , ifcfg-eno1-slave
> 
> 如果 Profile Name 設定錯了要修改，必須移除整個 Bond 設定，然後再重建；如果直接修改設定，關聯的設定檔名稱 ifcfg-XXX 並不會一起被更新，這會造成爾後管理上的困擾。

變更 Bonding 模式  
預設模式是使用 Load Balancing (Round-Robin)，將它變更為 Acitve-Backup

nmtui &gt; Edit a connection &gt; Bond: bond0 &gt; Edit &gt;

- Mode: Active Backup
- Primary: eno1 註:需要指定其中一張網卡為主要

> NOTE: 如果要使用預設的 Round-Robin 模式，網卡所連結的 Switch 設備必須設定 EtherChannel，否則 Switch 設備會偵測到 vlan XX is flapping between port YYY and port ZZZ。

重啟網路服務

```shell
# 注意:如果有使用 iSCSI Disks,重啟網路服務可能造成系統其他問題
systemctl restart network.service
or
nmcli networking off; modprobe -r bonding ; nmcli networking on
```

#### RedHat 8 with nmcli

網路環境: 三張網路裝置做 bonding

- ens3
- ens4
- ens5

```shell
# 檢視目前網路裝置
nmcli device status

# 新增 team 網路裝置
# 自訂名稱: team0 
nmcli connection add type team con-name team0 ifname team0 ipv4.addresses 192.168.10.20/24 ipv4.gateway 192.168.10.1 ipv4.dns 192.168.10.1 ipv4.method manual  connectio.autoconnect yes config '{"runner" : {"name" : "activebackup"}}'

# 檢查 team 0 狀態
nmcli device status

# 新增 team-slave 網路設備(綁定第一張網卡)
# 自訂名稱: team0-eth0
# master 指定剛剛新增的 team0
nmcli connection add type team-slave con-name team0-eth0 ifname ens3 master team0

# 新增 team-slave 網路設備(綁定第二張網卡)
# 自訂名稱: team0-eth1
# master 指定剛剛新增的 team0
nmcli connection add type team-slave con-name team0-eth1 ifname ens4 master team0

# 新增 team-slave 網路設備(綁定第三張網卡)
# 自訂名稱: team0-eth2
# master 指定剛剛新增的 team0
nmcli connection add type team-slave con-name team0-eth2 ifname ens5 master team0

# 檢查 team 0 狀態
nmcli device status
```

測試網路備援

```shell
# 檢視 team0 狀態
teamdctl team0 state
teamnl team0 options

# 關閉第三張網卡
nmcli connection down team0-eth2

# 檢查狀態
nmcli device status
teamdctl team0 state

# 開啟第三張網卡
nmcli connection up team0-eth2
```

#### LACP Mode

##### RedHat 7/8

nmtui &gt; Edit a connection &gt; Bond: bond0 &gt; Edit &gt;

- Mode: 802.3ad

Verify the state of the network

- Make sure the **Aggregator ID** are the same on the ports with the same Port Channel configured.
- Bonding Mode is 802.3ad.
- Aggreator ID 號碼在主機重啟後會自行改變。

> If you run into the issue with *Multiple LACP bonds have the same Aggregator ID*, check the link, [https://access.redhat.com/solutions/2916431](https://access.redhat.com/solutions/2916431) .

```
[root@tpeitptsm01 ~]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Peer Notification Delay (ms): 0

802.3ad info
LACP rate: slow
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
System MAC address: b4:7a:f1:4c:8b:1c
Active Aggregator Info:
        Aggregator ID: 1           <==== 所有 port 必須是同一個 ID，這也表示是同一個 portchannel.
        Number of ports: 4         <==== 這個 portchannel 有幾個 port
        Actor Key: 9
        Partner Key: 3
        Partner Mac Address: 70:18:a7:dc:ac:80

Slave Interface: eno1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: b4:7a:f1:4c:8b:1c
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: none
Partner Churn State: none  <===
Actor Churned Count: 0
Partner Churned Count: 0   <===
details actor lacp pdu:
    system priority: 65535
    system mac address: b4:7a:f1:4c:8b:1c
    port key: 9
    port priority: 255
    port number: 1
    port state: 61
details partner lacp pdu:
    system priority: 32768
    system mac address: 70:18:a7:dc:ac:80   <===
    oper key: 3
    port priority: 32768
    port number: 263
    port state: 61

Slave Interface: eno2
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: b4:7a:f1:4c:8b:1d
Slave queue ID: 0
Aggregator ID: 1
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
    system priority: 65535
    system mac address: b4:7a:f1:4c:8b:1c
    port key: 9
    port priority: 255
    port number: 2
    port state: 61
details partner lacp pdu:
    system priority: 32768
    system mac address: 70:18:a7:dc:ac:80
    oper key: 3
    port priority: 32768
    port number: 264
    port state: 61


```

#### Configure Switch

[![rhel_network_bonding_switch_config.png](http://www.osslab.tw/uploads/images/gallery/2021-11/scaled-1680-/rhel_network_bonding_switch_config.png)](http://www.osslab.tw/uploads/images/gallery/2021-11/rhel_network_bonding_switch_config.png)

#### Ubuntu with netplan  


- [A Beginner’s Guide to Creating Network Bonding and Bridging in Ubuntu](https://www.tecmint.com/create-network-bond-bridge-in-ubuntu/)

# RedHat 9 網路裝置命名規則

##### Reference

- [Chapter 1. Consistent network interface device naming](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/configuring_and_managing_networking/consistent-network-interface-device-naming_configuring-and-managing-networking)

##### 網路名稱列表

<table class="lt-4-cols lt-7-rows" id="bkmrk-scheme-description-e"><thead><tr><th align="left" data-col="1" data-row="1" id="bkmrk-scheme" scope="col" valign="top">Scheme</th><th align="left" data-col="2" data-row="1" id="bkmrk-description" scope="col" valign="top">Description</th><th align="left" data-col="3" data-row="1" id="bkmrk-example" scope="col" valign="top">Example</th></tr></thead><tbody><tr><td align="left" data-col="1" data-row="2" headers="idm139743499399616" valign="top">1

</td><td align="left" class="content--lg" data-col="2" data-row="2" headers="idm139743499398528" valign="top">Device names incorporate firmware or BIOS-provided index numbers for onboard devices. If this information is not available or applicable, `udev` uses scheme 2.

</td><td align="left" class="content--md" data-col="3" data-row="2" headers="idm139743499397440" valign="top">`eno1`

</td></tr><tr><td align="left" data-col="1" data-row="3" headers="idm139743499399616" valign="top">2

</td><td align="left" class="content--lg" data-col="2" data-row="3" headers="idm139743499398528" valign="top">Device names incorporate firmware or BIOS-provided PCI Express (PCIe) hot plug slot index numbers. If this information is not available or applicable, `udev` uses scheme 3.

</td><td align="left" class="content--md" data-col="3" data-row="3" headers="idm139743499397440" valign="top">`ens1`

</td></tr><tr><td align="left" data-col="1" data-row="4" headers="idm139743499399616" valign="top">3

</td><td align="left" class="content--lg" data-col="2" data-row="4" headers="idm139743499398528" valign="top">Device names incorporate the physical location of the connector of the hardware. If this information is not available or applicable, `udev` uses scheme 5.

</td><td align="left" class="content--md" data-col="3" data-row="4" headers="idm139743499397440" valign="top">`enp2s0`

</td></tr><tr><td align="left" data-col="1" data-row="5" headers="idm139743499399616" valign="top">4

</td><td align="left" class="content--lg" data-col="2" data-row="5" headers="idm139743499398528" valign="top">Device names incorporate the MAC address. Red Hat Enterprise Linux does not use this scheme by default, but administrators can optionally use it.

</td><td align="left" class="content--md" data-col="3" data-row="5" headers="idm139743499397440" valign="top">`enx525400d5e0fb`

</td></tr><tr><td align="left" data-col="1" data-row="6" headers="idm139743499399616" valign="top">5

</td><td align="left" class="content--lg" data-col="2" data-row="6" headers="idm139743499398528" valign="top">The traditional unpredictable kernel naming scheme. If `udev` cannot apply any of the other schemes, the device manager uses this scheme.

</td><td align="left" class="content--md" data-col="3" data-row="6" headers="idm139743499397440" valign="top">`eth0`

</td></tr></tbody></table>

By default, Red Hat Enterprise Linux selects the device name based on the `NamePolicy` setting in the `/usr/lib/systemd/network/99-default.link` file. The order of the values in `NamePolicy` is important. Red Hat Enterprise Linux uses the first device name that is both specified in the file and that `udev` generated.

If you manually configured `udev` rules to change the name of kernel devices, those rules take precedence.

##### Predictable network interface device names on the x86\_64 platform explained

The interface name starts with a two-character prefix based on the type of interface:

- `en` for Ethernet
- `wl` for wireless LAN (WLAN)
- `ww` for wireless wide area network (WWAN)

Additionally, one of the following is appended to one of the above-mentioned prefix based on the schema the `udev` device manager applies:

<div class="itemizedlist" id="bkmrk-o%3Con-board_index_num">- `o<span class="emphasis"><em><on-board_index_number></em></span>`
- `s<span class="emphasis"><em><hot_plug_slot_index_number></em></span>[f<span class="emphasis"><em><function></em></span>][d<span class="emphasis"><em><device_id></em></span>]`
    
    Note that all multi-function PCI devices have the `[f<span class="emphasis"><em><function></em></span>]` number in the device name, including the function `0` device.
- `x<span class="emphasis"><em><MAC_address></em></span>`
- `[P<span class="emphasis"><em><domain_number></em></span>]p<span class="emphasis"><em><bus></em></span>s<span class="emphasis"><em><slot></em></span>[f<span class="emphasis"><em><function></em></span>][d<span class="emphasis"><em><device_id></em></span>]`
    
    The `[P<span class="emphasis"><em><domain_number></em></span>]` part defines the PCI geographical location. This part is only set if the domain number is not `0`.
- `[P<span class="emphasis"><em><domain_number></em></span>]p<span class="emphasis"><em><bus></em></span>s<span class="emphasis"><em><slot></em></span>[f<span class="emphasis"><em><function></em></span>][u<span class="emphasis"><em><usb_port></em></span>][…​][c<span class="emphasis"><em><config></em></span>][i<span class="emphasis"><em><interface></em></span>]`
    
    For USB devices, the full chain of port numbers of hubs is composed. If the name is longer than the maximum (15 characters), the name is not exported. If there are multiple USB devices in the chain, `udev` suppresses the default values for USB configuration descriptors (`c1`) and USB interface descriptors (`i0`).

</div>##### 停用網路裝置連續名稱規則（Not Recommend）

> Warning  
> Red Hat recommends not to disable consistent device naming and does not support this feature on hosts with more than one network interface. Disabling consistent device naming can cause different kind of problems. For example, if you add another network interface card to the system, the assignment of the kernel device names, such as eth0, is no longer fixed. Consequently, after a reboot, the Kernel can name the device differently.

- [Disabling consistent interface device naming during the installation](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/configuring_and_managing_networking/consistent-network-interface-device-naming_configuring-and-managing-networking#disabling-consistent-interface-device-naming-during-the-installation_consistent-network-interface-device-naming)

# FAQ

##### ARP Cache 不會更新 (Send out Gratuitous ARP)

問題說明：兩部相同規格的 Linux 主機，平時互作備援，網路設定各有一個固定 IP 與共用一個 VIP，VIP 使用 Alias IP 方式。每次移動 VIP 至另一部主機時，都會遇到其他鄰近的不同 vLAN 的主機無法 ping 這 VIP，原因是它們的 Switch 設備與 Core Switch 不會立即更新 ARP Cache，直到 在那些 Switch 上手動清除舊的 ARP 紀錄。

解決方案：要讓 Core Switch 立即更新 ARP Cache，可以再切換 VIP 後，從**目的端** Linux 主機上執行任一個指令

```shell
arping -U -c 10 -I eth0:1 your.vip.address
arping -A -c 10 -I eth0:1 your.vip.address

```

其他解決方案：

啟動 VIP 時，使用 ifup 指令可以使主機傳送 Gratuitous ARP Request 更新訊號給 Switch

```shell
ifconfig eth0:1 10.4.1.110/24
ifup eth0:1
```

或者以下動作也可能可以傳送 Gratuitous ARP Request。

- 重啟網路服務
- 重啟主機

參考連結：

- [Gratuitous\_ARP (wireshark.org)](https://wiki.wireshark.org/Gratuitous_ARP.md)
- [Gratuitous ARP – Definition and Use Cases – Practical Networking .net](https://www.practicalnetworking.net/series/arp/gratuitous-arp/)

##### 移除 sit0 網路介面

停用 IPv6 功能後，sit0 就不再出現。

# Diagrams

##### OSI Model

[![osi_layer.jpeg](https://osslab.tw/uploads/images/gallery/2023-12/scaled-1680-/osi-layer.jpeg)](https://osslab.tw/uploads/images/gallery/2023-12/osi-layer.jpeg)

[![osi-layers.jpg](https://osslab.tw/uploads/images/gallery/2023-12/scaled-1680-/osi-layers.jpg)](https://osslab.tw/uploads/images/gallery/2023-12/osi-layers.jpg)

[![osi-model-cheatsheet.jpeg](https://osslab.tw/uploads/images/gallery/2023-12/scaled-1680-/osi-model-cheatsheet.jpeg)](https://osslab.tw/uploads/images/gallery/2023-12/osi-model-cheatsheet.jpeg)

[![osi-applications.jpeg](https://osslab.tw/uploads/images/gallery/2023-12/scaled-1680-/osi-applications.jpeg)](https://osslab.tw/uploads/images/gallery/2023-12/osi-applications.jpeg)

##### Power over Ethernet (PoE)

[![PoE.jpg](https://osslab.tw/uploads/images/gallery/2024-10/scaled-1680-/poe.jpg)](https://osslab.tw/uploads/images/gallery/2024-10/poe.jpg)

##### MAC Address

[![mac_addr.jpg](https://osslab.tw/uploads/images/gallery/2025-04/scaled-1680-/mac-addr.jpg)](https://osslab.tw/uploads/images/gallery/2025-04/mac-addr.jpg)

##### Ethernet Cable Types

[![ethernet_cable_types.jpg](https://osslab.tw/uploads/images/gallery/2025-04/scaled-1680-/ethernet-cable-types.jpg)](https://osslab.tw/uploads/images/gallery/2025-04/ethernet-cable-types.jpg)

# nc - Netcat

Linux 系統中一個多用途的網路工具程式，雖然它只是一個小程式，但是能夠做的事情很多，就像瑞士刀一樣，幾乎任何使用 TCP 或 UDP 封包的動作都可以用它來達成，是許多系統管理者（包含我自己）最喜愛的網路診斷工具之一。

##### 基本指令

```shell
# Scanning the port range (20 - 1024)
nc -z 192.168.21.202 20-1024

Connection to 192.168.21.202 22 port [tcp/ssh] succeeded!
Connection to 192.168.21.202 80 port [tcp/http] succeeded!
Connection to 192.168.21.202 111 port [tcp/sunrpc] succeeded!
Connection to 192.168.21.202 443 port [tcp/https] succeeded!
Connection to 192.168.21.202 514 port [tcp/shell] succeeded!

# Scanning the specified port
nc -zv 192.168.21.202 21
nc: connect to 192.168.21.202 port 21 (tcp) failed: Connection refused

# Port Scanning With netcat including displaying version #
echo "QUIT" | nc 192.168.2.17 22
echo "QUIT" | nc -v 192.168.2.254 ssh
# OR pass the -vv  to get remote OpenSSH version # 
nc -vv 192.168.2.254 ssh
```

##### 檔案傳輸

在不同的 Linux 主機上傳輸檔案

```shell
# Install nc and pv
yum install netcat pv

# Machine A with IP : 192.168.0.4
# Machine B with IP : 192.168.0.7
# On Linux Machine A
# [*] tar -zcf = tar is a tape archive utility used to compress/uncompress archive files 
#     and arguments -c creates a new .tar archive file, -f specify type of the archive file 
#     and -z filter archive through gzip.
# [*] CentOS-7-x86_64-DVD-1503.iso = Specify the file name to send over network, it can be file 
#     or path to a directory.
# [*] pv = Pipe Viewer to monitor progress of data.
# [*] nc -l -p 5555 -q 5 = Networking tool used for send and receive data over tcp 
#     and arguments -l used to listen for an incoming connection, -p 555 specifies the source port 
#     to use and -q 5 waits the number of seconds and then quit.
tar -zcf - CentOS-7-x86_64-DVD-1503.iso | pv | nc -l -p 5555 -q 5

# On Linux Machine B
nc 192.168.1.4 5555 | pv | tar -zxf -
```

複製目錄

```bash
# Receiver on hostB
nc -l 5000 | tar xvf -

# Sender on hostA
tar cvf - /path/to/dir | nc hostB.com 5000
```

Back up host A (/dev/sdb) to host B (sdb-backup.img.gz)

```bash
# On host B
nc -l 5000 | dd of=sdb-backup.img.gz

# On host A
dd if=/dev/sdb | gzip -c | nc hostB.com 5000
```

##### 測試 TCP Port

```bash
nc -v 192.168.0.175 5000
```

##### UDP 封包傳輸

```bash
# 本地主機，傳送字串至遠端主機
echo -n "foo" | nc -u -w1 192.168.1.8 5000

# 遠端主機，開啟 UDP port
nc -lu localhost 5000
```

##### 掃描目的主機網路埠

```bash
# For TCP 
nc -vnz -w 1 192.168.233.208 1-1000 2000-3000

# For UDP
nc -vnzu 192.168.1.8 1-65535
```

##### Cheat Sheets  


[![netcat_commands_s.jpg](https://osslab.tw/uploads/images/gallery/2023-12/scaled-1680-/netcat-commands-s.jpg)](https://osslab.tw/uploads/images/gallery/2023-12/netcat-commands-s.jpg)

![reverse_shell.gif](https://osslab.tw/uploads/images/gallery/2023-12/reverse-shell.gif)

# Linux Backup Tools

#### System Backup

##### Clonezilla

Clonezilla is a partition and disk imaging/cloning program similar to True Image® or Norton Ghost®.

- [Clonezilla - About](https://clonezilla.org/)

##### Rescuezilla

Rescuezilla is an easy-to-use disk cloning and imaging application that's fully compatible with Clonezilla — the industry-standard trusted by tens of millions.

- [https://github.com/rescuezilla/rescuezilla](https://github.com/rescuezilla/rescuezilla)

#### Data Backup

##### Restic

- [Restic](https://ostechnix.com/restic-fast-secure-efficient-backup-application/) - is a fast, open source, secure and cross-platform backup program. It allows saving multiple revisions of files and directories in an encrypted repository stored on different backends. 
    - 備份資料加密
    - 差異備份 - snapshot
    - 支援目錄與檔案
    - 備份媒體支援 本地/SFTP/HTTP REST/AWS S3/Azure Storage/Google Cloud Storage/rclone

##### Rsnapshot

- **Rsnapshot** is a powerful backup utility with installation options on any Linux-based machine. It can either back up a machine locally, or you can back up multiple machines, say servers for instance, from a single machine. **Rsnapshot** uses `rsync` and is written entirely in Perl with no library dependencies. 
    - [Backup Solution - rsnapshot - Documentation (rockylinux.org)](https://docs.rockylinux.org/guides/backup/rsnapshot_backup/)

##### Vykar

Vykar is a fast, encrypted, deduplicated backup tool written in Rust.

- [Home - Vykar Backup Documentation](https://vykar.borgbase.com/)

# ufw - Uncomplicated Firewall

Uncomplicated Firewall，簡稱 UFW，是 Ubuntu 系統上預設的防火牆組件。UFW 是為輕量化組態 iptables 而開發的一款工具。UFW 提供一個非常友好的介面用於建立基於 IPV4，IPV6的防火牆規則。UFW 在 Ubuntu 8.04 LTS 後的所有發行版中預設可用。

##### Tutorials

- [Linux Firewalls: Uncomplicated Firewall (ufw)](https://www.hackers-arise.com/post/linux-firewalls-uncomplicated-firewall-ufw)
- [Using Firewall With UFW in Ubuntu Linux \[Beginner's Guide\] (itsfoss.com)](https://itsfoss.com/ufw-ubuntu/)

##### Basic Commands

```bash
# Install
sudo apt-get install ufw

# Enable the UFW
sudo ufw enable
sudo ufw status

# Allow the services and ports
sudo ufw allow OpenSSH
sudo ufw allow 655
```

##### For Cloudflare Traffic  


Cloudflare [publishes the IP addresses of its servers online](https://www.cloudflare.com/ips/).

```bash
# For HTTP with IPv4 
sudo ufw allow from 173.245.48.0/20 to any port http
sudo ufw allow from 103.21.244.0/22 to any port http
sudo ufw allow from 103.22.200.0/22 to any port http
sudo ufw allow from 103.31.4.0/22 to any port http
sudo ufw allow from 141.101.64.0/18 to any port http
sudo ufw allow from 108.162.192.0/18 to any port http
sudo ufw allow from 190.93.240.0/20 to any port http
sudo ufw allow from 188.114.96.0/20 to any port http
sudo ufw allow from 197.234.240.0/22 to any port http
sudo ufw allow from 198.41.128.0/17 to any port http
sudo ufw allow from 162.158.0.0/15 to any port http
sudo ufw allow from 104.16.0.0/12 to any port http
sudo ufw allow from 172.64.0.0/13 to any port http
sudo ufw allow from 131.0.72.0/22 to any port http

# For HTTP with IPv6
sudo ufw allow from 2400:cb00::/32 to any port http
sudo ufw allow from 2606:4700::/32 to any port http
sudo ufw allow from 2803:f800::/32 to any port http
sudo ufw allow from 2405:b500::/32 to any port http
sudo ufw allow from 2405:8100::/32 to any port http
sudo ufw allow from 2a06:98c0::/29 to any port http
sudo ufw allow from 2c0f:f248::/32 to any port http

# For HTTPS with IPv4
sudo ufw allow from 173.245.48.0/20 to any port https
sudo ufw allow from 103.21.244.0/22 to any port https
sudo ufw allow from 103.22.200.0/22 to any port https
sudo ufw allow from 103.31.4.0/22 to any port https
sudo ufw allow from 141.101.64.0/18 to any port https
sudo ufw allow from 108.162.192.0/18 to any port https
sudo ufw allow from 190.93.240.0/20 to any port https
sudo ufw allow from 188.114.96.0/20 to any port https
sudo ufw allow from 197.234.240.0/22 to any port https
sudo ufw allow from 198.41.128.0/17 to any port https
sudo ufw allow from 162.158.0.0/15 to any port https
sudo ufw allow from 104.16.0.0/12 to any port https
sudo ufw allow from 172.64.0.0/13 to any port https
sudo ufw allow from 131.0.72.0/22 to any port https
```

##### Cheat Sheet

[![ufw_firewall.jpg](https://osslab.tw/uploads/images/gallery/2024-07/scaled-1680-/ufw-firewall.jpg)](https://osslab.tw/uploads/images/gallery/2024-07/ufw-firewall.jpg)

# Makefile

何謂 Makefile

- make 命令雖有很多內建的功能，但它也無法知道如何建立應用程式。故必須提供一個檔案，即 Makefile，告訴 make 如何建立應用程式。
- Makefile 與專案的原始碼檔案，通常放在同一個目錄中。
- 可以同時有很多不同的 makefile 管理專案的不同部分。
- make 命令和 Makefile 的結合，不僅控制原始碼的編譯，也可以用來準備使用手冊文件、安裝應用程式到目的目錄中。

使用 Makefile 的好處

- 如果這個專案沒有編譯過，那麼我們的所有程式碼都要編譯並被連結。
- 如果這個專案的某幾份程式碼被修改，那麼我們只編譯被修改的程式，並連結目標程式。
- 如果這個專案的標頭檔被改變了，那麼我們需要編譯引用了這幾個標頭檔的程式碼，並連結目標程式。

Files: [Makefile-sample](https://osslab.tw/attachments/80)

```
make docker-compose-build
make docker-compose
```

##### Tutorials

- [Makefile 語法和示範](https://hackmd.io/@sysprog/SySTMXPvl)
- [Makefile Tutorial](https://makefiletutorial.com/)

# Rsyslog

#### Tutorials

- [Remote Syslogging with rsyslog on Red Hat Enterprise Linux - Red Hat Customer Portal](https://access.redhat.com/articles/3549872)
- [Chapter 23. Viewing and Managing Log Files Red Hat Enterprise Linux 7 | Red Hat Customer Portal](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system_administrators_guide/ch-viewing_and_managing_log_files)
- [The Definitive Guide to Centralized Logging with Syslog on Linux (devconnected.com)](https://devconnected.com/the-definitive-guide-to-centralized-logging-with-syslog-on-linux/)
- [鳥哥私房菜 - 第十八章、認識與分析登錄檔 (vbird.org)](https://linux.vbird.org/linux_basic/centos7/0570syslog.php)

#### 常用指令

```bash
# Validate the rsyslog configuration
rsyslogd -N 2 -f /etc/rsyslog.conf

# Restart the rsyslog
systemctl restart rsyslog
```

#### 整合特定應用程式

##### 情境一: 寫入日誌檔

應用程式透過 rsyslog 協定寫入訊息，系統要輸出特定日誌檔。

`/etc/rsyslog.d/myapp.conf`

```
# Save db2audit log to db2audit
# Test command:
# logger -t db2audit -p user.info "Hello, This is Test Message"
if $programname == 'db2audit' then action(type="omfile" file="/var/log/db2audit")
& stop
```

> TIP: 如果不用這判斷式，只用 `user.*` 格式，其他不相關的應用程式日誌也會一併寫入。

##### 情境二: 讀取日誌檔

應用程式已經有自己的日誌檔，內容也符合 rsyslog 標準日誌格式，需要同步也寫到外部的日誌伺服器。

`/etc/rsyslog.d/myapp.conf`

```
$ModLoad imfile

$InputFileName  /app/your-file.log 
$InputFileTag   your-tag
$InputFileStateFile     your-tag 
$InputFileSeverity      info
$InputFileFacility      local7 
$InputRunFileMonitor
$InputFilePersistStateInterval 1000 
local7.*  @@remote-rsyslog-server:port
```

##### 情境三：過濾不需要的日誌

系統日誌檔 (/var/log/messages) 不想顯示以下的訊息

```
Jul 24 08:50:01 example.com systemd: Created slice user-0.slice.
Jul 24 08:50:01 example.com systemd: Starting Session 150 of user root.
Jul 24 08:50:01 example.com systemd: Started Session 150 of user root.
Jul 24 09:00:01 example.com systemd: Created slice user-0.slice.
Jul 24 09:00:02 example.com systemd: Starting Session 151 of user root.
Jul 24 09:00:02 example.com systemd: Started Session 151 of user root.
```

/etc/rsyslog.d/ignore-systemd-session-slice.conf

```
if $programname == "systemd" and ($msg contains "Starting Session" or $msg contains "Started Session" or $msg contains "Created slice" or $msg contains "Starting user-" or $msg contains "Starting User Slice of" or $msg contains "Removed session" or $msg contains "Removed slice User Slice of" or $msg contains "Stopping User Slice of") then stop
```

#### Central Log Server

##### Server Configuration

`/etc/rsyslog.d/10-from-remote.conf`

```
# Avoid the duplicate messages from local syslog
$template RemoteLogs,"/var/log/%HOSTNAME%/%PROGRAMNAME%.log"
if ($fromhost != "local-server-hostname" ) then ?RemoteLogs
& stop
```

`/etc/rsyslog.conf`

```
# Provides TCP syslog reception
# for parameters see http://www.rsyslog.com/doc/imtcp.html
module(load="imtcp") # needs to be done just once
input(type="imtcp" port="514")
```

##### Client Configuration  


`/etc/rsyslog.d/10-to-remote.conf`

```
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
# Use @@ for TCP protocol, @ for UDP protocol
*.*  @10.4.1.77:514;RSYSLOG_SyslogProtocol23Format
```

##### Restrict access to the log server (on Server)

/etc/rsyslog.d/9-acl.conf

```
# Restrict access to the log server that is sent from
# $AllowedSender <type>, ip[/bits], ip[/bits]
$AllowedSender TCP, 127.0.0.1, 10.15.9.31
```

#### FAQ

##### 日誌檔不明原因無法被寫入新日誌

日誌檔一旦被編輯過就無法再被寫入，必須重啟 rsyslog 服務後才會恢復。

##### AIX: 接收 AIX 主機的 syslog 時無法正確顯示來源 IP  


原因：AIX syslog 傳遞至遠端 Log Server 時，預設會自動加上 *"Message forwarded by $hostname"* 的資訊。要避免這個問題，在啟動 syslogd 服務加上參數 `-n`。

```bash
startsrc -s syslogd -a "-n"
```

# DNS

# DNS Server

##### Tutorials

DNS over HTTPS (DoH)

- [How to install dnscrypt proxy with adblocker on Linux](https://www.cyberciti.biz/faq/how-to-install-dnscrypt-proxy-with-adblocker-on-linux/)

Secure DNS

- [DNS settings to avoid email spoofing and phishing for unused domain - nixCraft (cyberciti.biz)](https://www.cyberciti.biz/security/dns-settings-to-avoid-email-spoofing-and-phishing-for-unused-domain)
- [How to test and validate DNSSEC using dig command line - nixCraft (cyberciti.biz)](https://www.cyberciti.biz/faq/unix-linux-test-and-validate-dnssec-using-dig-command-line/)

##### Adguard Home

- [GitHub - AdguardTeam/AdGuardHome: Network-wide ads &amp; trackers blocking DNS server](https://github.com/AdguardTeam/AdGuardHome)

##### NSD

An authoritative-only DNS server

- [NSD Tutorial Updated for 2024: Now With Zone Transfers! - LowEndBox](https://lowendbox.com/blog/nsd-tutorial-updated-for-2024-now-with-zone-transfers/)
- [How To Use NSD, an Authoritative-Only DNS Server, on Ubuntu 14.04 | DigitalOcean](https://www.digitalocean.com/community/tutorials/how-to-use-nsd-an-authoritative-only-dns-server-on-ubuntu-14-04)

##### Pi-hole

Pi-hole - A DNS-based advertisement blocker

- [Pi-hole – Network-wide Ad Blocking](https://pi-hole.net/)
- [How to Set Up Pi-hole to Get an Ad-free Life](https://itsfoss.com/setup-pi-hole/)
- [7 Things I Wish I Knew Before Running a Pi-hole](https://www.howtogeek.com/things-i-wish-i-knew-before-running-a-pi-hole/)

docker-compose.yaml :

- 網頁登入密碼：`FTLCONF_webserver_api_password: 'XXXX'`

```yaml
# More info at https://github.com/pi-hole/docker-pi-hole/ and https://docs.pi-hole.net/
services:
  pihole:
    container_name: pihole
    image: pihole/pihole:latest
    ports:
      # DNS Ports
      - "53:53/tcp"
      - "53:53/udp"
      # Default HTTP Port
      - "80:80/tcp"
      # Default HTTPs Port. FTL will generate a self-signed certificate
      - "443:443/tcp"
      # Uncomment the line below if you are using Pi-hole as your DHCP server
      #- "67:67/udp"
      # Uncomment the line below if you are using Pi-hole as your NTP server
      #- "123:123/udp"
    environment:
      # Set the appropriate timezone for your location (https://en.wikipedia.org/wiki/List_of_tz_database_time_zones), e.g:
      TZ: 'Asia/Taipei'
      # Set a password to access the web interface. Not setting one will result in a random password being assigned
      FTLCONF_webserver_api_password: 'correct horse battery staple'
      # If using Docker's default `bridge` network setting the dns listening mode should be set to 'ALL'
      FTLCONF_dns_listeningMode: 'ALL'
    # Volumes store your data between container upgrades
    volumes:
      # For persisting Pi-hole's databases and common configuration file
      - './etc-pihole:/etc/pihole'
      # Uncomment the below if you have custom dnsmasq config files that you want to persist. Not needed for most starting fresh with Pi-hole v6. If you're upgrading from v5 you and have used this directory before, you should keep it enabled for the first v6 container start to allow for a complete migration. It can be removed afterwards. Needs environment variable FTLCONF_misc_etc_dnsmasq_d: 'true'
      #- './etc-dnsmasq.d:/etc/dnsmasq.d'
    cap_add:
      # See https://docs.pi-hole.net/docker/#note-on-capabilities
      # Required if you are using Pi-hole as your DHCP server, else not needed
      - NET_ADMIN
      # Required if you are using Pi-hole as your NTP client to be able to set the host's system time
      - SYS_TIME
      # Optional, if Pi-hole should get some more processing time
      - SYS_NICE
    restart: unless-stopped
```

##### Technitium

Self host a DNS server for privacy &amp; security

- [Technitium DNS Server | An Open Source DNS Server For Privacy &amp; Security](https://technitium.com/dns/)
- GitHub: [https://github.com/TechnitiumSoftware/DnsServer](https://github.com/TechnitiumSoftware/DnsServer)
- [Technitium: The Self-Hosted DNS Server You Should Run - YouTube](https://www.youtube.com/watch?v=9buji0Vnbo0)

# dig - DNS Lookup Utility

The `dig` is a powerful CLI to find out information about domains and IP addresses without using any 3rd party tools or websites.

01\. What is the website’s IP address?

```bash
dig apple.com
```

 02. How do you identify the name servers(NS) associated with a domain?

```bash
dig NS apple.com +short 
dig NS com. +short
```

03\. Which eMail Servers(MX) are responsible for a domain?

```bash
dig MX apple.com +short
```

04\. Finding out the domain name associated with the IP address (reverse IP lookup)

```
dig -x 1.1.1.1 +short 
dig -x 8.8.4.4 +short
```

05\. Finding out the delegation path for any DNS zone (learn how DNS works)

```
dig apple.com +trace
```

06\. Finding out DNS answers from specific cache resolver (e.g. Cloudflare \[1.1.1.1\], Google DNS \[8.8.8.8\], IBM and so on for cyberciti.biz domain)

```
dig A cyberciti.biz @1.1.1.1 +short 
dig A cyberciti.biz @8.8.8.8 +short
```

07\. Finding out the cache expire time (TTL) for DNS

```bash
# AAAA is for IPv6
dig A www.cyberciti.biz +nocmd +noall +answer +ttlid 
dig AAAA www.cyberciti.biz +nocmd +noall +answer +ttlid
```

08\. Finding if a zone is synchronized with all authoritative name servers (look for serial number)

```
dig nixcraft.com +nssearch
```

09\. What is my public IPv4 or IPv6 address?

```
dig TXT o-o.myaddr.l.google.com @ns1.google.com +short 
dig TXT ch whoami.cloudflare @1.0.0.1 +short
```

10\. Getting help about the dig command

```
man dig
```

11\. Finding out the IP address associated the domain name.

```
dig yahoo.com +short | sort -V
```

# What's DNS

##### Tutorials

- [What is Round-Robin DNS? - GreenCloud](https://blog.greencloudvps.com/what-is-round-robin-dns.php)
- [What is a DNS Amplification Attack? - GreenCloud](https://blog.greencloudvps.com/what-is-a-dns-amplification-attack.php)
- [DNS Failover: How does it work? - GreenCloud](https://blog.greencloudvps.com/dns-failover-how-does-it-work.php)

##### How DNS Works

[![how_dns_works.jpg](https://osslab.tw/uploads/images/gallery/2023-12/scaled-1680-/how-dns-works.jpg)](https://osslab.tw/uploads/images/gallery/2023-12/how-dns-works.jpg)

[![how_dns_works_2.jpg](https://osslab.tw/uploads/images/gallery/2024-02/scaled-1680-/how-dns-works-2.jpg)](https://osslab.tw/uploads/images/gallery/2024-02/how-dns-works-2.jpg)

##### DNS Record Type

[![dns_record_type.jpg](https://osslab.tw/uploads/images/gallery/2023-11/scaled-1680-/dns-record-type.jpg)](https://osslab.tw/uploads/images/gallery/2023-11/dns-record-type.jpg)

# DNS Online Tools

##### DNS Check

- [DNS Check](https://dnscheck.app/)
- [DNSChecker](https://dnschecker.org/all-tools.php)
- [Google Admin Toolbox Dig](https://toolbox.googleapps.com/apps/dig/)
- [Google Admin Toolbox Check MX](https://toolbox.googleapps.com/apps/checkmx/)
- [Nslookup.io](https://www.nslookup.io/)

##### SPF/DKIM/DMARC Check

- [Postmaster Tools (google.com)](https://postmaster.google.com/managedomains)

##### WHOIS Search

- [WHOIS Search, Domain Name, Website, and IP Tools - Who.is](https://who.is/)
- [Domain WHOIS Search | EuroDNS](https://www.eurodns.com/whois-search)

##### Domain Tools

- [tldx](https://github.com/brandonyoungdev/tldx) - 以關鍵字搜尋可用網域名稱

# RAID - mdadm

[![raid_in_Linux.jpg](https://osslab.tw/uploads/images/gallery/2025-04/scaled-1680-/raid-in-linux.jpg)](https://osslab.tw/uploads/images/gallery/2025-04/raid-in-linux.jpg)