Block Volume Performance calculation

In the realm of modern computing, where data storage and retrieval speed are paramount, understanding the performance of storage solutions is crucial. One of the fundamental components of this landscape is Linux block volume performance calculation. Whether you're a system administrator, a developer, or an enthusiast, delving into the intricacies of block volume performance, including Fio-based tests, can empower you to make informed decisions about storage setups. In this blog post, we'll demystify the concepts behind Linux block volume performance calculation and explore the key factors that influence it, along with practical Fio-based tests.
 

Understanding Block Volumes:
Block volumes are a type of storage solution commonly used in modern IT infrastructures. They provide raw storage space that can be partitioned and formatted according to the user's needs. These volumes are often found in virtual machines, cloud instances, and even physical servers. They are characterized by their ability to handle data at the block level, meaning data is read from and written to storage in fixed-size blocks.
 

Factors Influencing Block Volume Performance:
Several factors play a pivotal role in determining the performance of Linux block volumes. Understanding these factors helps optimize storage systems for better efficiency and responsiveness.

1. I/O Operations Per Second (IOPS): IOPS refers to the number of input/output operations a storage device can handle in a second. It is a key metric in assessing storage responsiveness. The higher the IOPS, the faster the storage system can read from or write to the block volume.

2. Throughput: Throughput measures the amount of data that can be transferred between the storage device and the system in a given period. It's usually measured in megabytes or gigabytes per second. Throughput is a crucial metric when dealing with large data transfers.

3. Latency: Latency is the delay between initiating a data request and receiving the first byte of data. Lower latency indicates a more responsive storage system. Excessive latency can lead to delays in data-intensive operations.

4. Queue Depth: Queue depth refers to the number of I/O requests that can be in the queue to the storage device at a given time. A higher queue depth can lead to improved performance, especially in scenarios with concurrent I/O operations.

Calculating Block Volume Performance:
While calculating precise block volume performance can be intricate, here's a simplified approach:

1. IOPS Calculation: Determine the total IOPS required by considering the application's read and write demands. Divide this total by the number of block volumes to distribute the load. It's important to consider peak I/O requirements.

2. Throughput Calculation: Calculate the required throughput by estimating the data transfer needs of the application. Divide this by the number of block volumes for load distribution.

3. Latency Estimation: Latency is affected by various factors, including the speed of the storage media and the efficiency of the underlying technology. Faster media and optimized configurations lead to lower latency.

Optimizing Block Volume Performance:
To optimize Linux block volume performance, consider these strategies:

1. RAID Configuration: Implement Redundant Array of Independent Disks (RAID) configurations to enhance performance, data redundancy, and fault tolerance.

2. File System Choice: Select the appropriate file system based on the performance and features needed. Options like ext4, XFS, and Btrfs offer different trade-offs.

3. Read and Write Patterns: Understand the application's read and write patterns. Optimize storage settings based on whether your application is more read-heavy or write-heavy.

4. Caching and Buffering: Utilize caching mechanisms, both in hardware and software, to accelerate frequently accessed data.

Fio-Based Performance Testing:
Fio (Flexible I/O Tester) is a powerful tool for benchmarking and testing storage performance. It allows you to simulate various workloads and measure performance metrics. Here's how you can incorporate Fio-based tests:

1. Install Fio: Begin by installing Fio on your Linux system. You can use package managers like `apt` or `yum` to do so.

2. Create Fio Configuration: Craft a Fio configuration file that specifies the workload, block size, I/O pattern, and other parameters. Here's an example configuration for a random read test (`random-read.fio`):
 

[random-read]
rw=randread
bs=4k
ioengine=libaio
iodepth=16
numjobs=4
size=1G


3. Run Fio Test: Execute the Fio test using the configuration file:

# fio random-read.fio


Fio will simulate the specified workload and generate comprehensive performance metrics, including IOPS, throughput, and latency.

4. Analyze Results: Study the results generated by Fio. This information will help you understand how your block volume performs under different scenarios and workloads.

Review my other post to understand how to review FIO O/P.

For now, here are some fio test runs on an NVMe disk. Be ready to be amazed by the NVMe performance!

NVME Perf tests on Linux VM node with 8 cores (16 threads):

[root@custtest mnt]# df -Ph /mnt
 Filesystem Size Used Avail Use% Mounted on
 /dev/nvme0n1p1 2.0T 4.1G 1.9T 1% /mnt
 [root@custtest mnt]#

 

1. Random Read - 1 job

[root@custtest mnt]# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=32 --size=4G --readwrite=randread
 test: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
 fio-3.1
 Starting 1 process
 Jobs: 1 (f=1): [r(1)][100.0%][r=456MiB/s,w=0KiB/s][r=117k,w=0 IOPS][eta 00m:00s]
 test: (groupid=0, jobs=1): err= 0: pid=14067: Thu Nov 8 12:15:56 2018
 read: IOPS=117k, BW=456MiB/s (478MB/s)(4096MiB/8992msec)
 bw ( KiB/s): min=464742, max=467560, per=99.95%, avg=466228.41, stdev=936.28, samples=17
 iops : min=116183, max=116890, avg=116556.88, stdev=234.32, samples=17
 cpu : usr=31.73%, sys=68.26%, ctx=29, majf=0, minf=38
 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
 submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
 complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
 issued rwt: total=1048576,0,0, short=0,0,0, dropped=0,0,0
 latency : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
 READ: bw=456MiB/s (478MB/s), 456MiB/s-456MiB/s (478MB/s-478MB/s), io=4096MiB (4295MB), run=8992-8992msec

Disk stats (read/write):
 nvme0n1: ios=1037622/0, merge=0/0, ticks=86183/0, in_queue=77803, util=99.12%
 [root@custtest mnt]#

2. Random Read - 16 jobs

[root@custtest mnt]# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=32 --size=4G --readwrite=randread --numjobs=16 --group_reporting
 test: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
 ...
 fio-3.1
 Starting 16 processes
 Jobs: 16 (f=16): [r(16)][100.0%][r=1625MiB/s,w=0KiB/s][r=416k,w=0 IOPS][eta 00m:00s]
 test: (groupid=0, jobs=16): err= 0: pid=9089: Thu Nov 8 13:01:36 2018
 read: IOPS=584k, BW=2281MiB/s (2392MB/s)(64.0GiB/28725msec)
 bw ( KiB/s): min=97912, max=197496, per=6.27%, avg=146482.86, stdev=23546.32, samples=912
 iops : min=24478, max=49374, avg=36620.60, stdev=5886.58, samples=912
 cpu : usr=15.77%, sys=40.00%, ctx=3823197, majf=0, minf=621
 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
 submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
 complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
 issued rwt: total=16777216,0,0, short=0,0,0, dropped=0,0,0
 latency : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
 READ: bw=2281MiB/s (2392MB/s), 2281MiB/s-2281MiB/s (2392MB/s-2392MB/s), io=64.0GiB (68.7GB), run=28725-28725msec

Disk stats (read/write):
 nvme0n1: ios=16745719/0, merge=0/0, ticks=14087686/0, in_queue=20697740, util=100.00%
 [root@custtest mnt]#

3. Random write - 1 job

[root@custtest mnt]# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=32 --size=4G --readwrite=randwrite --numjobs=1 --group_reporting
 test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
 fio-3.1
 Starting 1 process
 Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=418MiB/s][r=0,w=107k IOPS][eta 00m:00s]
 test: (groupid=0, jobs=1): err= 0: pid=9129: Thu Nov 8 13:06:12 2018
 write: IOPS=107k, BW=418MiB/s (438MB/s)(4096MiB/9801msec)
 bw ( KiB/s): min=424160, max=430080, per=99.96%, avg=427792.11, stdev=1825.03, samples=19
 iops : min=106040, max=107520, avg=106947.89, stdev=456.21, samples=19
 cpu : usr=30.26%, sys=69.74%, ctx=8, majf=0, minf=6
 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
 submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
 complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
 issued rwt: total=0,1048576,0, short=0,0,0, dropped=0,0,0
 latency : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
 WRITE: bw=418MiB/s (438MB/s), 418MiB/s-418MiB/s (438MB/s-438MB/s), io=4096MiB (4295MB), run=9801-9801msec

Disk stats (read/write):
 nvme0n1: ios=0/1032133, merge=0/1, ticks=0/10936, in_queue=3542, util=10.21%
 [root@custtest mnt]#

4. Random Write - 16 jobs

[root@custtest mnt]# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=32 --size=4G --readwrite=randwrite --numjobs=16 --group_reporting
 test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
 ...
 fio-3.1
 Starting 16 processes
 Jobs: 9 (f=9): [w(1),_(2),w(1),_(1),w(2),_(1),w(1),_(1),w(2),_(1),w(2),_(1)][100.0%][r=0KiB/s,w=1031MiB/s][r=0,w=264k IOPS][eta 00m:00s]
 test: (groupid=0, jobs=16): err= 0: pid=9110: Thu Nov 8 13:05:41 2018
 write: IOPS=293k, BW=1145MiB/s (1201MB/s)(64.0GiB/57215msec)
 bw ( KiB/s): min=12024, max=97485, per=6.27%, avg=73570.12, stdev=9316.96, samples=1809
 iops : min= 3006, max=24371, avg=18392.45, stdev=2329.24, samples=1809
 cpu : usr=5.03%, sys=85.43%, ctx=3442512, majf=0, minf=128
 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
 submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
 complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
 issued rwt: total=0,16777216,0, short=0,0,0, dropped=0,0,0
 latency : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
 WRITE: bw=1145MiB/s (1201MB/s), 1145MiB/s-1145MiB/s (1201MB/s-1201MB/s), io=64.0GiB (68.7GB), run=57215-57215msec

Disk stats (read/write):
 nvme0n1: ios=0/16754526, merge=0/11, ticks=0/3866085, in_queue=4938740, util=100.00%
 [root@custtest mnt]#

In Conclusion:
Linux block volume performance calculation is a multifaceted process influenced by IOPS, throughput, latency, and queue depth. By comprehending these metrics and their implications, you can make well-informed decisions about storage configurations, leading to efficient and responsive systems. Additionally, integrating Fio-based tests into your performance assessment toolkit empowers you to measure and analyze storage performance empirically. Whether you're managing virtual machines, cloud instances, or on-premises servers, a grasp of block volume performance, coupled with Fio-based testing, will undoubtedly be a valuable asset in your IT arsenal.