![]() ![]() The PCI Express uses a split transaction model for reads. Read throughput is typically lower than the write throughput because reads require two transactions instead of a single write for the same amount of data. ![]() A write operation is a "Posted" operation that consists of a request only. Once the Transaction Layer Packet is handed over to the Data Link Layer, the operation completes. This is because while a PCIe read operation is a Non-Posted Operation, requiring both a request and a completion, a PCIe write operation is a fire and forget operation. We also observe that the read performance is lower than writes for thread counts from 8 to 128. The read performance registers an almost steady linear increase as we scale up the number of concurrent threads, and we observe the peak performance at 512 threads. At lower thread counts read and write performance are the same. We observe that the write performance increases monotonically up to 16 threads, reaches a plateau, peaks at 256 threads and then starts decreasing. The single thread performance is observed to be ~4 GB/s for writes and reads. The achieved peak read and write performance are 93% and 92% respectively of the theoretical peak performance. We have a total of 5 InfiniBand HDR links for the storage servers in the test bed.Įach link can provide a theoretical peak performance of 25 GB/s which provides an aggregate theoretical peak performance of 125 GB/s. However, here the network is the limiting factor. ![]() As per the technical specifications of the Intel P5600 1.6 TB NVMe SSDs, each drive can provide 3.5 GB/s peak read performance and 1.7 GB/s peak write performance, which allows a theoretical peak of 140 GB/s for reads and 68 GB/s for writes. We observe that peak read performance is 116 GB/s at 512 threads and peak write is 62 GB/s at 256 threads. The IOzone sequential write and read tests were performed three times, and the mean value is plotted in Figure 9 above, with the peak performance highlighted in the chart. Sequential IOzone Performance with aggregate file size 8 TB $ beegfs-ctl -getentryinfo /mnt/15g-perf/Benchmarks/ -cfgFile=/etc/beegfs/nfįigure 9. For all these tests, BeeGFS stripe size was chosen to be 2MB and stripe count was chosen to be 2 since we have two targets per NUMA zone as shown below: However, the chunk size and the number of targets per file can be configured on a per-directory basis. The default stripe count for BeeGFS is 4. Iozone -i 0 -i 1 -c -e -w -r 1m -I -s $Size -t $Thread -+n -+m /path/to/threadlist The commands used for Sequential N-N tests are as follows: IOzone was run in a combined mode of write then read (-i 0, -i 1) to allow it to coordinate the boundaries between the operations.įor this testing and results, we used a 1MiB record size for every run. The aggregate file size was chosen large enough to minimize the effects of caching from the servers as well as from BeeGFS clients. An aggregate file size of 8TB was selected which was equally divided among the number of threads within any given test. ![]() The processes were distributed across 16 physical client nodes in a round robin or cyclical fashion so that the requests are equally distributed and there is load balancing. At each thread count, an equal number of files were generated since this test works on one file per thread or the N clients to N file (N-N) case. These tests were conducted on multiple thread counts starting at 1 thread and increasing in powers of 2, up to 1024 threads. To evaluate sequential reads and writes, the IOzone benchmark v3.492 was used in the sequential read and write mode. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |