AWS c5 instance performance tests s3/efs/ebs

gistfile1.txt

Write-Tests:


Create a random 100Mb, 1GB, 10GB file on each file system and measure time for creation
Copy each of the test files from local EBS to each filesystem. Measure time.


1GB  dd if=/dev/urandom of=1GB_testfile bs=1024 count=1024000
100MB dd if=/dev/urandom of=100MB_testfile bs=1024 count=102400
10 MB dd if=/dev/urandom of=10MB_testfile bs=1024 count=10240
10 GB  dd if=/dev/urandom of=1GB_testfile bs=1024 count=10240000


Write:


Size
S3fs
EFS
EBS (LVM)


EBS (local root)
10 MB
8.1 MB/s
9.5 MB/s
11.8 MB/s


11.7 MB/s
100 MB
 7.8 MB/s
11.3 MB/s
11.8 MB/s




1000 MB
 8.2 MB/s
10.8 MB/s
11.8 MB/s




10000 MB


10.9 MB/s








Read-Test:
We stream data from tar files without untarring. Need to devise a test that mocks the same. 
tar -zxvf <tar_name> <file_to_untar>
andrew.diamond [4:34 PM]
It fairly common for a worker to process several files at once from within the tar archive. So it may have four open readers for the same file, each reading a different byte range. That's why SSDs work well with our ingest services, and magnetic disks are slower.


All that random access.
I know S3 supports byte range requests, I just haven't used that feature yet.
andrew.diamond [4:32 PM]
In ingest services and dpn validation services, the worker opens the tar file and reads the headers. Then it extracts specific files by telling the reader "Give me bytes 8800 - 35663 from the tar file." Something like that.




FileSystemBenchmarkCPP
https://github.com/GilbertoRibeiroPaz/FileSystemBenchmarkCPP


Summary:




1000 MB File / Throughput MiB/s
s3fs
LVM(EBS)
EFS
Read Sequential 
158.6459127162
167.4653024069 
141.2982197837
Read Random 
93.4104273864
99.6904259664
1.3955869698
Write Sequential 
77.4857459368 
144.3925940803 
148.0301626259
 Write Random  
22.8514458581
2.6540337084
 0.2427373954














Details below:
ubuntu@dpn-prod3:~/FileSystemBenchmarkCPP/dist/Release/GNU-Linux-x86$ ./filesystemthroughputbenchmark  /mnt/s3fs/ 2 1000M
Final average results:
--------------------------------------------------------------------------------------------------------------------------------------
      Operation Type        Throughput MiB/s      Average in seconds       Default Deviation          Execution time
     Read Sequential          158.6459127162            0.0063033455            0.0000296842            6.3033455000
         Read Random           93.4104273864            0.0107054435            0.0053168971           10.7054435000
    Write Sequential           77.4857459368            0.0129057210            0.0004256114           12.9057210000
        Write Random           22.8514458581            0.0437642805            0.0843687953           43.7642805000


Time for all testes: 147.3575810000 seconds




ubuntu@dpn-prod3:~/FileSystemBenchmarkCPP/dist/Release/GNU-Linux-x86$ ./filesystemthroughputbenchmark  /mnt/lvm/ 2 1000M
Final average results:
--------------------------------------------------------------------------------------------------------------------------------------
      Operation Type        Throughput MiB/s      Average in seconds       Default Deviation          Execution time
     Read Sequential          167.4653024069            0.0059714035            0.0000615847            5.9714035000
         Read Random           99.6904259664            0.0102092305            0.0003192612           10.2092305000
    Write Sequential          144.3925940803            0.0069259915            0.0018484858            6.9259915000
        Write Random            2.6540337084            0.4036540115            0.5758808196          403.6540115000