Skip to content

Instantly share code, notes, and snippets.

@BohuTANG

BohuTANG/ioarena.txt

Last active November 6, 2015 07:20
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save BohuTANG/8818804aae6e6f04ea35 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save BohuTANG/8818804aae6e6f04ea35 to your computer and use it in GitHub Desktop.
Download ZIP
Sophia, LMDB, nessDB ioarena benchmarks
Raw
ioarena.txt
https://github.com/BohuTANG/ioarena
Model Name: MacBook Air
Model Identifier: MacBookAir7,2
Processor Name: Intel Core i5
Memory: 8 GB
==================================1) small kv tests=================================
number of operations 20,000,000
operations set
key order random
key size 16 bytes
value size 32 bytes
LMDB:
[ t min, t max] ops count %
--------------------------------------------------
[ 0, 1] 19999476 100.00
[ 1, 2] 28 0.00
[ 2, 3] 5 0.00
[ 3, 4] 3 0.00
[ 4, 5] 3 0.00
[ 10, 12] 1 0.00
[ 25, 30] 3 0.00
[ 30, 35] 20 0.00
[ 35, 40] 35 0.00
[ 40, 45] 11 0.00
[ 45, 50] 3 0.00
[ 50, 60] 6 0.00
[ 60, 70] 5 0.00
[ 70, 80] 3 0.00
[ 80, 90] 4 0.00
[ 90, 100] 5 0.00
[ 100, 120] 16 0.00
[ 120, 140] 14 0.00
[ 140, 160] 16 0.00
[ 160, 180] 15 0.00
[ 180, 200] 8 0.00
[ 200, 250] 45 0.00
[ 250, 300] 54 0.00
[ 300, 350] 56 0.00
[ 350, 400] 113 0.00
[ 400, 450] 42 0.00
[ 450, 500] 3 0.00
[ 500, 600] 1 0.00
[ 600, 700] 1 0.00
[ 700, 800] 1 0.00
[ 800, 900] 2 0.00
[ 900, 1000] 2 0.00
--------------------------------------------------
total: 217339 msec 20000000 100%
min latency : 0.000000 * 1e-3 sec/op
avg latency : 0.010867 * 1e-3 sec/op
max latency : 951.329947 * 1e-3 sec/op
avg throughput : 92022 ops/sec
Sophia:
[ t min, t max] ops count %
--------------------------------------------------
[ 0, 1] 19998355 99.99
[ 1, 2] 807 0.00
[ 2, 3] 332 0.00
[ 3, 4] 149 0.00
[ 4, 5] 105 0.00
[ 5, 6] 56 0.00
[ 6, 7] 45 0.00
[ 7, 8] 41 0.00
[ 8, 9] 18 0.00
[ 9, 10] 12 0.00
[ 10, 12] 20 0.00
[ 12, 14] 14 0.00
[ 14, 16] 11 0.00
[ 16, 18] 6 0.00
[ 18, 20] 5 0.00
[ 20, 25] 7 0.00
[ 25, 30] 5 0.00
[ 30, 35] 6 0.00
[ 35, 40] 1 0.00
[ 40, 45] 2 0.00
[ 50, 60] 2 0.00
[ 100, 120] 1 0.00
--------------------------------------------------
total: 130662 msec 20000000 100%
min latency : 0.001907 * 1e-3 sec/op
avg latency : 0.006533 * 1e-3 sec/op
max latency : 115.455866 * 1e-3 sec/op
avg throughput : 153066 ops/sec
nessDB: (cache size: 512MB)
[ t min, t max] ops count %
--------------------------------------------------
[ 0, 1] 19998660 99.99
[ 1, 2] 176 0.00
[ 2, 3] 94 0.00
[ 3, 4] 87 0.00
[ 4, 5] 107 0.00
[ 5, 6] 79 0.00
[ 6, 7] 53 0.00
[ 7, 8] 43 0.00
[ 8, 9] 49 0.00
[ 9, 10] 89 0.00
[ 10, 12] 173 0.00
[ 12, 14] 118 0.00
[ 14, 16] 82 0.00
[ 16, 18] 59 0.00
[ 18, 20] 44 0.00
[ 20, 25] 59 0.00
[ 25, 30] 18 0.00
[ 30, 35] 9 0.00
[ 35, 40] 1 0.00
--------------------------------------------------
total: 59629 msec 20000000 100%
min latency : 0.000000 * 1e-3 sec/op
avg latency : 0.002981 * 1e-3 sec/op
max latency : 36.046982 * 1e-3 sec/op
avg throughput : 335407 ops/sec
==================================2) 1KB set tests=================================
number of operations 10,000,000
operations set
key order random
key size 128 bytes
value size 1024 bytes
LMDB:
[ t min, t max] ops count %
--------------------------------------------------
[ 0, 1] 9960176 99.60
[ 1, 2] 4391 0.04
[ 2, 3] 8308 0.08
[ 3, 4] 23820 0.24
[ 4, 5] 601 0.01
[ 5, 6] 356 0.00
[ 6, 7] 258 0.00
[ 7, 8] 159 0.00
[ 8, 9] 87 0.00
[ 9, 10] 36 0.00
[ 10, 12] 42 0.00
[ 12, 14] 16 0.00
[ 14, 16] 6 0.00
[ 16, 18] 3 0.00
[ 18, 20] 1 0.00
[ 20, 25] 2 0.00
[ 25, 30] 5 0.00
[ 30, 35] 1 0.00
[ 35, 40] 3 0.00
[ 60, 70] 1 0.00
[ 70, 80] 1 0.00
[ 80, 90] 1 0.00
[ 90, 100] 1 0.00
[ 100, 120] 1 0.00
[ 120, 140] 2 0.00
[ 140, 160] 2 0.00
[ 160, 180] 1 0.00
[ 180, 200] 5 0.00
[ 200, 250] 17 0.00
[ 250, 300] 15 0.00
[ 300, 350] 24 0.00
[ 350, 400] 38 0.00
[ 400, 450] 94 0.00
[ 450, 500] 111 0.00
[ 500, 600] 306 0.00
[ 600, 700] 717 0.01
[ 700, 800] 320 0.00
[ 800, 900] 51 0.00
[ 900, 1000] 14 0.00
[ 1000, 1200] 7 0.00
--------------------------------------------------
total: 2046133 msec 10000000 100%
min latency : 0.000000 * 1e-3 sec/op
avg latency : 0.204613 * 1e-3 sec/op
max latency : 1143.145084 * 1e-3 sec/op
avg throughput : 4887 ops/sec
nessDB:(cache size:512MB fanout:32 page size:16MB)
[ t min, t max] ops count %
--------------------------------------------------
[ 0, 1] 9995856 99.96
[ 1, 2] 2349 0.02
[ 2, 3] 769 0.01
[ 3, 4] 86 0.00
[ 4, 5] 19 0.00
[ 5, 6] 18 0.00
[ 6, 7] 11 0.00
[ 7, 8] 15 0.00
[ 8, 9] 21 0.00
[ 9, 10] 17 0.00
[ 10, 12] 38 0.00
[ 12, 14] 20 0.00
[ 14, 16] 12 0.00
[ 16, 18] 14 0.00
[ 18, 20] 21 0.00
[ 20, 25] 125 0.00
[ 25, 30] 119 0.00
[ 30, 35] 103 0.00
[ 35, 40] 107 0.00
[ 40, 45] 71 0.00
[ 45, 50] 54 0.00
[ 50, 60] 73 0.00
[ 60, 70] 19 0.00
[ 70, 80] 35 0.00
[ 80, 90] 6 0.00
[ 90, 100] 2 0.00
[ 100, 120] 12 0.00
[ 120, 140] 3 0.00
[ 140, 160] 2 0.00
[ 160, 180] 2 0.00
[ 180, 200] 1 0.00
--------------------------------------------------
total: 145004 msec 10000000 100%
min latency : 0.000000 * 1e-3 sec/op
avg latency : 0.014500 * 1e-3 sec/op
max latency : 195.000172 * 1e-3 sec/op
avg throughput : 68963 ops/sec
sophia:
[ t min, t max] ops count %
--------------------------------------------------
[ 0, 1] 9996288 99.96
[ 1, 2] 1612 0.02
[ 2, 3] 426 0.00
[ 3, 4] 255 0.00
[ 4, 5] 205 0.00
[ 5, 6] 166 0.00
[ 6, 7] 157 0.00
[ 7, 8] 123 0.00
[ 8, 9] 90 0.00
[ 9, 10] 93 0.00
[ 10, 12] 141 0.00
[ 12, 14] 93 0.00
[ 14, 16] 65 0.00
[ 16, 18] 66 0.00
[ 18, 20] 37 0.00
[ 20, 25] 64 0.00
[ 25, 30] 33 0.00
[ 30, 35] 16 0.00
[ 35, 40] 12 0.00
[ 40, 45] 11 0.00
[ 45, 50] 18 0.00
[ 50, 60] 11 0.00
[ 60, 70] 4 0.00
[ 70, 80] 5 0.00
[ 80, 90] 4 0.00
[ 90, 100] 1 0.00
[ 100, 120] 2 0.00
[ 140, 160] 1 0.00
[ 600, 700] 1 0.00
--------------------------------------------------
total: 132064 msec 10000000 100%
min latency : 0.002861 * 1e-3 sec/op
avg latency : 0.013206 * 1e-3 sec/op
max latency : 623.759031 * 1e-3 sec/op
avg throughput : 75721 ops/sec
@pmwkaa
Copy link

pmwkaa commented Nov 4, 2015

nice) Is this a -T set test?
I need to better optimize single-statement operation layering, in-memory transaction support has its cost.

Now i'm waiting for the pull request with nessDB support in ioarena :)

@BohuTANG
Copy link
Author

BohuTANG commented Nov 4, 2015

@pmwkaa
yes, since nessDB only supports set operation now.
I am not sure whether Sophia supports in non-txn mode, if so, we should have a try.

@pmwkaa
Copy link

pmwkaa commented Nov 5, 2015

It would be nice if @hyc could take a look at the bechmarking code driver: https://github.com/pmwkaa/ioarena/blob/master/src/ia_lmdb.c

I might messed up something.

Additionally mdbx (lmdb fork with some perfomance issues solved) worth to look at: https://github.com/ReOpen/ioarena

I will soon take patches to ioarena from it.

@hyc
Copy link

hyc commented Nov 5, 2015

Cool, will take a look. Has @leo-yuriev already run thru these tests?

The latency distribution for LMDB looks really odd.

@hyc
Copy link

hyc commented Nov 5, 2015

Hm, you're doing all of the LMDB operations in a single transaction, that's not the way a transactional DB is typically used. And you've set "batch" as unsupported, when in fact, a batch would be some number of ops in a single transaction.

Much of that LMDB driver needs to be rewritten, it's not a realistic use at all.

@BohuTANG
Copy link
Author

BohuTANG commented Nov 6, 2015

'Much of that LMDB driver needs to be rewritten, it's not a realistic use at all.'
No, we just add these codes to ia_lmdb_set:

if ((i % 1000) == 0) {
            mdb_txn_commit(s->txn);
            rc = mdb_txn_begin(s->env, NULL, 0, &s->txn);
            if (rc != MDB_SUCCESS) {
                ia_log("error: mdb_txn_begin(): %d", rc);
                return -1;
            }
  }

It works as 10000 sets per transaction, but it still slow and then the tail gets really slow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment