Benchmark Issue
Created by: Jotschi
Hi,
I took a look at the benchmark. Unfortunately there seems to be wrong assumption. The Tinkerpop 2.6.0 TinkerGraph implementation is faster compared to the TinkerGraph 3.2.0 implementation. But I think that does not reflect the Tinkerpop 3.2.0 performance. I have not yet determined why the TinkerGraph 3.2.0 is much slower but I believe it may have something to do with the way IDs are handled. (Just a hunch)
I did the same test using OrientDB TP2 and OrientDB TP3 (unofficial implementation) and came up with these figures:
=== testGetFramedVerticesAndNextUntyped ===
blueprints comparison: 99.53026124657256%
tinkerpop3 comparison: 65.92568822882211%
totorom comparison: 103.64681453936765%
frames comparison: Not capable
gremlin comparison: 107.2673447424247%
peapod comparison: Not Capable
=== testGetFramedVerticesUntyped ===
blueprints comparison: 99.51659220831277%
tinkerpop3 comparison: 62.21783215716792%
totorom comparison: 102.49149226551873%
frames comparison: Not capable
gremlin comparison: 101.86015225124282%
peapod comparison: Not Capable
=== testGetFramedVerticesTyped ===
blueprints comparison: Not Capable
tinkerpop3 comparison: Not Capable
totorom comparison: 102.792778909281%
frames comparison: 100.65792462081421%
gremlin comparison: Not Capable
peapod comparison: 65.89351246282055%
=== testGetFramedVerticesAndNextTyped ===
blueprints comparison: Not Capable
tinkerpop3 comparison: Not Capable
totorom comparison: 103.90106388168827%
frames comparison: 100.84983026562409%
gremlin comparison: 99.04836654974989%
peapod comparison: 61.78271340104785%
=== testGetAnnotatedAdjacencies ===
blueprints comparison: Not capable
tinkerpop3 comparison: Not Capable
totorom comparison: Not capable
frames comparison: 97.26275010670106%
gremlin comparison: Not capable
peapod comparison: 97.64909785802675%
This means that at least for the OrientDB the TP3 implementation is around 35% faster. I think it would be best to only compare TP3 implementations with eachother and not mix them with TP2 / TP2 OGM's.