Benchmark results

gistfile1.txt

============================================================
BENCHMARK SUMMARY
============================================================
Model: hamishivi/qwen2_5_openthoughts2
Total batches: 5
Batch size: 256
Unique prompts per batch: 32
Num rollouts: 8
Max tokens: 32000
------------------------------------------------------------
Total time: 5485.93s (99.9983% generating)
Total new tokens generated: 18,628,758
------------------------------------------------------------
Results (excluding first batch):
Average tokens/second: 3,500.95
Average MFU: 5.00%
Average generation time per batch: 1,072.42 s
Average new tokens per sample: 3,748,289.0 tokens
Wasted compute % (variable response length): 28.36 %
------------------------------------------------------------
HARDWARE SPECIFICATIONS:
GPU device: NVIDIA H100 80 GB HBM3
GPU peak FLOPs: 990 TFLOPs
GPU memory size: 80 GB
------------------------------------------------------------
COMPLETION LENGTH STATISTICS:
Total completions: 1,024

Response lengths:
- Min: 1,760 tokens
- Max: 20,437 tokens
- Mean: 14,641.75 tokens
- Median: 16,077.00 tokens

Response length percentiles:
- 25th percentile: 9,713.75 tokens
- 50th percentile: 16,077.00 tokens
- 75th percentile: 20,313.00 tokens
- 90th percentile: 20,397.00 tokens
- 95th percentile: 20,423.00 tokens
- 99th percentile: 20,434.00 tokens
============================================================