Skip to content

Instantly share code, notes, and snippets.

@Lyken17

Lyken17/lmeval_sglang.py

Created January 16, 2026 03:19
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save Lyken17/f4599655d67559fcd46b1b5c914e0079 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save Lyken17/f4599655d67559fcd46b1b5c914e0079 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
lmeval_sglang.py
python -m sglang.launch_server \
--model Qwen/qwen3-8B --port 8000 \
--dtype auto --data-parallel-size 8 --tensor-parallel-size 1
##################################################################
lm_eval --model local-completions \
--tasks gsm8k,gsm8k_cot,aime24,aime25 \
--model_args model=Qwen/qwen3-8B,base_url=http://127.0.0.1:8000/v1/completions,num_concurrent=32,max_retries=3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment