namgyu-youn

./benchmarks/quantization/measure_accuracy_and_performance.sh smoothquant_int8 meta-llama/Llama-3.2-1B
Skipping import of cpp extensions due to incompatible torch version 2.9.0+cu128 for torchao version 0.15.0             Please see https://github.com/pytorch/ao/issues/2919 for more info
torch.__version__='2.9.0+cu128'
torch.cuda.get_device_name()='NVIDIA A100 80GB PCIe MIG 2g.20gb'
torchao.__version__='0.15.0'
vllm.__version__='0.13.0'

processing quant_recipe smoothquant_int8

Skipping import of cpp extensions due to incompatible torch version 2.9.0+cu128 for torchao version 0.15.0             Please see https://github.com/pytorch/ao/issues/2919 for more info

namgyu-youn

> ./benchmarks/quantization/measure_accuracy_and_performance.sh awq_int4_weight_only meta-llama/Llama-3.2-1B

torch.__version__='2.9.0+cu128'
torch.cuda.get_device_name()='NVIDIA A100 80GB PCIe MIG 1g.10gb'
torchao.__version__='0.16.0+git529289d8'
vllm.__version__='0.13.0'

processing quant_recipe awq_int4_weight_only

namgyu-youn

import torch

import torchao.kernel.int8_scaled_mm_triton  # noqa: F401
from torchao.kernel.intmm import int_scaled_matmul


def benchmark_in_ms(warmup, iters, f):
    """Benchmark using CUDA events."""
    for _ in range(warmup):
        f()

namgyu-youn

torch.__version__='2.9.0+cu128'
torch.cuda.get_device_name()='NVIDIA A100 80GB PCIe'
torchao.__version__='0.15.0'
vllm.__version__='0.13.0'

processing quant_recipe int8_rowwise

Skipping import of cpp extensions due to incompatible torch version 2.9.0+cu128 for torchao version 0.15.0             Please see https://github.com/pytorch/ao/issues/2919 for more info

Running model_id='meta-llama/Llama-3.1-8B' with quant_recipe_name='int8_rowwise'

namgyu-youn

How to fix a corrupt zsh history file
Occasionally you may find you have a corrupt zsh history file preventing you from using the `fc` command or searching the history. Here's how to fix it.
Estimated reading time: 1 minutes

Table of contents
Corrupt ZSH history file
How to fix it
Making it a script
Corrupt ZSH history file
If you use zsh for your shell very occasionally you may find the following message appearing indicating a corrupt history file. This is normally after a reboot.