Skip to content

Instantly share code, notes, and snippets.

@jd-boyd
Created July 30, 2025 00:01
Show Gist options
  • Select an option

  • Save jd-boyd/11786455912d956ac55c68c07c865222 to your computer and use it in GitHub Desktop.

Select an option

Save jd-boyd/11786455912d956ac55c68c07c865222 to your computer and use it in GitHub Desktop.
llm llm-mlx Codestral error
% llm mlx download-model mlx-community/Codestral-22B-v0.1-4bit
config.json: 100%|█████████████████████████████| 716/716 [00:00<00:00, 6.01MB/s]
convert_mistral_weights_to_hf-22B.py: 11.3kB [00:00, 20.6MB/s]<00:03, 2.51it/s]
model.safetensors.index.json: 111kB [00:00, 45.4MB/s] | 0.00/716 [00:00<?, ?B/s]
special_tokens_map.json: 100%|█████████████████| 414/414 [00:00<00:00, 7.03MB/s]
tokenizer_config.json: 100%|███████████████████| 960/960 [00:00<00:00, 1.65MB/s]
tokenizer.json: 1.82MB [00:00, 18.5MB/s] | 0.00/1.84G [00:00<?, ?B/s]
tokenizer.model: 100%|████████████████████████| 587k/587k [00:01<00:00, 413kB/s]
tokenizer.json: 0.00B [00:00, ?B/s]
model-00003-of-00003.safetensors: 100%|████| 1.84G/1.84G [11:48<00:00, 2.60MB/s]
model-00002-of-00003.safetensors: 100%|████| 5.35G/5.35G [14:18<00:00, 6.23MB/s]
model-00001-of-00003.safetensors: 100%|████| 5.32G/5.32G [14:19<00:00, 6.19MB/s]
Fetching 10 files: 100%|████████████████████████| 10/10 [14:20<00:00, 86.01s/it]
Traceback (most recent call last): 99%|███▉| 5.26G/5.32G [14:18<00:00, 99.2MB/s]
File "/Users/jdboyd/.local/bin/llm", line 10, in <module>4:19<00:00, 87.1MB/s]
sys.exit(cli())
~~~^^
File "/Users/jdboyd/.local/share/uv/tools/llm/lib/python3.13/site-packages/click/core.py", line 1442, in __call__
return self.main(*args, **kwargs)
~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/Users/jdboyd/.local/share/uv/tools/llm/lib/python3.13/site-packages/click/core.py", line 1363, in main
rv = self.invoke(ctx)
File "/Users/jdboyd/.local/share/uv/tools/llm/lib/python3.13/site-packages/click/core.py", line 1830, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
File "/Users/jdboyd/.local/share/uv/tools/llm/lib/python3.13/site-packages/click/core.py", line 1830, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
File "/Users/jdboyd/.local/share/uv/tools/llm/lib/python3.13/site-packages/click/core.py", line 1226, in invoke
return ctx.invoke(self.callback, **ctx.params)
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jdboyd/.local/share/uv/tools/llm/lib/python3.13/site-packages/click/core.py", line 794, in invoke
return callback(*args, **kwargs)
File "/Users/jdboyd/.local/share/uv/tools/llm/lib/python3.13/site-packages/llm_mlx.py", line 56, in download_model
MlxModel(model_path).prompt("hi").text()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/Users/jdboyd/.local/share/uv/tools/llm/lib/python3.13/site-packages/llm/models.py", line 917, in text
self._force()
~~~~~~~~~~~^^
File "/Users/jdboyd/.local/share/uv/tools/llm/lib/python3.13/site-packages/llm/models.py", line 914, in _force
list(self)
~~~~^^^^^^
File "/Users/jdboyd/.local/share/uv/tools/llm/lib/python3.13/site-packages/llm/models.py", line 1023, in __iter__
for chunk in self.model.execute(
~~~~~~~~~~~~~~~~~~^
self.prompt,
^^^^^^^^^^^^
...<2 lines>...
conversation=self.conversation,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
):
^
File "/Users/jdboyd/.local/share/uv/tools/llm/lib/python3.13/site-packages/llm_mlx.py", line 211, in execute
model, tokenizer = self._load()
~~~~~~~~~~^^
File "/Users/jdboyd/.local/share/uv/tools/llm/lib/python3.13/site-packages/llm_mlx.py", line 203, in _load
self._model, self._tokenizer = load(self.model_path)
~~~~^^^^^^^^^^^^^^^^^
File "/Users/jdboyd/.local/share/uv/tools/llm/lib/python3.13/site-packages/mlx_lm/utils.py", line 264, in load
tokenizer = load_tokenizer(
model_path, tokenizer_config, eos_token_ids=config.get("eos_token_id", None)
)
File "/Users/jdboyd/.local/share/uv/tools/llm/lib/python3.13/site-packages/mlx_lm/tokenizer_utils.py", line 453, in load_tokenizer
AutoTokenizer.from_pretrained(model_path, **tokenizer_config_extra),
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jdboyd/.local/share/uv/tools/llm/lib/python3.13/site-packages/transformers/models/auto/tokenization_auto.py", line 1114, in from_pretrained
return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jdboyd/.local/share/uv/tools/llm/lib/python3.13/site-packages/transformers/tokenization_utils_base.py", line 2013, in from_pretrained
return cls._from_pretrained(
~~~~~~~~~~~~~~~~~~~~^
resolved_vocab_files,
^^^^^^^^^^^^^^^^^^^^^
...<9 lines>...
**kwargs,
^^^^^^^^^
)
^
File "/Users/jdboyd/.local/share/uv/tools/llm/lib/python3.13/site-packages/transformers/tokenization_utils_base.py", line 2259, in _from_pretrained
tokenizer = cls(*init_inputs, **init_kwargs)
File "/Users/jdboyd/.local/share/uv/tools/llm/lib/python3.13/site-packages/transformers/models/llama/tokenization_llama_fast.py", line 154, in __init__
super().__init__(
~~~~~~~~~~~~~~~~^
vocab_file=vocab_file,
^^^^^^^^^^^^^^^^^^^^^^
...<10 lines>...
**kwargs,
^^^^^^^^^
)
^
File "/Users/jdboyd/.local/share/uv/tools/llm/lib/python3.13/site-packages/transformers/tokenization_utils_fast.py", line 108, in __init__
raise ValueError(
...<2 lines>...
)
ValueError: Cannot instantiate this tokenizer from a slow version. If it's based on sentencepiece, make sure you have sentencepiece installed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment