apple mlx for Phi-3.5-mini-instruct

main.py

from mlx_lm import load, generate

model, tokenizer = load(
    "microsoft/Phi-3.5-mini-instruct",
    tokenizer_config={"eos_token": "<|end|>", "trust_remote_code": True},
)

messages = [
    {"role": "user", "content": "Hello, how are you?"},
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

response = generate(model, tokenizer, prompt=prompt, verbose=True, max_tokens=1024)

Until 2024/08/30, mlx doesn't support eos_token for phi-3.5, so we shall set it in the tokenizer_config by ourselves.