chore: document Cursor model name validation workaround
Explain mlx-openai-server served-model-name alias when HF ids are rejected. Co-authored-by: Cursor <cursoragent@cursor.com>docs/local-mlx-cursor
parent
e4ed08e3be
commit
c06666b7c8
|
|
@ -175,7 +175,7 @@ Official server docs: [`mlx_lm/SERVER.md`](https://github.com/ml-explore/mlx-lm/
|
|||
4. Set:
|
||||
- **Base URL:** `http://127.0.0.1:8080/v1` (the `/v1` suffix is required)
|
||||
- **API key:** any non-empty string (e.g. `local`)
|
||||
5. **Add model** — paste the exact id from `/v1/models`, e.g. `mlx-community/Qwen2.5-Coder-32B-Instruct-4bit`.
|
||||
5. **Add model** — see [If Cursor says “Model name is not valid”](#if-cursor-says-model-name-is-not-valid) below (often **not** the raw Hugging Face id).
|
||||
6. Click **Verify** if available.
|
||||
7. In **Chat** (Cmd+L), select that model from the dropdown.
|
||||
|
||||
|
|
@ -191,9 +191,55 @@ cloudflared tunnel --url http://127.0.0.1:8080
|
|||
|
||||
Set Cursor’s base URL to the tunnel URL with `/v1` appended (e.g. `https://….trycloudflare.com/v1`).
|
||||
|
||||
### If Cursor rejects long model names
|
||||
### If Cursor says “Model name is not valid”
|
||||
|
||||
Use the id returned by `/v1/models` exactly. If the UI still fails, try **[`mlx-openai-server`](https://github.com/cubist38/mlx-openai-server)** with a short `--served-model-name`, or temporarily test with a smaller 7B model id.
|
||||
This is **not** an MLX problem. Your server can return the correct id in `/v1/models` (e.g. `mlx-community/Qwen2.5-Coder-32B-Instruct-8bit`) while **Cursor still refuses to add it** — recent Cursor builds validate custom model names against an internal catalog before any request is sent. Slashes, `mlx-community/…`, and most Hugging Face repo ids fail with:
|
||||
|
||||
```text
|
||||
Model name is not valid: "mlx-community/Qwen2.5-Coder-32B-Instruct-8bit"
|
||||
```
|
||||
|
||||
**Workaround (recommended):** run **[`mlx-openai-server`](https://github.com/cubist38/mlx-openai-server)** with a **short alias** Cursor already recognizes, while loading your real weights in the background.
|
||||
|
||||
Requires **Python 3.11+** (separate from the Python 3.9 env `uv tool install mlx-lm` uses):
|
||||
|
||||
```bash
|
||||
uv tool install mlx-openai-server --python 3.12
|
||||
```
|
||||
|
||||
Stop `mlx_lm.server` if it is on port 8080, then:
|
||||
|
||||
```bash
|
||||
mlx-openai-server launch \
|
||||
--model-type lm \
|
||||
--model-path mlx-community/Qwen2.5-Coder-32B-Instruct-8bit \
|
||||
--served-model-name gpt-4o-mini \
|
||||
--host 127.0.0.1 \
|
||||
--port 8080
|
||||
```
|
||||
|
||||
In **Cursor → Models**:
|
||||
|
||||
- **Base URL:** `http://127.0.0.1:8080/v1`
|
||||
- **Add model:** `gpt-4o-mini` (the alias, not the Hugging Face path)
|
||||
|
||||
Verify with curl (should answer using your Qwen weights):
|
||||
|
||||
```bash
|
||||
curl -s http://127.0.0.1:8080/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "gpt-4o-mini",
|
||||
"messages": [{"role": "user", "content": "Reply with exactly: ok"}],
|
||||
"max_tokens": 16
|
||||
}'
|
||||
```
|
||||
|
||||
`mlx_lm.server` alone treats the request `model` field as a Hugging Face repo id, so `gpt-4o-mini` **fails** there (404 / “Repository Not Found”). Use `mlx-openai-server` when you need a Cursor-friendly alias.
|
||||
|
||||
**Other names to try in Cursor** if `gpt-4o-mini` is blocked on your build: `gpt-4o`, `gpt-4o-mini-2024-07-18`, or lowercase variants (some forum reports: casing matters). The alias must match what you pass in chat **and** what `--served-model-name` exposes on `/v1/models`.
|
||||
|
||||
**Ollama equivalent:** `ollama create gpt-4o-mini -f Modelfile` with `FROM qwen2.5-coder:…` — same idea, different runtime ([DEV walkthrough](https://dev.to/yann_ortodoro/running-qwen-25-coder-14b-locally-in-cursor-with-ollama-4436)).
|
||||
|
||||
## 6. Optional: start server on login
|
||||
|
||||
|
|
@ -303,6 +349,7 @@ First response after cold start may be slow while weights load into memory.
|
|||
| Wrong model / 404 | Model name must match `/v1/models` exactly |
|
||||
| First request very slow | Normal — one-time load into unified memory |
|
||||
| Poor code quality | Use **Instruct** variant; try 8-bit if you have RAM headroom |
|
||||
| `Model name is not valid` | Cursor catalog validation — use `mlx-openai-server` + `--served-model-name gpt-4o-mini` (§5) |
|
||||
| Cursor Verify fails | Tunnel with `cloudflared` or `ngrok`; use tunnel URL + `/v1` |
|
||||
| Agent unreliable | Expected — use local for chat/edits only |
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue