chore: document Cursor model name validation workaround
Explain mlx-openai-server served-model-name alias when HF ids are rejected. Co-authored-by: Cursor <cursoragent@cursor.com>docs/local-mlx-cursor
parent
e4ed08e3be
commit
c06666b7c8
|
|
@ -175,7 +175,7 @@ Official server docs: [`mlx_lm/SERVER.md`](https://github.com/ml-explore/mlx-lm/
|
||||||
4. Set:
|
4. Set:
|
||||||
- **Base URL:** `http://127.0.0.1:8080/v1` (the `/v1` suffix is required)
|
- **Base URL:** `http://127.0.0.1:8080/v1` (the `/v1` suffix is required)
|
||||||
- **API key:** any non-empty string (e.g. `local`)
|
- **API key:** any non-empty string (e.g. `local`)
|
||||||
5. **Add model** — paste the exact id from `/v1/models`, e.g. `mlx-community/Qwen2.5-Coder-32B-Instruct-4bit`.
|
5. **Add model** — see [If Cursor says “Model name is not valid”](#if-cursor-says-model-name-is-not-valid) below (often **not** the raw Hugging Face id).
|
||||||
6. Click **Verify** if available.
|
6. Click **Verify** if available.
|
||||||
7. In **Chat** (Cmd+L), select that model from the dropdown.
|
7. In **Chat** (Cmd+L), select that model from the dropdown.
|
||||||
|
|
||||||
|
|
@ -191,9 +191,55 @@ cloudflared tunnel --url http://127.0.0.1:8080
|
||||||
|
|
||||||
Set Cursor’s base URL to the tunnel URL with `/v1` appended (e.g. `https://….trycloudflare.com/v1`).
|
Set Cursor’s base URL to the tunnel URL with `/v1` appended (e.g. `https://….trycloudflare.com/v1`).
|
||||||
|
|
||||||
### If Cursor rejects long model names
|
### If Cursor says “Model name is not valid”
|
||||||
|
|
||||||
Use the id returned by `/v1/models` exactly. If the UI still fails, try **[`mlx-openai-server`](https://github.com/cubist38/mlx-openai-server)** with a short `--served-model-name`, or temporarily test with a smaller 7B model id.
|
This is **not** an MLX problem. Your server can return the correct id in `/v1/models` (e.g. `mlx-community/Qwen2.5-Coder-32B-Instruct-8bit`) while **Cursor still refuses to add it** — recent Cursor builds validate custom model names against an internal catalog before any request is sent. Slashes, `mlx-community/…`, and most Hugging Face repo ids fail with:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Model name is not valid: "mlx-community/Qwen2.5-Coder-32B-Instruct-8bit"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Workaround (recommended):** run **[`mlx-openai-server`](https://github.com/cubist38/mlx-openai-server)** with a **short alias** Cursor already recognizes, while loading your real weights in the background.
|
||||||
|
|
||||||
|
Requires **Python 3.11+** (separate from the Python 3.9 env `uv tool install mlx-lm` uses):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv tool install mlx-openai-server --python 3.12
|
||||||
|
```
|
||||||
|
|
||||||
|
Stop `mlx_lm.server` if it is on port 8080, then:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
mlx-openai-server launch \
|
||||||
|
--model-type lm \
|
||||||
|
--model-path mlx-community/Qwen2.5-Coder-32B-Instruct-8bit \
|
||||||
|
--served-model-name gpt-4o-mini \
|
||||||
|
--host 127.0.0.1 \
|
||||||
|
--port 8080
|
||||||
|
```
|
||||||
|
|
||||||
|
In **Cursor → Models**:
|
||||||
|
|
||||||
|
- **Base URL:** `http://127.0.0.1:8080/v1`
|
||||||
|
- **Add model:** `gpt-4o-mini` (the alias, not the Hugging Face path)
|
||||||
|
|
||||||
|
Verify with curl (should answer using your Qwen weights):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -s http://127.0.0.1:8080/v1/chat/completions \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{
|
||||||
|
"model": "gpt-4o-mini",
|
||||||
|
"messages": [{"role": "user", "content": "Reply with exactly: ok"}],
|
||||||
|
"max_tokens": 16
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
`mlx_lm.server` alone treats the request `model` field as a Hugging Face repo id, so `gpt-4o-mini` **fails** there (404 / “Repository Not Found”). Use `mlx-openai-server` when you need a Cursor-friendly alias.
|
||||||
|
|
||||||
|
**Other names to try in Cursor** if `gpt-4o-mini` is blocked on your build: `gpt-4o`, `gpt-4o-mini-2024-07-18`, or lowercase variants (some forum reports: casing matters). The alias must match what you pass in chat **and** what `--served-model-name` exposes on `/v1/models`.
|
||||||
|
|
||||||
|
**Ollama equivalent:** `ollama create gpt-4o-mini -f Modelfile` with `FROM qwen2.5-coder:…` — same idea, different runtime ([DEV walkthrough](https://dev.to/yann_ortodoro/running-qwen-25-coder-14b-locally-in-cursor-with-ollama-4436)).
|
||||||
|
|
||||||
## 6. Optional: start server on login
|
## 6. Optional: start server on login
|
||||||
|
|
||||||
|
|
@ -303,6 +349,7 @@ First response after cold start may be slow while weights load into memory.
|
||||||
| Wrong model / 404 | Model name must match `/v1/models` exactly |
|
| Wrong model / 404 | Model name must match `/v1/models` exactly |
|
||||||
| First request very slow | Normal — one-time load into unified memory |
|
| First request very slow | Normal — one-time load into unified memory |
|
||||||
| Poor code quality | Use **Instruct** variant; try 8-bit if you have RAM headroom |
|
| Poor code quality | Use **Instruct** variant; try 8-bit if you have RAM headroom |
|
||||||
|
| `Model name is not valid` | Cursor catalog validation — use `mlx-openai-server` + `--served-model-name gpt-4o-mini` (§5) |
|
||||||
| Cursor Verify fails | Tunnel with `cloudflared` or `ngrok`; use tunnel URL + `/v1` |
|
| Cursor Verify fails | Tunnel with `cloudflared` or `ngrok`; use tunnel URL + `/v1` |
|
||||||
| Agent unreliable | Expected — use local for chat/edits only |
|
| Agent unreliable | Expected — use local for chat/edits only |
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue