diff --git a/docs/dev/local-mlx-cursor.md b/docs/dev/local-mlx-cursor.md index eabad4d..f7c8345 100644 --- a/docs/dev/local-mlx-cursor.md +++ b/docs/dev/local-mlx-cursor.md @@ -175,7 +175,7 @@ Official server docs: [`mlx_lm/SERVER.md`](https://github.com/ml-explore/mlx-lm/ 4. Set: - **Base URL:** `http://127.0.0.1:8080/v1` (the `/v1` suffix is required) - **API key:** any non-empty string (e.g. `local`) -5. **Add model** — paste the exact id from `/v1/models`, e.g. `mlx-community/Qwen2.5-Coder-32B-Instruct-4bit`. +5. **Add model** — see [If Cursor says “Model name is not valid”](#if-cursor-says-model-name-is-not-valid) below (often **not** the raw Hugging Face id). 6. Click **Verify** if available. 7. In **Chat** (Cmd+L), select that model from the dropdown. @@ -191,9 +191,55 @@ cloudflared tunnel --url http://127.0.0.1:8080 Set Cursor’s base URL to the tunnel URL with `/v1` appended (e.g. `https://….trycloudflare.com/v1`). -### If Cursor rejects long model names +### If Cursor says “Model name is not valid” -Use the id returned by `/v1/models` exactly. If the UI still fails, try **[`mlx-openai-server`](https://github.com/cubist38/mlx-openai-server)** with a short `--served-model-name`, or temporarily test with a smaller 7B model id. +This is **not** an MLX problem. Your server can return the correct id in `/v1/models` (e.g. `mlx-community/Qwen2.5-Coder-32B-Instruct-8bit`) while **Cursor still refuses to add it** — recent Cursor builds validate custom model names against an internal catalog before any request is sent. Slashes, `mlx-community/…`, and most Hugging Face repo ids fail with: + +```text +Model name is not valid: "mlx-community/Qwen2.5-Coder-32B-Instruct-8bit" +``` + +**Workaround (recommended):** run **[`mlx-openai-server`](https://github.com/cubist38/mlx-openai-server)** with a **short alias** Cursor already recognizes, while loading your real weights in the background. + +Requires **Python 3.11+** (separate from the Python 3.9 env `uv tool install mlx-lm` uses): + +```bash +uv tool install mlx-openai-server --python 3.12 +``` + +Stop `mlx_lm.server` if it is on port 8080, then: + +```bash +mlx-openai-server launch \ + --model-type lm \ + --model-path mlx-community/Qwen2.5-Coder-32B-Instruct-8bit \ + --served-model-name gpt-4o-mini \ + --host 127.0.0.1 \ + --port 8080 +``` + +In **Cursor → Models**: + +- **Base URL:** `http://127.0.0.1:8080/v1` +- **Add model:** `gpt-4o-mini` (the alias, not the Hugging Face path) + +Verify with curl (should answer using your Qwen weights): + +```bash +curl -s http://127.0.0.1:8080/v1/chat/completions \ + -H "Content-Type: application/json" \ + -d '{ + "model": "gpt-4o-mini", + "messages": [{"role": "user", "content": "Reply with exactly: ok"}], + "max_tokens": 16 + }' +``` + +`mlx_lm.server` alone treats the request `model` field as a Hugging Face repo id, so `gpt-4o-mini` **fails** there (404 / “Repository Not Found”). Use `mlx-openai-server` when you need a Cursor-friendly alias. + +**Other names to try in Cursor** if `gpt-4o-mini` is blocked on your build: `gpt-4o`, `gpt-4o-mini-2024-07-18`, or lowercase variants (some forum reports: casing matters). The alias must match what you pass in chat **and** what `--served-model-name` exposes on `/v1/models`. + +**Ollama equivalent:** `ollama create gpt-4o-mini -f Modelfile` with `FROM qwen2.5-coder:…` — same idea, different runtime ([DEV walkthrough](https://dev.to/yann_ortodoro/running-qwen-25-coder-14b-locally-in-cursor-with-ollama-4436)). ## 6. Optional: start server on login @@ -303,6 +349,7 @@ First response after cold start may be slow while weights load into memory. | Wrong model / 404 | Model name must match `/v1/models` exactly | | First request very slow | Normal — one-time load into unified memory | | Poor code quality | Use **Instruct** variant; try 8-bit if you have RAM headroom | +| `Model name is not valid` | Cursor catalog validation — use `mlx-openai-server` + `--served-model-name gpt-4o-mini` (§5) | | Cursor Verify fails | Tunnel with `cloudflared` or `ngrok`; use tunnel URL + `/v1` | | Agent unreliable | Expected — use local for chat/edits only |