diff --git a/docs/dev/local-mlx-cursor.md b/docs/dev/local-mlx-cursor.md index 7b33ac7..f98272c 100644 --- a/docs/dev/local-mlx-cursor.md +++ b/docs/dev/local-mlx-cursor.md @@ -70,8 +70,18 @@ mlx_lm.server --help which mlx_lm.server # note path for LaunchAgent below ``` +If `which mlx_lm.server` prints **not found** after `uv tool install mlx-lm`, the binary is usually at `~/.local/bin/mlx_lm.server` but that directory is not on your `PATH`. Add to `~/.zshrc`: + +```bash +export PATH="$HOME/.local/bin:$PATH" +``` + +Then `source ~/.zshrc`, or run `~/.local/bin/mlx_lm.server --help` directly. + ## 2. Choose a model +### Defaults (recommended starting point) + | Role | Hugging Face repo id | Approx. RAM | |------|----------------------|-------------| | **Primary coder (recommended)** | `mlx-community/Qwen2.5-Coder-32B-Instruct-4bit` | ~18–22 GB | @@ -82,6 +92,43 @@ For Neon Sprawl (C# server tests, GDScript client, Bruno API collections), prefe Models download on first use into the Hugging Face cache. +### Model alternatives + +Qwen **Coder** models are a strong default for local codegen on Mac, but they are not the only worthwhile option. Rankings from public benchmarks (e.g. SWE-bench) change frequently; **try models on your own repo** before switching defaults. + +| Family | Example MLX repo id | Best for | Tradeoff vs Qwen2.5-Coder-32B | +|--------|---------------------|----------|-------------------------------| +| **Qwen2.5 Coder** (default above) | `mlx-community/Qwen2.5-Coder-32B-Instruct-4bit` | Stable, proven Cursor + MLX path | Baseline — not always newest benchmark leader | +| **Qwen3 Coder** | `mlx-community/Qwen3-Coder-Next-mxfp4` | Newer coding benchmarks; MoE efficiency on large Macs | Less “battle-tested” in guides; confirm id via `/v1/models` | +| **Qwen3 Coder (MoE)** | Search HF for `Qwen3-Coder-30B-A3B-Instruct-MLX` (e.g. lmstudio-community builds) | Strong code quality with fewer active params than dense 30B | Repo id varies by publisher — verify on [Hugging Face](https://huggingface.co/models?search=mlx+qwen3+coder) | +| **Gemma 4** | Search HF for `Gemma-4` + `MLX` + `4bit` | Local assistant balance; Apache 2.0 | Some 2026 comparisons put Qwen slightly ahead on pure coding | +| **DeepSeek Coder V2** | `mlx-community/DeepSeek-Coder-V2-Lite-Instruct-4bit-mlx` | Alternative coder; permissive license | Smaller ecosystem of Cursor walkthroughs | +| **DeepSeek R1 distill** | Search HF for `DeepSeek-R1-Distill` + `mlx` | Step-by-step debugging, “why does this fail?” | Weaker as a primary codegen model | +| **Llama 3.3 8B** | `mlx-community/Meta-Llama-3.3-8B-Instruct-4bit` (or current 8B MLX build) | Fast general chat; OK light edits | Noticeably weaker on code than dedicated coders | + +**Not practical as a full local Cursor daily driver** (API or huge weights): Kimi K2.x, GLM-5, full DeepSeek V3/V4 frontier models. Use cloud Cursor models for those workloads. + +**How to try an alternative:** use the same `mlx_lm.server` / Cursor flow; only change `--model` and the model name in Cursor settings. Example: + +```bash +mlx_lm.server --model mlx-community/Qwen3-Coder-Next-mxfp4 --host 127.0.0.1 --port 8080 +``` + +On **128 GB** unified memory, sensible experiments after the default works: + +1. **Qwen3 Coder** MLX build (likely best upgrade path in 2026). +2. Keep **Qwen2.5-Coder-7B** loaded or on a second port for quick questions. +3. Optional **R1 distill** when you want reasoning traces, not codegen. + +**Pick criteria (more useful than benchmark tables):** + +- MLX conversion exists under [mlx-community](https://huggingface.co/mlx-community) or a trusted MLX publisher. +- **Instruct** (or coder-instruct) weights, not base. +- Quality on *your* C#/GDScript files and test style in a 30-minute A/B. +- RAM at 4-bit fits comfortably alongside IDE + Godot + server. + +Search for new conversions: `https://huggingface.co/models?search=mlx+coder` + ## 3. Download and smoke test ```bash @@ -250,6 +297,7 @@ First response after cold start may be slow while weights load into memory. | Symptom | What to try | |---------|-------------| +| `mlx_lm.server` not found | `uv tool install mlx-lm`; add `$HOME/.local/bin` to `PATH` (see §1); or `~/.local/bin/mlx_lm.server` | | Connection refused | Confirm `mlx_lm.server` is running; `curl http://127.0.0.1:8080/v1/models` | | Wrong model / 404 | Model name must match `/v1/models` exactly | | First request very slow | Normal — one-time load into unified memory | @@ -257,12 +305,14 @@ First response after cold start may be slow while weights load into memory. | Cursor Verify fails | Tunnel with `cloudflared` or `ngrok`; use tunnel URL + `/v1` | | Agent unreliable | Expected — use local for chat/edits only | -## Alternatives +## Alternative runtimes (not MLX) + +Same Cursor setup (OpenAI-compatible `/v1`); different local stack: | Tool | Role | |------|------| | **[Ollama](https://ollama.com)** | Easier onboarding; may not use MLX under the hood on Mac | -| **[LM Studio](https://lmstudio.ai)** | GUI; local server often at `http://localhost:1234/v1` | +| **[LM Studio](https://lmstudio.ai)** | GUI; local server often at `http://localhost:1234/v1`; many MLX builds | | **`mlx-openai-server`** | OpenAI-compatible wrapper with extra options (short model aliases, multimodal) | ## References