# Local MLX models with Cursor (macOS) **Status:** Developer guide (not product architecture). **Audience:** Neon Sprawl contributors on Apple Silicon who want **private, offline-capable** chat and inline edits in Cursor, backed by a local model. Cursor does not talk to MLX directly. It needs an **OpenAI-compatible HTTP API** (`/v1/chat/completions`). This guide uses Apple’s **[MLX](https://github.com/ml-explore/mlx)** runtime via **[`mlx-lm`](https://github.com/ml-explore/mlx-lm)** and its built-in server. ## What works in Cursor | Feature | Local MLX | |---------|-----------| | **Chat** (Cmd+L) | Yes | | **Inline edit** (Cmd+K) | Usually yes | | **Composer / Agent** | Limited — local models are weaker at multi-step tool use; prefer cloud models for large refactors | | **Tab autocomplete** | No — Cursor tab completion uses separate proprietary models | Use local MLX for **privacy, offline use, and cost**. Keep **cloud models** for Agent mode, tab completion, and multi-file story work. ## Architecture ```mermaid flowchart LR subgraph cursor [Cursor IDE] Chat[Chat Cmd+L] Edit[Inline edit Cmd+K] end subgraph mac [Mac Apple Silicon] API["mlx_lm.server :8080/v1"] MLX[MLX runtime] Model["Qwen2.5-Coder-32B-Instruct-4bit"] end Chat --> API Edit --> API API --> MLX --> Model ``` ## Prerequisites - **macOS** with Apple Silicon (MLX is not for Intel Macs). - **Python 3.12+** (Homebrew or [uv](https://github.com/astral-sh/uv)). - Enough disk for model weights (~18 GB for the recommended 32B 4-bit coder). - **Cursor** with permission to override the OpenAI base URL (Settings → Models). Rough RAM for the recommended model: **~18–22 GB** active during inference. A Mac with **48 GB+** unified memory is comfortable; **128 GB** can also run 8-bit variants for higher quality. ## 1. Install `mlx-lm` **Option A — uv (isolated tool install):** ```bash brew install uv uv tool install mlx-lm ``` **Option B — venv + pip:** ```bash brew install python@3.12 python3.12 -m venv ~/.local/venvs/mlx-lm source ~/.local/venvs/mlx-lm/bin/activate pip install mlx-lm ``` Verify: ```bash mlx_lm.server --help which mlx_lm.server # note path for LaunchAgent below ``` ## 2. Choose a model | Role | Hugging Face repo id | Approx. RAM | |------|----------------------|-------------| | **Primary coder (recommended)** | `mlx-community/Qwen2.5-Coder-32B-Instruct-4bit` | ~18–22 GB | | **Higher quality (optional)** | `mlx-community/Qwen2.5-Coder-32B-Instruct-8bit` | ~32–35 GB | | **Fast / small (optional)** | `mlx-community/Qwen2.5-Coder-7B-Instruct-4bit` | ~5 GB | For Neon Sprawl (C# server tests, GDScript client, Bruno API collections), prefer the **Instruct** coder variant — not the non-instruct base weights. Models download on first use into the Hugging Face cache. ## 3. Download and smoke test ```bash mlx_lm.generate \ --model mlx-community/Qwen2.5-Coder-32B-Instruct-4bit \ --max-tokens 64 \ --prompt "Write a one-line C# xUnit test skeleton." ``` ## 4. Start the OpenAI-compatible server ```bash mlx_lm.server \ --model mlx-community/Qwen2.5-Coder-32B-Instruct-4bit \ --host 127.0.0.1 \ --port 8080 ``` Leave this process running while using Cursor. Verify the API: ```bash curl -s http://127.0.0.1:8080/v1/models | python3 -m json.tool curl -s http://127.0.0.1:8080/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "mlx-community/Qwen2.5-Coder-32B-Instruct-4bit", "messages": [{"role": "user", "content": "Say hello in one sentence."}], "max_tokens": 64 }' | python3 -m json.tool ``` Use the exact `"id"` from `/v1/models` as the model name in Cursor. Official server docs: [`mlx_lm/SERVER.md`](https://github.com/ml-explore/mlx-lm/blob/main/mlx_lm/SERVER.md). ## 5. Configure Cursor 1. Open **Cursor Settings** (Cmd+,). 2. Go to **Models**. 3. Enable **Override OpenAI Base URL** (wording may vary: “use own API key”, custom OpenAI endpoint). 4. Set: - **Base URL:** `http://127.0.0.1:8080/v1` (the `/v1` suffix is required) - **API key:** any non-empty string (e.g. `local`) 5. **Add model** — paste the exact id from `/v1/models`, e.g. `mlx-community/Qwen2.5-Coder-32B-Instruct-4bit`. 6. Click **Verify** if available. 7. In **Chat** (Cmd+L), select that model from the dropdown. ### If Verify fails on localhost Some Cursor builds restrict `127.0.0.1`. Expose the same server through a tunnel: ```bash # Server still on 8080; in another terminal: brew install cloudflared cloudflared tunnel --url http://127.0.0.1:8080 ``` Set Cursor’s base URL to the tunnel URL with `/v1` appended (e.g. `https://….trycloudflare.com/v1`). ### If Cursor rejects long model names Use the id returned by `/v1/models` exactly. If the UI still fails, try **[`mlx-openai-server`](https://github.com/cubist38/mlx-openai-server)** with a short `--served-model-name`, or temporarily test with a smaller 7B model id. ## 6. Optional: start server on login Replace `MLX_LM_SERVER` with the output of `which mlx_lm.server`. Save as `~/Library/LaunchAgents/com.neon-sprawl.mlx-coder.plist`: ```xml Label com.neon-sprawl.mlx-coder ProgramArguments MLX_LM_SERVER --model mlx-community/Qwen2.5-Coder-32B-Instruct-4bit --host 127.0.0.1 --port 8080 RunAtLoad KeepAlive StandardOutPath /tmp/mlx-coder-server.log StandardErrorPath /tmp/mlx-coder-server.err ``` ```bash launchctl load ~/Library/LaunchAgents/com.neon-sprawl.mlx-coder.plist ``` ## 7. Day-one helper script Save as `~/bin/start-mlx-coder.sh` (or anywhere on your `PATH`): ```bash #!/usr/bin/env bash set -euo pipefail MODEL="mlx-community/Qwen2.5-Coder-32B-Instruct-4bit" PORT=8080 echo "Warming $MODEL (first run downloads ~18GB)..." mlx_lm.generate --model "$MODEL" --max-tokens 1 --prompt "ok" >/dev/null echo "Starting OpenAI-compatible server on http://127.0.0.1:$PORT/v1" exec mlx_lm.server --model "$MODEL" --host 127.0.0.1 --port "$PORT" ``` ```bash chmod +x ~/bin/start-mlx-coder.sh ~/bin/start-mlx-coder.sh ``` Then point Cursor at `http://127.0.0.1:8080/v1`. ## 8. Neon Sprawl usage patterns **Good fits for local MLX** - Explain a server type, test, or Bruno request. - Draft a small GdUnit skeleton or C# AAA test outline. - Answer “why might this flake?” on an open file. **Prefer cloud Cursor models** - Agent mode across many files. - Story-sized implementation (Linear `NEO-*` branches). - Tab completion. ### Optional Cursor rule If local models hallucinate edits, add a project rule (e.g. `.cursor/rules/local-mlx.md`): ```markdown - Prefer small, file-scoped changes. - For edits, show a unified diff or exact replacement block. - Do not claim tools ran or tests passed unless shown. - Match existing C# AAA test layout and GdUnit `# Arrange` / `# Act` / `# Assert`. ``` ## 9. Sanity-check prompts in Cursor With the MLX model selected in Chat: 1. *“Summarize the open implementation plan in three bullets.”* 2. *“Suggest only the Assert section for this xUnit test — no other changes.”* 3. *“Minimal GdUnit test for a node that emits `health_changed`.”* First response after cold start may be slow while weights load into memory. ## 10. Troubleshooting | Symptom | What to try | |---------|-------------| | Connection refused | Confirm `mlx_lm.server` is running; `curl http://127.0.0.1:8080/v1/models` | | Wrong model / 404 | Model name must match `/v1/models` exactly | | First request very slow | Normal — one-time load into unified memory | | Poor code quality | Use **Instruct** variant; try 8-bit if you have RAM headroom | | Cursor Verify fails | Tunnel with `cloudflared` or `ngrok`; use tunnel URL + `/v1` | | Agent unreliable | Expected — use local for chat/edits only | ## Alternatives | Tool | Role | |------|------| | **[Ollama](https://ollama.com)** | Easier onboarding; may not use MLX under the hood on Mac | | **[LM Studio](https://lmstudio.ai)** | GUI; local server often at `http://localhost:1234/v1` | | **`mlx-openai-server`** | OpenAI-compatible wrapper with extra options (short model aliases, multimodal) | ## References - MLX LM server: [github.com/ml-explore/mlx-lm — `mlx_lm/SERVER.md`](https://github.com/ml-explore/mlx-lm/blob/main/mlx_lm/SERVER.md) - Recommended weights: [mlx-community/Qwen2.5-Coder-32B-Instruct-4bit](https://huggingface.co/mlx-community/Qwen2.5-Coder-32B-Instruct-4bit) - Repo stack context: [`docs/architecture/tech_stack.md`](../architecture/tech_stack.md)