neon-sprawl/docs/dev/local-mlx-cursor.md

Local MLX models with Cursor (macOS)

Status: Developer guide (not product architecture).
Audience: Neon Sprawl contributors on Apple Silicon who want private, offline-capable chat and inline edits in Cursor, backed by a local model.

Cursor does not talk to MLX directly. It needs an OpenAI-compatible HTTP API (/v1/chat/completions). This guide uses Apple’s MLX runtime via mlx-lm and its built-in server.

What works in Cursor

Feature	Local MLX
Chat (Cmd+L)	Yes
Inline edit (Cmd+K)	Usually yes
Composer / Agent	Limited — local models are weaker at multi-step tool use; prefer cloud models for large refactors
Tab autocomplete	No — Cursor tab completion uses separate proprietary models

Use local MLX for privacy, offline use, and cost. Keep cloud models for Agent mode, tab completion, and multi-file story work.

Architecture

flowchart LR
  subgraph cursor [Cursor IDE]
    Chat[Chat Cmd+L]
    Edit[Inline edit Cmd+K]
  end

  subgraph mac [Mac Apple Silicon]
    API["mlx_lm.server :8080/v1"]
    MLX[MLX runtime]
    Model["Qwen2.5-Coder-32B-Instruct-4bit"]
  end

  Chat --> API
  Edit --> API
  API --> MLX --> Model

Prerequisites

• macOS with Apple Silicon (MLX is not for Intel Macs).
• Python 3.12+ (Homebrew or uv).
• Enough disk for model weights (~18 GB for the recommended 32B 4-bit coder).
• Cursor with permission to override the OpenAI base URL (Settings → Models).

Rough RAM for the recommended model: ~18–22 GB active during inference. A Mac with 48 GB+ unified memory is comfortable; 128 GB can also run 8-bit variants for higher quality.

1. Install mlx-lm

Option A — uv (isolated tool install):

brew install uv
uv tool install mlx-lm

Option B — venv + pip:

brew install python@3.12
python3.12 -m venv ~/.local/venvs/mlx-lm
source ~/.local/venvs/mlx-lm/bin/activate
pip install mlx-lm

Verify:

mlx_lm.server --help
which mlx_lm.server   # note path for LaunchAgent below

2. Choose a model

Role	Hugging Face repo id	Approx. RAM
Primary coder (recommended)	mlx-community/Qwen2.5-Coder-32B-Instruct-4bit	~18–22 GB
Higher quality (optional)	mlx-community/Qwen2.5-Coder-32B-Instruct-8bit	~32–35 GB
Fast / small (optional)	mlx-community/Qwen2.5-Coder-7B-Instruct-4bit	~5 GB

For Neon Sprawl (C# server tests, GDScript client, Bruno API collections), prefer the Instruct coder variant — not the non-instruct base weights.

Models download on first use into the Hugging Face cache.

3. Download and smoke test

mlx_lm.generate \
  --model mlx-community/Qwen2.5-Coder-32B-Instruct-4bit \
  --max-tokens 64 \
  --prompt "Write a one-line C# xUnit test skeleton."

4. Start the OpenAI-compatible server

mlx_lm.server \
  --model mlx-community/Qwen2.5-Coder-32B-Instruct-4bit \
  --host 127.0.0.1 \
  --port 8080

Leave this process running while using Cursor.

Verify the API:

curl -s http://127.0.0.1:8080/v1/models | python3 -m json.tool

curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mlx-community/Qwen2.5-Coder-32B-Instruct-4bit",
    "messages": [{"role": "user", "content": "Say hello in one sentence."}],
    "max_tokens": 64
  }' | python3 -m json.tool

Use the exact "id" from /v1/models as the model name in Cursor.

Official server docs: mlx_lm/SERVER.md.

5. Configure Cursor

1. Open Cursor Settings (Cmd+,).
2. Go to Models.
3. Enable Override OpenAI Base URL (wording may vary: “use own API key”, custom OpenAI endpoint).
4. Set:
   • Base URL: http://127.0.0.1:8080/v1 (the /v1 suffix is required)
   • API key: any non-empty string (e.g. local)
5. Add model — paste the exact id from /v1/models, e.g. mlx-community/Qwen2.5-Coder-32B-Instruct-4bit.
6. Click Verify if available.
7. In Chat (Cmd+L), select that model from the dropdown.

If Verify fails on localhost

Some Cursor builds restrict 127.0.0.1. Expose the same server through a tunnel:

# Server still on 8080; in another terminal:
brew install cloudflared
cloudflared tunnel --url http://127.0.0.1:8080

Set Cursor’s base URL to the tunnel URL with /v1 appended (e.g. https://….trycloudflare.com/v1).

If Cursor rejects long model names

Use the id returned by /v1/models exactly. If the UI still fails, try mlx-openai-server with a short --served-model-name, or temporarily test with a smaller 7B model id.

6. Optional: start server on login

Replace MLX_LM_SERVER with the output of which mlx_lm.server.

Save as ~/Library/LaunchAgents/com.neon-sprawl.mlx-coder.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>com.neon-sprawl.mlx-coder</string>
  <key>ProgramArguments</key>
  <array>
    <string>MLX_LM_SERVER</string>
    <string>--model</string>
    <string>mlx-community/Qwen2.5-Coder-32B-Instruct-4bit</string>
    <string>--host</string>
    <string>127.0.0.1</string>
    <string>--port</string>
    <string>8080</string>
  </array>
  <key>RunAtLoad</key>
  <true/>
  <key>KeepAlive</key>
  <true/>
  <key>StandardOutPath</key>
  <string>/tmp/mlx-coder-server.log</string>
  <key>StandardErrorPath</key>
  <string>/tmp/mlx-coder-server.err</string>
</dict>
</plist>

launchctl load ~/Library/LaunchAgents/com.neon-sprawl.mlx-coder.plist

7. Day-one helper script

Save as ~/bin/start-mlx-coder.sh (or anywhere on your PATH):

#!/usr/bin/env bash
set -euo pipefail
MODEL="mlx-community/Qwen2.5-Coder-32B-Instruct-4bit"
PORT=8080

echo "Warming $MODEL (first run downloads ~18GB)..."
mlx_lm.generate --model "$MODEL" --max-tokens 1 --prompt "ok" >/dev/null

echo "Starting OpenAI-compatible server on http://127.0.0.1:$PORT/v1"
exec mlx_lm.server --model "$MODEL" --host 127.0.0.1 --port "$PORT"

chmod +x ~/bin/start-mlx-coder.sh
~/bin/start-mlx-coder.sh

Then point Cursor at http://127.0.0.1:8080/v1.

8. Neon Sprawl usage patterns

Good fits for local MLX

• Explain a server type, test, or Bruno request.
• Draft a small GdUnit skeleton or C# AAA test outline.
• Answer “why might this flake?” on an open file.

Prefer cloud Cursor models

• Agent mode across many files.
• Story-sized implementation (Linear NEO-* branches).
• Tab completion.

Optional Cursor rule

If local models hallucinate edits, add a project rule (e.g. .cursor/rules/local-mlx.md):

- Prefer small, file-scoped changes.
- For edits, show a unified diff or exact replacement block.
- Do not claim tools ran or tests passed unless shown.
- Match existing C# AAA test layout and GdUnit `# Arrange` / `# Act` / `# Assert`.

9. Sanity-check prompts in Cursor

With the MLX model selected in Chat:

1. “Summarize the open implementation plan in three bullets.”
2. “Suggest only the Assert section for this xUnit test — no other changes.”
3. “Minimal GdUnit test for a node that emits health_changed.”

First response after cold start may be slow while weights load into memory.

10. Troubleshooting

Symptom	What to try
Connection refused	Confirm mlx_lm.server is running; curl http://127.0.0.1:8080/v1/models
Wrong model / 404	Model name must match /v1/models exactly
First request very slow	Normal — one-time load into unified memory
Poor code quality	Use Instruct variant; try 8-bit if you have RAM headroom
Cursor Verify fails	Tunnel with cloudflared or ngrok; use tunnel URL + /v1
Agent unreliable	Expected — use local for chat/edits only

Alternatives

Tool	Role
Ollama	Easier onboarding; may not use MLX under the hood on Mac
LM Studio	GUI; local server often at http://localhost:1234/v1
mlx-openai-server	OpenAI-compatible wrapper with extra options (short model aliases, multimodal)

References

• MLX LM server: github.com/ml-explore/mlx-lm — mlx_lm/SERVER.md
• Recommended weights: mlx-community/Qwen2.5-Coder-32B-Instruct-4bit
• Repo stack context: docs/architecture/tech_stack.md