One email. No follow-up unless I ship something that fits.
WHAT YOU NEED
An Apple Silicon Mac. M1 or later, macOS 14 or newer. It runs on MLX, Apple's own framework, so Intel Macs, Windows and Linux are out
Python 3.11 or newer
About 850 MB of disk. The default model downloads from Hugging Face on first load. On my connection that took about 30 seconds, once
Nothing else. No PyTorch, no API key, no account
INSTALL
One package, one call
Install it, load the default English model, and ask it a typed question. The first load downloads the weights; every call after that is local.
terminalthe install
$ pip install laya-mlx
pythonfirst decision
import laya_mlx as layaagent = laya.load("aac6fef/laya-mlx") result = agent.predict( "I was billed twice. Please refund the duplicate.", {"department": {"type": "choice", "instructions": "Who should handle this?", "criteria": ["billing", "technical", "sales"]}}, ) print(result["answers"]["department"]["choice"])→ billing
Prove itbilling with a probability around 0.89 means it works. I ran five different tickets through this exact call and all five went to the right team, at about 11 ms each.
CONFIGURE
Pick the model, then leave the rest alone
There are three published models. The choice matters more than any setting, because it decides how much of a message the model can read.
model
reads up to
pick it when
aac6fef/laya-mlx
512 tokens
short English messages: tickets, form fields, chat lines
aac6fef/laya-multilingual-mlx
1,024 tokens
anything not in English, or longer messages. The project measures it as the faster one too
aac6fef/laya-typed-decisions-mlx
1,024 tokens
the upstream typed-decisions workflows
Defaults worth knowing: dtype="float16" (use float32 only if you need closer numbers, not better answers), batch_size=16 questions per pass. compile=True, pad_to_multiple=16 and cache_prompts=True are off by default; turn them on only for a loop that asks the same questions thousands of times, because compilation costs time on first use.
DAILY DRIVING
Three kinds of question
Laya doesn't write text. You give it a message and typed questions, and it returns probabilities. Ask several questions at once: they run in the same call.
My ruleRead answer_confidence on every answer and send anything under 0.7 to a person. In my five-ticket test the two lowest were a sales question at 0.59 and a login problem at 0.67. Both were right, but those are the ones to check. The project is clear that confidence does not guarantee accuracy.
WHEN IT DRIFTS
What goes wrong, and the fix
Long emails come back unsure
The English model reads 512 tokens and keeps the start of a text message. I padded a ticket so the request sat at the end: 202 tokens were dropped and confidence fell from 0.96 to 0.44. Check usage.truncated, switch to the multilingual model (1,024 tokens), or pass a conversation list, which keeps the newest turns.
Non-English text gets odd answers
The English models are not a substitute for the multilingual one. Use laya.Router, which detects the language and picks the right model.
Hundreds of options in one choice
They share one token budget, so each label gets a few tokens. Use laya.predict_shortlist to keep the top 20 first.
A RuntimeWarning on load
Expected. Since 0.3 it clamps a calibration value that made coin flips look certain, and names it. Nothing to fix.
THE SHAPE OF IT
Where everything lives
~/.cache/huggingface/the downloaded model weights, about 850 MB each examples/state.jsona sample message to test against examples/questions.jsonsample typed questions, CLI ready BENCHMARKS.mdthe project's timings and accuracy runs laya_mlx/snake/the Snake demo, every move a Laya call
The command line does the same as the Python API: laya-mlx predict --model aac6fef/laya-mlx --state-file examples/state.json --questions examples/questions.json from a clone of the repo.
That is the whole thing
One install, one model download, and the small yes, no and which-one calls in your app stop going to the cloud. They come back in milliseconds, on your Mac, for nothing.
I'm Raj Singh, founder of DebtOps.ai and a Claude certified partner. Over the last five years I've managed $25M in ads, built AI solutions for Fortune 100 companies, and installed AI operating systems and AI brains inside small businesses. Everything here is something I've shipped, debugged, or watched a client ship. My goal with AgileSingh is to make you irreplaceable in this AI age.
