Context
In secure and constrained environments, sending data to a cloud service is often not an option. Models have to run on the device itself.
Inference engine
A narrow-focused inference engine for large language models, built for Apple Silicon and other unified memory architectures (UMA), where CPU, GPU and neural engine share one pool of memory. Concentrating on a small set of models and hardware makes it possible to optimise for them properly.
Agents
Agent-based workflows on top of local inference — useful work without a connection to the outside world.
Platform evaluation
- Benchmarking inference in the browser with WebGPU and WebNN across runtimes and models.
- Following hardware developments such as NPUs and dedicated accelerators.
- Assessing which current models are practical at the edge, and feeding the results into platform strategy.