HNGlance

Top Best New Show

inferMLX: Simple Llama Model LLM Inference in macOS with MLX

https://github.com/peterc/infermlx

by petercooper • 1 year ago

2 1 1 year ago

GI

Related Stories

Loading related stories...

Comments

1

petercooper• 1 year ago

A distilled, refactored, and packaged variant of the inference tool from the MLX project. It runs on macOS if you have Python with the key feature being able to easily manipulate inference.

For example, it lets you make DeepSeek Llama 8B 'think' for longer by suppressing its thought closing tokens, or you can limit its thinking process to a certain number of tokens in just a few lines of code. This helps the smaller model solve things it gets wrong out of the box.

Source preview

github.com

HNGlance

© 2024-2026 HN Glance. Story content belongs to its original sources.

Privacy
Terms