chown

Shameless plug: if someone wants to try it in a nice ui, you could give Msty[1] a try. It's private and local.

[1] https://msty.ai

Save text as JSON as comments but the file itself is a PNG so that you can use it as an image (like previewing it) as they would ignore the comments. However, the OP’s editor can load the file back, parse the comments, and get the original data and continue to edit. Just one file to maintain. Quite clever actually.

I totally agree with this. I wanted to make it really easy for non-technical users with an app that hid all the complexities. I basically just wanted to embed the engine without making users open their terminal, let alone make them configure. I started with llama.cpp amd almost gave up on the idea before I stumbled upon Ollama, which made the app happen[1]

There are many flaws in Ollama but it makes many things much easier esp. if you don’t want to bother building and configuring. They do take a long time to merge any PRs though. One of my PRs has been waiting for 8 months and there was this another PR about KV cache quantization that took them 6 months to merge.

[1] https://msty.app

I wanted to present it as - “Msty As x Alternative”. But I think maybe I like Compare better. I will think about changing it. Thank you for the suggestion.

If anyone is looking for a one click solution without having to have a Docker running, try Msty - something that I have been working on for almost a year. Has RAG and Web Search built in among others and can connect to your Obsidian vaults as well.

https://msty.app

I am looking for a 55” 4K OLED. Do you have a recommendation? And are there any technical caveats with it? (I use a Mac primarily). Thank you

I am working on making using LLMs easy for everyone. Focus is to make users productive rather than having them configuring their environment to death. Wrote this app for my wife who, even as a smart software engineer, was getting overwhelmed with AI and LLMs.

https://Msty.app

I've recently started using Arc mostly out of FOMO and there are parts that I like and parts I don't. I have been a Firefox user for a long time before that so glad to see something similar that's one of the major priorities is the aesthetics and I very much appreciate that.

I tried to use it on my macOS Apple Silicone and got an error about it being broken and macOS suggested to trash it. Not sure if it is a bug issue. Will come back and try it again though to give it a second chance :)

Wow! The benchmarks are truly impressive, showing significant improvements across almost all categories. It's fascinating to see how rapidly this field is evolving. If someone had told me last year that Meta would be leading the charge in open-source models, I probably wouldn't have believed them. Yet here we are, witnessing Meta's substantial contributions to AI research and democratization.

On a related note, for those interested in experimenting with large language models locally, I've been working on an app called Msty [1]. It allows you to run models like this with just one click and features a clean, functional interface. Just added support for both 8B and 70B. Still in development, but I'd appreciate any feedback.

[1] https://msty.app

If you are still looking for it, I just made it available on an app[1] that I am working on with Gemma2 support.

https://msty.app

This is a great release! If you are looking to try it locally with a great interface, I am working on an app [1] and I just pushed an update to support Gemma2.

1: https://msty.app

I am working on something similar [1] that you might like. It doesn't have the canvas view but that along with a git graph like UI is something that I haven't thinking of adding.

1: https://Msty.app

I am working on an app [1] that does very similar as far as the branching goes (minus the right hand side visual which I have plans to support something very similar but along with a git like graph).

1: https://msty.app

Yes, if CUDA drivers are installed it should pick it up.

If anyone is looking to try 7B locally really quick, we have just added it to Msty.

[1] https://msty.app

You can try Msty as well. I am the author.

https://msty.app

For an app that I'm working on[1], I was using LangChain's Ollama integration and it was a headache. Things wouldn't work as documented as they were either wrong or outdated. Then when I had enough, I replaced it with the official Ollama JS library and so far it has been a great experience. It just works without any ceremony.

[1] https://msty.app

It will be paid down the road, but we are not there yet. It’s all offline, data is locally saved. You own it, we don’t have it even if you ask for it.

I am the author of Msty [1]. My goal is to make it as straightforward as possible with just one click (once you download the app). If you end up trying it, I would love to hear your feedback.

1: https://msty.app

I am the author of Msty [1]. My goal is to make it as straightforward as possible with just one click (once you download the app). If you try it, let me know what you think.

1: https://msty.app

Not right now but that’s something we plan to support soon. Supporting Ollama downloaded models is getting released either today or tomorrow, gguf support might go into the next release. Would love to chat with you to learn more about your use case. Mind saying hi on our Discord?

Wow! This is already very helpful and was the kind of feedback I was looking for. Thank you!

That's on the top of our list. It got pushed back because we want to support creating a character/profile (basically select a model and apply some defaults including a system prompt). But I feel like that was a mistake tomwait for it. Regardless, it is getting added in the next release (the one after something that is dropping in a day or 2, which is a big release in itself)

I was actually looking for one! What's the best way to reach you? Mind jumping on our Discord so that I can share the installer with you soon?

It's very nice that there exists something like that. I am an author of one of the similar apps [1] someone listed in a different thread. I was hoping I could get in touch with someone like you who could give me some feedback on how to make my app more accessible for users like you. I really want to it be an "LLM for all" kind of app but despite my best efforts and intention, I suck at it. Any chance of getting in touch with you and get some feedback? Only if you want and have time, no pressure at all.

[1] https://msty.app

Author of Msty here. Not yet but I am already working on the design for it to be added in very near future. I am happy to chat more with you to understand your needs and what you are looking in such apps. Please hop on the Discord if you don't mind :)

Give it a try and see how you feel. "Yes, it will" be a dishonest answer to be completely honest at least at this point. The app has been out for just about a month and I am still working in it. I would love a user like you to give it a try and give me some feedback (please). I am very active on our Discord if you want to get in touch (just mention your HN username and I will wace).

Hi. I'm the author of Msty app, 2nd on the list above. You are right about average users likely choosing ChatGPT over local models. My wife was the first and the biggest user of my app. A software engineer by profession and training but she likes to not worry about LLM world and just to use it as a tool that makes you more productive. As soon as she took Msty for a ride, I realized that some users, despite their background, care about online models. This actually led me adding support for online models right away. However, she really likes to make use of the parallel chat feature and uses both Mistral and ChatGPT models to give same prompt and then compare the output and choose the best answer (or sometimes make a hybrid choice). She says that being able to compare multiple outputs like that is a tremendously helpful. But that's the extent of local LLMs for her. So far my effort has been to target a bit higher than the average users while making it approachable for more advanced users as well.

I am the author of Msty app mentioned here. So humbled to see an app that is just about a month old that I mostly wrote for my wife and some friends to begin with (who got overwhelmed with everything that was going in LLM world), on the top of your list. Thank you!

Thanks! If you go to where you installed the model from and click on the download button, you can install additional models or remove installed models.

Now that I think of it, it could be a bit confusing. Thanks for asking, I feel like I need to improve this a bit.