HN user
If you want to take it even further a fully virtualized Opnsense with Proxmox is amazing. Your router can float between cluster nodes and each VLAN becomes a virtual interface in the hypervisor. What still blows my mind is how I can migrate the instance to a second server and bring the original server down for maintenance without my users noticing a thing.
Since you're releasing this under GPLv3 this is indeed open source. Having multiple license options does not affect this.
It's not a community per se but there's a lot of research and discussion going on directly in the llama.cpp repo (https://github.com/ggerganov/llama.cpp) if you're interested in the more technical side of things.
It certainly looks like it if you check their discussion board.
Likely it's hosted on the same infrastructure. The general recommendation is to host the status page on different provider so it's more likely to stay up.
It's not even source code, it's the fact that this just looks super sketchy in general. The website http://faction504.com/ listed just links directly to the Play Store page. There's no info on what Blackpage does or who runs it.
The issue with AMD and AI is, as always, the software stack. Even if the hardware is great ROCM simply doesn't have industry traction and accessiblity.
If you ask me to make an account before I can see your content (even if it looks interesting) that's a turn off and I'll go somewhere else. Maybe have a button at the top that says "explore as guest" or something like that?
Also you should advertise that this is an open-source project on the landing page, as that may cause more people to be interested in trying it out.
I don't like the centralization of Cloudflare, but it's the only way I can host public web pages out of my homelab with a DSL connection. I would rather have a CDN than put my content on a VPS as the servers remain 100% under my control and I can change providers anytime.
If we look at this from a technical perspective, what is the cost of running such a service?
Something like Facebook obviously costs a lot more to run, as it stores photos and video and also provides each user with an individually generated feed. In contrast HN is rather lightweight and basically serves mostly static cachable content to all users. I believe it was stated somewhere that the entire HN comfortably runs on a 64GB 4/8 bare metal server. On the extreme side I host my personal static web site on my home DSL connection fronted by Cloudflare, as the CDN does all the work and the bandwidth used is minimal.
To the people running these new communities the software cost is low as they're based off an open-source service of choice (Lemmy, Mastodon, etc.), and as we saw on Reddit moderation can be done by volunteers. Are the remaining costs for admins, hosting, etc. feasible for a non-profit with some user donations? And can we create more efficient platforms with a plainer style that will minimize the server and electricity costs?
Yeah as a long-time user I was really hoping for that myself, or at least have some sort of compromise. Certainly not that trainwreck of an AMA.
Wouldn't that be Hetzner?
It's not easy to move a group to a new platform, and there's very little people using Lemmy currently. The thing about Reddit despite its flaws is that it allows access to many large communities with a single account.
OpenLLAMA will be released soon and it's 100% compatible with the original LLAMA.
The fact that this is commodity hardware makes ggml extremely impressive and puts the tech in the hands of everyone. I recently reported my experience running 7B llama.cpp on a 15 year old Core 2 Quad [1] - when that machine came out it was a completely different world and I certainly never imagined how AI would look like today. This was around when the first iPhone was released and everyone began talking about how smartphones would become the next big thing. We saw what happened 15 years later...
Today with the new k-quants users are reporting that 30B models are working with 2-bit quantization on 16GB CPUs and GPUs [2]. That's enabling access to millions of consumers and the optimizations will only improve from there.
[1] https://old.reddit.com/r/LocalLLaMA/comments/13q6hu8/7b_perf...
[2] https://github.com/ggerganov/llama.cpp/pull/1684, https://old.reddit.com/r/LocalLLaMA/comments/141bdll/moneros...
Likely you can use a Bluetooth keyboard and mouse with it.
It also doesn't have ECC as well which was a staple of the previous Mac Pro line.
I personally like the Adminforge instance https://teddit.adminforge.de. It's much quicker than the original teddit.net.
Theoretically the ARC should work with llama.cpp using OpenCL, but I haven't seen benchmarks or even a confirmation that it works.
I think the tinybox is meant to be a training/inference server meant for tinygrad and filled with those AMD cards. Very likely it will run Linux.
I learned from the start on lead-free RoHS solder (doing SMT work) and had zero issues with it. Honestly I've never tried leaded solder as I just use lead-free all the time, though I know people who swear by it.
This is the reason why I still stick with OpenVPN on TCP 443 for my selfhosted VPN. Yes performance suffers a bit but it works absolutely everywhere including behind campus/corp firewalls as no one blocks TCP 443. I've tried running a seperate UDP instance on a different port for situations where I need higher performance but for my use cases TCP works fine.
From my experience UDP 53 like another commenter suggested does not always work as some firewalls forcibly route all UDP 53 packets to their own local DNS server in order to prevent people from using their own.
As a bonus OpenVPN has the "port-share" option which allows you to share the port with other services like an SSL web server. SSLH is also an option if you want to host both your VPN and a HTTPS site on TCP 443.
This project is doing LLMs of all things with WebGPU too!
“One kid kept opening up game sites” said one high school teacher who asked to stay anonymous, to protect the identities of their students. “I would wait for them to open one, add it to my list of blocked websites, refresh my settings, and then they would get locked out of it. Then they would open a new tab, find a new game site, and the cycle would repeat. This happened over and over over the course of about half an hour."
It strikes me as strange that the schools are playing a cat-and-mouse game rather than simply disciplining the students caught playing games or accessing these sites. Loss of privileges or so forth for violators would serve as an actual deterrent rather than encouraging students to simply find new ways to get around the block.
What's the difference between Viogpu and the older Virgil3d?
Honestly your Reddit comment was probably already archived on Pushshift as soon as you posted it.
Privacy and security is a big desire for the people running these lower-quality local models. These assistants are becoming more and more powerful and people want to use them for personal tasks. For instance someone might want to chat about their inner feelings or ask the AI to pen a confidential letter, things that they wouldn't want to send to a cloud service. We saw the same thing with Stable Diffusion and how people would spend a lot of effort getting it to run on their machines so they wouldn't have to rely on a public instance.
I'm talking about individual people here as the fact that this is a leak means that corps probably won't take the legal risk of trying this out (maybe some are doing so in secret). In the business world there definitely is a want for locally hosted models for employees that can safely handle confidential inputs and outputs.
The Llama models are not as good as ChatGPT but there are new variants like Alpaca and Vicuna with improved quality. People are actively using them already to help with writing and as chatbots.
For a single unit one could have it in their home or office, rather than a datacenter or colo. If the user sets up and manages the machine themselves there is no additional IT cost. The greatest operating expense would be the power cost.
While verification could be done for a cloud service like this one, what's more concerning is that locally run models with this tech will be coming soon (think of LLAMA and Stable Diffusion). KYC is merely a stopgap and honestly we'll need effective solutions for detecting vocal cloning impersonation in the future.
I guess I didn't really word this clearly, my intent was to state that the manual camera generally captures real-world conditions as a baseline. Then the photographer could in camera or in post manipulate the image to fulfill his artistic vision. While there are certain cameras which have inherent distortion for artistic purposes (e.g. Lomo camera) the majority of them are designed to realistically capture the scene in front of them.
In contrast the phone produces a specific artistic decision influence by the software. For instance in the backlit scenario the phone purposely boosts the highlights and cuts the shadows to create what it perceives as a more balanced image.