ddren

I was trying to distance myself from this situation, but this is just too painful to read. I am sincerely sorry that people have harassed you on my behalf, but I have no control over what some people say or do on an anonymous board based on publicly available information.

That doesn't mean that I am happy with the way our collaboration was handled. Why did you create a new converter when you knew there was already an existing pull request that addressed the same issue? Why did you modify the model format and break backwards compatibility when the current format was proven to work with mmap? Why did you change the magic string of the file format to include your initials, when there was an explicit version number field for this purpose? Why did you create a new pull request when you could have added your changes to mine? Why did you rush to merge the PR instead of taking your time to verify that everything worked properly, while listening to feedback from the other contributors and users? Why did you did you ignore concerns raised by other contributors in my PR? Why are you claiming that I was unable to make the WIN32 code work when the final version in your PR is virtually identical to mine, making me look incompetent?

Ultimately, it was my decision to move on, close my PR and allow yours to continue unchallenged, and I owned that decision every single time that I have commented about it, including in the PR linked in this post, where I recommended keeping your PR and working on fixing the issues being raised. I am sorry that some people have harassed you, but making me responsible about this is extremely unfair. There are plenty of reasons for people to feel disappointed about your behavior without me having to say anything about it.

I don't expect that anyone will believe me about this, after all I am just an "anonymous person". The truth is, I am extremely weary about posting this because I know how much damage you can do to me if you insist on this route to your followers. What is your theory, that because I am nobody I have nothing to lose? How are you not aware of the huge power imbalance between a "celebrity programmer" with thousands of followers and a nobody like me?

Anyway, all the information is publicly available on github for anyone who cares enough to verify it.

- slaren

So because ChatGPT exists now, less experienced programmers will be hired to developed critical software under the assumption that they can use ChatGPT to fill the gaps in their knowledge?

Even in that case, I would argue that is entirely a problem of the process, and should be fixed at that level. An experienced programmer doesn't become any less experienced just because they use ChatGPT.

What is the issue with the GDPR? If OpenAI is violating the GDPR, why should European governments ignore it?

I would trust ChatGPT code about as much as I trust the code produced by any human. All the Therac-25 code was written by a human, so what is the argument here exactly? At least when you tell ChatGPT that its code is wrong it agrees and tries to fix. Ok, it usually fails at fixing it, but it doesn't refuse to acknowledge that there is a problem at all, unlike the case of the Therac-25.

I like to think that it is not about who (or what) writes the code in the first place, it is about the review and testing procedures that ensure the quality of the final product. I think. Maybe it is just hopeless.

Price controls and rationing go hand in hand. Ultimately the goal is to ensure that everybody is able to acquire the goods that they need to survive. We are not talking about luxury yachts here, we are talking about food. If the source of the price increases is an insufficient supply of the raw materials, would you rather allow the market to increase prices until the point that some people are starving while others are buying ten times the food that you need to survive to feed their pets?

Gives anybody with elementary knowledge of econ a heart attack

Plainly, this isn't true. This opinion is not uncommon amongst so-called "Marxist economists" such as Richard Wolff. You can disagree that price controls and rationing would be the best way to solve the problem, but it is outright ignorant and needlessly insulting to make this claim.

I know this is probably going to give a heart attack to some of the libertarian tech-bros here, but there is an easy and tested solution for that: price controls. Inflation has a very clear source, it happens when someone decides to raise the prices. Why not stop the problem at the source?

I used it for a while but I found that too many suggestions are worthless and having to consider them makes me waste more time than just writing the code myself. For the things that it is useful like snippets, I found that ChatGPT is better anyway.

Most likely yes. I bought a new 980 Pro 2TB a few weeks ago and it came with the latest firmware. Manufacture date is Jan 2023. The firmware is already more than a year old so unless you get very old stock it should come with it.

Could you share what you did to convert the models? The mirrors aren't very reliable and it would be good to have it documented somewhere. There is some discussion about this at https://github.com/ggerganov/llama.cpp/issues/172

The llama models were trained with a context size of 2048. By default llama.cpp limits it to 512, but you can use -c 2048 -n 2048 to get the full context window.

llama.cpp is using RTN at the moment.

The python implementation[1] ran some tests using the same quantization algorithm as llama.cpp (4 bit RTN).

1: https://github.com/qwopqwop200/GPTQ-for-LLaMa

What are you comparing it to? Without instruction tuning and a two character prompt "He" I am not sure why you would expect it to perform any better.

The (quantized) 13B model is 7.6 GB on disk and the program uses around 8 GB to run. It runs without hitting the swap with just 9 GB assigned to WSL2.

They have recently merged support for x86. I get 230ms/token on the 13B model on a 8 core 9900k under WSL2.

Of course, in practice that is not a possibility for most people. You can't just choose to work hourly if nobody is willing to hire you on these terms. Without regulation the balance of power is strongly in the hands of capital and inevitably leads to employers working employees to death. We have plenty of examples of this in our past.

Is there anything about sitting in a line manufacturing electronics that requires 12 hours shifts?

I see many comments here every day that would get anyone instantly banned from most of the biggest subreddits. However, I think that most people would agree that this place is a lot less toxic than reddit. Smaller subreddits are somewhat better, but at this point this behavior has become pervasive in site, and users are expected to respond aggressively to any disagreement. As a result, any discussion becomes impossible and most comment sections are filled with users mindlessly agreeing with each other. Moderation is important but the way it is implemented in reddit has only managed to turn the site into an extremely toxic echochamber.

That's not my experience, but maybe it is my adblocker hiding the Chrome ads. You can't even use the chat feature in bing with firefox without spoofing the user agent.

Wasn't expecting to see Adam Ragusea here. He has a popular youtube cooking channel nowadays.

I have noticed this too while giving the new bing a try the last few days. It doesn't matter that you have an AI bot summarizing and answering questions about the search results if your search results are worthless anyway. I am back to google now, the search results aren't any better but at least it has a dark theme that won't burn my eyes and it doesn't try to get me to switch to Chrome at every chance like bing does with Edge.

I wonder how this is implemented in the GPU. From my time working on a 3D renderer a long time ago, triangles with offscreen vertices would be clipped into smaller triangles, so in the end you would still be rendering multiple triangles anyway. I imagine it would also be possible to clip the scanlines instead.

Isn't this problem solved with a chiplet design like AMD does? Just manufacture the SRAM using an older and cheaper node and keep the logic in the most advanced node.

Seeing the performance of implementations like FlexGen [1], I don't think it would be entirely unreasonable to run a 13B model on a single GPU for personal usage purposes. You are not going to a run a public service out of it, but it probably would be good enough to run your own ChatGPT or Copilot locally.

[1] https://github.com/FMInference/FlexGen

I was mostly talking about access to the trained model weights. The OpenAI API is certainly better than nothing, but it is very restrictive and cost prohibitive for many purposes. For instance, you have to adhere to the OpenAI usage policies, and while they offer fine-tuning services, it is not likely enough to implement techniques like RLHF, which is the basis for ChatGPT.

That said, if LLaMa can achieve performance competitive with GPT-3 with just 13B parameters, I imagine that it is only a matter of time until open source pre-trained models based on this architecture become available, which would render GPT-3 obsolete.

Not really, the LLaMA model is only available on request and access is granted on a "case by case basis" [1], which for most of us is more or less as available as GPT-3 is.

[1] https://ai.facebook.com/blog/large-language-model-llama-meta...

Out of curiosity, I searched the pet vacuum mentioned in the first example, and found it on amazon [0]. Just like Bing says, it is a corded model with a 16 feet cord, and searching the reviews for "noise" shows that many people think that it is too loud. At least in this case, it seems that Bing got it right.

[0] https://www.amazon.com/Bissell-Eraser-Handheld-Vacuum-Corded...

ChatGPT makes it easier to cheat, but cheating in that way was always an option, as long as you had enough money to pay someone else to do it for you. So in that sense, all that ChatGPT does is expose the flaws in the system by making this form of cheating available to everybody. I have to wonder why only now this is considered a serious problem that undermines education, but it was mostly ignored when the option was only available to the rich kids.

I think this can only solved by having a clear separation between learning and obtaining the credentials that certify that you possess the knowledge that it is expected of you to do some job. Universities should be centers of learning and students should be writing their essays because they recognize the value that doing so provides to them, not just because they need a certificate issued by the university to apply to the jobs that they desire. The certificate part should come from elsewhere, probably through means that make cheating much harder, such as exams and individual interviews. This would also open the door to other avenues of learning that may be more suited for some people, ie. self-taught people may have more success learning from a book or an online resource than from sitting through several hours of lectures every day.