jaehong747

Good interpretability work, but the problem is it's all in how you interpret it. Bridge concept neurons activating even while talking about something else, this seems pretty obvious to me. Input context activating related representations is just an engineering causal structure. Call it subconscious or don't, either interpretation works. But Anthropic keeps drawing these parallels to human consciousness, and it feels intentional, like they're trying to stir up some fantasy. Kind of like comparing condensation on a camera lens to human tears. The whole point of interpretability should be clarity, not stirring up confusion. Even if some form of consciousness does exist here, it wouldn't be magic, it would be an explainable principle. Would be good if they addressed that side too.

In the AI Era, Do Startups Still Have an Edge?

Startups have always competed with big companies based on three gaps: speed, risk tolerance, and incentives. The question is whether these advantages still hold in the age of AI.

* Speed Gap: Still valid. Might even widen. The bottleneck at big companies was never coding. It's approvals, alignment, and politics. AI makes implementation faster for everyone. But here's the thing—when the implementation bottleneck disappears, startups ship immediately. Big companies still get stuck at the decision-making bottleneck that comes before implementation. So the speed gap doesn't shrink. It might actually widen.

* Risk Gap: Still valid. AI doesn't change this. Big companies only want to solve proven problems. If something fails, someone has to take the blame. Startups can bet on unproven problems. If they fail, they don't have much to lose anyway. This is about org structure and decision-making culture. AI isn't going to suddenly make big companies comfortable with risk.

* Incentive Gap: Still valid. Actually stronger now. Employees at big companies get salaries. Founders get equity. This hasn't changed. If anything, AI makes it better for founders. Lower implementation costs mean you can get further with less capital. Less fundraising means less dilution. The math got better for founders.

All three gaps are still intact. AI hasn't weakened the structural advantages of startups. Some of them actually got stronger.

great job! it reminds me genaiscript. https://microsoft.github.io/genaiscript/

// read files

const file = await workspace.readText("data.txt");

// include the file

content in the prompt in a context-friendly way def("DATA", file);

// the task

$`Analyze DATA and extract data in JSON in data.json.`;

Graham says good writing sounds good and is more likely to be true. But his own writing is hard to read and confusing. His sentences are long and messy. If he’s right, then his own ideas must be wrong because his writing sounds bad.

Like you, I also find the paper's findings interesting. I'm not arguing that LLMs lack the ability to "think" (mechanically), but rather expressing concern that by choosing the word "thinking" in the paper, LLMs might become anthropomorphized in ways they shouldn't be.

I believe this phenomenon occurs because high-performance LLMs have probability distributions of future words already reflected in their neural networks, resulting in increased output values of LLM neurons (activation functions). It's something that happens during the process of predicting probability distributions for the next or future output token dictionaries.

Modern transformer-based language models fundamentally lack structures and functions for "thinking ahead." And I don't believe that LLMs have emergently developed human-like thinking abilities. This phenomenon appears because language model performance has improved, and I see it as a reflection of future output token probabilities being incorporated into the probability distribution of the next token set in order to generate meaningful longer sentences. Humans have similar experiences. Everyone has experienced thinking about what to say next while speaking. However, in artificial intelligence language models, this phenomenon occurs mechanically and statistically. What I'm trying to say is that while this phenomenon may appear similar to human thought processes and mechanisms, I'm concerned about the potential anthropomorphic error of assuming machines have consciousness or thoughts.

I’m skeptical of the claim that Claude “plans” its rhymes. The original example—“He saw a carrot and had to grab it, / His hunger was like a starving rabbit”—is explained as if Claude deliberately chooses “rabbit” in advance. However, this might just reflect learned statistical associations. “Carrot” strongly correlates with “rabbit” (people often pair them), and “grab it” naturally rhymes with “rabbit,” so the model’s activations could simply be surfacing common patterns.

The research also modifies internal states—removing “rabbit” or injecting “green”—and sees Claude shift to words like “habit” or end lines with “green.” That’s more about rerouting probabilistic paths than genuine “adaptation.” The authors argue it shows “planning,” but a language model can maintain multiple candidate words at once without engaging in human-like strategy.

Finally, “planning ahead” implies a top-down goal and a mechanism for sustaining it, which is a strong assumption. Transformative evidence would require more than observing feature activations. We should be cautious before anthropomorphizing these neural nets.

Wow, quickly skimmed the paper, this really is like a USB stick for knowledge. Thanks for sharing! Cool that you worked on something similar before.

Yeah, it's definitely attracting attention! Have you tried it yet?

FastHTML is an impressive and innovative idea. It seems like a web development tool similar to Streamlit, but with more precise control.

FastHTML's concept led me to consider a feature that allows direct deployment of PyQt code as web services, even without HTML knowledge, like "PyQtWeb."

PyQtWeb > FastHTML > [Streamlit, Gradio]

At the Bitcoin Conference 2024, presidential candidate Trump's speech increased the possibility of strategic Bitcoin holdings for the United States. Senator Cynthia Lummis also introduced a bill for strategic Bitcoin holdings. Of course, there will be difficult obstacles to passing and implementing the bill.

If the U.S. strategic Bitcoin holdings become a reality, what might happen?

If the U.S. doubles its energy production, strengthens its international dominance in Bitcoin mining power with cheaper electricity, and becomes the largest holder, there's a possibility they might propose using Bitcoin for international trade settlements. This is because the U.S. could mine Bitcoin at a low cost and sell it at a high price, while continuously increasing the Bitcoin price.

Because this scenario is foreseeable, China will have to make a decision, and according to game theory, they are likely to participate in the Bitcoin competition. Most countries, including European nations, Japan, and South Korea, will also join in the strategic stockpiling of Bitcoin. Without Bitcoin, they would have to suffer enormous losses and sacrifices in international trade.

JHP

Thank you. I didn't know that.

I wanted to argue that Llama 3.1 is not truly open source, and I wanted to discuss this with the public. Looking at previous "Ask HN" posts, it appeared to be a board where both questions and discussions were possible.

Thank you.

Is the Llama 3.1 model open source? Yes.

Really? No.

Training Data = Source Code

This time, the Llama 3.1 model was released as open source. However, the training data is not disclosed. In AI and deep learning, training data is the "source code," while the model is like an app. Therefore, Llama is not truly open source. It's more like a free language model app has been released.

When Meta's Zuckerberg released Llama, he mentioned Linux and posted, "Open Source Al is the Path Forward“.

While Zuckerberg looks cool and we appreciate him,

still, we should understand that Llama is not open source. To be open source, the training data must be disclosed. We should not be under the illusion that Llama is open source. (Training data size: approximately 15 trillion tokens)

I will provide an answer assuming that the question is about the criteria for model training.

We train the model based on the difference between the predicted price and the actual price as the criterion. Additionally, we also consider the accuracy of the predicted direction, not just the price.

Directional accuracy is a metric that measures how closely the predicted price and the actual price of an asset move in the same direction. For example, if the predicted price for the next day is higher and the actual price also increases, the DA result would be True (1). However, if the actual price moves in the opposite direction, the result would be False (0).

And, BTCGPT currently uses two models, one for daily predictions and the other for weekly predictions. Ideally, we would like both models to provide the same predictions, but in reality, due to differences in the training data and model architecture, they may produce different results. If you look at the direction accuracy at the bottom of the chart, you can see that there is also a difference in the prediction accuracy between the two models. Currently, the daily model is showing higher accuracy than the weekly model.

LSTM now. We are exploring different time series prediction models, such as TFT (Temporal Fusion Transformer) and the latest GPT models. Our aim is to find the best combination of performance and practicality. Currently, we have decided to use LSTM models for our service, but we are always open to exploring new and improved models in the future.