HN user
Co-founder & CTO at Superlinear (https://superlinear.eu)
For those who want to dive deeper, here’s a 300 LOC implementation of GRPO in pure NumPy: https://github.com/superlinear-ai/microGRPO
The implementation learns to play Battleship in about 2000 steps, pretty neat!
The name ‘late chunking’ is indeed somewhat of a misnomer in the sense that the technique does not partition documents into document chunks. What it actually does is to pool token embeddings (of a large context) into say sentence embeddings. The result is that your document is now represented as a sequence of sentence embeddings, each of which is informed by the other sentences in the document.
Then, you want to parition the document into chunks. Late chunking pairs really well with semantic chunking because it can use late chunking's improved sentence embeddings to find semantically more cohesive chunks. In fact, you can cast this as a binary integer programming problem and find the ‘best’ chunks this way. See RAGLite [1] for an implementation of both techniques including the formulation of semantic chunking as an optimization problem.
Finally, you have a sequence of document chunks, each represented as a multi-vector sequence of sentence embeddings. You could choose to pool these sentence embeddings into a single embedding vector per chunk. Or, you could leave the multi-vector chunk embeddings as-is and apply a more advanced querying technique like ColBERT's MaxSim [2].
[1] https://github.com/superlinear-ai/raglite
[2] https://huggingface.co/blog/fsommers/document-similarity-col...
You don’t have to reduce a long context to a single embedding vector. Instead, you can compute the token embeddings of a long context and then pool those into say sentence embeddings.
The benefit is that each sentence’s embedding is informed by all of the other sentences in the context. So when a sentence refers to “The company” for example, the sentence embedding will have captured which company that is based on the other sentences in the context.
This technique is called ‘late chunking’ [1], and is based on another technique called ‘late interaction’ [2].
And you can combine late chunking (to pool token embeddings) with semantic chunking (to partition the document) for even better retrieval results. For an example implementation that applies both techniques, check out RAGLite [3].
[1] https://weaviate.io/blog/late-chunking
[2] https://jina.ai/news/what-is-colbert-and-late-interaction-an...
Did you even need the D, wouldn't a PI controller be sufficient?
If TSMC buys its lithography machines, why should it even get any credit for 5nm at all?
Could you give an example of an unsolved riddle from linguistics?
In my opinion, the best solution to these issues is to:
1. Declare numbers as numbers in the configuration language. E.g. "decimal(1e1000)".
2. Parse declared numbers with a lossless format like Python's decimal.Decimal.
3. Let users decide at their own risk if they want to convert to a lossy format like float.
Where's the data that says Moore's law no longer holds? I see comments and articles asserting this but everytime with evidence. The data that I do find certainly still suggests Moore's law is doing fine.
Sounds great until the client realises they can hire someone else who does charge by the hour, saving them a massive 100k - 10k = 90k compared to your proposition.
Huh, makes a pretty big difference for us. We were using pandas' built-in to_parquet though, which seems to suffer from some overhead.
Are you sure about that? It depends on how Cloudflare defines what a cold start is. It might well include the initial loading of your code, with imports and init.
Have you benchmarked this against pickling those data files? In our experience, parquet's overhead isn't worth it for smaller data files.
Looks neat. Are you considering a flake8 extension like bandit for easy adoption (in CI and in VS Code)?
As a counterexample: we did actually run out of file descriptors on Lambda by not closing file descriptors.
Can you give an example of a case outside of the happy path?
My experience has been the opposite: building Docker images is much easier with conda than it is with pip. With conda you can start from miniconda3, copy an environment.yml, and then conda create it. With pip, you might need to take additional steps to install system dependencies like build-essential first, and you'll need different tools to manage your virtual environment and Python installation too.
I sympathize with your views in that a formalisation of meaning in life would seem to take away from it somewhat. However, your response also comes across as putting a stake in the ground and thereby closing your mind as to what the utility framework has to offer.
These are some of the problems with utility I took away from your comment:
1. Short term utility maximisation does not necessarily align with long term utility maximisation. 2. The utility is difficult to define because we don't know what can contribute to it and how much those components would contribute. 3. The utility, even if defined, can only be partially observed.
While these are certainly valid and interesting observations, that doesn't mean there isn't a latent utility that describes your lifetime's value. If you live a thousand lives and had access to a device to measure the (latent) utility, surely you could it to rank lives as more meaningful or valuable than others.
Looking at it this way, the problems you raise could be seen as opportunities: how should we align short term utility with long term utility? How can we find out which utility components there are and how much they would contribute? And how can we make the utility observable, so that it can be maximised?
Interestingly, attempting to maximise the utility may itself lower the utility as a result of spending time on the maximisation task and the dehumanisation of life as a result of its formalisation.
Dynaconf [1], though I think this article's approach in combination with pydantic is better.
In what sense has Intel vertically integrated design and fab according to you?
Nocedal has some papers in this direction.
I think you missed the 'easily'.
What is your preferred format for digesting content like this?
Any project with CI should recreate the environment a lot.
That's a very inefficient way to run your CI, with conda and pip alike.
Instead, you could build your environment once in a Docker image and use that as your build image.
Saves a lot of time on your builds, guarantees reproducibility, and will work even when package servers are unavailable.
Don't forget conda ;-)
It's not fast on large environment.ymls, but recreating your environment isn't something you do daily so it's not that big of a deal for us.
And even with limited APM, the response time can still be superhuman. Bit disappointed that they didn't add some human-level latency to the agents' actions.
Feels recognisable. We've outgrown Lambda and moved to Fargate. But now we're wondering whether we should dive into K8s. Worth the investment in your opinion?
What should we be publishing to?
Thanks, hadn't seen this before!
I wonder why they didn't have just one median function and control its behaviour with keyword arguments instead of creating 4 different median functions.
Sybill Trelawny?