jacky2wong

What I loved about this release was that it was hyped up by a polymarket leak with insider trading - NOT with nonsensical feel the AGI hype. Great model that's pushed the frontier of spatial reasoning by a long shot.

Surprised to see the list of companies using AWS powertools

I love this library - I'm surprised cohere doesn't jump in and have its logo in the front!

Yes - open source is a good go to market strategy. They should not open source if there's no good, strong business strategy around how they plan to commercialize. But there are many ways to commercialize an open source model. For many it's lead generation into a paid offering or a managed offering of the open source model.

Thanks for linking. I agree and strongly believe product companies are in the best position to monetize Gen AI. Existing distribution channels + companies being extremely fast to add AI features.

Where start-ups like Stability need to be rising to compete will have to be AI-native e.g. products re-thought of from the ground up like an AI image editor or as foundation-level AI research companies, agents or AI infrastructure companies.

There's no reason Stability can't play in both B2B and API if planned and strategized well and OpenAI can definitely pull it off with their tech and talent. But Stability has a few important differentiators from OpenAI where I believe if they launch an AI-native product in the multimodal space, they stand to differentiate significantly: - People join because they believed in Emad's vision of open source so it is their job to figure out a commercial model for open source. They can retain AI talent by ensuring a commitment to open source here. If they need to ensure their moat is retained and can commercialize, they should delay releasing model weights until a product surrounding the weights has been released first. Still open source and open weights but give them time to figure out a commercial strategy to capitalize their research. However because of this promise, they will not be able to license their technology to other companies. - Stability's strong research DNA (unsure about their engineering) is so badly fumbled by a lack of a cohesive product strategy that it leads to sub-par product releases. In agreement to the 3 'layers' argument, that's exactly Stability's greatest strength and weakness. Their focus on foundational models is incredibly strong and has come at the cost of the interface layer (and ultimately the data layer as it has a flywheel effect).

The company currently screams a need for effective leadership that can add on interface and data layers to their product strategy so they can build a strong moat outside of a strong research team which has shown it can disappear at any moment...

I think Stability is in an interesting situation. A few suggestions on its direction and current state: 1. Stability AI's loss of talent at the foundational research layer is worrying. They've lost an incredibly expensive moat and there's enough unsolved problems in the foundation layer (faster models, more energy efficient models, etc.) to ensure Stability provides differentiated offerings. Step 1 should be rectifying the core issues of employment and refocusing this more into the AI lab space. I have no doubt this will require a re-steering of the ship and re-focusing of the "mission". 2. Stability AI's "mission" of building models for every modality everywhere has caused the company to lose focus. Resources are spread thin. With $100M in funding, there should be a pointed focus in certain areas - such as imaging or video. Midjourney has shown there is sufficient value capture already in just 1 modality. E.g. StableLM seems like early revenue rush and a bad bet with poor differentiation. 3. There is sufficient competition on the API layer. Stability's commitment to being open-source will continue to entice researchers and developers but there should be a re-focus on improvements in the applied layer. Deep UX wrappers for image editing and video editing while owning the end to end stack for image generation or video generation would be a great focal point for Stability that separates itself from the competition. People don't pay for images, they pay for images that solves their problems.

Thanks for linking!

Good point - I'll research and create a more recent list.

Definitely. These solopreneurs all kept trying after failing and I wanted to share a bit of their story!

Fair point although entering a new industry often requires starting completely from scratch.

I wanted to get people's feedback on a community I'm thinking of building. A community for people who are building one-person businesses (also accepting solo-founders). Being a solopreneur is hard. You face problems no one else seems to understand. We don't all live in the SF bubble and often we don't have people to emotionally resonate with who understand what we're trying to do. Usually - you have to do it all. I searched online and it looks like solopreneurs are a growing category and while they often might find co-founders later on - starting out as a solopreneur is often also a good way to get the ball rolling. We can figure out game-tips and plans for people running these businesses (e.g. VAs that are amazing + how to screen for great VAs in seconds).

For those interested, I have a link for a form for people to fill in if there's interest - https://tally.so/r/3xaxlE

VoiceReplace is now in early alpha. We built the tool to help people tackle voice insecurities when they create videos. We are thinking of expanding this into a marketing tool for social media/SEO content. Would love HN's feedback on what the potential use cases can be!

I'm having a hard time imagining positioning here. Can you explain further how Diversion differs from DVC, Git and when using Diversion makes sense over other use cases. The GTM is slightly confusing to me (also yes - Git is hard - you cannot teach data scientists this. It'll take months).

Also agreed git is terrible right now for version-controlling workflows in AI (I have a fairly large .gitignore file with S3-hosted things ever for my NextJS + FastAPI apps - pain in the butt

Odd - we’ll take a look. If you would like - happy to process it on my end if you email jacky@floatlabs.io

Thanks. We're launching pricing in the coming days and didn't want to mis-price the product beforehand. We are currently not aiming to mimic the same inflections as certain inflections in human voices are a bug, not a feature. (e.g. I mis-pronounce a word or have random intonations sometimes).

Thanks!

I use VSCode + Vim. I found that largely replaces the need for a modeless vim. Is there an advantage of using it over my current setup?

I imagine Twitter would be the main one.

This is a simple side-by-side image generator I wanted to share! This helps you share photos easily that are next to each other.

can you share some stories? would love to learn how sam networked his way into sv

Quietly launching now on langvoice.dev :)

I'm building this right now actually! Launching this week - if you're interested in talking further - happy to share it. You can ping me on jacky@floatlabs.io :)

We will be adding a tutorial on how to use DeepEval as a Custom Validator for Guardrails very soon :)!

Thanks for checking us out! We measure factual consistency using a NLP task known as "natural language inference". We then compare 2 sentences to know if one is an 'entailment' of another (for example - if "there are 3000 oranges in the supermarket", an entailment would be "there are oranges in the supermarket" - which would get a high score whereas "there are no oranges in the supermarket" would be a contradiction. We use the score of the entailment class to get a measure of how factually consistent it is. We provide a high-level summary in the documentation (https://docs.confident-ai.com/docs/measuring_llm_performance...) and will be sharing an in-depth blog article on it soon!

We're thrilled to announce the launch of DeepEval, an LLM evaluation and testing suite designed to work nicely within a CI/CD pipeline.

As more companies integrate LLM/RAG applications into their operations, ensuring the effectiveness, reliability and safety of these models is hard.

About DeepEval We started with consulting on a few RAG projects and quickly realised how many issues came up when we iterated on our prompts, chunking methodologies, added function calls, added guardrails, etc. Very quickly we realised this had downstream effects that caused unexpected problems and results.

DeepEval, inspired by Pytest, aims to make iterating on these RAG and agent applications as easy as possible by building evaluation into part of their CI/CD workflow. The goal is to make deployment of LLMs as straightforward as getting all tests to pass.

Some features of DeepEval include: - Opinionated tests for answer relevancy, factual consistency, toxicness, bias. - Web UI to view tests, implementations, comparisons. - Opinionated flow for synthetic dataset creation.

We are currently in a fully operational beta release and would love your feedback and suggestions to continue improving DeepEval.

We are happy to answer any questions you have!

One complaint we heard over and over was that no one likes to write tests when evaluating LLMs.

We automate writing these tests by having ChatGPT generate sample queries and answers based on a given text. These test cases can then be run through the testing framework and can be run similarly to PyTest.

For those that need help migrating their models to OpenAI enterprise and ensure consistency results - would recommend checking the open source repo https://github.com/confident-ai/deepeval

To be honest - I had the opposite feeling - I found it pretty easy to wrap my head around their abstractions. If anything - I believe their abstractions are quite clean as I was able to navigate their codebase quite easily. Their "context" abstraction was an interesting choice admittedly but that's the only non-intuitive thing I found.

Disclaimer: I worked in this space for a few years as an ML engineer so it might be quite different for new entrants in the field.

Got it - that wasn't immediately intuitive from the suggested comment. Yes! Our roadmap will include ways to test other parts of the LLM application stack very soon!

A few publically shared evaluation datasets include:

- HuggingFace Evaluate (and their 3 dashboards) - OpenAIEval - AnthropicEval