sjnair96

It's completely wrecked. While previous versions were very usable, the beta 3 update is broken for all use cases. Extremely simple prompts trigger guardrails. Completely inert inputs.

What is the population of Sweden? What is the size of New York? What is the size of Sweden?

These all fail to generate a response.

Have you tried VoiceCraft?

Option + Sound menubar icon = choose individual input output devices. Solution I use for the occasion the OP's app was built for.

Usually, if I feel this, it means I have to get my AirPods cleaned (do it at the store, it's worth it). I have, subjectively, always felt that clean AirPods blow the Mac mic out of the water.

I also looked up the same. I wonder why. They must have a subsequent announcement regarding this I'd expect.

How do the multiple startups that import YouTube video into their platform work - one's that need the video/audio files? There seems to be a lot of webapps supporting this but I always wondered if they have a goto API for doing this. I myself have used microlink at some points for automated video source extraction.

https://microlink.io

Holy hell, was shitting bricks, considering I JUST migrated most services to Azure OpenAI (unaffected by outage) — right before our launch about 48 hours back. What a relief.

But with different, separate content filtering or moderation. I have deployed in prod and managed migration to Azure Openai form Openai, and had to go through content filter issues.

Really cool to see the Assistants API's nuanced document retrieval methods. Do you index over the text besides chunking it up and generating embeddings? I'm curious about the indexing and the depth of analysis for longer docs, like assessing an author's tone chapter by chapter—vector search might have its limits there. Plus, the process to shape user queries into retrievable embeddings seems complex. Eager to hear more about these strategies, at least what you can spill!

Some notes I made:

- Dropped Azure support (for now, maybe?)

- Switches out Requests and AIOHTTP packages for httpx.

- Switches to explicit client instantiation

Feel free to discuss what you expect this to/not to fix. So many Cloudflare 502 issues and bad gateway errors which we still get billed for, have caused the openai dev experience to be rather poor. The async/sync request timeouts don't even work as you'd expect. These may be fixed in this beta.

I cannot believe we are at a point where we can basically get Modafinil like substances in a can, sold openly. Curious what the roadmap must've been to get approval! This is just nuts!!

I may be missing something, but I'm willing to bet it's just you. I had the same line of reasoning as him and your snarky comment without explanation is unwelcome here. I like discourse. Not emotional knee jerk reactions or whatever it is that caused you to reply like that. If there is something fundamentally wrong, and obviously so with his line of reasoning, do let me know.

Can you share some insights/examples, if you can, on how you improved the prompts? One I feel is particularly poor is the next question generation/past question condensation prompts which are used to refine the user's input based on the history, so that the query includes all the context required for the question, and hence, incorporating "memory".

Well, they have Reid Hoffman as a cofounder and raised 235 mil before even launching anything at all. So.. maybe not all startups.

Check out my comments on this thread, if you have more findings reply there please, it would be useful for the community. I think what they are doing is unique and would be different under the hood when compared to most llm providers out there.

Reid Hoffman is a cofounder :)

The text is outputted in the same cadence that humans speak/think.

On memory Wow: - I use a technique called episodic memory to retrieve information from my long-term memory. This is a memory retrieval process that humans also use. When I need to access information from my long-term memory, I use episodic memory to retrieve it. This allows me to access information from any point in time, rather than just the most recent information - I have a short-term memory buffer that can hold about 10 minutes worth of information. This is the information that I’m actively processing and using at any given time. It’s a limited amount of information, but it allows me to have a conversation with someone without having to constantly access my long-term memory. The information in my short-term memory is constantly being overwritten by new information, so it’s constantly being updated.

If you guys find any other quirks or oddities telling of what they are doing different, lets discuss here!

- It has access to current data, but it's not quite realtime, but likely routinely updated

- It matches cadence of human speech/thought with its output. Very cool.

- It says it's not quite the same as an LLM

- You can chat over Whatsapp, Messenger, iMessage, text etc

- It can speak. (TTS options in bottom right)

Not so secret, and also precisely how Langchain (1) and GPT Index (Llama Index) (2) got so popular. Here's a quick rundown:

0) You can't add new data to current LLMs. Meaning you can't train them on additional data, or fine-tune, leave that more for understanding structure of the language or task.

1) To add external corpus of data into LLMs, you need to fit it into the prompt.

2) Some documents/corpus are too huge to fit into prompts. Token limits.

3) You can obtain relevant chunks of context by creating an embedding of the query and finding the top k most similar chunk embeddings.

4) Stuff as many top k chunks as you can into the prompt and run the query

Now, here's where it gets crazier.

1) Imagine you have an LLM with a token limit of 8k tokens.

2) Split the original document or corpus into 4k token chunks.

3) Imagine that the leaf nodes of a "chunk tree" are set to these 4k chunks.

4) You run your query by summarizing these nodes, pair-wise (two at a time), to generate the parent nodes of the leaf nodes. You now have a layer above the leaf nodes.

5) Repeat until you reach a single root node. That node is the result of tree-summarizing your document using LLMs.

This way has many more calls to the LLM and has certain tradeoffs or advantages, and is essentially what Llama Index's essence is about. The first way allows you to just run embeddings once and make fewer calls to the LLM.

[1] https://langchain.readthedocs.io/en/latest/ [2] https://gpt-index.readthedocs.io/en/latest/guides/index_guid...

Indeed interesting.

On that note, a core Nvidia NeMo developer I follow posted this: https://twitter.com/HaseoX94/status/1572748653189791745

He calls it a "T5 for ASR" paper :) More insights in there, have a look! Curious to see what your blog would put up as well!

you can also get speaker labels with existing open source software.

Hello Nickolay :)

Diarization has always been the hard part for me, especially since it is very difficult to do comparisons within your domain. The evaluation metrics are not descriptive enough imo.

Would you say Titanet or EcapaTDNN are decent for use in production alongside, say, Whisper, or any other ASR output, if given the timestamps, so as to bypass running VAD? I'm just about to run experiments to try pyannote's diarization model and google's uis-rnn to test out how well they work, but it's a tad beyond my ability to evaluate.

I also wonder if Whisper architecture would be good for generating embeddings, but I feel it's focused so much on what is said rather than how it's said that it might not transfer over well to speaker tasks.

Damn, why shoot themselves in the foot for image processing tasks. Appreciate the insights!

Midjourney - founded by David Holz, co-founder of Leap Motion. Check out their discord :)

Ha, no one has pointed out who the author is yet - a CTO of one of India's recent unicorns.

Oh Alan Adler comes in twice! He's credited for inventing the Aeropress coffee brewer and the Aerobie frisbees!

Yes, but that would involve us digging into the code of the gRPC server, which is unfortunately closed to us :(

Ahh, we don't have access to the server. It's closed - an NVIDIA inference engine. Which under the hood talks to their Triton engine. Unfortunately, while Triton allows configuring the limit, the layer in front of it eats our channel options which have the message size configurations.

Have you guys run into the issue of gRPC being used for ML inference engines/platforms, and having the message size be an issue?

Recently, I was working on building the backend for our Speech Recognition engine and it seems like the message size limit came in the way of being able to perform batch/offline inference. I might be lacking some of the intuition behind designing and deploying such services, and am curious if you guys have run into this issue. The platform doesn't seem to accept the channel options from the client.

That and the issue of no easy way to do gRPC calls from a browser yet, and the resulting architecture then needing to have been built the ground up with considerations for an extra piece - a gateway, or something like Envoy to make it more accessible over the web. I feel like that would be okay if I was working on a project with k8 ingress set up already and already utilizing microservices. But for getting running it seems like added complexity. Not hating on it or anything, just sharing my experiences and thoughts, hoping that some of you might've had experience reasoning and dealing with such designs and can offer some insight.