HN user

NameError

269 karma
Posts6
Comments38
View on HN
Slack Is Down 1 year ago

It seems that Slack being fully down and unusable only reaches the "incident" category on their scale - I'd hate to see what counts as "outage"

I think the core problem at hand for people trying to use AI in user-facing production systems is "how can we build a reliable system on top of an unreliable (but capable) model?". I don't think that's the same problem that AI researchers are facing, so I'm not sure it's sound to use "bitter lesson" reasoning to dismiss the need for software engineering outright and replace it with "wait for better models".

The article sits on an assumption that if we just wait long enough, the unreliability of deep learning approaches to AI will just fade away and we'll have a full-on "drop-in remote worker". Is that a sound assumption?

Easy way to get a fair result from an unfair coin toss: Flip the coin twice in a row, in this case starting with the same side facing up both times, so it's equally unfair for both tosses. If you get heads-heads or tails-tails, discard and start over until you get either heads-tails or tails-heads, which have equal probabilities (so you can say something like HT = "heads" and TH = "tails").

This works even if the coin lands heads 99% of the time, as long as it's consistent (but you'll probably have to flip a bunch of times in that case).

Reminds me of a cool proof I saw recently that there are two numbers a and b such that a and b are both irrational, but a^b is rational:

Take sqrt(2)^sqrt(2), which is either rational or not. If it's rational, we're done. If not, consider sqrt(2) ^ (sqrt(2) ^ sqrt(2)). Since (a^b)^c = a^bc, we get sqrt(2) ^ (sqrt(2))^2 = sqrt(2)^2 = 2, which is rational!

It feels like a bit of a sleight of hand, since we don't actually have to know whether sqrt(2)^sqrt(2) is rational for the proof to work.

When my primary care doc referred me to a dermatologist for a suspicious mole, I could not find an actual dermatologist who would see me in less than ~8 months. I ended up seeing a physician's assistant, which I'm still uneasy about since there's been a study that shows that PA's seem to have a lower success rate vs. doctors [1], and the educational requirements are very different for PAs.

As a layperson, it seems like we (patients / society) would benefit from having more doctors, i.e. opening up more residency slots and admitting more people to med school, but there's probably a lot I don't understand about the issue. Not sure if it's a lack of political willpower to do this, or if there are other reasons why the number of doctors we train is so restricted.

[1] https://pubmed.ncbi.nlm.nih.gov/29710082/ ("PAs performed more skin biopsies per case of skin cancer diagnosed and diagnosed fewer melanomas in situ, suggesting that the diagnostic accuracy of PAs may be lower than that of dermatologists")

This article really resonates with me - I've heard people (and vector database companies) describe transformer embeddings + vector databases as primarily a solution for "memory/context for your chatbot, to mitigate hallucinations", which seems like a really specific (and kinda dubious, in my experience) use case for a really general tool.

I've found all of the RAG applications I've tried to be pretty underwhelming, but semantic search itself (especially combined with full-text search) is very cool.

I just made an update which should enable "population density" type queries to work a bit better - there is now an option to divide by 'LAND_AREA' for any variable (though this should probably be limited a bit), and this option will be automatically selected for queries including 'density' or a few related strings e.g. 'per sq. mile'

Hi - thank you for trying it out! These are both definitely real issues with the current approach. I've tried to reign in the "selecting an overly specific table" issue in the final "LLM-selects-from-search-results" stage but clearly have some work left to do there.

As far as the second issue - when people search for things way outside of the available data - I have not done much to address this, but really should. This happens for more plausible queries too, e.g. "Crime Rate" seems like it could be cataloged by the Census, but is not part of the tables indexed by the site (ACS Detailed Tables). It selects variables somewhat randomly here when it should really say something like "no relevant results found"

This lines up with how I make tech stack decisions for my own projects. But I think it's not always obvious going into something if it's going to end up being a money-making endeavor or just an educational project in the end, so I'll fall somewhere in between.

What makes the most sense to be is to be really selective about what new technologies to use, and try to really learn ~one thing per project. E.g. my current project is a small search engine, and I've spent a lot of time exploring / figuring out how to use LLM Embedding models and vector indices for search relevance (vs. falling back on using ElasticSearch the same way we use it at work), but I'm using tools that are familiar to me for the UI/db/infrastructure.

This article is claiming that the Ticketmaster breach from a few days ago was actually a much broader hack affecting 400+ companies, all through a Snowflake employee's stolen credentials. This seems like a pretty big story that's only being reported on hudsonrock.com now.

I haven't heard of Hudson Rock before, does anyone know if they are a reputable source?

The GPT Store 3 years ago

Agreed - plugins had the ability to see responses directly and I'm not sure why they removed this for actions, it's a big step down.

I released a plugin on the OpenAI plugin store when that was first released and am currently migrating it to be a GPT. At least for my use case, plugins and the "actions" feature of GPTs are too similar to complain about the change. My plugin worked on the first try when I tried to import the API spec as GPT Actions. In my experience so far, the GPTs UI actually seems a bit nicer and more stable.

One big complaint I had about plugins was that there were some kinda unclear policies on what a plugin was allowed to do. Specifically, it was really helpful to include strongly-worded prompts in your plugin response (e.g. my plugin provides public data and the JSON response includes an "INSTRUCTIONS_TO_MODEL" key with stuff like like "This data is about <x>. The AI Assistant MUST tell the user exactly what data they are seeing, and MUST provide a link to the original source data")

This was more or less necessary for a lot of plugins to work at all, but it wasn't totally clear that it was even allowed by OpenAI policies - their ToS says that plugin responses cannot "attempt to steer or set model behavior". I tried a couple of times to reach out to the company to ask about this and it was more or less radio silence from OpenAI - the closest thing I found was a discussion on their forums between (non-OpenAI employee) developers trying to figure out what was allowed and what wasn't.

GPTs seem to clear this up by explicitly encouraging custom system prompts etc., but sandboxing it into a "GPT". I guess GPTs seem like a slight improvement over plugins to me - if anything, I'm worried they're too similar. The plugins product has been a bit of a failure in terms of adoption and I'm not totally convinced GPTs will fix everything.

I'm not sure there's a good way to answer this question, people seem to have very different definitions of AI. The definition I learned in school was "AI is when machines do things that most people would say requires intelligence to do", which is (maybe deliberately) vague and prone to shifting over time.

See https://en.wikipedia.org/wiki/AI_effect - the term "AI" used to include things like chess engines (and even simpler search algorithms), but once computers are able to do those things for a while and people get used to it, the term "AI" tends to shift and only refer to things that machines can't do yet.

It's admittedly a bit of an unsatisfying answer, but whether AI is here is more a question of your definition of "intelligence" than anything else.

I think it was a mistake for Github to name the whole product "Actions" and then re-use the same word for a specific component within system. It's really natural to say something like "we need a push action for this repo" when it might be more correct to say "a push workflow"

The release of plugins was underwhelming for a lot of users, I wonder if they're worried about the same thing happening again. I had fun making a plugin when they were first announced but the developer experience was clunky and frustrating, and there was a lot of confusion around how the plugin store actually worked.

I haven't made the time to convert my plugin into a GPT but I'm planning to, hopefully it'll be a bit of a smoother experience this go around.

AWS us-east-1 down 3 years ago

I suppose "increased error rates and latencies" is technically true when the error rate is 100% and the latency is "until we fix it"

It's right that only places between the tropics will ever see the sun directly overhead. I asked it the same question and it was kinda subtly misleading:

"The Sun is directly overhead, or at the zenith, at a given location only when that location is on the Tropic of Cancer (23.5 degrees north latitude) during the Summer Solstice (around June 21st each year) or on the Tropic of Capricorn (23.5 degrees south latitude) during the Winter Solstice (around December 21st each year)."

There are other situations besides these - you don't have to be -on- one of the tropics for the sun to be overhead, you have to be -on or between- them.

I am in the US and had the same issue - ended up installing it by finding the announcement post and clicking through from there.

I'm curious about this, in the plugins section:

Do:

BarkBytes plugin for ChatGPT

BarkBytes ChatGPT plugin

Don’t:

BarkBytes plugin for OpenAI

BarkBytes plugin compatible with ChatGPT

BarkBytes plugin works with ChatGPT

BarkBytes plugin integrated with ChatGPT

A plugin is basically just an API that the provider allows ChatGPT to access, and a specification of that API. There's no reason you couldn't allow other non-OpenAI models to use your plugin too. Is OpenAI saying plugin providers -must- refer to their plugin as ChatGPT-exclusive, even if it's not?

OpenAI Tokenizer 3 years ago

The API does tell you how many prompt and completion tokens each request used, if you're okay knowing after-the-fact