tomsayer

-> You could create a mountain of educational material, apps, and tools with LLM assistance and verify it. Same result. I very strongly disagree with this. Teaching is not the same as a bunch of educational material and apps. The use of AI in education does not replace teachers, but it can get much closer to the responsive, learner centered, high-touch learning experiences that teachers provide, and that are proven to deliver the best outcomes.

-> That’s a powerful emotional point made to shut down criticism. You made a blanket statement - without any substantive points - that we shouldn't put LLMs in front of kids. I was arguing that there are advantages that we should take into account, not trying to shut down criticism.

-> not sure about the certainty of what “making sure” a 4-year old knows is real or not is. I agree. We're being very cautious here. My co-founder, who is a clinical child psychologist, has a blog post on exactly this you can find it in the footer of our website.)

I'm Tom, cofounder of Ello.

-> I think teaching a child to trust an LLM from a formative age is horrifically irresponsible.

I largely agree, although with some nuance. One of the most important skills a young child today is going to need as they grow up is the ability to understand what's AI and what isn't, and how to interact with things that are. We actually work hard to make sure a child knows they aren't talking to something real - that they're talking to a computer program.

If part of the comment was on the accuracy of LLMs; our system of multiple harnessed LLMs running in parallel does deliver pedagogically solid and accurate content and experiences. There's a few comments in this thread about LLM hallucinations - We have curriculum specialists constantly evaluating the interactions that Ello delivers, and it just isn't a major factor right now.

So we need to ensure that we're delivering a high-quality, accurate experience for a child, while at the same time making sure that they understand what's reality.

-> Actually, better, don't put an LLM in front of children. At all.

I've responded above to your reason against using AI in education. But to judge whether it's right to avoid using AI with children entirely, we also need to weigh the positive side. So many children get a substandard education that isn't responsive to them and doesn't set them up for success - whether that's a child stuck in a class of 30 being treated as average, or one of the 270 million kids who don't have access to any education at all. Saying "don't put an LLM in front of children" misses the reality of what education actually looks like for so many of them IMO. If your kid gets everything they need from their school, their home, their private tutor and so on, then great.

So our answer is to use the benefits of AI to build technology that teaches - that understands where a child is and gives them the next learning experience they need to keep growing - while making sure they understand they're dealing with a computer program, and without trying to replace the critical role of the people in their lives.

Hey there, I'm Tom - cofounder of Ello.

On the first Q: that popup is for the website. We're a consumer company and use paid marketing to reach parents. There are no advertisements in the product, nor is any data in the product sold or used for any purpose apart from delivering the experience.

On the second, we absolutely do not just tell an LLM to be safe and hope for the best. Every output of the LLM that is going to be presented to a child runs through a three-layer safety system. But - as importantly - everything a child says to the LLM goes through that same system.

1. Programmatic, anything that could easily be flagged with hardcoded rules. 2. An LLM evaluator that can interpret meaning to catch anything that the programmatic rules miss. 3. A human in the loop that reviews everything that could be slightly concerning.

We then, in turn, have a multi-tiered response system, including giving parents full access to everything that's happening so they're informed, flagging in a clear, but non-alarmist way, anything somewhat concerning, and finally direct outreach to the parents and/or authorities for anything critical.

yeah, had that feedback from a few people. Need to change that! Thanks for the feedback

Thanks!

Thanks for signing up but even more, thanks for the wonderful feedback.

- We actually find the opposite. Without the promo code people are reluctant to try it. I think because it's new and - to you next point - not cheap. - This is a real concern for us and one of our primary product develop tracks it to change this. Our mission contains the phrase 'regardless of resources' and as a $25 DTC product we are not doing that yet. Hopefully much more to come on this soon. - Interesting. We kinda have to know the grade level to send the right books. About 10% of parents know their child's reading level but 90% the only info we get is grade level and ahead/ behind etc... I haven't heard that as a privacy concern. We could do more to explain why we want this. - We can explain how to add siblings better up front. It is there, but it's small and easy to miss. - Agree. Right now as we are getting started we have made it binary - our main goal was that Ello would work for those who really wanted no data sharing, and to give parents that option very easily and clearly. As we start to build out more product features that rely on processing audio we should make privacy modular rather than binary. - Again agree. We're working on that. We definitely need to get parents/kids to the 'aha' moment sooner and not have a 5 day shipping delay. What do you mean by it feeling like a growth hack? It feels like the opposite to me - a growth killer!

Would love any more thoughts as you get the box and they start using it. Thanks you!

Yup- It's a pretty settled debate now I think. There's a good podcast on it called 'Sold a story' that just came out. Thanks for the NYT link!

We are actually adding another language soon! (for somewhat random reasons) - we hope eventually to expand much more.

And it really depends on how strict you want it - 'strict' is a bit subjective. What we cannot do yet, but want to, is put those controls in the hands of the caregiver.

If you try it, I'd love your feedback. If youve been thinking about building this yourself I'm sure you'll have some great thoughts on what we've done well and what we could do better with.

- Five is definitely insufficient for some children (and too much for kids who really struggle and want to perfect books). We're working on giving parents the ability to get more; either we ship more, or once you have read the physical books you can unlock digital books. Lots of considerations there of course. We also want you to be able to use books from e.g. the school library. Q for you - for the 1 to 2 readers a day your child reads, where do those books come from? - That's a great one. I haven't had that question before but I could certainly see the challenge there. Let me ask Elizabeth and Lauren (the experts) what they think on that. - It goes one of two ways; both which we're happy with. The first way is the child builds their confidence and doesn't need Ello anymore. Not the best for business but a great outcome :) The other is that the child continues to value the 'safe space' to practice reading, and does it in conjunction with reading elsewhere. - Research pretty strongly shows that the most important skill - to begin with - is decoding (learning the sounds words makes.). Comprehension comes after that. What has been fairly strongly disproven is that context in reading is sufficient (and some would even say helpful) in learning the fundamentals of reading. Having said that, comprehension is a crucial latter part of reading - we're excited for some of the potential for ChatGPT here in allowing us to go beyond basic questions.

Great questions for us to think about as to what would be in the back of parents minds. And Ill get back to you on the second!

Hi John :)

Good feedback! Will work on updating the website to show that. Were you wanting to just see the specific interaction of how it works?

You can see a bunch of videos users have uploaded here: https://www.reviews.io/company-reviews/store/ello

(And can also see reviews if you click on product reviews.)

We still get some very confused people come to our site every now and then :)

Right now it will likely work well. It is trained on a largely US based data set and so accents within that data set will do better, but the way the model works I wouldn't expect many challenges with Australian.

Interestingly, as we dive deeper into phonics and phonemes it will become more challenging as what may be correct for one accent, would be incorrect for another. We will need the ability to select - by child - what is 'allowed'. This applies to dialects as well.

Great questions! - When a child makes a mistake or needs help we take the child through a phonics decoding exercise (scaffolded 'sounding it out'.) Right now we support reading practice rather than provide full systematic instruction - i.e. we don't replace the rigorous, structured program a child should get in school. However, we are working right now on building out our own scope and sequence and detailed phonics instruction and hope to add that soon. - This one is beyond me and I don't want to pretend to be the expert here. That's Elizabeth and Lauren. Let's definitely connect and I'll get an answer from them. - Our speech recognition system does an OK job at the phoneme level. We are releasing an updated model that will go hand in hand with the scope and sequence that should be the best there is and be great at the phoneme level.

Would love to connect. My email is my first name at our company url!

Tom

We do lots of things, but one of the most important steps is an in-depth role-play where we go through numerous scenarios. We're not just looking for the knowledge (which given their qualifications and experience they should have) but also for their ability to use that knowledge in the context of a parent's own goals and philosophy. If someone brought their own philosophy or values, and not just their experience and expertize, then we wouldn't bring them on.

This is helpful feedback. Thank you. We had the counter feedback off a free call making the service seem low value. Ideally we get the best of both worlds. Easier said than done of course!

This is a harder Q! I don't 100% know but two things give me hope.

First, there are so many incredible experts in early childhood right now that - IMO - are undervalued. We have so many that want to have a home to use their skills in support of parents alongside whatever they currently do.

Second, we're investing heavily in the selection and then ongoing support and / professional development of our coaches.

Over time we will need to increase the support and development even further but for the foreseeable future we're just happy how many incredible people are out there.

Didn't take it that way but thank you for saying that :)

And agree! The organization Zero to Three does a great job advocating for this in the US but we're still way behind.

It's not very long term, but kraamverzorgsters in the Netherlands are a great example!

Thanks :)

Thanks for the kind message!

Trust takes time to build definitely - and honestly, we don't do it - we wish we could! The trust-building really starts to happen in the first call when each side gets to know each other (and beyond just the expert's credentials and bio.)

Great questions!

On the first, we're very cautious; we do not do diagnosis. What we do do is support parents in seeking out a diagnosis if that was the appropriate path. Our experts have all either done diagnosis or facilitated them in the past so they're good and knowing when to suggest that. The balance is in making sure we don't give parents a false confidence, but we find that the pairing of an expert with the family is much more effective than a parent wondering on their own.

On the second point, let me know if I misunderstood, but our goal is to give the right advice. We know that it won't always work and often that will be due to the unique nature of the specific family. Sometimes we will just get it wrong; we have a very rigorous selection process for the coaches and strong QA and ongoing support so we hopefully don't get it wrong very often. But we know we won't be perfect.

I think we're waayy to early for that but I hope we get to the stage where this is a question :)

My first reaction is that our goal is to support parents and we'd want to be very careful before doing anything with their data. BUT if we can support the field at the same time as being 100% transparent with our families it would be great (e.g. they'd have to explicitly opt-in.)

But again, a great problem for the future :)

We don't really think of ourselves as an app (although we use an app to facilitate the service right now.) It's direct, personal support that provides the value; the app is the access mechanism.

We have plans to offer different access mechanisms in future to make this accessible to more people.

Our hope (and belief) is that all types of parents will appreciate having access to someone with expertise - how they access them can differ.

(That felt like a very odd way of responding but I hope it makes sense!)

Thank you! Changed. I also see I said three ways in that FAQ and only have two.

Would love for you to try us! We offer a full 30-day money back guarantee as we know this is something parents want to be confident in.

(I'm Tom, Elizabeth's cofounder) - I agree with the societal reflection. Bowling Alone by Robert Putnam really speaks to this so well.

While one of the changes we think about is that, the more positive one is that we now know SO much about how the young brain develops. We want parents to have access to that in a meaningful way. I often use the healthcare analogy; when someone wants support with their health (whether proactive wellness or reactive treatment) they'll often go to a doctor which is great (and why it's so bad when people don't have healthcare). Raising a child is of course different in many ways, but the internet is awash with generic, impersonal advice whereas we hope to bring back the human connection. We're excited to be able to bring this growing field of expertize to parents in a way that is deeply human at it's heart.

But yes... changes in communities are still depressing.

This is really good feedback! I HOPE that most of the work we have to do is in communicating what we do rather than actually what we do. But without trust we've got nothing )given this is about a parents kids) and so it's a really important thing to get right either way.

I realize this doesn't change your first impression, but the website is our 'business frontpage.' None of the service (which does definitely include sensitive information) takes place through the site.

Our PP is a bit boiler plate and I think the main thing is to make it clearer about the things you've mentioned and separate the service from use of the website. So for example we do track who clicks on what on the website using things like Segment, but we don't do that in the tech product you use to communicate with the expert.

I'd love to chat more about learning if it's in the communication or if it's what we're doing? If you're willing, could you email tom@trustle.com? (And of course no expectation to do so! You've been so helpful with what you've shared so far.)

This is probably going to be unsatisfactory as it will be a bit vague but I'm going to try ... :)

I think part of the challenge we're trying to help with is that lots of places (blogs, websites) say 'this is good for children.' Which could be true (if backed by the research) but that takes an averages approach (I love the book End of Average by Todd Rose.) It sounds a little millenial, but there is no average child and so an approach the uses averages often doesn't work.

Our experts get to know the specific family, philosophies and kids, and then bring their expertise (which includes the population scale research) to support them in a way that makes sense for them.

It's a bit like where we do population-level research on health, but we don't then expect that everyone take the same approach based on a few inputs. We still get them to talk to a doctor.

:) We're new so these questions are great as they help us learn!

Healthgrades is for finding a clinical service; we are not a health service. We often use the term coach (which isn't a perfect term but the best we have right now.) We hope it is an ongoing relationship and not just used when somethings 'going wrong.' Another way we sometimes talk about it is that it's like a really good friend who happens to have an incredible background in child development. With that in mind, we don't fall under hippa as we don't discuss medical issues, although we aim to be at hippa level regardless.

Yup - Catalin has a background in ML. We're not working on any ML stuff right now - he's just an incredible technologist! The closest I can see right now is us being able to maintain a body of knowledge similar to how most doctors use Up To Date. We could then use ML to surface the information to the coach / expert so they always have the most up to date knowledge at their finger tips.

We want to be as efficient as possible so we can support as many parents as possible, and as effective as possible so we provide the best support. But we never want to replace the direct human connection, nor to use data in any way that parents wouldn't want - we know if we lose any trust we've got nothing!

:) That's the magic we hope to recreate.

We support all families! If there is an additional factor (such as autism) we take care to make sure parents see us as a support for them, and not a replacement for whatever supports they have in place for their child. But lots of parents appreciate having a partner in their support system. Lots of our coaches have extensive experience with specific diagnoses.