HN user

travisleestreet

24 karma
Posts2
Comments15
View on HN

We don’t respect robots.txt because estate agents often block listing pages - not necessarily to prevent indexing, but for SEO reasons (Google penalises large sites with transient pages).

That said, we do crawl responsibly i.e. we use reasonable request delays, respect rate limits etc. We want agents to like us, ultimately, and blowing up their servers doesn't help with that. If an agent prefers opt-out, we always honour it.

For sure, those searches are more about showing what’s possible (and making browsing fun) than what a high-intent buyer would actually type in.

That said, from speaking to a lot of London buyers, people are often more flexible on location than you’d expect. The real criteria tend to look more like: “3-bed house under £700k, 30 min commute from [office], near a park, low crime, good schools” rather than “3-bed in Hackney.” Basically along the lines of the location / mapping data you're suggesting.

We’ve already built travel-time search and plan to layer in more of that other 'services' style data in the next few months.

For sure big real estate portals functionally work. People buy homes through them, even if the experience is clunky. But because they’re entrenched via massive network effects / SEO, and their actual customers are estate agents, there’s basically no incentive to innovate. That's why the top property portals are such great businesses - Rightmove in the UK has like 70% margins.

I think what you're saying is there's that there’s deeper pain further down the value chain? If so, agreed. We think there’s an opportunity to solve some of that, and that building a better product / search experience at the top of the funnel makes it easier to fix problems later in the journey.