Show HN: TidyFinder

by brildum • 15 years ago

36 21 15 years ago

brildum5• 15 years ago

I've worked on elaborate projects with the goal of world domination over the past few years, and all have failed. Now, I'm working on building a portfolio of smaller apps to slowly build passive income (with the potential for a big win).

TidyFinder is the first step towards that goal. I'd appreciate any feedback/criticism/suggestions that you can offer. I value the HN community's thoughts. Thanks.

philfreo2• 15 years ago

Would love to hear some details on how you built it / got your data

brildum5• 15 years ago

Some basic details below, maybe I'll write a blog post with more thorough descriptions...

App and scrapers are written in Python with MongoDB storage. I was pleasantly surprised with MongoDB's query speed given I only index keywords. There are a small number of products in the database, but its still very fast.

To get the data I've created scrapers for each site. Currently, only Target and Amazon are implemented (with Container Store soon to come). Target and Container Store provide a product feed when you sign up as an affiliate. They FTP their product info to your site and I use that to parse the links. Using each link, I then scrape the page to store the relevant information for various products in the database. Amazon provides a nice API and there is a Python wrapper which makes it easy to query (python-amazon-product-api).

Luyt• 15 years ago

I guess you scrape websites of retailers to populate your database? Have there been any retailers which said you are not allowed to scrape their site for gathering product info, pictures and prices? (I can imagine that certain retailers only want their products to be showcased in the context of their own site)

fishtoaster• 15 years ago

Kinda cool- it's clean and very focused. Reminds me of http://milo.com/, but more specific.

johnrob• 15 years ago

Is storage furniture the best niche for this? I wouldn't know, since I'm not in the market for that, but I don't think the pain of comparison shopping in that subset is particularly high (I could be wrong of course). Another slight foible - how often does someone buy a dresser? You only have a brief window to acquire a user between the time they decide to buy but before they make the purchase. The polar opposite would be something like diapers where parents purchase on a regular basis. Are there any niches more like that?

brildum5• 15 years ago

These are all good questions, of which I don't have answers for (yet). I created this in response to my cousin (a professional organizer) complaining about finding products within certain dimensions. If she had this problem, I figured others might to. Only time will tell if this will actually provide value, but its certainly not all dressers.

andrewflnr• 15 years ago

Is this just furniture? I searched 3-6 x 7-9 x .5-1.5, which is about the size of art supply box I was looking for a while back, and it couldn't find anything.

brildum5• 15 years ago

No, this is not just furniture. It is everything in Storage & Organization categories from Target and Amazon. I'm not sure whether the art supply box fits that category, but I'd hope so. I do not have the dimensions for all products as they are not always formatted identically on product pages, and my parser is fairly dumb at the moment.

As people may notice, there are some issues with the dimensions search. For instance: how do you differentiate between width/height/depth when there is no real "front" of the product. The dimensions depend on the orientation the user expects, not necessarily what is stored in the DB.

These are both issues I hope to fix, but at the moment I still think it performs fairly well. (and will improve)

philfreo2• 15 years ago

Many times I go shopping for something relatively expensive where I don't know exactly what I want. In almost every case I am disappointed with the typical online research/purchasing decision and wish there were way more intelligent vertical shopping engines.

Recently I can think of this happening to me for video cameras, regular camera, honeymoon spots, projectors, health insurance, apartments, outdoor camping gear, etc. Flight & hotel search has made real progress in the last couple years here but there's still tons of opportunity.

I can't say this category of items is super important to me, but I certainly welcome the idea of making purchasing anything online smarter/easier. With so many affiliate programs out there they should also lead to decent/good businesses.

Terretta2• 15 years ago

This thread is useless without Ikea.

But seriously, Ikea's EXPEDIT series should be in here, at the very least, and considering they have a whole "Storage Furniture" section, is a better target than Target.

    Storage furniture:

    Pantry
    Wardrobes
    Bed storage
    Headboards
    Nightstands
    Dressing tables
    Bookcases
    Cabinets & sideboards
    Chests of drawers
    Clothes storage systems
    Drawer units
    DVD & CD furniture
    Filing cabinets
    Heavy duty storage systems
    Shelving units
    TV stands & media solutions
    Wall shelves

I was planning to drive to Ikea tomorrow and was thinking about a search that would show me the options that fit the wall space in question.

Your app is great -- just needs the goods.

pbreit• 15 years ago

First, Ikea doesn't sell online. Second, can you just go to ikea.com?

JoshTriplett• 15 years ago

Ikea does indeed sell online. But I agree: I don't think this would provide any significant value compared to just going to the site.

brildum5• 15 years ago

Many retailers aren't on the site. Target and Amazon are the first I added, but I felt it was good enough to release and get some feedback.

Terretta2• 15 years ago

This is feedback.

derwildemomo• 15 years ago

in case you need some help including ikea: http://momo.brauchtman.net/2011/08/25/fun-with-nokogiri-scre... :-)

devth• 15 years ago

Pretty cool.

Make the price adjuster control move in increments of at least $10 (maybe $100).

gyardley2• 15 years ago

Assuming this is to make some actual money, and not just a proof of concept -- sites like this live or die on the strength of their SEO. People need to find it when they're searching Google for storage products.

You need an individual product page with its own URL for every item, and you need a strategy for getting inbound links, including to those deep product pages. You need content on those pages - reviews and ratings (scrape Amazon, maybe, or product catalogues).

In other words, to make some cash out of this (and the five gajillion other sites you could set up in other niche verticals), you have to content spam the holy hell out of the Internet. Not something friendly to the HN crowd, but that's what works.

Once you've done that, and you have some traffic, you can sell CPC traffic to online stores in exchange for favorable placement. And once you've done that, you're probably making enough money to start measuring the expected value of a user for each inbound search term and buying more, paid traffic - especially when your SEO doesn't have you in position #1.

And then you'll have a comparison shopping site, which can print money - until Google updates its search engine and pulls the rug out from under you. Print the money, but put it in your pocket.

Immediate steps for expanding your product selection / hopefully making more cash - most of the major comparison shopping sites today have affiliate product feeds.

palish• 15 years ago

Thank you very much!

"content spam the holy hell out of the Internet"

How? (In what way would you go about doing that?)

gyardley2• 15 years ago

Well, back in the day, it was stuff like setting up blogs that linked back to your site, making somewhat-relevant blog comments elsewhere, submissions to directories, issuing 'press releases', making forum posts (especially with signature links), making somewhat-relevant contributions to wikis, 'contributing' to bookmarking sites like del.icio.us, providing somewhat-relevant answers on Yahoo! Answers, you get the idea. If you could stick a link in it, you stuck a link in it.

For all I know any or all of these tactics could currently be the search engine kiss of death - so don't do any of them without doing your research first. Search engines have probably gotten a wee bit smarter.

The other, more ethical way to do it is to produce great content (reviews, tutorials, photo shoots, etc.) that people might actually want to link to of their own free will, but this is time-consuming and hard to scale up.

Show HN: TidyFinder

Related Stories