davidsojevic

https://david.sojevic.com/

It's currently just where I'm posting miscellaneous experiments I've tinkered with (mainly just about trying to abuse CSS at this stage)

The most comprehensive tool I've built so far was a JSONPath playground [0] when I was working on a game where I found myself writing a fair bit of JSONPath (well, JSONPath Plus specifically) expressions by hand and wanted to be able to test them out in the same way that I would on regex101 when writing regular expressions.

I realistically probably would have only saved myself less than an hour of crafting those expressions if the tool had already existed (with this level of detail, there are lots of many simpler ones already for it), but I would have spent a solid 40-50 hours of bouncing between manually crafting and writing detailed instructions to direct the agents to get this tool there.

[0] https://jsonpath101.com/

This is actually a very well-polished game and I think you've definitely got that core gacha distillation down pat!

I love that you can solve math problems to unlock more rolls and that you can buy packs and then the "slicing" of the packs is very satisfying too.

I work at SerpApi [0], and we offer a free tier that may serve your needs if you're just looking to do programmatic searches periodically.

Much of the reason people go with a service like ours is because of the difficulty with rolling your own reliable solution. Happy to answer any questions you might have as well!

[0] https://serpapi.com/

The most recent project I’ve been working through has been a tool for JSON query evaluation and debugging [0] inspired by how easy regex101 is to use.

I couldn’t find any that were as nice or as powerful to use for writing JSONPath queries, so instead of spending an hour crafting and testing them manually, I spent >40 hours building this tool to save myself half an hour.

[0] https://jsonpath101.com/

I've got a few things on my plate that I bounce between building for fun:

1. A point-and-click adventure game: making it with incredibly heavy technical constraints "just because"

2. A coding puzzle game of rapidly escalating difficulty

3. Part of #2 had me needing to craft some JSON Path queries and I felt like there wasn't anything nice to build and test them with, so I built this tool for it (inspired by the amazing regex101): https://jsonpath101.com/

4. A website where I write about text-based browser games

I don't mean that users are following the links to `acme.com` and `demo.com` type domains in documentation; I mean that bots are likely finding and following many links to them because of their widespread use in documentation.

If you search for `site:github.com "acme.com"` in Google, you'll find numerous instances of the domain being used in contrived links in documentation as an example of how URLs might be structured on an arbitrary domain and also in issues to demonstrate a fully qualified URL without giving away the actual domain people were using.

This means that numerous links are pointing to non-existent paths on `acme.com` because of the nature of how people are using them in documentation and examples.

I suspect part of the issue is that people are still using things like `acme.com` and `demo.com` as an example domain in their documentation and tests instead of relying on `example.com` which is reserved exactly for this purpose [0]

[0] https://www.iana.org/domains/reserved

SerpApi | https://serpapi.com | Junior to Senior Fullstack Engineer multiple positions | Customer Success Engineer | Hiring Coordinator | Python/Ruby/PHP/Js/Rust/Cotlin/C#/Crystal/Nim/Elixir Developer Advocate positions | Based in Austin, TX but remote-first structure | Full-time | ONSITE or FULLY REMOTE | $150K - 180K a year 1099 for US or local avg + 20% for outside the US SerpApi is the leading API to scrape and parse search engine results. We deeply support Google, Google Maps, Google Images, Bing, Baidu, and a lot more.

Our current stack is Ruby, Rails, MongoDB, and React.JS. We are looking for more Junior and Senior FullStack Engineers. We have an awesome work environment: We are a remote first company (before Covid!). We do continuous integration, continuous deployments, code reviews, code pairings, profit sharing, and most of communication is async via GitHub.

We value super strongly transparency, do open books, have a public roadmap, and contribute to the EFF.

Apply at: https://serpapi.com/careers

SerpApi | https://serpapi.com | Junior to Senior Fullstack Engineer multiple positions | Customer Success Engineer | Hiring Coordinator | Python/Ruby/PHP/Js/Rust/Cotlin/C#/Crystal/Nim/Elixir Developer Advocate positions | Based in Austin, TX but remote-first structure | Full-time | ONSITE or FULLY REMOTE | $150K - 180K a year 1099 for US or local avg + 20% for outside the US SerpApi is the leading API to scrape and parse search engine results. We deeply support Google, Google Maps, Google Images, Bing, Baidu, and a lot more.

Our current stack is Ruby, Rails, MongoDB, and React.JS. We are looking for more Junior and Senior FullStack Engineers. We have an awesome work environment: We are a remote first company (before Covid!). We do continuous integration, continuous deployments, code reviews, code pairings, profit sharing, and most of communication is async via GitHub.

We value super strongly transparency, do open books, have a public roadmap, and contribute to the EFF.

Apply at: https://serpapi.com/careers

I like the idea of it; though I think the latest update (around puzzle solveability?) may have broken the game as I'm unable to begin in any browser I've tried:

    Uncaught TypeError: can't access property "puzzleLog", G is null

If you're interested in giving the tool a shot, feel free to shoot me an email (take my username and insert an @ after david and a .com at the end) and I'll happily give you access after I get it up somewhere publicly accessible -- possibly in the next couple of weeks.

We have ASIC (Australian Securities & Investments Commission) that handles company registrations, notices, etc. and they maintain a register of insolvency notices and liquidations.

This can be a reasonable place to go to look for distressed businesses/assets too and I've considered using them as a source with my aggregation/search engine, though they don't really have the same type of information as a business for sale listing so they fall somewhat outside of the main type of results that I otherwise display.

Other reasonable places I've seen too, though in incredibly low volumes (think 0-3 listings a month), are commercial auction houses/sites where they'll list a business for sale or the full assets of a business. The main issue with that it is that they're so low volume that I'm not sure it's worth spending the time ingesting them this early on while there's still many other larger listing sources.

In Australia, brokers need to be licenced real estate agents (even as a buyers agent in many states), so I suspect there's a relatively decent culture of people who are serious about selling their business going through brokers and resulting in listings on at least one of the major platforms.

There's just shy of 90,000 unique listings I'm tracking (i.e. after de-duplication) on these platforms.

On the traditional classifieds sites and things like Facebook groups focusing on these, there's a significantly smaller number of listings/ads for business sales (e.g. a couple of thousand).

I think where there are definitely hidden gems is where there are many small business owners at or close to retirement age where they haven't planned for a sale at all. For example, a family member nearing retirement age has a small business they're just intending to shut down because they "couldn't be bothered" selling it. I've heard people have had reasonable success just approaching local businesses like this that have older owners OR asking accountants if they have any clients that are thinking of selling.

I've been working on a "businesses for sale" aggregation/search engine that sources data from all of the major "business for sale" type platforms in Australia and de-duplicates listings, extracts data like revenue/profit/etc, and normalises it all for quick browsing.

I have a couple of family members and friends who are looking to buy businesses (separately), and it's been much more time-consuming than you'd expect just to browse through listings to determine if they're relevant to you or not.

The platforms seem to mostly follow the same format as real estate listings (as the brokers seemingly rely on the same software/data formats), with one big blob of freeform text that contains the various information that you'd ideally just be reading at a glance.

Add to the fact that there are over 15 "business for sale" type platforms in Australia where they have a minimum of 1,000 listings and at least 10 platforms with between 100-1,000 listings, you can easily burn hours looking through them individually.

I'm currently covering 12 of the top 15 (ranked by number of listings they contain) platforms and I just tinker away once or twice a month, adding support for new platforms.

I should probably release it and get some feedback at some point, but I suffer a bit from "it needs more polish before I let people other than my family and friends use it"

I've been working on and off on a client-side* SERP rank tracker: https://serpowl.com/

I wanted a simpler alternative to the self-hosted SerpBear tool that I could use and share, so this is the result.

It uses SerpApi (where I work) as the data source for what actually executes the SERP scraping because it's much too complex to have purely client-side, but 100% of the rank tracking portion is client-side.

It's not fully complete and there's definitely rough edges with it, but because of the data source, it supports a large number of search engines right off the bat.

I work at SerpApi and we offer a Bing Search API[0] that is very easy to integrate with.

[0] https://serpapi.com/bing-search-api

SerpApi | https://serpapi.com | Junior to Senior Fullstack Engineer positions | Customer Success Engineer | and more... | Based in Austin, TX but remote-first structure | Full-time | FULLY REMOTE or ONSITE | $150K - 180K a year 1099 for US or local avg + 20% for outside the US

SerpApi is the leading API to scrape and parse search engine results. We deeply support Google, Google Maps, Google Images, Bing, Baidu, and a lot more.

We are still hiring - we have so much work and we're after great people to help out! I only started at SerpApi late last year myself and can't recommend it enough.

We do continuous integration, continuous deployments, code reviews, code pairings, profit sharing, and most of communication is async via GitHub.

Our current stack is Ruby, Rails, MongoDB, and React.JS. We are looking for more Junior and Senior FullStack Engineers. We have an awesome work environment: We are a remote first company (before Covid!).

We very strongly value transparency, do open books, have a public roadmap, and contribute to the EFF.

When you apply, please mention that you saw David's post on HackerNews.

Apply at: https://serpapi.com/careers

If you've previously applied and didn't make it through, please feel free to reapply if it's been a while and you think you would make a better fit than previously.

You can contact me at david at serpapi.com if it's been a few days and you think your application may have fallen through the cracks.

Making it harder for bots usually means that it drives up the cost for the bots to operate; so if they need to run in a headless browser to get around the anti-bot measures it might mean that it takes, for example, 1.5 seconds to execute a request as compared to the 0.1 seconds it would without them in place.

On top of that 1.5 seconds is also that there is a much larger CPU and memory cost from having to run that browser compared to a simple direct HTTP request which is near negligible.

So while you'll never truly defeat a sufficiently motivated actor, you may be able to drive their costs up high enough that it makes it difficult to enter the space or difficult to turn a profit if they're so inclined.

Very impressive work! I always enjoy a good write up about reverse engineering efforts and yours was really simple to follow.

Many popular/large websites and bot protection services usually have environment checking as a baseline and mouse-movement tracking in some of the more aggressive anti-bot checks.

It's always interesting to see how long it takes from when the measures have been defeated/publicised until the service ends up making changes to their mechanism to make you start over (hopefully not from scratch).

Some link shortening services allow you to either bring your own domain or register a domain through them to use, so in the latter case it would be "via" the link shortener.

It's sometimes available on other types of SaaS services where your URL would otherwise be customer-facing as well.

There's a fork of this that has some great improvements over to the top of the original and it is also actively maintained: https://github.com/lexiforest/curl-impersonate

There's also Python bindings for the fork for anyone who uses Python: https://github.com/lexiforest/curl_cffi

Given the large delta between what you've experienced and what I can find on my side, the only thing I can think is that your application has been handled outside of our standard process (perhaps directly with someone?) and/or that the SaaS we use to manage the process isn't surfacing you in search.

The code challenge linked has been used for the software engineer candidates long before I began at SerpApi and, in the spirit of transparency, candidates have been directed to make their submissions via a PR to the public repository since almost our earliest employees, e.g. Ilya's submission can be found here: https://github.com/serpapi/code-challenge/pull/6

Can you please forward the email chain(s) to me at david at serpapi.com so I can look into it? Even if you've withdrawn from the process, I would very much appreciate having a copy of these to see what's gone wrong.

We're always looking to improve where we can to reduce any gaps in the process and to make sure no applications slip through the cracks.

I can't find any calls/interviews with your name and also can't find any submission from you on our code challenge[0] (which is the only take home task that we ask devs to do), but it may just mean that you've applied under a different name and GitHub profile than you have listed on the website in your about page/your HN username.

If you can reach out to me at david at serpapi.com and provide me with some more detail about your application I can try and find out what's happened here.

[0] https://github.com/serpapi/code-challenge

Definitely not fake, we are most certainly hiring!

I'm a somewhat newer hire at SerpApi and I found the role through one of the monthly "Who is hiring?" threads late last year.

Please feel free to reach out to me at david at serpapi.com with more detail on what part of the process you had issues with.

SerpApi | https://serpapi.com | Junior to Senior Fullstack Engineer positions | Customer Success Engineer | Talent Acquisition Specialist | Based in Austin, TX but remote-first structure | Full-time | ONSITE or FULLY REMOTE | $150K - 180K a year 1099 for US or local avg + 20% for outside the US

SerpApi is the leading API to scrape and parse search engine results. We deeply support Google, Google Maps, Google Images, Bing, Baidu, and a lot more.

Our current stack is Ruby, Rails, MongoDB, and React.JS. We are looking for more Junior and Senior FullStack Engineers. We have an awesome work environment: We are a remote first company (before Covid!).

We do continuous integration, continuous deployments, code reviews, code pairings, profit sharing, and most of communication is async via GitHub.

We very strongly value transparency, do open books, have a public roadmap, and contribute to the EFF.

Apply at: https://serpapi.com/careers

The Marginalia Search project has been such a great one to follow over the years and I think Viktor is very well deserving of both the last grant and the new grant.

Following the technical journey that he has documented through the blog posts over time has been great to read and I dream about one day trying out the same thing myself at the same sort of scale that he has managed to do as a solo dev.

I'm very glad to see that he's going to be able to keep working on the project for at least the immediate future!

Daniel Tiger's Neighborhood [0] is directly based on the character by the same name from Mister Rogers' Neighborhood.'

Daniel Tiger is one of the very, very few TV shows we let our daughter watch, and it teaches emotional regulation so incredibly well and has plenty of tunes that I've caught her singing to herself, such as "it's okay to feel sad sometimes, little by little, you'll feel better again"

I'm in Australia and never grew up watching Mr Rogers, but after seeing the "A Beautiful Day in the Neighborhood" movie I recognised the tunes and then spent an inordinate amount of time looking into the origins.

[0] https://en.wikipedia.org/wiki/Daniel_Tiger%27s_Neighborhood

Fortunately the Internet Archive has a copy in the Wayback Machine: https://web.archive.org/web/20190611150403/https://www.msn.c...

The last time I saw a Game Boy emulator written in Ruby was part of a RubyConf presentation[0] a few years back, and the performance at the time was understandably not amazing.

The performance here pre-optimisation is more than playable, and with all of the optimisations stacked on top, it was amazing to see it go from ~40FPS to >270FPS.

Very impressive feat and a great write-up on the whole process!

[0] https://www.youtube.com/watch?v=WbO2FEpNPvQ