I’ve been playing around with using llms to recommend me books I can get from my local library based on the model’s latent knowledge of books it has ‘read’ and a conversation with it about books I’ve liked.
HN user
kidnoodle
Data engineering manager at https://www.humara.com/
I had a theory that the way to solve this was a location intelligence data union which sold safely anonymised aggregates and shared the profits, while also litigating on behalf of members under available legislation to stop other people using their data.
Alas, I was stymied by not having any cash to work on it, and the unit economics were not very VC friendly (at least I assume that’s one of the reasons why I didn’t get any traction from VCs).
A new spin on my slow baking location intelligence data union (https://wherelabs.info). This week I’m thinking about whether it makes sense to provide a location history ‘vault’, designed to let users expose their location history to LLM’s as context.
A new spin on my slow baking location intelligence data union (https://wherelabs.info).
This week I’m thinking about whether it makes sense to provide a location history ‘vault’, designed to let users expose their location history to LLM’s as context.
Still slowly working away on my location intelligence data union..
I’ve spent a while understanding what sort of market would make it viable. I think it does actually work if you can square: 10K participants per major metro area, revenue of about 2.9M per metro area (so say, 5K monthly recurring with about 50 customers).
At that point you could pay data union participants about $5 a week to share their location data with you.
From talking to some previous data union folks, the major challenges are paying out (my target is much higher than any union managed), and people dropping out over time.
My bet is that these are both solvable things by selling data products rather than just bundles of data, and the data source being very passive.
I’m also interested in the idea that such a union should act more like a union than previous efforts in this space, by actively defending members’ data from brokers.
I’ve been working more on the unit economics of my data union/trust idea (https://wherelabs.info/).
What I’m trying to understand is whether it is viable to pay people ~$5 per week for sharing their location data and demographics based on a 90% share of revenue from sales of data products built on that data. (But without ever selling or exposing individual level data).
I think at least initially the ‘product’ is datasets which don’t show individuals. You could of course build out a future direction which is differentially private.
More or less my initial approach to this is you take a grid, and you show movements/density on that grid. If necessary you coarsen the grid to avoid reidentification of individuals, and ultimately to get a good picture of the population given the biased sample which is the union membership, you need a statistical model on top which also helps from a privacy perspective.
State actors demanding individual location history is definitely an issue. I have a few possible approaches in mind to defend against that.
I’m still slowly working on a location intelligence data union (https://wherelabs.info) - the idea is people install an app and agree in a very upfront way to have their location tracked over time.
The union monetises this by selling privacy preserving aggregates (think ‘where is everyone in London right now’, or ‘where did people commute from?’), and acts on behalf of union members to stop data brokers selling their location data.
I built an app for iOS and android recently with a combination of Claude, and cursor. I started off by drawing Claude a picture of what I wanted the app to look like, and telling it to give me the skeleton of an iOS app. (It did a surprisingly good job)
Then I switched to cursor and iterated with it on making it work. Once I had the iOS app running, I got cursor to create an android app that looked and functioned the same.
Super interesting experience - took a couple of days of iterating here and there, helped to have them in git repos for when cursor went too far off piste.
Both apps work, they look pretty much like my original picture.. and the code is absolute trash. I don’t know swift, or particularly kotlin, but the code is horrendous. If you wanted to take them any further, you’d rewrite it from scratch.
This was. Rating combinations of Claude 3.7, 4.0 thinking, Gemini 2.5, and chatgpt 4. (Switching between agents when one was getting stuck.)
Observationally, they all had a much harder time with android and rarely produced anything that didn’t need fixing to build.
Great for rapid prototyping in a stack I don’t know, but 100% tech debt.
I’m playing around with the idea of a location intelligence data union. I work in an adjacent space, and it drives me crazy that there’s all this data about humans moving around that could make a huge positive difference in the world but it doesn’t, because ‘good’ actors won’t touch the shadily data broked data.
I figure the solution is to pay people for their location data, and be up front and transparent about collecting it.
Flowminder | Southampton, UK | Full time/part time | Remote (UK) | Full stack software engineer (frontend bias) | python/react | 52000
Still looking for someone to join my small but wily team of engineers and help me build a new, open source (https://github.com/Flowminder/flowkit-ui and https://github.com/Flowminder/flowkit-ui-backend), data platform for the humanitarian and development sector. (There's lots of other interesting and impactful things to get your teeth into as well, but the role is primarily for this project to start with).
Flowminder is a pioneering, internationally operating non-profit organisation that supports disaster relief and development efforts through several products including the analysis of data gained from mobile phones and household surveys in ethical, privacy and security preserving manners.
We use things like: React, Linux, Mapbox, Deck.gl, Python, FastAPI, Docker, make, PostgreSQL/MySQL, Terraform, CircleCI, Github, GCP, OpenAPI, Auth0
I can also promise you no stupid interview process - you, me, and a couple of the engineers have a chat about you, us, and things you have worked on and the kind of work you’d do with us. If you’ve got a GitHub/bitbucket/gitlab/whatever that’s great, and we’ll look at it, but not everyone’s work is public, and not everyone develops as a hobby, and those things are entirely reasonable as well!
https://www.flowminder.org/about-us/work-with-us/full-stack-...
Flowminder | Southampton, UK | Full time/part time | Remote (UK) | Full stack software engineer (frontend bias) | python/react
I'm looking for someone to join my small but wily team of engineers and help me build a new, open source (https://github.com/Flowminder/flowkit-ui and https://github.com/Flowminder/flowkit-ui-backend), data platform for the humanitarian and development sector. (There's lots of other interesting and impactful things to get your teeth into as well, but the role is primarily for this project to start with).
We use things like:
React, Linux, Mapbox, Deck.gl, Python, FastAPI, Docker, make, PostgreSQL/MySQL, Terraform, CircleCI, Github, GCP, OpenAPI, Auth0
https://www.flowminder.org/about-us/work-with-us/full-stack-...
Flowminder | flowminder.org | Infrastructure Engineer | Remote in the UK | Full or part-time | https://www.flowminder.org/about-us/work-with-us/infrastruct...
Flowminder is a pioneering, internationally operating non-profit organisation that supports disaster relief and development efforts through several products including the analysis of data gained from mobile phones and household surveys in privacy and security preserving manners.
We are now looking for an Infrastructure Engineer with excellent communication skills to facilitate and (depending on experience) drive and lead the expansion of our small but worldwide infrastructure and product installations in cooperation with key technology partners.
We use terraform, docker, linux, python, Postgres, airflow, GCP and anything else that gets the job done. Our infrastructure sits in mobile phone operators' data centres in interesting places around the world, and we use it to help people.
We value autonomy, a drive to make stuff better, and being curious.
A thoroughly panic inducing example is the basis for Anna Meredith’s Sawbones - https://music.youtube.com/watch?v=TsgNYUfdv9A&feature=share
Flowminder | Senior developer Python, SQL | Southampton, UK & Geneva| Remote possible | Fulltime but part-time possible | 58K
Flowminder Foundation is looking for a senior developer to help us open source, and augment our mobile phone data analysis stack.
We’ve used this tool to do things like monitor displacement after natural disasters and conflicts and help understand disease spread in low and middle income countries.
Built on docker, python, and Postgres. We also increasingly use JavaScript for visuals, R if appropriate, and anything else that’s the right tool for the job at hand.
Diverse and friendly team of mostly academics (although we’re looking to increase our diversity on that score), free terrible coffee, lot of opportunity for travel to unusual places. We can’t currently sponsor visas, but are open to some degree of remote working. Full job spec is at http://www.flowminder.org/vacancies/senior-developer
Feel free to drop me a mail at Jonathan.gray at flowminder.org if you’ve got any questions!
Flowminder | Senior developer Python, SQL | Southampton, UK & Geneva| Remote possible | Fulltime but part-time possible | 58K
Flowminder Foundation is looking for a senior developer to help us open source, and augment our mobile phone data analysis stack.
We’ve used this tool to do things like monitor displacement after natural disasters and conflicts and help understand disease spread in low and middle income countries.
Built on docker, python, and Postgres. We also increasingly use JavaScript for visuals, R if appropriate, and anything else that’s the right tool for the job at hand.
Diverse and friendly team of mostly academics (although we’re looking to increase our diversity on that score), free terrible coffee, lot of opportunity for travel to unusual places.
We can’t currently sponsor visas, but are open to some degree of remote working.
Full job spec is at http://www.flowminder.org/vacancies/senior-developer
Flowminder Foundation | Product Manager | Onsite, Southampton UK | £50-60K
We're looking for an awesome product manager to help us take several internal software tools open source, and help guide the vision for their future development.
Flowminder Foundation is a non-profit, working to analyse mobile phone data for good. We work with telecoms operators all over the world, and a bunch of humanitarian and development agencies, providing unique insights on human mobility.
Friendly, interesting place to work, with a really varied team of academics from all over the world, incredible variety of projects, and the opportunity to make a huge difference to people's lives.
We're also looking for a kickass analyst/data scientist. Applications here: http://www.flowminder.org/vacancies
Flowminder Foundation | Product Manager | Onsite, Southampton UK | £50-60K
We're looking for an awesome product manager to help us take several internal software tools open source, and help guide the vision for their future development.
Flowminder Foundation is a non-profit, working to analyse mobile phone data for good. We work with telecoms operators all over the world, and a bunch of humanitarian and development agencies, providing unique insights on human mobility.
Friendly, interesting place to work, with a really varied team of academics from all over the world, incredible variety of projects, and the opportunity to make a huge difference to people's lives.
We're also looking for a kickass analyst/data scientist. Applications here: http://www.flowminder.org/vacancies
I work 4 days a week, which allows me to spend one looking after my daughter. I had the option to do that, or take a (hugely) better paid job. Don't regret it for a second, and I don't think it has actually had that significant of an impact on how much I actually get done in a week.
At least until my child starts school, I wouldn't even consider working full time. Even then, I think the positive difference the extra time has made to me, and my family would make me think hard about going back to a 5 day working week.
Location: London, UK
Remote: Possible
Willing to relocate: No
Technologies: Python, SciPy/Numpy, R, Java, Javascript
Résumé/CV: http://greenape.github.io/cv/j_gray_cv.pdf
Email: j.gray@soton.ac.uk
I'm just finishing up a PhD in complex systems simulation and social statistics, where I've mostly been crunching social survey data, and building agent-based models in Python. Looking to apply my data wrangling and python powers to interesting problems.
I have a blog post I've never quite finished writing, walking through some analysis of the 34,686,770 Amazon reviews in the Stanford SNAP dataset (https://snap.stanford.edu/data/web-Amazon.html, but there's a newer and better one at http://jmcauley.ucsd.edu/data/amazon/).
Somewhat surprisingly, the median review score is 5, and if you look at median scores for products, the skew is very much left. So, the data pretty much backs up your observation!
A todo list with dependencies, using webcola (http://marvl.infotech.monash.edu/webcola/).
Yuki Sugiyama and co did some lovely research on this phenomena and were able to reproduce it on a circular road. A rather splendid video of said experiment: http://www.youtube.com/watch?v=7wm-pZp_mi0