marcua

Thank you for sharing!

https://blog.marcua.net/

The content has varied over the years, but in the past few years, I've used the blog to explore side projects outside of work. This has allowed me to separate my primary responsibility at work (manager/unblocker/collaborator/prototyper) from my personal interests in hacking.

(author)

It's so helpful to read the headline through your eyes! While I can't change this title on HN, I'll tone down/relegate the multi-tenant bits to the features section in the future. Thank you for this feedback!

(author here)

Thank you for your kind words and the great question!

To your compliment: I agree ayb can help with easier prototyping today.

To your concern: I would not use ayb in a production setting today. To be explicit, while the roadmap [1] is long, I think that ayb will be useful in production once we've implemented a v1 of auth, permissions, persistence beyond the node, and isolation.

To "how to prevent that," here's a rough outline that I'm open to feedback on:

  - Implement the v1 features above

  - Host a public instance (and encourage others to, not trying to empire-build)

  - Build some fun applications on top of the public instance myself to stress test it

  - Encourage others to do the same and/or run some "learn SQL" classes to better understand where beginners get stuck and address those issues

[1] https://github.com/marcua/ayb#roadmap

Fascinating! It's great to see lots of people working on this problem. If you have any running examples, I'd love to see them to understand how the different actors in your documentation work. Thank you for sharing this!

(Hi there! Surprised author here---I hadn't realized this made its way to the front page)

I love your questions because they get to the heart of how to make this stuff easier for more people, and in transparency, I only have some of the answers! :)

What's the permissions/ACL model, and how do you keep that from getting too confusing for the average person? (You asked a lot more here, but I think this is the root of this question) The admittedly naive permissions/ACL model I'm envisioning/speccing now is at the database level, similar to GitHub at the repository level. If you create a database, you can add read/write and read-only collaborators, one of which is `public`/`world`, which would make the database accessible to unauthenticated users.

Your questions around table slices/views are excellent, and in the model I'm proposing, ayb won't be able to help. The model I'm proposing will be good enough for "here's my dataset, and you can build on it" or "I spun up a project and have a private DB that my webapp is gating" but not "I want user X to have access to row Y." Row-based auth would thus be pushed into the application layer, which seems to come with the territory with SQLite as best I can tell. To contrast, something like Supabase is able to provide both a database and row-based auth because Postgres provides better support natively, and Supabase then made it easy to add common auth providers.

what makes your CLI wrapper that accepts SQL (or your HTTP API that accepts SQL) any easier for average users to consume than just installing SQLite themselves and running the exact same SQL If the goal is to write/learn SQL, I agree that ayb offers nothing on top of SQLite. As SQLite is the database and ayb is the database management system, the things ayb makes simpler are on the "management system" side --- without ayb, it's hard to create a new one, it's hard to control access, and it's hard to access one from a web application. You're right that by that definition, it's more developer-friendly than power user-friendly, and I hope we can do better with future iterations.

Thank you for sharing this! One thing I noticed is that, to benefit from the natural language processing tooling, you had to first transcribe the audio. Is there some mechanism for avoiding that pre-processing step with the help of speech recognition models (e.g., OpenAI's new Whisper model)?

This is really helpful! Thank you so much!

Thank you! I wish I had found this sooner to try it out. Similar question to the other one I asked on this thread: what sort of fidelity have you seen in `archivebox`'s rewritten static asset URLs? Having to fix URLs that weren't properly rewritten ended up taking me the most time.

This is great! It looks like you can even grab WARCs using `wget`: https://golangexample.com/put-a-web-archive-warc-on-an-s3-bu...

I'm curious: what sort of fidelity have you seen in `grab-site`'s rewritten static asset URLs? Having to fix URLs that weren't properly rewritten ended up taking me the most time.

Good question! When a designer is extracting content from a customer's old website, they save all of the structured information about that customer in a single context, keyed by the customer's ID.

Hi all! B12 co-founder here. We'd love to answer any questions you have, or help you get started in using Kinase!

Thanks for your reply, @mamikonyana!

Through Orchestra, the human-assisted AI work platform we open sourced (http://orchestra.b12.io/), our customers benefit from a high-touch experience - equivalent to what you would expect with an agency custom website build - and a self-optimizing, intelligent website at a fraction of the cost/time.

Meanwhile, our automation-augmented experts are free to focus on what they do best: creative and analytical work. Orchestra and our algorithmic design tooling allow the machines to automate away the nagging repetition of mundane tasks, like staffing, process check-ins, and quality assurance.

I'm happy to expand on this answer, and you can find a bunch of papers that this is all based on at the Orchestra website.

Hi there! I'm Adam, the co-founder and CTO of B12. Happy to answer questions!

(Orchestra dev here)

If you're curious how it can be used, Daniel Haas put together a wonderful example of how Orchestra could be used in a newsroom:

http://orchestra.readthedocs.org/en/latest/example_use.html

Happy to take questions!

We're actually as interested as you in this. Doing data analysis on hipsters was fun, but we're hoping it's the start of much deeper and more meaningful studies. Joining our data with other macro- and micro-economic datasets will be awesome, as will looking at the data over time to build things like cost of living indices.

As to your chain question: they are included, and we're hoping to build a data explorer that lets folks like you filter down the data to make analyses more sound, as you suggest. You can do that sort of stuff right now on our API at http://dev.locu.com/. Let me know if you need help along the way!

Awesome! We're looking into making a more general-purpose city explorer.

And thanks for the feedback---we're on it!

Thanks a bunch! On it!

Thanks for the feedback! Sending to our designer!:)

It's OK to feel sad. If you start feeling worse, reach out to someone. Happy to talk if you need---email in profile.

Thanks for submitting! The nice thing about our publishing model is that I can edit the document with suggestions, so send them my way!

You don't ask assembly line workers to build an amazing car on their own in a single step. Similarly, you shouldn't ask low-paid information workers to synthesize amazing text on their own in a single step.

I think that your HIT design highlights several common mistakes requesters make on MTurk:

- You are underpaying for the task (would you write a good review of Berkeley, CA for $1 for a stranger?)

- You provide no aggregation or verification step, to ensure that turkers know their work should jive with other turkers' output. You also give no indication that such verification is possible or likely to happen.

- Your task output is poorly defined and open to interpretation. You may have asked a straightforward question, but I assume you placed a blank textbox on the screen and expected well-formed paragraphs in return.

If you want a great example of text synthesis of relatively high quality using MTurk for prices in the range of your budget, see http://borismus.com/crowdforge/

If you want to learn more about how to design HIT workflows, see http://projects.csail.mit.edu/soylent/ (disclosure: I share an office with and work with Michael Bernstein, but not on this work). One of Soylent's contributions was the Find-Fix-Verify design pattern, which helps with some of the problems you raise.

Your task is even harder, of course, since you require subject-matter experts in a fictional location. So perhaps MTurk is the wrong crowd for your task.

That's probably the balanced approach. I'll try to do it if I get some free time.

Thanks! My problem is not being able to find a discussion that already exists. The "post to YC" bookmarklet would create a duplicate submission, so I needed something slightly different.

Thanks! I thought about that, but there might be multiple threads for that URL, so I wanted to let the user know there might be multiple. Should I open multiple tabs?

You avoid locks by serializing transactions at any site. Since you're not waiting on disk (in memory DB) and each partition runs on its own block of memory and has its own cpu/thread, you simply don't let two transactions on the same partition run concurrently.

See http://cs-www.cs.yale.edu/homes/dna/papers/hstore-cc.pdf for cases where you want to run two transactions in the same location concurrently in h-store (the academic precursor to VoltDB).

Depending on 1) The storage engine you're using, and, 2) What other queries you run against this table

You could cluster the table on the 'time' column. That would store the data in the table in the same order as it appears in the index, and make time-based range queries a lot faster.

Whereas HDFS is to be used programatically or through a shell, it appears (though the documentation is sparse) that Ceph was designed to be mountable like most traditional unix FSs. There's the MountableHDFS [1] project for Hadoop, and so these could end up being equivalent interface-wise. At that point it's all up to how they implement create/append/delete/seek/replication semantics, which Hadoop has way more documentation on than Ceph.

The Ceph docs also imply that they have designed it so that it's easy to snapshot directories---I'm not sure whether HDFS has facilities for this.

[1] http://wiki.apache.org/hadoop/MountableHDFS

Thanks for posting! I am one of the folks working on FeedMe. If you'd like to see what we're up to now, head over to http://feedme.csail.mit.edu/ to try it out for yourself! Also, feel free to ask questions.

Glad to help:)