ljw1004

I think our memories are rooted in time and space. "Which years did we go on our family holiday to Scotland? Can I see all my videos from when my kids were toddlers? Let's see that specific photo of sunset at the Grand Canyon, I think around 2021? What was the cafe we stopped at on our road trip? How did my flower garden change since I bought my house?"

(I put in a timeline (histogram) of photos, and keyword search, to serve these kinds of needs.)

My team has a large ocaml codebase and a dual build system (buck2 and dune). The two have roughly similar: dune faster at the raw speed of invoking ocaml build tools, buck2 winning out when a lot of stuff has to be rebuilt and it uses distributed build system.

The major pain point is LSP integration, which has to be closely tied to the build system, since it's only by building that the LSP server can know a file's dependencies. Everything is all neatly available with dune. We've cobbled something together a bit with buck2 but it's not as nice.

I'm struggling to understand how a moat can have a CRACK in it.

In my web-scraping I've gravitated towards the "cheerio" library for javascript.

I kind of don't want to use DOMParser because it's browser-only... my web-scrapers have to evolve every few years as the underlying web pages change, so I really want CI tests, so it's easiest to have something that works in node.

I presume it means things like "does cross-file go-to-def and find-all-references work instantly after opening a file?" In most IDEs these things take seconds or minutes to become available, depending on project size

That's a good example because it leads to truth that NO up-front tagging will ever anticipate all the searches you might make in future. There are so many possible searches.

I figure that a combination of wetware and software is the current sweet spot. My brain usually has enough associations and context to turn every photo search into a time or place filter - "I think it was downtown last year" or "some time in summer at home" or "it had my wife in it". The photo storage system need only provide search/filter on date and place to narrow it down to a few hundred thumbnails, plus machine-learning to tag people. Which is basically what iOS provides, no more, no less.

Any other up-front categorization or tagging is basically wasted effort.

For me, I wrote a bulk tool that renames my photo file names by reverse geocoding the GPS information via Open Street Maps. That way I can do text search for place, as well as 2d map search. It's at https://unto.me

I got into ultramarathon swimming because of that movie. I've since done numerous unaccompanied 12mile swims in nearby Lake Washington (Seattle), several accompanied sea crossings, and one 24mile swim. On every swim I think of Vincent and that phrase.

How it started -- I was swimming along the shore of the lake, figured that if I saved nothing for the journey back then the worst that could happen is I run out of energy and have to haul myself into someone's back yard and ask them for help. That wouldn't be too bad, so I might as well. It's amazing how much I can trick my brain+body into going further than it otherwise would.

What does it mean to open-source or MIT-license a "file format"?

The MIT license is a license about copyrighted software, allowing people to use/modify/publish that software. But a file format isn't a piece of software.

Are you open-sourcing the specification document for the file format? (people are still free to write software that reads+writes the file format even if the specification document isn't open-sourced).

Are you open-sourcing your particular library for reading the file format? (I'm confused here, because you stressed that the file format was so simple, so I'd have expected it easy and maybe even desirable for many people to come up with libraries for reading+writing the file foramt?)

I wrote https://github.com/ljw1004/seaoflogs - an interactive filtering tool, for similar ends to what's described here. I wrote it because my team was struggling to analyze LSP logs (that's the protocol used by VSCode to communicate with language servers). But I made it general-purpose able to analyze more log formats too - for instance, we want to correlate LSP logs with server logs and other traffic logs.

(1) I wanted something where colleagues could easily share links in workplace chat with each other, so we could cooperatively investigate bugs.

(2) For LSP we're often concerned with responsiveness, and I thought the best way to indicate times when viewing a log is with whitespace gaps between log messages in proportion to their time gap.

(3) For LSP we have lots of interleaved activity going on, and I wanted to have visual "threads" connecting related logs.

(4) As the post and lnav say, interactivity is everything. I tried to take it a step further with (1) javascript, (2) playground-style updates as you type, (3) autocomplete which "learns" what fields are available from structured logs.

My tool runs all in the browser. (I spent effort figuring out how people can distribute it safely and use it for their own confidential logs too). It's fast enough up to about 10k lines of logs.

Those numbers are easy to find on google! Iowa 14-18" average topsoil depth in 1900, 6-8" in 2000. I'm not sure anyone could tell acceleration/deceleration over this timescale.

That was a deeply beautiful interview. Such profound yet relatable thoughts, humbly expressed.

I felt exactly the same. Silicon valley attitude "woah dude that's awesome". UK fenland attitude "don't get your hopes up and know your place".

I think there is an easy way to prove this to users. Make your thing be a single page self contained html file which they save into the hard disk. Then they can trust the restricted permissions with which chrome runs such local files.

If you have a tech savvy audience they can also view your thing in an iframe with only sandbox="allow-scripts" to prove that it's not making network requests.

I wrote an html/js log viewer with those security models https://GitHub.com/ljw1004/seaoflogs - it handles up to 10kline log files decently, all locally.

In what sense is this solution "fair"? What definition of fairness is being used?

This seems like a solution where rich people get the good things and poor people don't. I wouldn't call that fair.

In a democracy, everyone has an equal weight, an equal say, in the governance of the country. We often call that fair, or sometimes just.

In capitalism, every dollar has an equal weight, an equal say in what gets produced or built or done.

My job over the past several years has been to work on IDE integrations for a language (Hack, Flow). Prior to that I worked on C#/VB "Roslyn" system at Microsoft, which is entirely about exposing the internals of the compiler to plugins.

Autocomplete isn't really helped by the AST. Here instead is how it works:

(1) Query the backend "at the specified line+col in the file, what context are we in?"

(2) The context might be a qualified identifier, e.g. "foo.|" or "foo.ba|" or "MyFoo::|", in which case the context will have to tell you the type of the thing to the left of the qualifier. You then make a second request to the backend to ask for a list of members of that type. (This requires you have some common language with the backend for referring to types).

(3) Or, the context might be an unqualified identifier, e.g. "|" or "fo|". In this case you have to assemble a list of all possible identifiers in this position. (You might also want to assemble keywords here, but it's hardly necessary). This list will include all local variables; for this, you can use the AST so long as the semantics of your language aren't too tricky about what variables are in scope, or you might query the backend for them. The list will also include every single top-level symbol in your entire project. (Even in languages which require import or #include statements, you still want to list top-level identifiers that haven't yet been imported, because that's what a nice IDE experience is -- you help the user type this and then help them auto-insert the import).

This fact means that autocomplete has to be powered by the build system, has to know about all files in the project. Also, for large projects, there might be countless top-level identifiers across the project, maybe in the tens of thousands, so autocomplete needs some serious pre-compute assistance.

So how would one achieve any part of the above with an AST? The AST solely helps with part of the first step, allowing you to identify which AST node you're in. Then you still have to query the backend using the exact same API as above to learn the type of the qualifier to the left of the current AST node. Then you still have to query the backend to find all members of that type. And you still have to query the backend for its build-system knowledge of every type in the file.

In other words, the AST has barely helped you at all in your quest to implement autocomplete.

WinRT isn't tied to store apps. You can use most of it just fine from .NET/C++ desktop apps. (not the XAML bits obviously)

https://www.nuget.org/packages/UwpDesktop/