munin

Matrix is an unserious project and the client ecosystem is a train wreck. The server ecosystem is not much better. The Element people, who are kind of the default Matrix people because as far as I can tell are the only people getting paid, will tell you that this is because a bunch of IT integrator companies unjustly profit off of the open source work by selling services to European companies but contributing none back to either Element or other open source Matrix projects.

The first issue I'd like to address is that one: as a small business, I tried to purchase software from Element and was told that I was not large enough to justify their time. Fair enough, I only wanted a 200 seat license and I was willing to pay per seat, but I guess they really want the high value contracts if they have a limited sales team. However, it is a bit much to go from that experience to their justification about the structure of their project. Maybe they should think about taking some sales opportunities that present themselves?

Then there are branding and release decisions around the clients that Element makes. There are two projects in the client space from Element: a client called Element, and a client called Element X. Element X is the newer one. Element (do you see how this is getting confusing yet) is simultaneously at different times an Electron desktop app, a mobile app, and a web app. Element X is becoming all of those things but the feature parity is not even between them. Element supports "legacy" Jitsi for voice and video calling while Element X supports newer Element call - which is different from legacy Element, Element call is a webRTC implementation native to the Matrix ecosystem while the "legacy" Jitsi is a way to send clients a URL for Jitsi calls and have them shell out to another app to actually implement the call. Fair enough. However, the desktop Element X client does not yet support new Element call but the "old" Element client does support both "legacy" Jitsi and new Element call. And the Element X mobile app cannot call the old Element mobile app - but I think the other way around can. Even getting your head around this as an IT person is confusing.

To add insult to injury the new Element X app on mobile is in some ways a downgrade because they integrated the cloud vendor push notification services into the app, so even though you have "sovereign" and "self-hosted" infrastructure you're still, on a good day, leaking meta-data about your chats back through to the people you were trying to decouple yourself from anyway. You can run your own push notification services for this mostly if you want and all your mobile clients are Android but like, why.

Then, there's desktop client usability. During account setup, Element/Matrix makes a big ceremony out of establishing your cryptographic identity. Perfect. And as part of that you write down a 10-ish something word passphrase that is a recovery sequence for said identity. Perfect. Then some network hiccup happens that disturbs the Element client like some kind of prey animal and it spontaneously logs you out. You log back in, but there are no fields or options visible to use that recovery passphrase to restore your cryptographic identity. Your only option is to reset your identity, which makes all prior chats you have had unreadable. That part at least makes sense but why have this recovery story if it is not tested or usable in the app? This is probably an Element thing but in my research I have not found a client that people say is more robust, though at this point I'm open to trying.

It's also possible that the way most people use this is as a web app, which is to be fair more robust. It does seem worse from a security point of view to have one central web server dealing in most of your users plain text, though. At that point, why not use Mattermost? I guess they're even more hostile to their users/customers, for some reason.

Finally, there's the server ecosystem. The thing that is frustrating to me here is the interplay between Synapse, Matrix Authentication Service (MAS), and OIDC. This, as far as I can tell, is all intentionally hostile to drive you into Element's commercial product offering. Which I find especially galling because they won't sell your their commercial offering anyway, so you're going to have to figure it out for yourself. Synapse has some legacy support for OIDC which you are going to need to enable for backwards compatibility. However, for forwards compatibility with Element X, you are going to need MAS. Synapse is a large, mature Python project. MAS is a single Rust binary which is simultaneously a server and CLI to do user management. You'll need both configured against your OIDC provider. Why didn't the new OIDC features just get integrated into Synapse?

I think that a lot of this is an outcome of the fact that Element is very literally in a "the old world is dying and the new world struggles to be born" situation at this time. I do have a lot of sympathy for being in the position of having huge companies - especially companies as annoying as IT outsourcing and integration - make a line of business out of configuring and installing your open source software. However, I have to say, having spent some of my professional life now also configuring and installing this open source software, I understand why those IT outsourcing companies have a moat. If the open source software was easier to install and use, perhaps those companies would have less of a moat. It seems to me that at least some of the story from Element is that if they make the ecosystem harder to use and understand, then people will take their money and the business will survive. However, in my experience, they won't take your money anyway.

I suspect this is a staged Zoom call for marketing purposes. How do I know this? Everyone is smiling.

Oh come on, everyone is smiling because there's an infant in the leftmost 2nd from top video. Or a picture of an infant. Someone did "hey look at my cute kid" and everyone loses it because that's what people do.

I appreciate this but in the context of this discussion when a PhD asks “is it better in industry” they do not mean going to work at a meat packing plant.

Your larger point is true though and it would be great for there to be equity and dignity for all people of all professions!

Oh not to worry, academia innovated on this concept too and for non-US citizens advisors can simultaneously hold both a students F-1 visa and their career future over them.

Your points are valid though.

science would be healthier for having more staff scientist positions in larger groups

I think this is a good idea but not for these reasons. Big labs frequently do have research scientist type roles but those roles are harder to fund so it is sort of a "rich get richer" deal. As I understand it, just cast your eyes around in a university and ask "who pays for that." Professors are paid for by the students they teach and the grant money they bring in. A research scientist might not have either so why would the university commit to paying for them perpetually and how would they afford that?

They could commit to funding some number of research scientists at the expense of some well paid administrators (gasp) but I think this only adds a constant factor of slack to the system, and those research scientist jobs would quickly become as competitive and cut throat to get as TT prof jobs, perhaps more so because many TT profs really don't want to teach or do service work so if you gave them a role that was just research but with the job permanence and freedom of a TT position they would kill for it.

It’s not as common but the incentive structures in academia are uniquely enabling to abusers. In industry it’s much easier to leave tyrant bosses. I agree with your points.

Many of the positive incentives don’t exist in industry though, an “up and out” culture is rarer in industry. When it works in academia your supervisor is positively invested in your growth, in general in industry your supervisor doesn’t care about your growth. If they need someone with new skills and you learn them fine but they can also fire you and hire someone with those skills (even though this is probably a net negative for them due to retraining on job specific stuff, it’s still seen as a net positive by management)

There is less abuse though. Full stop. Sorry you’re in this situation.

Teaching classes effectively is very hard and very few people can do it. It also doesn’t pay that well and is draining and often times not that rewarding. Universities try to get people that are good at it as professors by bribing them with the respect and autonomy that the title “professor” confers. Outside of the classroom, the professor does whatever they want to advance their own agenda.

I would think online only programs would have a hard time competing with universities for people that can design and execute effective courses. There are so few that most universities don’t even employ many of them. Why would someone like that work for an online school when they could work for one that would give them tenure, funding, lab space, grad students, etc?

Not to worry, he also doesn't donate to his university.

2000 or so.

I dunno, 40 years of living in other places, stuff like this only happens in Texas and the South. There are definitely assholes in NY and Boston but they won't get up in my grill for "looking gay" on the street so that's pretty neat.

I lived in Dallas and it is the only place I've lived where people would scream homophobic slurs at me in the street. I wouldn't say I'd never move back, but the situation would have to be dire for me to consider it. Definitely cut your hair if you visit.

Sounds like Ravens progressive matricies.

How do you decide which scientists to pay? Pay them all? Then everyone becomes a “scientist” but does no work and draws a free paycheck.

Pay the ones that produce good work? What is good work? Right now, that is decided by number of papers. That metric is gamed to death, that is what publish or perish culture did. Choose any other metric, it will be gamed too.

What if the PhD meant something such that having it was enough? That was all you needed, then you would just get paid to do science. Such a gatekeepeing would require graduating a lot fewer PhDs. How to do that is open: admit fewer? Or psd fewer? If the latter, maybe master out a lot more? Anyone that makes it through gets a guaranteed paycheck, but almost no one makes it through.

Where you are mistaken is that there are a large number of graduates because there is a large demand for research. There isn’t. A given professor needs N papers to get tenure. Say that N = c*k for c researchers and k papers. N is so large that c must be >2, so for the professor to get tenure, they need assistants. Where do they come from? Graduate students. What do those students do when they graduate? The professor doesn’t care because they have tenure now.

Research funding has been decreasing over time, which means N increases, which, counter-intuitively, means c increases, which starts a negative feedback loop because the amount of researchers competing for funding increases while the amount of funding available decreases, still further increasing the number of graduate students needed. That explains what you see.

From the letter linked in the article:

In sum, your own oral and written statements to the OIE investigator, the Dean’s Office, and others clearly establish the following:

On May 8th, 2019, carrying bolt cutters and leading a group of non-affiliates to campus, you forcibly entered Garland Hall in the middle of the night. By your own admissions, your actions were premeditated and you expected that your actions could result in a violent confrontation with students and others in or around Garland Hall. In fact, you believed the group of non-affiliates you brought with you could become violent. As a faculty member at Johns Hopkins University, you created a dangerous situation that could have ended in serious harm to our students, yourself, and others in the community.

Prior to May 8th, 2019, JHU administrators had clearly and repeatedly instructed you that Garland Hall was closed and that you were not permitted to enter, despite your requests for permission to access the computer servers in the building. You acted in deliberate defiance of the administration’s directives. You have flagrantly and unapologetically violated JHU directives and your actions have endangered the University community. Further, you stated you feel no remorse or regret for your actions.

I don't understand how these events are acceptable. Neither Daniel Povey nor JHU seem to dispute any of the critical facts. How is recruiting some muscle to go bust some heads, in any universe, an acceptable course of action? For any employer?

Smart people get everyone else to bog themselves down with "second-order thinking" perpetually tying themselves in mental knots wondering endlessly "what happens next" and "should I or shouldn't I." That is because smart people that get shit done will not think twice, do the thing, and then quickly re-evaluate the results and proceed apace. Smart people that get shit done, outperform by actually executing. You can't predict the future. Asking "but what about in ten years" is the height of hubris. The only answer can be "who knows."

We could tell my story instead. My uncle was an electrician, I didn't take high school that seriously, so my parents wanted me to apprentice for my uncle. I said no thanks, got a low level tech job, worked my way through undergrad and then PhD school, and now I have a cushy, well paid job where I sit indoors in a private office for eight hours a day and think for a living. My uncle had to crawl through attics on hot summer days, fight with rodents and snakes, and deal with live electrical wires and people who don't pay their bills. On top of that, he wound up dead from mesothelioma, probably from spending most of his days just breathing in insulation fibers.

So, thanks but no thanks, basically. You can romanticize the trades all you want but from my experience watching my family and friends from back home, "the trades" are exchanging the better years of your life in time and health to rich people in exchange for an early and painful death.

Sort of, but not really. From the original report:

It was the decision to cease the processor operation which finally proved fatal. Restart is not feasible since attitude is too difficult to re-calculate after a processor shutdown; therefore the Inertial Reference System becomes useless. The reason behind this drastic action lies in the culture within the Ariane programme of only addressing random hardware failures. From this point of view exception - or error - handling mechanisms are designed for a random hardware failure which can quite rationally be handled by a backup system.

Although the failure was due to a systematic software design error, mechanisms can be introduced to mitigate this type of problem. For example the computers within the SRIs could have continued to provide their best estimates of the required attitude information. There is reason for concern that a software exception should be allowed, or even required, to cause a processor to halt while handling mission-critical equipment. Indeed, the loss of a proper software function is hazardous because the same software runs in both SRI units. In the case of Ariane 501, this resulted in the switch-off of two still healthy critical units of equipment.

The original requirement acccounting for the continued operation of the alignment software after lift-off was brought forward more than 10 years ago for the earlier models of Ariane, in order to cope with the rather unlikely event of a hold in the count-down e.g. between - 9 seconds, when flight mode starts in the SRI of Ariane 4, and - 5 seconds when certain events are initiated in the launcher which take several hours to reset. The period selected for this continued alignment operation, 50 seconds after the start of flight mode, was based on the time needed for the ground equipment to resume full control of the launcher in the event of a hold.

This special feature made it possible with the earlier versions of Ariane, to restart the count- down without waiting for normal alignment, which takes 45 minutes or more, so that a short launch window could still be used. In fact, this feature was used once, in 1989 on Flight 33.

The same requirement does not apply to Ariane 5, which has a different preparation sequence and it was maintained for commonality reasons, presumably based on the view that, unless proven necessary, it was not wise to make changes in software which worked well on Ariane 4.

Even in those cases where the requirement is found to be still valid, it is questionable for the alignment function to be operating after the launcher has lifted off. Alignment of mechanical and laser strap-down platforms involves complex mathematical filter functions to properly align the x-axis to the gravity axis and to find north direction from Earth rotation sensing. The assumption of preflight alignment is that the launcher is positioned at a known and fixed position. Therefore, the alignment function is totally disrupted when performed during flight, because the measured movements of the launcher are interpreted as sensor offsets and other coefficients characterising sensor behaviour.

Steering was controlled by the on-board computer, which mistakenly thought the rocket needed a course change because of numbers coming from the inertial guidance system. That device uses gyroscopes and accelerometers to track motion. The numbers looked like flight data -- bizarre and impossible flight data -- but were actually a diagnostic error message. The guidance system had in fact shut down.

The real question is, why was the diagnostic error message interpreted as flight data?

I recommend reading the original report, which has much more meat: http://sunnyday.mit.edu/nasa-class/Ariane5-report.html

The author obviusly doesn't realize that the company making the wafers for Raytheon ISN'T ALLOWED TO USE THE CLOUD.

This is probably going to change. People like you said the same thing about health data, and student data. The savings were so tantalizing that the regulators and stakeholders figured out how to make it work. What do you think GovCloud is for? C2S and "Secret cloud"?

Our university had a 3-4 person dedicated Exchange team. When "Google Apps" came out, people wanted us to switch to that from our old mail server stuff. Go figure, why would you keep using pine and squirrelmail when you could use gmail? "It can't hold student data" the IT team said, "it isn't certified for FERPA or ITAR." Okay, true. Fast forward two years, now Google's "Apps for Education" can deal with both. The switch was sudden and brutal and the university no longer has a 3-4 person dedicated Exchange team or an Exchange deployment of any kind.

I was over-simplifying to communicate the repetitive and dull nature of a job I'm very happy to no longer be doing. Really the game was more about "anomaly detection" and the sensor indicators/measurements were far more continuous than categorical, and their outputs had to be weighed against past experience and context of the monitored components.

Two points: first, even if "solid" universities are right to run away from this (and IMO they don't run away from it, they just move the IT program into the business school where you don't see it any more) there are still many other universities pumping out students into a dead career field, which should be concerning. Second, this is not just about IT programs. Computer science programs are impacted to. From the OP:

But instead of five backend developers and three ops people and a DBA to keep the lights on for your line-of-business app, now you maybe need two people total.

All nine of those people would probably have been CS graduates, or at least many of the backend developers would be (and perhaps the DBA). Or they would be people that thought of themselves as "developers" and not "IT" for whatever that distinction is worth now.

This kind of change has also impacted me. It shows up when I'm trying to give students advice about starting their careers, and I realize that the first jobs I had (system administrator, SOC worker) have been replaced by robots. Especially in the SOC, I was a "Tier 1" analyst that would do monitoring (watching a bank of green lights waiting for one to turn red) and first level triage and analysis. This has been replaced by ML driven data processing systems.

So I think the apocalypse is double-bladed: while automation kicks a bunch of current workers out by making them immediately redundant, it also freezes out the next generation by removing entry level jobs and not really replacing them with anything equivalent. Meanwhile, universities and vocational ed programs won't get this memo for another ten years so they will continue to happily propel waves of students onto a set of closed and locked doors.

What's hard to get from this is a sense of quality. For one, professionals have a hard time nailing down what a "quality bug" is - if given a few different bugs and told to rank them, you could probably come up with some ranking of your own, but it would probably be different from your coworkers, bosses, customers, etc. So that sucks. However there are also reported findings that most people could agree are low to no value. How much of those show up in bounty programs vs. internal finds or commissioned audits? Of the people at the top of the distribution, how many of them are reporting many high quality finds, or spray-and-pray with lots of chaff? Are there "diamonds in the rough" near the bottom that only get out one or two a year but they are high quality? Is that an indicator that if those reporters could be given full time jobs doing audits, their productivity would rise?

There are cynical and practical reasons to use LaTeX.

The cynical: While most conferences in CS advertise both a LaTeX and Word template for submissions, if you submit an article in Word, it seems quite likely it will be desk-rejected for being quackery, possibly without even being read. Is this fair? No. Has the scientific/mathematical community lost out on any great ideas due to this prejudice? Probably not. Think of this as the ultimate, final form of the clear plastic binder from Calvin and Hobbes, except Calvin was right all along and you probably can get away with saying that bats are bugs if you write it in LaTeX. This says more about the process of science than it does LaTeX.

The practical: I can think of three pragmatic reasons.

It seems that nowadays, good science is collaborative. Single author papers are rare. How is it that multi-author papers are written? Writing in a text-based markup language allows co-authors to easily send each other changes and keep a versioned history without having to worry about "track changes" and using file names as tags, and worrying about cross-compatability between ancient version of Office on Mac and Windows and that one person who only has Linux and so opens things in Libre.

In my experience, people do want to get into the weeds with formatting in LaTeX for many reasons, good and bad. For example, how can we shave off two lines of text so we're within the page limit (yes, even though the proceedings will all be published electronically we will get desk rejected for being one page over the page limit where the only thing on the last page are two lines from the bibliography, c'est la vie). That is a bad reason. But good reasons could include improving the look and feel of a document, or controlling where figures are placed, or drawing figures using TikZ. Additionally, having the sum total of the formatting in plain text has advantages. Do you think that undo really un-does everything in Word? Can't you remember some times where you do something, but then un-do and then do "the same thing" again and a different result occurs? LaTeX has many foibles but that is not one of them. Though, of course, sometimes I will wish you luck in understanding exactly why what you wrote in LaTeX works the way it does now. But for example, it is possible to define new mathematical symbols, operators, etc. in LaTeX in a way that is not possible in word processors.

The third pragmatic reason is that everyone else already uses it and some day or another your text will need to be turned into LaTeX for publication. I used to keep a lab notebook in Markdown and sometimes I would feel the desire to copy and paste some text from my lab notebook into a paper (if what I wrote was particularly lucid or well written, or was at least a good place to start from). This almost never ended well. If you have a draft of the paper in Word you might easily spend many days converting it from Word into LaTeX. If it's sufficiently complicated you might never get it converted. This could be a barrier to getting collaborators or submitting at all.

I think that everyone that works with LaTeX, knows, deep down, it is absurd, and the arguments the author runs into are not really arguments anyone would stand by if pressed. I think if anyone is going to write something entirely in prose, that they will never care about the versioned history of or want collaborators on, and don't care if the finished product looks good or bad, you can use a word processor, or at least, this is my criteria for whether or not I use a word processor. Am I drafting an outline of how we're going to write the proposal? That happens in Google docs. Am I actually writing the proposal? That happens in LaTeX, because I want control over the typesetting to fit in the page limit, I want it to look good, and I want collaborators to be able to edit the document and send me plaintext patches rather than docx with track changes maybe or maybe not enabled.

It would be cool to see an evaluation of this against baseline. Software is a giant warehouse filled with soggy pinatas. Swing any kind of bat in there and you'll get some candy.

Cigarettes are legal and people still smuggle and sell them on the black market to avoid paying taxes. Your scheme would be mostly indistinguishable from the current prohibition and would still be dominated by cartels and SWAT teams.

I spent two summers of my PhD at research labs with private offices. It was awesome. At grad school, I work in an open plan lab. When reading or writing papers I find it very difficult to concentrate when people are having overlapping Skype calls on speaker phone or loud conversation near my desk. In a private office, I could take a paper and a whiteboard and really focus.

Also, interruptions become less jarring. If I'm focusing deeply in an open office, I have headphones on. That means if someone wants my attention, they have to come up close behind me and tap me on the shoulder, which is pretty intrusive of the personal space. In my private office, I wouldn't have to wear headphones in the first place, and if someone wanted to interrupt, they would knock on the door. Much less intrusive.

Interruptions also become less frequent. In an open office people seem to think that they can just come over to your desk and grab you to ask you, well, anything. The closed door has a good deterrent effect, people will interrupt you only for serious things, like lunch, "the building is on fire," and so on.

That's not really what I meant. What I meant was, you seem to act as if enforcement of this policy would require a lot of logistical work on the part of the airline to ensure that the passenger wasn't separated from their bags, or that they were compensated for whatever they had to ditch due to security. However, that is not actually the policy or law they have to follow, and they are free to do arbitrary things. This makes enforcement much easier.

Whether or not that is "sane" or "right" or "wrong" isn't material to my counter-point.

If they wanted to, once detecting a battery, they could:

1. Open the luggage themselves

2. Remove the item containing the battery

3. Replace it with a note saying that the item was removed

4. Close the luggage back up

5. Put the luggage back on the plane

6. Discard the item

What are you going to do, sue them? Post bad things about them on social media?

This paper is cited by "Taming Performance Variability" but is probably worth a read if you haven't read it before and find this paper interesting: https://www.seas.upenn.edu/~cis501/papers/producing-wrong-da...

The authors of "Producing Wrong Data Without Doing Anything Obviously Wrong" find a bunch of "optimizations" that don't do anything but change performance by some percentage points. I think about this every time someone says big companies care about "sub percentage points" optimizations - how much of that is in the noise? Work like these papers ask these questions, which I think is good. Computer scientists seem to have a worse time with empirical evaluation, statistics, and experimental design. Not that any field of science does super good at this all the time, but CS seems to have this as an "unknown unknown" deficiency.

It's backed by z3, an SMT solver. There are theories in SMT for strings that can sometimes answer questions of your first flavor, but I don't know if they're in z3.

Solvers can also answer questions of your second variety, but again, sometimes they can't, and they can't be guaranteed to in general. For an example, consider this example: https://rise4fun.com/Dafny/Cube where the "ensures" on the return value enforces exactly that.

You also should be able to write set-inclusion style queries like your second clause, i.e. a function that takes an element and a list of elements as an input and only returns true if the element is contained within the list of elements. I think? I'm not sure why you couldn't.

Of course, whether or not that does what you think it does or not depends on how something can get written to that database - if there was a path to write something new to your exclusion list from elsewhere in your application, then the "verified" code would return true when you would think it would return false but it was doing exactly what you told it to do. Is this a problem with your design, or the verification? I'd argue the design, but verification-nihilists would probably say it's a problem with the verification.