dfraser992

Tech lead for WEMC here - see https://tealtool.earth Straightforward charts of climate related data for different countries and regions around the globe

For temperature and a few other variables, it shows historical data from the EU Copernicus service (C3S) along with three different projected series out to 2100

for CO2, it shows the latest historical data

The charts are concerning and I am sure my co-workers are not hell bent on faking data to scare people just to get more funding; they work too much and go to too many meetings.

There is no way to DM on HN, as far as I am aware. I would be interested in talking - I have some Ml/NLP based ideas and some free time... See my profile info for my website/email

Interesting idea.... I have been idly working on an web crawler / indexing / organizing system that uses NLP etc to sort of solve the OP's problem(s) - if you don't mind, could you send me an email about your needs? See my HN account for contact info.

I have been collecting bookmarks for some time now as well - all unsorted... Doing it manually would be a nightmare so I have been thinking about making a SaaS - webcrawler + AI (NLP + clustering)

It would at least accomplish a preliminary sort/grouping; manual cleanup or fine tuning would always be needed, I think, but at least the bulk of the work could be done in an automated fashion to give the user a head start.

Would anyone else want such a SaaS? I thought about how to charge for it, but ideally it is a one-time operation, so charging anything more than $1 to $5 doesn't seem reasonable. And the privacy issues ... bunch of practical problems, so I may just write some OSS.

Given you already have a few years of experience which is a bit more important than just a degree (IMO), I would not worry so much. "Computing" is not "IT support" (no one gets a degree in that) and those are good unis you mention, so why in the world would they be offering a silly degree in "IT support"? I have never heard of such a thing actually.

If the company (manager / HR people) you are interviewing with actually think like what you worry about, you do not want to be working there because that is a sign of a bad company that knows nothing of IT or respects developers. We are not mere plumbers (I am from the US and the UK attitude towards developers has been a bit annoying over the years, but it does seem to be changing). Although it'd be nice to be paid as much as plumbers...

My M.Sc. from Edinburgh was in "Informatics" because that is what they call Computer Science, and my M.Sc from UEA was in "Knowledge Discovery and Data Mining" aka AI/Machine Learning. I would assume you will have to do a final-year project, so a line in your CV describing that ought to be enough to show you are a developer and not a help desk jockey. (along with your previous work experience)

But I wouldn't call it "Computer Science" though - if some jobsworth actually decides to validate the contents of your CV (they usually don't and you do have work experience), the renaming of the degree might influence their opinion of you etc... e.g. "did they actually go to ICL?"

Between this and FTX, I am really starting to think the death penalty ought to be applicable for financial crimes beyond a certain magnitude. The intangible cost to society / peoples' lives is incalculable, but given the fundamental hypocrisy of the West anymore, the only thing that truly matters is money, especially rich people's money and thus grifters and con artists like SBF and Trump are allowed to skate by with a slap on the wrist. 11 years (no way is she serving that) is a slap on the wrist.

Be very explicit, from the beginning, that you do not do fixed price contracts (which is essentially really what clients always want) unless the job is obviously trivial enough to estimate properly - and even then, there is always a chance something might occur.

Make better estimates to account for such unknowns like having to install packages because you're dealing with a codebase someone else developed - be very detailed in these estimates you provide as well and provide not a single number but a range.

If they still grumble about the cost, then maybe offer to reduce the cost of trivial work like setting up the necessary environment by 1/3 or something (only if the client is a decent one and just needs a bit of education...)

Provide good info about the reasons for extra time - you're trying to get the client to ultimately trust you, that you know what you're doing, and are focused on trying to not spend as much of their money as you'd like. So keep them in the loop as much as possible w/o being annoying about what is going on. A daily or weekly update? A heads-up before diving into something time consuming? E.g. if the code you're dealing with is garbage, explain that in a professional way (and clearly, but not too much detail) so they understand the issue (to the extent they need to understand it)

So basically it's a process of educating the client, over time or however efficiently enough, about the true nature of IT work. Metaphorically, you're a plumber or a general contractor, or a carpenter, not a factory worker. Everyone knows that sort of work always involves more time than expected sometimes. If they still complain, then they're a bad client and let them go find someone on Upwork instead.

Why hasn't anyone mentioned IR35?????

I am in the UK - it seems the company you are working for is in violation of the IR35 regulations (but I'm not entirely sure of what nuances there are given you are in another country). i.e. you are a contractor, but are being treated as a full time employee with none of the benefits of such. e.g. they 'should' be paying the government for National Insurance etc.

I have been contracting for 10+ years so know something of IR35 but have always made sure the contracts I sign are quite clear about me not being an employee etc - no long term contracts etc. So I'm not an expert here.

If they are violating IR35, which I really think they are, then you have quite a lot of power at your disposal if you can play things right. You can either get them to start respecting you (but it doesn't sound like that is the best idea because staying is a __bad idea__) or sue them or just turn them over to HMRC.

In any case, you should talk to a solicitor - get everything you can documented and well organized, then any good employment lawyer can tell you the cost/benefits of turning them over to HMRC / the authorities. Google IR35 and employment law / solicitors ... pick one that has a well designed website. Talking to them ought to be free for the first 1/2 hour (I would think) or you should be able to send an email with a good summary of the situation w/o incurring any costs.

An upvote for you (if I could). I'm American but have lived in the UK for near 20 years. My great-whatever grandfather signed the Declaration of Independence so I'm hardly a royalist... So WTF am I doing living in the UK??? (the NHS etc...)

My general sense is that of respect for the Queen as a symbol. She did it right and wasn't a useless numpty like ... oh... all of the rest of them. Primarily nothing but B list celebrities. William and Kate seem fine enough, Harry and Meghan are .. irrelevant except to the nonces who have no actual lives, and let's not discuss Andrew...

Hopefully Charles will use the "soft power" he supposedly has to corral the professional sociopaths destroying this country (e.g. wind and solar power, given his supposed environmental leanings) but I don't know.... it very well may be all downhill from now. England (and by extension all of the UK) is destined to become a failed state.

Which is why I am looking hard at moving to Scotland (soon to be independent!) or even the EU to get the F out of here ASAP. It really is a transitional point.

No, in the UK, the loser will pay for court costs, in general. I had a bit of legal trouble a few years ago - the other side was trying to play "solicitor" to use the law as a way of extorting me (aka threatening to make criminal complaints over various (bullshit) things) so I had to hire someone to write a polite 'go fuck yourself' letter eventually. Solicitors are duty-bound to try and get things to settle out of court, so dealing with wanna-be solicitors is always a problem for them.

https://www.ashurst.com/en/news-and-insights/legal-updates/q...

What's unique about the UK is that unlike actually civilized countries (incl. the US), there is no legal requirement to negotiate in good faith. It is expected for you to do so, but if the other side figures out that you haven't and tries to take you to court over that, then the English court system will say "tough luck". Manipulating the other side to your advantage should be expected, they said. There have been High Court cases that have established this principle. so doing business with the English is always possibly problematic (e.g. Brexit and all the lies, the mentality behind all that etc). Anecdotally, I have heard from foreigners that doing business with English people is more of a headache than with ones from other countries. And I have lived here long enough to 'understand' the English better than most Americans...

I might have such a thing, though it has been over a decade since I did any webdev; not sure I still have the code. I was doing the backend for a project while the frontend work was being done by a guy in SF (I was in London). So remote working, all those issues, etc.

He kept delaying on giving me code until the client (and I) got upset enough. The code he gave me was garbage despite his claims of really knowing Javascript (very good interviewer...). It was horribly slow, unfinished and I spent two weeks trying to comprehend it so I could make a decision as to throw it all out or patch it.

One of the biggest problems turned out to be a function or something (details are fuzzy) that was generating a object that corresponded to the data/information the user would see in the UI. His code was not tracking whether a specific object had already been generated (i.e. no need to generate a new copy if the data was already been shown) so in the end, far too many copies of these little objects were being created - and kept around.

So it was a memory leak, but the memory wasn't getting lost, just forgotten about (if that makes sense). Enough time and the browser would get really slow, etc. Thank god for infinite RAM! (not)

Fortunately I learned my trade fixing problems with Borland's heap management (don't ask me for any details, that was 30 years ago) so I had the leet skills and patience to track down every last byte of RAM... This was before any fancy tools from Chrome were available, AFAI remember.

Never used Valgrind or know what the current state of the art is in regard to web dev / Javascript, but keeping track of RAM usage in Javascript (or Python or ...) is always a necessary task to pay attention to.

Pattern recognition

I print the webpage to a PDF and store it that way, then fix up the metadata as I see fit. And that is a good feature - make a request on the Github repo and see if the lead developer responds.

No one has mentioned iLibrarian yet. No idea how its feature set compares to Zotero or Mendeley (I haven't used them much as I found them annoying in some ways) but there is a free self hostable version as well as the SaaS product. It is sufficient for managing my archive of papers.

https://i-librarian.net/

I forget why I didn't like Zotero - I am using iLibrarian [0]. It is positioned as a SaaS, but the source is available and you can host it yourself. But I can't speak to how it compares to other software as I am not continually using it. It seems I just like collecting scientific papers rather than actually reading them.... Maybe if I live to 150 will I manage to read them all.

[0] https://i-librarian.net/

And I wholeheartedly second the fact Mendeley is Evil. I got sick of the bugs.

I have used Neo4j and Tigergraph. Neo4j's community version being a toy is something I agree with - haven't used the enterprise version, though. But its memory requirements do seem to be ridiculous. Tigergraph, OTOH, their free version could readily handle much much more data (and faster loading) than Neo4j.

But Tigergraph is UNIX only and built on C, not Java (probably an asset, TBH). Some UNIX sysadmin experience would be useful, but not mandatory, if you want to understand the innards of the software.

Cypher is perhaps a bit friendlier / easier to learn than GSQL. The necessary mental model for understanding GSQL takes a bit of time to comprehend.

I haven't looked at Neo4j in quite some time now, so I may be wrong here now - Tigergraph comes with some capabilities Neo4j doesn't like offering a built in REST functionality and easily embedded custom functions. When I thought about trying to do the equivalent with Neo4j, it would have required a microservice sort of proxy.

Neo4j has a lot of extra "stuff" in their ecosystem, like Bloom, but Tigergraph has been catching up in various ways, I think. Their documentation has gotten better since I first started using it and more add-ons etc.

My immediate reaction was "why not Parallel?" but realized I'm not immediately sure how to deal with multiple servers... Off the top of my head.... have Parallel manage threads that execute remote scripts (using SSH)? SCP to copy files as needed to servers... You would have to figure out how to manage the utilization of resources (i.e. the servers) to not overload them and stuff like that.

using Dask is also possible. I have a Dask based system I built that farms out jobs/objects to members in a simple cluster I set up by hand. The Python class is responsible for managing an external process (such as the Java based Stanford NLP toolkit) or a Python script that uses spaCy. Each job gets sent a block of text which is them turned into features by whatever tool is being used. This class uses the 'subprocess' library in Python to deal with the external processes. Dead simple multiprocessing on a cluster w/o the complexities of SLURM. Setting up the venv on the servers in the cluster is the main hassle, but Dask works fine. Send me a PM if you want a copy of this class to get an idea of how it works.

But what if it wasn't clearly labeled? I did my MSc thesis on fake reviews and discussed the phenomena known as "covert marketing" a bit. e.g. a guy you're talking to in a bar at some point steers the conversation to the excellent beer he is drinking and heavily recommends it to you. Good enough actors will be very convincing. "Influencers" are a somewhat more ethical alternative that takes advantage of humans' lemming-like nature.

I mean, quite a lot of people truly believe Hilary Clinton is the mastermind behind a DNC run pedophile ring. Yes, she is a problem, but that theory is completely schizophrenic. A NPC masquerading as a real person who spouts positive talking points about Tucker Carlson's respect for Hungary is quite reasonable compared to that and it will suck some people in.

So all it takes is some right wing developers for a not-entirely-just-a-game like Second Life or Minecraft to introduce a bug that allows certain instances of NPC to be unlabeled... or a mod to a game that drives a NPC... and an equivalent to GPT-3 funded by the Kochs or the Mercers...

Very hypothetical, very hand waving. But it is possible. So I can see the PR and legal departments flat out stopping this idea.

What are you going to do in retirement? I read an article recently about how people tend not to really change their life or behaviors once they 'retire', i.e. those plans to completely reinvent yourself are not entirely realized.

For example, me - I kept pushing off my impulses to ditch IT to work on the novel/play/screenplay... a compulsion kept me focused on the job despite going thru what you are a few times. It was that creative drive, I guess and I could never redirect it. And "responsibility", getting money for my hobby, etc. So what am I doing now, now that I've "officially" mostly retired (aka not looking for a job because interviews suck and why aren't in management and you're too old / can't code like leet 20 year olds...)

Am I working on that novel? No, I'm chasing down a bug in scikit right now so I can finish this ML research paper. Christ... And I don't even have to write this paper, I'm doing it for the hell of it. Maybe someday someone will read this paper (or the next couple) and get something useful out of it.

I guess what I'd say is figure really figure out "why" you did all the things you did and what motivated you despite all the headaches. For me, it was the idea of "service" (once I got past the ego trip of being in my 20s/early 30s) - my role was to create some bit of order in the universe for people or whatever that needed it in the way of IT stuff. That has what carried me along despite working (unknowingly) for sociopaths with actual criminal records a few times (don't ask...) Businesspeople, capitalism... don't get me started.

If you can't afford to quit now, then find some sort of external motivator like "service" to keep you going. Or perhaps a completely different tech to deal with - have you experienced the full scope of all of it?

Or perhaps this is a roundabout way of saying "get a hobby". IT has always been fundamentally a hobby for me. Actual coding isn't so much anymore, but now that I've been exposed to academia the last few years, I am sliding into research-y stuff, writing papers no one else is going to bother to. It is still tech, but parallel to my other impulses - the fundamental impulse of creating something out of pure ideas.

When I REALLY retire, I'll write that novel!

My suggestions (I have just decided to officially retire, so money things are on my mind):

1. Read Rich Dad, Poor Dad. I have not personally read it, but it is supposed to be pretty good at explaining the basics. I also vaguely remember some criticism, but can't remember the specifics.

https://en.wikipedia.org/wiki/Rich_Dad_Poor_Dad

2. Index funds are good for people like you (you sound like my sister). If you're worried global capitalism will collapse (along with everything else) in the next 30 years <which it actually might> then... well, no one is going to be worrying about money at that point.

3. Investing in stocks takes some effort - you need the emotional intelligence to control your impulses (I am still working on that) as well as some understanding of the economy, economic forces, etc. How pump and dump schemes work is another good topic to understand. People can go down the rabbit hole as far as they want - I am not a "research the financials of a company in detail" person for instance. And options... very gambling-like. I have done ok over the years, but more just because I understand tech and have a decent sense of how companies might thrive -- i.e. invest in what you understand.

4. Relying on state pensions, given the social and economic trends, is a recipe for disaster. Politicians are kicking the can down the road all the time and if the social trends I spoke of become real problems, then pensions will be the least of it. I am certainly not counting on seeing any Social Security or National Insurance (I live in the UK) when I hit 65.

5. From the sound of it, you probably should focus on never getting into debt and building your business / passive income. Then you need only worry about what you are entirely responsible for and not on other people which seems to be a motif in what you're saying. But that means not being an employee which is a lot more secure...

6. If I'd held onto my 10 Bitcoins for longer.... well, if I'd bought some when I heard about when it was $0.01... if I'd sold that NSOL stock in August 2001... You can't be terrified of what might happen (or what didn't) and become paralyzed as a result. Go read the Chinese parable about the farmer who loses everything after gaining everything. Or the Book of Job.

7. And like you say, don't blindly trust other people with your money; the world is getting more corrupt, I fear. So that means YOU HAVE to be responsible for it and figure out how to take calculated risks and what your limits are. Otherwise you will become reliant on that paltry state pension - or you'll have to keep working all the way into your 70s like some people.

For what I was doing, and how I was doing it, working with graph structures directly with Cypher would have been easier. Perhaps "logical" is the wrong word; my intent was to relay my specific experience, not express a general "principle".

I investigated using Neo4j back in 2010 for storing the schedules/routes of ocean shipping freight companies, forwarders, etc. The first cut of the system stored the schedule data in a graph like way in MySQL, but to do recursive SQL queries (needed for the types of queries being performed) was annoying. Things worked well enough, but a graph database would have resulted in a much more logical representation of the data structures being used - and the code behind the queries easier to develop.

That said, the 2nd system never got off the ground, I quit the badly run startup before finishing it. And now that I have a bit more experience with Neo4j, I'd say it would have been a bear to fully implement. Java is too heavy and Neo4j is a memory pig. It works, I can't say it is bad or perhaps iffy like Tinkerpop, but it is "Enterprise Software" and everything that is associated with that meme.

I have been using Tigergraph for my latest research into modeling the schedules etc of rail transport. It is much faster than Neo4j, requires far less memory (I can store every bit of data I need to in it unlike with Neo4 (would need multiple 64GB of RAM servers)), and its programming language is pretty nice once you get the hang of it.

So I'd recommend Tigergraph. The downside is that it is not as 'plug and play' as Neo4j, does not has all the mindshare/fancy bells and whistles, and is entirely C++/Unix based. So having some UNIX sysadmin experience is helpful unless you want to use their cloud solution.

"Drug Warriors and Their Prey" takes a look at the WoSD from another angle [0]

Basically, the WoSD 'was' (still is?) a mechanism by which authoritarians could impose control on society by oppressing society's undesirables. Echos of the Nazis, engender a police state, control Black people, etc etc.

But the WoSD hasn't gone so well, so it (as the main phenomena) was been supplanted by the War On Terrorism (cue the Patriot Act...) That has gotten a bit stale (but is still in effect?), so now it's ... Russians? Covid-19 and all the relevant restrictions?

Or the GOP has decided to stop being indirect and now is going blatantly full out towards overt authoritarianism using The Big Lie as the mechanism? In some parallel universe, liberals are now being rounded up and put into FEMA re-education camps on President-For-Life Trump's orders because they tried to steal the election (and every future one as well).

There is no one Single True Motivation to this sort of social phenomena - they are all different expressions of the sociopaths in charge's drives for power.

[0] https://www.amazon.co.uk/Drug-Warriors-Their-Prey-Police/dp/...

You're right, I got a bit strident there at the end... unresolved emotions I guess. Perhaps she isn't playing games, I suppose I am reading into the situation, but she just is overwhelmed and can't get the software or system finished fast enough. If it is obvious at this point that nothing works (you were not clear on that), then perhaps try to have a non-confrontational meeting with her with a mediator in charge.

The goal would be to get her to understand it's obvious things are not working and she needs to start co-operating, otherwise everything will then get problematic - and that no one wants this. A totally autistic engineering we-need-to-solve-the-problem-and-get-things-on-track meeting.

"appeasement"? You can't - it is obvious (based on my experience with people like this) she is deliberately stalling because it isn't finished, the demo was a canned one, etc.

One guy we hired (let's call him Bob) (I was contractor / lead engineer for a startup) kept saying for months he was working on the frontend (I was on the backend and so had lots to do, couldn't full time manage Bob because there was only me and him) He had a full time job as well, so the client and I gave him a pass for not meeting deadlines, but it got to be too much and the client asked to see code. Bob sent some bits and pieces, I could tell (sort of) what he was trying to do, but it was hardly enough progress. Long story short, after another couple months of this, I eventually flipped out (entirely my fault, I'd been too stressed out), called him names and demanded code.

The client had a talk with Bob, a hard deadline was set... then a week later, I noticed a weird IP address in the SVC logs. It turned out it was from Russia and the client was of course alarmed. It was using this guy's account -- Bob had hired a 3rd party contractor to try and do the work - without telling the rest of us. The founders were naturally very angry and so I left them to deal with Bob....

In the interview, Bob did very well to the point of he could sling the buzzwords around about Javascript like no one's business aka better than me. But good at marketing yourself doesn't equal competent coder! I should have given him FizzBuzz. Later, the VP of the startup told me Bob had at the very beginning tried to float the idea he'd manage a team of subcontractors. The VP did not really like that idea, given Bob was obviously trying to get into management pronto / set up his own company, was only 23 and had no real experience with such a thing - it might have worked.... but was more complex / expensive than the founders planned for. And me having to deal with a team of people when Bob said from the outset he could do it all...

I ended up doing the frontend from scratch in 2 months.

The second version of this story, I got called in by another former client to redo from scratch the work the guy who'd replaced me (Bob) hadn't finished. It turned out he'd completely thrown out the design of the J2EE app I'd built and started from scratch - doing exactly everything like the first app (which I'd replaced) which crashed 4 or 5 times a day. <That was a C++ app jammed into a J2EE container, so explicit threading etc etc>

This was for a gambling website, so any crash at all meant lots of lost money. :( Edit: my version was clocked at 4.5 nines; the client was very happy with me. But they unfortunately did not listen to their gut about Bob and gave him too much slack...

Bob was hired to MERELY PORT from Weblogic to JBoss. It would have been the simplest job ever, but .... he didn't know what he was doing and my client said he exhibited all the behaviours this "Ada" is doing

Like another commenter says, get onto AWS, see what is there, hope you can salvage something, and fire this person immediately. My gut says a product release is probably not going to happen on time because a large amount of the system needs to be built. So do plan for that. If it is actually finished, then... why is she trying to lose her job? Why is she afraid of losing her job? You say "employee" - if so, then Max has a lot of legal rights in this situation (IANAL).

If she is holding the AWS account hostage, then lawyers are the first resort, not the last. Appeasing her means giving her power, unless you can come up with some scheme to outfox her. I'm sorry, I guess I will be downvoted for being evil, but you don't mess around with people like this.

I have pretty much retired (i.e. suck at interviews and can't deal with the nonsense in industry anymore and am not one for management) and am thinking of going to get a PhD just because these days, I like doing research for the hell of it. Someday I will finish writing my first paper.... If I ever get a full time job again, it will have to be some sort of researchy sort of thing, so a PhD will be useful to get past HR etc.

Your PhD program is what you make of it. If you are not interested in going into full time academia, the uni will still take your money if you look like a good candidate and can get through the program. I suppose you have to worry about competing with other students if you both are trying to get the one open position with a professor... So make sure you can pay your own way and then you won't be dependent on being a wage slave for the university and all that industry-lite crap.

But really, ask yourself over and over "why do I want a PhD?" until you're sure of the answer - it is 3+ years of your life doing only that and it could be brutal due to the workload etc.

Engineering/CS programs are probably more laid back or less 'political' (i.e. those scary stories) than humanities or other STEM degrees. I have noticed some interpersonal drama in some sense just hanging around the uni these last few years, e.g. students and dealing with them, a general sense of the academic environment. But I make sure to stay out of it.

Or something deep in her psychology, e.g. she was raised by a pedophile and surely there were some dysfunctional family dynamics... That will color your entire life.

Oh dear... I assume you're being sincere, so no shade or anything like that. I just realized exactly how old I am relative to you and the other person who posted the same sentiment. ;)

Both. She was a big big name of my parent's generation (think Rhianna version 2, Rhianna is version 3 or 4 or 5... if you go back to Maria Callas or Josephine Baker being version 1)

It's the same defense that got Tucker Carlson and FOX News off, so it logically should work.

Were they "real" M.Sc degrees or "conversion course" M.Sc degrees (the equivalent of an expensive boot camp with a official piece of paper at the end)? I'm part time staff at a UK university which offers a 'conversion course' degree. It is vaguely annoying (from my vantage point of 25 years in industry) that these students are getting M.Sc degrees - the good students learn the basics of programming, so I'm not throwing shade on them, but I would hire someone with a B.Sc over them if I had to choose. Those with a B.Sc are more knowledgeable due to having 3 years experience in IT stuff versus just one.

But they are getting M.Sc degrees because part of the course involves learning how to "do research", i.e. the process. That apparently is a general requirement for students getting an accredited M.Sc degree. But it doesn't have to be original research... or even good research. Honestly, the whole program is sort of a diploma mill and given that students typically go into the program in order to retool and get a job in IT, they would be better served learning more focused stuff like one does in a boot camp than "how to do research". Or maybe a not-MSc degree, but something that indicates they're good enough to be junior programmers but are not really MSc level. The uni is trying to play the "boot camp" game while adhering to "academic standards".

I OTOH went thru a "real" MSc program from a much better uni (Edinburgh) and you absolutely had to have the basics of IT under your belt before taking that program.