HN user

kinow

523 karma

From Casa Verde, Sao Paulo, Brazil. After 8 years in Auckland, New Zealand, now in Barcelona, Spain. Working with workflow managers at the BSC. Past work includes government, research, programming, GIS/fuzzy logic/web development/NLP/ontologies/etc., web development, UIUX, illustrations and design.

https://twitter.com/kinow https://github.com/kinow/ https://kinoshita.eti.br/ https://www.instagram.com/brunokinoshita/ https://www.deviantart.com/kinow

Posts11
Comments318
View on HN

I work in a project with a similar goal, I think, but that was not related to AI: climate digital twins. I worked on two, EDITO (European Digital Twin of the Ocean, edito.eu), and currently on DestinE ClimateDT (Destination Earth Climate Adaptation Digital Twin, https://destination-earth.eu/).

These digital twins aim to simulate climate, oceans, atmosphere, wind speed, temperature, etc., regionally or globally. They apply physics, mathematics, software engineering, and high-performance computing to simulate the world. Both digital twins I mentioned use EuroHPC resources like CSC LUMI, BSC MareNostrum5, CINECA Leonardo, and soon others (and we also use other smaller HPCs).

Now there are some ongoing work about integrating AI/ML, LLM, etc.. I find it a better approach than building for LLM/AI-first, as it integrates existing components, and allows for AI to be used too. I think this could possibly be integrated as a component in a solution like this described in the article, as a component.

(There are other digital twins, for heart, human body, cities, urban climate, rivers, biodiversity, etc.)

I just finished a master's on HPC where I had to take some classes on CUDA, MPI+CUDA, OpenCL. Reading an article like this before the classes would have been a lot helpful! Especially the part just before and after "What does it mean for a warp to be eligible?".

I'm finishing a masters here in Spain about HPC (uploaded the thesis last night). It's a joint program by two universities from Galicia (USC/UDC). I paid about the same the author did. The programme is great, loved the professors --- I contacted them after the class and the interactions were helpful/interesting. Like in the author's case, my master had a lot of practical work (which is good, being in computing/HPC). But the workload is *really crazy*. Assignment pretty much every week, with overlapping deadlines.

There's something here called "second chance", which I don't remember seeing this in Brazil. My wife did a masters at University of Barcelona and they also had a second chance. In every subject if you fail, then you take the exam or submit some assignments to get the required grade. Some students realised they wouldn't get a good grade, and preferred to skip the exam or assignment, just to get a better grade on the second chance to graduate with a good score/honours/etc.. I found that interesting, but I focused more learning things that would help me at $work and tried to avoid stress (I was probably the oldest student in my class, 40+ yo).

My company is involved in research and production of RISC-V-based chips in Barcelona, with partners. There is or was a partnership with Intel at some point, and I think NVIDIA collaborated in some tasks in one of the projects. But the idea that I heard from the presentations is to produce chips in EU with little dependency on US, China, etc.

https://catalonia.com/w/barcelona-supercomputing-center-laun...

https://www.bsc.es/join-us/excellence-career-opportunities/d...

I think there is a partition in our supercomputing facility for these new types of technologies, but since my work is running climate models, I only hear news from other teams like our AI factory, the quantum computer, or people involved with these new chips and some emulators (that I think work together).

Nice post! Short and simple to read. Is epublish on GitHub or somewhere? It looks simpler than previous approaches I've seen.

I had the chance to help a bit with one of the AOSA books. It was a very powerful pipeline, but also quite complex to manage. IIRC it used pandoc (and a lot other tools).

So having a simpler alternative like epublish would be interesting if I have to work on another book in the future.

EDIT: sorry, just went to the main thread, and saw there you replied to another user it's not open source "yet" (hooray)

That idea is different than what most are talking here in other comments.

The grammar and vocabularies don't match, but I think the worst are the expressions. Both sides have *a lot* of expressions that vary per context and location.

But about time the Internet Archive had a US-independent backup.

Agreed!

The Internet Archive Switzerland, online at https://internetarchive.ch/, is a newly-formed Swiss non-profit foundation that will operate independently within its national context.

I think the Wikipedia Editors will have to decide whether they will add it to the existing page. The Operations section is still listing only U.S. data centers: https://en.wikipedia.org/wiki/Internet_Archive#Operations

I had a class in my masters about data centers (HPC Infrastructures). The professor was using some data centers somewhere in the middle of USA, in an area with hot weather as example. He compared that with ideal scenario (weather, power source, etc.).

In one of the slides, there were factors that influence the decision of where to build a data center, and several of the items involved finding a place with enough space and skilled people to work at this data center. He also commented sometimes there is politics involved on choosing the place for a next data center.

Just got an email from one HPC I have access in Germany. I guess all HPCs ans services like GH Actions are going to be offline for a bit. I think last time was on a Friday too, so it might be another Friday to organize emails, files, rotate backup/passwords...

I'm in my 40s too. I haven't switched roles in ~4 years, and probably won't for the next ~4, 5 years. But I have always painted and drawn as hobby. Seriously considering trying that for a short time during a sabbatical or -- more likely -- find a job as contractor from Mon to Wed, and spend the rest of the time drawing and painting.

If you are concerned about employability then I think going back to school or investing in a masters or some technical courses could be interesting. Or even moving to coordination/leader/engineer roles?

But if you have a hobby, maybe you could consider trying something different like either doing it in parallel, or maybe combining with engineering. e.g. I'm considering something like Blender3D + drawing using Grease pencil. Blender can be programmed with Python too, and this way I'd combine two things that I like.

Congrats, the description sounds like a good mystery! It'd be interesting to read more about the tooling and process you used, even if you don't release everything open, maybe you could write/blog about it?

I was also looking if there was a Wikipedia page about Software Engineers/Programmers who were also fiction writers. I know Andy Weir from Martian was a programmer. I thought Neal Stephenson would have some background in programming, but looks like he never wrote software professionally.

I also didn't know much of the difference between the two, and I also used RSS for my Hugo site.

At the bottom of the article there's, under "See Also", a link to this page comparing RSS and Atom: https://www.intertwingly.net/wiki/pie/Rss20AndAtom10Compared...

It seems like the last update is from 2008, but the section on the differences has a few interesting items. I am not sure if it changed, but it says:

"The RSS 2.0 specification is copyrighted by Harvard University and is frozen. No significant changes can be made (although the specification is under a Creative Commons licence) and it is intended that future work be done under a different name; Atom is one example of such work."

The Wikipedia RSS page has also a small section comparing RSS and Atom: https://en.wikipedia.org/wiki/RSS#RSS_compared_with_Atom

"Technically, Atom has several advantages: less restrictive licensing, IANA-registered MIME type, XML namespace, URI support, RELAX NG support.[35]"

Yeah, I considered applying once, but saw others online saying their process was long and outdated. In my case, I applied anyway, but during the screen call I asked if I would have to use Ubuntu even if I didn't use, and also their new (at that time) Juju for all tasks, even if that wasn't the best tool for the job. The position was related to automation of services. They told me I would to use both Ubuntu and Juju, and I couldn't use other tools if those two worked, which I understand, but I thought being stuck using Juju probably wouldn't help my career after a few years.

Yeah, I checked an Intel GPU some years ago and I think it was scoring near 1000 or below in Blender's open data. Glad it's slowly improving, although I have to check if the price is also increasing or not, although I suspect it must still be cheaper than the other options.

I hadn't heard about Cloudflare WARP. Found this Reddit thread with questions/comments I also had, https://www.reddit.com/r/CloudFlare/comments/ldejnt/how_is_c..., and also what I think is CF's main website for WARP info, https://one.one.one.one/ (which I must confess I had never head even though I use 1.1.1.1).

I struggle with LaLiga's filter during matches, but I am more interested if it'll help with latency/speed. Have you noticed any different when using WARP vs. without it regarding Internet speed?

Thanks!

For those that use Blender, in their section about Blender:

We hope that, in the future, there will be real options other than NVIDIA for GPU-based rendering, as it is an area where competition is nearly non-existent.

And Checking opendata.blender.org, a NVIDIA GeForce RTX 4080 Laptop GPU scores 5301.8, while Intel Arc Pro B70 is still at 3824.64.

So there is still a bit more to go before Intel GPUs perform close to NVIDIA's.

I remember the announcement some years ago of the Auckland-Hamilton train. I think initially it had limited schedule, but from what I recall the usage was quite good after the launch (I think it was before or just after the pandemic). Good to know it's in operation now.

And also the vague idea of local rail service around Christchurch (interestingly, a private company bought the old DMUs from Auckland's local fleet after electrification and are just starting to run special trains for Rugby games).

Hadn't heard about this! Interesting, and good idea to have a service for the games.

So the only potentially viable intercity route is Auckland to Wellington.

This one would probably be quite busy. I had to fly Auckland-Wellington quite a bit as an engineer, and our managers & executives travel quite more (NIWA, a CRI that now I believe has merged with another one and changed its name).

Eventually I had to go to the capital to vote or for the embassy, or for a tech event. All these trips were always via airplane, but I'd be happy to get a fast train or a night train as in Europe.

If you are serious about the Christchurch to Invercargill, there is now a private company offering the occasional weekend trip: https://www.mainlander.co.nz/train-trips/the-mainlander-rail...

Learned another new thing, thank you! I plan to go to Invercargill when I visit again to see if I can see the Aurora Australis or maybe check out where they have the MetService radiosonde launch. From what I recall MetService used to launch one from Invercargill near the airport (or I could be confused with a weather station or another climate monitoring station they had there).

What about options for those living up north? When I was still in Auckland some of my co-workers were looking into moving further West/North (there were too many people moving to South Auckland/Raglan/Hamilton at that time). But I remember the transport options involved one or more buses, and a ferry in the case of a co-worker that bought a house in... Hobsonville I believe. But the ferry didn't run all days, and had a limited schedule compared to the one for Waiheke or Devenport. Has that improved?

I always thought it'd be nice if there was a short train line connecting Devenport to Cape Reinga, as all the times I went to Cape Reinga or to take someone to Russel I'd have to drive or find a private shuttle.

Thanks!

Didn't know about these historical facts, looks like timing really contributed to the current situation in New Zealand. When I was in Auckland some years ago, I remember NZ trying to bring some railway services back, before the pandemic.

I never got to travel on these, but I'm hoping to do that when I'm there again, probably next year. I see the website is still the same, so if anyone is going to NZ: https://www.greatjourneysnz.com/.

And to be fair, looks like you can more or less cross the country, as long as you don't plan to get all the way to Invercargill.

That is a good point but I think it doesn't apply everywhere.that has a similar shape. New Zealand has a similar shape but without railways interconnecting cities. You cannot cross the country, the islands, or even regions by train.

I think this could be a variable to contribute to a good coverage and infrastructure... but there are probably more factors involved.

Churn of the user base could be playing a role in this, but I think it may not be too significant. In Europe there are multiple universities with HPC masters, which provide new users/devs to HPC. I worked with HPCs in New Zealand, and now I am doing the same in Spain. We hire multiple people from other HPC centers in Germany/UK/Italy, and equally lose people to those sites.

I think the field is actually increasing with AI, digital twins, more industry projects (CFD, oil, models for fisheries, simulations for health diseases, etc.).

I think hpc devs need an extra set of skills that are not so common. Such as parallel file systems, batch schedulers, NUMA, infiniband, and probably some domain-specific knowledge for the apps they will develop. This knowledge is also probably a bit niche, like climate modelling, earthquake simulation, lidar data processing, and so it goes.

And even knowing OpenMP or MPI may not suffice if the site uses older versions or heterogeneous approaches with CUDA, FPGA, etc. Knowing the language and the shared/distributed mem libs help, but if your project needs a new senior dev than it may be a bit hard to find (although popularity of company/HPC, salary, and location also play a role).

Seeing the comments, I think one thing missing is the availability of water. The authors mention it briefly, saying it's not part of the report,

We focus primarily on temperature rather than precipitation or soil moisture, as water availability can be more readily controlled through human interventions, such as irrigation and paddy field construction.

Traditional rice cultivation normally involves flooding. So I am not sure how much one can rely on irrigation for rice. When I was younger, in Sao Paulo some Japanese immigrants were trying to raise Japanese rice species with flooding when I was young, but they gave up and moved to other crops as the weather in the southern area and Uruguay were better (that's what I was told growing up -- FWIW we had corn, sugarcane, orange until there was a huge pest in Barretos region, bamboo and lots of other fruits and veggies in the farm).

A couple of years ago my company (BSC in Spain) had an internal talk about impact of climate change in European vineyard. I don't remember the article they were talking about, but it was similar to this one: Climate change impacts and adaptations of wine production https://www.nature.com/articles/s43017-024-00521-5

From what I recall, what the researcher explained was that the change in water sources like rivers and lakes is already affecting wineries that are not able to grow certain crops due to the lack of water.

But due to the reduction in water levels, some wineries and types of grapes were not being able to be harvested in specific parts of Italy. The article above mentions something similar,

Existing producers can adapt to a certain level of warming by changing plant material (varieties and rootstocks), training systems and vineyard management. However, these adaptations might not be enough to maintain economically viable wine production in all areas.

I guess there might be some genetic modification, and other techniques that do not require flooding and use less water. But that will likely affect small/medium producers, as well as and communities that depend on rice cultivation in certain areas.

Even with genetically modified rice, it might not be viable to bring water, or move families to other areas. So higher temperatures and the reduction of area that is suitable for harvesting rice might make genetically modifying rice useful only to a few.

I guess larger producers may be able to afford workarounds but that may increase cost to end users. Which is already a lot higher in Brazil than 10 years ago for normal and for the Japanese rice.