stefanpie

Very cool to see you work! Early in my PhD I did some work with GNN accelerators on FPGAs (which I think later ended up in some form as a colab with some CERN or Fermilab folks) and have chatted a bit in the past with the FastML, HLS4ML, and HEP folks.

I have since pivoted a lot of my PhD work (still related the HLS and EDA). But I wonder what is the current main limitation/challenges of building these trigger systems in hardware today. For example, in my mind it seems like the EDA and tooling can be a big limitation such as reliance on commercial HLS tools which can be buggy, hard to use, and hard to debug. From experience, this makes it harder to build different optimized architectures in hardware or build co-design frameworks without having high HLS expertise or putting in a lot of extra engineering/tooling effort. Also tool runtimes make the design and debug cycle longer, especially if you are trying to DSE on post-implementation metrics since you bring in implementation tools as well.

But I might be way off here and the real challenges are with other aspects beyond the tools.

Prof. Cunxi Yu and his students at UMD is working on this exact topic and published a paper on agents for improving SAT solvers [1].

I believe they are extending this idea to EDA / chip design tools and algorithms which are also computationally challenging to solve. They have an accepted paper on this for logic synthesis which will come out soon.

[1] "Autonomous Code Evolution Meets NP-Completeness", https://arxiv.org/abs/2509.07367

I am not the original author, but I posted this since it mirrors some experiences I have had in my PhD so far submitting papers. This kind of tweaking in paper and writing even happens when writing the first draft or sometimes even in the conception of the research idea or how to go about the implementation and experimentation.

There is a half-joke in our lab that the more times a paper is rejected, the bigger or more praised it will be once it's accepted. This simply alludes to the fact that many times reviewers can be bothered with seeing value in certain ideas or topics in a field unless it is "novel" or the paper is written in a way that is geared towards them, rather than being relegated to "just engineering effort" (this is my biased experience). However, tailoring and submitting certain ideas/papers to venues that value the specific work is the best way I have found to work around this (but even then it takes some time to really understand which conferences value which style of work, even if it appears they value it).

I do think there is some saving grace in the section the author writes about "The Science Thing Was Improved," implying that these changes in the paper make the paper better and easier to read. I do agree very much with this; many times, people have bad figures, poor tables or charts, bad captions, etc., that make things harder to understand or outright misleading. But I only agree with the author to a certain extent. Rather, I think that there should also be changes made on the other side, the side of the reviewer or venue, to provide high-quality reviews and assessments of papers. But I think this is a bit outside the scope of what the author talks about in their post.

To offer a non-AI alternative but a similar game concept, I highly recommend TimeGuesser: https://timeguessr.com/.

You are shown historical photos (from the past 100 years, even up to the past year) and need to guess the location on a map and the year of the image. Extremely fun and varied gameplay because of the varied events the photos capture (some mundane while others more recognizable).

While watching, I started playing a fun game where I try to guess the location of the video, GeoGuessr style. Very interesting when it comes to the odd handheld angles and low quality of some of the video clips. Would recommend.

Am I lost or are the cores not open source? I cannot find any Verilog, VHDL, or bundled IP blocks to downloaded. Very strange for what on the surface appears to be a hobby FPGA project.

This heavily overlaps with my current research focus for my Ph.D., so I wanted to provide some additional perspective to the article. I have worked with Vitis HLS and other HLS tools in the past to build deep learning hardware accelerators. Currently, I am exploring deep learning for design automation and using large language models (LLMs) for hardware design, including leveraging LLMs to write HLS code. I can also offer some insight from the academic perspective.

First, I agree that the bar for HLS tools is relatively low, and they are not as good as they could be. Admittedly, there has been significant progress in the academic community to develop open-source HLS tools and integrations with existing tools like Vitis HLS to improve the HLS development workflow. Unfortunately, substantial changes are largely in the hands of companies like Xilinx, Intel, Siemens, Microchip, MathWorks (yes, even Matlab has an HLS tool), and others that produce the "big-name" HLS tools. That said, academia has not given up, and there is considerable ongoing HLS tooling research with collaborations between academia and industry. I hope that one day, some lab will say "enough is enough" and create a open-source, modular HLS compiler in Rust that is easy to extend and contribute to—but that is my personal pipe dream. However, projects like BambuHLS, Dynamatic, MLIR+CIRCT, and XLS (if Google would release more of their hardware design research and tooling) give me some hope.

When it comes to actually using HLS to build hardware designs, I usually suggest it as a first-pass solution to quickly prototype designs for accelerating domain-specific applications. It provides a prototype that is often much faster or more power-efficient than a CPU or GPU solution, which you can implement on an FPGA as proof that a new architectural change has an advantage in a given domain (genomics, high-energy physics, etc.). In this context, it is a great tool for academic researchers. I agree that companies producing cutting-edge chips are probably not using HLS for the majority of their designs. Still, HLS has its niche in FPGA and ASIC design (with Siemens's Catapult being a popular option for ASIC flows). However, the gap between an initial, naive HLS design implementation and one refined by someone with expert HLS knowledge is enormous. This gap is why many of us in academia view the claim that "HLS allows software developers to do hardware development" as somewhat moot (albeit still debatable—there is ongoing work on new DSLs and abstractions for HLS tooling which are quite slick and promising). Because of this gap, unless you have team members or grad students familiar with optimizing and rewriting designs to fully exploit HLS benefits while avoiding the tools' quirks and bugs, you won't see substantial performance gains. Al that to say, I don't think it is fair to comply write off HLS as a lost cause or not sucesfull.

Regarding LLMs for Verilog generation and verification, there's an important point missing from the article that I've been considering since around 2020 when the LLM-for-chip-design trend began. A significant divide exists between the capabilities of commercial companies and academia/individuals in leveraging LLMs for hardware design. For example, Nvidia released ChipNeMo, an LLM trained on their internal data, including HDL, tool scripts, and issue/project/QA tracking. This gives Nvidia a considerable advantage over smaller models trained in academia, which have much more limited data in terms of quantity, quality, and diversity. It's frustrating to see companies like Nvidia presenting their LLM research at academic conferences without contributing back meaningful technology or data to the community. While I understand they can't share customer data and must protect their business interests, these closed research efforts and closed collaborations they have with academic groups hinder broader progress and open research. This trend isn't unique to Nvidia; other companies follow similar practices.

On a more optimistic note, there are now strong efforts within the academic community to tackle these problems independently. These efforts include creating high-quality, diverse hardware design datasets for various LLM tasks and training models to perform better on a wider range of HLS-related tasks. As mentioned in the article, there is also exciting work connecting LLMs with the tools themselves, such as using tool feedback to correct design errors and moving towards even more complex and innovative workflows. These include in-the-loop verification, hierarchical generation, and ML-based performance estimation to enable rapid iteration on designs and debugging with a human in the loop. This is one area I'm actively working on, both at the HDL and HLS levels, so I admit my bias toward this direction.

For more references on the latest research in this area, check out the proceedings from the LLM-Aided Design Workshop (now evolving into a conference, ICLAD: https://iclad.ai/), as well as the MLCAD conference (https://mlcad.org/symposium/2024/). Established EDA conferences like DAC and ICCAD have also included sessions and tracks on these topics in recent years. All of this falls within the broader scope of generative AI, which remains a smaller subset of the larger ML4EDA and deep learning for chip design community. However, LLM-aided design research is beginning to break out into its own distinct field, covering a wider range of topics such as LLM-aided design for manufacturing, quantum computing, and biology—areas that the ICLAD conference aims to expand on in future years.

The author has a really fun talk about the experience as well: https://www.youtube.com/watch?v=IWT6Y0YDJUo

The main author of KANs did a tutorial session yesterday at MLCAD, an academic conference focused on the intersection of hardware / semiconductor design and ML / deep learning. It was super fascinating and seems really good for what they advertise it for, gaining insight and interpret for physical systems (symbolic expressions, conserved quantities , symmetries). For science and mathematics this can be useful but for engineering this might not be the main priority of an ML / deep learning (to some extent).

There are still unknowns for leaning hard tasks and learning capacity over harder problems. Even choices in for things like the chosen basis function used for the KAN “activations” and what other architectures these layers can be plugged into with some gain is still unexplored. I think as people mess around with KANs we’ll get better answers to these questions.

Very cool to learn where my Linux file manager GUI app came from.

One point that is light on details in the wiki was what exactly their monetization plan was or if they even had one at all? It sounds like they wanted to integrate business/enterprise features and support, or perhaps features that plug into other internet services that they could possibly monetize. It sounds like the "network user experience" with "Eazel Online Storage" and "Software Catalog" were their initial monetization ideas.

As much as I prefer to do everything in a text editor and use open-source EDA tools/linters/language servers, Xilinx's Vivado deserves major credit from me for its block editor, schematic view, and implementation view.

For complex tasks like connecting AXI, SoC, memory, and custom IP components together, things like bussed wires and ports, as well as GUI configurators, make the process of getting something up and running on a real FPGA board much easier and quicker than if I had to do it all manually (of course, after I can dump the Tcl trace and move all that automation into reproducible source scripts).

I believe the biggest advantage of the Vivado block editor is the "Run Block Automation" flow that can quickly handle a lot of the wire connections and instantiation of required IPs when integrating an SoC block with modules. I think it would be interesting to explore if this idea could be successfully translated to other styles of visual programming. For example, I could place and connect a few core components and let the tooling handle the rest for me.

Also, a free idea (or I don't know if it's out there yet): an open-source HDL/FPGA editor or editor extension with something like the Vivado block editor that works with all the open source EDA tools with all the same bells and whistles, including an IP library, programmable IP GUI configurators, bussed ports and connections, and block automation. You could even integrate different HDL front-ends as there are many more now than in the past. I know Icestudio is a thing, but that seems designed for educational use, which is also cool to see! I think a VSCode webview-based extension could be one easier way to prototype this.

The short answer to your first question (in my opinion) is no, ASICs and FPGAs are not really equivalent (generally) in the context of how they are used in different applications. ASICs are baked-in circuits; once you make them, they are fixed. You can make the circuits and digital logic in them dynamic at runtime (for example, a CPU, or a more complex example, a domain-specific runtime accelerator like Sohu). An FPGA lets you reprogram the circuit inside it (via one level of extra abstraction, the logic and routing configuration inside it). In essence, an FPGA makes the ultimate tradeoff of being fully reconfigurable down to the "gate" level at the cost of other things like clock speed, area, transistor size, power, and so on. These tradeoffs may be so significant that it looks like GPUs and ASICs are worth it for deep learning inference rather than FPGA.

In general, you can implement deep learning accelerators on both FPGA and ASIC. Xilinx and now AMD have been slowly adding more and more stuff to their "FPGA chips," like an AI engine (vector processors and a network-on-chip), as well as high bandwidth memory, in addition to their configurable logic already there, to make it more viable as a possible solution for companies to deploy deep learning stuff to their devices or in applications where they want to integrate deep learning alongside regular FPGA processing stuff. I don't know how that shakes out in the industry, but I do know that lots of academics use FPGAs as a good platform for experimenting and prototyping accelerator architectures and such.

Note that you can find the full version PDF from Lynn Conway's website:

https://ai.eecs.umich.edu/people/conway/VLSI/VLSIText/PP-V3/...

https://ai.eecs.umich.edu/people/conway/VLSI/VLSIText/VLSITe...

Hey thanks for the reply! I'm assuming your the first author on the paper; if so, the signup button on the Normal Computing website is not working at the moment (at least for me, even with ad blocker turned off).

I didn't realize they included details about the hardware. Lie you said these just look like analog computers, compute in memory, analog arrays, which have also made a resurgence with deep leaning.

I know they mainly present results on deep learning/neural network training and optimization, but I wonder how easy it would be to use the same optimization framework for other classes of hard or large optimization problems. I was also curious about this when I saw posts about Extropic (https://www.extropic.ai/) stuff for the first time.

I tried looking into any public info on their website about APIs or software stack to see what's possible beyond NN stuff to model other optimization problems. It looks like that's not shared publicly yet.

There are certainly many NP-hard and large combinatorial or analytical optimization problems still out there that are worth being able to tackle with new technology. Personally, I care about problems in EDA and semiconductor design. Adiabatic quantum computing was one technology with the promise of solving optimization problems (and quantum computing is still playing out with only small-scale solutions at the moment). Hoping that these new "thermodynamic computing" startups also might provide some cool technology to explore these problems with.

Cool nugget from the article: "the software that Studio Ghibli used to make Spirited Away has been free for years"

https://opentoonz.github.io/

Funnily enough, Matlab offers a high-level synthesis (HLS) tool to lower Matlab code and Simulink models to Verilog [1]. I have not personally used it, but the last I heard, it works well enough for those who need it and Matlab puts in the work to support it. However, HLS is another whole can of worms, and it can be confusing when discussing new HDLs and the distinction between an HLS tools and high-level HDL dialects.

[1] https://www.mathworks.com/discovery/high-level-synthesis.htm...

This brings flashbacks of standing next to part bins in circuit labs with a multimeter in hand, trying to match components. Unfortunately, the TAs and course instructors did not emphasize your point, and the measured part values and thus the measured filter cutoffs had to be close to perfect to be checked off.

In the past, I have looked at using optimizers to solve for component values of complex analog circuits. I only looked at optimizing accuracy at one corner, but it would be interesting to see what people have done to optimize multiple variables, like including noise, for multiple corners. I think I've seen some Monte Carlo simulation mentioned once to suggest fuzzy solutions within some specification. I would be curious to see if others know more about this.

Hypergraphs are awesome, and I wish they got more attention. They actually make intuitive sense in contexts like circuits and other hardware design problems. However, the last time I checked, all the Python and Rust Hypergraph libraries did not have support for directed edges (or it wasn't documented very well).

The "tersest route" problem seems to be interesting as I have never come across this before. It seems almost like a packing or covering problem.

One big related point not discussed in the actual software used for semiconductor design at all levels (note: I'm biased toward this point since I work in this area on a day-to-day basis)(also this is most related to computing devices rather than things like solar panels and LEDs).

Unlike the software world (which has great open-source foundations for a lot of the core tools used, such as compilers and operating systems), many of the software tools used in electronics design automation (EDA) and semiconductor fabrication are developed by industry giants at high costs (edit: high costs to the users), with little consideration given to the quality and performance of the software itself (not all parts are bad, but typically most are).

Fortunately, academia and open source have been accelerating progress within the EDA stack with long-standing tools like Yosys, full-stack academic projects for tape out with good backing like OpenROAD, and workgroups like CHIPS Alliance.

However, I believe that there is still a gap between academic + open-source efforts and industry research. To really enchippify everything, the academic and open-source communities need more open and accessible tools for people to use, without having to rely solely on industry tools, funding, and higher motives.

I also want to note that some people are looking at using ML/DL to generate placement solutions, even though DREAMPlace is still a key recent work in this area that at the core uses analytical placement, although cleverly using deep learning libraries and features.

For example, the VTR people are looking at RL to pick what actions to take during FPGA placement for their simulated annealing placer (see: https://ieeexplore.ieee.org/document/9415585).

I also know this paper/poster was indexed recently under OpenRevew but I'm still waiting for some more implementation details to look at: https://openreview.net/forum?id=6GR8KqWCWf

FPGA routing I don't think anyone has touched on using ML/DL but I do know that there is some talk about using ML/DL models with current routing approaches to replace search heuristics (think like replacing or augmenting something like A*) or do routability predictions. Certainly, there are probably many ways to use RL in routing as there are many places in current algorithms to intelligently make certain heuristic decisions.

Edit: I also want to note that there are a ton of works that also use ML/DL to tune the "hyperparameters" of EDA tools such as placers and routers. Think ML/DL for back-box non-linear optimization.

I believe that nextpnr has integrated both a simulated annealing placer and an analytical placer (based on this publication from 2019: https://arxiv.org/abs/1903.10407) (see here for the work the analytical placer is based on: https://ieeexplore.ieee.org/abstract/document/6339278). I think they are also working on electrostatic placement but I'm not sure what the current status of that is.

Based on the other comments, they are correct in that when doing annealing you usually want to accept some moves that do lead to regression that might not improve the regression to escape early local minimums in your objective.

I abused the definition of annealing a lot in the post but I briefly touched on the idea:

"At first, you might want to make moves or swaps over large distances or you might want to accept some percent of moves that don't improve the objective, but as time goes on, you want to make smaller moves and be less likely to select moves that don't improve the objective. This is the "annealing" part of simulated annealing in the context of FPGA placement."

I think I might have made the writing confusing because I mixed the original definition of the annealing approach (of accepting moves that don't improve the objective) with the use of "annealing" for other things like action parameters (ex. swap distance between two nodes). Something I should edit to clarify better.

Note that, yes, the thing I implemented doesn't do any annealing but rather just pick actions that only improve the objective. I am working on some extensions to add real annealing but that turned out to have a lot of more in-depth technical work that is not obvious.

Should be fixed now, thank you for the note!

This personal account resonated with me since I also had some personal experiences in both the "math competition" and "competitive high school" world up until I graduated high school in ~2018.

---

I went to a middle school where I somehow ended up far ahead in the curriculum due to my own interest in science and math, thanks to my personal interests and parental support at the time. I think I caught the attention of my math teachers and got to participate in a county-wide middle school math competition with some other students at my school. It was cool, and I wasn't, and still am not, a super competitive person, so I thought it was a fun experience to try; I don't think we won anything significant. I also think I took an AIME-style competition exam (sit-down test), but I don't remember much about it. All of this was cool but not super interesting to me at the time.

Some wise teacher or fellow parent at the time must have tipped off my parents about Math Circles, and I am incredibly grateful that my parents took me there. Math circles (at least in my experience) are like local meetups for younger (usually non-college) students to go and learn about cool exploratory and advanced (non-standard curriculum) math topics from college students and professors. I used to be so excited to go on Saturdays to math circles and sit and watch an old professor from a local college teach topics like logic, number theory, advanced geometry, and other math topics that were definitely not taught in my middle school. I also distinctly remember that I loved the exploratory side of the math circles as opposed to competitive math where I felt like I wouldn't learn anything and I had to get everything correct. If you have kids who are curious about math or even science, nature, and art, I would highly look into whether there are Math Circles in your local area and at least try it out. Some of my fondest memories during this time are from Math Circles.

I somehow ended up getting a full scholarship to attend a local (non-religious) private school with one of the top competitive high school math teams in the nation (among other extracurriculars like debate, model UN, ...). The summer school started there, and I won't forget when I went to the information sessions for the math competition team. The "vibe" was very off as soon as I walked in and not what I was expecting. The competitiveness was very evident in your attendance and performance in the multiple-times-a-week after-school math competition classes, how the best get to be on the A and B teams, and so on. I think I walked into it with the preconceived notion that I was going to get to learn more math for fun by joining the math team (like I did in math circles), but my naive worldview was very much shattered. I think I attended one of the local math competition events that first semester of high school and took one of the sit-down style tests, but there was no way I scored any kind of decent amount to be considered for anything related to math competition at that school. I also had one good close friend who stuck with math competition for the four years of high school, and so I got to get an idea of what that was like, and it was just as crazy as I thought it would be the whole time. I am actually grateful for him since I also got to talk to that friend about a lot of cool advanced math topics before I took those math classes, as well as help me learn enough of the math ahead of time to take the advanced AP physics classes (the electricity and magnetism one) that I had genuine interest in.

Luckily, by my second year, I was able to land on "science research" (think liek a really competitive science fair, e.g., ISEF) as a nice extracurricular I enjoyed. Even in that, you cannot escape the competitive nature of it, but there are alternative ways to branch out on your own (real publication, working with local colleges and professors, just being able to say I completed this cool research project and made this cool thing) and put something on your "college resume" without having to pigeonhole yourself into competing against other students all the time. I did have to cycle through some other extracurriculars (model UN and FRC robotics) where I "crashed and burned" before I finally landed on one where I would feel "safe" in a sense to enjoy what I was doing with research.

Finally, the big push from the school onto the students to get into the most prestigious schools they can get into really led to some weird dynamics among students and really unhealthy kinds of stress and anxiety to cope with. I think (but memories are fuzzy) that I was able to dodge a lot of that just by the nature of not being super competitive, but there were many students who would lose their minds if they didn't get above a 90% on an exam in an AP level class, as in genuinely freak out and stress over it and then go to the teacher and try to sort it out. Any grade that would jeopardize their perception of being smart or getting the highest GPA possible to get into the best schools possible was an anxiety and stress-inducing nightmare for them. There was very much an idea that "I'm not going to tell everyone else what I got on an exam" to also hide class rankings and your own GPA. I'm also sure that none of us got any healthy amount of sleep needed; there were people pushing insane courseloads with all AP classes to pad their "college resumes." I remember taking some architecture classes instead, which I found much more fulfilling and fun, and then feeling out of place trying to talk to some of my other classmates at the time who were trying to min-max their GPA and college applications.

---

Not to ramble on for too long, I just want to give an idea to some readers of another related personal perspective on this topic. Having been in some related environments in middle school and high school, I can really empathize with some of the genuine, agonizing, and sometimes lighthearted in hindsight, moments the author experienced. I can't imagine what it's like in 2023/2024, or even having kids in the future, what high school education will be like. Can students really find peace and comfort to just learn and pursue what interests them the most?

Quick epilogue: I did apply to all the prestigious schools that everyone else applied to (bless my parents for covering all the unnecessary application fees), but also thanks to my parents' pragmatism, I applied to some strong schools for engineering. I ended up down the Ph.D. route, and the exploratory open-endedness aspect of research is still what keeps me excited and happy in my day-to-day work.

If you look under Casio's website under the category of "Standard Scientific Calculators" (https://www.casio.com/intl/scientific-calculators/standard-m...) they have some calculators where you can see the input and output at the same time and navigate the input with arrows. I had one that was like the "fx-350ES plus" (https://www.casio.com/intl/scientific-calculators/product.FX...) in high school but they have a selection of newer looking models that support the same kind of many-featured (fractions, exponents, integrals, ...) pixel graphic inputs rather than the simple 2 line LCD.

To add to this this, many of the aspects that make MRI's uncomfortable for kids and even adults is all the physical sensations. Even the claustrophobia can get to many adults if they have never been in one of these machines. Also aspects such as the having the transmit/recoveree coil harness on the patient are physicals sensations you have to experience (for example sensing coils for the head) and really cant be emulated. The sound itself is also very physical as it is somthing not just heard but also felt by the patients. The sound is caused by electromechanical (fast high current switching of the electromagnet coils of the gradient coils) interactions in the operation of the machine itself so you might also feel the sound via the machine or coming from many directions in the bore.

I remembers from my class that there are some published works that design MRI sequences ("pattern" in which you operate the MRI machine to switch/modulate the electromagnets of the gradient coils, send RF pulses, and record the received RF pulses) in such a way that they reduce or don't cause the majority of the noise from the switching of the gradient coil in a typical MRI sequence.

At Emory university I got to see a cool idea to help kids get comfortable with MRI machines. Some Georgia Tech students built a fake MRI machine (a realistic mechanical structure with a sliding bed) that kids could still get into. The cool thing was that it had Bose speakers built into the fake machine so that they could play the sounds of a real MRI machine makes as that is a big aspect that makes kids get anxiety and fear of the machine. It was all in a separate room right by the waiting room and the room with the real MRI. It was so cool to see such a practical solution to patient care especially for kids.

Edit: found some pictures online https://www.cores.emory.edu/csic/resources/imaging/mri_for_h...

This is my advise to people who ask me for advise when applying to PhD / graduate school for research. The advisor is probably the only other main factor besides finances and any other personal constraints. If you have a great advisor all around (not just the best researcher in their field, as in they also know how to mentor, teacher, network, and help you reach your goals), they will know how to navigate your different research interests and goals of your PhD and get you to where you want to go.