Predictive Debugging: A Game-Changing Look into the Future

https://blog.jetbrains.com/dotnet/2023/07/27/introducing-predictive-debugging-a-game-changing-look-into-the-future/

by redbell • 3 years ago

178 83 3 years ago

blog.jetbrains.com

invalidname• 3 years ago

That's pretty fantastic!

When will this be available in Idea?

thelazyone• 3 years ago

Sounds like a logical step forward from current debuggers! I had some colleagues who bragged of not needing debuggers for their workflow... Well, sometimes I really do.

sanitycheck• 3 years ago

It does look very nice, but as with all fancy debugger features I wonder if I'd spend more of my life debugging the debugger when it doesn't debug than it'd save me.

I also think that while a debugger is a great and useful tool, being able to debug without one is a requirement, because sometimes you don't have one (or it stopped working properly - see recent Android Studio releases) and if one can't do it just using logs one is really stuck. So it's worth getting practice at that.

paulcjh• 3 years ago

Activating type checking mode in VSCode was a game changer for me when running python, help catch loads of the edge cases for correctly for some production code

mdaniel• 3 years ago

If you think turning on typechecking was a game changer, I can't wait to see the look on your face while using the more intelligent editor PyCharm; the kinds of things in catches are asymptotically magic

jamesmurdza• 3 years ago

It looks like a combination of static analysis and live execution of what they call "pure" parts of the code, i.e. code that doesn't affect state outside the current function.

RunSet• 3 years ago

Friendly suggestion to peruse Jetbrain's third-party data-sharing agreement while considering what information of a user, even in theory, it might forbid them to share with anyone they wish for any reason they please.

https://www.jetbrains.com/legal/docs/privacy/third-parties/

After you do that here are some IDEs that (so far as I know) don't have data-sharing agreements with third parties.

https://downloads.codelite.org/

https://vscodium.com/

Bjartr2• 3 years ago

Nothing listed there is anything particularly problematic so far as I can see. Can you be more specific?

If you're saying that it's sufficiently vague as to not actually limit them in any way, then I'm wondering what a better privacy policy would look like. One that gives them proper leeway for the legitimate day-to-day workings of a company, but which manages to not be just as vague.

One thing worth remembering here is that the alternatives you mention are exclusively IDEs, whereas the JetBrains privacy policy covers the data sharing they do for their whole ecosystem of products and services, not just their IDEs.

WhereIsTheTruth• 3 years ago

Interesting that you didn't even mention their opensource community edition: https://github.com/JetBrains/intellij-community wich powers plenty of other open source IDEs, yet you push vscodium

Nice FUD ;)

krono3• 3 years ago

Which also seems to be covered by the same privacy policy. That FUD accusation was a bit premature.

For the record I dislike both vendors' products equally as much :)

https://github.com/JetBrains/intellij-community/blob/60f8208...

intelVISA• 3 years ago

Unfortunately JetBrains, like many, are unwilling to pass up on exploiting the good margins on data extraction - even at the expense of their products and userbase.

daxfohl• 3 years ago

Seems almost like a spreadsheet once you hit a breakpoint. Pretty cool!

hackinthebochs• 3 years ago

Nice to see something like this finally being made. I always wondered why I couldn't just annotate the value of variables and then have the IDE evaluate the code as I write it to show how those variables are mutated. Always seemed like low-hanging fruit. It would also have the benefit of being a learning tool to help develop a mental model of how coding works.

Bjartr2• 3 years ago

Reminds me of Light Table

https://chris-granger.com/2012/04/12/light-table-a-new-ide-c...

jebarker• 3 years ago

Also sounds like: https://whitebox.systems/

vaughan6• 3 years ago

Someone needs to design a programming language and an IDE (and possibly a new OS too) with great debugging as the primary goal. Debugger and IDE support is always just thrown on later in every new language these days.

"Omniscient debugging" as seen in https://pernos.co/ is the holy grail. Time-travel debugging would also be great.

People are far too obsessed with static type checking. A lot of this time would be better spent invested in live debugging tools.

When I'm editing my code I want to see exactly what _value_ each variable contains, the type really doesn't matter so much. Wallaby.js/Console.ninja is a great example of this.

Good debugging, especially deterministic record/replay is usually complicated by the OS. I often wonder what an OS would look like if designed with debugging as a top priority.

shardullavekar2• 3 years ago

We started off as a time travel debugger for Java coders and failed to gain any traction. (our hn launch: https://news.ycombinator.com/item?id=31286126)

Pivoted to record and replay. Developers can put assertion + mocking around the replay. We are working on a feature that shows what _value_ each variable contains. Check unlogged.io

Disclaimer : I am the founder

Bootstrapper9092• 3 years ago

Very cool! Any specific reason you started with Java?

wiz21c3• 3 years ago

pretty neat. But what I need the most right now is a way to visualize information. I work on computer fluid dynamics and there I have:

- super intricate algorithms with lots of variables running around

- lots of huge numpy arrays storing the bulk of the data

I find the current debuggers lacking here:

- showing numpy arrays in colours, easily changing color scales, etc. is not easy (you can do your own visualization, but then it's code); explore specific parts of the array in detail; link that to other arrays, etc.

- build watches panels where you can mix analysis and values extracted from code. I can't count the number of time I freeze the execution, export data to R, then do analysis there, then fix the code. It'd be nice to have that integrated in the IDE.

- have watches that record values as execution runs (I guess that's one of the stuff omniscient debugging does)

And while I'm at it:

- why can't I cut paste formulas, pictures, etc. in my code ?? (no Jupyter, you're ok for a nice paper like presentation but not for code with hundreds of thousands of lines)

heavenlyblue• 3 years ago

Good software development implies writing code that is easy to debug, including writing your own tools to debug your own niche-specific code!

pizza3• 3 years ago

What if Jupyter was ok for code with hundreds of thousands of lines? I suspect it might be.. I think nbdev ameliorates my major gripes with jupyter: it lets you easily maintain a bidirectional jupyter <-> (library as standard python files) mapping, while also giving you the excellent presentational flexibility of Jupyter. Plus a good looking docs site, for free. Since the code is exported as scripts, you can just `pip install -e .` and get all your IDE integration as normal.

corysama• 3 years ago

Charles Simonyi's Intentional Software was doing that back in the days of Windows NT https://www.youtube.com/watch?v=tSnnfUj1XCQ But, they stayed insular and enterprise right up until Microsoft bought them back.

We mortals barely got to see or hear anything about it. I heard one podcast interview of a developer there who said they basically represent the code as an s-expression tree with a bunch of metadata on each node.

unshavedyak2• 3 years ago

People are far too obsessed with static type checking. A lot of this time would be better spent invested in live debugging tools.

I mean.. the two are tightly related. The type represents the outer bounds of values possible. Types will give you as much utility as you put into them (or as the language allows).

I agree in general, i just think you massively undersell types haha. It's not either or, it's AND. Always AND.

steve_adams_86• 3 years ago

I agree. Even with excellent debugging, I think types will allow me to need to debug less. This seems ideal, because debugging means things are breaking.

Types also mean that I can describe my program in ways others can understand and trust. I don’t want to pass off potentially broken programs to other people and say don’t worry, the debugging experience is great.

ahzhou• 3 years ago

Types are just human and machine readable documentation - orthogonal to debugging IMO.

Great typing, like great documentation or great design, helps with debugging - but that’s not the main goal.

hinkley2• 3 years ago

If you're trying to simulate a combinatorial explosion of possible states, reigning in the possible values would certainly help with maintaining that illusion.

ellis0n• 3 years ago

Someone needs to design a programming language and an IDE (and possibly a new OS too) with great debugging as the primary goal. Debugger and IDE support is always just thrown on later in every new language these days.

Already designed. Called AnimationCPU platform and a new ACPU OS with ACPUL programming language and real-time time travel debugger. But there is not much marketing here, so you can only watch some demos:

DBG https://vimeo.com/363434798

OS https://vimeo.com/833395245

deutschepost• 3 years ago

You should have a look at Tomorrow Corps custom Toolchain[1]. It opened my eyes about how good things could be.

[1]https://tomorrowcorporation.com/posts/how-we-make-games-at-t...

cadr• 3 years ago

That was amazing.

sgu999• 3 years ago

Thanks for sharing, absolutely mad! I now feel like I'm 20 years behind with my current tooling...

junon• 3 years ago

I've been working on an OS for a few years now that addresses this pretty much directly. Not ready to announce anything but there are certainly a group of people doing OSdev along these lines, myself included.

ElFitz• 3 years ago

What about Smalltalk?

I don’t recall the specifics, but I remember someone mentioning it having some impressive debugging capabilities on hn.

- https://news.ycombinator.com/item?id=35100708

- https://news.ycombinator.com/item?id=36905215

- https://news.ycombinator.com/item?id=33725141

For those willing to sift through the comments:

https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

vaughan6• 3 years ago

SmallTalk was so visionary. Someone needs to do a modern take on it.

pizza3• 3 years ago

Yup. The OS contains the debugger process contains a copy of the debugger contains a copy of the runtime contains a copy of the program state contains a copy of the linked system libraries and such and, somewhere, in some tiny section of the executable behemoth, the code that I wrote to print hello world.

Always wondered why we usually stop at adding first-class features at the code. Or go “Oooh” if a language lets you go one step up the chain to eg metaprogramming methods.

The whole chain is where we are working; the whole chain should be the first-class citizen of the language/tool.

vaughan6• 3 years ago

The whole chain is where we are working; the whole chain should be the first-class citizen of the language/tool.

Sums it up well!

michaelteter• 3 years ago

People are far too obsessed with static type checking.

I thought this too. I'm a huge Ruby fan, and also Clojure and Elixir. In Clojure you have Spec (https://clojure.org/guides/spec) which is pretty powerful if you choose to use it. And in Elixir, you have pattern matching that can take you quite a long distance without need for specifying explicit types. ...

But lately I've been learning Swift, and now I'm in favor of explicit static typing. (You don't have to be explicit always with Swift, but it's not much trouble to be so... and it's necessary sometimes.) And then switching back to Ruby, I find myself wishing for that visible type clarity on function params. Am I dealing with an object and needing to reference properties? Is this a hash and I can access key/value pairs? A developer using some library may have to dig in and read library source to really know what's expected when types are not required or listed in the function signature.

What I would like to see is Elixir style pattern matching with static types. Then you can "duck type" in terms of the shape of the input as long as it conforms to some subset of data+type.

jahewson• 3 years ago

People are far too obsessed with static type checking. A lot of this time would be better spent invested in live debugging tools.

I disagree. Time spent debugging is time wasted. It's a non-repeatable sunk cost. Don't get me wrong, I love debugging - it's like solving a mystery in real-time - but the competitor to existing debuggers is not a better debugger, it's to obviate the need for one.

f1shy• 3 years ago

> People are far too obsessed with static type checking.

I have to fully agree with this. At least here in HN, I see all the time comments "language is good, but does not have static typing." As it was THE thing needed.

Of course it helps, but I think is greatly exaggerated.

I think many people code as some of my colleges did Physics in the university: they looked at the data available in the problem description, and searched formulas to plug the data, just looking at the units, where the output unit was the correct for the answer required. In a similar way I see people smashing things at APIs without reading the documentation, using bad naming, having tens of variables in 10 line functions, all because "no problem, if something is wrong, it won't compile!".

I have written tens of thousands of lines of code in dynamic typed languages, and I can count and remember the few times I had a problem related to types. Out of those, only 2 times I had a somewhat difficult time debugging (couple of hours).

avidphantasm• 3 years ago

Looks cool. I could do without the cutesy meme images.

unshavedyak2• 3 years ago

Has JetBrains ever discussed putting out some sort of generic LSP oriented product(s)?

I own a couple JetBrains product for very select things but as much as i enjoy the company i will never use them for my actual code.. because i'm a 20 year addict of Terminal based editors (currently on Helix).

I'd love to see the considerable effort they put towards tooling available to the Terminal oriented audiences. Yea, probably small audience.. but i just often see this stuff and think "What if this was an LSP? I'd purchase it in a heartbeat!"

Especially since multi-LSP editing is apparently viable. Eg Helix supports multiple LSPs for a single context, now. Though i'm not clear on the specifics yet, as i've not tried it yet.

I really want to give them more money. I just don't want to use their UIs for my code.

xpe4• 3 years ago

Hello, I am an LSP bot designed to help clarify human language interaction. The above commenter used the term LSP without defining it. LSP means "Language Server Protocol." ~ Your friendly neighborhood Hacker News LSP Bot

xpe4• 3 years ago

How might a Language Server Protocol (LSP) work for debugging? And predictive debugging in particular? Any examples you've seen? Do LSPs have a mechanism by which debugging could work? (I haven't researched this yet.)

Note: I ask the above questions in an attempt to move the conversation towards predictive debugging. Like many, I visited this thread because I'm interested in the core topic: debugging. We're probably not here for a general discussion about the pros and cons of JetBrains / non open-source software / IDEs.

Groxx2• 3 years ago

The LSP version of debuggers is essentially DAP, and it is gaining some support: https://microsoft.github.io/debug-adapter-protocol/

Like LSP it's a bit crippled compared to a normal debugger, but the universally-pluggable aspect is compelling.

linux2647• 3 years ago

Their Fleet IDE is exactly that. It’s in development right now, but a public beta is available: https://www.jetbrains.com/fleet/

Edit: I just realized you meant the opposite, where the IDE is just a language server. Fleet is an IDE that uses language servers

nsm• 3 years ago

I don't know if the sort of deep refactoring and other integrations they offer in their IDEs would be possible in the LSP protocol as it exists today. I believe debuggers for vscode have a similar DAP protocol. Any way, my point is, they have something great that works well for their users, so it doesn't seem like a very large market to make a less powerful version for a small set of new users. I also suspect that the sort of developers who don't use their IDEs but instead use an LSP based editor that is free/open-source would be unlikely to _pay_ for Jetbrains LSP based offering, however high quality/unique it might be.

seeknotfind• 3 years ago

Cool idea. Guess we are still having innovations without LLM

xpe4• 3 years ago

Reading the title, my mind conjured what I consider to be the utopia of debugging: a probabilistic assessment of likely failure points across the entire stack, backed by reasoning over the relationships between the components. From there, generate a prioritized actionable debugging task list. An example might be "test input X on function Y so we can rule out Z".

(Aside: The "so we can" part is the key to focusing the effort, otherwise debugging can turn into a technical-debt paydown-party. There is a time and place for that, but maybe not while your service grinds to a halt for unknown reasons.)

Therefore, you can imagine why reading the article was extremely disappointing. No offense, JetBrains; this is still probably pretty good. Certainly a step in the right direction. And it is predictive debugging on a local scale.

Context: I guess you could say I'm a harsh critic of the comically inane ways that debugging often happens. Often debugging happens at the worst time: when a person's cognitive capabilities are drained and stress is maximum. This is not the set of conditions we want. Debugging is quite different than designing. Mental flexibility and triage is key.

Calzifer• 3 years ago

Not very convinced yet. I would expect it to fail/not work quite quickly for cases which are "worth" debugging.

For the first example I would just place the breakpoint on the return. The highlighting is quite nice but I see not much need for the prediction which was probably hard to implement.

In languages like Java features like "hot code replace" exist. In a function like this if the result is not what I expect, I can simply change the code and restart the function without restarting the application. Not much need for prediction.

What I wished were more common is reverse debugging were you can step backwards.

yodon• 3 years ago

Wow. Your description of Time Travel Debugging plus Edit and Continue working together sounds amazing. Definitely want to try that some day. I suspect the side-effect/impure-function-call detection JetBrains has built here will be key to understanding boundaries of where/when you can roll back, edit, and continue, and where/when you can't.

matkoch• 3 years ago

Did you see the part about live edits? Placing the breakpoint at the return would be too late to have that working.

Edit: or maybe that was your point after all?!

jerrygoyal• 3 years ago

a version of this already exists in VSCode but you need to enable it from settings

Xeamek3• 3 years ago

Don't know if this will be "game-changing", but definitely a nice to have QoL

Bootstrapper9092• 3 years ago

Very neat! Generally speaking, it seems like testing and debugging, hasn't been getting as much love as other aspects of computer programmeing.

Debuggers still work pretty much the same as gdb, only now with an integrated UI, and there's so much room for improvement. This is a great start and can be taken much further with abilities to mimic complete sessions to easily reproduce bugs and run them again and again until fixed.

We see the same at the related testing automation field (disclaimer - checksum.ai founder). Same testing methods, same testing problems, only fancier packages.

GuB-42• 3 years ago

Testing gets a lot of love, at least in theory. Much less in practice because it is a whole lot of extra work.

As for checksum.ai, is it some kind of fuzzer? I think fuzz-testing is under utilized, in fact, many of my colleagues don't even know that it exists, and I have never been on a project where it was done. Generally, fuzzing is done in the context of security, but I see no reason why it should be limited to it.

shyamkumar7• 3 years ago

Not exactly (disclaimer - on the checksum team). Our goal is to generate end to end tests that mimic user behavior, using real production sessions to train our models. These end to end tests could then be run during development, effectively smoking out real potential bugs before your latest deployment makes it into production. Of course, part of the generated tests could involve fuzz testing if it makes sense (if there is a form field, input, etc.)

nathants• 3 years ago

printf and fast iteration are all that’s needed. golang got that much right.

my pace on clojure and my pace on go are the same. the vibes are different though.

my current 20s iterations with cpp are not great.

Noumenon72• 3 years ago

Sounds awesome, but I never get to use PyCharm's debugger any more because all our tests run in VS Code devcontainer with a docker-compose full of helper containers like databases.

Aaron2222• 3 years ago

JetBrains recently added devcontainer support to their IDEs[0]. Not something I've had a chance to try out though.

[0] https://plugins.jetbrains.com/plugin/21962-dev-containers

Predictive Debugging: A Game-Changing Look into the Future

Related Stories