BugJail: New Java debugger – debug across time with mouse

by jacquesm • 7 years ago

263 124 7 years ago

giorgioz• 7 years ago

As a constructive feedback on just the presentation of the landing page I think the screenshots are showing some code with theme colors that are kind of ugly. I understand this is a landing page but if you care about Developer Experience DX I'm sure you can find a theme that is better looking or inspiring.

DailyHN• 7 years ago

Those images aren't visible on mobile. And I was expecting much worse when I viewed them in desktop mode.

Based on the stage they're at, I'd save optimizing the landing page design for the future.

Though, I would put effort to better describe the problem on the landing page.

HenriNext46• 7 years ago

Thanks for the feedback. Do you mean the code syntax highlighting colors, or the dark blue UI colors overall, or both?

ylyn• 7 years ago

This looks like rr or GDB reverse debugging but for Java.

wheresvic32• 7 years ago

From the main page:

  Isn't BugJail just a reverse debugger?

     No. While BugJail does have similarities with reverse debuggers it is actually quite different ...

dhlin• 7 years ago

Have a question to rr or any replay debuging tool, does it still work if the program has database access while the replaying machine doesn't have or not have the same database?

cfallin• 7 years ago

Yes, it records all syscall return values, so the replay doesn't actually need to perform them. This means that IO (GUIs, network accesses, etc.) shouldn't happen on replay, in general. (The replay does show stdout/stderr data, so perhaps those are special-cased?)

wellpast2• 7 years ago

I wonder how they are able to trace and capture variable and stack state at a rate of 20M invocations — per second!?

How is this possible on the JVM?

(I’m aware of the instrumentation API that profilers use but I thought they had to do sampling to limit interference with program performance and they are not capturing all the data req’d for debugging either.)

Anyone know?

Liquid_Fire• 7 years ago

Pure speculation: You don't need to record everything, only enough information to reconstruct what happened later (e.g. outcomes of branches, results of system calls, etc). You can then run a post-processing phase after execution (or in parallel on another thread) to fill in the extra detail that is needed for efficiently presenting what happened.

HenriNext46• 7 years ago

Actually, the 20 million events per second is the real event count, and on top of that comes processing..

HenriNext46• 7 years ago

There is no single trick, just lot of hard work.

If there's interest, I can write a blog post about this.

wellpast2• 7 years ago

I would love to read this. Thank you.

notkaiho• 7 years ago

The page needs an editor, stat. The grammar is _not good_. I'd be happy to do it to be fair, doesn't seem that big a job.

HenriNext46• 7 years ago

Thanks, actually I'd really appreciate your help! My email address is the contact page.

lowercased2• 7 years ago

android debugging to start in late 2018...

is there any progress on this project (bugjail in general I mean, not android)

HenriNext46• 7 years ago

Auh crap, that is just typo, should have been 2019. BugJail is under very active development.

carstenhag• 7 years ago

Was also about to comment about this, haha. I'll be interested once this is android/kotlin/android studio-ready!

lowercased2• 7 years ago

thanks for responding here!

reminds me of chronon. any similarity?

netsharc• 7 years ago

Late 2019 is right now, any time after now is really really late!

brian_herman__• 7 years ago

This looks really good. I like the fact you can drill down and have a replay debugger with a slider! Great job! I wish it worked for python!

wheresvic32• 7 years ago

This is really neat and I wish I had this around 10 years ago when I had to migrate Java apps.

However, these days I rarely need to debug programs while developing - I usually use some combination of TDD with unit/integration testing and it gets the job done with the added bonus of being easy to maintain down the road.

Of course if you're on a project where time to market is critical and you need to sacrifice tests, I suppose a debugger is required. But then again, you're probably not using Java either.

bni2• 7 years ago

I don't see a conflict between using debugger and TDD.

I use the debugger to set breakpoints when running my unit tests all the time. It lets me understand the code better when developing it.

Also convenient when pasting in data from production into the unit test to find what is causing the bug.

HenriNext46• 7 years ago

Hey, I'm Henri, founder and lead developer of BugJail. I just noticed traffic spiking and came to see what is going on :)

The landing page wasn't quite ready for spotlight, but such is life :)

All feedback is highly appreciated, no matter how brutal.

And if you have any questions, feel free to ask!

w0utert• 7 years ago

Since you asked for it ;-)

> All feedback is highly appreciated, no matter how brutal.

You should really get a native English speaker to proofread and correct the website. The content itself is fine but there are so many spelling & grammar errors in the text that it really starts to detract, and it honestly affected my perception of the quality of your software in a bad way.

Also not a big fan of the long-scroll style landing page in general, but that seems to be the flavor of the day now. I think some structure (index, menu) would make it a lot easier to navigate to the relevant information, right now reading top to bottom I had no idea what kind of other information could/would/should have been lower down the page.

HenriNext46• 7 years ago

Yep, absolutely agreed. The landing page is work in progress and hasn't been proofread. Didn't except it to end up in HN :)

Apologies for typos and grammar, and I hope you can also brutally criticize everything else beyond the surface! :)

mlyle• 7 years ago

He should really remove the political content too. Yes, many of us think Trump is an asshat, but why alienate a potential customer who disagrees? I think it's an attempt to be edgy/cool (along with "Frigging") but I think it risks hurting more than it helps.

thenewnewguy2• 7 years ago

All feedback is highly appreciated, no matter how brutal.

Some complaints about the linux packaging:

The website mentions a license, but doesn't appear to provide a link to one, and my .deb download didn't come with a license either.

You're redistributing Oracle Java for some reason? Might want to just mark that as a dependency, and I hope you have a license for that.

You have two copies of libEGL.so for some reason.

Sometimes the directory is BugJail and sometimes it's bugjail.

I don't use a distro that takes rpm or deb packages, maybe there could be a .tar.gz download for us?

HenriNext46• 7 years ago

Thanks for the feedback. Quick replies:

- The Linux packaging will be re-done from scratch (previous beta had completely re-done packaging for Windows and Mac, re-done Linux packaging coming in next beta) and I'll make sure all these get fixed.

- The license is displayed in the launcher screen (before you click "Enter BugJail"), but i'll make it also available online, good point, thanks.

- Redistributing Oracle JRE as part of BugJail package is done exactly according to their redistribution terms, no problems there.

- The reason for redistributing JRE has more to do with Windows and Mac, where 'marking it as dependency' is not an option. I'll reconsider how to do this when the Linux packaging is rewritten.

- Would AppImage packaging work for you? What distro do you use?

palerdot• 7 years ago

Next is BugJail for Android, with beta starting late 2018, hopefully.

Why does the website showcase outdated content? Personally, I will drop off at this point. You might have put this message pre 2018, but you should keep the landing page up-to-date.

Also, as one of the other commenters pointed out, you should spend some extra bucks and proof read the landing page. Sentences like "And cheap perpetual license after that" does not go well for most customers who are willing to shell money for a product. If your product is worth it, people will pay for it.

Tech is just a part of the equation. If you don't take care of other stuffs like messaging you are sending wrong signals to your potential users.

HenriNext46• 7 years ago

That 2018 is a really unfortunate typo, but I feel hesitant to deploy fix now as the traffic is heavy.

I agree on all points, and have no excuses; the landing page wasn't ready for being surprise featured in HN.

Many thanks for your feedback & apologies that I provided such a crap experience.

lallysingh3• 7 years ago

I think you need to make the experience more concrete.

On the visceral side: put up some example videos of use. If you can't do it to the level of polish you like, get someone else to do it for you as an unofficial one. IntelliJ is already slow, and I'm worried about event-recording debuggers being only useful for small programs -- larger ones eat up too much RAM and cpu overhead to be tractable.

On the power side: show some sophisticated use cases using real libraries and real problems to fix. What stuff can I find out directly that I can't easily do in IntelliJ?

Can I directly query the DB? Can I use those queries' results in regression tests?

When I read the page, I thought "Oh, like the valgrind family of tools for Java, nice." So perhaps some pre-baked validations using the tech you've built could be useful.

touisteur2• 7 years ago

If you can't do it to the level of polish you like, get someone else to do it for you as an unofficial one.

Yes, have one (or several) customer(s) write a blog (that you can help proof-read, add links to documentation, your example database, your reference guide...) on one or two features. "here's how I did it before with eclipse/intellij, how it was long and painful and manual and involved and xxxx. Now with bugjail I can do this with xxx and xxx". A blog I can read and you can polish bit by bit, asynchronously. Video can be hard to follow, copy-paste, with good and clear simple English, or you could caption but then again, lots of energy to spend!".

Maybe offer some incentive to this (those) customers of some kind (5 year free support, a number of enhancement tickets...).

The best way to demonstrate (I feel) is sometimes a happy customer with complex but satisfied needs.

HenriNext46• 7 years ago

Yep, I think video would be a perfect fit.

About performance impact and program size: BugJail can be used also with hardcore programs, like databases and compilers, where event based record-and-replay debuggers typically struggle. There's still significant performance hit, around "twice as slow", but it won't be orders of magnitude slower.

Also, BugJail is tested with large applications. For example one test is startup sequence of Eclipse, on 5 year old box with 32GB. The result is ~700 million rows in database, in ~30 seconds. It does consume all memory, and your workstation will scream for mercy, but it gets there :)

Direct DB query is not available in current beta, but it is partially implemented in dev version: it shows the results, but there is no navigation from the query results to CallTree, Source code view etc. So: coming, but not yet.

I hope I answered your questions at least partially? It's 7AM in the morning, after one of the hottest days in Australian history, and a surprise appearance in HN front page, so I might not be quite the sharpest tool at the moment :)

j15t2• 7 years ago

Hi, I'm interested in program execution state and the different ways it can be represented/modelled. The "relational model" you describe sounds very interesting. Do you have a technical whitepaper on this topic, or perhaps some third-party literature that has guided you?

HenriNext46• 7 years ago

Neither actually. But if there's interest for the technical side, I could perhaps write a blog post?

cryptonector4• 7 years ago

Is this like DTrace/BTrace with lots of enabled probes married to a database?

HenriNext46• 7 years ago

If you also add GUI after the database, then it mostly is :)

proc0• 7 years ago

I think JS should be higher on the priority list, considering the demand to effort ratio (I'm thinking it's probably easier to implement for JS than Java). I would guess that it would also be easier to host it and load JS on the fly to debug online. Just my 2c, will try the Java one anyway to test it out.

HenriNext46• 7 years ago

I totally agree on the demand, but on the effort side the dynamic nature of JS makes it really tricky to do right. Thanks for your comment & let me know how it goes!

shujito• 7 years ago

First of all, this looks promising

I have just one question, what will the price be?

HenriNext46• 7 years ago

I don't know! How much would you pay for it?

However, I can promise that the pricing model will be perpetual license based, NOT a subscription that "only costs you one cup of coffee per day"..

lvh• 7 years ago

Sounds similar to sayid, though generalized to the entire JVM?

[sayid]: https://github.com/clojure-emacs/sayid

bni2• 7 years ago

I find this very interesting. Can you use it integrated in IntelliJ. How about as a paid plugin?

HenriNext46• 7 years ago

There's no plugin yet, but definitely coming (I've been using IntelliJ almost 20 years now, so also want that integration for my own use).

jhvkjhk• 7 years ago

The `direct answers` looks fascinating! Do we have some equivalent tools for c++ or python? I'm tired of step step step...

trishume• 7 years ago

https://pernos.co/ is making a very similar omniscient debugger based on rr for AOT compiled languages with DWARF info.

ccleve3• 7 years ago

Two comments on the website:

There is some reference to a beta in 2018. Should remove that.

Also, a video of the thing in use would be helpful. Put it right up at the top. Show me something I can't do now. Include specific examples.

HenriNext46• 7 years ago

Thanks, that 2018 is a very unfortunate typo, should have been 2019.

And agreed, video would be great, but it takes fair bit of time away from development, so I'm undecided.

ccleve3• 7 years ago

Shouldn't take long. Google "screencast software". Record yourself: "Hi, this is Henri. Let show you how find bugs with BugJail..." Do a 3-minute video with some simple examples. Put it on YouTube and embed it in your site.

If I were doing it, I'd budget 4-8 hours of work and close to zero cost.

ctataryn• 7 years ago

How is this different than http://chrononsystems.com/ ?

localhost• 7 years ago

I would imagine that the execution logs would get rather large. Is there a way to dynamically trigger logging as well?

HenriNext46• 7 years ago

The execution logs (or "Captures", in our Jail terminology..) do get fairly large if you capture something hardcore, like database or compiler.

For normal web/business/enterprise applications you can configure BugJail to only capture your own packages, which massively reduces the file size (and performance impact).

To give concrete example of hardcore scenario, capturing startup of Elasticsearch results in about 85MB file.

There is no way to enable capturing dynamically, and this is likely never going to be supported, due to the way how BugJail works (it reconstructs the program execution, so if beginning of the execution is missing, "things are not fine").

baud147258• 7 years ago

No Java 11 support? I was interested in giving it a try after the holidays, since it look like it would be a good tool for the type of bugs we have.

I think it could be a big barrier to adoption, considering how Java 8 is EOL.

HenriNext46• 7 years ago

Yeah, Java 9+ is on roadmap for next beta.

When the previous major beta was released there was one dependency that didn't support Java 9 and locked everything to 8.

If you'd like get notified when Java 11 support is ready, you can just send one line ping email (my email address is in the 'contact' page).

Many thanks for the feedback!

aarbor9892• 7 years ago

I tried using it and my first impression is that it's pretty difficult to use for a webapp (or maybe just specifically a Spring application since there are a ton of added proxy classes) without the ability to search. You need to know exactly which thread accepted your http request and then dig your way down the call stack to find your method invocation. You should be able to search for your method invocation and then go back up the call stack from there.

Other than that, seeing exactly how long every single method call took is awesome. I think there's a lot of potential, but without the ability to search I can't see myself reaching for this again

HenriNext46• 7 years ago

I totally understand your point.

The search functionality is currently under development, and it's by far the most critical 'missing' feature.

As a workaround for webapps: using the 'Capturing Assistant' and configuring BugJail to capture only your own packages helps a lot. That way you usually see clearly which thread handled the request, and also have less app-server / framework level crap that you need to expand through before you get to your own code.

If you'd like to get notified when the search functionality is available, just shoot me an email (my email is in the contact page).

Thanks for providing the feedback, and sorry that the initial experience wasn't better.

aarbor9892• 7 years ago

Thanks for your reply. I actually did specify my packages in the agent -javaagent:/Users/aa/.BugJail/agent.jar=--snapshot:.--primary:io.mycompany. but it didn't seem to help too much. I'll shoot you an email to get notified - is there an estimated timeline?

carapace• 7 years ago

Can I get the source? In other words, how do I debug the debugger?

cryptonector4• 7 years ago

I'm guessing "no", but it sounds a lot like DTrace/BTrace with many enabled probes married to a DB.

cryptonector4• 7 years ago

See https://news.ycombinator.com/vote?id=21846545&how=un&auth=99...

HenriNext46• 7 years ago

Nope, at least not at the moment: BugJail is closed source.

stevemk14ebr• 7 years ago

This doesn't work if source code is not available. I can get a trace of the function calls but no arguments, or return values or anything. This makes it entirely useless as a reverse engineering tool, something i was very excited to use it as due to it essentially being a tracer. It also NEEDS modern java support.

HenriNext46• 7 years ago

Actually, it does work without source code (or, should work)

The left side 'CallTree' is fully functional, and shows owner instances, arguments and return values. If you click those, then the Object/Array is displayed at bottom and is expandable just like in normal debugger (+slider to change debug time). So only the right side "Source" view is missing if you don't have source.

If the above doesn't work, could you get in touch and I'll get it sorted out? https://support.bugjail.com

And fully agreed on Java 9+ support, coming in next beta.

Thanks for the feedback!

thethirdone2• 7 years ago

I tried playing around with this and I can't see it being useful for me currently. If the source and tracing information were more tightly integrated it would be a lot more useful. Offering a more traditional debugging experience with the ability to rewind time to the last time a value is set would also make it valuable.

A specific issue I ran into was that multi dimensional arrays do get logged as changing when a value is set in the bottom layer. This meant for my application the were a bunch of init calls setting that value, but none of the actual logical changes were tracked.

HenriNext46• 7 years ago

The source and tracing sides will definitely get fully integrated. This is only the second beta that even has the source side at all, so it's just "early days".

The line/statement level traditional debugging is possible to implement with currently captured information, but the implementation is seriously hard, likely in 2021+ timeframe.

About the specific issue: I'd really like to get that fixed, do you think you could shoot me an email so that I can followup in case that i cannot repro the bug (my email is in the contact page)?

Many thanks for the feedback & apologies that the initial experience wasn't so great

thethirdone2• 7 years ago

I made a gist [0] that demonstrates the issue. Line 5 does not make a node in the timeline.

I look forward to seeing what bugjail becomes in the future.

[0] https://gist.github.com/TheThirdOne/7af9356dfb89d21816250017...

josteink• 7 years ago

Offering a more traditional debugging experience with the ability to rewind time to the last time a value is set would also make it valuable.

Visual Studio has this.

It’s called time-travel debugging, and it makes your code run really slow :)

kij• 7 years ago

Bugjail was mentioned in a thread about a xor bloomfilter alternative. Or at least that's where I saw it first mentioned. My bet is it triggered op's posting.

jacquesm• 7 years ago

Absolutely.

dang• 7 years ago

That was quite interesting: https://news.ycombinator.com/item?id=21841921

Alupis2• 7 years ago

Looks like a pretty nice way to debug Java programs! I look forward to testing it.

One quick note though, you might want to consider removing the politics from your screenshot before really releasing this product. You've got a great product here, but there's no need to inject politics - even if it's amusing to some. It takes away from the seriousness of your project and makes it feel more like a personal side project instead of something I should pay for and build into my workflow, in my opinion. Just my two-cents.

HenriNext46• 7 years ago

Yep, it was already mentioned somewhere here, but basically: I agree & will do.

I look forward to your feedback on the actual product, be as brutal as you can / want! :)

Alupis2• 7 years ago

Great! I've shared your link with a few other java devs and the consensus so far is this looks like an amazing tool. I can't wait to take it for a spin! Keep up the great work!

knutwannheden• 7 years ago

Since "database" and "relational model" is mentioned, I assume the collected data is indeed being written to an RDBMS. Would be interesting to be able to poke around in that with some SQL, even if not available as an official API.

HenriNext46• 7 years ago

Sure, I can add that to next major beta. Thanks for the suggestion!

petagonoral• 7 years ago

How does it compare to Chronon DVR[0] for Java. Seems similar functionality wise

[0] http://chrononsystems.com/

datastoat• 7 years ago

Database-backed debugging is great! It means that debugging can be done as a data science exercise, which is very useful when, for example, a rare confluence of external factors make the program behave badly.

Earlier examples: "Debugging in the (very) large", which describes how Microsoft used Windows telemetry, especially for debugging weird slow-down glitches. In the networking domain, where there is AppDynamics for trying to debug the interactions between network and distributed services.

I'd love to see data-science style debugging much more accessible (especially in Python). And it's great to see that you're making a serious effort to get the user experience right -- though I hope that the underlying data will also be accessible from more capable data science platforms.

HenriNext46• 7 years ago

BugJail writes its data files in open source format, Apache ORC.

You can load the data files directly to Spark, Hive or anything else that reads ORC, and then use Python to query, visualize or get statistics from the captured program execution.

Give it a try and let me know how it goes?

rmetzler• 7 years ago

I think it might be really useful to run BugJail as an (optional) CI step and offer the user to download, integrate with the IDE, or have historical data. But I didn't fully understand the license regarding such usecases.

BugJail: New Java debugger – debug across time with mouse

Related Stories