Scrumdog – a program to download Jira Issues to a local database

https://whoek.com/b/jira-to-sqlite-with-scrumdog.html

by bosveld • 4 years ago

149 61 4 years ago

This is what I need but for Mac! Any idea of alternatives? I have multiple Jira instances I want to combine -- literally can't find an app that does this.

bosveld• 4 years ago

Author here - Yes that is exactly what it does. I'll compile it for the Mac later today and load it on the Scrumdog.app website. Thanks for your comments.

dijondreams2• 4 years ago

Wow thank you so much!!

ellisv2• 4 years ago

Any alternatives?

You could use Steampipe w/Jira plugin: https://hub.steampipe.io/plugins/turbot/jira

ellisv2• 4 years ago

Looks like a cool project!

ananthakumaran• 4 years ago

I had used https://github.com/go-jira/jira in my previous job. custom-commands is a very useful feature. I had built a set of commands to filter out issues for different use cases. The command line approach was way faster than waiting for the jira page to load and click the correct set of filters.

digerata• 4 years ago

No explanation necessary. Anyone that has followed Atlassian in the news knows that downloading Jira Issues is an excellent idea!

brtkdotse• 4 years ago

Are you suggesting they’re going out of business?

verst• 4 years ago

Probably a testament to performance and reliability of the hosted offering (we all remember the multi day outage not long ago).

onion2k• 4 years ago

Apparently they've never made a profit...

outworlder• 4 years ago

Until the current crop of managers retire, Atlassian will still have business. Some don't know anything other than Jira, so will recommend it on corporate purchases.

christophilus• 4 years ago

Missed naming opportunity: should have been “scrumbag”.

mxuribe• 4 years ago

Agreed, but i also think the -dog portion of the name likely refers to *fetching* of data...you know, like a dog fetches...well, i woud guess anyway. ;-)

tomjakubowski• 4 years ago

"slumdog"

np_tedious• 4 years ago

They are definitely hoping to make millions of dollars off this tool, and then become a...

mnd999• 4 years ago

How about Jira-is-dog-slow-and-the-interface-is-garbage-so-just-give-me-the-data? Not catchy enough?

Presumably if you built a front end for the database you’d then have a not shit issue tracker.

kderbyma• 4 years ago

if he sold it for high enough, could be a Scrumdog millionaire

simonw2• 4 years ago

Absolutely love this. I have a whole bunch of similar tools for importing stuff from different services into SQLite but I never got up the courage to take on Jira!

Here's my github-to-sqlite tool which imports GitHub Issues https://datasette.io/tools/github-to-sqlite - being able to query them in a local SQLite database is super-handy. I run a demo instance of the resulting database here: https://github-to-sqlite.dogsheep.net/

iampims• 4 years ago

Heads up:

Error 500 cannot import name 'etree' from 'markdown.util' (/usr/local/lib/python3.8/site-packages/markdown/util.py)

simonw2• 4 years ago

Thanks for letting me know - fixed that here: https://github.com/dogsheep/github-to-sqlite/issues/74

zerop3• 4 years ago

Just to get ideas, what custom reports are users looking for that can not be built by Jira.

standardly• 4 years ago

I think this is more of a way to automate pulling large amounts of data from Jira rather than doing it directly from a dashboard. Also could be utilized as a backup repository.

jrockway• 4 years ago

I've been wanting to suck in Jira tickets to my own database so that I can full-text index them along with code, docs, customer requests, Slack, etc. Then when you're looking for information, you only have to go to one place.

(My ulterior motive is to write a blog post that's like "how I used the product I make at work to index all information at the company". I can usually just do a linear scan across the backlog to find the issue I want, but hey, when marketing and software combine, I can't complain.)

If there's some service I can buy that does this, though, I'd rather use that.

SloopJon• 4 years ago

In my case, I find it hard to query related issues of different types. It's kind of hard to explain unless you see how we use Jira, but I basically want to do joins. You can do some interesting things with the Structure plugin, or issueFunction() from the Adaptavist ScriptRunner plugin. The other thing I've done is to pull a few REST queries into a single-page app., but that takes more work.

zerop3• 4 years ago

If program/project managers having coding love/passion, a lots of innovation could come around Jira

zerop3• 4 years ago

Atlassian may want to provide SQL interface to Jira data. JQL is limited in many ways.

bokchoi• 4 years ago

This is my favorite limitation: https://jira.atlassian.com/browse/JRACLOUD-21372

sokoloff• 4 years ago

Wow. Going on 12 years to get exact text matches working; that sounds crazy.

tgv• 4 years ago

There's an sql database backing Jira, but it looks rather ugly. I guess that's why they translate it.

emmelaich• 4 years ago

I think it's more about access control. They don't do any access control at the db level afaik.

bicx2• 4 years ago

I built a very similar tool for my company to download a set of DataDog logs into SQLite. Products like Jira and DataDog have their own query engines for sure, but you really can't beat the capabilities of true SQL for deeper analysis.

mdonahoe• 4 years ago

link?

bicx2• 4 years ago

Wish I could! It's locked away in a private company repo.

nonameiguess• 4 years ago

My manager a few jobs back did something like this, but just using Microsoft Excel with Power Query. No need to even install sqlite. Normally, that probably isn't much of an obstacle, but this was in a classified environment where it was nearly impossible to get the local IT admins to install additional software on your workstation. Every corporate Microsoft box is going to already have the Office suite.

crestfallen2• 4 years ago

I'm curious about this. I would say I'm fairly versed in Excel but haven't dug too deep into Power Query / the data model. If you wouldn't mind, how is this done?

ako• 4 years ago

Looks like it's documented on the atlassian website: https://community.atlassian.com/t5/Jira-Core-Server-question...

vorpalhex• 4 years ago

I liked the approach to slowly narrowing down the language choice based on needs. A good example of finding the right tool for the job.

Benjammer• 4 years ago

FYI - This type of workflow automation and connecting to Saas product APIs and moving data around can also be easily built using a no-code RPA tool like PixieBrix[0].

[0] https://www.pixiebrix.com/

bbkane• 4 years ago

You might also try https://www.beekeeperstudio.io/ or https://dbgate.org/ for working with databases (SQLite and others). I really like the UIs of each and they're both open source- supported by donations if you find either useful.

evantahler• 4 years ago

Just noting that you can use an EL(T) tool like Airbyte (OSS) for this as well! The integrations are already built - Jira -> mongo, Jira -> Postgres, etc.

https://docs.airbyte.com/integrations/sources/jira/

Tostino• 4 years ago

Yup it makes a lot of sense to use the already built tools for this.

nathanwallace2• 4 years ago

Readers may also enjoy Steampipe[1], an open source [2] tool that uses Postgres FDW's for live SQL queries against Jira, Confluence, GitHub, GitLab, AWS, Slack and many more tools [3].

1 - https://steampipe.io 2 - https://github.com/turbot/steampipe 3 - https://hub.steampipe.io

adren123• 4 years ago

There is also git-bug [1], an offline bug tracker fully embedded in git which can bridge to Github, Gitlab, Jira

[1] https://github.com/MichaelMure/git-bug

tomrod• 4 years ago

Steampipe sounds interesting (and will be looking into scrumdog too).

Is Steampipe a community + services model, or just open source + community?

nathanwallace2• 4 years ago

The Steampipe open source project and community are managed by Turbot [1]. We also have Steampipe Cloud [2] in preview if you'd prefer a hosted version focused on teams.

1 - https://turbot.com 2 - https://cloud.steampipe.io

lloydatkinson• 4 years ago

Maybe someone can build a functioning UI for jira with this because Jira is terrible.

theknarf• 4 years ago

Why are you hosting compiled binaries on GitHub and no source code...

zaphar• 4 years ago

Jira's querying and reporting facilities leave a lot to be desired. I have some scripts to pull the data down into a sqlite database so I can query it much better.

Jedd• 4 years ago

If you're forced to use Jira, and setting up a local replica database is surplus to your interest / requirements, I can recommend the Jira Assistant chrome extension.

The licence is a bit wonky (source-available, basically) but the feature set fills a few gaps with the regrettable stock UI.

https://jiraassistant.com/ or https://chrome.google.com/webstore/detail/jira-assistant-wor...

pix64• 4 years ago

Is possible to create single file applications in C#

https://docs.microsoft.com/en-us/dotnet/core/deploying/singl...

earleybird• 4 years ago

While it is possible, it is certainly less than stellar compared to the other languages that were reviewed. Granted my experience was 1 - 2 yrs ago with .Net Core 5. I recall having to deal with setting different temp directories each time I wanted to run a another instance of a 'single binary' because the first thing it did was unzip the bundled runtime into the system wide TEMP directory. It certainly was a single file app but behaved more like an installer.

Whoever came up with that obviously didn't understand the why of single .exe applications.

tus666• 4 years ago

I like the name. Scrumdawg would have been better though.

GekkePrutser• 4 years ago

I wish there was a program to enter hours into Jira (and related tasks of course) and upload them.

We have to use Jira for logging hours at work now because they think that makes us agile. Needless to say the only agile related thing we do is logging hours in Jira so that's a complete joke. Our project management department is full of complete idiots.

But Jira is such a dog to work with. Especially if you only enter stuff for the sake of it. It won't accept "7h33m", no it has to be "7h 33m". For every change you need to refresh all pages before they're picked up. It's ridiculously slow.

A frontend where I could just dump my made-up hours that would then take its time to upload that shit to Jira would really help me to be less stressed.

eliaspro• 4 years ago

You might also be interested in the excellent Bugwarrior [1] which syncs issues from a wide variety of issue trackers (JIRA is amongst them) [2] to your Taskwarrior [3] database.

[1] https://bugwarrior.readthedocs.io/en/latest/

[2] https://bugwarrior.readthedocs.io/en/latest/services.html

[3] https://taskwarrior.org/

Scrumdog – a program to download Jira Issues to a local database

Related Stories