Show HN: Linkwarden – An open source collaborative bookmark manager

by daniel31x13 • 3 years ago

280 88 3 years ago

Hey there HN! Meet Linkwarden, a fully self-hostable, open-source collaborative bookmark manager to collect, organize and archive webpages.

Please also visit/star our GitHub repo [1].

Linkwarden was built using TypeScript and NextJS, backed by a PostgreSQL database for the lighter-weight data. The rest of the data can be chosen either to be stored on the filesystem, or stored on the cloud on Digital Ocean Space/AWS S3, the reason for the cloud storage solution was for the Cloud offering [2], we realized that the preserved webpages (archives) take up space pretty quickly and S3 was much more efficient for this task. On the front-end we used TailwindCSS for styling and Zustand for state management.

You could either use our Cloud offering (with 14-day free trial) to directly support this project and experience Linkwarden, or you could self-host it on your own machine and have maximum flexibility.

Feel free if you had any questions, we'll do our best to answer it.

[1] https://github.com/linkwarden/linkwarden

[2] https://cloud.linkwarden.app/register - Hosted in Digital Ocean's datacenter located here in Toronto, ON.

pratio2• 3 years ago

I'll definitely give it a short this weekend. Are there any plans to support different authentication methods? Like LDAP, OAuth2 etc?

I'm using linkding at the moment https://github.com/sissbruecker/linkding which also has a browser addon, the only missing thing is some form central user auth but we're using it as it is.

DaniDaniel50057• 3 years ago

Currently the only authentication methods are using plain username/password as default.

And if the extra environment variables are set properly, you could hook it up using the email provider, taking care of the confirmation emails and one time links.

squiggy22• 3 years ago

If its on nextjs I've a feeling there are auth providers kicking about to implement sso at least.

DaniDaniel50057• 3 years ago

Absolutely, the authentication is being handled by next-auth so there are lots of providers that can be added in the future.

jhot• 3 years ago

Linkding does support header auth if your provider supports that (I run authelia backed by ldap).

pratio2• 3 years ago

Thanks for pointing that out. I'm already using authelia backed by ldap, it'll be an easy setup. I might've to figure how to merge the existing users and their data though.

10000truths• 3 years ago

Any relation to Bitwarden, or just a happenstance similarity in names?

DaniDaniel50057• 3 years ago

No we’re not related to Bitwarden, we both just have a nice name and are opensource :)

codegladiator• 3 years ago

It is a Linkedin for Bitwarden.

rounakdatta• 3 years ago

No please no. We have one LinkedIn, and that's enough pain to humanity.

j452• 3 years ago

Looks really clean.

A few questions:

- It’s not clear if this saves highlight in Ng and annotations (notes about the highlights). More than saving a bookmark we think about a sentence that can be searchable.

- Is there any plan to save the entire webpage as text (to maintain the annotations in it) in addition to pdf and screenshot?

One product I am overly dependant on is Diigo - I would love a replacement even if it was self hosted.

DaniDaniel50057• 3 years ago

Saving webpages as text was actually something we wanted to do before launch but just went for the “MVP” for now.

So yeah we’re definitely bringing more archive formats.

xurbax• 3 years ago

Please consider integrating SingleFile support as well, perhaps at least in coordination with the planned browser addon. (ArchiveBox supports SingleFile on the server-side running under nodejs.)

j452• 3 years ago

Great to hear, will keep an eye out for it. I take the saved pages and pull them through my workflow via API.

RevoGen• 3 years ago

Are there full-text-search capabilities?

DaniDaniel50057• 3 years ago

If by full-text-search, you mean the website contents, not really.

But if you mean, searching the link details, yep.

keepamovin• 3 years ago

If you want full-text-search with archiving check out my project, DiskerNet. https://github.com/dosyago/DiskerNet --> also well done on LinkWarden! Looks like a great product! :)

uzername• 3 years ago

Hey, this looks great!

In your readme, in the "A bit of history", it should be `has many fewer features`

On a more technical note, I wondered if you have any stories working with Prisma and Next? It works but every ORM has its pros and cons. My annecdote with the two is on a project recently, I had issues bundling the appropriate prisma packages during a Next standalone mode build.

DaniDaniel50057• 3 years ago

Prisma is great and I definitely recommend it to anyone who’s either starting out or on a more advanced level.

vsviridov2• 3 years ago

Oof, any time I see next/prisma I already know that my tiny VPS will likely choke building this... So yeah, self-hostable, but not for everyone.

Got burned with this by cal.com self-hosted version: https://blog.vasi.li/cal-com-is-making-me-lose-faith-in-the-...

FireInsight• 3 years ago

I'm making a similar thing with SvelteKit and Kysely so we'll see how that turns out.

thelazyone• 3 years ago

Heh. Not a fan of js apps (npm or not), but your article was enjoyable to read.

vsviridov2• 3 years ago

Thank you.

adr1an• 3 years ago

Same. I don't think I need the collaboration aspect of this app, so I will keep being a happy user of linkding, see: https://news.ycombinator.com/item?id=21872488

sodimel• 3 years ago

Here's a (my own) lightweight alternative, built using django & no javascript: https://gitlab.com/sodimel/share-links

It allows you to store links (title & language of the page, a pdf of the page, assign tags, to include them in collections), it has a very simple (moderated) comment system, set status of the link (online: direct link, offline: replace link by a webarchive one) a lightweight ui (remember: no js), multi-accounts (permissions), translations, some rudimentary stats and some other things (access a random page!).

See my own instance for an example with thousands of links: https://links.l3m.in/

daniel31x132• 3 years ago

Actually Linkwarden was tested on machine with only 2GB of memory and it ran pretty smoothly.

awestroke2• 3 years ago

Build it on your own computer, rsync the result to your vps

efff• 3 years ago

When will docker version arrive?

alexktz2• 3 years ago

Precisely when it means to.

janvdberg• 3 years ago

Not to diminish the effort here, but I just want to point out (as someone who has tried lots of bookmark managers) that Floccus is everything I want from a bookmark manager (effortless sync across devices and just using the bookmark manager in your browser).

I am pointing this out, because I wish someone would have pointed it out to me.

https://j11g.com/2023/03/04/floccus-is-the-bookmark-manager-...

neontomo• 3 years ago

Thank you, seems like what I wanted.

freedomben2• 3 years ago

Thank you! This is exactly what I needed, and what I've been looking for for years! Open source, lightweight, and stable.

attentive4• 3 years ago

xBrowserSync is another one.

You can use https://github.com/ishani/xSyn for self-hosting.

slivanes2• 3 years ago

Another reason why Safari shouldn't be considered a user friendly browser.

dewey4• 3 years ago

Which reason are you referring to?

crossroadsguy• 3 years ago

People who consider Safari a friendly browser do not look before iCloud and do not look after iCloud. No matter how it behaves/performs. That’s how Apple ecosystem rolls. And the ones who do not consider it a user friendly browser do not use it.

neura• 3 years ago

Is this simply because the bookmark manager linked (Floccus) is not available for Safari?

Or better yet, can you elaborate on how any of the content up the chain from your comment that shows why Safari shouldn't be considered a user friendly browser?

gooob• 3 years ago

doesn't look like i can use my own server with floccus

johnnyworker• 3 years ago

If you have a WebDAV server, you can use that.

I use several browser profiles (stuff like social, entertainment, dev), and now I can put the usual sites I visit with each of those in the top level of the their bookmark bar directly, but also have a single folder for the ones I want to share between all of them, yay! I am very happy right now. Thanks GP.

saulpw• 3 years ago

"collaborative" is the key feature that Floccus and all other "syncing" bookmark managers are missing.

awestroke2• 3 years ago

Missing features: a good UI for managing and organising bookmarks, automatically archiving bookmarks in case they go offline

lannisterstark3• 3 years ago

eeeeh.

Shiori looks like it'd work infinitely better compared to floccus. It has an extension, tags, and everything is stored in a central repository you can visit from web (or server itself) any time you want. It also archives your bookmarks. It has been working flawlessly for me for a couple of years now.

https://github.com/go-shiori

attentive4• 3 years ago

That looks like a web app, not an actual browser bookmark sync.

Apples and oranges.

There is a value in using native browser bookmarks and syncing them cross browsers/OS's.

nirav722• 3 years ago

I've been self-hosting and using Shiori for couple years now. Originally it was my replacement for pocket. It’s the closest thing I found to Pocket. Not only for read-it-later, but also as my archiving tool. My only issue with Shiori is that I have not been able to find a working browser extension or get the official Shiori extensions to work. So adding URLs is a manual process. So I only use it archive dev and longform articles for reference later.

hk1337• 3 years ago

I ended up just creating a page in Notion and imported a CSV file.

danShumway• 3 years ago

Genuine question, not trying to bash the project -- the link here seems to really stress that floccus is just for syncing, but can't you just use Firefox Sync for that?

I already have the ability to send my tabs across devices or sync bookmarks, it's built right into Firefox. The UI could be better, but it doesn't look like Floccus changes the browser UI, which is my primary complaint with Firefox bookmarks.

I'm not sure what I'm missing.

trinsic2• 3 years ago

Where's the documentation? I get page not found eror

kornhole2• 3 years ago

This looks slick. Because archive.org is getting a little problematic by not allowing more sites to be archived, decentralized archiving is becoming more important. I have been using archive box on my server. It does not have the collaboration features, but that is what my fediverse instances and other collaboration tools provide.

__jonas• 3 years ago

Because archive.org is getting a little problematic by not allowing more sites to be archived

I haven't heard anything about this, could you elaborate or link to some article?

kornhole2• 3 years ago

I am sorry that details escape my memory at this point, but I have seen a couple instances recently where journalists tried to archive news stories and were served a response that someone has barred articles from this site from being archived. There is also no guarantee that something once archived there will not be removed when they are put under pressure or terms of service change.

Tokumei-no-hito• 3 years ago

I’ve heard no such claim. Interested in an offices statement as well

freedomben2• 3 years ago

This looks really neat! Can you share more about the project? Such as:

1. What is the driving vision behind this project? For example is this just scratching a personal itch with hopes it helps others, or is the hope to expand this into a product or company in the future?

2. Is the goal to monetize somehow in the future? If so, what sort of monetization strategies are being considered? For example, "open core", "paid hosting" (what happens to self-hosted?)

daniel31x132• 3 years ago

Great question, Linkwarden was initially a personal project but then we decided to scale it up into a fully fledged product. Regarding monetization, we already included the paid hosting plan for the users who don’t want to self-host, but the self-hosted option will remain free forever and will always be supported alongside the paid hosting.

ecliptik2• 3 years ago

I've used Raindrop[1] for the last few years and it works well - cross device support, archived pages, and tags/folders.

Going to check out Linkwarden since I really like the idea of being able to self-host something similar since Raindrop could one day disappear (#googlereaderneverforget).

A feature Raindrop has is it can export bookmarks to a standard xml file, which I then have a script that automatically adds them to Archivebox[2] for a local copy and to add them to archive.org[3].

Does Linkwarden, have a feature to automatically submit a bookmark to archive.org along with the local copy? That would greatly reduce this setup and have it all in one tool.

1. https://raindrop.io/

2. https://archivebox.io/

3. https://ecliptik.com/bookmarking-with-raindrop/

DaniDaniel50057• 3 years ago

Being able to bookmark a Link to archive.org was actually something we wanted to do earlier, but we had to do it a opt-in solution per each link since there might be a website that you don’t want to archive for the public and instead only keep it to yourself.

But note that it is on the roadmap (but not top priority).

dewey4• 3 years ago

How has your experience with archivebox after running it for a while? After trying to set it up multiple times I gave it another try a few days ago and it always feels like it's doing too much and is therefore very sluggish and buggy.

I was looking for alternatives but couldn't really find something great with a decent UI and full-text search.

Tomte• 3 years ago

I’ve tried Archivebox (using docker compose) several times, and every single time it just stops.

I import around 3k bookmarks, it starts archiving them. Immediately some archival methods fail (usually screenshot and pdf), and after archiving a few hundred bookmarks it never continues to archive the rest. I‘ve let it sit and do its thing for several days, it never manages to get through all of them (or even a sizable minority).

Different machines, different filesystems, different networks. No idea what‘s wrong.

ecliptik2• 3 years ago

It isn't horrible. I have it running in a docker-compose stack and after initial setup I haven't really thought about it other than checking the Raindrop script I have is still populating it.

I don't really use it interactively, it's more to have a "backup" of websites I find useful after finding some I used to reference for years disappeared and were never added to archive.org or occasionally sending the Readability/PDF versions to my Kindle.

I also setup YaCY[1] at one point with the idea of having my own local personal search engine for the archived sites, but I ended up never using it.

1. https://github.com/yacy

qwerty456127• 3 years ago

Can it import a list of URLs and aut-tag them using some API or pre-trained ML? If yes, I bloody want it! No matter the price.

swozey• 3 years ago

This looks really nice, great work. I'll definitely give it a try.

Have you considered a free tier where you could monetize it maybe via sponsorships/ads with the goal to have a social aspect?

I'm a huge fan of Githubs social trending/explore/lists/topics section for finding new tools for specific things that I work on, rust, go, aws, etc. for myself and my teams. Also things like dev.to, daily.dev, etc but they're not really as useful as I thought they'd be. You can see an example of the Lists I've created here https://github.com/mikejk8s?tab=stars - I wind up putting these lists into a team notion doc right now.

There's those "Awesome-XXYZ" lists but I don't think they're the best way to do this at all. They also wind up very out of date. My Github lists aren't collaborative, I can't give people a way to contribute to them and as far as I know they're not something you can search globally to find if someone has some interesting lists.

It's quite a bit different than what you're doing here but what I've been hoping to find was some sort of technology Looking Glass/aggregator where I could click a topic/Collection, say Rust, and see rss feeds, blogs, curated and very well organized bookmarks, hashtags of other related lists, etc in a collaborative manner with lots of contributors.

I was sort-of beginning to do this via a published notion domain and treating it like a wiki.. https://mrj84.notion.site/Go-Wiki-c637ff57e00046bfbe22fb2562... - that's the closest I've been able to brain storm as something remotely near what I'm aiming for.

Sorry for the long post, maybe it'll give you some ideas or maybe someone has some ideas for me.

andrewrothman• 3 years ago

I like to save the best / most interesting links I come across as I browse the web. It can come in handy to pull up a blog post I read a while ago or remember some new sass product or developer tool I wanted to check out. I'm using https://raindrop.io now which works great for this.

When I looked into it I was surprised that browsers don't have this kind of bookmark management built-in. I'd be very happy with two small additions to browsers: (1) display by / sort by date added and (2) a small separate freeform text box for notes (so I can describe why I saved the link).

(Optionally it could be nice if browsers adopted some standard sync mechanism for bookmarks, maybe based on WebDAV like the Floccus extension).

Then again, these dedicated external bookmark managers do have nice features like tags, search, and offline downloads or page screenshots. Those are all great!

Linkwarden looks like a nice product. Looks like it would tick all the boxes for my use-case and the design is pleasant. I like that it's open source and has a fair price for the hosted offering. Maybe I'll give it a try!

bachmeier• 3 years ago

To save anyone else the clicks, the pricing is $4/month for unlimited links. Currently, no export functionality.

meiraleal• 3 years ago

Why would I want to save a click? The project is very cool and I will use it.

slushh• 3 years ago

Easily share curated collections with the public

Do you have a page that shows the most popular collections?

burkesquires• 3 years ago

I have saved this to my bookmark manager! :-)

pacomerh• 3 years ago

Cool project, quick design feedback, in 'Exploring the use cases' the left column is too narrow? https://ibb.co/f4Q5mnB

stavros• 3 years ago

Just a bit of advice: You wrote a sentence about what the service does, and a large paragraph on what it was built on. When you're pitching your service, tell people what's different about your service, why it's better, why they'll want to use it, etc.

I understand that HN tends to be more technical, but the technical details can be a single link. Right now, all I know about your project is that it's a bookmark manager and S3 is better for storing files than the filesystem.

Good luck!

Modified30192• 3 years ago

Does this have the capability of setting an option to periodically check the page for updates and save a revision?

My ideal bookmark/page archiver would have this workflow:

1) Find a page I like or find valuable for whatever reason, so I click on a browser addon button.

2) A little dialog would then show up from the button, allowing me to set the following

2a) Add tags, as well as offer suggested tags I could add or remove.

2b) Set an optional update frequency, preferably with an option that would slowly reduce the frequency of checking for changes, first if no changes are found, and eventually as an absolute regardless of changes.

2c) Set specific technical page save settings

3) Once done, I click a “save” button in the dialog, and the page would be saved at a single html file, like the browser addon “SingleFile”, (which has some adjustable default settings previously mentioned). This allows saving pages with very simple javascript/dynamic functionality instead of essentially an static image. It also inlines some media: see https://addons.mozilla.org/en-US/firefox/addon/single-file/. That said, perhaps a WARC file may be better when it comes to handling things like compression, multiple revisions, indexing, and possibly following links to download and store linked media.

4) Then it would automatically open the saved page in the browser, so I could have a quick look can make sure it’s not broken for some reason

5) Finally it would then occasionally check for updates, saving a revision. On future visists to the page, the addon would have a little badge to let me know the page has already been saved and is being watched.

It kinda sounds like I want a browser integrated front end with sane and intuitive settings for HTTrack. As and example, let’s say I find a post on hackernews full of insightful comments about something and want to save it. The post might be new, so comments are going to continue to be added (or possibly removed, though this is more of a reddit problem) after I’ve saved the link and page. It’d also be nice to automatically grab the linked webpage for the context. Something that makes this easy would be great.

It might also be nice to be able to select comments (select elements like ublock does?) for highlighting.

Modified30192• 3 years ago

Other things that come to mind as complications.

Saving the page as presented in my current browser session can be vastly different vs a non-logged in guest with no changes from browser addons.

Many websites require browser addons to be tolerable. Reddit likes to hide the end of comment chains to artificially inflate their fucking click metrics, and addons are required to load those comments inline. Saving pages with ublock enabled is also a must. I think selenium can do this: https://stackoverflow.com/questions/52153398/how-can-i-add-a...

So being able to use a login token or auto login with an would be useful. It’s probably best to create a special archive only user for each website. Otherwise it’d be a nightmare trying to remove the elements such as username, favorites, subscribed, etc and make sure the redactions aren’t broken by a future site design update.

hamsterbase• 3 years ago

I suggest trying out HamsterBase (HamsterBase is not open source).

1. It supports direct binding with SingleFile, enabling one-click web page saving. Because it saves in the browser, all other plugins will take effect.

2. It provides an open-source plugin https://github.com/hamsterbase/hamsterbase-highlighter, allowing you to annotate directly in the browser, and it automatically saves a snapshot of the web page when you annotate. When you visit the page again, it automatically displays the previous snapshots.

3. All data is stored on your local device, with both a docker version and a desktop version available. Different versions support P2P synchronization.

4. Provide full-text search function, which can search all the articles on the webpage.

asielen• 3 years ago

In addition to PDF and PNG, died it store searchable text from the page.

If really live a way to do a text search of my bookmarks.

alexktz2• 3 years ago

I wonder. Combining this with ocr in obsidian or something might get you there. It’s not a single tool solution though.

joekrill• 3 years ago

I threw together a quick docker-compose.yml to try this out. For anyone else that wants to spin this up and give it a try locally: https://gist.github.com/joekrill/cc503e21e14f95fefa91acc5f86...

(Though I'm seeing crashes on my Mac M1, which I think is being caused by Prisma)

jerryzh• 3 years ago

I am always curious why don't people use zotero directly for archiving everything

ipiz0618• 3 years ago

This is great! I've always wanted something like this, and even tried launching my own app for this. My app failed to gain traction but happy to see the same idea succeed.

Hiko0• 3 years ago

Is the browser extension coming for Safari as well? And what about an iOS App?

totalconfusion• 3 years ago

Amazing. Collaborative bookmark management needed to happen.

Imagine deciding you want to learn a new skill and not having to start from square one learning the lay of the land around your newly chosen subculture but rather it's already been curated for you and you can hit the ground running with great links to various assets and communities like software tools, forums, discords and video tutorials. I love the archiving functionality too.

Your website looks great and all the information I needed was right there. Concise, informative and neat.

I think the real value here is a tool that will allow people of all kinds to aggregate relevant content and help onboard people to the communities they know, live and love.

I feel like a freemium model would allow this tool to reach a massive scale as I see awesome utility in it but I don't expect normal people would self host or pay for it even if it cost 10 cents. It's just too much inertia.

I feel so dirty telling you how to charge for your own product.

Pagerank and SEO killed search. What you have here sounds a hell of a lot better.

Awesome work mate! I'm aways up for a chat.

Show HN: Linkwarden – An open source collaborative bookmark manager

Related Stories