hamsterbase

Feel free to try out my product: https://hamsterbase.com/

Supports self-hosting via Docker.

https://hamsterbase.com/docs/install/install-with-docker.htm...

There's one more thing. Leader of antv is the developer of echarts.

When it comes to web archiving, I've found that Markdown has some real limitations. Sure, it's great for basic text, but it struggles with things like embedded content and non-standard layouts. Try archiving a Twitter thread or an app-style webpage in Markdown, and you'll see what I mean. It just doesn't capture the full picture.

That's why I've come to prefer formats like webarchive, mhtml, or single HTML files for archiving. They're incredibly faithful to the original content - you get almost perfect rendering of the original page, complete with styling and layout. Plus, they can capture stuff behind paywalls or on logged-in pages, which is a huge plus.

The real challenge, though, isn't just about saving the content. It's about making that saved content useful. These archive formats are great for preservation, but they can quickly become a mess of unorganized files that are hard to search through or make sense of.

I think the key is finding ways to organize and interact with these archives more effectively. Things like full-text search across all your saved pages, the ability to add notes or highlights directly on the archived content, and smart tagging systems could go a long way. And it'd be really powerful if we could integrate these archives with other knowledge management tools we use.

I develop a tool called HamsterBase that seems to address a lot of these issues we've been discussing. t's a local-first app. That means all your data stays on your own device - no need to worry about your personal archives being stored on someone else's servers. There's no sign-up or registration required, which is refreshing in today's cloud-centric world.

When it comes to web archiving, I've found that Markdown has some real limitations. Sure, it's great for basic text, but it struggles with things like embedded content and non-standard layouts. Try archiving a Twitter thread or an app-style webpage in Markdown, and you'll see what I mean. It just doesn't capture the full picture.

That's why I've come to prefer formats like webarchive, mhtml, or single HTML files for archiving. They're incredibly faithful to the original content - you get almost perfect rendering of the original page, complete with styling and layout. Plus, they can capture stuff behind paywalls or on logged-in pages, which is a huge plus.

The real challenge, though, isn't just about saving the content. It's about making that saved content useful. These archive formats are great for preservation, but they can quickly become a mess of unorganized files that are hard to search through or make sense of.

I think the key is finding ways to organize and interact with these archives more effectively. Things like full-text search across all your saved pages, the ability to add notes or highlights directly on the archived content, and smart tagging systems could go a long way. And it'd be really powerful if we could integrate these archives with other knowledge management tools we use.

It's an interesting problem space, and I think there's a lot of room for innovation in how we approach personal web archiving and knowledge management.

You could try hamsterbase. All functions are offline and data is stored locally.

If you need to save all the pages you've seen, you can use singlefile, an open source plugin that works directly with hamsterbase.

Ant Charts was actually developed by the author of echats.

I have a read-it-later tool (https://hamsterbase.com/) that I've been maintaining as a hobby for 2 years, supporting Mac, Linux, Windows, iOS, Android, and self-hosted Docker.

All these versions share the same codebase.

VSCF: I extracted part of the source code from VS Code and developed a framework called VSCF. It includes commands, themes, dependency injection, key bindings, IPC.

https://github.com/hamsterbase/vscf

Local server: On top of VSCF, I developed the underlying business logic, and file IO, SQLite, and the logic related to synchronization will be placed here.

Frontend: On top of VSCF, I used TypeScript and React for front-end development.

nodejs-mobile: This is an open-source project that allows me to use Node.js on mobile devices.

https://github.com/nodejs-mobile

Self-hosted version = local server + frontend, using WebSocket for communication.

Desktop version = Electron + local server + frontend, using Electron's IPC for communication.

Mobile version = nodejs-mobile + self-hosted version. Users use webview to access the UI. It looks just like a native application. They can even use their phones as servers to access data on their phones from a computer.

I also maintained a project at an extremely low cost, other than the $99 for the mac app store and the domain, my other costs were $0.

This is an read-it-later app https://hamsterbase.com/

• Official website deployment: render.com

• Code hosting: GitHub’s private repository

• Issues management: GitHub issue

• Product release: GitHub release , dockerhub

• Email: Free email service from larksuite.com

• Document management: logseq, a free, open-source note-taking software.

• Newsletter: substack

• Design: Figma

I suggest trying out HamsterBase (HamsterBase is not open source).

1. It supports direct binding with SingleFile, enabling one-click web page saving. Because it saves in the browser, all other plugins will take effect.

2. It provides an open-source plugin https://github.com/hamsterbase/hamsterbase-highlighter, allowing you to annotate directly in the browser, and it automatically saves a snapshot of the web page when you annotate. When you visit the page again, it automatically displays the previous snapshots.

3. All data is stored on your local device, with both a docker version and a desktop version available. Different versions support P2P synchronization.

4. Provide full-text search function, which can search all the articles on the webpage.

I made a reader-it-later app。

Even though I have been developing for over a year, I still feel that it's not perfect. Currently, no account system has been developed in the software, only a donation website address is left on the official website. Users can leave their email after paying, and I promise to send them activation codes in the future.

The software price is 60, and the income in the recent 30 days is 300 dollars. The total income is 420 dollars.

In terms of PKM applications, believe CRDT is the future。

In response to this, I've developed a "read later" app based on CRDT. The following are my expectations for knowledge management software, as well as the goals I had in mind when designing this application:

1. I want my data to be stored locally so that I can access it anytime and anywhere, thus eliminating the need for expensive hard drives.

2. I hope the software can be purchased for a one-time fee, rather than by subscription. Since all the data is stored locally, this indicates that there are no additional costs for the developers. Therefore, I believe a one-time payment would suffice. At the same time, I'm open to the game DLC model, where enhanced features can be obtained through additional payments.

3. I hope the software can support synchronization so that I can access the most recent data on any device.

Singlefile supports batch saving URLs to local storage.

Right-click on the webpage > select Singlefile > choose "Batch save URLs".

Using this method, I have converted hundreds of my bookmarks into html files.

I also think that Hamsterbase is not worth $60, so I offered a free trial. Only after it is released on iOS and Android will I consider charging.

https://www.techrepublic.com/article/how-to-archive-websites...

You can take a look at this article.

I will sync web archives to my Mac through iCloud, and then import it in bulk into Hamsterbase.

You can try the desktop version first, and then consider buying it after the mobile version is released. Until iOS and Android are developed, there will be no charge.

Currently hamsterbase provides rss subscription export, you can take any cell phone rss client subscription.

https://hamsterbase.com/developer/rss/

You are right, I added the price link in the footer.

"Thank you for your reminder. May I ask if you are accessing the website on your mobile phone? The pricing page is located at the top of the official website for desktop version. If you are using the mobile version, you will need to click on the hamburger button.

I have updated the wording to: “Free during the beta period, Provide Believer plan, one-time purchase (60$) with lifetime updates.”

https://hamsterbase.com

Most "read it later" services save web pages on their servers, which cannot preserve paywalls, ad-free content, pages that require login or registration, and pages on a local network.

Sharing why I decided to develop my own "read it later" software.

1. I have a habit of saving web pages I like, most of which are in MHTML format, some are saved as single-file HTML, and others are web archives saved on iOS. Altogether, I have accumulated thousands of them.

2. On my computer, I can preview them one by one, but I cannot search through them. So, I developed a Node.js service that parses web pages locally and stores them in an SQLite FTS for full-text search. I deployed the service using Docker on my NAS.

3. To enhance my learning experience, I also developed an annotation feature that allows me to make notes and annotations directly on the offline HTML. For good articles, I save them to read slowly over time.

4. Gradually, the app gained some users. Since they were not familiar with Docker, I wrapped it with Electron and developed a standalone desktop version. The desktop version and Docker version use CRDT for peer-to-peer synchronization.

5. Some users provided feedback that it was inconvenient to annotate after saving, so I developed an open-source browser extension. Users can now annotate web pages directly in the browser, and the annotations and snapshots are saved automatically. When visiting the page again in the future, previous annotations can be restored.

If you want to save articles, you can try using Hamsterbase.

It doesn't collect any information and doesn't require login or registration.

It offers a self-deployed version and provides desktop clients for Windows, Mac, and Linux.

It supports full-text search.

It supports annotating web pages.

There is an open-source and free browser plugin that allows real-time annotation in the browser: https://github.com/hamsterbase/hamsterbase-highlighter.

you can try hamsterbase.com

this tool will index all of your html. Support take highlight, full text search.

I have recorded all the operations of a document. Each operation has an id .

When the document is loaded into memory, it has a unique uuid as the agent id.

If the current version is

hash(a1 b2 c3)

If two people edit the document at the same time the name will become

hash(a1 b2 c3 e4)

hash(a1 b2 c3 d4 d5)

After merging, the document name will become

hash(a1 b2 c3 d4 e4 d5)

a1, b2 is "Lamport timestamp" https://en.wikipedia.org/wiki/Lamport_timestamp

the full version (a1 b2 c3 e4) is “vector clock” https://en.wikipedia.org/wiki/Vector_clock

This solution is suitable for processing documents. For example, figma uses a technique similar to CRDT.

It is not suitable for processing orders.

Couldn't agree more.

I just developed a local-first read later software using CRDT technology. aka

https://hamsterbase.com/

1. all data is stored locally, one page corresponds to one CRDT file. CRDT file is a single source of information

2. use sqlite as cache for query and full text search.

3. use the version of CRDT as the file name. It is guaranteed that there will be no duplicate file names and never file content changes. Only files are deleted and added.

In this architecture

Users could synchronize their data with any intermediate media. Such as hard disk, iCloud, http interface, webdav. It is guaranteed that there will always be no conflicts.

You can try the hamsterbase I developed

1. currently free

2. completely offline, no registration required

3. full text search, tagging, highlighting

You can try hamsterbase.com which I have developed.

1. perfect support for 99% of websites, support import html, mhtml, webarchive.

2. full text search support、provide public api.

3. all functions are completely offline, no account registration, no credit card required, no information collected.

4. currently free in beta

5. support singlefile, singlefile can automatically save all pages viewed.

It requires the user to open terminal ,and then run "xattr -cr /Applications/hamsterbase_desktop.app"

We don't have an Apple developer account yet, so we have this problem. We won't have this problem once we have the mac app store up.

https://hamsterbase.com/docs/importing-website.html#automati...

yes ,hamsterbase supports automatic import of all html in a folder.

They will be compressed by tgz and stored in workspace/objects

Thanks for the tip, I realised I had written the wrong thing on the front page. Currently supports linux,mac,windows

Currently officially released

linux : https://hamsterbase.com/docs/install/install-with-docker.htm... macos: https://hamsterbase.com/docs/install/macos.html windows: https://hamsterbase.com/docs/install/windows.html

Not yet officially released but available

windows desktop : https://github.com/hamsterbase/hamsterbase/releases/tag/0.6.... macos arm desktop : https://github.com/hamsterbase/hamsterbase/releases/tag/0.6.... macos intel desktop :https://github.com/hamsterbase/hamsterbase/releases/tag/0.6.... : https://hamsterbase.com/docs/install/install-with-docker.htm... macos: https://hamsterbase.com/docs/install/macos.html windows: https://hamsterbase.com/docs/install/windows.html

Not yet officially released but available

Download from https://github.com/hamsterbase/hamsterbase/releases/tag/0.6....

windows desktop ,macos arm desktop , macos intel desktop

The desktop version is available, it's just not officially released yet (for the app store), you can download it from github.

https://github.com/hamsterbase/hamsterbase/releases/tag/0.6....

hamsterbase supports singlefile. You can save web pages directly with singlefile. You can see the documentation below

https://hamsterbase.com/docs/integrations/singlefile.html

hamsterbase offers an api with an open source sdk that could perhaps be combined with openai.

https://hamsterbase.com/developer/