HN user

ksri

431 karma
Posts32
Comments64
View on HN
github.com 2mo ago

Show HN: CLI to for working with complex Google Docs using Markdown

ksri
2pts1
github.com 3mo ago

Show HN: Treat Google Docs as Markdown

ksri
2pts2
github.com 4mo ago

Show HN: ExtraSuite – pull / push for Google Drive

ksri
1pts0
github.com 6y ago

JinjaSQL – Template Language for SQL

ksri
3pts0
github.com 6y ago

Show HN: JinjaSQL – Template Language for SQL

ksri
3pts0
www.cypress.io 6y ago

Cypress now supports Firefox and edge

ksri
1pts0
www.confluent.io 7y ago

A Cloud-Native Experience for Apache Kafka in Confluent Cloud

ksri
3pts0
rdbtools.com 7y ago

Show HN: RedisOverflow – Redis and Node.js Playground

ksri
2pts0
rdbtools.com 7y ago

How to Bulk Rename Fields in a Hash in Redis

ksri
4pts0
rdbtools.com 7y ago

Show HN: RDBTools, GUI for redis

ksri
1pts0
rdbtools.com 8y ago

Show HN: RDBTools – Why's Redis Eating Up Memory

ksri
4pts1
rdbtools.com 8y ago

Redis Memory Optimization Cheat Sheet

ksri
2pts0
rdbtools.com 8y ago

Show HN: RDBTools – Redis Memory Analyzer

ksri
9pts0
hashedin.com 9y ago

Designing Modules in Python [pdf]

ksri
2pts0
hashedin.com 9y ago

Designing Modules in Python [E-Book]

ksri
1pts0
blog.2ndquadrant.com 9y ago

Postgres row level security and application users

ksri
1pts0
blog.hashedin.com 9y ago

Elastic Beanstalk Gotchas

ksri
1pts0
blog.hashedin.com 9y ago

5 gotchas with elastic beanstalk

ksri
1pts0
docs.google.com 9y ago

Cheat Sheet: Authentication Techniques

ksri
5pts0
blog.hashedin.com 9y ago

Choosing Right Authentication Technique for REST APIs

ksri
1pts0
github.com 9y ago

Show HN: JinjaSQL – Generate SQL Queries using a Jinja Template

ksri
77pts38
github.com 9y ago

JinjaSQL – Templating Language to Generate SQL Statements

ksri
7pts1
docs.google.com 9y ago

Authentication Techniques for APIs

ksri
2pts0
0x74696d.com 11y ago

Analytics on the Cheap

ksri
78pts27
groups.google.com 12y ago

Redis 3.0.0 beta 3 is out

ksri
2pts0
www.musanim.com 13y ago

Magical Number 7 - Limits on our Capacity to Process Information

ksri
1pts0
raw.github.com 14y ago

Redis 2.6.0 RC1 Released

ksri
77pts14
github.com 14y ago

Show HN : Analyze Redis Memory Usage

ksri
2pts0
github.com 14y ago

Convert Redis dump.rdb files to JSON

ksri
2pts0
github.com 14y ago

Redis RDB File Format Explained

ksri
1pts0

Countless tools exist to create a google doc from markdown. A few tools exist to edit google docs - but they are very limited. Either they replace the entire google doc - losing comments, formatting, images, colours or anything else that you added that markdown doesn't support. Or they are so limited that they only allow find-and-replace, or perhaps limited operations.

The core problem is that google docs' only provides a single API called batchUpdate. This is really painful to use - because the underlying model uses indexes - and it is very painful to keep track of indexes as you update the google docs.

Tools like gws and gogcli technically support batchUpdate. But because the underyling API is so painful - the agent / LLM cannot do anything more than basic updates.

My team uses google docs extensively. And these documents have their own life. Team members leave comments / feedback. Others perhaps apply formatting. Different people contribute to the document - that is the whole point of using google docs. In such a scenario, the agent must be able to play along well - it cannot mess up the document for others.

To solve these problems, I have been working on a CLI tool - ExtraSuite. See https://github.com/think41/extrasuite

The core workflow is just two commands:

`extrasuite docs pull <url> <folder>` downloads the google docs and saves it as markdown files, one per tab in the google doc. It also saves a `comments.xml` - so that the agent can address your comments / feedback.

The agent then edits the markdown files. No special instructions - it knows how to edit markdown files. These can be simple edits, complex rewrites, adding new files, creating tables, adding images or anything that github flavoured markdown allows.

`extrasuite docs push <folder>` - then figures out what the agent changed by comparing locally. This is markdown to markdown diff. Then it figures out how to appy those exact changes to google docs and ultimately reconciles the google doc to match the markdown

ExtraSuite makes a few guarantees:

- It won't mess up formatting, images, headers/footers, styles, colours etc. They will continue to work properly - In general, anything that cannot be represented in markdown won't be touched in the write process.

ExtraSuite has several other features: - Leave comments in the document and ask your agent to work on them. - Full support for tables, images, code blocks, and github flavoured markdown.

If you have either of gws or gogcli installed, you can directly start using extrasuite without any setup.

Our teams have been using this for several months now, and happy to answer any questions about it. Would love your feedback!

It handles it gracefully. The tool cannot edit them, but it will ensure it does not mess them up.

Some examples: - Table of contents can't be updated via the API. But the tool will ensure that the ToC doesn't disappear - Styling can't be represented in markdown. That's fine - new content in the same paragraph will inherit the styles. The tool won't change the styles.

The core approach - don't mess with anything that isn't represented in markdown.

The way it works: we take the original google document and only apply the specific changes that the author intended to make. So this ensures we don't change anything else.

Related. We have several third party web apps in use. These apps don't expose a public api, but they are all single page web apps. We wanted to connect claude code to these web apps for our limited use case.

We opened chrome, navigated the entire website, the downloaded the network tab as an har file. The asked claude to analyze and document the apis as an openapi json. Worked amazing.

Next step - we wrote a small python script. On one side, this script implements stdio mcp. On the other side, it calls the Internal apis exposed by the 3rd party app. Only thing missing is the auth headers..

This is the best part. When claude connects to the mcp, the mcp launches a playwright controlled browser and opens the target web apication. It detects if the user is logged in. Then it extracts the auth credentials using playwright, saves them to a local cache file and closes the browser. Then it accesses the apis directly - no browser needed thereafter.

In about an hour worth of tokens with claude, we get a mcp server that works locally with each users credentials in a fairly reliable manner. We have been able to get this working in otherwise locked down corporate environments.

I am working on a way to edit google docs using markdown. Many tools exist to convert google docs to markdown and to import markdown to google docs - but none of them make in place edits. The core logic is to convert the google docs to md. The user then edits the md. Then diff the markdown files, and apply the changes back to the source google docs. This way, features not represented in markdown do not get overwritten.

Lots of effort has gone into testing against real world docs. Its beta quality right now.

https://github.com/think41/extrasuite

I am working on a declarative CLI for google docs/sheets/slides etc. The general idea is a "pull" command that converts the proprietary document into a local files like tsv or xml. The agent (claude code) then simply edits these files in place and calls "push". The library then figures out the diff and applies only the diff, taking care to preserve formatting and comments.

The hypothesis is that llms are better off getting the "big picture" by reading local files. They can then spend tokens to edit the document as per the business needs rather than spending tokens to figure out how to edit the document.

Another aspect is the security model. Extrasuite assigns a permission-less service account per employee. The agent gets this service account to make API calls. This means the agent only gets access to documents explicitly shared with it, and any changes it makes show up in version history separate from the user's changes.

Early release can be found over here - https://github.com/think41/extrasuite.

re. comparing with other editing agents - actually, I didn't find any that could work with google docs. Many workflows were basically "replace the whole document" - and that was a non-starter.

re. what happens if its a big book - each "tab" in the google doc is a folder with its own document.xml. A top-level index.xml captures the table of contents across tabs. The agent reads index.xml and then decides what else to read. I am now improving this by giving it xpath expressions so it can directly pick the specific sections of interest.

Philosophically, we wanted "declarative" instead of "imperative". Our key design - the agent needs to "think" in terms of the business, and not worry about how to edit the document. We move all the reconcilliation logic in the library, and free the agent from worrying about the google doc. Same approach in other libraries as well.

I have been working on extrasuite (https://github.com/think41/extrasuite). This is like terraform, but for google drive files.

It provides a git like pull/push workflow to edit sheets/docs/slides. `pull` converts the google file into a local folder with agent friendly files. For example, a google sheet becomes a folder with a .tsv, a formula.json and so on. The agent simply edits these files and `push`es the changes. Similarly, a google doc becomes an XML file that is pure content. The agent edits it and calls push - the tool figures out the right batchUpdate API calls to bring the document in sync.

None of the existing tools allow you to edit documents. Invoking batchUpdate directly is error prone and token inefficient. Extrasuite solves these issues.

In addition, Extrasuite also uses a unique service token that is 1:1 mapped to the user. This means that edits show up as "Alice's agent" in google drive version history. This is secure - agents can only access the specific files or folders you explicitly share with the agent.

This is still very much alpha - but we have been using this internally for our 100 member team. Google sheets, docs, forms and app scripts work great - all using the same pull/push metaphor. Google slides needs some work.

Struggling with the same issues with junior developers. I've been asking for an implementation plan and iterating on it. Typical workflow is to commit the implementation plan and review it as part of a pr. It takes 2-3 iterations to get right. Then the developer asks claude code to implement the based on the markdown. I've seen good results with this.

Another thing I do is ask for the claude session log file. The inputs and thought they provided to claude give me a lot more insight than the output of claude. Quite often I am able to correct the thought process when I know how they are thinking. I've found junior developers treat claude like a sms - small ambiguous messages with very little context, hoping it would perform magic. By reviewing the claude session file, I try to fix this superficial prompting behaviour.

And third, I've realized claude works best of the code itself is structured well and has tests, tools to debug and documentation. So I spend more time on tooling so that claude can use these tools to investigate issues, write tests and iterate faster.

Still a far way to go, but this seems promising right now.

SmolLM2 2 years ago

Is there a way to run this in the browser as yet? Transformers js doesn't seem to support this. Is there another way to run this in the browser?

Yes, following relationships is a key feature of Squealy. Here is an example for a recent-questions API - https://github.com/hashedin/squealy/blob/master/squealy-app/....

In short, with 1 GET API call that uses 3 SQL queries, you are getting a list of questions, related answers and related comments.

You should parse that example as follows - 1. The first SQL query fetches all the questions, filtered by API parameters 2. The second SQL query fetches all comments for the questions selected in the first query. See the `WHERE c.postid in {{ questions.id | inclause }}` part - questions.id is coming from the output of the first query 3. The output of the first and second query is merged together on the basis of the questionid that is present in both the query output. 4. Similar approach is used for all the answers

I have been working on an idea to leverage SQL + REST APIs instead of GraphQL to solve the similar problems, and I would love to get some feedback on it.

GraphQL is great during development, but in production you often use "persisted queries" to prevent abuse. Persisted queries are also a way to use HTTP GET instead of POST, and thereby leverage HTTP caching. As such, if you swap out graphql and use sql during the development phase, you perhaps can get similar benefits.

My solution (https://github.com/hashedin/squealy) uses SQL queries to generate the API response. Squealy uses a template language for the sql queries, so you can directly embed where clause from the API parameters. The library internally binds the parameters, and is free from SQL injection.

Handling many-to-many relation, or many one-to-many relations is done in-memory after fetching data from the relational database. This provides the same flexibility as GraphQL, but of course requires you to know SQL.

Here is an example that builds various StackOverflow APIs using just a YAML+SQL based DSL - https://github.com/hashedin/squealy/blob/master/squealy-app/...

Squealy is still work in progress, so don't use it in production. But I would love to get some feedback!

I have been working JinjaSQL[1] for a while to simplify complex SQL Queries.

JinjaSQL is a templating language, so you can use interpolation, conditionals, macros, includes and so on. It automatically identifies bind parameters, so you don't have you to keep track of them.

1. https://github.com /hashedin/jinjasql

Great categorization!

Re. Guide comments - I usually try to make a smaller function with an appropriate name and then call the function. So long function become a series of invocations to smaller functions. This eliminates most guide comments IMO.

Absolutely. First was the Apache commons clause license, and responding to random folks who assumed Redis was going closed source. Then the current issue of changing master slave termibology,and having to hear negative things from both the camps.

Lolwut is the perfect break from madness.

Some background on RDBTools. Over time, more and more data ends up in Redis. It becomes difficult to analyze what is stored within redis, and so most projects end up increasing the memory / switch to a higher redis instance.

RDBTools tries to address that problem. It will analyze a RDB file, identify key patterns, show a breakup of memory by various dimensions including expiry, data type & encoding, key patterns and so on. It will also provide recommendations to reduce memory consumption. For example, it will tell you the right configuration for hash-max-ziplist-* and other small object settings based on your data.

RDBTools is available as a docker image. The analysis is done offline, so there is no impact to your production redis servers.

You'd add that to Squealy.

JinjaSQL simply generates a SQL query with an associated array of parameters. It isn't even aware of the SQL query that's written. JinjaSQL would work just fine with any database.

Squealy is a django project, so if you can get the right database driver configured and installed - in theory, squealy would just work fine with MSSQL. After all, the user is writing the query, so squealy doesn't care much about the underlying differences in the way SQL is written.

Let's assume you have a python object "userquery" that has the user specifications - including dimensions, metrics, filters and the cube to query. I'm also assuming you have whitelisted all the parameters in this object per your database model, otherwise you are open to security issues.

Here's how you could write that in jinjasql. I'm writing this out of memory, and it has errors/issues - but I hope you get the idea.

    SELECT
    {% for metric in userquery.metrics %}
        {{ metric.id }} as '{{metric.description}}'
    {% endfor %}
    FROM {{ userquery.cube }}
    WHERE 
    {% for filter in userquery.filters %}
        {{ filter.key }} {{ filter.operator }} {{ filter.value }}
    {% endfor %}
    GROUP BY 
    {% for dimension in userquery.dimensions %}
        {{ dimension }}
    {% endfor %}
This way, your template doesn't have to map to exactly one visualization.

Now, another interesting this in the security aspect. You can also pass the logged in user data model, and then append a where clause that restricts the rows based on the logged in user id or his role or whatever else is your application security model.

I'm not sure how much effort it would have saved you, but it does help us a lot - mostly because we don't have to translate between SQL and the way SQL ALchemy / ORM works.

Jinjasql is meant for reporting use cases, so yes - it should be able to help you. However, when we made JinjaSQL, we didn't want to build a super-expressive reporting API. Instead, for every report/chart we wanted to make, we wrote a comparatively simpler query and used jinjasql to get it working.

If you have a single complex query that is generating all your reports based on database configuration - I'm guessing you need the full power of python. So while JinjaSQL could have save some effort, I'm not sure it'd save you a lot.

Hope that helps!

Author of jinjasql. The whole point of jinjasql is to dynamically build queries.

There comes a time when you need the power of sql, and an ORM gets in the way. Think unions, sql functions, group by, more complex sql. Think dynamically generated sql queries based on some data structure.

In such cases, you have to keep track of how many variables you are capturing in the query, and then manually bind them.

Jinjasql is just a query generator. At the end of the day, it doesn't actually execute the query. You take the query it generates, and the array of parameters it gives - and then execute it using traditional means using bind parameters.

This approach gives you the power of a template language to generate the query. You can create reusable macros, conditionals and other features that a template language provides.

Author here. The doc does acknowledge basic auth as a valid approach for server side usage, but the language was a bit too pessimistic about basic auth.

I just modified it to say the following

    If you use HTTPS, then basic authentication is a good choice for server side only applications. It is the easiest to get started, and is well supported by clients and server frameworks.

For clients outside of the browser, you have a few choices. JWT token passed in authorization header, or a secure random token passed in authorization header, or a signature that verifies the client has access to a secret.

The easiest way is to create a secure random token and store it in database or in-memory cache like Redis. When a request comes in, you extract the token and check your database/cache if the token is valid. The drawback is an additional network call to the database/cache every API call. If your application can afford this extra call, this is a pretty good approach.

With JWT, you don't have to make an extra network call, because the token itself has everything you need to verify if it's valid. But, if you want the ability to revoke the token - then you will need to maintain a blacklist of tokens somewhere, and check against this blacklist on every request. Arguably, at that point, you lose the original benefit of JWT - the ability to bypass the database/cache.

And finally, if your clients are server side applications only, then you can opt in for signature based approaches.