jasonni

What coding agent do you use with StepFun-3.5-flash? I just tried it from siliconflow's api with opencode. The toolcalling is broken: AI_InvalidResponseDataError: Expected 'function.name' to be a string.

The python MLX version of Parakeet indeed support streaming: https://github.com/senstella/parakeet-mlx It requires modification of the inference algorithm. In this implementation, I see the author even uses a custom metal kernerl to get maximum performance. The Parakeet model batch inference logic is simple. But for streaming, it may require some effort to get the best performance. It's not only the depencency issue.

dots.ocr requires requires a considerable amount of computational resources. If you have Mac device with ARM CPU(M series), you can try my dots.ocr.runner(https://github.com/jason-ni/app.dots.ocr.runner).

There is a pipeline solution with multiple small specific models that can run only with CPU: https://github.com/RapidAI/RapidOCR

I uploaded a picture of handwritten note. Why doesn't it return latex code? Below is the output(source code part) of your website:

二. 毕萨伐尔定律: B=∫dB = ∫(μ₀I dl × eᵣ)/(4πr²)

1. 载流长直导线. [图示：三角形，角度θ₁、θ₂，电流方向向里（叉号）] B = (μ₀I)/(4πr₀) (cosθ₁ - cosθ₂) = (μ₀I)/(4πr₀) [cosθ₁ + cos(π - θ₂)] 无限长直导线: B = μ₀I/(2πr) 半无限长直导线: B = μ₀I/(4πr₀)

推导: dB = (μ₀)/(4π) · (Idz sinθ)/r² B = ∫(μ₀I dz sinθ)/(4πr²) 由于各电流元产生的磁场方向相同 z = r₀ ctg(π - θ) = -r₀ ctgθ dz = r₀ dθ/sin²θ r = r₀/sin(π - θ) = r₀/sinθ B = ∫(μ₀I r₀ dθ sinθ)/(4π r₀²/sin²θ · sin²θ) = ∫(μ₀I sinθ dθ)/(4πr₀) = (μ₀I)/(4πr₀) ∫_{θ₁}^{θ₂} sinθ dθ = (μ₀I)/(4πr₀) (cosθ₁ - cosθ₂) = (μ₀I)/(4πr₀) [cosθ₁ + cos(π - θ₂)]

载流圆线圈在轴线上: 注 若有N匝则 需要乘以N [图示：圆线圈，轴线上点P，半径R，距离x，电流元Idl，磁场方向沿x轴] Bₓ = (μ₀p²I)/[2(p² + x²)^(3/2)] ① x=0, 即电流环中心的磁感应强度: B = μ₀I/(2R) ② x≫a, (p² + x²)^(3/2) ≈ x³, B = (μ₀Ip²)/(2x³) 引入磁矩, \(\vec{m} = I\vec{S} = IS\vec{e}_n\) 对任意形状的平面载流线圈都适 轴线上的磁感应强度, \(\vec{B} = \frac{\mu_0 I p^2 \vec{e}_n}{2x^3} = \frac{\mu_0}{2\pi} \frac{\vec{m}}{x^3}\)

推导: 由图和右手定则可以判断出磁场只有沿x的分量 dB = (μ₀I)/(4πr²) dl cosα = p/r = p/√(p² + x²) Bₓ = (μ₀I cosα)/(4πr²) ∮dl = (μ₀I p)/(4πr² r) · 2πp = (μ₀I p²)/(2r⁴) = (μ₀I p²)/[2(p² + x²)^(3/2)]

I'm working on implementing Nvidia's parakeet tdt ASR model inference in GGML framework. The performance result compared to the MLX python version surprised me. My ggml implementation is 1000x slower than the MLX python version. Any help/comments/suggestions are welcome. THanks a lot!

No one is sure that Transformer model is the final best structure. However, you can still use RTX3090 or RTX2090 to run AI models today, no matter the neural network structure is LTSM/RNN/Transformer. Programmablity and compatibility have more value in some aspect of economic considaration.

I hope in the future, when Chip manufacture cost is no longer the bottleneck of AI, we can have more options.

from their announcement, "Isn’t inference bottlenecked on memory bandwidth, not compute?", it seems weights are still in memory. It may have limit onchip cache for computing. Input tokens go through a batch pipeline to relieve memory bottleneck. Similar to Groq.

In their announcing page, the section "How can we fit so much more FLOPS on our chip than GPUs?" tells some details. It's said "only 3.3% of the transistors on an H100 GPU are used for matrix multiplication". They trade off programmbility with computation density. And from the "Isn’t inference bottlenecked on memory bandwidth, not compute?" section, I guess they use similar tricks like Groq. Looking forward to more architecture details and comparation with Groq.

Last year, when I want to find a tool to do the sanpshot and OCR job, I found flameshot. However, the OCR feature hasn't been added as native function due to some issues I'm not very clear.

So I spent some time added the OCR function into flameshot. I didn't choose to compile tesseract into flameshot, but using the rest api way to call a server running remotely. The reason for this way is I also added llama.cpp translation feature after OCR.

Here're github repositories for my fork of flameshot and the OCR and translation server which is written causually in Rust.

https://github.com/jason-ni/flameshot https://github.com/jason-ni/flameshot-ocr-server

Glad to see a https://virtaitech.com/en/index competitor. As I know VirtAI doen't provide freeware. But they provide RDMA network and GPU pooling features. For guys interested in how this is done, I suggest have a look of https://github.com/ut-osa/gpunet and https://github.com/tkestack/vcuda-controller