Ask HN: Is there any tool for benchmarking responsiveness for Linux?

https://news.ycombinator.com/item?id=30209366

by c0deR3D • 4 years ago

100 44 4 years ago

news.ycombinator.com

System76 recently announced their responsiveness optimizer, System76 Scheduler [0], which basically works as a daemon, adjusting the nice value and the CFS knobs for processes in the box for increased responsiveness. They've claimed that the responsiveness is therefore increased, which I'm also believed so.

However, this got me wondering, is there exists any tool that can report the "numbers" (e.g., scheduling latency) regarding to the responsivenss? Maybe Google has such tool for testing regression for Android or Chrome OS, sadly, I didn't managed to find one.

Thanks!

[0] https://github.com/pop-os/system76-scheduler

ahartmetz2• 4 years ago

For input device to screen measurement, high frame rate camera on your phone. Accuracy isn't the best, but what you measure is as real as it gets.

jrockway2• 4 years ago

To me the question is how do you use that method to measure keyboard responsiveness? On the camera, you can't quite tell when the key was "supposed" to trigger. I've done these measurements by having a keypress toggle an LED on the keyboard (in firmware), but that glosses over the matrix scan (and potentially debouncing if you implement that wrong). All in all, I've never been fully satisfied there. Maybe you need a one-key keyboard that triggers the LED from an interrupt, and then handles the keypress normally. (Only one key, so no matrix. Triggers on the first edge, so no debounce latency.)

Underphil3• 4 years ago

Or use the caps lock key and have that reflected on screen.

ahartmetz2• 4 years ago

AFAIK you'd have to have a pretty bad keyboard for it to have a significant role in latency. I guess specialized hardware (based on a microcontroller with built-in USB host support) wouldn't be too hard to construct if you really needed to know.

For the camera measurement, I would just press the key as fast as possible and pick about 3/4 of the way down as the trigger point.

andi999• 4 years ago

What about fpga which issues a (fake) ps2 keypressed signal and synchronous trigger (led in this case) for the camera?

c0balt• 4 years ago

NVIDIA published LDAT[1] for click-to-photon latency measurements a while ago. The 'Intended' purpose was to measure input latency for gaming but it could probably be reused for measuring responsivity. This would implicitly also measure the hardware but that shouldn't matter when a constant hardware/driver combo is used.

[1] https://www.nvidia.com/en-us/geforce/news/nvidia-reviewer-to...

ziggus• 4 years ago

I believe the question is less about input latency than (as mentioned,) scheduling latency or the amount of time from when the scheduler marks a process as 'ready to execute' to the time it actually starts executing.

SchedLat (https://tanelpoder.com/posts/schedlat-low-tech-script-for-me...) is one tool/approach to doing this, but it's a bit out of date.

hulitu2• 4 years ago

Con Kolivas (maintainer of BFS) had at one time some tools.

hulitu2• 4 years ago

https://github.com/ckolivas/interbench

eloh• 4 years ago

I can recommend this one: https://apps.apple.com/de/app/is-it-snappy/id1219667593

Was mentioned on HN some time ago.

rob_c• 4 years ago

on L I N U X...

Animats• 4 years ago

Tracy? It's mostly used by game devs.

assemblylang2• 4 years ago

IIRC the Zen Linux kernel[0] is designed to be more responsive for desktop usage than the mainline Linux kernel. I wonder what metrics these tools would provide on the same exact system except with the 2 different kernels. (also see Liquorix patch set for additional patches on top of Zen[1])

[0] https://github.com/zen-kernel/zen-kernel

[1] https://liquorix.net/

1_player• 4 years ago

Liquorix isn't the Zen kernel, but it's a set of patches on top of Linux, including Zen among others.

https://github.com/zen-kernel/zen-kernel

assemblylang2• 4 years ago

Thanks for the correction, updated the post

sgc2• 4 years ago

Anybody out there using this? Thinking of trying it on my older thinkpad.

timetraveller26• 4 years ago

I use it on my w530. It feels fast and responsive, but I haven't done a proper test or anything.

One of the good things it's that includes patches that allow Anbox to run so I can use Android apps.

seanw444• 4 years ago

I hear it doesn't yield considerable differences on beefier hardware, but you can tell the difference on older stuff. Thinkpad may benefit.

Note that Liqourix is not the Zen kernel though.

MonaroVXR• 4 years ago

I'm using Pop!_OS, I dont know if I have the update installed or not. For me Pop!_OS is the best distro, since Fedora keeps bugging me out. I'm mostly a RHEL person.

beardog• 4 years ago

I use it on a "just works" Garuda install. No issues for it that i haven't experienced with standard Linux

techietim• 4 years ago

I use Zen on a newer Thinkpad T14, but I feel there is a difference. Applications like Firefox and Intellij feel snappier than with the standard Kernel.

smoldesu• 4 years ago

You're probably looking for sysprof: http://www.sysprof.com/

throwaway8582• 4 years ago

I've used `cyclictest` for this before, it's a simple command line application that measures how long it takes to wake up and run a user space process in response to an interrupt. I've used it in the context of minimizing latency for network and audio applications. I'm not sure how much this relates to desktop application responsiveness as there are additional layers involved.

AtlasBarfed• 4 years ago

That would be some tough things to measure.

How much can you slow down a cpu anyway? Can you slow it to 1/10th its clock speed? I would think the best way to measure things like this is to slow down the overall system speed as much as possible.

timpattinson• 4 years ago

Yeah you can generally underclock a modern intel or amd chip down to 400MHz, but of course other bottlenecks (storage, RAM, networking) will be massively out of proportion

stefan_• 4 years ago

The linux-rt project has a variety of test programs for this:

https://wiki.linuxfoundation.org/realtime/documentation/howt...

But beware, "scheduling latency" has very little to do with responsiveness as it would appear to a user of some desktop environment.

marcodiego• 4 years ago

If the computer is slow enough under load, you can use wall clock to count many seconds it takes to start a program that needs to load many libs and data (LibreOffice, Firefox, Blender, GIMP).

You can also watch FPS of supertuxkart when the system is under load: https://www.youtube.com/watch?v=c5bAOJkX_uc https://github.com/hakavlad/le9-patch

But yes, we need a popular well behaved benchmark to measure desktop responsiveness.

unwind• 4 years ago

I wonder if Ben Heck's "Controller Monitor" [1] would help. It's a dedicated low-level (microcontroller-driven) high-speed input sampling/logging platform, with support for reading a light sensor which I believe is used to sense on-screen changes.

It has the advantage of using game console joypads as inputs, and hacking them to read the buttons etc directly, i.e. without relying on USB. I imagine a specialized version that just sampled a mouse button would be more useful in the context of a computer running desktop apps.

Apologies if I completely misunderstood the intent of this post.

[1] https://www.benheck.com/xbox1monitor/

halz• 4 years ago

Weird, the measurement scale they use does not jive with the scale of the knobs being tweaked. For example, `sched_cfs_bandwidth_slice_us` is adjusted by the project's `bandwidth_size` knob. The default is 5000 us, or 5 ms. But the knob they mention it being '5 us'. Same goes for scheduling latency-- its not `us`, its `ms`.

And afaik, the 'sauce' used in Liquorix is MuQSS, by Con Kolivas. But the project has been put on ice somewhat recently[0].

[0]https://ck-hack.blogspot.com/2021/08/514-and-future-of-muqss...

tanelpoder• 4 years ago

In addition to what others have said:

- eBPF script: https://www.brendangregg.com/blog/2016-10-08/linux-bcc-runql...

- perf sched: https://www.brendangregg.com/blog/2017-03-16/perf-sched.html

- schedlat: https://tanelpoder.com/posts/schedlat-low-tech-script-for-me...

The last one (written by me) is using /proc/PID/schedstat pseudofile for monitoring % time spent in the CPU runqueue of a single process (not systemwide). I don't always have (root) access for running perf or eBPF scripts (or the old Linux/kernel version don't allow that). But I can still measure how much time a process on a request's critical path (like a RDBMS transaction log writer on commit) spent waiting to get back onto CPU before doing any work.

Like others have already said, the OS CPU queuing/scheduling latency is just one component contributing to the total responsiveness/reaction latency of your app.

PaulDavisThe1st• 4 years ago

https://wiki.linuxfoundation.org/realtime/documentation/howt...

This is the goto tool for measuring SCHED_FIFO (realtime) scheduling performance. If you really care about responsiveness in a broad sense (not just user/GUI), this is likely to play some role your testing process.

ahub• 4 years ago

Not sure if it's relevant but I liked the approach of Pavel Fatin in measuring editor latency [0]. It uses OS level input events and actual rendering on screen. So testing the same editor/file with different settings might yield results that are different depending on your scheduling config, while staying close to a "real world" example. He shared [1] the tool he made for the tests.

I'd be curious to know your results regardless of the technique used.

[0] https://pavelfatin.com/typing-with-pleasure/ [1] https://github.com/pavelfatin/typometer

Bancakes• 4 years ago

Consider also compiling a realtime configuration of the kernel with the periodic timer ticks.

jonahbenton• 4 years ago

My major latency hangups are not around happy path input to response in a single app but around memory management across multiple apps including multiple browser hosted apps. Optimizing swap to occur AHEAD of the user's next task switch is an entirely unsolved problem.

Ask HN: Is there any tool for benchmarking responsiveness for Linux?

Related Stories