Awk for multimedia

https://arcan-fe.com/2017/10/05/awk-for-realtime-multimedia/

by poweredbyptrs • 9 years ago

154 24 9 years ago

egwynn• 9 years ago

Cool! Reminds me of GStreamer (https://en.wikipedia.org/wiki/GStreamer). It’s definitely like AWK insofar as it’s pipeline-based. But to me the big thing about AWK is that the program doing the transformation is usually super easy to write down in the CLI itself. Either way, sounds fun!

jcelerier• 9 years ago

But to me the big thing about AWK is that the program doing the transformation is usually super easy to write down in the CLI itself

GStreamer is also very easy to use from the CLI:

    https://gstreamer.freedesktop.org/documentation/tutorials/basic/gstreamer-tools.html

j_s3• 9 years ago

Awesome!

Other video tools (besides ffmpeg):

https://www.mltframework.org/ (lightweight LGPL C99 perhaps similar to OP)

http://www.emmgunn.com/ (mac VIDEOtoolbox), https://subler.org/ (mac mp4)

https://sourceforge.net/projects/megui/ (windows, recommended for SD sources)

https://github.com/stax76/staxrip (windows, a better handbrake)

Further pointers to similar tools are appreciated! Grabbed several of these from this awesome discussion of a weak article on Handbrake: https://news.ycombinator.com/item?id=13492646 (Jan 2017, 59 comments)

PS. Also looking for pros/cons/alternatives re: print LUT http://juanmelara.com.au/print-film-emulation-luts-for-downl... and histogram equalization http://docs.opencv.org/3.1.0/d5/daf/tutorial_py_histogram_eq...

aikinai• 9 years ago

Thanks for sharing all of these tools! Do you happen to know if any of these can write metadata to mp4 or mov files that will be recognized by Mac/iOS Photos?

I've been trying to write location, camera, date, etc. to encoded videos, but Photos won't recognize anything except files straight off an iPhone. The closest thing I've found is that ffmpeg can write Quicktime tags and the structure looks identical to me when I view it in exiftool, but Photos still doesn't respect it, so I assume there's some minute difference that breaks Photos' parsing.

0bit• 9 years ago

I've used Atomic Parsley[1] and MP4Box[2] successfully for various MOV and MP4 related muxing and metadata handling before.

[1] http://atomicparsley.sourceforge.net [2] https://gpac.wp.imt.fr/mp4box/

j_s3• 9 years ago

Unfortunately I do not have any personal experience with your problem since I purposely avoid Apple Photos (perhaps helpful though not related: I rely on Google+ Photos as my free [lossy] offsite backup).

- Date/time? http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,790... recommends https://github.com/HayoBaan/matchDateTime and/or

  exiftool -a -G -s -time:all [filename]

- Lat/long location? works for JPG, only recently for PNG http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,844...

- camera? for Lightroom http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,858...

softwarelimits• 9 years ago

Also take a look at https://github.com/Zulko/moviepy

acj• 9 years ago

Natron is worth a look as well: https://natron.fr

Mac/Linux, Python API, mostly usable docs, and the source is available.

j_s3• 9 years ago

I left this special-purpose tool out on my initial round-up, but anyone still paying attention might find a use for it:

https://github.com/slowmoVideo/slowmoVideo

AdmiralAsshat• 9 years ago

This seems more like pipe-based processing for multimedia than AWK-based. I don't see what AWK does in relation to this that any good UNIX-y utility that is pipe-friendly is doing (grep, sed, etc).

anitil• 9 years ago

I was completely prepared to gush about how much I love AWK. I suppose that can wait for another day

agumonkey2• 9 years ago

more like imagemagick in a way

diggernet• 9 years ago

How does this compare with AviSynth/AvxSynth?

ComodoHacker• 9 years ago

Looks like a good alternative.

jerkstate• 9 years ago

this looks pretty cool, reminiscent of the "shake" compositing application for video

agumonkey2• 9 years ago

haa the glorious shake. Compositing tools were lovely graph and lazy dataflow gems.

sengork• 9 years ago

This reminds me of AWK Music http://kmkeen.com/awk-music/

Bromskloss• 9 years ago

Using the fifth root gives five intervals for each octave, a pentatonic scale. Why five notes? It is really easy to make something that sounds good in the pentatonic scale and really hard for a pair of notes to be dissonant.

Wait a minute! Is the five-note equal-division of the octave really one of those scales that "just sound good"? It's definitely not the same scale as the example he links to.

anamexis• 9 years ago

I tried this out using overtone, and it doesn't sound too terribly dissonant, but I agree, nothing like the major pentatonic scale linked in the video.

(Also, overtone is fun! here's my repl session)

  (require '[clojure.math.numeric-tower :as math])
  (use 'overtone.live)
  (def freqs (map #(* 440 (math/expt 2 (/ % 5))) (range 0 5)))

  (map #(demo (sin-osc %)) freqs)
  (map #(demo (sin-osc %)) (take 2 freqs))
  (map #(demo (sin-osc %)) (take 3 freqs))
  (map #(demo (sin-osc %)) (drop 2 freqs))

boomlinde• 9 years ago

The pentatonic scale does not imply equal temperament between the five notes. The "good-sounding" pentatonic scales use very simple harmonic ratios, for example 1, 9/8, 5/4, 3/2 and 5/3 for the major pentatonic scale. In the equally tempered 12-note octave, these roughly correspond to the first, second, third, fifth and sixth. An equal division of an octave into five notes doesn't yield as harmonic results.

That said, 5-EDO sounds good in my opinion. Not very harmonic, but somewhat resembles the scales used Gamelan music.

navaati• 9 years ago

For those like me that want to hear it on a modern distribution: instead of redirecting to >/dev/dsp, pipe into |pacat. Works like a charm.

golergka• 9 years ago

May be I didn't pay it enough time, but this seems to be just pipe-based video processing. I thought that the main feature in awk was to work with meaning in the text itself - so I expected some machine-learning magic to find all dogs and replace them with cats, for example.

poweredbyptrs• 9 years ago

The pipes are only used to communicate higher-level IPC primitives. The real data exchange and processing is bidirectional. This is why it is possible for the last client in the chain to provide the inputs that control the game that was being run by the first client in the chain. There is also audio.

The part that I found cool is that the possible data sources aren't just videos, it's emulators, virtual machines, terminals, and so on.

"the example here only works with a single pipe-and-filter chain, but there is nothing preventing arbitrary, even dynamic, graphs to be created."

(and AWK is a DSL for streaming text processing - it's from the late 70ies.. it's dumb as a rock - there's no high level analysis features)

defined• 9 years ago

Sorry, but this article has nothing to do with AWK other than mentioning its name. AWK is a pattern-action, implied-loop text processing scripting language that supports user-defined functions and interaction with the OS via pipes and system().

It may have been better titled "Arcan is to multimedia as AWK is to text", or something.

Awk for multimedia

Related Stories