Please enable JavaScript eh!

 ⌘ Web Mechanic ⌘ 

Bash Scripting

Piping

This is one of the most powerful features of bash (or any shell) programming. So far we've looked at individual commands that do a particular task.

While this can accomplish many tasks you might have on the computer, the bash language has a powerful command-line feature:

OK, that might require a shake of your head, but think about it ...
What if we could send the output of a command directly to another command, instead of doing 2 steps? Or 4 steps?

So far we have only used a command to do a single task. If we needed to do more with that data, we have to write another command to do that.

Piping is how we do that.

Here is a job that I've had to do with my digital music collection. I can get certain data from several types of audio files, including how long the track plays in hours, minutes and seconds.

The tool to do that is ExifTool.

Here is what that data looks like, for Airto Moreira - Homeless:

Duration                        : 0:04:47
Duration                        : 0:04:46
Duration                        : 0:04:39
Duration                        : 0:03:03
Duration                        : 0:05:03
Duration                        : 0:04:16
Duration                        : 0:06:19
Duration                        : 0:06:59
Duration                        : 0:05:58
Duration                        : 0:04:42

The command to do that is:

exiftool *.flac | grep Duration

The symbol | is a vertical bar (AKA pipe), and has special meaning in this case: it sends the output of exiftool to the grep command.

This is what is known as piping one command to another. In this way we can build a pipeline of commands.

This in itself is useful, but in my case I only want the column of actual times, not the whole line.

Another command in the bash toolbox is cut. Unsurprisingly, it cuts specified columns from a string!

exiftool *.flac | grep Duration | cut -c 35-
0:04:47
0:04:46
0:04:39
0:03:03
0:05:03
0:04:16
0:06:19
0:06:59
0:05:58
0:04:42

And if we wanted to sort those by longest to shortest times:

exiftool *.flac | grep Duration | cut -c 35- | sort -r
0:06:59
0:06:19
0:05:58
0:05:03
0:04:47
0:04:46
0:04:42
0:04:39
0:04:16
0:03:03

When we have a line of commands like that, it is called a pipeline.

We used exiftool, grep, cut, and sort to achieve a very useful result in 1 line of code. Now that's K00L.

Resources Used

• ExifTool
  ExifTool is a platform-independent Perl library plus a command-line application for reading, writing and editing meta information in a wide variety of files. ExifTool supports many different metadata formats including EXIF, GPS, IPTC, XMP, JFIF, GeoTIFF, ICC Profile, Photoshop IRB, FlashPix, AFCP and ID3, Lyrics3, as well as the maker notes of many digital cameras by Canon, Casio, DJI, FLIR, FujiFilm, GE, GoPro, HP, JVC/Victor, Kodak, Leaf, Minolta/Konica-Minolta, Motorola, Nikon, Nintendo, Olympus/Epson, Panasonic/Leica, Pentax/Asahi, Phase One, Reconyx, Ricoh, Samsung, Sanyo, Sigma/Foveon and Sony.
• grep
  general regular expression print
  Search for a pattern in a string. Regexes as they are known, are very powerful and can be complex. The expressions are sometimes thought of as their own computer language.
• cut
  The cut command allows users to select specific fields from lines of text, making it useful for processing structured data like CSV (comma-separated values).
• sort
  Sort, merge, or compare all the lines from the files given (or standard input.)

The only tool you have to install is ExifTool. Follow the instructions on the site. The other tools are already available on your Mac.