ZSH Gem #2: Extended globbing and expansion

Posted by | Comments (9) | Trackbacks (0)

You are probably familiar with globbing. Globbing is a way to select files on the command line based on a simple pattern. For instance, to remove all files within a directory, you can use:

rm dir/*

That's basically it. The asterisk says: “match all characters, regardless of how often they appear”. Other globbing operators are ? meaning “match any single character” and [xyz] meaning “match any character within the square brackets” (you can also specify ranges such as [0-9] or [a-z]).

That's about what all shells support. Newer versions of Bash can do a lot more, but I'll show you what the Z Shell can do.

Before we dive right into globbing and expansion, we have to clarify some basics. Unlike Bash, ZSH can expand almost everything. Therefore you won't find much about this topic if you search the manual for globbing. It's all called expansion or filename generation because that's what it is: ZSH globbing is not just a pattern, ZSH globbing is more like a different way of naming things and you can always hit TAB, to expand the expression. For instance, try this on the command line:

% ls dir/*<TAB>

Once you hit the TAB key, ZSH will expand the globbing operator and list all matched files instead. This works with any other globbing operator as well.

That's it for the basics, now let's start with the fun part. I promise, this is much more powerful than normal globbing or even normal regex matching rules.

Note: to be able to use all the extended expansion/globbing features, you need to set the option EXTENDED_GLOB first. So execute:

setopt extended_glob

before doing anything else or better place it in your .zshrc.

Recursive globbing

This is one of the most important things to know and it is probably also the feature, you'd be presented with first, when reading any Z Shell globbing guide. Recursive globbing is the feature, that makes the find command superfluous in most circumstances because with this you can search the current directory including all subdirectories. To match any file foo within the current directory also including subdirectories, run:

ls **/foo

Notice the double asterisk. This will list all files named foo in the current directory tree. If you also want to follow symlinks, use *** instead of **.

Negation

Nearly every globbing pattern can be negated. To match any file not starting with the letters a or b the following pattern can be used:

ls [^ab]*

The negation operator can be used in front of any pattern, not just inside square brackets. For example:

ls ^foo

will find any files except foo and

ls ^(foo|bar)

will find any files except foo and bar. You see: you can also use parentheses to group patterns and use the pipe character to separate alternatives just like in normal regular expressions.

Approximate matching

This is a very useful feature. With approximate matching you find files even if their names contain spelling mistakes such as differing, missing or transposed characters.

ls (#a1)foobar

matches all files with the name foobar but also files with the names fobar, foobra or foxbar. The number after the a defines how far the correction goes. A number of 1 corrects up to one mistake. Higher numbers do more correction. But be careful: the more correction you allow, the more false positives you'll get.

Qualifiers

With qualifiers it is possible to select files by certain attributes and this is where ZSH globbing becomes very powerful. The list of qualifiers is extremely long, so I'll pick the most important ones. Qualifiers are parenthesis expressions at the very end of the whole globbing expression. If the BARE_GLOB_QUAL option is set, you can use bare parentheses or otherwise use (#qfoo) where foo is the qualifier.

First of all there are file type qualifiers. We have . for regular files, / for directories, @ for symbolic links, = for sockets and p for named pipes. Now to list all symbolic links inside the directory foo use

ls foo/*(#q@)

More qualifiers are * for any executable files, r, w and x for owner readable, writable or executable files. These are usually enough, but of course there are lots of other qualifiers, including extended access rights, time and file size selectors. All of these can of course be combined as you like. Definitely have a look at the man page.

Conclusion

That concludes this little introduction. If what you've read here does not sound exciting, then be aware that this is just a very little part of what's possible with ZSH-style globbing. If I'd write about everything you could do with it, it would grow to the size of a book. But I strongly advise you read the manual which I linked below.

To give you one example of how powerful the whole thing is, let me show you one expression to recursively match all normal files which have no uppercase characters or numbers in the name, which are executable for the owner who must have the UID 1002 but not for the rest of the world, have a file size above 30MB and have been modified within the last month:

ls -l **/([^A-Z[:digit:]])##(#q.x^X^u1002Lm+30mM-1)

And if you hit TAB instead of Enter, ZSH will of course expand the whole expression on the command line. Got it? Fine! smile

Read more about extended globbing and expansion:

Trackbacks

No Trackbacks for this entry.

Comments

There have been 9 comments submitted yet. Add one as well!
EJ
EJ wrote on : (permalink)
Absolutely fantastic. Looking forward to the entire series. This is exactly what I need to climb the learning curve away from bash. Keep up the great work!
Warren P
Warren P wrote on : (permalink)
I've been using Linux and ONLY ever bash shells, since the early slackware days and I never understood why people raved about zsh. Now I get it. A bit. Warren
Yan McB
Yan McB wrote on : (permalink)
At a couple of times I've been tempted to try zsh. I might right now, just to play a bit with expanding stuff! Two questions: Is the TAB expansion easily undoable? E.g. by SHIFT+TAB? I guess I'd find that out soon enough.. Can vi commands be used to edit the command line? As in 'set -o vi' in Bash and similar settings in Readline.
Janek Bevendorff
Janek Bevendorff wrote on : (permalink)
Unfortunately, I don't know about any undo command but if you find some in the depths of the manual, let me know. With vi-like editing, do you mean something like "command line editing":http://zsh.sourceforge.net/Intro/intro_10.html ? If that's what you mean, then yes: ZSH can do that. You can emulate vi as well as emacs. I thought about making an article about that in the series as well, but I decided against it.
Chad Walker
Chad Walker wrote on : (permalink)
I don't know about in vi-bindings (I use the OTE: Emacs), but in emacs-bindings, ^_ is undo, and it will undo a completion.
Guillaume
Guillaume wrote on : (permalink)
In the spirit of emacs, one can use Ctrl-g to undo a TAB expansion. This assume the default emacs like binding.
Jordan Stadler
Jordan Stadler wrote on : (permalink)
With vi bindings zsh defaults to insert mode, so after a TAB expansion simply pressing ESC to exit insert mode and then 'u' will undo the expansion. Side note: I just started reading this series and it has been the much needed push to get me to try ZSH. I'm just playing catch up on the articles now and will follow it daily. Thanks for these articles. :D
Janek Bevendorff
Janek Bevendorff wrote on : (permalink)
bq. Side note: I just started reading this series and it has been the much needed push to get me to try ZSH. I’m just playing catch up on the articles now and will follow it daily. Nice to read that. Glad you like it. :-)

Write a comment:

E-Mail addresses will not be displayed and will only be used for E-Mail notifications.

By submitting a comment, you agree to our privacy policy.

Design and Code Copyright © 2010-2024 Janek Bevendorff Content on this site is published under the terms of the GNU Free Documentation License (GFDL). You may redistribute content only in compliance with these terms.