ZSH Gem #13: Parameter expansion

Posted on 13 Dec at 12:00 AM by Janek Bevendorff | Comments (0) | Trackbacks (0)

Today I want to show you some examples of parameter expansion or parameter substitution as it is called in Bash. I've already used parameter expansion a few times in this series but now I want to show you how it works (at least some basics, I don't want to report the whole manual here).

First of all: what is a shell parameter? A shell parameter is nothing more than an entity which can store one or more values, that is a variable. You assign a value to a shell parameter with the = parameter without any surrounding white space:

myparam=value

That should indeed be nothing new to you. I guess you also know how to access the value of a parameter. For this we have to ${...} operator:

echo ${myparam}

And here you see why some shells call it parameter substitution: the parameter name myparam is substituted with the value which is assigned to it. ZSH calls it parameter expansion because in ZSH terms the name is not substituted but expanded to the value.

${NAME} is the simplest form of parameter expansion and if it's directly followed by a white space character the braces can be omitted. But there is more of course. I wouldn't write an article here about what shell parameters/variables are and how to use them. That's something you can find anywhere on the Internet.

But what makes parameter expansion so powerful? It's the fact that you can use other operators inside the ${...} operator to perform nearly any string operation you like without needing any extra tools. In general I could end here and give you a link to the manual where you find a list of all those parameters, but that wouldn't be very satisfying. So let me advise you of some important ones.

Remove the beginning or end of a string

With the syntax ${NAME#PATTERN} you can remove substrings at the beginning of the value of NAME (or better: replace the value with the matched portion deleted) and with ${NAME%PATTERN} you can delete substrings at the end of the value of NAME.

What does this mean and what does PATTERN stand for? PATTERN is nothing more than some ZSH globbing pattern. So the following parameter expansion expression outputs the value of our parameter with the foo portion deleted:

param=foobar
echo ${param#foo}

But also the following would be possible:

param=foobar
${param#f[[:alnum:]][^b]}

Because we can use globbing patterns there might be more than one possible match. What would be the result if we had the pattern f*o and the string foobar? Would it be obar or bar?

You can specify that by modifying our expansion operator. If you write it in the form ${NAME##PATTERN} instead of ${NAME#PATTERN} (or ${NAME%%PATTERN} instead of ${NAME%PATTERN}, respectively) the longest possible match (i.e. bar) is preferred, otherwise the shortest (i.e. obar).

Search and replace

You can do search and replace operations (again based on ZSH globbing patterns). The expansion syntax is ${NAME/PATTERN/SUBST} for replacing the first match of PATTERN with SUBST and ${NAME//PATTERN/SUBST} for replacing all matching parts (similar to the g flag in most regular expression implementations). With this we can replace the foo part with baz:

echo ${param//foo/baz}

resulting in bazbar. You can also define anchors. If the pattern begins with # the pattern must match the beginning of the string (similar to ^PATTERN in PCRE), if it begins with % it must match the end of the string (similar to PATTERN$). And if it begins with #% it must match the entire string (similar to ^PATTERN$).

Of course you can also use the search and replace syntax instead of the removal syntax I showed you above by specifying an empty substitute string.

Perform word split based on $IFS

Yes, though automatic word split is disabled by default, ZSH can do even that. In the linked article I showed you how to split a string on user-defined delimiter characters, but of course ZSH can also do that based on the contents $IFS and you don't need to enable SH_WORD_SPLIT for that. Just use the following parameter expansion syntax for explicitly splitting words of a string into an array:

param="foo bar baz"
array=(${=param})
echo $array[1]
echo $array[2]
echo $array[2]

Expansion flags

That's it for the basic operators (yes, there are more, please read the manual!). Now let me show you some parameter expansion flags. These flags also perform string operations, but you can combine them to your liking without nesting the expressions. The notation for flags is as follows: the opening brace must directly be followed by an opening parenthesis. Everything up to the next closing parenthesis is interpreted as flags (as many as you want).

Make letters uppercase or lowercase

You can do that with the flags U and L:

echo ${(U)param}
echo ${(L)param}

Split or join strings

Additionally to the splitting operators above you can do some more advanced splitting and joining with flags.

The first splitting and joining flags I want to show you are f and F. With these split strings on newlines (\n) or join them with a newline as delimiter.

param="foo
bar"
array=(${(f)param})
echo $array[1]
echo $array[2]
string=${(F)array}
echo $string

f and F are shorthands for ps:\n: and pj:\n:. These are combinations of the flags p and s:DEL: or p and j:DEL:, respectively. p makes sure that print-like escape sequences are recognized and s:DEL: and j:DEL: split or join strings using the delimiter characters DEL. I used ps:-: in the word split article linked above for splitting a string on dashes. You see: DEL can be any string.

Conclusion

I think I showed you enough to understand how parameter expansion works. Everything else can be looked up in the manual.