A Plethora of PowerShell Pitfalls: Part 2

Comments 0

Share to social media

A Portion of Potential Puzzles

In the previous installment of the Plethora of PowerShell Pitfalls, you had a chance to test your PowerShell acumen against issues with parameter assignments in PowerShell, both direct and across function-calling boundaries. This article rounds out my top ten pitfalls with a few more challenges regarding constructing commands in PowerShell. To reiterate from last time, here are my suggested guidelines for moving forward:

  • First, try to answer the short list of questions without executing the code. Exercise your mental muscles. Give it some thought. Try to figure out the what.
  • Next, go ahead and execute the code to see if you were right. If not, now given the answers, see if you understand the why.
  • Finally, continue reading through the answers and explanations to reveal the what and the why.

The Questions

Try to answer these questions before executing the code.

Question 1

Here is  a function Count-Args that merely returns the count of the number of arguments passed to it…

…so, for example, invoking Count-Args 1 2 "" abc" returns the value 4.

Now assume you have an equivalent executable program ( CountArgs.exe) that performs the same function: What then is the result of executing this statement? (a copy of CountArgs.exe is is provided in a link at the bottom of the article in case you’d like to try this out)

Question 2

Consider a function that outputs the concatenation of its two arguments:

What is the output of this code?

Question 3

You have created the Greetings.psm1 PowerShell module containing only this function:

You import the module (which is in your current directory) and run the function:

Only then do you realize that you meant to say “everyone” instead of “world”, so you open Greetings.psm1, change “world” to “everyone”, save the file, then rerun the same two lines. What is the output now?

Question 4

Define a simple array, then display all its items, and finally display the first two items:

Similarly, for a hash table, define its key/value pairs, and then display the hash table contents:

What is the output when you execute this?

The Answers

As with most endeavors, the more you put in the more you get out, so have a go at answering the above questions before continuing on to the answers. Each exposition below repeats the question but this time with the answer. Challenge yourself yet a second time: see if you can figure out why you get the given answer before reading the subsequent explanation.

Question 1: Beware the Empty String in Positional Parameters

We have a function Count-Args that merely returns the count of the number of arguments passed to it…

…so, for example, invoking Count-Args 1 2 "" abc" returns the value 4.

Now assume you have an equivalent executable program ( CountArgs.exe) that performs the same function: What then is the result of executing this statement?

Answer:

In DOS, some commands are built-in to the shell, like dir and echo, and some are external applications or executables, like ping.exe and ipconfig.exe. In PowerShell, you have three tiers instead of two: cmdlets, scripts, and executables. If you pass-from a DOS shell-an empty string as one of several arguments to an executable, that empty string is, quite naturally, one of those arguments. However, if you do the same from a PowerShell window, that empty string vanishes! That does not sound too good, but wait-it gets worse! While you could rationalize that as a PowerShell difference to be aware of, it is not quite as simple as that. PowerShell is inconsistent: if you pass an empty string to a PowerShell cmdlet or to a PowerShell script, then it keeps that empty string.

Let’s look closer. The versatile PowerShell Community Extensions module (PSCX) contains a collection of supplemental cmdlets to make you more productive in PowerShell, but it also contains a diagnostic utility called echoargs.exe (located in C:\Program Files (x86)\PowerShell Community Extensions\Pscx3\pscx\Apps on my machine once I installed PSCX). Consider this command executed both in DOS and in PowerShell-I have omitted the path to the executable for brevity:

That clearly passes four arguments to the echoargs executable. But examine the results: DOS sees four arguments but the empty string argument is invisible to PowerShell!

DOS

PowerShell

Arg 0 is <word>

Arg 1 is <two words>

Arg 2 is <> # correct

Arg 3 is <123>

Arg 0 is <word>

Arg 1 is <two words>

Arg 2 is <123> # wrong

  The Fix

To make PowerShell behave correctly in this regard, it is necessary to use PowerShell’s verbatim parameter (see section 8.2, Pipeline Statements, in the PowerShell Language Specification ) that effectively makes anything after it on the command line act like a DOS command line.

Author’s note: the below should read PS> echoargs hyphen hyphen percent

PowerShell with verbatim parameter

PS> echoargs –% word “two words” “” 123

Arg 0 is <word>

Arg 1 is <two words>

Arg 2 is <> # correct

Arg 3 is <123>

Thus, using the verbatim parameter is a reasonable fix-when you know you have an empty string on the command line. But what about when you don’t know that you have one? We are back to the original problem-the empty string disappears!

PowerShell with empty string disguised in a variable

PS> $myVariable = “”
PS> echoargs word “two words” $myVariable 123

Arg 0 is <word>

Arg 1 is <two words>

Arg 2 is <123> # wrong

 Unfortunately, the verbatim parameter cannot fix this case, because $myVariable is no longer being parsed by PowerShell:

Author’s note: the below should read PS> echoargs hyphen hyphen percent

PowerShell with empty string disguised in a variable

PS> $myVariable = “”
PS> echoargs –% word “two words” $myVariable 123

Arg 0 is <word>

Arg 1 is <two words>

Arg 2 is <$empty> # wrong

Arg 3 is <123>

 The fix for this case is to use a peculiar quoting trick (kudos to Keith Hill for this):

PowerShell with escaped, quoted empty string disguised in a variable

PS> $myVariable = “”
PS> echoargs word “two words” `”$myVariable`” 123

Arg 0 is <word>

Arg 1 is <two words>

Arg 2 is <> # correct

Arg 3 is <123>

Know Your Arguments
When Passing To an External
Application (a .exe file)

 While ugly, this last technique works both for variables and for literals, and does not constrain the rest of your line to be DOS-like, so it is the best choice for a challenging situation. (For even more on this, see this StackOverflow post.)

Question 2: Beware the String, Too

Consider a function that outputs the concatenation of its two arguments:

What is the output of this code?

Answer:

PowerShell is used by some as a powerful scripting language. By others, it is used as a powerful shell language. Because it is both, however, there are occasions where its convenience as a shell language causes problem when you are scripting. Consider this typical shell command:

 If I typed dir instead of Get-ChildItem, that would also work just fine in a DOS shell; if I used ls, it would also work fine in a Linux shell. But there is something rather peculiar about it if you think from the perspective of a programming language-the string constants (*.tmp and *.txt) are not quoted! That is, many conventional programming languages would require something like this:

 And that is also just fine in PowerShell. Unlike conventional programming languages, quotes around a string constant are often not necessary, to make typing shell commands more convenient. The downside of all this, though, is that if you meant to type a variable (e.g. $stuff) but accidentally omitted the dollar sign, then you might not get any complaint from PowerShell because it will just be taken as an unquoted string constant. Consider:

That will return “two” and “three”. But omit the $ on the last argument like this:

…and there is no error. The return value is quite different though; it returns “one”.

Never Assume Your
 Code Works-
Verify That it Does

The Fix

Aye, there’s the rub. There is no “strict” switch or other automatic checking capability to detect such a situation; you will need to resort to the old-fashioned way: code review, unit tests, etc.

Question 3: Your Code Changes are Ignored

You have created the Greetings.psm1 PowerShell module containing only this function:

You import the module (which is in your current directory) and run the function:

Only then do you realize that you meant to say “everyone” instead of “world”, so you open Greetings.psm1, change “world” to “everyone”, save the file, then rerun the same two lines. What is the output now?

Answer:

Once you start writing your own PowerShell functions, you will soon realize that you need to organize your functions into modules for ease of use and ease of maintenance. So let’s say you have a module called MyGoodStuff. You would load this module for use with the Import-Module command:

 You then proceed to exercise some function contained within that module and realize there is a bug. You correct that bug, save the file, then re-import the module:

 You try your function again, confident your bug has been fixed… but without ever seeing your code, I can guarantee that your function will still fail.

The Fix

The problem is not your code, it is your command. Once you load a module into a given PowerShell session, attempting to reload it normally will not do anything. You either need to unload the module then reload it:

Read the Fine Print
(the documentation)
 on Standard Cmdlets
to Avoid Surprises

 Or you can do it in a single command by adding the -Force parameter:

Question 4: Piping a Hash Table

Define a simple array, then display all its items, and finally display the first two items:

Similarly, for a hash table, define its key/value pairs, and then display the hash table contents:

What is the output when you execute this?

Answer

Many PowerShell cmdlets are designed so that the output of one cmdlet may be used as the input to another cmdlet by using a pipeline. A cmdlet that generates suitable output to do this is simply one that outputs a stream of objects of a single type (or a set of compatible types). Typically, such a stream can be visualized as just an array. This example uses an explicit array of simple integers piped to ForEach-Object:

 …or this:

The output of a suitable cmdlet works the same way-here the output of Get-Process is a stream of System.Diagnosis.Process objects, and that output is piped to Select-Object where it reports just the specified properties from each object:

This convenient and powerful pipelining mechanism runs into a problem, though, when you have another common PowerShell data structure, a hash table. A hash table is essentially a dictionary: you can look up a value by its key. They are easy to create and easy to use. Consider this hash table:

 You could then reference $myHashTable[‘alpha’], or even $myHashTable.alpha, to retrieve the value 1, while $myHashTable.beta returns the value 2, and $myHashTable.gamma returns “a string”.

What seems to follow naturally, then, is that you should be able to do this:

Compare what happens with an array vs. what happens with a hash table… and it looks like it worked just fine. (Note that %{$_} is just an abbreviation for ForEach-Object { $PSItem} )

  Array Array through pipe HashTable HashTable through pipe
Input $myArray $myArray | %{$_} $myHashTable $myHashTable | %{$_}
Output 1 1 Name Value Name Value
2 2
3 3 alpha 1 alpha 1
4 4 beta 2 beta 2
5 5 gamma a string gamma a string

But looks can be deceiving! Let’s take just the first item in the pipeline and see what happens:

  Array in pipe HashTable in pipe
Input $myArray | Select -First 1 $myHashTable | Select -First 1
Output 1 Name Value
alpha 1
beta 2
gamma a string

 The first item of an array is just the first item of an array; no surprises there. But the first item fed through the pipeline with a hash table is the entire hash table! As it turns out, that is by design: a hash table passes through the pipeline as a single object.

The Fix

In order to feed individual key/value pairs from a hash table one at a time, you need to explicitly use the hash table’s enumerator. This will turn the single hash table object into a collection of DictionaryEntry objects. You obtain the enumerator with the hash table’s GetEnumerator method. The next table shows the difference. At left you can see that using the enumerator on the entire hash table yields identical results to what you have already seen-so it is certainly no worse(!)-but is it better? At right, you can see that, yes, when we try to take just the first item, that is exactly what it does!

  HashTable HashTable in pipe
Input $myHashTable.GetEnumerator() | % {$_} $myHashTable.GetEnumerator() | Select -First 1
Output Name Value Name Value
alpha 1 alpha 1
beta 2    
gamma a string    

 Having successfully placed a stream of DictionaryEntry objects in the pipeline, you can now handle hash tables with more dexterity. Say, for example, you want to sort a hash table by its keys in reverse order. Without the enumerator, it fails because, as you now know, it passes the entire hash table as a single object. With the enumerator, we have DictionaryEntry objects, each of which has a Key and a Value property, so we can specify a sort on the Key in a descending order:

  Sort Without Enumerator Sort With Enumerator
Input $myHashTable | Sort-Object -Descending $myHashTable.GetEnumerator() | Sort-Object -Property Key -Descending
Output Name Value Name Value
alpha 1 gamma a string
beta 2 beta 2
gamma a string alpha 1

 Finally, consider the standard filtering/projection operation, performed with Where-Object. Applied to an array, you get a smaller array:

 

Whole Array

Filtered Array

Input

$myArray

$myArray | Where { $_ -gt 3 }

Output

1

2

3

4

5

4

5

 It would be handy if we could similarly filter a hash table and just get a smaller hash table. But since PowerShell requires that the first step is to convert the hash table to a collection of DictionaryEntry object, the challenge is to convert the list of DictionaryEntry objects, after filtering, back to a hash table. PowerShell does not provide any built-in way to do this, though. Jeffrey Hicks posted a useful function to convert general objects to a hash table (Convert PowerShell Object To Hashtable, revised) but for our purposes, this is all you need:

 

Raw Filter

Filter Then Convert to HashTable

2186-img60.gif $result = $myHashTable.GetEnumerator() |           Where-Object Key -eq "gamma" $result.GetType().Name $result = $myHashTable.GetEnumerator() |           Where-Object Key -eq "gamma" |           ConvertTo-HashTable $result.GetType().Name
2186-img5E.gif

DictionaryEntry

…or Object[]

…or error!

Hashtable

When Working
with a Hash Table,
Remember its
Enumerator

 If you have read my previous installment of A Plethora of PowerShell Pitfalls, and specifically the section on what I call The Conundrum of None, One, or Many, you will recall that I explained that the type of object returned depends on the quantity of objects returned. Indeed, at left above the return type could be (depending on your Where-Object predicate) either DictionaryEntry as shown, or it could be Object[], or it could even be an error! Conveniently, the ConvertTo-HashTable function exercised at right above does not care how many objects it gets: it will always return a hash table!

(With thanks to Klaus Graefensteiner in his post “When PowerShell hash table magic backfires” for making me think more about this whole topic.

Conclusion

PowerShell has been around for some time now, but it is still fairly new to a lot of people. Often developers try to glean just enough understanding to get through a given task at hand. But PowerShell offers some unique challenges that could cause problems without a thorough understanding of the mechanics and semantics. I have tried to illuminate some of these here. Besides keeping an eye out for these, try to reduce your potential for subtle errors by letting PowerShell help you: Use PowerShell’s strict mode, use strong typing (which is quite optional in PowerShell!), use modules for encapsulation, and write unit tests (see my article on PowerShell unit tests with Pester here).

Load comments

About the author

Michael Sorens

See Profile

Michael Sorens is passionate about productivity, process, and quality. Besides working at a variety of companies from Fortune 500 firms to Silicon Valley startups, he enjoys spreading the seeds of good design wherever possible, having written over 100 articles, more than a dozen wallcharts, and posted in excess of 200 answers on StackOverflow. You can also find his open source projects on SourceForge and GitHub (notably SqlDiffFramework, a DB comparison tool for heterogeneous systems including SQL Server, Oracle, and MySql). Like what you have read? Connect with Michael on LinkedIn