PowerShell has a surprising number of different ways of creating objects that have data and methods. Although it is only recently that PowerShell has supported objects directly by means of classes, dynamic modules are a great way of getting the same result. It enables the programmer to create type-constrained data structures that have methods. The problem has been that they don’t seem that easy to understand, and in order to explain them in a way that makes sense, it really needs something convincing and practical by way of an example; what is needed is a demonstration of when a dynamic module would be useful.
One use that seems obvious to me is the stack, also known as the ‘LIFO’ buffer, which is an abstract data structure that is like a stack of plates in a canteen. It has some classic methods, ‘pop()’ where you grab a plate, ‘push()’ where you put a plate on the stack, ‘peek()’ where you see what plate is on the top of the stack and ‘count()’ where you find out how many plates there are. You may need a ‘clear()’ method too. When you’re debugging the workings of a stack, it is nice to be able to see the data.
I use stacks for writing expression analysers. Generally I like at least two stacks, probably up to four. They tend to be different sizes and may have other differences. If written as objects, the code becomes much cleaner and easier to read. Why do I write expression analysers? You might imagine that you would never need such a thing, but once you have one, reasons for using it just keep appearing. They are handy for parsing document-based hierarchical data structures, for parsing grammars and for creating Domain-Specific Languages (DSLs). A DSL is handy when you’re writing a complex application because it cuts down dependencies, and allows you to develop scripts for complex workflows without recompiling .
What I describe here is a cut-down version of what I use, just to illustrate the way of creating and using stacks. I extend this basic algorithm, originally from Dijkstra’s shunting algorithm, into a complete language interpreter. All it needs is a third stack to implement block iterations and ‘while’ loops. Why not use PowerShell itself as your DSL? I’ve tried that, but my experience is that it is pretty-well impossible to eliminate script-injection attacks, and I also find that there isn’t quite enough control.
You’ll notice that the function has dynamic modules that are created as custom objects, one for each stack we need. (the return stack isn’t necessary at this stage, and you may also need an iteration stack too)
1 2 |
$ValueStack = New-Module <code>-ScriptBlock</code> @TheStackCode -argumentlist 40 -name 'ValueStack' -AsCustomObject $FunctionStack = New-Module -ScriptBlock @TheStackCode -argumentlist 20 -name 'FunctionStack' -AsCustomObject |
The initialisation code that we passed as a scriptblock named @TheStackCode is called with whatever arguments you need. In our case we just need to specify the height of the stack that we require: (-argumentlist 20).
The value stack is saved in a variable for use later. You’ll see that each function has become a method (a ScriptMethod) so that this will work …
1 2 3 4 5 6 7 8 9 10 |
$ValueStack.Push('one') $ValueStack.Push('two') $ValueStack.Push('three') $ValueStack.Push('four') $valueStack.TheStack $ValueStack.Pop() $ValueStack.Pop() $ValueStack.Pop() $ValueStack.Pop() $valueStack.TheStack |
The Push() method pushes the string object supplied as a parameter onto the stack and the Pop() method takes each off in turn. The array that is used to store the stack values can be accessed via the object’s TheStack member (a PowerShell NoteProperty) if required for debugging.
The only other part is the initialisation code.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
$theStackcode = { #create the scriptblock containing the data and logic for the stack object. param ([int]$length) $stacklength = $length; $Stackpointer = 0; $TheStack = @($null) * $stacklength #initialise the stack array Function Count { $script:stackpointer } #returns the entries on the stack Function Pop # pop the entry off the stack { if ($Script:Stackpointer -gt 0) { $Script:TheStack[--$Script:Stackpointer]; $Script:TheStack[$Script:Stackpointer] = $null } else { $null } #return a null if there are no entries to pop off the stack }; Function Push ($value) #push the value on to the stack { if ($Script:Stackpointer -lt ($Script:stacklength - 1)) # check for overflow { $Script:TheStack[$Script:Stackpointer++] = $value } #add the value } Function Peek #see what is on the top of the stack without taking it off { if ($Stackpointer -gt 0) { $TheStack[$Stackpointer] } else { $null } #null if there is nothing on the stack } Function Count #see how long the stack is { $TheStack.Length } Export-ModuleMember -Function Pop, Push, Peek, Count Export-ModuleMember -Variable TheStack } |
Yes, a lot of magic goes on under the covers to make it so simple, but you have an object that provides information to intellisense and plays well with the Get-Method cmdlet.
So here is the complete code for the expression analyser.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 |
<# .SYNOPSIS Simple expression analyser .DESCRIPTION This function was developed to test out a simple PowerShell expression analyser that is intended for processing syntax expressions. However, it was easier to test doing numeric expressions .PARAMETER Expression The string expression that should be evaluated. This is a mathematical expression that currently supports just the basic mathematical operation, just to get the .EXAMPLE PS C:\> Evaluate-Expression -Expression 'pi*(23-4)/2^2' .NOTES =========================================================================== Created on: 07-Jul-16 2:37 PM Created by: Phil Factor Organization: Simple-Talk Filename: =========================================================================== #> function Evaluate-Expression { [CmdletBinding()] [OutputType([psobject])] param ( [string]$Expression #the string you want to evaluate ) $ErrorActionPreference = "Stop"; $theStackcode = { #this is the constructor #create the scriptblock containing the data and logic for the stack object. param ([int] $length) #you can specify the length of the stack $stacklength = $length; $Stackpointer = 0; $TheStack = @(0) * $stacklength #initialise the stack array Function Count { $script:stackpointer } #returns the entries on the stack Function Pop # pop the entry off the stack { if ($Script:Stackpointer -gt 0) { $Script:TheStack[--$Script:Stackpointer] } else { $null } #return a null if there are no entries to pop off the stack }; Function Push ($value) #push the value on to the stack { if ($Script:Stackpointer -lt ($Script:stacklength - 1)) # check for overflow { $Script:TheStack[$Script:Stackpointer++] = $value } #add the value } Function Peek #see what is on the top of the stack without taking it off { if ($Stackpointer -gt 0) { $TheStack[$Stackpointer-1] } else { $null } #null if there is nothing on the stack } Export-ModuleMember -Function Pop,Push,Peek,Count Export-ModuleMember -Variable TheStack } #now we create the dynamic module as a custom object. We can add noteproperties, #scriptproperties $ValueStack = New-Module -ScriptBlock @TheStackCode -argumentlist 40 -name 'ValueStack' -AsCustomObject $FunctionStack = New-Module -ScriptBlock @TheStackCode -argumentlist 20 -name 'FunctionStack' -AsCustomObject $Token = @{ '{' = @{ precedence = 0; type = 'structural'; meaning = 'start' }; '*' = @{ precedence = 8; type = 'binary'; meaning = 'mul' }; '%' = @{ precedence = 8; type = 'binary'; meaning = 'mod' }; '/' = @{ precedence = 8; type = 'binary'; meaning = 'div' }; '-' = @{ precedence = 7; type = 'binary'; meaning = 'minus' }; '+' = @{ precedence = 7; type = 'binary'; meaning = 'plus' }; '(' = @{ precedence = 5; type = 'structural'; meaning = 'OpenBracket' };; ')' = @{ precedence = 1; type = 'structural'; meaning = 'CloseBracket' }; '^' = @{ precedence = 9; type = 'binary'; meaning = 'Power' }; '<<' = @{ precedence = 9; type = 'binary'; meaning = 'shl' }; '>>' = @{ precedence = 9; type = 'binary'; meaning = 'shr' }; 'abs' = @{ precedence = 9; type = 'unary'; meaning = 'abs' }; '}' = @{ precedence = 1; type = 'structural'; meaning = 'end' }; } # $FunctionStack.Push('{') #indicate start of expression $expression = $expression + '}' $i = 0; $state = 'begin'; $unexecutable = $false; #$expression.Substring($i) #skip leading spaces #if ($expression -eq $null -or $expression.Length -eq 0) {Throw 'no expression'} While ($i -lt $expression.Length) { #take out any leading space characters if ($expression.Substring($i) -cmatch '(?m)\A\s+') { $i = $i + $matches[0].Length } #first pick up your literal (number in our case because we haven't done strings yet!) #check for numeric value.If the token is a number, then add it to the value stack if (($state -ne 'val') -and ($expression.Substring($i) -cmatch '(?im)^[-+]?(\d+(?:\.?\d*)|\d*\.\d+)')) {# OK. we can add it to the value stack and change state (this copes with unary operators) $ValueStack.Push($matches[0]); $i = $i + $matches[0].Length; $state = 'val' }#we have moved the index $i past the end of the number we've identified. elseif ($expression.Substring($i) -cmatch '^(e)|^(pi)') #do any constants { switch ($matches[0]) #we only do pi and E but you'd also deal with any ones you define { 'e' {$i=$i+1;$ValueStack.Push([math]::E)} 'pi' {$i=$i+2;$ValueStack.Push([math]::PI) } default {$ValueStack.Push($null) } } } #If the token is a function token, then push it onto the stack. elseif ($expression.Substring($i) -cmatch '^(\+)|^(\-)|^(\*)|^(\%)|^(\\)|^(\/)|^(\()|^(\))|^(\})|^(\{)|^(\^)|^(abs)|^(<<)|^(>>)') { $currentToken = $matches[0]; $i = $i + $matches[0].Length; $state = $Token[$currentToken].type; # we have a special treatment for brackets. if ($currentToken -ne '(') { if ($FunctionStack.Peek() -eq $null){throw "missing '(' bracket in expression"} while (($FunctionStack.Peek() -ne $null) -and $Token[$currentToken].Precedence -lt $Token[$FunctionStack.Peek()].Precedence) { #while the precedence of this token is less than the token at the top of the function stack $operator = $FunctionStack.Pop(); #pop the function off the function stack write-verbose "popped $operator after comparing with $currenttoken" $rvalue = $ValueStack.Pop(); #get the top value which becomes the right-side value if ($Token[$operator].type -eq 'binary') { #and get the next from the stack if it is a binary op $lvalue = $ValueStack.Pop(); if ($lvalue -eq $null) { throw "syntax error: too many operators, or missing close-bracket" } } #OK we've bottomed-out a bracketed expression so we can push the value it resolved to if ($operator -eq '(' -and $currentToken -eq ')') { $ValueStack.Push($rvalue); $state = 'val'; break; } if ($Token[$operator].type -eq 'structural') { throw "syntax error: missing bracket while processing '$operator' with '$currentToken'" } if ($rvalue -eq $null) { throw "syntax error: missing value(s)" } # now we can provide the code for operators without direct powerShell-equivalence if ($token[$operator].meaning -eq 'Power') { $repl = "[math]::pow($lvalue , $rvalue)"; $unexecutable = $true } elseif ($token[$operator].meaning -eq 'abs') { $repl = "[math]::abs($rvalue)"; $unexecutable = $true } elseif ($token[$operator].meaning -eq 'shr') { $repl = "$lvalue -shr $rvalue" } elseif ($token[$operator].meaning -eq 'shl') { $repl = "$lvalue -shl $rvalue" } else #we can do it with a simple repl (not in production!) { $repl = "$lvalue $operator $rvalue" }; write-verbose "executing '$repl' at point of '$currentToken' with '$operator' ($state)" $result = Invoke-Expression $repl; $ValueStack.Push($result); #and we just push the value onto the stack if ($currentToken -eq ')') { $state = 'val' }; } } if ($currentToken -ne ')') { write-verbose "pushing $currentToken at $i"; $FunctionStack.Push($currentToken) } } else { throw "unrecognised token at $($expression.Substring($i))" } # } $valueStack.Pop() write-verbose "ValueStack=$($valueStack.count()) Value=$($valueStack.peek()) and FunctionStack=$($functionStack.count()) Function=$($functionStack.peek())" while ($functionStack.count() -gt 0)# just to help debugging. {write-verbose "$($functionStack.pop())"} } |
And I have a simple unit-test on the same page to check out that the basic operations are working (it doesn’t catch all the edge cases!). Why are the simple expressions so much easier to do? This is because wherever I can, I am checking against the same expression in PowerShell. Sometimes the operators are different or don’t exist in PowerShell, which uses the static maths class from .net when it can.
1 2 3 4 5 6 7 8 9 10 11 12 13 |
#and a collection of simple unit tests to catch anything major. if ([math]::Round((evaluate-expression 'pi*4*4'),2) -ne 50.27) { evaluate-expression 'pi*4*4' -verbose } if ((evaluate-expression '1024 >> 2') -ne 256) {evaluate-expression '1024 >> 2' -verbose} if ([math]::Round((evaluate-expression 'e*2'),2) -ne 5.44) { evaluate-expression 'e*2' -verbose } if ((evaluate-expression '2+abs(-345)+8') -ne 355) {evaluate-expression '2+abs(-345)+8' -verbose} if ([math]::Round((evaluate-expression 'pi*(23-4)/2^2'),2) -ne 14.92) { evaluate-expression 'e*2' -verbose } if ((evaluate-expression '3 + 4 * 2 / 1 - 5) ^ 2 ^ 3') -ne 1679616) {evaluate-expression '3 + 4 * 2 / 1 - 5) ^ 2 ^ 3' -verbose} @('(1+2)+3','(23+4)*7','456+(2*3)+4','(456+2*3)+4','456+(2*3+4)','(456+2*3+4)','456+(2)*3+4','-456+-4', '(9*0)+5*6','45*(8+89)','9%5')| foreach-object{if ((evaluate-expression $_) -ne (Invoke-Expression $_)) { "oops! $($_) was calculated as $(evaluate-expression $_ -verbose) but should be $(Invoke-Expression $_)" } } |
Load comments