PowerShell Day-to-Day SysAdmin Tasks: Events and Monitoring

The automation of routine admin tasks isn't the end of the story. You have to be aware of whether they have succeeded, and how long they've taken. When you have a lot of tasks, you have to consider how you can oversee logs and results from a single console that provides both detail and overview, and warns of problems. Nicolas explains how.

Certainly, it makes sense to automate as many of the routine SysAdmin tasks as you can, but is it possible to get a bit carried away and forget to include the essential feedback that allows us to supervise the work? However much you automate, the administrator needs to keep in control of the processes. Without this feedback information how, for example, do we check all those automated processes to see if a task has failed? How do we analyse the behaviour and performance of our tasks? It is important for us to be able to monitor tasks and respond to events in order to retain firm control of our systems.

To have a hope of doing this, we need to be able to:

  • Read the information in the Microsoft Windows events logs
  • Create our own events within PowerShell scripts
  • Improve the display of events so as to be easily warned of potential problems
  • Trigger actions as soon as a system activity is generated

It makes sense for our processes to use Windows events to alert us to problems and keep track of what tasks are completed on the system, just as the Windows services do. The PowerShell console isn’t sufficient. We must be able to receive alerts or notifications that indicate the result of a task or a PowerShell script.

Reading Information in the Windows Events Logs

There quickly comes a point where our workload is such that it is no longer possible to scan all the logs regularly for untoward events. PowerShell can then assist us to search for events that could spell problems

To do this, there are two PowerShell commands that we can use to interact with Windows logs:

Get-EventLog

I shall not go into details about Get-EventLog because it has become obsolete and is clearly less effective than Get-WinEvent. It was introduced in PowerShell version 1 but has to be mentioned because itremains important if you still work on systems such as Windows 2003 Server. Please note, however, that the new logs (such as “Desired State Configuration” (DCS)) are accessible solely via the Get-WinEvent command.

Get-WinEvent

This allows quicker access to logs. The recovery of events from a remote machine is also quicker. This is easily explained: If you use a filter in your command, then the events will be filtered before rendering results through the network. At last, Get-WinEvent uses the built-in Windows event log remote technology instead of using PowerShell remoting.

If you still have to use the Get-EventLog command over a slow network link, then I’d advise you to avoid any risk of flooding the network by doing the following:

  1. Execute the command on the remote machine
  2. Save the result in a file on that remote machine
  3. Recover this file at a more opportune time

Let’s start by asking the machine to list all the present logs. We will use the listlog parameter followed by * so as to display them all.

2337-figure1_new-1e547e10-4e08-4da2-92da

Figure 1 – List log

The logs are displayed in the order that Get-WinEvent gets them.

Given the large number of results, it is possible to limit the search to only the logs containing events. That is that we exclude all the logs that have no events in them:

When you have identified an interesting log to analyse, you can opt to display just the latest events; a hundred in this case:

So as to be more precise in my analysis, I wish to recover solely the events of the ‘Error’ type contained in the ‘Application’ Log, just fetching the latest three hundred events:

2337-figure2_new-07569e64-d3cf-4b31-93b1

Figure 2 – List events where Level = Error

Quite soon, we’ll probably decide that a better approach would be to specify that we search by filtering the date of the event. Here I display the events of the last twenty-four hours:

So far, so good, but we need to do this on all our servers. To avoid having to connect to each server, the argument -ComputerName allows us to interrogate the logs remotely:

This will have got our process up and running to a point where it is being effective.

From here on, it becomes more interesting. If you have tried the previous commands on your systems, you will certainly have noticed that the result takes longer and longer to be displayed. The problem is that a lot of unnecessary data is being passed down our PowerShell pipeline, only to be filtered out later on by adding the Where-Object condition to the pipeline. This generic syntax is fine where there is no better way of filtering in the source cmdlet, but it slows this process considerably because there is so much data. To filter the data at source, there is a more suitable parameter that is called -FilterHashTable. It is a HashTable: A HashTable is a collection of ‘key = value’ pairs. So I can filter on several values without slowing down my search.

It is easy to show how much quicker this is. Let’s measure the response time of the previous command and then compare it with the same command doing the same thing but integrating this parameter -FilterHashTable:

2337-figure3_new-ec52865b-018f-490d-9de0

Figure 3 – Measure performance of FilterHashTable

You’ll notice that the command is much more rapid during the process of data and is simpler to use. Indeed, I can simultaneously analyse several logs by specifying them with only one command:

Let’s imagine that we wish to obtain all events with these conditions:

  • For both the ‘Warning’ and ‘Error’ levels
  • For The last twenty-four hours
  • For both the ‘application’ and ‘system’ logs

If the search gets much more complex than this, you may need to improve the readability of the results. To do this, I identify each text that enables me to categorize them:

Note: In the above command “l” = label and “e” = expression

2337-figure4_new-db305a6f-aee2-4d12-98ef

Figure 4 – List event with specific ID value

It is much easier to read this information rapidly with the added ‘Category’ field.

Creating One’s Own Events

In a production environment, it is much more convenient to analyse the behaviour of your scripts from within the Windows Event viewer. You need to use this anyway in order to monitor your other services. The advantage is to centralise the information at the same place, and to avoid having to use a text file on the server. For this, I can use an already-existing log or create my own log for a specific requirement. In this example, I will create a log named “Simple-Talk”:

Note: the deletion of a log is done with the command Remove-EventLog -LogName 'Simple-Talk'

The recording of an event is done with the Write-EventLog cmdlet. Here, the source utilised is the one indicated previously when the log was created:

As regards the Event ID, you are free to use that one you wish but it may not exceed 65535.

Lastly, the parameter -EntryType accepts the following values:

  • Verbose (level = 5)
  • Information, (level = 4)
  • Warning (level = 3)
  • Error (level = 2)
  • Critical (level = 1)
  • Undefined (level = 0)

Obviously one must be aware that the events do not overload the server. For this, the maximum size is fixed at 20MB and the new entries will overwrite the older ones like a ring-buffer:

You can, alternatively, use an existing log and create a name for your source process so as to differentiate your scripts:

Then you simply create an event in the ‘system’ log.

Improving the Display Of Events and the Alerts

It isn’t always easy to read events from the PowerShell Console. It is often better to display this in a more readable format. In this example, I use the HTML format to do this. It also enables me to produce more concise and accurate reports for managers.

2337-appview-ede47be3-abc2-4a49-839e-bda

Figure 5 – Display events in HTML format

I receive numerous emails every day at work from automated processes and use the same method in processing the results of my scripts so as to centralise the information. It is important to be able to send information by email. Therefore, to send an email, there are two alternative ways of doing this in PowerShell:

Send-MailMessage

The command Send-MailMessage is very simple to use and works perfectly. You may also send an attachment with a message the following way:

Net.Mail.MailMessage

Sometime, it happens that the mail server “listens” on another port than the standard port 25. Of course, this necessitates more lines of code but it offers more flexibility.

From this, it is relatively simple to implement the sending of an email when an event appears. For this, I associate the script that follows to each event that I wish to supervise.

2337-attach-b82bdf25-1d18-4675-854d-4e34

Figure 6 – Attach task to an event (From event viewer)

From now on, as soon as an event appears with a “level = information”, an email will be sent with the details of that event. On the contrary, no process will be executed. This is only a basic example but it can prove to be very useful when overseeing a particular event efficiently. You may adapt this script to your environment and for all types of events.

Exporting Events

In some cases, you may find that you need to export events so as to allow the processing and analysis of this data by another tool. Let’s imagine that a colleague asks you to export SQL events in a XML format. PowerShell makes the task easy. For this, the Export-CliXml cmdlet saves the data in an XML format:

If you wish to re-import the previously exported file, you just need to use the command Import-CliXml:

Another possibility is to export data in the .CSV format. In my example, I recover all events from the last 7 days, from the ‘application’ log where the level is ‘Critical, Error and Warning’:

At last if you must absolutely export the events in the “evtx” format, you must use the following DOS command:

The processing of these files may be executed via PowerShell with the -Path argument:

To Go Further

Now that we have the basics of Windows events, we can tackle the process of subscribing to events. Let’s go back to the previous example that showed how a PowerShell process could alert us via email when a specific type of event appeared in a log. It is an event based on the result of a script. With PowerShell, it is possible to subscribe to .NET objects so as to be able to respond when an event actually happens. This is much more efficient than polling by scanning event logs for a specific message. By subscribing to an event, a PowerShell script block can be executed whenever an event happens. Not all .NET objects are necessarily compatible with subscriptions. I’ll use, as an example, the System.IO.FileSystemWatcher object that allows us to monitor the activities of a file. This is typically used when a certain type of file is placed in a directory, and a process needs to be kicked off. I will show three parameters:

  • Path: the path of the files to monitor
  • Filter: the types of files to monitor
  • NotifyFilter: the properties of the files to monitor

How would we know which properties to use? Simply use the Get-Member cmdlet. This way you will obtain the events to which one can subscribe, the methods of the object as well as its properties:

2337-figure7_new-a8a4fe6a-6fcb-44c0-bb69

Figure 7 – Get-Member cmdlet

Note: this example is basic as it lets one fully understand the functioning of the subscriptions.

And now, how to find the values accepted by the NotifyFilter property? They are listed here.

When you have found the events that you need, the subscription is done with the Register-ObjectEvent cmdlet:

The above code creates three subscriptions:

  • The first when a file is created (Created)
  • The second when the file is removed (Deleted)
  • And the last when the file is renamed (Renamed)

Note: Ensure that you adapt your subscriptions to the level of your monitoring. Here I subscribe to these three events as I monitor the name of the file that may be created, renamed or deleted. If you monitor, for example, the security changes of a file, then you must subscribe only to event “Changed”.

The parameter -SourceIdentifier is a friendly name that enables the naming of your subscription.

The parameter -Action is a script block that is executed when an activity is detected.

To summarise my explanations, I took the following screenshot and drew two areas:

  • Red area: six available subscriptions
  • Orange area: nine available properties. In my example, I decided to use the “path”, “filter” and “NotifyFilter” properties.

2337-figure8_new-9fb07f4e-3d88-4e18-a517

Figure 8 – Event subscriptions and object properties

There is a PowerShell variable created automatically that is called $Event and contains the details of the event. This variable is scoped to the script block. Here is an example of use:

You may also create a global variable: $Global:MyEvent = $Event so as to reuse it in your PowerShell session.

When the subscriptions are created, it is possible to list them with the following cmdlet:

2337-figure9_new-987db2c7-f516-47be-935c

Figure 9 – List subscriptions

How would you test them? You just need to create a text file in the directory “C:\Temp”, then rename it, and finally delete it. Therefore, you will have triggered your three subscriptions and you will see this in your PowerShell Console:

2337-figure10_new-28c624a2-c4af-49a2-8b6

Figure 10 – Test subscriptions

Two important points to remember:

  • A subscription is only available during a PowerShell session. This means that after having closed your console, the subscriptions will be deleted completely. They will no longer be available.
  • A subscription will create a System.Management.Automation.PSEventJob object. This means that it is possible to list it with the Get-Job cmdlet. Its status is “Running” when at least one activity has been detected:

2337-figure11_new-f69aca47-e5b5-4384-ad8

Figure 11 – Get event job

Conclusion

The main objective of this article was to assist you with monitoring of your tasks and the management of your scripted processes. Events are a good way of following, and tracking, your automated processes. You may also use this method to put in place a simple and relatively cheap auditing system via PowerShell.

As with all my articles, the idea is to illustrate how PowerShell can allow us to simplify our daily tasks. We see here that events subscriptions can become complex, but if you take time to understand this functionality, you’ll find that it has many possibilities, and you will be able to react faster with your various administration tasks.

In this series of articles, I’m trying to show the potential of PowerShell solutions rather than attempting to present you with complete solutions. All these examples are a basis to let you explore the management of events in more detail, so feel free to modify and adapt them to your environment.