SSIS 2012 Projects: Deployment Configurations and Monitoring

The Project Deployment Model introduced in SSIS 2012, which was explained in the first part of this series, speeds up the deployment of database projects in which there may be hundreds of SSIS packages per project. Not only that, but deployments can be configured differently for each environments such as test and staging, and there are now ways of monitoring the status and performance of packages and of versioning the SSIS Catalog.

In my first article on Project Deployment in SSIS 2012 ‘Setup, Project Creation and Deployment’, I described the basics of the project deployment model for SSIS. I then showed how to deploy a project to a SQL Server instance.

This article aims to go into more detail about the way of configuring the deployed projects to fit different needs and different environments. It will then go on to describe the ways of monitoring and versioning the SSIS Catalog.

There are two general options for SSIS environments setup:

  • The SSIS projects being deployed to a server which handles SSIS operations and runs the packages towards different environments (TEST and PROD, for example)
  • Or, the SSIS projects can be deployed multiple times – once on every environment (TEST and PROD)

Configuring SSIS projects hosted on the same server for multiple environments

In the previous article, we created a project with an SSIS package. This project had a Project parameter called DatabaseName. The idea behind it was to be able to easily manage the different names of the databases, based on environment (let’s suppose that the database in the TEST environment is called ProjectDeployment_Test and the one in the PROD environment is called ProjectDeployment_Prod).

To achieve our goal in this case, the Project Deployment model provides us with the means of creating Environments – which is a collection of specific settings that map to the project variables.

To create an Environment, we have to connect to our SQL Server instance via SQL Server Management Studio (SSMS) and right-click on the ‘Environments‘ folder in our deployed project:

2095-74628903-309a-45ad-9e5f-95bf6473738

After creating the two environments (Test and Prod), our project will look like this:

2095-978ee684-1527-45aa-a048-803ef7a9fdd

Now we must create a variable in each environment that has the same name as our Project Variable. We double-click each Environment and, in the ‘Variables‘ tab, we enter the Variable name and the value for it. Keep in mind that the Variable name is the same, but the value for it will differ, depending on the environment.

Here is how the Test environment will look:

2095-1-80111907-7871-4197-9522-3d2e6cc59

And here is how the Prod environment will look:

2095-1-93bedb3a-82cb-4351-b29d-f1825de62

Now we have to map the environment variables to our project. Right-click on the deployed project and select ‘Configure…

2095-811175b5-4eb4-4ae5-a66b-785adb1de0b

Then, in the ‘References tab, add both the Test and Prod environments. After adding them, the screen will look like this:

2095-0cf3a777-b959-4acf-a94d-fd50f10473e

Then go to the ‘Parameters‘ tab in the same wizard and add the Parameter value:

2095-c4d83cf4-c452-4232-8784-cf54cdd3a88

In the end, our screen will look like this:

2095-1-1dd3d2d7-afd0-4df4-baa7-ea65db446

Now that we have set up the Environments and mapped them to the project, it is time to execute the SSIS package.

Right-click on the SSIS package and click Execute. Select the appropriate environment and click ‘OK‘.

2095-1-7feb7cbf-e9b0-4e53-bbcb-e6c974c14

This execution is just a single test execution. However, the dialog and configuration is very similar when we schedule SSIS packages via the SQL Server Agent.

In this case, when we have multiple environments configured on the same server, we have to make sure that every time we schedule a package for execution, we have configured the right Environment. I.e. in each SQL Server Agent job we need to make sure that the configuration is correct according to the destination (Test or Prod).

When we make changes to our BIDS project and when we re-deploy, we do not need to worry about any configurations – the SSIS packages in the project are deployed, but the Environments and the configurations are kept as-is (unless we change the names of the project variables).

Configuring SSIS projects hosted on multiple environments

If the SSIS projects are deployed to more than one server, the configuration is a bit different.

Let’s say we have a server called Test and a server called Prod, and we have to deploy the same project to both environments and configure it accordingly.

We could, of course, use the same technique as above, where we deploy our project to each server, and then set up one Environment in each server and point the project variable to it.

We don’t really need to do this, though. Instead, we can just deploy the projects to each server, and then right-click on the project name in the SQL Server instance and click ‘Configure:

2095-77223e8d-a359-4e75-87ff-66fb6691757

Then we just need to enter the name of the project parameter:

2095-1-7533a7cd-4498-4033-b953-fea749402

When we deploy next time, the settings will remain the same as we have configured them in every server, even though in our project the value for the project variable is different.

Monitoring of SSIS Packages in the SSIS Catalog

First of all, when we run a manual test execution of a package in the SSIS Catalog, we are immediately asked whether we would like to navigate to the monitoring page in SSMS:

2095-1-c79f7a7d-3a96-423c-b40a-1efdfa31c

If we click ‘Yes‘, we are automatically taken to the SSMS Report which shows the execution status of the current package.

It looks similar to this:

2095-d10e3041-907b-4826-a253-4a34ae19e95

If we wanted to see what SSIS packages are currently executing on our instance, we would right-click on the SSISDB catalog and select ‘Active Operations.

2095-a9cf8081-5a1d-4824-8896-d2ff0329c52

The dialog would look similar to this:

2095-d59c49a5-cf32-4797-9634-c17b3478363

Unfortunately we do not have the option to click on the currently executing package and go directly to the execution statistics report from this tab.

In order for us to see the package execution statistics and the current execution status, we have to navigate to the reports. One way to do this is to right-click on the SSISDB catalog and select the ‘Integration Services Dashboard report.

2095-0f2f939e-6dbf-47b7-b7e3-fa9f62bf2bd

This report shows an overview of all current and past executions:

2095-a23bda93-14ae-49da-82ca-2e7c21496ef

As we can immediately notice, there is a problem with our package execution. In this case, the failure is due to a validation error in my project. I have forgotten to specify the connection for one of my task components.

This is a great learning opportunity: keep in mind that, in the project deployment model, there may be validation errors and the project can still be deployed even with them. If this happens, the execution of all packages in the project will fail on the server, even though the error is in one of the packages. As I mentioned previously, there is no way to deploy only a single package; we have to deploy the entire project every time we make changes. This is why we have to be extra careful when we deploy with the project deployment.

Performance monitoring

Now that we have our project working successfully and we can execute packages, we are concerned with the performance of our packages.

It is very easy to monitor the performance of an SSIS package on our server. All we have to do is click on the ‘View Performance link in the Package Overview report.

2095-e0b288e6-89dc-446d-ba27-936b9d8e9b1

Then we will see the performance report, which looks similar to this:

2095-d8c722fb-5494-4d88-a414-20fe6243871

As we can see, there has been some improvement in run duration of the package.

Conclusion

In this article I’ve described how to deploy SSIS projects with the project deployment model for use in different environments by means of configuration. I have then described the ways of monitoring and versioning of the SSIS Catalog. The management is almost effortless, since the metadata and SSIS project versions are stored in the SSISDB database within the SQL Server instance.