Easy peasy I thought, I just create a new Event Collection rule, pick ‘Enterprise Vault’ as my datasource, and target Windows 2003 Server for the rule, disable it by default and use an override to apply it to my evault computer group. But to my suprise I was getting alerts like this:

Alert description: The Windows Event Log Provider was unable to open the Enterprise Vault event log on computer 'xxxx' for reading.
The provider will retry opening the log every 30 seconds.
Most recent error details: The system cannot find the file specified.

Very strange and after a debug session with procmon (thank god for that tool), I found out that the name ‘Enterprise Vault’ had a space behind it in the registry … so it was actually ‘Enterprise Vault ‘, and the Event Log picker in Opsmgr didn’t care for that extra space at the end. Ofcourse E-vault is also to blame for adding that extra space (nice QA)

So we had to go into the registry editor (HKLM\System\CurrentControlSet\Services\EventLog) and remove that space. And all was well again in the universe .. for now.

So I decided to let the alerts be processed by Nagios, which uses a backend for sms throttling/aggregation and is integrated in our own on-call system.

The setup at the opsmgr side is as follows:

I used NSCA win 32 client for send the alerts to Nagios

For example for the Exhange alerts, I created a Notification command Channel
* channel name: “NSCA Exchange Alerts”
* Full path to file: “c:\send_nsca_win32_bin\send.bat”
* Command line parameters: $Data/Recipients/To/Address/Address$ $Data/Context/DataItem/ResolutionStateName$ $Data/Context/DataItem/AlertName$
* Initial directory: “c:\send_nsca_win32_bin”

This seemingly unnecessary complex batchfile is used because there is some issue in using quotes in the Command line parameters of the Notification channel. And I’m not the only one, it seems.

Ugly hack, but it works for me

Ofcourse scom.domain.com and nagios.domain.com must be changed to your own.

* New notification recipient
* Choose: general
   o Notification recipient display name: "Nagios Exchange Alerts Recipient"
   o [x] always send notifications
* Choose: Notification Devices
* Choose: Add
   o Channel - Notification Channel: "Nagios Exchange Alerts"
   o Delivery address: "exchange"
   * Choose: Next
      o Schedule: [x] Always send notifications
   * Choose: Next
      o General - Notification device name: "Nagios Exchange Alerts Device"

And now you can create a subscription for Exchange alerts where you can add this recipient to.

So what happens, if you have an Exchange alert about LDAP search time, it will be sent like this ‘exchange New: LDAP Search Time – sustained for 5 minutes’

$Data/Recipients/To/Address/Address$ = exchange
$Data/Context/DataItem/ResolutionStateName$ = New
$Data/Context/DataItem/AlertName$ = LDAP Search Time – sustained for 5 minutes

The notification command will sent the alert message to send.bat which will call sendnsca.vbs which will actually start the nsca client which will send the alert to nagios which will process the alert which ultimately send you a sms.

The nagios configuration will be done in another post

A video of these extensions and a distributed application using mysql/apache/php in action can be found on Edge.technet.com

I think that management types will love the uniform reports of all their server regardless of the OS

A whitepaper is available on the new blog about Cross Platform Extensions


$operatorroles=@{"ISA Operators"="DOMAIN\GS.ISA.admins";"Exchange Operators"="DOMAIN\GS.Exchange.admins";"SQL Operators"="DOMAIN\GS.SQL.admins"}

function addOperatorRole {
param([string]$operatorname)
write-host "adding operator: $operatorname"
$mg=(get-item .).ManagementGroup
$operator=$mg.GetMonitoringProfiles() | where {$_.Name -eq "Operator"}
$obj = new-object Microsoft.EnterpriseManagement.Monitoring.Security.MonitoringUserRole
$obj.Name=$operatorname
$obj.DisplayName=$operatorname
$obj.MonitoringProfile = $operator
$mg.InsertMonitoringUserRole($obj)
}

function addOperatorRoles {
write-host "adding Operator Roles"
$operatorroles.getEnumerator() | foreach -process { addOperatorRole $_.Name}
}


First run

select NetbiosComputerName,DNSName from OperationsManager.dbo.MT_Computer where DNSName is NULL

to find the DNSName-less servers. Then to fix it:

update OperationsManager.dbo.MT_Computer set DNSName = 'exchangecluster.test.com' where NetbiosComputerName='exchangecluster'


Another solution is to use the discoveryfix MP from Marius Sutara

A third solution is to wait on Microsoft to release a new and hopefully native instead of converted management pack for Exchange 2007 (SP1)


function Pause ($Message="Press any key to continue...")
{
Write-Host -NoNewLine $Message
$null = $Host.UI.RawUI.ReadKey("NoEcho,IncludeKeyDown")
Write-Host ""
}


function checkMemberDatabase {
#http://suppport.microsoft.com/kb/942865
write-host "checking if memberdatabase needs fixing"
pause
$cn = new-object system.data.SqlClient.SqlConnection("Data Source=SQLSERVER\INSTANCE;Integrated Security=SSPI;Initial Catalog=OperationsManagerDW");
$ds = new-object "System.Data.DataSet" "dsOpsdata"
$q = "SELECT * from MemberDatabase"
$da = new-object "System.Data.SqlClient.SqlDataAdapter" ($q, $cn)
$da.Fill($ds)
$dtOpsdata = new-object "System.Data.DataTable" "dtOpsData"
$dtOpsdata = $ds.Tables[0]
if ([string]::IsNullOrEmpty($dtOpsdata)) {
write-host "upgrading member database - fixing kb92865"
$q="EXEC MemberDatabaseAttach 'SQLSERVER\INSTANCE', 'OperationsManagerDW', 1, 1, 1"
$da = new-object "System.Data.SqlClient.SqlDataAdapter" ($q, $cn)
$da.Fill($ds)
checkMemberDatabase
} else {
write-host "member database ok"
}
}


Don’t forget to replace “SQLSERVER\INSTANCE” (2 times) in your SQL server (and instance) containing the OperationsManagerDW

I’ve scripted the whole installation, SP1 upgrade, and post-installation configuration in powershell and .NET managed code as much as possible, the only things I couldn’t script were the dbcreatewizard and some post-installation configuration like installing agents (problems with permissions).

In a future post I’ll come back about the post-installation scripting.

Also some errors/bugs fixed and new ones introduced, like performance counter bug which -ofcourse- popped up on my server too. But all in all, a must-have release, it almost feels like a finished product now.

I found a solution for this problem on http://support.microsoft.com/kb/942865/en-us

Again very strange that following the installation instructions can cause errors like this.

Error1:

Event Source: OpsMgr SDK Service
Event ID: 26319
Description:
An exception was thrown while processing GetRelationshipTypesByCriteria for session id uuid:17ce6e56-aef7-47e9-810b-47a9a433eaec;id=2.
Exception Message: The creator of this fault did not specify a Reason.
Full Exception: System.ServiceModel.FaultException`1[Microsoft.EnterpriseManagement.Common.UnauthorizedAccessMonitoringException]: The creator of this fault did not specify a Reason. (Fault Detail is equal to Microsoft.EnterpriseManagement.Common.UnauthorizedAccessMonitoringException: The user DOMAIN\scom_datareader does not have sufficient permission to perform the operation.).

Apparently the culprit is the DNS management pack, the fix on
ihaveablog didn’t work for me though. Removing the DNS management pack did.

Error 2

Event Source: Health Service Modules
Event Category: Data Warehouse
Event ID: 31551
Description:
Failed to store data in the Data Warehouse. The operation will be retried.
Exception ‘SqlException’: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.One or more workflows were affected by this.Workflow name: Microsoft.SystemCenter.DataWarehouse.CollectEventData
Instance ID: {61F1F03D-351A-CBA1-377C-84DF6FFAAD89}

This error only pops up occasionally.. to be followed with event id 2115 and id 31554, basically it succeeds to write the data in the datawarehouse after a few minutes. No idea why this happens and no solution found on the interwebs