We have recently discovered that the Normalization Rules provided by HPE in the scan.gz and snmp.gz files are not being loaded into the Normalization Engine in UCMDB 10.11 after completing the upgrade to CUP 3. This issue was reported by another user and confirmed by HPE and is now visible in QCCR1H99001 as of Monday 3/23/15.
If left unattended, this is serious issue that will negatively impact your your data quality in discovery and integration. This issue has been confirmed both in both our client environments and non-customer environments, and the condition may exist in prior versions and updates.
We formulated the working hypothesis that, since the Normalization Engine is running as expected, the size (270mb and 12mb, respectively) and length (7 million and 370,000 lines, respectively) of the two files are causing the update process to fail. When the new CUP is installed, the probe is instructed to download the new normalization files for use. We believe that either the installation of the normalization files and initialization of the normalization rule base is interrupted as the updater restarts the probe, orthat the installer insufficiently handles the step and continues on its merry way, leaving the package undeployed.
We have validated that the problem does not appear to occur with the UCMDB 10.20 upgrade from 10.11 and may be limited to CUPs. The solution is to simply reinstall the NormalizationRules package. HPE has provided two workarounds in the QCCR, but we find the package manager to be the easiest and safest way to resolve the issue. We provide more background, the test case, and step by step instructions to remedy the problem in your environment below.
The UCMDB Normalization Engine checks incoming discovery or integration CIs for certain empty attributes. If these attributes are defined in the Normalization Rules, the engine will determine the correct rule based on the existing qualifiers, then augment the CI with additional attributes. These additional attributes may include node role, OS family, and even more specific manufacturer information that would otherwise be inconsistent or absent.
The normalization process occurs after each discovery or integration result chunk is processed, before they are sent to UCMDB. Normalization does not act on CI attributes that have an existing value, only those attributes which are qualified and are blank. For example, a Unix CI with a discovered_os_name attribute of “SunOS” will fill the os_family as “unix”, insert “oracle_corp” as as its os_vendor, set the extended_os_family as “sun_os”, and add the node_role of “server” (Fig. 2, below).
While the Normalization Engine-inserted attributes are not necessary for reconciliation in UCMDB, they can serve as helpful qualifiers in TQLs for modeling and reporting and are often vital in integrations between other tools.
Verifying the Problem
While upgrading many of our client and internal systems to 10.11 CUP 3, we noticed that some key data was not being updated by Normalization. We initially added the required rules to the sample.xml (a user-defined normalization xml) in order to override the scan.gz and snmp.gz files; after that change, the Normalization Rules functioned correctly (as evidenced by the previously missing attribute data being supplied).
A few days later we had a conversation with another practitioner, and realized there was a bigger issue at play which potentially affects many users. We scheduled a meeting, and immediately saw via Webex that their Normalization Engine results were taking no more than 1ms to process changes and that 0 changes were being processed, similar to our environment (Fig. 1). So, as in our instances, their engine was running but the rules were absent.
To verify that this was indeed the case, we pulled several rule IDs from the Out of the Box Scan files and checked their presence in the Probe’s JMX console. Below (Fig. 2) is an example of one such normalization rule found in the scan.gz file:
After confirming that the desired rules were indeed present in the package, we confirmed they were not presently available in the Normalization Rule engine on the probe (Fig. 3).
We also tested an integration between both two 10.11 instances, as well as between a 10.11 and 10.20 instance. Our test involved removing attributes that would be affected by the above normalization rule in both the upstream and downstream UCMDB systems. As expected, without the rules being present in the Normalization Engine, the attribute values were not updated (Fig 4).
To restore the functionality of the normalization rules, both primary files (SCAN_FILE and SNMP) must be reloaded by the probe by redeploying the package in UCMDB as follows:
- Login to UCMDB with an administrative account with access to the package manager.
- In the Package Manager, find the NormalizationRules package.
- Select the NormalizationRules package and click the Export package to local directory button.
- Save the package to an easily accessible directory
- In Package Manager, click Deploy Packages to Server (from local disk)
- Navigate to the exported NormalizationRules.zip package and select it
- (Optional) Deselect the samples.xml.
- Click Deploy, and wait for UCMDB to report that it has finished uploading the resources.
- The probe will receive the redeployed files and reload the normalization rule engine (this process may take some time 2-5 minutes depending on speed of your system).
- To confirm that the normalization rules are in place:
a. Log into the Probe JMX console.
b. Navigate to the NormalizationRuleBase mBean.
c. Using the viewNormalizationRuleByNiceId method, search for the ID number of the file suffixed with @SNMP or @SCAN_FILE.
d. If no results are being returned, then the normalization rulebase has not finished processing the file.
Once the normalization file is reloaded, the changes from discovery or integration will be recorded in the destination UCMDB (Fig. 5a):
And data will update properly, so you’ll see the history changes (Fig. 5b):
As you can see now in the CI properties (Fig. 6):
The image below (Fig. 7) is a further example of a properly functioning Normalization Engine and rulebase outcome:
The preceding fix should resolve this issue for UCMDB versions 10.11 and earlier. If you require any assistance, or have any further questions, please don’t hesitate to