Putting the “Universal” in Universal Discovery

Putting the “Universal” in Universal Discovery

HPE Universal Discovery/UCMDB discovers a huge range of hardware and operating systems, device and network information from infrastructure. But it doesn’t discover everything, at least not automatically. Expanding its capabilities takes time, knowledge, and a lot of planning and coordination.

We’ve spoken before about Discoverable and Non-Discoverable data. Business Elements  are largely non-discoverable. They may exist in several different tools, spreadsheets, new and old documents, and common understandings within an organization. We typically have to do a combination of methods to gather a comprehensive list of Services; pull software and database name reports after we’ve discovered, review docs, interviews, examine tools and databases, iterate, iterate, iterate. But we’ve managed to map over 2000 applications just to servers for a single customer in less than 12 weeks.

However, there is no best practice across the industry. That’s part of the core issue. How do we bring in new organizations? Their resources, consultants, management and internal customers? How can we honestly discuss where and what a tool is capable of? If we consider HPE or any discovery solution “automatic,” there can be no maturity. It either works or it doesn’t.

So if that’s the case, then no “automated discovery” tool actually works. This same point applies to any other piece of software used in IT today. They all either work or don’t work, and tools are mixed in every environment. This stymies progress.

What if instead, we had a way to collaborate on all ITIL related data, regardless of where or how that ITIL data was created? What if we had a common maturity understanding for how well and how deeply we understand and manage that data?

The essentials are that you need to have as comprehensive discovery of infrastructures as possible. You also need to make sure you have one normalized service model list (these can, of course, be pursued in tandem). But to actually attempt to map an application “as is” state in UCMDB manually is just not feasible. It takes a lot of wasted time and effort, and causes adoption problems due to labor, cost, and complexity.

Within the core concept of ITIL is the Configuration Management System (which replaces the Configuration Management Database that drove the creation of HPE UCMDB and Atrium, etc.). At Effectual we use the HPE UCMDB in combination with a number  of different tools to create our Packed Integration Enhancements (PIE). The PIE solutions (particularly those for ITSM, ITAM, and UD) combine to form an Advanced Configuration Management System (CMS). The CMS allows us to bring together all available sources of data, from HPE and third party tools and collect and normalize things automatically the way one would manually with ITIL.

Third Party Packages and Vertical Mapping for Universal Discovery

There also are a number of third party vendors making great innovations in this department. For instance Leverage Discovery makes packages that take the capabilities of HP UD/UCMDB to a whole new level. Chris from Leverage Discovery is the single person we’ve seen who can perform “automated application discovery” with HP UCMDB. His team has developed what we call vertical discovery packages. They have the skill and experience to build whatever discovery packages your customer and organization may need. You won’t get capability like this ANYWHERE else with HPE UCMDB in as short a time or at as reasonable a cost as Chris uses to write content packs for HPE.

Kevin Keller from Discount Tire, another smart veteran UCMDB practitioner, raised a very strong point in the HPE Discover CMS round table this past June in Las Vegas. To paraphrase, he asserted that each organization can make this process easier by digitally fingerprinting their own deployed software and systems accurately from the beginning.

While that won’t necessarily help on day one of your project, Kevin raised a really important point; we should be discovering, mapping, identifying and then working with Ops and Apps people to tag the CIs and systems that are in the wild. I believe he was also pointing out that the HPE UCMDB/UD tool needs to be better at identifying patterns in order to help this process along.

In this and other discussions, it was also made clear that HPE UCMDB/UD’s primary focus of development is automated modeling and discovery, so we should be seeing some improvements in this area in forthcoming versions.

A Three-Level Model

HPE UCMDB is really nothing more than a giant ITIL-specific data normalizer. So data coming from any number of sources can be mixed and merged to create a better understanding of what the entire enterprise knows. The more data and sources you pull together the more mature that information becomes.

But how do you measure what you have along the way? We use a three level model to address informational maturity:

Level 1 denotes the “presence” of a piece of information or a Node. Level 1 means we know roughly where and what that node is, and that it exists. In HPE UCMDB, this would mean we have connected to the device and know its OS, IP, basic information.

Level 2 indicates we know the “inventory” of that particular piece of information and all relevant contained facts. It means we know what exactly is running on that Node or device.

Level 3 indicates that we know the application internals, configuration specifics and interdependencies.

NOW (stick with us for a minute here)…

So you’re thinking, from a business service perspective, that you can automatically discover all of the software, hardware, and knowledge elements of an actual service. You can’t. No way. Not even if you had better software to do it. Unless some person told you enough about what makes up the system and its parts. There is no discovery software in the Universe that can bridge itty bitty components to one another logically without some input from a non-automated source.

The same L1-L3 model works for other aspects of working with UCMDB data (such as modeling and grouping services, building a service catalog, or monitoring business services in BSM). It’s simple to understand, and simple to apply. It allows you to measure progress without a skeptic shooting you down at every turn because you’re “missing something.”

So we’ve tried to indicate how we measure maturity for an individual device, but the same applies to any other model in the UCMDB. Level 1 – basic info, Level 2 – complete but not completely related info, Level 3 – completely inventoried, understood and related info.

So do you want to search through a giant haystack for a few needles?

We’ll bet your answer is no. Yet this is how most organizations go about their Discovery and Mapping mission. They end up going silo by silo looking for Level 1, 2 and 3 information simultaneously. Too often they go searching for something they can’t find, because they lack mature organization of both approach and results in their UCMDB. They don’t understand that in order to get to Level 3 detail they need to sort ALL the haystacks out first, pulling out every needle along the way.

When you want to build a service catalog or a business service model to use across the enterprise, you need to sort out and remove the haystacks from the equation entirely. This way, you can work with just the small pile of needles.

Our Approach: Horizontal Mapping

Across all of our projects, we refuse to get into the “vertical” portion of discovery prematurely. We prefer instead to take an organizational/global “horizontal” approach where we work to identify and organize everything first.

Where other people might use new CIs which would add to the Universal CI Type Data Model, we use CICollections (groups). We identify every network range, every single dmz/firewall, location and aspect of the physical network. We take input from customers (which is never correct or entirely complete without multiple iterations), we take evidence from the tool. We iterate, iterate, iterate until we have ample evidence that we haven’t missed anything. We build a complete “logical footprint” of the entire company, and all of their dependent networks and locations.

Why? Because without this complete fence-post-to-fence-post understanding we cannot measure what percentage of the world we’ve found with any degree of certainty. The vast majority of the HPE UCMDB customers out there never think this way, because “We don’t want desktops in the UCMDB,” or “We don’t have licenses for so many things.” Issues like these usually limit scope from the beginning. They simply don’t get started the right way.

If you don’t have a 100% understanding of the edges of the world, you can never fully understand what exists within it it. You’ll never be able to have confidence in the information because there will always be a big “BUT” in every conversation you have about the tool.

Having as close to 100% of the Level 1 information about an organization means that we have identified every possible node, every network, even secured ones. We can say that yes, we know where everything IS, which location and country, which city and facility. This can all be organized and automated. The automation and understanding of locations helps with the Level 2 and 3 maturity. Without the Level 1 capability we’re back to our haystack.

The goal before moving on to Level 2 is to have as close to 100% of your Level 1 information as possible. Physical location, virtual and physical IPs, computers and devices and where they live on the network. Even if it’s only 90, or 99%, whatever it is… at least you know what you don’t know.

Once you have Level 1, Level 2 is very simple. You’ll need all this Level 1 information to obtain Level 2 inventory and relationships.You’ve organized your network, domains, regions, locations, etc., so credentials and access are simplified. But you’ll still need to know which user, which organization to request access from, and with whom to coordinate. This process is much easier when you already have a “map of the world.”

You can gather documents, get App names, and query the UCMDB with each piece of information. You can easily see where software is installed, and which databases are used (from either the documents or the UCMDB). With some practice and good data Discovery, you can do an Application model in about 20 minutes because you’ve prepared your work environment and have consistent, good data to work with.

Level 1 can take professionals like Effectual as little as 2-3 weeks to sort out. Then Level 2 can be done in just a couple of days depending on the credential and firewall access. After Level 2 is complete, you can begin the modeling of Services and Applications.

So, assuming you have gathered all the horizontal data you need, if you approach the horizontal Level 1 and 2 maturity, it makes achieving Level 3 maturity (the app internals, relationships, dependencies, and logical app layers that aren’t clearly visible as infrastructure) much easier. Out-of-the-box HPE marketing, and solutions like Neebula, lead you to believe you can skip the foundational work and go straight to the Application view. For modern web-based technologies you may be able to. But not for the entire range of device types, access requirements, and infrastructure components. Not unless you build to the application capability by layering in and building that good Level 1 and 2 foundation.

Vertical vs. Horizontal Application Mapping

Vertical modeling, on the other hand, is typically done from the “top down.” Eager customers who leap into this approach find that vertical models are difficult to achieve with consistency. Some Discovery methods find all relevant bits of information, but not all applications have the same types of CIs and available discovery methods. So this leaves the perception of a gap. Most vertical projects that we’ve witnessed have stalled out because of this perceived gap in Discovery capability. This, in turn, leaves stakeholders displeased and mistrusting of what the tool is actually capable of because too much emphasis is placed on what it is not.

The lowest common denominator in the IT Operations process is getting everyone on the same page as to what service runs on which devices. The only people who are going to get into specifics about what App runs on which application instance is the actual developer. But every other person in the enterprise will use the server name.

So we say that you should build your UCMDB and discovery capability from the ground up (rather than from the top down) before you begin a vertical effort. Ensure that you’ve completed the foundation “left to right.” This method maintains accuracy and delivers the same set of information to every member of the team.

Build as much as 100% of your Level 1 info. Exclude only after you’ve initially discovered everything (you can’t decide what you don’t want unless you already know what you have). Work to have new networks added immediately. Automate. And finally discover every device at least every day (we have made improvements which circumvent the bottlenecks that used to stifle this process).

Then once you’ve got this good foundational set of data, you can go about organizing services, infrastructure groups, business services and applications into CICollections. You’ll know where everything is, what everything is, and who it belongs to. This is the same way that ITAM and ITSM think about their servers and objects in each tool – who, what, when, where, why – at the server and device level.

Horizontal mapping isn’t initially as detailed as vertical mapping, but by ensuring your base is there and organizing the most common elements first, highly detailed vertical mapping can be much more reliable and uniform. Since you already have the service maps, you have the agility to go deeper into building a good application model. It actually becomes easier to get vertical discovery details.

A Building Is Only As Strong As Its Foundation

In short, to succeed with an application map that is conclusive and accurate, you need a good foundation. So we suggest you set out to win from the start. Then you’ll always stay relevant by having the built-in capability to make vertical application maps with the best possible accuracy.

Find 100% left to right. Once you’ve mapped the basic groups and know you have everything you might need to do it, then you can selectively model vertically. When you’ve done this L1-L3 journey without vertical maps, you don’t really need an application-specific model because you can simply go to the service you’re interested in and reveal these facts as they’re needed. Why spend 20 or more hours per application model, when can get the answer you need in 2-3 minutes?

If you’d like to know more, please

Contact Us