In the Field with our Experts

Real-Time Data Quality Through MDM

By Bob Wall, Senior Consultant

I have found that one of the biggest challenges for data migration, data integration and data warehousing projects that I’ve worked on, is the requirement for clean data. Typically, one of a plethora of data quality/cleansing tools is deployed. The tool employs sophisticated heuristic, probabilistic, deterministic, phonetic, linguistic and empirical methods and algorithms to perform data quality analysis. For example, for customer data being integrated into a data warehouse, we want the tool to reconcile Easthartford, Hartford East, Hartford and East Hartford to the same physical address. The data quality process is usually run in a batch process, first for the initial load and then continually after that.

Today, operational MDM hubs present an even bigger challenge. We have to ensure synchronization across multiple source systems and all data must consistently be correct. The data quality checks must be applied at various stages within the master data lifecycle, and must support federated sharing across and between systems, databases and applications. In a sense operational MDM requires real-time data quality.

Digressing for a moment, I have also worked on projects that employed various search engine technologies. On one in particular, a client wanted to have an automated way to categorize structured and non-structured web data (emails, documents, images, etc.); search for certain conditions and do it in an automated fashion. We investigated some technology that used what was referred to as the semantic web technology, which incorporated advanced semantic and linguistic analysis with classification schemes (using Hyper Text Markup Language (HTML), eXtensible Markup Language (XML), Resource Description Framework (RDF), and Web Ontology Language (OWL)) to render web content machine-readable and make it capable of being searched in an automated fashion.

Software tools that support inline SOA data quality services combined with advanced semantic and linguistic analysis/machine learning capabilities are beginning to evolve. Microsoft’s purchase last year of Zoomix is a testimonial to the strategic value of these types of products. The convergence of data quality and semantic web technologies may provide operational MDM projects with the ability to automate ongoing classification, matching, and standardization of master data records. I think it is definitely worth keeping an eye on to see if it leads to real-time data quality capabilities embedded in MDM tools.

photo by Blude (via Flickr)

---

Bob Wall is a senior consultant with Baseline Consulting. He is an information technology specialist with 30 years experience in all areas of data warehouse administration, data architecture, data resource management, training, and applications systems development, as well as in corporate management.

Triple Play Package for MDM

By Bob Wall, Senior Consultant

Several months ago I decided to upgrade my home internet connection to broadband and discovered that several of the carriers offered a bundled package that allowed me to not only get an upgrade for my internet service, but also for my telephone and television services at the same time. It would offer better speeds, consolidate my bills and most importantly save me money. This bundle is often referred to as a “triple play” deal. So recently I decided to upgrade to a triple play package and I have been extremely pleased with the services and the cost savings.

Baseline offers a similar set of services that we deliver to companies to help out with with their master data management strategy. It’s called the “MDM Masterplan” and it combines a set of MDM specific tasks and skills to deliver an overall MDM roadmap. This can save companies an inordinate amount of time and money, focusing them quickly on their objective and how to reach them. The service is its own type of  “Triple Play for MDM.”  Instead of an internet, telephone and television bundle, the service bundles three core MDM activities:  Business Scoping, Architecture & Technology Review, and Roadmap Development:

Business Scoping is analogous to the internet service in the home triple play package because it allows you to discover the company’s readiness and chart a path—kind of like a Google search!

Through review of business and IT documentation and interviews of key IT executives, MDM sponsors, key business and IT stakeholders, try to understand their concerns for MDM, opportunities for further efficiencies, the organization’s business priorities and challenges, and MDM business requirements.

Architecture & Technology Review is analogous to the telephone service in the home triple play package because the strength of the telephone service is the backbone architecture design and technology network of the service provider.

Through review of the “current state” technology development and processing environment, a gap versus the “desired state” environment, and  identification of the existing methods, practices, and functional capabilities available to support the business capabilities associated with MDM, come up with a blueprint for an MDM architecture and the required technology to support it.

Roadmap Development is analogous to the television service in the home triple play package because you can tune in and plan your future viewing.

Reconcile the business and technical findings to produce a moving forward MDM implementation plan that is based on the company strategy, business priorities, and technical activities necessary to deliver an MDM initiative in an iterative, business-driven approach. 

In summary, this “Triple Play for MDM” can assist a company in preparing for an MDM implementation and ensure that MDM is not only well-planned, but well-packaged and priced for success!

photo by rick (via Flickr)

---

Bob Wall is a senior consultant with Baseline Consulting. He is an information technology specialist with 30 years experience in all areas of data warehouse administration, data architecture, data resource management, training, and applications systems development, as well as in corporate management.

Welcome to the MDM Show, “What’s My Role?”

By Bob Wall, Senior Consultant

Years ago, I regularly watched a television show called “What’s My Line”, moderated by John Charles Daly. The show featured a panel of four celebrities (e.g. Bennet Cerf, Steve Allen, Dorothy Kilgallen and Arlene Francis) who questioned the contestants and tried to discern their occupation. The panelist would begin asking the contestant yes-or-no questions. If they received a “yes” answer they continued, but if they received a “no” questioning passed to the next panelist and $5 was added to the prize tallied on cards Daly would flip over. A contestant won by receiving ten "no" answers (as Daly occasionally noted, "10 flips and they are a flop!") or if time ran out.

If you’ve worked on a Master Data Management (MDM) project, specifically a Customer Data Integration (CDI) effort,  you may think you’re in a remake of “What’s My Line”, but now it is called “What’s My Role.” Because we are trying to integrate customer data from possibly dozens of systems, we need to understand what role the customer played in that system.

For instance, we may designate Jane Doe as a customer. For argument’s sake, a customer defined as a person or organization who has purchased one of our products, but in the shipping system she shows up as ship-to-customer, in the accounts payable system as a bill-to-customer, and in the HR system as an employee.

One way to get around this is to use a generic Party data model (with subtypes of Person or Organization) and to use 2 types of roles for that Party; Declarative and Contextual. For a Party, we might define a “role” for the party, and declare that role as “customer,” the part that an organization or person plays within the enterprise. We then might look at the associative relationship between a source application system and Party and define a contextual role, how a party is or was involved within the context of another entity. Here the roles might be ship-to, billed-to, or employee, but we always establish one instance of the specific party, in this case Jane Doe. A good discussion of using roles can be found in Len Silverston’s “The Data Model Resource Book, Volume 3, Universal Patterns for Data Modeling.”

Using a generic approach to roles for your MDM/CDI project may help you to integrate these roles and not wind up like panelists on “What’s My Role,” where you could face 10 flips and you’re out!

---

Bob Wall is a senior consultant with Baseline Consulting. He is an information technology specialist with 30 years experience in all areas of data warehouse administration, data architecture, data resource management, training, and applications systems development, as well as in corporate management.

Ready, Shoot, Aim: Gaining Control of Your Data

By Carol Newcomb, Senior Consultant

You’re already familiar with the common symptoms: rapid heart rate, shallow breathing, sweaty palms and jumpy behavior.  When you are asked to identify the strategy for information management in your organization, your mind draws a blank.  “So, I’m not the CIO.  Go ask him!”  When asked about the most reliable source for master data on customers or contracts or sales force, you mumble, “Try the ERP system.  That’s where I found it last time.”  Or if an outside contractor were to ask about your 5-year plan for information management and BI strategy, you shrug, puzzled.  “What good is a 5-year plan when I don’t even have a 1-year plan?  Leave me alone.”  Treading water is the mode of survival.

The single most common problem organizations face these days is too much data.  It comes from all directions for all sorts of purposes.  It comes from dozens of applications designed to fulfill specific functions.  It has grown over the years until reluctantly, one of the organization’s leaders says “Enough already!”   Data is now costing more to manage than it is contributing to informed decisions.  Data is not an asset, it’s a liability!  How did this happen?  It’s like a teenager’s fantasy: wanting and having, wanting and having, bright shiny objects, more and more, until the sobering realization sinks in that the dream is no longer just a dream—it’s a full-fledged nightmare! 

Information management is all about control.  Stop.  Step back.  Get some perspective.  Assess the true situation. Itemize what’s good and what’s subpar.  Think about what the organization needs to achieve.  Think about how the data supports those objectives, not how the organization manages despite its data.  Data management is a deliberate, stepwise set of practices designed to ensure that value can be extracted from the data, and that the overhead associated with poor quality, inconsistent definitions, unaligned source systems and discontinuous data management practices are orchestrated to work as a whole.  The objective is to deliver information value.

So how do you go about changing from ‘Ready, Shoot, Aim’ to ‘Ready, Aim Shoot’?  First, get ready.  Admit that survival is not a strategy.  Survival is survival, and it is getting more difficult to maintain your organization’s survival, much less its competitive edge.  Next, invest the time and resources in drawing that big ugly spaghetti diagram, an inventory of ALL the source applications, software systems, ETL points, staging systems, landing points and delivery channels.  Then, take a true inventory of the end-users.  This is critical, because this is where data begins to result in business value.  Take time to really research who does what with certain pieces of data.  Do they further augment it with external data?  Do they merge it with other internal or external data?  Do they modify it to match other data sources?  How much time do they spend cleaning, standardizing or massaging data?  Start to build your ROI equation.  Data Sources X Time X People = Business Information Cost.  How much is this really costing?  Where is the value?

As you learn more about how the business uses data, start to lay out a business value matrix (the other side of the ROI equation).  This will help prioritize your data management roadmap.  Estimate the amount of effort required to integrate each source against the perceived business value, and chart a business value/implementation effort matrix.  Consider drafting more than one, as this reflects perceptions more than actual fixed units.  Share this information with end users to validate it and socialize a plan.  Then start to lay out a tactical roadmap.  Aim!

Now you’re getting somewhere!  Having faced the ugly reality (Ready) and incorporated real-life facts from business end-users (Aim), you can start to craft your strategy to streamline and rationalize the multiple data lifecycles supported by your organization.   Shoot!  Prioritize the data sources and systems to begin with, and estimate the expected value.  Launch your governance, metadata, quality and delivery resources, and engage your business users each step of the way.  Deliver value and repeat.  Evaluate the process, refine, and keep going.  It’s a deliberate, engaged, transparent and proactive approach, and it’s the only way you will ever conquer those adolescent nightmares!

photo by Erik Charlton (via Flickr)

---

Carol Newcomb is a Senior Consultant with Baseline Consulting. She specializes in developing BI and data governance programs to drive competitive advantage and fact-based decision making. Carol has consulted for a variety of health care organizations, including Rush Health Associates, Kaiser Permanente, OSF Healthcare, the Blue Cross Blue Shield Association and more. While working at the Joint Commission and Northwestern Memorial Hospital, she designed and conducted scientific research projects and contributed to statistical analyses.

Get on Board the MDM Express

By Bob Wall, Senior Consultant

Imagine if you will you're the CIO of a mid-market pharmaceutical company and you've just been to a TDWI conference that had a session (“Introduction to MDM for BI Professionals”) that provides an overview of master data management. You’re heading back to your corporate headquarters by train. You feel convinced that like this train, MDM is speeding along gathering momentum, and you don’t want to be left at the platform and miss the opportunity to get your company on the MDM Express!

You may have heard that MDM offers a single version of the truth, enterprise-wide, about customers, products, and other information types. As CIO, if you look around at your company’s issues you realize right off the bat that customer data integration is a real issue. You think to yourself that it would be nice to be able to take an MDM strategy and implement it to consolidate, integrate and synchronize all of the customer data so that it would be more accurate and it would enable your sales and marketing teams to better leverage the market value of your customer relationships. However, you're faced with the daunting fact that MDM is a very complex concept—and potentially an expensive one. Moreover, many MDM vendors price their products such that they’re out of reach for the small to medium size company.

How do you get started?  Here are four proven starting-points for MDM:

1. Find the business sponsor. I know it’s trite and overused, but since you’re the CIO it’s likely you know where the pain is among your C-level colleagues. Enlist the CFO whose numbers don’t reconcile across billing and invoicing systems, or the CMO who struggles with a growing number of customer lists.
2. Engage a readiness assessment. You can do this with internal or external resources, but do it. Gauging readiness has little to do with whether people know what MDM is and more to do with your incumbent skills, data management practices, toolsets, and needs.
3. Define a Small, Controlled Project. Work with the sponsor and cut a slice out of the overall problem that you can use to not only demonstrate the value of MDM, but establish the technology foundation.
4. Build the team. You can’t just “assign” MDM to your BI or CRM team. You need to construct a new team of data-savvy practitioners who will own enterprise master data for the long-term.

Sure, there are other deliberate steps. Maybe you want to do an educational workshop to get everyone on the same page, or talk to some incumbent vendors about their offerings. But these tactics and others could backfire if they are ill-timed. Better to think globally but act locally, and reap the rewards.

photo by jurvetson (via Flickr)

---

Bob Wall is a senior consultant with Baseline Consulting. He is an information technology specialist with 30 years experience in all areas of data warehouse administration, data architecture, data resource management, training, and applications systems development, as well as in corporate management.

Public Sector takes SaaS for a Spin

By Rob Paller, Consultant

There has been a lot of talk about the growth and potential impact of Software as a Service (SaaS) in the realms of business intelligence, MDM, and to some extent data warehousing on Twitter and numerous industry blogs.  The premise behind SaaS is to move the hardware and software from your data center into the cloud. By doing so you reduce the costs associated with deployment and maintenance of the hardware and software. Much like Google has done with your email, documents, and photo libraries.

Google is turns out to be a great example because of their partnership with Panorama Software. Panorama Software is helping lead the charge by providing one of the first OLAP services in the cloud. Using Google Docs and Panorama’s SaaS solutions allow you to create pivot tables or take your cubes from SQL Server and move them into the cloud for collaborative analysis.  However, that has nothing to do with what is going on in the public sector and is best saved for another discussion.

Recently, one of our clients in the public sector has taken the concept of SaaS and applied it across the agencies and departments for many of the same reasons a small or medium sized business would consider looking to the cloud. Our client has established a group within its IT department whose purpose is to share services at an enterprise level to help drive down costs by reducing multiple contracts with the same vendors and standardize on the solutions offered. The solutions offered by this group range from data cleansing, data integration, business intelligence and data warehousing.  Going forward, this group is also working toward establishing a master data management solution as they begin to address the requirements to provide electronic medical records. A substantial benefit in applying the SaaS concept across the departments and agencies is found in the reduced cost in training staff that transfer from one department or agency to another and reduce time to delivery of new solutions. 

While the traditional SaaS concept may work great for the small and medium sized business the fact of the matter is that large corporations and even the public sector is less likely to make the jump into SaaS right now.  Do you think the spin being applied to SaaS concept at this client could work for large companies or other areas of the public sector? Let me know in the comments below or on Twitter.

photo by David Accampo (via Flickr)

---

Rob Paller is an expert at business analytics and database administration. Since joining Baseline, Rob has been responsible for developing a case analysis system to streamline the oversight of food assistance benefits, implementing a common citizen data model, and assisting in the rollout of a new public assistance data model integrating data from over 10 years of legacy with a new benefit eligibility determination system.

The Project Manager as News Reporter

By Stephen Putman, Senior Consultant

photo by rabbleradio

I recently read a post on the PM Blog relating to Data Integration Management Processes. In the post, the author details the seven steps in the Project Management Book of Knowledge (PMBOK) Guide relating to a Data Integration project for completion of a project. The steps are pretty much what you would expect-- develop a project charter and scope, develop the project plan, implement change control, etc.-- for any project, not just Data Integration. I found one of the steps particularly interesting for what it left out:

4. Direct and Manage Project Execution The Project Manager needs to make sure that everybody is doing what they should be doing...(emphasis mine)

The italicized statement is a huge assumption-- it supposes that the manager and team knows what tasks are required, how they are related, how long they should take, and why they are required. In other words, it supposes that the project manager can answer the famous "six questions" that all reporters ask about a news story. Unfortunately, many times this is a faulty assumption, since project managers do not necessarily have experience in data integration solutions, and may not have access to this expertise.

At Baseline, we answer these questions every day:

• Who - Who should be involved in each step of the implementation process
• What - What should be done to successfully accomplish the project, based on vast experience in developing and implementing industry best practices
• When - What sequence in which the tasks should be performed, and how they are related to each other
• How - The skills required to accomplish the tasks, and the ability to identify these skills in client staff
• Where - Assumed to be the client site (the least important question)
• Why - How detailed tasks relate to a proven project implementation methodology that has been developed by Baseline

The simple act of answering these six questions puts you well on the road to successful project completion. Of course, that?s not all there is to it, but it?s a good start?one that a surprising number of experienced project managers forget!

---

Stephen Putman has over 20 years experience supporting client/server and internet-based operations from small offices to major corporations.  He has extensive experience in a variety of front-end development tools, as well as relational database design and administration, and is extremely effective in project management and leadership roles.

Business Intelligence: Finding the Edges

By Rob Paller, Consultant

photo by H. Michael Karshis

When was the last time you did or witnessed something creative? Did you say to yourself, “That’s what happens when you think outside the box!”? Contrary to the cliché, creativity isn’t found by thinking outside the box but from finding the edges and stretching them. Over the past couple years the BI industry has gone through its share of mergers and acquisitions. Many of the big players have been absorbed as the consuming companies tried to round out their offerings and becoming the data equivalent of a Walmart supercenter.

While the big players fight to find room inside the box, start-ups and open source players (OS) are left to find ways to remain on the radars of prospective customers who are otherwise looking for stability in a vendor during a time where VC money is getting harder to come by and the distinction of being open source is losing its sheen. This leaves the start-ups and OS vendors the opportunity to find the edges of the box. That is where these vendors must find new ways to distinguish themselves.

There is a fair amount of discussion going on among the BI tribe regarding their BI predictions in 2009 including how business intelligence can help a company become greener, the impact that SaaS will have on BI, and defining BI 2.0. While these are all great ideas on their own, what is seemingly being lost is the ability for a BI environment to remain nimble once deployed allowing new features to be deployed sooner and enabling knowledge users to stretch the edges of BI within their companies. Toss all of those vendor contracts, licensing costs, and maintenance agreements into in a box for now. (And I don’t mean the recycling container under your desk!) Instead let’s walk around the edge for a minute and see if there is potential to push the boundaries.

Consider Ubuntu, an open source operating system, with its six month release cycle. Initially released in 2004, Ubuntu has adhered to this cycle while also offering a long-term support version providing a stable foundation for business making a long term investment in Ubuntu. Imagine the potential for a BI solution that delivers new features every six months fostering those who thrive on creativity. Is this pushing the edge too far?

---

Rob Paller is an expert at business analytics and database administration. Since joining Baseline, Rob has been responsible for developing a case analysis system to streamline the oversight of food assistance benefits, implementing a common citizen data model, and assisting in the rollout of a new public assistance data model integrating data from over 10 years of legacy with a new benefit eligibility determination system.

Disruptive Technologies: Governance in Action

By Carol Newcomb, Senior Consultant

photo by roland

I recently consulted on a Customer Data Governance program implementation project.  I must say, most companies get to the point where they design to death what governance should look like, how it should operate, who should be on which committees and who should have specific decision rights.  But how often have you ever seen data governance work well?  What does a governance policy even LOOK like?
Yeah, yeah, governance is the “4 pillars of People, Processes, Decisions Rights and Controls”, or it’s the “5 boxes of Alignment, Oversight, Measurability, Visibility, and Transparency”, but come on—what does it FEEL like?  How does it really WORK??  What makes it happen??  How can people understand it when they are told it is so important?  Who does what?

Face it.  Governance is disruptive.  We speak of disruptive technologies, like the cell phone or FaceBook, the Hybrid automobile, or the laptop computer.  These were all disruptive in their day.  They changed the way we think about social networks, distances between people and getting around.  Governance is disruptive because it seizes control from the hands (silos) of data analysts, IT staff and report writers, and puts it squarely into the laps of business people who (unknowingly) need the data to make accurate and timely business decisions.  Oh, and did we mention—share the data with each other?  Just think of the power of sharing trusted data and how much more efficient and effective the business would become with strong dependencies instead of weak, redundant independencies.

“Don’t put those decisions in MY LAP.”  We hear this all the time.  Why not?  It’s your bread and butter.  Stand up and be accountable for the fact that you don’t trust the data, but you still report it up the chain of command.  Show that you’re willing to expose who made what data decision, when, and how broad the impacts on actually are.  Be proud of the data that you use to run your portion of the business.    This is uncomfortable stuff.  Trust?  You want me to TRUST the data you lob over the fence? 
Well, let me think about that and I’ll get back to you.

Governance is disruptive because it breaks down the weaknesses we have that permeate most corporations, no matter what industry you’re in.  It forces people to talk to each other.  It forces people whom we call Data Stewards, to beat the bushes, get people to admit what they’ve been doing and how they’ve been doing it.  It wrests the decisions out of the IT cubicle and puts it squarely on the Director’s desk.  Data quality?  Let me show you our data quality.  Ouch.  Metadata?  How do you spell that?

So, get used to the fact that all change is disruptive.  But, the changes that are REALLY disruptive are those that break down the social and cultural barriers (aka personal barriers) that corporations have allowed to take control of their data, weakened its use as a decision-making tool, and contributed to massive cash outlays in data correction, cleaning and rework just to manipulate data so that it can serve its ultimate purpose: to measure the effectiveness of decisions made and to guide more profitable decisions in the future.  It’s that simple.

---

Carol Newcomb is a Senior Consultant with Baseline Consulting. She specializes in developing BI and data governance programs to drive competitive advantage and fact-based decision making. Carol has consulted for a variety of health care organizations, including Rush Health Associates, Kaiser Permanente, OSF Healthcare, the Blue Cross Blue Shield Association and more. While working at the Joint Commission and Northwestern Memorial Hospital, she designed and conducted scientific research projects and contributed to statistical analyses.

What’s Your (MDM) Type?

By Bill O'Kane, Senior Consultant

 

photo by Dan4th

As part of an interview recently given by Evan Levy, he was asked to explain the various flavors of MDM implementation. The three generally accepted categories are persistent (all master attributes stored and maintained exclusively in the data hub for access by client systems), registry (sets of master attributes stored and maintained in client systems are registered, or “pointed to” by the data hub), and hybrid (also sometimes referred to as co-existent, where a copy of some or all of the client systems’ master attributes are stored and maintained both in the data hub and the client systems). While each MDM implementation is inherently different due to the uniqueness of each business and operating environment, these categories sufficiently encompass the vast majority.

I’ve come to realize that all of the projects I’ve been involved in have been of a single type: they began life as persistent-style hubs (and in fact, were justified this way in terms of a business case), and later became hybrid, as the costs associated with ancillary tasks (such as remediating all of the legacy batch processes to access the data hub) came to light during detailed source system integration planning and analysis.

I often discovered a heavy reliance on vendor-driven operational systems, which many times limited the ability to remediate certain technical processes. However, in spite of the fact that it became necessary to retreat from the completely persistent view from a tactical perspective, I believe that all of these projects remained successful because the persistent-style vision was kept on as a strategic guiding principle. In other words, the project teams were held accountable for creating data architectures that would support a persistent-style implementation, even if the source systems were allowed to keep their copies of the master data for a while longer. In fact, on some occasions, the project team was even asked to submit project plans for these future remediation tasks for use as time and budget might permit.

Although I’m less familiar with pure registry-style implementations, the idea of implementing something like this without a more persistent-style strategic vision to guide the proceedings makes me a bit uneasy. For example, questions I’ve heard come up more than once in regard to registry style: How is this different from the Virtual Data Warehouse concept we tried and failed to implement fifteen years ago?, or Won’t we just be creating another data silo? I have to confess, I don’t have a great response to these. Moreover, one of the things that originally drew me to MDM is that its nature (when persistent or hybrid) forces organizations to at last come to grips with the sins of their pasts in regard to data architecture, data governance, and metadata. 

---

Bill O'Kane has twenty years of experience in the design, development, and implementation of large-scale Master Data Management (MDM) systems and Data Warehouses, with an emphasis on Customer Data Integration (CDI). His experience includes both the development of internal systems and the integration of vendor-based solutions in complex environments. Bill has achieved success in diverse corporate environments including Fortune 100 financial services and health management companies, mutually held insurance companies, and privately held software vendors.