Perceptive Information Strategies

That’s my new title. On March 1, I began working with TDWI (The Data Warehousing Institute), an organization well known to business intelligence (BI) and data warehousing professionals for education, training, certification, news and research. The World Conferences, Executive Summits, Solution Summits and other events have long been critical educational gatherings for the professional community engaged in BI, analytics, data warehousing, data integration and other topics. The next event on the TDWI calendar is in Washington, D.C., and I will be there. 

In my new position, I will be heading up research reports; writing blogs, articles and columns; contributing to the conference programs; judging the annual Best Practices Awards; consulting with TDWI members; and more. I will have the pleasure of working closely with Philip Russom, TDWI’s Senior Manager of Research and Services, who will focus more on the data management side. I’ve known Philip going back to my Intelligent Enterprise days, where he wrote an excellent column for us. He is an important thought leader in this industry and has a real skill in educating professionals. And, he’s a pianist and singer, with a fondness for The Great American Songbook. I also look forward to working with all the great people associated with TDWI as speakers, educators and vendors, many of whom I have known and respected throughout my career.

While my focus will be on my work at TDWI, this doesn’t spell the end of this blog. I will definitely be busy, but I hope to keep this spot active with commentary on other areas of interest that may not fit into my work at TDWI. I will have a new blog on the TDWI Web site though, which will have its first installment next week.

Meanwhile, Opening Day, Giants vs. Dodgers, is just around the corner!

With margins for transactional operations getting thinner, organizations in many industries are focused on leveraging more value from their data. This could not be truer than in the financial services industry, where the onrushing spread of algorithmic trading is changing…well, everything, including the role of information in guiding trading and investment decisions. This rather unnerving article in Wired by Felix Salmon and Jon Stokes captures what’s happening quite well. The speed with which organizations need to turn data into both better business insight and marketable data services has many looking closely at data virtualization.

As an analyst, I am fortunate to be a (remote) participant in the Boulder BI Brain Trust (BBBT), the Boulder, Colorado-based brainchild of Claudia Imhoff. A couple of times a month, this illustrious group of experts gathers for half-day briefings with vendors in the business intelligence, analytics and data warehousing space; the briefings are always highly informative (on Twitter, watch for the hash tag #BBBT). On Friday, January 14, the topic was data virtualization: the BBBT met with Composite Software, a leading vendor of tools and solutions for data virtualization. Composite brought along a customer – and no small customer, either – NYSE Euronext. Emile Werr, NYSE Euronext’s VP of Global Data Services and head of Enterprise Data Architecture gave us a briefing on how the company is developing a data virtualization layer using Composite’s products.

Wikipedia has a good definition of data virtualization: “to integrate data from multiple, disparate sources – anywhere across the extended enterprise – in a unified, logically virtualized manner for consumption by nearly any front-end business solution, including portals, reports, applications, search and more.” As the Wiki entry notes, data virtualization (or “data federation”) is an alternative to data consolidation or replication into data warehouses and data marts. When this concept was first introduced, it was the cause of fiery debates at TDWI events and elsewhere. Now, it has settled in as a complement to the other more entrenched approaches.

Virtualization helps when organizations can’t really wait for standard data integration and loading into a data warehouse. That is very much the challenge facing NYSE Euronext, which is using Composite tools to develop a virtual layer to improve data access for internal executives and to establish a platform for creating data services. “We have so many companies trying to connect into us, and we want to serve standardized information out to companies around the world,” Werr said. NYSE Euronext is moving away from its old method of dumping transaction data into a warehouse; it wants to put more intelligence into the virtual layer. And to help build this layer, it is hiring people with business skills who understand processes and how to derive business value from data. “[These professionals] are the most productive people on my team right now,” he said.

The BBBT session featured an interesting debate about how data governance fits with data virtualization. Can data quality and governance rules be managed and invoked from the virtual layer? Should they be managed at the source or as part of extract, transformation and loading (ETL) processes, as many organizations do now? The discussion began to turn toward master data management and the option of creating a central hub or registry to implement governance for access to multiple sources. Highly regulated industries such as financial services and healthcare should consider this approach because of the need to invoke regulatory provisions for data access and sharing. Werr discussed these requirements and how his organization hopes to use the Composite virtual layer to support metadata governance and access from multiple BI tools.

Putting intelligence into a virtual layer fits with the IT infrastructure trend toward virtualization and cloud computing, and may become even more important because of this trend. Service-oriented applications running on cloud and virtual platforms frequently require access to multiple, disparate data sources. From a business standpoint, data virtualization is going to be critical to moving data quickly from the back office outward, to where it can be packaged into “information-as-a-service” offerings that customers will buy – and that will improve the seller’s profit margins.

Down in the land of razzle-dazzle, IBM drew on the spirit of Las Vegas by “unleashing” Cognos 10 onstage on October 25, with fireworks, explosions and a lot of light fantastic. The Cognos 10 release was the headline of IBM’s Information on Demand (IOD) 2010 Global Conference. Rob Ashe, CEO of Cognos before it was acquired by IBM and now general manager of Business Analytics for the IBM Software Group, said the release, six years in the making, was about “eliminating the boundaries that keep you from gaining and sharing insights.”

Cindy Howson of BIScorecard nicely summarizes the high points in her Intelligent Enterprise blog. She writes: “In theory, though, it is the ability for users to collaborate around a task, with supporting BI dashboards, wikis, blogs, instant messages and emails, where I see the biggest potential.” Cindy is not alone; many analysts and writers are excited about the possibilities surrounding the integration of business intelligence tools and collaboration platforms. Cognos 10 is a big step for IBM in this direction; it takes advantage of Cognos’ standard of innovation in performance management and dashboards to push the collaborative user experience beyond basic capabilities.

Collaborative BI has attracted other big BI vendors, and has lit up innovative new players. Lyza, for example, is an impressive product in this vein; it offers a breakthrough that radically changes how people use business intelligence, or looking at it the other way, how they flow through information in the course of collaboration. Cognos 10 may not induce the same intense reaction as Lyza, but it is definitely a meaningful development, both for the market and for IBM internally.

The collaboration thrust joins Cognos with Lotus through the inclusion of a free license of Lotus Connections. IBM bought Lotus in 1995 for about $3.5 billion, which made it at the time the biggest acquisition in the history of the software industry and signified IBM’s strategic move into applications. Twelve years later, IBM picked up Cognos for nearly $5 billion. With Cognos 10, IBM could be on the road to higher return on investment from these enormous acquisitions as organizations try to use collaboration and social networks to improve employee productivity.

Threading collaborative interaction with business intelligence – or “business insight,” the term Cognos seems to prefer – holds tremendous potential for employee productivity. At IOD, I met with Jeff Shick, vice president of Social Software in the IBM Software Group. We discussed many of promising ideas from the realm of knowledge management: capturing tacit knowledge, finding repeatable patterns in how people use and share information artifacts and the potential of recommendation engines for finding the most relevant information and sources, including people. Knowledge management, once a kind of blue sky topic, never seemed so grounded in the fact-based decisions and actions that people must perform.

Organizations will discover more about their knowledge assets by integrating BI and collaborative tools and practices. The combo could free users from their traditional BI silos that serve single users or communities and allow information insights to flow across organizational divisions and out into the external world.

Of course, combining collaboration and BI is not without challenges. For one thing, many organizations are very careful about who gets BI, and how much they get to use it. Thus, the combination will change user demand for BI and challenge organizations to rethink security, user requirements, data availability and more. But, the catalyst for change is good; organizations should never stop innovating with BI.

“Steve Jobs is my best friend,” said Oracle CEO Larry Ellison, early in his Oracle OpenWorld keynote on September 23. “I love him dearly, and he’s someone I watch very closely, watch what he does over at Apple.” While it’s unlikely that we’ll see iPod or iPad-like billboards featuring silhouettes of IT managers rocking out to their Oracle Exadatas, Ellison is unabashed in expressing his desire to emulate what Jobs has done. “We believe that if you engineer the hardware and software pieces to work together, the overall user experience – the product – is better than if you do just part of the solution.”

Exadata and the new Exalogic Elastic Cloud, which Oracle announced at OpenWorld: Do they represent commoditization or innovation? The answer is both. For obvious reasons, vendors tend not to market their ability to provide commodity products. IT buyers are consumers, too, and “commodity” has that word “common” in it – not too inspiring. In the consumer market, however, Apple has been highly effective at turning mass-produced machines into objects that symbolize innovation. While it’s certainly possible to imagine devices engineered to produce sound that is superlative to an iPod, Apple was the first to create the right package at the right time for the broader market.

Perhaps someday, just like an iPod, you’ll be able to buy an Exadata or Exalogic box from a dispensing machine at the airport. Ok, perhaps not. But Oracle seems convinced that mainstream IT buyers are similarly ready for a “commodity” package. They are weary from the pressure of supporting complex and costly existing systems that give them little room for discretionary spending. Configuration and change management are two of the most time-consuming tasks; it has also been difficult for IT developers to adapt software applications to parallel hardware platforms to make it easier to scale up and out. Plus, having boxes hanging off intelligent networks and existing out in the cloud appears to be the approach taken by architects of many newer systems.

Successful but relatively exotic products exist, such as Netezza (just recently acquired by IBM) and Teradata. In fact, the database market is teeming with more exotic products than at any time in the past 20 years or so. But Oracle is betting that the mainstream market will only adopt the advanced and esoteric stuff if it is inside the box. “The car industry delivers complete cars,” Ellison observed in his keynote. “There are a lot of computer systems involved in running a modern car, but it’s all been engineered and tested to work together – whether it’s a Prius or my favorite commuting car, the Bugatti.”

Given the number of software suppliers that Oracle has acquired in recent years, it’s no surprise that Oracle would be obsessed with product consolidation and integration. However, it was interesting to see how this objective is affecting strategies throughout the company. Business intelligence, for example, “should not be separate,” Ellison said. “The core of the system should be intelligence; it should be everywhere.”

This sort of consolidation and integration can be innovative, in that it will propel the majority of IT buyers into feeling that they are enabling innovation in their businesses by buying packaged machines such as Exadata and Exalogic. But time will tell, and competitors will respond. Early adopter sessions at OpenWorld that I attended made migration to Exadata sound straightforward, with considerable benefits accruing from reductions in cost and time-to-deployment compared to traditional systems. It won’t be on the bleeding edge that Oracle’s success will unfold; it will be in the mainstream.

Former Hewlett-Packard CEO Mark Hurd’s severance vacation and search of the help-wanted listings proved to be short. Oracle announced that Hurd will be joining the company’s board of directors and will assume the position of president. Current co-president Safra Catz is staying, but Charles Phillips has departed. Phillips deserves credit for improving Oracle’s customer relationships during his era; it’s easy to forget now how raw those relationships were at the time he took the helm. However, like the end of Hurd’s reign at HP, Phillips’ more extended finale came with embarrassing personal distractions.

HP is not accepting Hurd’s new position peaceably. The company has filed a lawsuit, noting that by joining Oracle, Hurd is violating a confidentiality agreement and “[putting] HP’s trade secrets and confidential information in peril.” Given that Oracle is increasingly more of a competitor than a partner to HP, it’s easy to see HP’s difficulty with this turn of events. I don’t know whether HP’s lawsuit will succeed in stopping Hurd’s appointment, but a quick read of experts quoted in the business press seems to suggest that it will not.

The arrival of Mark Hurd at Oracle could be a huge development that will send shock waves across the industry. As evidenced by recent trading, investors seem bullish about Oracle’s prospects with Hurd’s arrival. Here as six points that I see as significant with regard to his hiring:

-          Hurd will bring leadership to Oracle’s hardware, systems and data center push. This is new: Oracle will now have a top executive with experience in the hardware and systems business. Oracle’s steady core has always been database software, with services and other types of software and services subject to shifts in strategy and restructuring. Will Hurd, after looking at the business prospects for the Sun offerings, try to change Oracle’s balance to go strong after HP and IBM? Or will Hurd conclude that Oracle needs to mend its relations?

-          Hurd is Oracle’s first strong #2 since Ray Lane. Oracle experienced significant growth during Lane’s years as president and COO with Oracle, but his relationship with Oracle CEO Larry Ellison soured, and he was gone after eight years. Hurd is no shrinking violet, especially after having been the CEO of two different companies. What will happen when Hurd and Ellison disagree? And will they agree on the possible acquisitions that everyone expects will happen?

-          Hurd will be tempted to “cut the fat” inside Oracle. No doubt, part of Wall Street’s positive reaction to Hurd’s appointment reflects the view that he could make Oracle even more profitable by taking an axe to internal costs, as he did at HP, and hold managers more accountable. Oracle is run more efficiently than HP was when Hurd took over, but it is an established company with over 100,000 employees. Surely an outsider with the practiced eye of Hurd will find areas to trim. How will this affect Oracle’s internal psyche?

-          Hurd could invigorate Oracle in the channel. Hurd has a reputation for valuing channel partner relationships – an area where Oracle has had trouble. Hurd could make improvement here a focus, and help Oracle counter stronger direction from IBM and Microsoft.

-          Hurd’s hiring will be a catalyst for change in the industry. If Hurd does make Oracle a more serious player in the server, storage and data center business, competitors will have to react. If its stock price continues to fall, would a Cisco Systems, EMC or even Microsoft make a play for HP? Wild thoughts, but then the IT industry has not really had to deal with Oracle as the sort of broad-based player it could now become.

-          Hurd could give more oomph to Oracle’s BI, analytics and data warehousing. With the 11g releases, Oracle is in the early stages of an important product cycle for business intelligence, analytics and data warehousing. Hurd was clearly a believer in the power of BI at HP; I would expect him to give Oracle’s offerings an even higher profile.

Larry Ellison, whether fired up by a friends’ misfortune or the opportunity to bruise a competitor, has succeeded in planting a couple of banderillas in the HP bull, angering and distracting it. But now, by bringing Hurd on board, he could be about to change Oracle itself.

I arrived at my first meeting at TDWI in San Diego late, still hyperventilating from legging out a long hot walk from my hotel, where I had dumped my bags after a gnarly drive down to “America’s Finest City” from Los Angeles. So, perhaps appropriately, my first meeting was with a company in the healthcare industry: Blue Cross and Blue Shield of Kansas City, co-winner of TDWI’s Enterprise Data Warehouse Best Practices Award. The company won the award as a customer of HP Business Intelligence Solutions.

The healthcare industry is obviously undergoing tremendous change, with new government policies and economics challenging the business models of insurance and managed care providers such as “Blue KC.” The transition is away from controlling benefits – that is, denying benefits – and increasing rates as the primary tools for managing costs. The future is “wellness,” or helping members get or stay healthy. “We want to improve the health of our members so that they don’t become ‘patients,’ said Darren Taylor, vice president, Information Access Division with Blue KC. “If we can do that, then we can take care of costs from within the context of that objective.”

Wellness requires knowing more about members, which means that the companies need vastly improved data management and analysis. Connecting to disparate data systems and establishing a single enterprise data warehouse (EDW) are proving critical to accomplishing Blue KC’s objectives with its membership. Previously, Blue KC had outsourced diabetes or other disease management programs “to good companies,” Taylor said, “but we did not have enough insight into these proprietary systems.” The company could not integrate or analyze multiple sources of data about one member to understand how, for example, their heart conditions, asthma or other issues were related. Gaining this single view is essential. With the EDW in place, the company is able to bring these disparate data sources in house.

Taylor was VP of the IT group, but his group now reports to Blue KC’s CFO. “There’s more accountability. IT is usually about waiting for requirements. We’re now about anticipating needs, and bringing business and IT together to execute on our data warehouse strategy.”

As we pulled out into San Diego harbor, the dusk was calm. The increasing dark settled slowly over the lingering sunset hues of pink and orange. A gentle breeze filled the sails of the big schooner as we began our cruise with members of Hewlett-Packard’s Business Intelligence Solutions group and customers, who were in town this week to attend The Data Warehouse Institute (TDWI) conference. The captain invited guests to help hoist sails, which we did. Now it was time for drinks and hors d’oeuvres. Dinner would follow.

The only disconcerting – but at the same time, entertaining – part of the cruise was what appeared to be a U.S. Navy SEALs training session going on all around us. Helicopters swooped low and gunboats raced by; as darkness fell, more copters lit the water with searchlights, putting the bay under close surveillance. San Diego mostly basks in sunny, happy weather, but overhead and out on the water, you see constant reminders of San Diego’s key role in the serious business of hosting naval forces that defend the country and project American power.

As expected, HP personnel kept their lips sealed about the continuing saga of Mark Hurd, HP’s former CEO, the HP Board and actress/marketing consultant Jodie Fisher. With media such as The New York Times, The Wall Street Journal and the blogosphere filling in details daily, it was hard not to bring it up. Some HP folks smiled guardedly in agreement when it was suggested to them that while the mess is not pleasant, HP might be better off replacing Hurd’s fearsome regime of performance accountability and tight spending control with more creative, inventive and inspiring leadership.

I couldn’t help but reflect on the excitement Hurd generated when he joined HP in 2005 after previously heading up NCR and its then Teradata division. When Hurd became CEO, he made it clear that he “got it” when it came to BI, analytics and data warehousing, both with regard to the company’s internal management and his strategy for HP’s software products and services. He hired Randall Mott away from Dell to become HP’s executive VP and CIO; before his time at Dell, Mott was CIO at Wal-Mart, where he led its successful and influential enterprise data warehouse foray. Hurd directed Mott to consolidate HP’s numerous data centers and data marts to reduce costs and improve BI and analytics.

Under Hurd’s leadership, HP increased investment in database technology, particularly NeoView. However, strategies kept shifting, internal cost control became the focus and NeoView did not have the market impact that HP once hoped it might have. A big challenge was balancing its database software development strategy with its ongoing partnerships with Oracle, Microsoft and other players entrenched in the BI and data warehousing market. So, rather than software, HP’s BI foot forward became consulting services and solutions. In late 2006, HP bought the respected Knightsbridge Solutions, which has become the centerpiece of its BI and data warehouse consulting services. An HP BI Solutions customer, Blue Cross and Blue Shield of Kansas City, was selected as co-winner of TDWI’s 2010 Best Practices Award for Enterprise Data Warehousing.

I had the opportunity to talk to Darren Taylor, vice president, Information Access Division with “Blue KC.” I’ll write about this discussion in my next blog, which will offer some quick takes about my meetings at TDWI in San Diego. Closing out this one, it will be interesting to see if HP installs a new CEO with the same kind of vision Hurd seemed to have about the power of BI and data warehousing – but with more soul, and more consistency in terms of the strategic drive and investment needed to succeed in a tough marketplace.

When Graeme McDowell tapped home his putt to seal a championship at the U.S. Open on Sunday (June 20), spectators who packed the stands and stood shoulder-to-shoulder around the green roared their approval. Tiger Woods and Phil Mickelson, the Open’s superstars, were humbled by the Pebble Beach course and its famously changeable weather. The little-known McDowell “survived,” as several commentators put it. But that doesn’t really give him enough credit. He played a smart, safe game that adapted well to course conditions.

(Photo credit: Lance Iversen, The Chronicle)

The same might be said about IBM’s technology operations, which in partnership with the U.S. Golf Association’s Digital Media team stood the test of a massive number of virtual fans visiting online and mobile U.S. Open sites. IBM and the USGA said that over four million visitors came to the U.S. Open’s Web site, about 8 percent more than last year. This was the first big year for the mobile site, which had nearly two million visits. A major attraction was the “Playtracker” application, which enabled users to fly over the course and get visualizations of how the course was playing through heat maps based on scoring feeds. You can imagine the potential for future data-driven visualizations based on historical data about courses, players, pin positions on the greens and much more.

IBM’s technology management of the U.S. Open site offered a case example of how virtualization and workload management are becoming the essential ingredients of scalability, availability and agility, certainly for consumer Web sites like the Open’s. The USGA is no stranger to IBM’s virtualization technology; IBM has a close services partnership with the USGA, which includes running a variety of cloud services for the Association from its data center in North Carolina. When I visited the trailer near the Pebble Beach course where Web site and scoring services technicians were holed up, I couldn’t help but be amazed at the simplicity of the dashboards that offered real-time views of workload performance on a virtual platform of servers located across the country.

As John J. Kent, IBM Program Manager for Worldwide Sponsorship Marketing explained, virtualization is critical to utilization efficiency, enabling IBM to combine several workloads onto a single platform. “Virtualization basically makes the distributed environment into a mainframe, which has had this virtualization capability forever,” he said. Kent heads up IBM’s technology partnerships with other events, including this week’s Wimbledon Championships tennis event. Kent said that tennis is actually the more data-rich game, with fans already interested in analysis of “all the potential data points – such as unforced errors and rally counts – that can help you understand the strength of a player’s performance.”

In distributed environments, scaling up has always meant adding more hardware; with virtualization and cloud computing, organizations can avoid the long “cap x” procurement process and simply request more of what they need, and it can be made available rapidly over the network. What’s key, then, is to understand and monitor their workloads so that they can be optimized as demand rises and falls; then, organizations don’t have to spend on procuring enough servers to match peak workloads – but otherwise let them sit idle.

The other performance throttle IBM needed during the Open was to regulate content flow. Bandwidth is now the chief bottleneck; the explosion of advanced mobile devices in particular has moved users ahead of what networking providers are able to offer. IBM and the USGA’s Digital Media team needed the ability to make dynamic decisions about regulating content flow. “We needed to understand content demand well,” said Kent. “We were able to slow scoring updates, for example, if we were reaching a threshold in demand for content access and live streaming.” Thus, workload intelligence is critical to managing unstructured content as much as it is for data.

The USGA needs to provide a rich virtual experience on mobile devices to capture a younger demographic, which is important not only for the continued success of professional golf but also for attracting advertising on its Web site. However, as fans grow more dependent on the experience delivered by their mobile devices, it will be interesting to see if the USGA responds to pressure to allow those who attend the Open to bring them, which they are currently prohibited from doing. While there are good reasons not to have onsite fans working their mobile devices and interrupting the lovely hush before a player takes a swing, I wonder if the USGA will have to bow to the inevitable. Otherwise, fans might prefer to stay outside, where they can enjoy a rich, virtual experience.

But in any case, from an IT perspective, the key to victory in the U.S. Open and similar high performance events is clear: Know the workload and optimize it through the virtualized infrastructure. The victorious Graeme McDowell set a good example.

Oracle’s acquisition of Sun Microsystems continues to reshape the competitive landscape in the software and IT industry. Perhaps nowhere else is this more apparent than in the long-running battle between Oracle and IBM. By packaging together database software and systems in Exadata – especially with storage and server technology from Sun, which was already an arch IBM competitor – Oracle has ignited a “stack” war with Big Blue. Pick up the Financial Times or other business press, and you are likely to see one of Oracle’s trademark no-frills advertisements claiming Exadata’s benchmark performance superiority to IBM database systems.

IBM has hardly taken Oracle’s jabs lightly; for example, it has responded recently with benchmark results that assert lower overall database system costs compared with Oracle/Sun systems. I’m not going to write about benchmark wars here, but I will say that as stacks turn into pre-configured appliances, apples-to-apples comparisons of price and performance get tougher. It is important to examine closely the specifics of what the marketed benchmark results are reporting: in other words, whether the price-performance numbers account for all software and hardware costs, just hardware or whatever. This is especially true for organizations evaluating database appliances that offer pre-configured systems that integrate software, storage and server technology.

My focus here is on how IBM, with its recent software and systems announcements to support its “smarter systems for a smarter planet” strategy, is aimed at changing the basis of competition. To be sure, IBM has been traveling in this direction for a lot longer than just since Oracle’s acquisition of Sun. However, at the launch event (April 7, IBM Almaden Research Center, San Jose), you could feel the temperature in the room rise whenever Oracle came up. IBM needed to respond to Oracle, but there’s a lot more going on than a battle of database machines.

The April announcements brought technology substance to IBM’s long-running campaign to educate the market about why “the planet” needs to be smarter. In short, the context IBM has been articulating is that public and private organizations in all industries are growing increasingly dependent on the flow of data for everything they do. This includes the data tsunami arriving in the form of sensor data, online clicks and comments, surveillance and more. If they wish to improve processes, performance, customer service, market intelligence and innovation, they need to use data effectively and be “smarter.” This must happen with all information activities, including analytics and transaction processing.

The significance from a technology perspective is that software and systems can’t be part of the problem. Organizations need technology that does not simply add to the headaches of poorly integrated information silos, no “end-to-end” view of performance and prohibitive costs for scalability and speed. It’s not good enough just to deliver a souped-up database machine; the technology must offer something more, so that the organization can become smarter, not dumber.

Each of the announced systems (preconfigured for x86, Unix/Linux and System Z platforms) has distinctive features; those based on the POWER7 processor were the most impressive. However, the unifying theme for all was workload optimization. Arvind Krishna, general manager for Information Management in the IBM Software Group contrasted “closed” appliances with IBM’s workload optimized systems, which he said are designed to take on additional capabilities and flex to the workload demand so that the system remains efficient, cost-effective and scalable. IBM Research’s Bijan Davari, IBM Fellow and vice president for Next Generation Computing touted IBM’s workload optimization leadership “spanning decades,” particularly in mainframes, and described ongoing research. IBM demonstrated how the DB2 pureScale Application systems optimize workloads for transaction-intensive systems, and the Smart Analytics systems do so for BI and analytics.

Pardon me for using political consultant James Carville’s ugly phrase, but to have smarter systems, it’s the workload, stupid. Organizations must understand their current and anticipated workload, and then choose the technology platform (or external service) appropriately. They must ensure that the systems they choose enable continuous analysis, management and optimization of workloads from the end-to-end, user experience perspective, so that as demand intensifies and more data flows through, performance does not suffer and costs don’t skyrocket. Move over, speeds and feeds: workload is what matters.

As we move further into our information-rich age of multiple sales and service channels, social media and surveillance, identity is becoming a hot topic. First, there’s identity theft: According to a recent study by the Ponemon Institute (sponsored by Experian’s ProtectMyID.com and reported by The Medical News), “nearly 1.5 million Americans have been victims of medical identity theft.” Credit fraud, reputation fraud and more are additional negative results of having sensitive information about ourselves spread across the information ecosphere.

Then, there’s identity surveillance. Law enforcement and intelligence services must deal every day with identity confusion as they try to work within legal constraints to find wanted criminals and potential terrorists. Adding complexity, law enforcement will need to determine identity not just from traditional data but multimedia as well; an example is this current caper reported by the Tallahassee (Florida) Democrat.

Identity surveillance and watch lists are rising as political and policy challenges. Canada and the United States are in the news here and here, tussling over implementation of Secure Flight, the plan to collect more passenger data for watch lists that will be implemented by the Transportation Security Administration of the U.S. Department of Homeland Security. See this Intelligent Enterprise blog from last June by Rajan Chandras for some background.

In the middle of all of this are software providers, primarily IBM InfoSphere Identity Insight Solutions, Infoglide (which is providing software for the DHS) and Informatica. In February, I attended the Informatica Analyst Conference and had a chance to talk to execs there about the Informatica Identity Resolution (IIR) solution and how it fits with other solutions and technologies such as master data management (MDM). I came away with a strong sense of how IIR is opening doors to new business opportunities for Informatica in government, but also potentially in areas where Informatica has greater market strength but where identity recognition and resolution software has not traditionally been applied.

Identity recognition and resolution systems enable organizations to use data matches to gain a better understanding of identity across multiple systems. This could include not just individual identities but also networks and relationships: that is, who people know and how they are connected. The tools generally apply algorithms and rules engines to automate and systematize steps that would obviously take gumshoe detectives far longer as they seek clues, patterns and a risk assessment about possible terrorists, fraudsters, money launderers and regulatory violators.

When Informatica acquired Identity Systems from Nokia in the spring of 2008, it looked like simply a smart addition to the company’s data quality toolbox. However, it is clear now that the acquisition was one of a series of decisive steps that have turned Informatica into a more broadly relevant information management (IM) solutions provider. The Identity Systems deal was followed in 2009 by the acquisition of AddressDoctor GmbH, a tool for postal address cleansing and verification. And of course, Informatica recently made its biggest move early this year by acquiring Siperian, a provider of MDM tools.

IIR is an important component of Informatica’s complete MDM solution, and will help organizations implementing MDM gain the much-sought single view of identities (customers, patients, criminals and more) across multiple data sources. A key capability to look for in identity recognition and resolution tools is functionality in multiple languages and countries. Combined with AddressDoctor, Informatica has tools for locating and matching identities around the world. And thinking beyond law enforcement uses, global corporations with diverse markets need better tools for identity network analysis to improve marketing, billing, service and more, especially in this age of social media.

IIR can also help internally, given that data is often hidden in applications and obscure databases. A healthcare firm at the Analyst conference described how it is using IIR for operations between its mainframes and users’ 30,000 Microsoft Access databases. Finally, one of the more interesting technology pairings I learned about at the conference was the real-time application of IIR for “identity-aware” event processing using Informatica’s CEP engine Agent Logic. Watch lists and other espionage uses are an obvious application of this combination, but it could also be applied in systems for financial services, healthcare, retail and other industries.

In the olden days, identity might have seemed a simpler, more innocent matter, although viewing film noir and reading detective novels from the ‘40s and ‘50s might make you wonder. Today, however, there’s no question that identity is a complex topic that includes sensitive political and privacy ramifications. Software providers such as Informatica should be in for a wild ride.