Perceptive Information Strategies

Auld Lang Syne has been sung, but since it is still January, there remains time for New Year’s resolutions. Losing weight, sleeping more, eating vegetables and working less on weekends are perennial favorites; to those I now add “blogging more often.” For some, this resolution would be a cinch. Not for me. In 2010, with one project due after another, I struggled to carve out time to blog. I hope to correct that in 2011, even as I anticipate a new year full of interesting and demanding projects.

In competition with writing one single blog, of course, are all the social media options that have many of us splayed across the Web, including Facebook and Twitter. These need care and feeding, too. In my business, despite its high inanity quotient, Twitter has become essential for communicating news briefs and quick-take analyses. Facebook is many things to many people, but for me it is just a “fun” forum for sharing observations and artifacts along life’s journey. Maybe someday it will be more. Can’t forget LinkedIn. Finally, the social network experience has to include commenting on other people’s blogs, at major news source sites, on Yelp, on Amazon and so on. Have to give the text analytics engines something to chew on!

Most industry pundits have already published their predictions and prognostications for 2011. Rather than add to the pile, I would like to offer a few quick information management “resolutions”: priorities that I believe will shape what happens in 2011.

Integrate the integration. In 2010, the business value of information integration hit home to many organizations. Improved integration can lower the cost of information management and help eliminate downstream business problems caused by poor data quality and inconsistency. Yet, across enterprise departments and business functions there are usually numerous data, application and process information integration steps. With vendors such as IBM, Informatica, Kalido, Oracle and Talend beginning to provide better tools for developing and governing the use of data through master data management and “semantic” metadata layers, organizations have the opportunity to work toward comprehensive, end-to-end visions of information integration.

Don’t be blinded by automated analytics: The good news coming out of 2010 is that advanced analytics involving large (ok, “big”) data sources are becoming mainstream. More organizations than ever before will be able to afford analytics, especially as they deploy data appliances and use services, templates and other tools to shortcut the development of models, variable selection and other steps that are difficult and time-consuming. However, organizations need to “keep it real”: this is important stuff, involving critical decisions about customers, patients, pricing, demand chains, fraud prevention and other factors that are differentiators. Despite the hype, automated analytics are not entirely ready to replace wetware, gut feel or moments of irrational inspiration.

Respect “keeping the lights on.” It’s fashionable these days to dismiss non-strategic IT tasks as merely “keeping the lights on.” I found in 2010 that some of the most complicated and important decisions organizations are making these days have to do with IT infrastructure. Virtualization and cloud computing are completely remaking the map, which means that IT has to move to the next generation of tools and analysis of network optimization, application performance management, dependency mapping and more. Organizations need more sophisticated tools and analysis to make the right decisions about IT infrastructure.

Encourage vendor “coopetition.” The sensational story of Mark Hurd’s departure from HP and resurfacing at Oracle dominated headlines in 2010. The story is not over; InformationWeek’s Bob Evans offers an insightful blog about the continuing friction between HP and Oracle. In the wake of Oracle’s Sun acquisition, the two companies are in the midst of a tectonic shift away from what had been a longstanding partnership. Organizations should remind vendors such as HP and Oracle that despite competitive antagonism, they expect them to work together effectively on behalf of their interests. Customers have that kind of clout. Fasten your seatbelts, though, because I’m sure we’re in for more M&A activity, possibly involving HP and Oracle, which will further reshape the competitive landscape.

Be smart about being “dynamic.” A major watchword this year is “dynamic.” Cloud computing, virtualization, appliances, workload optimization, workforce optimization and other technologies are helping organizations flex to meet changing business requirements without the usual steps of adding people, software and hardware resources that sit idle when not needed. To be more just-in-time requires knowledge; otherwise, organizations could be caught short. In 2011, before going fully dynamic, organizations need to evaluate whether they have adequate knowledge about user and business requirements. If not, it may be time to evaluate tools and practices for understanding workloads, process requirements, dependencies and more. Old ways – and divisions between business and IT – have to change.

That’s all for now. Happy New Year to all!

When Graeme McDowell tapped home his putt to seal a championship at the U.S. Open on Sunday (June 20), spectators who packed the stands and stood shoulder-to-shoulder around the green roared their approval. Tiger Woods and Phil Mickelson, the Open’s superstars, were humbled by the Pebble Beach course and its famously changeable weather. The little-known McDowell “survived,” as several commentators put it. But that doesn’t really give him enough credit. He played a smart, safe game that adapted well to course conditions.

(Photo credit: Lance Iversen, The Chronicle)

The same might be said about IBM’s technology operations, which in partnership with the U.S. Golf Association’s Digital Media team stood the test of a massive number of virtual fans visiting online and mobile U.S. Open sites. IBM and the USGA said that over four million visitors came to the U.S. Open’s Web site, about 8 percent more than last year. This was the first big year for the mobile site, which had nearly two million visits. A major attraction was the “Playtracker” application, which enabled users to fly over the course and get visualizations of how the course was playing through heat maps based on scoring feeds. You can imagine the potential for future data-driven visualizations based on historical data about courses, players, pin positions on the greens and much more.

IBM’s technology management of the U.S. Open site offered a case example of how virtualization and workload management are becoming the essential ingredients of scalability, availability and agility, certainly for consumer Web sites like the Open’s. The USGA is no stranger to IBM’s virtualization technology; IBM has a close services partnership with the USGA, which includes running a variety of cloud services for the Association from its data center in North Carolina. When I visited the trailer near the Pebble Beach course where Web site and scoring services technicians were holed up, I couldn’t help but be amazed at the simplicity of the dashboards that offered real-time views of workload performance on a virtual platform of servers located across the country.

As John J. Kent, IBM Program Manager for Worldwide Sponsorship Marketing explained, virtualization is critical to utilization efficiency, enabling IBM to combine several workloads onto a single platform. “Virtualization basically makes the distributed environment into a mainframe, which has had this virtualization capability forever,” he said. Kent heads up IBM’s technology partnerships with other events, including this week’s Wimbledon Championships tennis event. Kent said that tennis is actually the more data-rich game, with fans already interested in analysis of “all the potential data points – such as unforced errors and rally counts – that can help you understand the strength of a player’s performance.”

In distributed environments, scaling up has always meant adding more hardware; with virtualization and cloud computing, organizations can avoid the long “cap x” procurement process and simply request more of what they need, and it can be made available rapidly over the network. What’s key, then, is to understand and monitor their workloads so that they can be optimized as demand rises and falls; then, organizations don’t have to spend on procuring enough servers to match peak workloads – but otherwise let them sit idle.

The other performance throttle IBM needed during the Open was to regulate content flow. Bandwidth is now the chief bottleneck; the explosion of advanced mobile devices in particular has moved users ahead of what networking providers are able to offer. IBM and the USGA’s Digital Media team needed the ability to make dynamic decisions about regulating content flow. “We needed to understand content demand well,” said Kent. “We were able to slow scoring updates, for example, if we were reaching a threshold in demand for content access and live streaming.” Thus, workload intelligence is critical to managing unstructured content as much as it is for data.

The USGA needs to provide a rich virtual experience on mobile devices to capture a younger demographic, which is important not only for the continued success of professional golf but also for attracting advertising on its Web site. However, as fans grow more dependent on the experience delivered by their mobile devices, it will be interesting to see if the USGA responds to pressure to allow those who attend the Open to bring them, which they are currently prohibited from doing. While there are good reasons not to have onsite fans working their mobile devices and interrupting the lovely hush before a player takes a swing, I wonder if the USGA will have to bow to the inevitable. Otherwise, fans might prefer to stay outside, where they can enjoy a rich, virtual experience.

But in any case, from an IT perspective, the key to victory in the U.S. Open and similar high performance events is clear: Know the workload and optimize it through the virtualized infrastructure. The victorious Graeme McDowell set a good example.

Oracle’s acquisition of Sun Microsystems continues to reshape the competitive landscape in the software and IT industry. Perhaps nowhere else is this more apparent than in the long-running battle between Oracle and IBM. By packaging together database software and systems in Exadata – especially with storage and server technology from Sun, which was already an arch IBM competitor – Oracle has ignited a “stack” war with Big Blue. Pick up the Financial Times or other business press, and you are likely to see one of Oracle’s trademark no-frills advertisements claiming Exadata’s benchmark performance superiority to IBM database systems.

IBM has hardly taken Oracle’s jabs lightly; for example, it has responded recently with benchmark results that assert lower overall database system costs compared with Oracle/Sun systems. I’m not going to write about benchmark wars here, but I will say that as stacks turn into pre-configured appliances, apples-to-apples comparisons of price and performance get tougher. It is important to examine closely the specifics of what the marketed benchmark results are reporting: in other words, whether the price-performance numbers account for all software and hardware costs, just hardware or whatever. This is especially true for organizations evaluating database appliances that offer pre-configured systems that integrate software, storage and server technology.

My focus here is on how IBM, with its recent software and systems announcements to support its “smarter systems for a smarter planet” strategy, is aimed at changing the basis of competition. To be sure, IBM has been traveling in this direction for a lot longer than just since Oracle’s acquisition of Sun. However, at the launch event (April 7, IBM Almaden Research Center, San Jose), you could feel the temperature in the room rise whenever Oracle came up. IBM needed to respond to Oracle, but there’s a lot more going on than a battle of database machines.

The April announcements brought technology substance to IBM’s long-running campaign to educate the market about why “the planet” needs to be smarter. In short, the context IBM has been articulating is that public and private organizations in all industries are growing increasingly dependent on the flow of data for everything they do. This includes the data tsunami arriving in the form of sensor data, online clicks and comments, surveillance and more. If they wish to improve processes, performance, customer service, market intelligence and innovation, they need to use data effectively and be “smarter.” This must happen with all information activities, including analytics and transaction processing.

The significance from a technology perspective is that software and systems can’t be part of the problem. Organizations need technology that does not simply add to the headaches of poorly integrated information silos, no “end-to-end” view of performance and prohibitive costs for scalability and speed. It’s not good enough just to deliver a souped-up database machine; the technology must offer something more, so that the organization can become smarter, not dumber.

Each of the announced systems (preconfigured for x86, Unix/Linux and System Z platforms) has distinctive features; those based on the POWER7 processor were the most impressive. However, the unifying theme for all was workload optimization. Arvind Krishna, general manager for Information Management in the IBM Software Group contrasted “closed” appliances with IBM’s workload optimized systems, which he said are designed to take on additional capabilities and flex to the workload demand so that the system remains efficient, cost-effective and scalable. IBM Research’s Bijan Davari, IBM Fellow and vice president for Next Generation Computing touted IBM’s workload optimization leadership “spanning decades,” particularly in mainframes, and described ongoing research. IBM demonstrated how the DB2 pureScale Application systems optimize workloads for transaction-intensive systems, and the Smart Analytics systems do so for BI and analytics.

Pardon me for using political consultant James Carville’s ugly phrase, but to have smarter systems, it’s the workload, stupid. Organizations must understand their current and anticipated workload, and then choose the technology platform (or external service) appropriately. They must ensure that the systems they choose enable continuous analysis, management and optimization of workloads from the end-to-end, user experience perspective, so that as demand intensifies and more data flows through, performance does not suffer and costs don’t skyrocket. Move over, speeds and feeds: workload is what matters.

As we move further into our information-rich age of multiple sales and service channels, social media and surveillance, identity is becoming a hot topic. First, there’s identity theft: According to a recent study by the Ponemon Institute (sponsored by Experian’s ProtectMyID.com and reported by The Medical News), “nearly 1.5 million Americans have been victims of medical identity theft.” Credit fraud, reputation fraud and more are additional negative results of having sensitive information about ourselves spread across the information ecosphere.

Then, there’s identity surveillance. Law enforcement and intelligence services must deal every day with identity confusion as they try to work within legal constraints to find wanted criminals and potential terrorists. Adding complexity, law enforcement will need to determine identity not just from traditional data but multimedia as well; an example is this current caper reported by the Tallahassee (Florida) Democrat.

Identity surveillance and watch lists are rising as political and policy challenges. Canada and the United States are in the news here and here, tussling over implementation of Secure Flight, the plan to collect more passenger data for watch lists that will be implemented by the Transportation Security Administration of the U.S. Department of Homeland Security. See this Intelligent Enterprise blog from last June by Rajan Chandras for some background.

In the middle of all of this are software providers, primarily IBM InfoSphere Identity Insight Solutions, Infoglide (which is providing software for the DHS) and Informatica. In February, I attended the Informatica Analyst Conference and had a chance to talk to execs there about the Informatica Identity Resolution (IIR) solution and how it fits with other solutions and technologies such as master data management (MDM). I came away with a strong sense of how IIR is opening doors to new business opportunities for Informatica in government, but also potentially in areas where Informatica has greater market strength but where identity recognition and resolution software has not traditionally been applied.

Identity recognition and resolution systems enable organizations to use data matches to gain a better understanding of identity across multiple systems. This could include not just individual identities but also networks and relationships: that is, who people know and how they are connected. The tools generally apply algorithms and rules engines to automate and systematize steps that would obviously take gumshoe detectives far longer as they seek clues, patterns and a risk assessment about possible terrorists, fraudsters, money launderers and regulatory violators.

When Informatica acquired Identity Systems from Nokia in the spring of 2008, it looked like simply a smart addition to the company’s data quality toolbox. However, it is clear now that the acquisition was one of a series of decisive steps that have turned Informatica into a more broadly relevant information management (IM) solutions provider. The Identity Systems deal was followed in 2009 by the acquisition of AddressDoctor GmbH, a tool for postal address cleansing and verification. And of course, Informatica recently made its biggest move early this year by acquiring Siperian, a provider of MDM tools.

IIR is an important component of Informatica’s complete MDM solution, and will help organizations implementing MDM gain the much-sought single view of identities (customers, patients, criminals and more) across multiple data sources. A key capability to look for in identity recognition and resolution tools is functionality in multiple languages and countries. Combined with AddressDoctor, Informatica has tools for locating and matching identities around the world. And thinking beyond law enforcement uses, global corporations with diverse markets need better tools for identity network analysis to improve marketing, billing, service and more, especially in this age of social media.

IIR can also help internally, given that data is often hidden in applications and obscure databases. A healthcare firm at the Analyst conference described how it is using IIR for operations between its mainframes and users’ 30,000 Microsoft Access databases. Finally, one of the more interesting technology pairings I learned about at the conference was the real-time application of IIR for “identity-aware” event processing using Informatica’s CEP engine Agent Logic. Watch lists and other espionage uses are an obvious application of this combination, but it could also be applied in systems for financial services, healthcare, retail and other industries.

In the olden days, identity might have seemed a simpler, more innocent matter, although viewing film noir and reading detective novels from the ‘40s and ‘50s might make you wonder. Today, however, there’s no question that identity is a complex topic that includes sensitive political and privacy ramifications. Software providers such as Informatica should be in for a wild ride.