What is Advanced Analytics?

A significant number of blog posts have been devoted to the subject of Big Data Analytics, so it is a good time to revert back to what Advanced Analytics delivers to the business.

I view Advanced Analysis as the use of data and models to provide insights to guide decisions. Advanced Analysis is therefore about data analysis, and for advanced analytics to be useful, quality data from trusted sources is critical. This is the key difference with Big Data analysis which is focused on large and varied formats of data, accumulating at high velocities.

Advanced Analytics requires structured data. Where there is structured data, Enterprise Content Management would help, but where there is no structure in the data, the analysis will be fundamentally flawed. The saying "garbage in, garbage out" holds true for Advanced Analytics.

Advanced Analytics begins with the analysis of data. Once analysed, it is presented in executive dashboards with advanced visualisation capabilities. There are 4 terms noteworthy terms in the definition of Advanced Analytics...data, models, insights and decisions.

With Advanced Analytics, mathematical algorithms are used to evaluate data a context relevant to the user. Complex mathematical models discover patterns in the data which would have previously been unknown.

Models are the complex mathematical formulae used to augment the data available to knowledge and insights. The purpose of the models is to uncover insights that drive better decision making.

I have been engaged in conversations with business leaders, demonstrating how PureApplication System can be used to accelerate the delivery of Advanced Analytics capabilities. Using patterns of expertise, we are able to significantly reduce time to value, enabling line of business executives to benefit from innovative IT capabilities in less than a couple of months, where it would have previously taken years to implement.

The Promise of Collaborative Development and Operations

I came across a  DeveloperWorks article by IBM Distinguished Engineers Ruth Willenborg and Steve Abrams where they discussed Collaborative Development and Operations.

In my view, this is about creating a collaborative development platform that aligns all stakeholders, e.g. line of business, development, testing teams, suppliers and users. The ultimate goal is to accelerate the delivery of services by the business to its customers in a way that is measurable and where continuous feedback can be provided for process improvement.

IBM PureApplication System is a platform developed with these principles in mind. On one end of the scale, you have the development organization working with business analysts to design, prototype and build software assets. These assets are integrated and consumed on PureApplication system which is optimized for  web application delivery and lifecycle management. The platform's cloud computing capabilities enable operators to interact with it in a self service manner, resulting in faster delivery of services to consumers and end users. 

In today's agile world of mobile application and services delivery, collaborative development and operations is about enabling continuous innovation by software development organizations  and continuous delivery of application services to end users and consumers. This requires a platform that supports collaborative development, continuos integration, continuos testing, continuous release management and continuous monitoring, all in an iterative lifecycle.

This article shows how applications can be on boarded onto PureApplication System using patterns of expertise. Patterns offer significant potential in enabling IT organizations to transform themselves and become better aligned to the business.

Part 2: PureSystems Analyst Roundtable Discussion

A key discussion topic for this week's Analysts roundtable is the PureData System for Transactions. 

PureData System for Transactions has been designed and built for clients with the highest demands for performance and availability. To many of our clients, such systems are core to their business and any downtime can result in loss of revenue, customer churn, tarnished image, etc.). PureData System is a  "Tier 1" system, designed for such mission critical applications.  

At the heart of PureData System for Transactions is DB2 pureScale, which provides extreme levels of availability and scalability, with application transparency. PureData System for Transactions inherits these capabilities to deliver high levels of reliability, performance and scalability out of the box. Existing DB2 applications are supported with no changes, and Oracle Database applications are also supported with minimal, and in some cases, no changes are necessary.

One of the most striking features of the IBM PureSystems family is the consistent, single web user interface through which all hardware and software components are managed and monitored. This significantly reduces the additional knowledge required to operate any offering within the PureSystems family, once a user is familiar with one of the offerings. Think about the effort required to transition from an iPod to an iPhone. Once a system is delivered, the PureData System for Transaction adoption process is as simple as

1. Power-on
2. Integrate system to the network
3. Create and deploy database clusters
4. Configure access and allocate storage to databases
5. Upload objects and data into the created databases

Part 1: PureSystems Analyst Roundtable Discussion

I have a date with a group of European Industry Analysts later in the week, and as I reflect on the key messages I would like to convey, I thought it might be useful to share my thoughts on this blog. I will start off in part 1 of the series with an introduction to the PureSystems family, the topic of interest to the Analysts.

You might recall that in April 2012, IBM introduced PureSystems with the first 2 family members, PureFlex System and PureApplication System. The PureData System joined the family in October 2012.  

The PureFlex System integrates computing, storage, networking resources and system management to simplify and accelerate the delivery of infrastructure services.  It is ideal for those clients who want to accelerate deployment and simplify management of infrastructure for building custom application platforms and application stacks. PureApplication System is an integrated application platform, delivering platform services that include pattern-based application stack deployment and management, web application serving and database management. PureData System is an expert integrated system optimized exclusively for delivering data services.

PureSystems address the challenges associated with an increasing complexity in the provisioning of IT infrastructures capabilities (storage, compute and networking) and associated web application middleware software; and the increasing volume, velocity and variety of data used today in all aspects of business.

Best practices and expertise are built into the system, reducing and in many cases, eliminating the need for operators to go through extensive documentation and best practice white papers to understand how to configure, tune, deploy and operate applications and data on the systems. 

To ensure clients are able to get value from the platform faster, system is designed, integrated and optimized for specific high performance applications in the factory. On delivery, the client simply focusses on integrating the system into their infrastructure, rather than spending time connecting the various pieces together and installing software. Standardization significantly increases time to value by eliminating the need for extensive installation and configuration. Additionally, because each PureSystem is design, configured and optimally tuned in the factory in a standard manner, troubleshooting is made easier. 

In today's highly competitive business environment, competitive advantage arises from being able to take advantage of capabilities faster that anyone else. While some might focus on delivering a vast array of features and functions, a key design principle of PureSystems is to simplified the user experience, exposing what matters most to system users and operators. This enables them to be most productive. With PureSystems, procurement is simplified with a single line of support and part number.  No assembly of components is required. Applications and data are ready to load application in hours. Open integration to 3rd party software is possible, with the whole systems managed through an integrated management console that greatly simplifies system upgrades and maintenance across the stack. 

PureSystems comes in different models that have been designed, integrated and optimized to deliver application and data services to today's demanding applications with simplicity, speed & lower cost. For more information can be found here.

What IT systems does a Smart City need?

A McKinsey Global Institute article "Urban world: Cities and the rise of the consuming class", reflects on the speed and unprecedented scale of city expansion. This is placing an increasing demand for new environmentally friendly infrastructure for water and energy, buildings, transportation and communication. Migration is motivated mostly by the search for a better life, and in additional to the infrastructure challenges previously outlined, this also places a strain on the cities to deliver citizen based services such as education, health, public safety, economic development and social programs.

Smarter city operations rely on the ability to capture data that can be used to anticipate and proactively resolve problems. Problem resolution can be achieved by coordinating processes and resources for more efficient operations. Take public safety as an example. IT systems can be deployed to predict, monitor and mitigate crisis situations. This can be achieved by automatically analyzing video streams for threats based on known criminal patterns. Similar capabilities can be applied to transportation for more effective traffic management, and for analyzing water use and consumption patterns, thereby enabling utilities to identify leakages and optimize repair jobs for improved service delivery.


The figure to the left is from  "Competing on Analytics, Davenport and Harris, 2007". 

To be competitive, Smart Cities need to deploy Analytics capabilities that enable the delivery of a high quality of services that meets and exceeds citizen expectations today, and  can accommodate future needs for real-time dynamic access to innovative new services. 

Additionally, these new services need to be resilient, secure, compliant with local requirements, and sufficiently agile to address new risks posed by an ever more connected and collaborative world. And all of this needs to be achieved cost effectively. IBM's PureData System for Analytics is designed specifically for this use case.

Imagine a situation where city operations can be collaboratively managed via Executive, City Operations and Agency dashboards that include domain key performance indicators for standard operating procedures. Such a system could support centralized planning, execution and monitoring for more efficient operations.  

IBM's Intelligent Operations Center delivered on a Cloud enabled PureSystems platform, enables cities to not only contain operational cost and complexity, but achieve breakthrough productivity gains through rapid time to value via virtualization, optimization, energy stewardship.

 

The Economics of Cloud Computing

To be competitive in today's tough economy climate, IT organizations need to find ways of delivering innovative business services while taking cost out of their operations. IT costs are normally categorized as operating expense and capital expense, and effective cost containment requires the right balance between the two.

To reduce capital expense, many IT organizations have turned to virtualization, the ability to pool and share IT resources. Pooling resources reduces capital expense of hardware, software and facilities. Less infrastructure, and correspondingly, less energy is required to deliver the same quality of service. 

Standardization on a common software stack, common operational policies, service delivery and management processes, is considered one of the most effective ways of reducing operating expense. With standardization and automation, less human resources and skill is required to deliver the same quality of service. Expertise is generally recognized as one of the fastest growing piece of the IT spend.  

Addressing standardization and virtualization is key to reducing infrastructure costs, while meeting the dynamic needs of the business. With increased infrastructure standardization, organisations are able to achieve greater economies of operating expense. Similarly, IT organizations that leverage virtualization within their infrastructure realize greater economies of capital expenditure.  

Leading IT vendors have been ceasing on this opportunity to support a closer alignment of IT to the business. IBM's Expert Integrated Systems were motivated by the increasing need for IT organisations to achieve cost optimization. 

They are built on industry standards and designed to enable organizations to accelerate cloud adoption. Cloud computing enables business agility, making it possible for organizations to improve service delivery by applying engineering discipline and economies of scale in an Internet inspired architecture. Expert Integrated Systems can be deployed in a customer controlled private network to drive efficiency, while retaining control and customization in a private cloud model.

The Promise of Mobile Business

Until very recently, I was mostly oblivious to the Bring Your Own Device (BYOD) trend that appears to be getting more and more embedded in the fabric of enterprise IT. I now find myself owning a personal iPhone, Samsung Galaxy and a MacBook Air, all of which I use for work. In my case, the acquisition was largely driven by a desire to use what I believed would enabled me to be most productive. I had also become more and more frustrated with the length of time it took to boot up my laptop, let alone the weight. The MacBook Air was therefore a pretty compelling proposition. Email on my iPhone was more pleasing, and the Samsung Galaxy enabled me to create a mobile private Wifi hotspot using my personal 3G broadband sim card. All I need is an iPad Mini to complete the picture!

From a business point of view, these trends are having a  transformational impact, creating opportunities for new business models as user's expectations drive what IT needs to deliver. Think about your mobile experience for a minute. I find myself contemplating switching banks based on the user experience of the mobile banking application. And for enterprise IT, it is not simply a matter of getting the best application out of the door. Mobile applications have a much faster lifecycle, and are much more iterative than traditional applications. Enterprise IT therefore needs to adopt a much more agile application development and release lifecycle. And the challenges do not end there. They also have to cater for a fragmented landscape of devices and network providers.

That said, the opportunities that can be realized through mobile business are immense. In a recent tweet, I shared a link to an article on 19 Surprising Facts About Africa’s Mobile Market. The potential for leveraging the mobile platform for business is clear. What makes the mobile platform different and compelling is the flexibility it offers in terms of the multitude of applications available, the wide range of connectivity options, multimedia support, location awareness and other sensory capabilities.

While many recognize the opportunities available in the business to consumer space, the business-to-enterprise benefits are not very well understood. The mobile platform, when used effectively, can significantly enhance worker productivity, by enabling mobile workers to have access to the information and resources they need, wherever they are and when ever they need it. Throughout most of my IT career, I have been a mobile worker. The benefits to me personally and to my employer are obvious. I am able to work from home when family needs dictate without always having to take a day off. My colleagues are able to reach me even while I am traveling. The ability to work from home  makes me more productive. I do not have to incur the cost and time of traveling into the office every day. I reduce my carbon footprint, while my employer is able to reduce their personnel costs with schemes like Bring Your Own Device to work and by reducing office space to a bare minimum. In many instances, these benefits can be realized by simply extending existing applications to mobile workers and customers. I have personally witness many of my clients significantly improve customer satisfaction and increase revenue by leveraging the mobile platform to deepen their customer engagement. Developing a mobile strategy is clearly one of the ways organizations can differentiate themselves.

To many, context awareness is the new killer application. This relates to the ability to dynamically capture contextual information from mobile devices and use it to optimize and change communication and business flows. With context awareness, you can increase customer loyalty by delivering loyalty reward to a customer based on their unique needs at a given point in time. This concept can be extended to the enterprise with mobility-linked business processes. In this instance, business processes can be improved by capturing and automating process interactions where and when they occur.

Let's discuss the scenarios above using examples. In a customer loyalty use case, a retailer can offer an application for creating and managing shopping list. As a customer creates a list, their intended purchase could analyze against their historical shopping patterns, enabling the retailer to offer personalized shopping vouchers. The retailer's objective is to retain the shopper's business and capture a greater share of spend. The process of analyzing spending patterns and making an offer needs to be done at realtime. The PureData Systems for Analytics is designed to address petascale analytical workload requirements. In a business to business context, similar principles can be applied to deliver information to field workers just in time. In other words, consider an engineer working on an offshore oil rig. As they go about their job and input data into their mobile workpad, information can be analyzed in real time and the business process changed on the fly so  additional checks or maintenance is conducted. The same principles can be applied to other scenarios, e.g. optimizing supply chain and logistics planning, organizational learning to improving business outcomes etc.

Recognizing the opportunity mobile business presents, we have been working closely with some of our clients, leveraging the Worklight platform to enable them to create rich, yet cost-effective mobile apps in this fragmented technological landscape. A key piece of the capability we deliver is enterprise integration, enabling clients to connect to enterprise back-end services in a secure and scalable manner, while securely managing and controlling the growing portfolio of deployed enterprise applications.

Redefining the Economics of Enterprise IT

Did you followed IBM closely in 2012? If you did, you would (should) have come across PureSystems, a new approach to computing that is set to radically redefine the economics of IT. 

PureSystems introduces a new approach to IT system design, procurement, lifecycle management and retirement. Systems are designed and optimized in the factory for a particular set of related tasks (workloads) and are not meant to be general purpose or highly customizable. Generally, they have a static, predefined stack of hardware and software that cannot be altered. Where the system design does not meet the needs of a particular use case, other custom solutions may be more appropriate.

So why design a system with such "limitations" ?

As it turns out, more is less in this instance. PureSystems are purpose-built systems that are intended to be easy to setup, configure and manage. Much of the burden of setting up and managing traditional IT system have been removed or radically simplified, making it possible for clients to adopt highly complex enterprise IT systems without having to understand all the complex internal details. When one looks deeper into the time spent on the phases of implementing IT projects, it becomes clear that there is significant opportunity for improvement to accelerate time-to-value. Forrester did some research for IBM which showed that the top causes for delays were related to the time spent integrating, configuration and tuning systems, as well as deploying applications. These are key areas address by PureSystems.

Think for a moment about what Apple did with the original Macs and subsequently the iPod and iPhone. All Apple devices were  integrated and optimized in the factory for a well defined set of tasks, the Mac for personal computing, the iPod for multimedia entertainment, and the iPhone for mobile telephony and related functions. Apple was able to optimized the software running on these devices as they understood and tightly controlled the hardware. This enabled them to create a much more pleasing user experience, and this observation is not simply related to the Software Ergonomics, but they way the device feels; its weight, its thickness, the way it looks. There is much more to an Apple device that the functionality it offers, and this in my view, was the one of the triggers for Apple's cult following.

Now, turn to Google's Android. Google's Android operating system is freely available for all to use, and Samsung's innovation has ensured the Andriod operating system remains dominate outside of the USA. While openness and ease of adoption is most certainly a competitive advantage for Android, the loose coupling between the Operating System and the hardware makes it more challenging for Android to achieve the same level of brand specificity as Apple. In the Android world, users have a choice of devices from many players with their very own unique device designs. This does not in any way guarantee Apple's dominance or vice versa. In my view, there are merits in both approaches.

Let's get back to PureSystems...

With PureSystems, IBM has made available, the benefits of both approaches. The "Apple" of PureSystems are the Expert Integrated Systems that ship from the factory pre-racked, with all the storage, compute, networking and middleware software components in place, and ready for use. Like the iPhone, once the system is out of its packaging, you plug it into the power supply, connect it to the network, and go through a mostly automated set up process to initialized the systems and begin installing applications from an internal application store. As with Apple's devices, PureSystems has various offerings designed to address different usage profiles. There is an application system for running transactional web applications, a database system for transactional database applications, and an analytics systems for analyzing large data sets. This is akin to the iPhone, iPad, Mac etc. And as with the iPad and iPad Minis, these also come in different sizes and configurations that are easy to understand and adopt based on the anticipated workload. PureSystems offers the best user experience, enabling IT organisations to adopt enterprise computing capabilities while realizing much faster time to value.

But IBM does not stop there. Recognizing that others might have already invested in hardware from IBM and other vendors, IBM introduced IBM Workload Deployer. A subset of the capability used for installing and life-cycle managing applications in the PureSystems application store is available in IBM Workload Deployer (IWD). With IWD, clients can automate the installation and lifecycle management of applications onto a member of the PureSystems family, PureFlex, a pre-integrated, optimized infrastructure system, as well as other 3rd party and IBM custom built hardware systems. This model is akin to the Android model, where various device manufacturers build and design their custom hardware for the Android Operating System. While the software user experience might be consistent, the physical experience, i.e. the look and feel of the device varies across device manufacturers. So while the Samsung Galaxy S3 might run the Android Operating System as well as the Samsung Galaxy Note or LG, Motorola or any other vendor's device, the total experience is undeniably different.

In my view, investing in the PureSystems family is akin to investing in a range of Apple devices. The customer service experience, the look and feel, and experience of working with the offering is very consistent across the range of products. If you know how to use one, learning to use the other is relatively easy, when compared to learning something new. Investing in IBM Workload Deployer on the other hand has its advantages. You are able to deploy applications across a hybrid set of hardware infrastructures. This requires IBM, the client or a third-party to configure the hardware platform for use with IWD. Included in the PureSystems family is PureFlex, an offering well positioned for this use case. By acting as a middleware deployment and lifecycle management broker for a range of hardware platforms and operating systems, some of the optimizations possible from tight integration and control of the underlying hardware are not possible. Lifecycle management activities such as hardware firmware patch management would vary between platforms.

In conclusion, it is worth stating that Workload Deployer and PureSystems deliver a comprehensive set of cpabilities that address the need for agility and simplicity in IT services delivery to the business. Where an IT organization is focused on time to value, PureSystems has some distinct benefits, and where there is a need to leverage existing infrastructure investments, IWD is uniquely positioned. We see many of our most innovative clients adopting both systems effectively to transform their IT operations and differentiate themselves over their competitors. 

Explaining how IBM is making the cities work better...

I was at home last night watching CNBC (could have been Bloomberg) with my 10 year old daughter when an IBM advertisement campaign came on and got her excited. Needless to say I was left with the task of explaining what IBM does today. What's the IBM company of 2013? What capability would encapsulate what "Building a Smarter Planet" signifies?

As I pondered through this, I settled on the view that the best way of approaching the issues is to focus on a challenge most of us experience / encounter on a daily basis. The need to leverage technology to sustain economic growth and enhance the quality of life of citizens in a rapidly changing world appeared to be a good choice.

From an offerings point of view, I narrowed my focus on IBM's Intelligent Operations Center portfolio. Our city leaders are being held to account, and need to improve their decision making. IT can help by enabling them to derive insights from the information and data they have at their disposal. In this instance, the volume of data needs to be sufficiently comprehensive and readily available for analysis. Additionally, where historical data is not available, IT can also be used to anticipate problems, so the right people can get in front of events and problems before they become a crisis. And all of these activities require coordination; coordination of people, resources, systems etc.
 
The need to empower city leaders, from mayors to the police chiefs, heads of transportation, and beyond is clear. The Intelligent Operations Center Portfolio of services enables users to

    1.    Leverage available information to make better decisions
    2.    Analyze problems and resolving them proactively
    3.    Coordinate resources and processes to operate effectively

The initial 3 areas of focus was the intelligent management of operations, whatever they might be, traffic and water management.

Intelligent Operations Management is about planning, organizing, monitoring and sharing information between various stakeholders. We are in an era where governments are being held to account, so the ability for public sector executives to access information on dashboards that include domain specific key performance indicators is critical. 

With respect to traffic management, the need for realtime visibility is a obvious. The ability to analyze historical data and correlate to traffic incidents can enhance future planning. Water is one of the world's most precious resources, and in many parts of the world needs to be managed carefully. IBM has invested in developing capabilities that enable utility companies to instrument, collect, analyze and visualize data on water usage using Geographical information systems. IBM is also innovating on the delivery of these capabilities to its clients, offering both the ability to deploy this on premise, in which case the PureApplication System would be a attractive platform, or as a Software as a Service offering on IBM's Smart Cloud.

What is driving the increasing adoption of Cloud Computing ?

The challenges associated with growing IT complexity and its impact on business agility has been the focus of recent conversations on this blog. There is clearly a need to address increasing operating costs associated with the delivery of IT services. Studies conducted by IBM, IDC and others has shown that over the last few years, management and administration costs have increased significantly against flat or decreasing new system spend, and marginal increases in power and cooling costs.

Cloud computing is the new consumption and deliver model inspired by consumer Internet services.  Underpinning cloud computing are proven capabilities of virtualization, automation and standardization, which is driving IT simplification and increased efficiency. Line of business executives are attracted to the self-service characteristics of Cloud, the new sourcing options it presents, and its economy of scale. Cloud essentially represents the industrialization of delivery for IT supported services. Leading cloud vendors like IBM provide clients with the flexibility to adopt the cloud solution appropriate for their organization, be it a Private Cloud, Public Cloud or a Hybrid Cloud including elements of both. In a recent newsletter, IDC predicted the increasing emergence of workload specific Clouds in 2013.

I see organizations adopting Cloud in 3 steps

1) Consolidation
2) Virtualization
3) Automation

Consolidation reduces server sprawl and results in a reduction in infrastructure complexity. Reduced complexity translates to less effort to operate and manage, which should improve operational costs and reduced total cost of ownership

By adopting a shared, virtualized infrastructure, IT organizations are able to significantly increase hardware utilization, resulting not only in a reduction in new hardware spend, but also with regards to associated costs like cooling and power. A virtualized infrastructure can also simplify application deployment and ongoing operation by removing physical resource boundaries.

Add automation to a consolidated and virtualized infrastructure, and you can dramatically reduce deployment cycles. Standardized delivery of IT services provides an opportunity to introduce granular metering and billing. This enables the flexible delivery of new processes and services.

IBM's PureApplication System is designed to accelerate cloud adoption. It enables customers to deploy Cloud services in less than 4 hours, from power on and connection to the client's network. It is integrated in the factory with advanced virtualization that enables IT resources to be shared among many applications. This results in more efficient utilization of IT resources and reduced hardware costs through economies of scale.

Patterns of Expertise, the ability to embed application configuration, deployment and lifecycle management best practices significantly reduces IT cycle times and management costs by enabling applications to be deployed with minimal skill in the shortest time possible.

Workload optimization algorithms and tight integration of all components, including software and hardware, results in a platform that can scale up and down to optimize IT resource utilization, with the network and storage system optimized to the workload.

The promise of Cloud enabled Analytics

The promise of analytics is all too familiar. The ability to optimize traffic flow in realtime, conduct fraud and risk detection as transactions are taking place, understand and act on customer sentiments in social networks among many others. Analytics has truly evolved from a business initiative to a business imperative.

In IBM's 2011 CIO study, a majority of CIOs ranked Analytics as the #1 factor contributing to an organization's competitiveness. A 2011 study conducted by IBM Institute of Business Value and MIT  Sloan Management Review confirmed that organizations that embraced analytics were 2x as likely to outperform their peers.

Recently, we have witnessed a shift of focus from Enterprise Data Analytics to Big Data Analytics. This is being accelerated by the significant amount of data being generated daily; 12 terabytes of tweets, 5 million trade events every second, thousands of video feeds from surveillance cameras etc. A quote from John Naisbitt summarizes the state of today's economy very well...“We have for the first time an economy based on a key resource [Information] that is not only renewable, but self-generating. Running out of it is not a problem, but drowning in it is.”

In this new economy, there is a need for complementary approaches to Analytics to handle these new  sources of data. Traditional sources of enterprise data, which are usually structured and logical can be handled quite effectively by Transactional, ERP and Data Warehouse Systems. Emerging Sources of unstructured data, e.g. machine generated data like RFID, log data and data from sensors, or Cloud Data which is typically includes a combination of text, multimedia and other forms of unstructured data, require a platform tuned to their unique workload characteristics. There is therefore a need to develop capabilities that bridge the need for Analytics on structured and unstructured data.

The emergence of these trends has shifted IT into the center of business, but IT faces challenges realizing the value Analytics can deliver. Due to the increasing complexities of IT infrastructure, most organizations are unable to shift budgets away from maintaining and operating existing systems. The increasing data volumes, formats and sources of data is driving the need for new solutions that minimize complexity and reduce time to value.

Cloud computing can minimize these barriers and reduce complexity through standardized service delivery. Clouds computing platform optimize investments through a shared infrastructure with elastic scalability to handle variable workloads. This results in a faster time to value. 

IBM's PureData System accelerates cloud adoption. PureData System is optimized for very high transactional throughput and high speed peta scale analytic and transactional data workloads. IBM clients are using PureData System for Database & Analytics workload consolidation on a highly scalable and resilient infrastructure.

Why PureSystems ?

In a previous blog post, I presented my views on one of the key motivation behind IBM's introduction of PureSystems into the marketplace, citing the need for IT to adopt solutions that are optimized to achieve the business agility needed in today's business environments. Most c-level executives anticipate significant change and complexity ahead, with only a handful prepared to handle it. With complexity on the rise, agility becomes a key competitive advantage.

In the past, IT used to be the control point. The lines of business specified their requirements and IT designed and implemented what they needed.  The line of business had their own application silos, with IT providing safely through separate ownership and segregation of resources. Increasingly, the line of business teams are making the call, with business owners having greater control.  In the cloud space, evidence of this trend can be confirmed by the increasing adoption of Software as a Service applications.  Without IT consulation, the business is reaching out to external IT services providers to fulfill their requirements.  In one fell swoop, the entire IT organization can be cut out, and to make it worse, the IT organization is left to deal with subsequent integration problems.  And clients are demanding more too.  The rise of mobile computing puts pressure on IT organizations to deliver support for new types of applications, additional capabilities that enterprise applications never had to deal with before.  

This need for agility is transforming the relationship between IT and the business in very profound ways. The true promise of IBM's PureSystems isn’t just about addressing the complexity of IT,  it is about redefining the economics of IT.  Over half of the business executives believe that cloud computing enables business transformation; leaner, faster, and more agile processes.  This is strategic thinking.  Organizations that approach cloud in a tactical fashion risk adding complexity and inefficiency due to fragmentation, redundancy and operating silos. PureSystems is designed to accelerate cloud adoption, enabling organizations that embrace cloud strategically, from a business as well as IT perspective, to capture new business value through innovation, flexibility, and speed, with integrity and security, while reducing cost and complexity. 

The new bread of IT practitioners...

In 2012, a majority of CEOs interviewed during an IBM funded CEO Study identified technology as having the greatest impact on their business. Technology had always featured highly, but this is the first time it was ranked #1.

What does this mean for the IT profession?

You can argue that in future, there will be a need for IT practitioners to focus on enabling business users to consume and control business content for competitive advantage, through a reasoned application of information technology. This requires an IT practitioners to be a

•  Consensus builder
•  Results oriented
•  Generalist
•  A technology expert
•  Not just a top level software designer
•  Not (just) a programmer
•  Not the project manager
•  Not a product expert
•  Not a lone scientist

Against this backdrop of increasing expectations and reliance on IT by the business, IT is not being provided with the resources it needs to implement new capability. On average, IT budgets are growing at less than 0.8% per year, resulting in most spend being applied to maintaining on going operations and support of existing IT infrastructures. This is a reflection of the complexity of today's IT systems, and the need for significant resources to keep the running.

Most c-level executives and technical leaders I have interacted with recently, mostly across Europe, have cited integration and collaboration between IT and Lines of Business as critical success factors for innovative projects. As the CEO study shows, innovation is increasingly delivered through software, applications and technology.

In April 2012, IBM introduced a new bread of Expert Integrated Systems.

Given that solutions are manifested as architectures, Expert Integrated Systems are integrated in the factory to include systems, applications and process components optimized for specific workloads. PureSystems integrates a broad variety of products, technologies and services, various systems and applications architectures, and diverse hardware and software components into a ready to use system. By taking complexity out of enterprise IT systems,  IT practitioners are able to dedicate more of their time to creating strategic business outcomes, aligning business needs with appropriate Solutions and technologies.

This is how I see the professional evolving over time, with IT practitioners collaborating across lines of business and IT to align with strategic business planning, solution design and delivery; leveraging ready to use workload optimized Expert Integrated Systems that accelerate time to value.

What can you expect from a Smart Stadium?

As home entertainment systems improve (e.g. 3D HD TV), stadiums are coming under increased pressure to differentiate themselves and offer a pleasing entertainment environment for their fans and visitors. 

To continue to attract visitors, there is a need to address the challenges of managing traffic flows,  parking, the need for collaboration with public safety officials (police, fire, ambulance) etc.

The most progressive stadiums are investing in technology that can be used to improve the operation of their facilities, essentially enabling them to create a Smarter Stadium.

The ability to monitoring gate throughput can result in a better understanding of how fast the stadium is filling, enabling operations staff to optimize the flow of traffic into and out of the stadium, as well as within the grounds. Data from the turnstiles can be aggregated into a dashboard and analyzed against key performance indicators. Operators can use dashboards to view the flow rate across each gate, and if the traffic becomes too high, they can take corrective action by either slowing the flow further downstream, or diverting traffic to other gates with more capacity. As fans drive into the stadium, they could be notified by SMS to park at an alternative location to optimize traffic flow into the stadium.

Performance of concession stands can also be monitored to understand how each is performing. Monitoring sales channels and offers performance should give a good insight into POS revenue and concourse level revenue, making it possible for decisions to be taken in realtime to improve sales performance on the fly. Data could be aggregated further into concession location revenue and terminal revenue to analyze and understand buying behavior and trends across different parts of the stadium. Sales transaction information could also be combined with other queue monitoring systems to offer realtime navigation assistance to stalls with the shortest queues, or ensuring stalls do not run out of items in popular demand.

Another interesting use case could be related to compliance. Let's assume that alcohol sales are not allowed past the 4th quarter of a football game. Sales data from the POS terminals could be aggregated and the transactions validated against business rules. When an alcohol sale violation incident occurs, an alert is triggered and relevant information could be displayed on the stadium operations consoles and stadium maps, and passed onto an Incident Management system where a notification could be sent to other systems and devices, e.g. to alert staff with mobile devices. All data generated can be stored in a data warehousing system for subsequent analysis and reporting. 

The capabilities and use cases described in this article can be implemented on IBM's PureApplication System, a system designed to reduce IT complexity and accelerate time to value.  It  ships with all storage, networking and compute capabilities integrated into a rack in the factory, and pre-configured and optimized for Web Application and Database workloads. Patterns are used to capture best practice and accelerate the deployment and lifecycle management of pre-integrated, optimized industry solutions like the one described in this article.

Visit the IBM Smarter Stadium Solution for more information 

PureData for Analytics Value Proposition

The PureData for Analytics offering, a member of the PureSystems family of Expert Integrated Systems offers a simple-to-use approach for serious Analytics on structured data.

Analytics activities such as data exploration, discovery, transformation, model building and scoring can be performed where the data resides, in the warehouse.

This reduces the time it takes to build and deploy Analytics models, accelerating fact-based decision based on insightful Analytics.

Practitioners can also experiment iteratively with different models, operationalizing and making advanced Analytics more accessible.

What's a Field Programmable Gateway Array?

While conducting a deep dive into IBM's PureData for Analytics system, it became obvious that Field Programmable Gateway Array played a key part in its performance. Little did I know that this technology is used in many every day electronic devices, e.g. DVDs make use of it to facilitate reads of high quality compressed digital data off spinning discs without jitters.

The device itself is a semi conductor chip equipped with a large number of internal programmable gates. When programmed, it acts as a specialized hardware for specialized tasks requiring high performance.

In PureData for Analytics, data is delivered from disk to memory as quickly as it can be streamed off disk, compressed and cached in memory by the FPGA using a smart algorithm which ensure that frequently used data is served out of memory. The embedded engines in the FPGA can be dynamically modified and extended programmatically, and act on streaming data at extremely high speed. In addition to compressing the data using semiconductor based technology, the FPGA filters out unnecessary columns and rows to boost performance.

My first "in depth" look at the PureData for Analytics System

Hosts

The primary interface to the PureData for Analytics system are high performance Linux hosts. External tools and applications, e.g. reporting, backup and recovery etc, interact with the host via standardized interfaces, e.g. JDBC etc. The host compiles SQL queries into executable code snippets, creates optimized query plans and distributes the snippets to massive parallel processing nodes for execution. The host is in an active-passive high availability cluster configuration, mirroring data to the standby hosts which monitors the primary host and takes over in case of a failure.

S-blades

The bulk of the analytics workload processing occurs on intelligent massively parallel processing nodes called S-blades. S-blades are optimzed for processing analytics workloads at massive scale. They contain multi-core CPU, multiengine Field-Programmable Gate Architectures, and gigabytes of RAM, all optimized to work together to deliver peak performance. Continuous availability is made possible my the systems management software, which monitors the s-blades (including memory), and automatically takes a failed S-blade out of service and moves the processing load to a spare one.

Disks

The S-blades are connected to disk enclosures via a high-speed interconnect that enables streaming of data to the S-blade memory at the fastest rate possible. The disk enclosures contain high density, high performance disks. Redundancy is built into the data path from each S-blade to the disks. Each drive is mirrored in a RAID 1 configuration, and should a disk fail, the storage subsystem simply redirects I/O processing to the mirror without interruption of service. Spare drives are included, allowing the system to replace failed drives and regenerate content for full redundancy.

Network
The communication in the MPP grid occurs on an optimized IP based network designed for high volume data warehousing traffic patterns. It allows maximum utilization of the network bandwidth without overloading it, thereby allowing predictable performance close to the data transmission speed of the network. There are 2 completely independent networks for redundancy. The data network is also completely separate from the management network. This enables the system to assess the health of its components even where there might be data network problems.

Optimizing Database Warehousing Operations

The biggest challenge or bottleneck across date warehousing operations is the speed at which the database engine can read from and write to disk. This is commonly known as disk I/O bottleneck.

Most efficient analytics platforms minimize data movement, and are able to process streaming data from disk to memory in parallel, and on a massive scale. In the case of IBM's Netezza, this is accomplished by using innovative hardware acceleration. It uses Field Programmable Gate Arrays (FPGA) to filter extraneous data as early in the stream and as fast as the data can be streamed off disk.

By eliminating data that is not required close to the data source, downstream components like CPU, memory and network do not have to deal superfluous data, significantly reducing I/O bottlenecks and improving system performance.

Expert Integrated System Design

Expert Integrated Systems are a new breed of IT systems that ship pre integrated in the factory with processing, storage and networking components for faster time to value.

They are born optimized for a specific workload and built to deliver industry leading price-performance ratio with appliance simplicity.

The challenge for IT providers is to create a system that combines the best components, (rather than the most advanced or most highly performing), to create a design where the component s work together to deliver the best price -performance. In other words, an elegant design to overcome common IT operational challenges.

Data Warehouse in a Big Data World. What is the use case?

I like the analogy in the IBM Big Data Platform book titled "Harness the Power of Big Data".

It tells the story of the days long gone, when miners could easily spot nuggets or veins of gold with the naked eye. This made investing easier, as its value could be seen and therefore the resources required to extract it considered against its perceived value. Using a Big Data analogy, we can consider this gold to be "high value-per-byte of data".

Assume for a moment that there is more gold nearby, but it is just no visible to the naked eye. Trying to find this gold is a bit more of a gamble and potentially more expensive. This would be "low value-per-byte of data" due to the challenges associated with finding gold not visible to the naked eye.

With the right equipment however, it might be possible to economically process lots of dirt and keep the flakes of gold found. This flakes can be taken for processing and combined to make flakes of gold.

Back to our Big Data analogy...

In this scenario, it would make sense to keep all the dirt we could find (in a Big Data System), so that as new, economical dirt processing techniques emerge (Big Data Analytics on commodity systems) we would have an opportunity to extract the flakes of gold (value / insights) and store it for processing into gold bars (in our Data Warehousing system).

Hadoop is a Big Data batch system that allows users to store all data in its native business object format and get value out of it through massive parallel processing on commodity components.

Data Warehouse is characterized by "speed-of-thought response times" requirements where sustainable data with  proven value stored and delivered interactively.

It is therefore clear to see that in a Big Data world, there is value and a place for both Hadoop (Big Data) and Data Warehouse systems.

IBM's Hadoop system is Infosphere Big Insights. For simplified Big Data Analytics, look no further than IBM PureData for Analytics powered by Netezza, and Infosphere Warehouse for your Data Warehousing needs.