The Economics of Cloud Computing

To be competitive in today's tough economy climate, IT organizations need to find ways of delivering innovative business services while taking cost out of their operations. IT costs are normally categorized as operating expense and capital expense, and effective cost containment requires the right balance between the two.

To reduce capital expense, many IT organizations have turned to virtualization, the ability to pool and share IT resources. Pooling resources reduces capital expense of hardware, software and facilities. Less infrastructure, and correspondingly, less energy is required to deliver the same quality of service. 

Standardization on a common software stack, common operational policies, service delivery and management processes, is considered one of the most effective ways of reducing operating expense. With standardization and automation, less human resources and skill is required to deliver the same quality of service. Expertise is generally recognized as one of the fastest growing piece of the IT spend.  

Addressing standardization and virtualization is key to reducing infrastructure costs, while meeting the dynamic needs of the business. With increased infrastructure standardization, organisations are able to achieve greater economies of operating expense. Similarly, IT organizations that leverage virtualization within their infrastructure realize greater economies of capital expenditure.  

Leading IT vendors have been ceasing on this opportunity to support a closer alignment of IT to the business. IBM's Expert Integrated Systems were motivated by the increasing need for IT organisations to achieve cost optimization. 

They are built on industry standards and designed to enable organizations to accelerate cloud adoption. Cloud computing enables business agility, making it possible for organizations to improve service delivery by applying engineering discipline and economies of scale in an Internet inspired architecture. Expert Integrated Systems can be deployed in a customer controlled private network to drive efficiency, while retaining control and customization in a private cloud model.

The Promise of Mobile Business

Until very recently, I was mostly oblivious to the Bring Your Own Device (BYOD) trend that appears to be getting more and more embedded in the fabric of enterprise IT. I now find myself owning a personal iPhone, Samsung Galaxy and a MacBook Air, all of which I use for work. In my case, the acquisition was largely driven by a desire to use what I believed would enabled me to be most productive. I had also become more and more frustrated with the length of time it took to boot up my laptop, let alone the weight. The MacBook Air was therefore a pretty compelling proposition. Email on my iPhone was more pleasing, and the Samsung Galaxy enabled me to create a mobile private Wifi hotspot using my personal 3G broadband sim card. All I need is an iPad Mini to complete the picture!

From a business point of view, these trends are having a  transformational impact, creating opportunities for new business models as user's expectations drive what IT needs to deliver. Think about your mobile experience for a minute. I find myself contemplating switching banks based on the user experience of the mobile banking application. And for enterprise IT, it is not simply a matter of getting the best application out of the door. Mobile applications have a much faster lifecycle, and are much more iterative than traditional applications. Enterprise IT therefore needs to adopt a much more agile application development and release lifecycle. And the challenges do not end there. They also have to cater for a fragmented landscape of devices and network providers.

That said, the opportunities that can be realized through mobile business are immense. In a recent tweet, I shared a link to an article on 19 Surprising Facts About Africa’s Mobile Market. The potential for leveraging the mobile platform for business is clear. What makes the mobile platform different and compelling is the flexibility it offers in terms of the multitude of applications available, the wide range of connectivity options, multimedia support, location awareness and other sensory capabilities.

While many recognize the opportunities available in the business to consumer space, the business-to-enterprise benefits are not very well understood. The mobile platform, when used effectively, can significantly enhance worker productivity, by enabling mobile workers to have access to the information and resources they need, wherever they are and when ever they need it. Throughout most of my IT career, I have been a mobile worker. The benefits to me personally and to my employer are obvious. I am able to work from home when family needs dictate without always having to take a day off. My colleagues are able to reach me even while I am traveling. The ability to work from home  makes me more productive. I do not have to incur the cost and time of traveling into the office every day. I reduce my carbon footprint, while my employer is able to reduce their personnel costs with schemes like Bring Your Own Device to work and by reducing office space to a bare minimum. In many instances, these benefits can be realized by simply extending existing applications to mobile workers and customers. I have personally witness many of my clients significantly improve customer satisfaction and increase revenue by leveraging the mobile platform to deepen their customer engagement. Developing a mobile strategy is clearly one of the ways organizations can differentiate themselves.

To many, context awareness is the new killer application. This relates to the ability to dynamically capture contextual information from mobile devices and use it to optimize and change communication and business flows. With context awareness, you can increase customer loyalty by delivering loyalty reward to a customer based on their unique needs at a given point in time. This concept can be extended to the enterprise with mobility-linked business processes. In this instance, business processes can be improved by capturing and automating process interactions where and when they occur.

Let's discuss the scenarios above using examples. In a customer loyalty use case, a retailer can offer an application for creating and managing shopping list. As a customer creates a list, their intended purchase could analyze against their historical shopping patterns, enabling the retailer to offer personalized shopping vouchers. The retailer's objective is to retain the shopper's business and capture a greater share of spend. The process of analyzing spending patterns and making an offer needs to be done at realtime. The PureData Systems for Analytics is designed to address petascale analytical workload requirements. In a business to business context, similar principles can be applied to deliver information to field workers just in time. In other words, consider an engineer working on an offshore oil rig. As they go about their job and input data into their mobile workpad, information can be analyzed in real time and the business process changed on the fly so  additional checks or maintenance is conducted. The same principles can be applied to other scenarios, e.g. optimizing supply chain and logistics planning, organizational learning to improving business outcomes etc.

Recognizing the opportunity mobile business presents, we have been working closely with some of our clients, leveraging the Worklight platform to enable them to create rich, yet cost-effective mobile apps in this fragmented technological landscape. A key piece of the capability we deliver is enterprise integration, enabling clients to connect to enterprise back-end services in a secure and scalable manner, while securely managing and controlling the growing portfolio of deployed enterprise applications.

Redefining the Economics of Enterprise IT

Did you followed IBM closely in 2012? If you did, you would (should) have come across PureSystems, a new approach to computing that is set to radically redefine the economics of IT. 

PureSystems introduces a new approach to IT system design, procurement, lifecycle management and retirement. Systems are designed and optimized in the factory for a particular set of related tasks (workloads) and are not meant to be general purpose or highly customizable. Generally, they have a static, predefined stack of hardware and software that cannot be altered. Where the system design does not meet the needs of a particular use case, other custom solutions may be more appropriate.

So why design a system with such "limitations" ?

As it turns out, more is less in this instance. PureSystems are purpose-built systems that are intended to be easy to setup, configure and manage. Much of the burden of setting up and managing traditional IT system have been removed or radically simplified, making it possible for clients to adopt highly complex enterprise IT systems without having to understand all the complex internal details. When one looks deeper into the time spent on the phases of implementing IT projects, it becomes clear that there is significant opportunity for improvement to accelerate time-to-value. Forrester did some research for IBM which showed that the top causes for delays were related to the time spent integrating, configuration and tuning systems, as well as deploying applications. These are key areas address by PureSystems.

Think for a moment about what Apple did with the original Macs and subsequently the iPod and iPhone. All Apple devices were  integrated and optimized in the factory for a well defined set of tasks, the Mac for personal computing, the iPod for multimedia entertainment, and the iPhone for mobile telephony and related functions. Apple was able to optimized the software running on these devices as they understood and tightly controlled the hardware. This enabled them to create a much more pleasing user experience, and this observation is not simply related to the Software Ergonomics, but they way the device feels; its weight, its thickness, the way it looks. There is much more to an Apple device that the functionality it offers, and this in my view, was the one of the triggers for Apple's cult following.

Now, turn to Google's Android. Google's Android operating system is freely available for all to use, and Samsung's innovation has ensured the Andriod operating system remains dominate outside of the USA. While openness and ease of adoption is most certainly a competitive advantage for Android, the loose coupling between the Operating System and the hardware makes it more challenging for Android to achieve the same level of brand specificity as Apple. In the Android world, users have a choice of devices from many players with their very own unique device designs. This does not in any way guarantee Apple's dominance or vice versa. In my view, there are merits in both approaches.

Let's get back to PureSystems...

With PureSystems, IBM has made available, the benefits of both approaches. The "Apple" of PureSystems are the Expert Integrated Systems that ship from the factory pre-racked, with all the storage, compute, networking and middleware software components in place, and ready for use. Like the iPhone, once the system is out of its packaging, you plug it into the power supply, connect it to the network, and go through a mostly automated set up process to initialized the systems and begin installing applications from an internal application store. As with Apple's devices, PureSystems has various offerings designed to address different usage profiles. There is an application system for running transactional web applications, a database system for transactional database applications, and an analytics systems for analyzing large data sets. This is akin to the iPhone, iPad, Mac etc. And as with the iPad and iPad Minis, these also come in different sizes and configurations that are easy to understand and adopt based on the anticipated workload. PureSystems offers the best user experience, enabling IT organisations to adopt enterprise computing capabilities while realizing much faster time to value.

But IBM does not stop there. Recognizing that others might have already invested in hardware from IBM and other vendors, IBM introduced IBM Workload Deployer. A subset of the capability used for installing and life-cycle managing applications in the PureSystems application store is available in IBM Workload Deployer (IWD). With IWD, clients can automate the installation and lifecycle management of applications onto a member of the PureSystems family, PureFlex, a pre-integrated, optimized infrastructure system, as well as other 3rd party and IBM custom built hardware systems. This model is akin to the Android model, where various device manufacturers build and design their custom hardware for the Android Operating System. While the software user experience might be consistent, the physical experience, i.e. the look and feel of the device varies across device manufacturers. So while the Samsung Galaxy S3 might run the Android Operating System as well as the Samsung Galaxy Note or LG, Motorola or any other vendor's device, the total experience is undeniably different.

In my view, investing in the PureSystems family is akin to investing in a range of Apple devices. The customer service experience, the look and feel, and experience of working with the offering is very consistent across the range of products. If you know how to use one, learning to use the other is relatively easy, when compared to learning something new. Investing in IBM Workload Deployer on the other hand has its advantages. You are able to deploy applications across a hybrid set of hardware infrastructures. This requires IBM, the client or a third-party to configure the hardware platform for use with IWD. Included in the PureSystems family is PureFlex, an offering well positioned for this use case. By acting as a middleware deployment and lifecycle management broker for a range of hardware platforms and operating systems, some of the optimizations possible from tight integration and control of the underlying hardware are not possible. Lifecycle management activities such as hardware firmware patch management would vary between platforms.

In conclusion, it is worth stating that Workload Deployer and PureSystems deliver a comprehensive set of cpabilities that address the need for agility and simplicity in IT services delivery to the business. Where an IT organization is focused on time to value, PureSystems has some distinct benefits, and where there is a need to leverage existing infrastructure investments, IWD is uniquely positioned. We see many of our most innovative clients adopting both systems effectively to transform their IT operations and differentiate themselves over their competitors. 

Why is Open Source fidelity important ?

IT vendors who maintain fidelity to open source and ensure their products are 100% compatible with the open source packages they rely on, can realize some tangible benefits.

Fidelity to open source makes it easier to integrate with other products designed to be compatible with a common set of open source packages. This approach has enabled IBM to amass 100s of partners, accelerating the adoption of products like IBM's Big Insights which builds on the open source Hadoop distribution. Clients and software vendors using open source libraries and interfaces for Hadoop can rest assured that their applications will run with Big Insights.

This approach of embracing and maintaining the integrity of open source projects enables vendors to adopt open source enhancements and innovate on top for competitive differentiation. There is an opportunity for software vendors to test and certify that each component works well together in a manner suitable for enterprise production deployment. Additionally, vendors can address backward compatibility issues prevalent in some open source projects by undergoing rigorous regression testing.

As with many other open source projects, the IBM Big Insights development team has committers to the Hadoop project and its related projects. These committers contribute to open source innovation and ensures that IBM plays a role in shaping Hadoop and is aligned to its future evolution. 

Part 3 : High Availability Enterprise Hadoop Clusters

Hadoop is primarily made up of 2 elements 1) The distributed Hadoop Filesystem (HDFS) and a programming paradigm, MapReduce.

Redundancy is built into Hadoop clusters. Data is redundantly stored in multiple places across the cluster, with portions of the functions running on various servers in the cluster. Hadoop is designed with the expectation that node failures will occur. Hadoop is fault tolerant. If a failure occurs, it automatically heals itself by nominating another node in the cluster to perform the work destined for the failed node.

That said, for enterprise deployment, a hot standby for the NameNode and the JobTracker server in the MapReduce processing layer is required. This was achieved in Hadoop 1.0 by having an active / passive failover solution, where data is replicated across two separate Hadoop clusters. An alternative solution is to have a dedicated backup for the master node, which includes the NameNode and can also include the JobTracker service. Should the NameNode fail, the Hadoop cluster can restart using the backup NameNode.

IBM committers have been working on Hadoop 2.0 with the Hadoop open source community to address the single point of failure challenges. In Hadoop 2.0, it is possible to designate a hot standby for the HDFS NameNode, and MapReduce2 has been enhanced to eliminate the potential single point of failure in the JobTracker by distributing its functionality across nodes in the cluster.

Part 2 : What is IBM's Enterprise Hadoop ?

In a previous blog post, I introduced Hadoop at a very high level. 

Hadoop is clearly a technology that can be used to overcome the volume and variety challenges of data at rest. Traditional analytics tools work very well on structured data, but today, only 20% of the data is structured. The need for an alternative solution is clear.

While many organisations have been experimenting with Hadoop to bridge this gap, few have been able to leverage it to gain insights that creates meaningful value. This is partly due to the deep skills required to work with Hadoop, and also the lack of enterprise capabilities that address information governance concerns, security and data lifecycle management.

IBM introduced Infosphere Big Insights, its Enterprise Hadoop platform to enable organisations to leverage Hadoop in a way that could have a transformational impact on their business and deliver significant competitive advantage. To achieve this, IBM introduced focussed on delivering 3 key capabilities. 

1. Analytics support to enhance consumability, enabling analysts with minimal programming skills to get value out of big Data. 
2. Enhancing the Hadoop platform so that it integrates with the rest of the enterprise, enabling Analysts to query Big Data from their existing Data Warehouse and vice versa.
3. Delivering additional tools and capabilities that address governance, security, administration and performance concerns inhibiting the enterprise adoption of Big Data 

For more information on IBM's Infosphere Big Insights, visit this link.

Part 1 : What is Hadoop ?

Imagine networking thousands of computers together. Each computer has its own processor and hard disk drive. All these computers are running software that makes the computers appear as a single powerful "super computer" with lots of processing capability and storage space. 

Now assume you are Amazon, and using this "super computer" to store click stream records from your amazon.com web site. You now want to understand if there are trends that lead to customers not completing transactions after having added items into their shopping carts. Recall that the click stream data is spread across the local disk drives on all of these thousands of computers.

To gain the insights required, a copy of your Analytics (application) logic is sent to each individual computer. Each computer  then runs the application logic against data stored locally. Instead of bring data to the application, the application (or function) is moved to the location where data is stored. Moving data across a network has a significant impact on performance, and by avoiding this, near linear scalability is achieved. Increasing data processing requirements can be accommodated simply by adding more computers.

Hadoop is the term used to describe this distributed filesystem(HDFS) and data processing engine that can be used to handle extremely high volumes of unstructured data at Internet scale. These group of computers make up a Hadoop cluster. Each computer in a Hadoop cluster is referred to as a node. The programming model used to bring the function (application logic) to the data is known as Map Reduce.

Technologies like Hadoop is what enables companies like Facebook, Google and Yahoo! to store millions of digital images and elements of our conversations, without having to design or understand up front, the format of the information or content they need to handle. This flexibility and ability to scale in a near linear fashion is one of the key attraction of Hadoop. Yahoo! reportedly has over 40,000 nodes spanning its Hadoop clusters which store over 40PB of data.