How do you work files securely in the cloud without decrypting them?

A group of IBM researchers has released a Github project that implements a homomorphic encryption system –a way to work on encrypted data in a file without first decrypting the whole file. 

Why would anyone want to do that? Partly because if you have to decrypt the file to work on it, it's going to exist as plaintext somewhere. IBM has other ideas about this as well: leaving the encrypted file encrypted would keep data protected in the cloud while still letting users work on it. 

More here 

What is Advanced Analytics?

A significant number of blog posts have been devoted to the subject of Big Data Analytics, so it is a good time to revert back to what Advanced Analytics delivers to the business.

I view Advanced Analysis as the use of data and models to provide insights to guide decisions. Advanced Analysis is therefore about data analysis, and for advanced analytics to be useful, quality data from trusted sources is critical. This is the key difference with Big Data analysis which is focused on large and varied formats of data, accumulating at high velocities.

Advanced Analytics requires structured data. Where there is structured data, Enterprise Content Management would help, but where there is no structure in the data, the analysis will be fundamentally flawed. The saying "garbage in, garbage out" holds true for Advanced Analytics.

Advanced Analytics begins with the analysis of data. Once analysed, it is presented in executive dashboards with advanced visualisation capabilities. There are 4 terms noteworthy terms in the definition of Advanced Analytics...data, models, insights and decisions.

With Advanced Analytics, mathematical algorithms are used to evaluate data a context relevant to the user. Complex mathematical models discover patterns in the data which would have previously been unknown.

Models are the complex mathematical formulae used to augment the data available to knowledge and insights. The purpose of the models is to uncover insights that drive better decision making.

I have been engaged in conversations with business leaders, demonstrating how PureApplication System can be used to accelerate the delivery of Advanced Analytics capabilities. Using patterns of expertise, we are able to significantly reduce time to value, enabling line of business executives to benefit from innovative IT capabilities in less than a couple of months, where it would have previously taken years to implement.

The Promise of Collaborative Development and Operations

I came across a  DeveloperWorks article by IBM Distinguished Engineers Ruth Willenborg and Steve Abrams where they discussed Collaborative Development and Operations.

In my view, this is about creating a collaborative development platform that aligns all stakeholders, e.g. line of business, development, testing teams, suppliers and users. The ultimate goal is to accelerate the delivery of services by the business to its customers in a way that is measurable and where continuous feedback can be provided for process improvement.

IBM PureApplication System is a platform developed with these principles in mind. On one end of the scale, you have the development organization working with business analysts to design, prototype and build software assets. These assets are integrated and consumed on PureApplication system which is optimized for  web application delivery and lifecycle management. The platform's cloud computing capabilities enable operators to interact with it in a self service manner, resulting in faster delivery of services to consumers and end users. 

In today's agile world of mobile application and services delivery, collaborative development and operations is about enabling continuous innovation by software development organizations  and continuous delivery of application services to end users and consumers. This requires a platform that supports collaborative development, continuos integration, continuos testing, continuous release management and continuous monitoring, all in an iterative lifecycle.

This article shows how applications can be on boarded onto PureApplication System using patterns of expertise. Patterns offer significant potential in enabling IT organizations to transform themselves and become better aligned to the business.

Part 2: PureSystems Analyst Roundtable Discussion

A key discussion topic for this week's Analysts roundtable is the PureData System for Transactions. 

PureData System for Transactions has been designed and built for clients with the highest demands for performance and availability. To many of our clients, such systems are core to their business and any downtime can result in loss of revenue, customer churn, tarnished image, etc.). PureData System is a  "Tier 1" system, designed for such mission critical applications.  

At the heart of PureData System for Transactions is DB2 pureScale, which provides extreme levels of availability and scalability, with application transparency. PureData System for Transactions inherits these capabilities to deliver high levels of reliability, performance and scalability out of the box. Existing DB2 applications are supported with no changes, and Oracle Database applications are also supported with minimal, and in some cases, no changes are necessary.

One of the most striking features of the IBM PureSystems family is the consistent, single web user interface through which all hardware and software components are managed and monitored. This significantly reduces the additional knowledge required to operate any offering within the PureSystems family, once a user is familiar with one of the offerings. Think about the effort required to transition from an iPod to an iPhone. Once a system is delivered, the PureData System for Transaction adoption process is as simple as

1. Power-on
2. Integrate system to the network
3. Create and deploy database clusters
4. Configure access and allocate storage to databases
5. Upload objects and data into the created databases

Part 1: PureSystems Analyst Roundtable Discussion

I have a date with a group of European Industry Analysts later in the week, and as I reflect on the key messages I would like to convey, I thought it might be useful to share my thoughts on this blog. I will start off in part 1 of the series with an introduction to the PureSystems family, the topic of interest to the Analysts.

You might recall that in April 2012, IBM introduced PureSystems with the first 2 family members, PureFlex System and PureApplication System. The PureData System joined the family in October 2012.  

The PureFlex System integrates computing, storage, networking resources and system management to simplify and accelerate the delivery of infrastructure services.  It is ideal for those clients who want to accelerate deployment and simplify management of infrastructure for building custom application platforms and application stacks. PureApplication System is an integrated application platform, delivering platform services that include pattern-based application stack deployment and management, web application serving and database management. PureData System is an expert integrated system optimized exclusively for delivering data services.

PureSystems address the challenges associated with an increasing complexity in the provisioning of IT infrastructures capabilities (storage, compute and networking) and associated web application middleware software; and the increasing volume, velocity and variety of data used today in all aspects of business.

Best practices and expertise are built into the system, reducing and in many cases, eliminating the need for operators to go through extensive documentation and best practice white papers to understand how to configure, tune, deploy and operate applications and data on the systems. 

To ensure clients are able to get value from the platform faster, system is designed, integrated and optimized for specific high performance applications in the factory. On delivery, the client simply focusses on integrating the system into their infrastructure, rather than spending time connecting the various pieces together and installing software. Standardization significantly increases time to value by eliminating the need for extensive installation and configuration. Additionally, because each PureSystem is design, configured and optimally tuned in the factory in a standard manner, troubleshooting is made easier. 

In today's highly competitive business environment, competitive advantage arises from being able to take advantage of capabilities faster that anyone else. While some might focus on delivering a vast array of features and functions, a key design principle of PureSystems is to simplified the user experience, exposing what matters most to system users and operators. This enables them to be most productive. With PureSystems, procurement is simplified with a single line of support and part number.  No assembly of components is required. Applications and data are ready to load application in hours. Open integration to 3rd party software is possible, with the whole systems managed through an integrated management console that greatly simplifies system upgrades and maintenance across the stack. 

PureSystems comes in different models that have been designed, integrated and optimized to deliver application and data services to today's demanding applications with simplicity, speed & lower cost. For more information can be found here.

NoSQL or Know SQL ?

I have spent the last few months exploring the Big Data landscape, discussing some of the technologies that are underpin the promise of Big Data like Hadoop and Map Reduce.

NoSQL is a set of capabilities that have emerged over the last few years, motivated by the fact that relational databases technologies are limited by the need to model an application up front, define a schema and operate around the schema. Proponents of NoSQL believe that this presents a significant limitation for some emerging classes of applications, particularly where the schema definition can not be determined in advance. It is important to note that NoSQL is a movement, as opposed to a specific technology. What motivates proponents of this technology is the need for SQL schema flexibility. The actual implementation varies, with proponents adopting various approaches, e.g. key store, document store, in-memory or graph oriented databases etc. 

Most of the clients I have interacted with understand the value of relational databases and the merits of being able to model applications and design around a specific scheme. This is how relational technologies are able to deliver scale, robustness, and transaction guarantees also referred to as the ACID properties. The dilemma clients have is how to retain the goodness of relational databases, while adopting elements of the NoSQL promise that has the potential to improve their overall effectiveness. This is where the notion of "Know SQL" comes in. In my view, it is not about the irrelevance of relational SQL based database technologies, but rather about knowing when more schema flexibility is desirable, particularly in the Big Data world.

Recognising this trend, IBM has been investing its relational database technologies, adding XML support into DB2 to deliver more schema flexibility, and introducing NoSQL Graph Support, also known as DB2-RDF. As a result, clients are able to achieve schema flexibility, without having to give up the proven capabilities and robustness of relational database engines. More information on DB2 NoSQL support can be found here.

What IT systems does a Smart City need?

A McKinsey Global Institute article "Urban world: Cities and the rise of the consuming class", reflects on the speed and unprecedented scale of city expansion. This is placing an increasing demand for new environmentally friendly infrastructure for water and energy, buildings, transportation and communication. Migration is motivated mostly by the search for a better life, and in additional to the infrastructure challenges previously outlined, this also places a strain on the cities to deliver citizen based services such as education, health, public safety, economic development and social programs.

Smarter city operations rely on the ability to capture data that can be used to anticipate and proactively resolve problems. Problem resolution can be achieved by coordinating processes and resources for more efficient operations. Take public safety as an example. IT systems can be deployed to predict, monitor and mitigate crisis situations. This can be achieved by automatically analyzing video streams for threats based on known criminal patterns. Similar capabilities can be applied to transportation for more effective traffic management, and for analyzing water use and consumption patterns, thereby enabling utilities to identify leakages and optimize repair jobs for improved service delivery.


The figure to the left is from  "Competing on Analytics, Davenport and Harris, 2007". 

To be competitive, Smart Cities need to deploy Analytics capabilities that enable the delivery of a high quality of services that meets and exceeds citizen expectations today, and  can accommodate future needs for real-time dynamic access to innovative new services. 

Additionally, these new services need to be resilient, secure, compliant with local requirements, and sufficiently agile to address new risks posed by an ever more connected and collaborative world. And all of this needs to be achieved cost effectively. IBM's PureData System for Analytics is designed specifically for this use case.

Imagine a situation where city operations can be collaboratively managed via Executive, City Operations and Agency dashboards that include domain key performance indicators for standard operating procedures. Such a system could support centralized planning, execution and monitoring for more efficient operations.  

IBM's Intelligent Operations Center delivered on a Cloud enabled PureSystems platform, enables cities to not only contain operational cost and complexity, but achieve breakthrough productivity gains through rapid time to value via virtualization, optimization, energy stewardship.