Driving Innovation starts with enabling Developers

Arrows_2_HiRes.jpg

Over the past few years, startups and small businesses have been threatening and attacking traditional, established enterprises in an unseen aggressive fashion. As an example, Netflix has drastically disrupted business of traditional cable and video companies by providing an on-demand streaming platform. Undeniably, Netflix has changed the media market and also shaped how customers want to consume media. On the business side, e.g., Salesforce has been heavily disruptive to traditional CRM software vendors. When analyzing why these companies disrupted whole industries, it usually boils down to one single fact: Their ability to innovate at very high pace. This has been made possible by service oriented and highly scalable platforms on which their developers built new products. Often, enterprise IT is struggling with offering adequate services. However, there is a way how enterprise IT can become the enabler for developers and innovation.

 

When we look at the industry, most applications on the market (especially web facing products) consist of (at least) the following components:

  • Webserver
  • Database
  • Object-based storage
  • Load Balancer and algorithms/scripts on the infrastructure to dynamically scale the infrastructure
  • Messaging components
  • Etc.

Furthermore, developers use a set of tools to successfully develop software:

  • Source code repositories
  • Continuous integration software
  • Testing frameworks for automated testing
  • Mechanisms to deploy the software
  • Etc.

Until a couple a of years back, those components have been setup up from scratch for each new project, over and over again. Having been a developer myself for many years, I eventually stopped counting how often I had to deploy subsets of these components.

In the late 2000's, Amazon Web Services and Google App Engine came along. With a few clicks (or better: with a few API calls) it was possible to instantiate those components within seconds. Making those components work together just took another few API calls. Embracers of those cloud services were mostly smaller startups. By using those cloud services, they suddenly had a large advantage over enterprise IT: Within a few minutes, they were able to start developing business-relevant code and applications. At the same time, developers in enterprise IT had to deal with three different parties: the server department, the networking guys, and the storage admins. After days or weeks, they finally had been provided with the bare resources to start developing on. But hold on, no, they did not start developing right away - they first had to set up their platform and install the relevant components. Finally, weeks or months later, they started writing business-relevant code. By then, startups already had running products and were grabbing large chunks of market share.

 

Even if developers in enterprises would have been instantly provided with the required resources (servers, network, storage), they still would have had a disadvantage. Bringing up their platform to work on still would have taken them weeks. Not only did they have to manager their code, they also had to manage their infrastructure and platform. The cloud vendors just offered a self-service web interface or an API endpoint and startup developers were able to get resources within seconds. Nowadays, it is more important than ever to give enterprise developers those same capabilities in house. If not, innovation will slowly vanish as enterprise developers are not able to keep up with the pace of the industry.

 

A_Day_In_The_Life_2.pngPlatform as a Service (PaaS) is the concept of providing developers those tools and components in an automated and self-service fashion. This enables developers getting to the point where they are implementing applications and business logic much faster, rather than spending time on setting up the application infrastructure themself. Furthermore, it enables developers to leverage pre-built services (e.g., webservers, databases, load balancers, etc.) without requiring in-depth knowledge on how those components exactly work. Rather than spending large chunks of cash on market research, those tools give developers platforms to quickly develop prototypes, which can be tested against the real target market.

 

Google App Engine and AWS are most likely the leading players in offering public cloud based PaaS services. While those public PaaS solutions are very powerful and simple to use, they usually create a deep vendor lock-in with the cloud provider. This is something that especially enterprises are more and more trying to avoid. Migrating an application built on top of AWS to e.g., Google App Engine or a private cloud may require re-architecting the application in the worst case. Instagram is just one example of how labor-intensive a migration from AWS to a private cloud can be (see here). Migrating more complex applications is most likely even more challenging. In many cases, migration will be driven from the business side in order to achieve cost savings: Prototyping and deploying a new application in a public cloud is initially fast, simple, and usually fairly cheap, as no hardware needs to be purchased. However, as soon as the application reaches widespread adoption, it can become interesting to move the application back to the on premise datacenter. This is mostly because of the lower TCO when operating applications on large scale in house.

 

The good news is that many open source PaaS solutions have appeared on the market over the past few years. They have the advantage that they can be deployed on public clouds (e.g., AWS), but also on private clouds (e.g., OpenStack). Some examples for open-source PaaS include Cloud Foundry, OpenShiftApache Stratos which can be deployed on either private or public clouds. Relying not on the cloud provider's PaaS capabilities, but rather using a standalone product, has several benefits: On the one hand, it simplifies migration dramatically and gives a lot more flexibility, especially when it comes to fine tuning performance. On the other hand, owning and having complete control of the PaaS solution can be appealing, especially for enterprises. When relying on a public cloud vendor's PaaS solution, dependence on the cloud vendor in case of an outage may not be acceptable. Owning the solution gives the ability to fix the issues on one's own and may also exhibit a lower TCO (most likely when deployed on a large scale). 

 

However, there are also cloud specific solutions appearing on the market. Those solutions are tightly coupled with the underlying cloud solution and can not be deployed on different infrastructures. For example, Project Solum is an ongoing project of building an OpenStack-native PaaS solution. In this case, integrating directly into OpenStack can yield many efficiencies and has the chance to make Solum extremely competitive to Google App Engine and AWS - just in your own datacenter. Solum is built bottom up, and, one day it could theoretically even become a part of OpenStack. However, at the same time, this creates a PaaS lock-in, which won't allow the applications built on the platform to run on other clouds (e.g., AWS or Azure).

 

Knowlege_Tree_1_HiRes.jpgIt is still unclear which of the PaaS models will become the dominant one: deeply integrated, cloud native PaaS solutions or standalone, cloud independent solutions. Most certainly, businesses will drive the requirements for the applications and the discussions will focus on points like migration capabilities, ease of operation, security, cost, and last but not least, about the features of the solution.

Regardless of the outcome, PaaS will and is becoming an inevitable component in enterprise IT and will become a cornerstone in enabling and accelerating innovation across enterprises.


If we drill down another layer in the enterprise datacenter, Infrastructure as a Service adoption is rapidly increasing. Infrastructure as a Service (IaaS) is the idea of making compute, networking, and storage resources available to end consumers via an UI or APIs. Nowadays, OpenStack is on the way to becoming the de-facto standard in open source IaaS solutions. Enterprises across all industries are on the way of adopting OpenStack and the list of users keeps on growing (see OpenStack User Stories). The OpenStack User Survey from November 2014 at the OpenStack Summit in Paris, gives insight into what is happening in the industry. The top drivers why companies are adopting OpenStack are the following:

  1. Increase ability to innovate
  2. Open technology
  3. Cost savings
  4. Avoiding vendor lock-in

While embracing open technology, achieving cost savings, and avoiding vendor-lockin, the most important piece for OpenStack adopters is the ability to innovate. This brings us back to the initial point: Enterprise IT needs to give developers the power and the platform to work like if they were developing on Google or AWS. OpenStack is the first step in providing such a platform.

 

Out of a big pool of resources, OpenStack gives developers the ability to quickly provision these resources in a self-service fashion:

  • Virtual Machines
  • Storage (Object-based, file-based, block-based)
  • Networking (to glue it all together)

Those components alone do not drive innovation. However, an IaaS solution like OpenStack is the first step as it serves as the base for a PaaS solution. This gives us the following picture:

 

iaaspaasapp.png

 

Applications, being used by the end user, are or will be sitting on top of the PaaS solution. The PaaS solution is consumed by developers and the devops teams through APIs. The PaaS layer sits on top of the IaaS layer. The IaaS layer is either directly integrated into the offerings (in case of a public cloud provider), or is deployed on top of an in-house IaaS solution like OpenStack. Those in house IaaS solutions then run on top of servers, networking and storage gear in the own datacenter. The IaaS layer and the hardware is operated by the IT staff. Important are the downward facing API dependencies. Each layer uses the APIs of the underlying layer, down to the API of the public cloud provider or of the IaaS solution.

 

Enterprises are nowadays not only threatened by other enterprises, but, more than ever, also by small startups. The platforms used by startups enable their developers to bring products to the market without a lot of capital invest and preparation time, but rather enable them to quickly start prototyping. The IT of many enterprises is still too rigid to defend those attacks. They will have to adapt and offer their developers Platform as a Service solutions, thus give them the power to quickly prototype applications. The old way of spending large amounts of cash on market research will not allow enterprises to keep up. However, giving their developers the platform to quickly innovate and bring prototypes to the market will do so.

 

In the next posts, we'll look into how NetApp and our partners can help achieving those goals.

 

 Follow me on Twitter: @clemenssiebler