The first time I remember hearing about Eucalyptus, an open source product for building private clouds, was when Canonical launched “Ubuntu Enterprise Cloud” in Ubuntu 9.04, April of 2009. UEC, a version of Eucalyptus, allowed users of Ubuntu Server to build their own clouds behind the firewall.
Eucalyptus was developed by a team of academics headed by UC Santa Barbara Professor Rich Wolski, who had originally been researching grid computing. Wolski and his team had an application they wanted to demo on both Amazon Web Services and their custom university clusters, in order to do this they ported their application to AWS, and then wrote code to have their cluster imitate AWS. Seeing the commercial potential in this open source technology, the authors formed Eucalyptus Systems in late 2009. Eucalyptus Systems adopted an “open core” model, where an Open Source core is supplemented by proprietary extensions. Eucalyptus Systems hired ex-MySQL CEO Mårten Mickos, a prominent defender of open core licensing, to run the company.
The next major thing I heard about Eucalyptus was that everyone was jumping ship. The media reported that none less than NASA (the spiritual home of Rocket Scientists) had tried to make Eucalyptus work for their Nebula cloud platform project, but had failed to make it scale. On top of this, Nova, the replacement controller that NASA had built (and were willing to commit to the Eucalyptus open source project) was reportedly in direct competition with a component that Eucalyptus were building to sell in their proprietary version. Thus, in July 2010, NASA engineers and their partners, Rackspace, announced OpenStack, a new open source project which united the Nova controller and Rackspace’s storage systems under one umbrella.
OpenStack has certainly gone from strength to strength since its announcement. It has picked up major backers, such as Cisco, Dell, HP and (reportedly) IBM. Canonical announced in May 2011 that they would shift to OpenStack as the default cloud provider for Ubuntu Enterprise Cloud, while still offering Eucalyptus as an unsupported option. Citrix purchased Cloud.com, another AWS-compatible private cloud system, and has stated a goal to “bring them together“.
However, OpenStack has not been without its detractors. Governance of the project was uncertain until Rackspace announced the OpenStack Foundation in October 2011. Randy Bias of Cloudscaling.com said in November 2011 that “the desire [for an open cloud environment that scales] is ahead of the technology, and there’s more smoke than fire in OpenStack at the moment”. According to Gigaom.com, as of February 2012, Rackspace Cloud was still running on the original software platform they acquired from Slicehost.com, and still only had “hundreds” of instances in alpha test running on OpenStack. They have since moved into a beta test of “tens of thousands” of instances. Rackspace CTO John Engates told GigaOm that he hoped Rackspace-labeled OpenStack clouds would be available in Q3 or Q4 of 2012.
A number of commentators have criticized OpenStack’s creation of a net new API, rather than continued usage of the popular and “de-facto standard” EC2 API. Rackspace suggest that their new OpenStack APIs are better than the EC2 APIs, and it’s definitely true they are more modern. However, there’s presently a strong ecosystem of tools and a community of developers familiar with the EC2 API, which OpenStack is keeping around as a second-class citizen. There is talk in some OpenStack communities of a preference for completely deprecating the EC2 API; but this is not official policy.
Although Eucalyptus (and other entrants such as Cloud.com and Nimbula) used the AWS API without obvious permission, Amazon remained largely ambivalent toward its private cloud ‘competitors’, with CTO Werner Vogels dismissing private cloud as “little more than marketing”. Safe in the knowledge that their scale protected them from competition, and their pace of innovation was unlikely to be matched, Amazon gained the benefits of more tools that used their APIs. When Amazon eventually moved to adding more enterprise features, they were primarily along the lines of Virtual Private Cloud, allowing ‘legacy’ datacenter components to be connected to dynamically creatable, private sections of Amazon’s public cloud.
The tide started turning for Eucalyptus recently with two announcements; one that slipped under the radar, and one that arrived with a bang. Earlier this month, Mickos announced to Brian Proffitt of IT World that Eucalyptus Systems would be dropping the dual branch (one commercial, one open source) model, and returning to a single open source branch licensed under the GPLv3, in their upcoming 3.1 release. Eucalyptus’s VP of Community, Greg DeKoenigsberg elaborated in a blog post: the enterprise features were built into a parallel code base rather than as separate modules, for ease of integration, and it was always the company’s goal to merge them back together.
Proffitt also clarified the story of the NASA submission, recounting Mickos saying the four or five members of the Eucalyptus team who were dedicated to the NASA project did not communicate as effectively as they could have, and citing competing motivations within NASA. He acknowledges that the incident caused a lot of friction in the Eucalyptus community, but NASA continues to be a Eucalyptus customer, and that it was only the Nebula team that dropped the Eucalyptus software.
Eucalyptus Systems has been growing steadily while out of the press, claiming 25,000 cloud deployments, but under 100 paying customers. Their recent (commercial) 3.0 release included features such as IAM support and high availability, and they opened an office in EMEA in October.
The big news of the week for the cloud watcher is Eucalyptus’s March 22nd announcement that Amazon Web Services will formally support them in their use of the AWS APIs. DeKoenigsberg told ReadWriteWeb that “Amazon chose to partner with Eucalyptus because we were a great technical fit. Meeting AWS’s technical standards is a significant achievement, and we are extremely proud to be up to that lofty standard”. The agreement is described as non-exclusive, though DeKoenigsberg added “I personally don’t see anyone out there who provides the same technical ability to match the AWS API that we provide”.
DeKoenigsberg says, “In the world of web services, it’s all about the API. Securing an agreement with Amazon, in which they actively support the development of an open source reference implementation that supports the AWS API, is a monumental step forward in cementing the AWS API as the de facto API for cloud computing, be it public, private, or hybrid.” RedMonk analyst Stephen O’Grady believes “OpenStack is not likely to either ask for or receive the same blessing from Amazon that Eucalyptus received.”
The AWS and Eucalyptus agreement promises compatibility with what are considered the building block services of AWS: EC2, EBS, S3, IAM, and the AMI format. These supported services, while a small subset of the myriad services Amazon have on offer, still make up more services than are found in OpenStack, and are more than enough for many workloads. There hasn’t been a technical announcement about what this means for Eucalyptus – their implementation of these services exists today, and has for some time. Also, there isn’t a publicly available timeline yet for either the 3.1 release or any future enhancements made available by the Amazon partnership. The open source download pages only mentions up to version 2.0.3, though the source code is available for 3.0.
Today, many more business use Amazon Web Services than a private cloud. A product that is compatible with the AWS APIs, has commercial support available, and is blessed by Amazon, will be a much easier sell to higher ups in companies that have adopted AWS. Eucalyptus has just been given “top tier” billing by the only cloud player whose opinion really matters in the IaaS market, and formalized the “Amazon and Eucalyptus vs. OpenStack and Everyone Else” divide.
Does this development mean you’re more likely to adopt Eucalyptus? Respond in the comments below…