The Apache CloudStack Blog
Two late-announced security advisories
Today I sent out two CloudStack-related security advisories: CVE-2015-3251 (related to VM credential exposure) and CVE-2015-3252 (related to VNC authentication). Details about these issues can be found on the CloudStack user and dev mailing lists, as well as on the Full Disclosure and BUGTRAQ security mailing lists.
While these vulnerabilities are of moderate and low severity (respectively), the reason for this post is because the advisories were announced approximately 5 months after the first release of the patches in 4.5.2. This is personally embarrassing, unacceptable, and in a more severe case could be downright dangerous.
The CloudStack security team worked through the related vulnerabilities through the summer of 2015. We had advisory drafts, patches, and mitigations all ready well before the release. Far enough ahead, actually, that we forgot about the release and weren't paying attention to the release (at least I wasn't - I know others were), and didn't send out the advisories at the appropriate time. Part of this is due to me having become an unofficial lead/spokesperson for the security team; In the past there has been at least one occasion when others released advisories when I was not available, but usually I'm coordinating issues and publishing announcements.
Luckily, the CloudStack Security Team works with and under the direction of the ASF security team. During one of their periodic reviews, they noticed CloudStack had loose ends on these two advisories, and asked for an update. Earlier today I realized the advisories had not been released, so here we are.
How will we improve?
Obviously, we don't want to be in this situation again. Here's some steps we're taking to minimize the chance of a repeat performance:
- I've modified the Release Procedure to specifically request the release manager give the security team a heads up that a release is about to be announced. This can be a simple non-blocking email that shouldn't slow down the release process, but still ensure that we're aware of the upcoming release.
- I'll be ensuring that other members of the security team feel comfortable crafting and releasing advisories. Like the rest of CloudStack and other ASF projects, the CloudStack security team does not have a named leader and should be able to operate if I or others are unavailable.
In the past I've referred to CloudStack as "critical infrastructure" - CloudStack powers infrastructure clouds for many large cloud providers. We take information security seriously, realizing that many depend upon our work. Vulnerabilities happen in most software at some point in time - the important part is how they are responded to. While in this case we did respond quickly to the issues and created and applied patches, we let the community down by not quickly releasing the advisories. This is an unfortunate chink in our armor, but we'll be taking steps to ensure it doesn't happen again.
Apache CloudStack 4.6 is released
The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today the availability of Apache™ CloudStack™ v4.6, the turnkey Open Source cloud computing software platform used for creating private-, public-, and hybrid cloud environments.
Apache CloudStack clouds enable billions of dollars' worth of business transactions annually across their clouds, and its maturity and stability has led it to has become the Open Source platform for many service providers to set up on-demand, elastic public cloud computing services, as well as enterprises and others to set up a private or hybrid cloud for use by their own employees.
"This 4.6 release of Apache CloudStack marks a significant shift in how we release CloudStack," said Sebastien Goasguen, Vice President of Apache CloudStack. "With a focus on quality and speed of releasing software, we implemented a new release workflow which allows us to have a production-ready release branch all the time, and allows us to quickly release new features. From now on, CloudStack will be released much faster without regression and with increased quality in each version."
Recognized as the Cloud orchestration platform that "just works", CloudStack includes an intuitive user interface and rich APIs for managing the compute, networking, software, and storage infrastructure resources.
Under the Hood
CloudStack v4.6 reflects dozens of new features and improvements, including:
- NuageVsp Network Plugin
- Bind integration with Globo DNSAPI
- SAML 2.0 Plugin
- Managed storage for KVM
- Improved CloudByte Storage Plugin
- Use SSH for commands sent to Virtual-Router
- Baremetal Advanced Networking Support
- Instance Password Generation length can now be changed
A complete overview of all new enhancements are detailed in the project release notes
CloudStack v4.6 reflects more than 200 bug fixes from previous releases.
Apache CloudStack is in use/production at thousands of organizations worldwide that includes BIT.Group GmbH, BT Cloud, China Telecom, CloudOps, DATACENTER Services, DataCentrix, Datapipe, EVRY, Exaserve, Exoscale, IDC Frontier, iKoula, Imperial College, INRIA, KDDI, Korea Telecom, LeaseWeb, M5 Hosting Inc., Melbourne University, Reliable Networks, Redbridge, SafeSwiss Cloud, Schuberg Philis, ShapeBlue, Tranquil Hosting, Trader Media Group, University of Cologne, and the University of Sao Paulo, among others.
"With the 4.6 release the Apache CloudStack continues to mature adding important new functionality that will benefit CloudCentral's customers," said Kristoffer Sheather, Founder & Chief of Australian cloud services provider CloudCentral, who have been using Apache CloudStack since 2010. "The new Redundant Routers for Virtual Private Cloud (VPC) networks feature will ensure continuous availability of customer VPC networks, and Browser Based Template & Volume Upload will make it easier for our customers to import and use their choice of operating system ISO images and import VM templates from other cloud systems."
"CloudOps is very excited about the release of Apache CloudStack 4.6, which represents significant improvements in feature set and quality," said Ian Rae, CEO of CloudOps. "We are proud of our involvement in this landmark release and look forward to supporting our customers achieve operational success in upgrading to and operating clouds based on this release. Apache CloudStack is the best kept secret in open source cloud computing and has a global user base of cloud operators many of whom contribute to the project."
"The 4.6 release of Apache CloudStack brings new features and fixes bugs which are critical for our Aurora cloud offering at PCextreme," said Wido den Hollander, CEO of PCextreme. "We've worked hard with the community to get 4.6 released. The committers working at PCextreme resolved multiple issues and also introduced new features in CloudStack including a new StatsCollector output to Graphite and better support for CEPH. This new release allows us to grow our cloud even further."
"I'm very excited with launch of the Apache CloudStack version 4.6," said Cyrano Rizzo, CIO of the University of Sao-Paulo. "This version brought many new features and benefits, such as the case in resilience with the new redundant router for VPC, the capability to rapid deployment, demo and test to run the Apache CloudStack inside Docker that will speed the growth, the possibility to manage the resources with Graphite, the ease of upload templates and volumes, among many others, this version also brought many improvements, I'm very happy with one in particular that makes SAML plugin to production grade, this functionality is helping me to build a huge project called interCloud that intend to federate many public universities across the Brazil with Single Sign On."
Apache CloudStack welcomes contribution and community participation through mailing lists as well as attending face-to-face MeetUps, developer trainings, and user events. Catch Apache CloudStack in action at the next CloudStack European User Group on 3 March 2016 in London
CloudStack and OpenSSL CVE-2015-1793
Updated July 11th, 2015:
After reviewing CloudStack components and seeing Debian's advisory on CVE-2015-1793 (CloudStack's "system VM" is Debian based), it looks like CloudStack is not affected by this vulnerability.
Original post follows...
On the 9th of July, the OpenSSL project announced a high severity vulnerability within the OpenSSL library. While this particular vulnerability does not seem to affect SSL servers, there are security issues with SSL clients powered by OpenSSL. Because of this, we suspect there may be issues with parts of CloudStack which initiate SSL connections.
At this point we are still reviewing which particular versions of OpenSSL are used by different versions of CloudStack. Once this review is complete, we will further update the community and this post as to our next steps.
CloudStack and the "Ghost" glibc vulnerability
UPDATE: mitigation instructions have been improved (don't update openswan) and we forgot to mention rebooting.
UPDATE: Links to updated System VM templates are now below
Yesterday, a buffer overflow vulnerability was announced in glibc that affects most current Linux distributions. In CloudStack, the system VMs contain a vulnerable version of glibc.
CloudStack community members have built an updated system VM template, which ShapeBlue is hosting at http://packages.shapeblue.com/systemvmtemplate/ (More information on the packages at http://shapeblue.com/packages).
For instructions on how to update the SystemVM template in CloudStack, see here.
For those who wish to patch their running system VMs, ssh into each one and run:
apt-mark hold openswan apt-get clean apt-get update && apt-get upgradeAfter updating glibc, the system will need to be rebooted.
Information about how to connect to your System VMs is available here.
Other CloudStack-related systems may be affected!
Please review security updates from Linux distributions you use on your management server, storage systems, hypervisors, as well as other Linux VMs and bare-metal systems running in your environments. This post provides instructions for determining if a system is vulnerable, as well as patching directions for common Linux distributions.
[CVE-2014-7807] Apache CloudStack unauthenticated LDAP binds
CVE-2014-7807: Apache CloudStack unauthenticated LDAP binds
The Apache Software Foundation
Apache CloudStack 4.3, 4.4
Apache CloudStack may be configured to authenticate LDAP users. When so configured, it performs a simple LDAP bind with the name and password provided by a user. Simple LDAP binds are defined with three mechanisms (RFC 4513): 1) username and password; 2) unauthenticated if only a username is specified; and 3) anonymous if neither username or password is specified. Currently, Apache CloudStack does not check if the password was provided which could allow an attacker to bind as an unauthenticated user.
Users of Apache CloudStack 4.4 and derivatives should update to the latest version (4.4.2)
An updated release for Apache CloudStack 4.3.2 is in testing. Until that is released, we recommend following the mitigation below:
By default, many LDAP servers are not configured to allow unauthenticated binds. If the LDAP server in use allow this behaviour, a potential interim solution would be to consider disabling unauthenticated binds.
This issue was identified by the Citrix Security Team.
Posted at 08:11PM Dec 08, 2014 by jlk in News | |
The Apache CloudStack Project Announces Apache™ CloudMonkey™ v5.3.0
The Apache CloudStack project is pleased to announce the 5.3.0 release of CloudMonkey, the command line interface tool for Apache CloudStack configuration and management.[Read More]
CloudStack's realhostip service to retire in less than a week!
Citrix is reporting that they are still seeing DNS queries against the domain; Those who have not reconfigured their CloudStack installations will find part of their installations breaking once the realhostip service is retired on September 30th.
If you are running a version of CloudStack older than 4.3 and you have not reconfigured your installation to not use realhostip.com, please take the time to do so now before users are affected. Instructions are available in the CloudStack Wiki as well as other blogs on the Internet.
Posted at 06:41PM Sep 26, 2014 by jlk in News | |
Realhostip Reprieve for CloudStack Users
As mentioned previously, the realhostip.com dynamic DNS resolver service is being retired this summer. During testing of Apache CloudStack version 4.3, we found a few more issues related to realhostip.com that have been addressed for the 4.4 release.
In order to give everybody a reasonable window to update their CloudStack installations to use the updated code, the retirement date for the realhostip.com service has been pushed back to September 30th, 2014. This provides an additional 3 months from the original June 30th date.
Any questions related to the retirement of the realhostip.com service and it's affect on CloudStack installations should be send to the CloudStack Users or Development mailing lists. Further information about how to subscribe and interact with the mailing lists is available at https://cloudstack.apache.org/mailing-lists.html.
[CVE-2013-6398] CloudStack Virtual Router stop/start modifies firewall rules allowing additional access
Product: Apache CloudStack
Vendor: Apache Software Foundation
Vulnerability type: Bypass
Vulnerable Versions: Apache CloudStack 4.1.0, 4.1.1, 4.2.0
CVE References: CVE-2013-2136
Risk Level: Low
CVSSv2 Base Scores: 2.8 (AV:N/AC:M/Au:M/C:P/I:N/A:N)
The Apache CloudStack Security Team was notified of a an issue in the Apache CloudStack virtual router that failed to preserve source restrictions in firewall rules after a virtual router had been stopped and restarted.
Upgrading to CloudStack 4.2.1 or higher will mitigate this issue.
This issue was identified by the Cloud team at Schuberg Philis
Posted at 02:00PM Jan 10, 2014 by ke4qqq in News | |
[CVE-2014-0031] CloudStack ListNetworkACL API discloses ACLs for other users
Product: Apache CloudStack
Vendor: Apache Software Foundation
Vulnerability type: Information Disclosure
Vulnerable Versions: Apache CloudStack 4.2.0
CVE References: CVE-2014-0031
Risk Level: Low
CVSSv2 Base Scores: 3.5 (AV:N/AC:M/Au:S/C:P/I:N/A:N)
The Apache CloudStack Security Team was notified of a an issue in Apache CloudStack which permits an authenticated user to list network ACLs for other users.
Upgrading to CloudStack 4.2.1 or higher will mitigate this issue.
This issue was identified by Marcus Sorensen
Posted at 02:00PM Jan 10, 2014 by ke4qqq in News | |
Apache CloudStack Weekly News - 4 September 2013
Welcome back to another exciting issue of the Apache CloudStack Weekly News. This week, 4.2.0 enters it's fourth round of voting, we welcome several new committers and look at some of the major discussions on the Apache CloudStack mailing lists, and much more.
4.2 is Now being Voted On
The fourth round of voting is now open on the 4.2 release. This release is full of new features, fixes and thousands of hours of work from everyone in the community. It's important to test and cast your vote on the release. Remember that all members of the community are eligible to cast a vote and note any issues that they have with the current release candidate.
4.2 Issues Closure
Sudha Ponnaganti has throughout the 4.2 put together a list of the the current blocker and critical issues that need to be reviewed. If you have issues that have been resolved please review, test, and close out please.
High Quality Documentation
For some time now there has been discussion around a possible replacement to our current use DocBook for our primary document editor. Sebastien Goasguen started a discussion to look at Markdown by Daring Fireball. With there being concern about how to create and maintain high quality documentation, this is an important thread to participate in for anyone interested in the release documents.
After seeing lots of frustrated people with folks I decided to try something out with markdown.
I used pandoc to convert some docbook files to markdown and I used a structure for a book based on 'The little mongodb' book.
We can generate epub and pdf using latex.
There are two "books" aimed at being step by step recipes. Not long, not convoluted, single OS, etc…simple step by step.
I am still sanitizing the installation one based on 4.2 .
Comments, flames ?
Speaking in Tech Podcast - The Register
Aaron Delp joined in as a part of talking cloud and especially CloudStack as part of an interview with The Register and their "Speaking in Tech" podcast series.
Aaron's section on ACS is from 17:45 to 26:00 - http://www.theregister.co.uk/2013/08/01/speaking_in_tech_episode_69/
CloudStack Appliances Released
Ilya Musayev a committer of the ACS project and founder of CloudSands project has recently announced the release of a set of pre-built management server appliances available for open use based off the ACS 4.1.1 code base. There are appliances for VMware, Xen and KVM hypervisors.
Objective: Speed up the Apache CloudStack adoption by abstracting the need of going through install process and using pre-installed package instead. Especially useful for a quick POC.
1 CPU x 2 GB of RAM
Please spend few minutes on testing these out, you can import it as a template into your ACS - power on and see the details on initial start.
I've tested vSphere and KVM version. I don't have XEN instance to try.
- Build a Cloud Day - London being hosted by Sebastien Goasguen, being held on September 5.
- Cloud Plug Fest offers a variety of Tutorials and sessions, including OpenStack and CloudStack, in Madrid, Spain September 16-20.
- Build a Cloud Day - Switzerland has Sebastien Goasguen teaching you and helping you build clouds across Europe on September 26.
- CloudStack Collaboration Conference planning is well underway for Amsterdam, Netherlands. Put it on your calendar now for November 20-22.
New Committers and PMC Members
- Ilya Musayev has been invited to join the CloudStack PMC, and has accepted.
- Vijay Bhamidipati has been invited by the PMC to become a committer and has accepted.
- Toshiaki Hatano has been invited by the PMC to become a committer and has accepted.
- Kirk Kosinski has been invited by the PMC to become a committer and has accepted.
- Ian Duffy has been invited by the PMC to become a committer and has accepted.
Announcing the CloudStack Collaboration Conference - Europe
With two very successful events in the United Stated we know it is time to bring this conference to Europe. This time we’re gathering the community in The Netherlands. More specific, right in the center of Amsterdam in one of its historical landmarks, the Beurs van Berlage.
Starting November 20th with a hack day and continuing with a two day conference, this will be your opportunity to dive into all things CloudStack. Meet the community, discuss new ideas and learn about existing and upcoming features. We have setup the conference to provide an exciting environment to participate in workshops, attend presentations or just sit back and have a drink with other CloudStack enthusiasts.
The Call for Papers is open right now, so send your abstract to email@example.com. If it’s relevant to Apache CloudStack development, deployment, and integration, we’re interested in what you might have to say. We can accommodate workshops, hack sessions, presentation and we want to work with you to make sure you can share what you want with the community. Check the website for more details, http://www.cloudstackcollab.org/call-for-papers
The conference website http://www.cloudstackcollab.org will be regularly updated with new content to keep you informed about the conference. Please check it regularly to be informed about the latest developments regarding the CloudStack Collaboration Conference Europe.
The Call for Papers will run from today (August 16th) to September 30th. We will send out notifications shortly after closing the the Call for Papers.
The Conference Hack Day will be November 20th
The Conference talks and planned sessions begin on November 21th
The Conference ends on November 22th
We will announce the registration in a short while, please keep an eye on the website http://www.cloudstackcollab.org/ for more details.
The conference will be at the Beurs van Berlage in Amsterdam, The Netherlands. Located in the city center it is close to quite a number of hotels and hostels in Amsterdam. We are looking at the possibility to make a deal with one of the hotels in the immediate vicinity of the conference location. We will update the conference website when we have the details.
Sponsoring opportunities are available for the CloudStack Collaboration Conference. At the conference website http://www.cloudstackcollab.org/sponsors some of our sponsors will explain you the benefits in a video message. If you’d like to see the sponsorship prospectus or ask about sponsoring, contact firstname.lastname@example.org.
We’re very pleased to invite the community to Amsterdam and we hope you’ll join us! See you in Amsterdam!
Apache CloudStack Weekly News - 24 July 2013
Welcome back to another exciting issue of the Apache CloudStack Weekly News. This week, we take a look at the progress towards 4.2.0, major discussions on the Apache CloudStack mailing lists, and much more.[Read More]
CloudStack Weekly News - 10 July 2013
The community is busy working on 4.2.0, and there's much to be done before the release is ready. This week, we're taking a look at some of the interesting discussions going on in the the community about the next generation of Apache CloudStack, and functionality we can provide, as well as procedural changes that everyone should be aware of.[Read More]
Apache CloudStack Weekly News - 1 July 2013
We are half way through the year and a lot of work is done, and lot more is yet to be done. This week we look back at some of the CloudStack Collaboration Conference, work continues on 4.1.1 and 4.2.0, and we have some interesting discussions on how we should release the CloudMonkey and Marvin tools used with CloudStack. There's a by-laws vote underway to look at how and where we decide non-technical issues, and some discussion on the best way to to discuss and do code reviews.[Read More]