Dell to opensource software to ease Hadoop install & management

November 8, 2011

It wouldn’t be surprising if you were surprised to learn that Dell is developing software.  To say that this is an area we haven’t been known for in the past would be an understatement.  While we may not pose a direct threat to Microsoft any time soon, we have been coding in a few focused areas.  One of those areas is cloud installation and management and is represented by our project Crowbar.  While Crowbar began life simply as a way to install Openstack on Dell hardware, it has expanded from there.

Today’s news is that we have developed and will be opensourcing “barclamps” (modules that sit on top of crowbar) for: Cloudera CDH/Enterprise, Zookeeper, Pig, Hbase, Flume and Sqoop.  All these modules will speed and ease the deployment, configuration and operation of Hadoop clusters.  But don’t take my word for it.  Take a listen to Crowbar’s architect Rob Hirschfeld as he explains Crowbar and today’s announcement:

Look for the code on Crowbar GitHub repo by the last week of November.  If you want to get involved, learn how.

Extra-credit reading:

Pau for now…


Crowbar: Where its been and where its going

October 24, 2011

Rob Hirschfeld, aka “Commander Crowbar,” recently posted a blog entry looking back at how Crowbar came to be, how its grown and where he hopes it will go from here.

What’s a Crowbar?

If you’re not familiar with Crowbar, its an open source software framework that began life as an installation tool to speed installation of OpenStack on Dell hardware.  The project incorporates the Opscode Chef Server tool and was originally created here at Dell by Rob and Greg Althaus.  Just four short months ago at OSCON 2011 the project took a big step forward when, along with the announcement of our OpenStack solution, we announced that we were opensourcing it.

DevOps-ilicous

As Rob points out in his blog, as we were delivering Crowbar as an installer a collective light bulb went off and we realized the role that Chef and tools like it play in a larger movement taking place in many Web shops today: the movement of DevOps.

The DevOps approach to deployment builds up systems in a layered model rather than using packaged images…Crowbar’s use of a DevOps layered deployment model provides flexibility for BOTH modularized and integrated cloud deployments.

On beyond installation and OpenStack

As the team began working more with Crowbar, it occurred to them that its use could be expanded in two ways: it could be used to do more than installation and it could be expanded to work with projects beyond OpenStack.

As for functionality, Crowbar now not only installs and configures but once the initial deployment is complete, Crowbar can be used to maintain, expand, and architect the instance, including BIOS configuration, network discovery, status monitoring, performance data gathering, and alerting.

The first project beyond OpenStack that we used Crowbar on was Hadoop.  In order to expand Crowbar’s usage we created the concept of  “barclamps” which are in essence modules that sit on top of the basic Crowbar functionality.  After we created the Hadoop barclamp, others picked up the charge and VMware created a Cloud Foundry barclamp and DreamHost created a Ceph barclamp.

It takes a community

Crowbar development has recently been moved out into the open.  As Rob explains,

This change was reflected in our work on OpenStack Diablo (+ Keystone and Dashboard) with contributions by Opscode and Rackspace Cloud Builders.  Rather than work internally and push updates at milestones, we are now coding directly from the Crowbar repositories on Github.

So what are you waiting for?  Join our mailing list, download the code or ISO, create a barclamp, make your voice heard.  Who’s next?

Extra-credit reading:

Pau for now…


Structure: Learning about DevOps & Crowbar from Jesse Robbins

June 27, 2011

Last week on Day two of Structure the morning sessions ended with an  interesting discussion moderated by James Urquhart.  The session was entitled “DevOps – Reinventing the Developers Role in the Cloud Age” and featured Luke Kanies – CEO, Puppet Labs and Jesse Robbins – Co-Founder and CEO, Opscode.

After lunch I ran into Jesse and got him to sit down with me and provide some more insight into DevOps as well as explain what Opscode was doing with project Crowbar.

Some of the ground Jesse covers

  • (0:21) What is DevOps
  • (1:00) The shift that happens between developers and operations.  Writing code and getting it into production faster and how it shifts responsibilities between the two groups.
  • (2:52) Who are the prime targets for DevOps and how has this changed over time.
  • How DevOps began in web shops who needed to do things differently than legacy-bound enterprises.
  • How enterprises faced with greenfield opportunities are now embracing devops
  • (5:36) The crowbar installer which employs Opscode’s Chef and allows the rapid provisioning of an OpenStack cloud.

Extra-credit reading:


Citrix to launch OpenStack Distro (with a little help from Rackspace and Dell)

May 25, 2011

Today at Citrix Synergy, Citrix announced “Project Olympus,” their up coming OpenStack distribution.  In case you’re not familiar with it, OpenStack is an open source cloud platform based on the code from NASA’s Nebula cloud as well as Rackspace’s storage code.  The OpenStack project kicked of last summer and already has gathered support from over 60 commercial hardware and software vendors.

Mt. Olympus and the Cloud

Citrix’s OpenStack Distro

Citrix’s Project Olympus will produce a commercial distribution of the OpenStack infrastructure-as-a-service platform.  This “Olympian” distribution will be made up of two main components: a Citrix-certified version of OpenStack and a cloud-optimized version of XenServer.  While Citrix will lead with their Xen technology, thanks to OpenStack the distro will support all leading hypervisors.

Project Olympus is targeted at both public cloud providers as well as enterprise customers looking to build out private clouds.  The distribution will be available later this year.

But I want it now — The Citrix/Rackspace/Dell Early Access Program

For those who don’t want to wait until the official distribution is ready, don’t fret you can get started today through the Early Access Program (EAP).  The EAP is designed to help customers kick-off pilots and proof-of-concept deployments.  The program provides access to a beta version of the Citrix distro plus Dell hardware and deployment software as well as deployment services, training and on going customer support for customer clouds via Rackspace’s Cloud Builders program.

To get going with Citrix’s Project Olympus Early Access Program, please visit  http://www.citrix.com/olympus

Dell, Crowbar and Reference Architectures

Dell’s above-mentioned deployment software, aka “crowbar,” was a big hit at the last OpenStack Design summit.  The software which leverages Opscode’s Chef, allows folks to get an Openstack cloud up in running in less than four hours (instead of  days).  In addition to the deployment software and systems, to support the project Olympus EAP, Dell will also be providing reference architectures so keep your eyes peeled for those.

If you have any questions about what Dell is doing with OpenStack or want to get started, email us at OpenStack@Dell.com.

Extra-credit reading

Pau for now…


OpenStack Installer Demo at SXSW

March 15, 2011

Last week and this, Austin’s downtown has been taken over by the South by Southwest festival (SXSW).  What started out nearly 25 years ago as a music festival/conference has grown to include parallel film and interactive events as well.  During the event every bar and venue downtown is occupied with bands, films or tech companies showing their stuff.

Yesterday Rackspace commandeered the Kung Fu Saloon in the name of the OpenStack project.  As part of this event, and before the drinking began, Dell did a demo of “Crowbar,” our OpenStack installer that we recently announced.

Check out the mini montage below that includes a quick interview I did with Dell solutions engineer Greg Althaus right after he finished the demo.

Don’t take our word for it

Dell is currently field testing crowbar and plans to donate the code to the OpenStack community after testing.  If you are interested in testing crowbar yourself, email us at OpenStack@Dell.com.

Some of the ground the video covers

  • Intro montage: Welcome to the Kung Fu Saloon, setting up and a snippet of the demo
  • [0:40] Talking to Greg
    • What actually is crowbar and how does it work with OpenStack compute and storage?
    • How fast can you spin up a cloud using it?
    • Where does OpsCode’s Chef fit in?
    • Our plan to donate this code to the community after field testing.  

Extra-credit reading

Pau for now…


OpenStack heads into Interstellar Overdrive

March 8, 2011

One small step for OpenStack, one giant step for mankind!  Ok, that may be taking it a bit far but today there is some exciting news that Dell and Rackspace have announced.

(I grabbed this image from a blog by varundev kashimath)If OpenStack is new to you it’s the open source cloud platform based off of the Nebula cloud code base donated by NASA (hence all the rocket allusions) and the storage code base from Rackspace.  The project is now about eight months old and its community is over 50 organizations strong including NASA, Citrix, AMD, Intel, NTT, cloud.com, RightScale, Dell, Opscode and Puppet labs.

(Graphic source: varundev kashimath’s blog)

The News

To help the community and organizations get up and going building out their own OpenStack based clouds we are announcing the following:

  1. Proof of Concepts: Rackspace and Dell are working together to deliver an OpenStack solution to market later this year.  As the first step along the way we are currently seeking customers for OpenStack proofs of concept (POC).  If you are interested in learning more about the POCs and possibly participating or getting engaged, email: OpenStack@Dell.com.
  1. Code: Dell has developed an OpenStack installer that can be used to quickly install OpenStack on bare metal Dell PowerEdge C servers.  This tool is being field tested, and Dell will contribute the code to the OpenStack community once testing is complete.  To help users out further, once the OS is installed, the servers can be updated with all the latest packages and be automatically configured by Opscode’s Chef to provide their services in the OpenStack infrastructure.
  2. Technical whitepaper: A technical whitepaper, “Bootstrapping OpenSource Clouds” which lays out the design of a hyperscale OpenStack cloud on Dell PowerEdge C servers.

But wait, there’s more, “Cloud Builders”…


To help support OpenStack deployments, Rackspace announced today that its launching  a new division, “Cloud Builders.”  Cloud Builders “will provide training, certification, deployment services and ongoing support and management to enterprises and service providers via the team of OpenStack experts and its partner network.”

Specifically the offerings will include (taken from the Rackspace blog):

  • Training & Certification – Rackspace Cloud Builders will provide training classes and certification testing for designing, developing and administering OpenStack Clouds. Rackspace also plans to empower a network of training and certification experts.
  • Deployment Services – Rackspace Cloud Builders, in conjunction with other experts from the OpenStack community, will help customers design and deploy OpenStack Clouds.
  • Support & Management – Customers will have ongoing access to remote support and escalation assistance from the OpenStack specialists, including proactive monitoring and fixes.

OpenStack is picking up momentum, stay tuned for more and in the meantime, set the controls for the heart of the sun! :)

Extra-credit reading and resources:

Pau for now…


Follow

Get every new post delivered to your Inbox.

Join 95 other followers

%d bloggers like this: