Open source Crowbar code now available for Hadoop

November 29, 2011

Earlier this month we announced that Dell would be open sourcing the Crowbar “barclamps” for Hadoop.  Well today is the day and the code is now available at our github repo.

Whats a Crowbar barclamp?

If you haven’t heard of project Crowbar it’s a software framework developed at Dell that started out as an installation tool for OpenStack.  As the project grew beyond installation to include monitoring capabilities, network discovery, performance data gathering etc., the developers behind it, Rob Hirschfeld and Greg Althaus, decided to rewrite it to allow modules to plug into the basic Crowbar functionality.  These modules or “barclamps” allow the framework to be used by a variety of projects.  Besides the OpenStack and Hadoop barclamps written by Dell, VMware created a Cloud Foundry barclamp and DreamHost created a Ceph barclamp.

To help you get your bearings

As I mentioned in the opening  paragraph, the code for the Hadoop barclamp is now available.  To help you get started, below are a couple of videos that Rob put together.  The first walks you through how to install Crowbar and the second one explains how to use Crowbar to deploy Hadoop.

Extra-credit reading

Pau for  now…


Crowbar: Where its been and where its going

October 24, 2011

Rob Hirschfeld, aka “Commander Crowbar,” recently posted a blog entry looking back at how Crowbar came to be, how its grown and where he hopes it will go from here.

What’s a Crowbar?

If you’re not familiar with Crowbar, its an open source software framework that began life as an installation tool to speed installation of OpenStack on Dell hardware.  The project incorporates the Opscode Chef Server tool and was originally created here at Dell by Rob and Greg Althaus.  Just four short months ago at OSCON 2011 the project took a big step forward when, along with the announcement of our OpenStack solution, we announced that we were opensourcing it.

DevOps-ilicous

As Rob points out in his blog, as we were delivering Crowbar as an installer a collective light bulb went off and we realized the role that Chef and tools like it play in a larger movement taking place in many Web shops today: the movement of DevOps.

The DevOps approach to deployment builds up systems in a layered model rather than using packaged images…Crowbar’s use of a DevOps layered deployment model provides flexibility for BOTH modularized and integrated cloud deployments.

On beyond installation and OpenStack

As the team began working more with Crowbar, it occurred to them that its use could be expanded in two ways: it could be used to do more than installation and it could be expanded to work with projects beyond OpenStack.

As for functionality, Crowbar now not only installs and configures but once the initial deployment is complete, Crowbar can be used to maintain, expand, and architect the instance, including BIOS configuration, network discovery, status monitoring, performance data gathering, and alerting.

The first project beyond OpenStack that we used Crowbar on was Hadoop.  In order to expand Crowbar’s usage we created the concept of  “barclamps” which are in essence modules that sit on top of the basic Crowbar functionality.  After we created the Hadoop barclamp, others picked up the charge and VMware created a Cloud Foundry barclamp and DreamHost created a Ceph barclamp.

It takes a community

Crowbar development has recently been moved out into the open.  As Rob explains,

This change was reflected in our work on OpenStack Diablo (+ Keystone and Dashboard) with contributions by Opscode and Rackspace Cloud Builders.  Rather than work internally and push updates at milestones, we are now coding directly from the Crowbar repositories on Github.

So what are you waiting for?  Join our mailing list, download the code or ISO, create a barclamp, make your voice heard.  Who’s next?

Extra-credit reading:

Pau for now…


OpenStack Installer Demo at SXSW

March 15, 2011

Last week and this, Austin’s downtown has been taken over by the South by Southwest festival (SXSW).  What started out nearly 25 years ago as a music festival/conference has grown to include parallel film and interactive events as well.  During the event every bar and venue downtown is occupied with bands, films or tech companies showing their stuff.

Yesterday Rackspace commandeered the Kung Fu Saloon in the name of the OpenStack project.  As part of this event, and before the drinking began, Dell did a demo of “Crowbar,” our OpenStack installer that we recently announced.

Check out the mini montage below that includes a quick interview I did with Dell solutions engineer Greg Althaus right after he finished the demo.

Don’t take our word for it

Dell is currently field testing crowbar and plans to donate the code to the OpenStack community after testing.  If you are interested in testing crowbar yourself, email us at OpenStack@Dell.com.

Some of the ground the video covers

  • Intro montage: Welcome to the Kung Fu Saloon, setting up and a snippet of the demo
  • [0:40] Talking to Greg
    • What actually is crowbar and how does it work with OpenStack compute and storage?
    • How fast can you spin up a cloud using it?
    • Where does OpsCode’s Chef fit in?
    • Our plan to donate this code to the community after field testing.  

Extra-credit reading

Pau for now…


What’s happening — A Dell / OpenStack Update

September 15, 2010

A couple of days ago Bret Piatt, who handles Technical Alliances for OpenStack, came up to Austin to have further discussion with our team’s software engineers around OpenStack.  If you’re not familiar with OpenStack, it is an open source cloud platform founded on contributed code from Rackspace and NASA’s Nebula cloud.

The project was kicked off a couple of months ago at an inaugural design summit held here in Austin.  The summit drew over 25 companies from around the world, including Dell, to give input on the project and collectively map out the design for the project’s two main efforts, Cloud Compute and Object Storage.

Since the summit, and the project’s subsequent announcement the following week at the OSCON Cloud Summit, the community has been digging in.  The first object storage code release will be available this month and the initial compute release, dubbed the “Austin” release, is slated for October  21.  Additionally, the second OpenStack Design Summit has been set for November 9-12 in San Antonio, Texas, and is open to the public.

OpenStack visits Dell

During Bret’s visit to Dell he met with a bunch of folks including two of our software architects, Greg Althaus and Rob Hirschfeld.  The three talked about how things were going with the project since the summit as well as specific ways in which Dell can contribute to the OpenStack project.

Below you can see where I crashed the three’s whiteboard session and made them tell me what they were doing.  I then followed them, camera in hand, down to the lab where Greg and Rob showed Bret the system that we have targeted for running OpenStack.

Some of the topics (L -> R) Bret, Greg and Rob touch on:

  • Bret:  Getting ready for the object storage release in September and compute in October.  Looking to get the right hardware spec’d out so that people can start using the solution once its released.
  • Rob: Learning about how the project is coming together since the design summit.  Interested in how the 3 code lines, storage, NASA compute and Rackspace compute, along with the input that was gathered at the Design summit and community input, are coming together.
  • Greg and Rob take Brett to the lab to show him the C6100 which could be a good candidate for open stack.
  • Next step, getting OpenStack in the lab and start playing with it.

Extra-credit reading:

Pau for now…


Follow

Get every new post delivered to your Inbox.

Join 99 other followers

%d bloggers like this: