As I mentioned in my previous entry, the code for the Hadoop barclamps is now available at our github repo.
To help you through the process, Crowbar lead architect Rob Hirschfeld has put together the two videos below. The first, Crowbar Build (on cloud server), shows you how to use a cloud server to create a Crowbar ISO using the standard build process. The second, Advanced Crowbar Build (local) shows how to build a Crowbar v1.2 ISO using advanced techniques on a local desktop using a virtual machine.
I’m always interested in what’s happening at Canonical and with Ubuntu. Last week at Hadoop World I ran into a couple of folks from the company (coincidentally both named Mark but neither Mr. Shuttleworth). Mark Mims from the server team was willing to chat so I grabbed some time with him to learn about what he was doing at Hadoop World and what in the heck is this “charming” Juju?
Some of the ground Mark covers
Making the next version of Ubuntu server better for Hadoop and big data
(0:34) What are “charms” and what do they have to do with service orchestration
(2:05) Charm school and learning to write Juju charms
(2:54) Where does “Orchestra” fit in and how can it be used to spin up OpenStack
(3:40) What’s next for Juju
But wait, there’s more!
Stay tuned for more interviews from last week’s Hadoop world. On tap are:
Yesterday at Cloud Expo I bumped in to Dr. Rich Wolski, CTO and co-founder of Cloud player, Eucalyptus. It had been a while since we had last talked so I grabbed some time with him and got him to give me the skinny:
Some of the ground Rich covers:
Eucalyptus’s major release which is coming out in the next 4 weeks
[0:40] The RightScale myCloud integration that they announced yesterday (linking Eucalyptus private clouds with various public clouds)
[2:01] Eucalyptus’s relationship with Canonical and how their interests are diverging
[3:15] Where specifically Eucalyptus is targeted
[4:25] What are some of their goals and product features they’d like to add over the next year
Dell’s Data Center Solutions (DCS) group has some pretty colorful folks. One of the more interesting members is Jimmy Pike, the man IDG New’s James Niccolai refered to as the “Willy Wonka of servers.” Jimmy, the self-proclaimed “chief geek” of the DCS team is the consummate tinkerer whether that involves constructing a data center in a brief case or thinking of new ways of driving down data center power consumption by leveraging alternative forms energy.
Last Spring I visited Jimmy’s home to check out what he was working on in his “free time.” Here’s what I saw (he keeps telling me he’s got much cooler stuff since I shot this so I may have to do a “geekquel”)
Some of the things Jimmy show us:
The low-power chips he’s playing with
His experimentation with user interfaces and superman glasses
His mini rack of servers
The various forms of desktop virtualization and OS’s he uses
Yesterday, the announcement went out that the Dell | Canonical Enterprise Cloud, Standard Edition was out and ready for consumption. What this cloud-in-a-box allows folks to do is to set-up affordable Infrastructure-as-a-Service (Iaas)-style private clouds in their computer labs or data centers. The cool thing is that, because the Ubuntu Enterprise Cloud (UEC) software is compatible with Amazon Web Services EC2 and S3 services, it enables IT admins and developers to move workloads between public and private clouds.
Who cares?
Application developers and IT service providers and admins who are setting up cloud POC’s are perfect candidates for this pre-configured testing and development environment. With regards to industries, areas where there is a lot of software development work like Hosters, Telco & Communications, Media & Entertainment and Web 2.0 businesses are prime markets for the Dell UEC solution.
So what’s in it?
The solutions’ basic components are Dell PowerEdge C systems plus a Dell-specific download of the Ubuntu Enterprise Cloud (made up of the Ubuntu operating system and the Eucalyptus platform for private cloud computing). To simplify getting the whole shebang up and running Dell and Canonical are providing the following:
Walrus Controller (W) – the cloud’s storage repository
Cluster Controller (CC) – the controller for a up to 1024 compute cores grouped together as a cluster
Storage Controller (SC) – the controller for cluster’s storage repository
Compute Node (CN) – cloud’s compute node
And on the support side…
If you’re looking for systems management and support services with your order, you are in luck. Canonical, the company behind Ubuntu, has put together UEC Assist, a support service built specifically for Dell customers deploying SE Edition and which is delivered by Canonical’s Global Services and Support team.
Its all about efficiency
From a Dell DCS (the group at Dell behind this) point of view, this offering fits in well with our strategy of bringing total solutions to market that optimize efficiency at every layer, from code to servers to storage. The open source Dell UEC solution is tailor made to deliver a ready to go IaaS solution.
Mark Shuttleworth, the ever gracious founder of Ubuntu, stopped by Dell this morning to talk to various folks about various subjects. I was able to grab some time with him between meetings and get his thoughts on a few topics.
I was particularly interested in getting his thoughts on Ubuntu Enterprise Cloud (UEC) which will be available this week from Dell as the Dell | Canonical UEC Solution (along with the UEC software, the solution is based on our PowerEdge C2100 and C6100 and comes with a reference architecture and deployment guide). The other topic I wanted to get his insight into was OpenStack.
Here’s what Mark had to say:
What Mark talked about
How Mark has settled into his role as non-CEO (he is still chairman).
What he is focusing on these days: the cloud and product design e.g. Unity.
[1:45] The thinking behind UEC and the combined Dell Canonical offering.
[3:45] OpenStack and Canonical’s participation
Working with both OpenStack and Eucalyptus and how both of these are central to the process of standardization that we are starting to see at the infrastructure layer of cloud computing.
Continuing in my series of videos from the OpenStack design summit this week in San Antonio, here is an interview I did yesterday with Eucalyptus systems co-founder Graziano Obertelli.
Eucalyptus allows enterprises to set up open source infrastructure-as-a-service private clouds. Eucalyptus is also one of the key ingredients in the Ubuntu Enterprise Cloud that is being certified to run on Dell’s PowerEdge C systems as part of our cloud ISV program.
Here is what Graziano had to say:
Some of the ground Graziano covers:
What goals do the Eucalyptus team have for the summit
They’ve recently hired a community manager – Mark Atwood
Last week a couple of us went down to San Antonio to help represent the OpenStack project at Rackspace’s partner summit. While there I met up with the VAR Guy. Mr. Guy got me chatting about Dell’s Data Center Solutions group, where we’ve been and where we’re going. Below is the resulting video he put together featuring myself and San Antonio’s greenery. (See the original article this came from).
Some of topics I tackle:
How Dell’s Data Center Solutions Group is designing servers for high-end cloud computing
How Dell is integrating hardware with software in cloud servers
Coming soon: Dell Cloud Solution for Web Applications/Leveraging Joyent‘s software
After the cloud summit last week at OSCON, I sat down with Neil Levine of Canonical to see what was in store for Ubuntu cloud-wise (Canonical is a partner of ours in our cloud ISV program). Neil is the VP of Canonical’s corporate services division which handles their cloud and server products.
Here’s what Neil had to say:
Some of the topics Neil tackles:
The next Ubuntu release “Maverick Meerkat” and its geek-a-licious launch date: 10.10.10.
Look for Maverick to make Eucalyptus even easier to deploy and use.
Data processing and data analytics is one of the key use cases in the cloud and Canonical is looking to move up the stack and provide deep integration for other apps like Hadoop and NoSQL.
What are some of the areas of focus for next year’s two releases i.e. 11.04 and 11.10.
Project ensemble: what it is and what its goals are.
Tuesday after the OSCON cloud summit I sat down with Rick Clark over a well deserved beer. Rick is the chief architect and project lead for the OpenStack compute project that was announced on Monday.
Last week I interviewed Rick on the first day of the inaugural OpenStack design summit and I wanted to catch up with him and get his thoughts on how it had gone. This is what he had to say:
Some of the topics Rick tackles:
How it went engaging a very large technical group (100+) in an open design discussion patterned after an Ubuntu Developer Summit.
Some of the decisions he thought would be no brainers, turned out differently e.g. OVF (open virtualization format) and keeping the storage and compute groups separated.
Since the summit involved representatives from over 20 companies, some of them competitors, how good were people at putting away their business biases/agendas?
How far they got (hint they got requirements from everyone for the first release).
They’ve already gotten their first code contributions.
How they plan to build a community: actively looking to hire a community manager. In the meantime its actively growing and in a week they’ve gone from 10 people in the IRC channel to 150 on Tuesday.
Rick Clark used to be theengineering manager at Canonical for Ubuntu server and security as well as lead on their virtualization for their cloud efforts. He’s now at Rackspace and is applying much of what he learned while at Canonical to his new gig as project lead and chief architect of the just announced OpenStack Compute.
Rick talked to me about what he brought with him from Canonical as well as the details behind OpenStack Compute.
Some of the topics Rick tackles:
What is the OpenStack Compute project (hint its a fully open sourced IaaS project)
Leveraging what Rick learned from the Ubuntu community, including a regular six month cadence.
Rick’s goals for design summit: develop a roadmap for the first release, spec out the software and spend the last two days prototyping and hacking.
Why they went with the Apache 2 license and why not AGPL?
The Rackspace API (NASA had already started to switch from the Amazon API before combing
Yesterday at the GigaOM Structure conference here in San Francisco, I ran into Marten Mickos, the recently appointed CEO of Eucalyptus systems. Eucalyptus is one of the key ingredients in the Ubuntu Enterprise Cloud that is being certified to run on Dell’s PowerEdge C systems as part of our cloud ISV program.
Marten, the former CEO of MySQL took the helm of Eucalyptus about three months ago, and was at Structure both as an attendee and participant, sitting on two panels at this two-day cloud-a-polooza. At the end of the day-one I got some time with Marten and asked him about his new gig.
Some of the topics Marten tackles:
How he made the decision to go to Eucalyptus. (Hint: he asked the question, what’s bigger than Open Source)
What is Eucalyptus and whats it based on?
How will Marten’s experience at MySQL and Sun help him in his new role at Eucalyptus?
MySQL was a disrupter of the old whereas Eucalyptus is an innovator of the new.
Sun’s company culture was phenomenal, the technology was phenomenal, the business…um…
Tom Henderson and Brendan Allen of ExtremeLabs published a great walk-thru of the Ubuntu Enterprise Cloud (UEC) last week in NetworkWorld. Canonical, the commercial sponsor behind Ubuntu, is one the first members of our Cloud Partner Program and we will soon be offering UEC running on top of our PowerEdge C line accompanied by reference architectures.
If you’re not familiar with UEC, which leverages the open source Eucalyptus private cloud platform, here is a quick backgrounder:
Basically, Ubuntu Enterprise Cloud can be deployed on internal hardware to run job/batch applications. The idea is to initially allocate storage, then rapidly build multiple virtual machines to process data, collect the data, then tear down the infrastructure for re-use by a subsequent purpose.
Ubuntu Enterprise Cloud provides internal cloud control methods that closely mime what can be done on Amazon’s public cloud infrastructure. Its tools can be used to process recurring jobs or one-shot distributed applications, like DNA analysis, video rendering, or database table reformatting/reindexing.
Walk this way
The Review, which is a concise 3 and a half pages, steps you through:
Getting started
Installation*
Setup/configuration
Image Bundles
Usage/Monitoring
*My favorite line from this section is: “Installation was very simple; we inserted the Ubuntu Server CD, selected Ubuntu Enterprise Cloud, and drank energy drinks.”
If you’re interested in learning about UEC this article is a great place to start.
Extra-credit reading
If the above whets your appetite, you may want to dig into the following:
Last but not least in our tour of the first flight of servers in Dell’s PowerEdge C line is the C2100. This was filmed last month at our launch right after we closed down our whisper suite. Click below and join solutions architect Rafael Zamora as he leads you through the thrills, chills and spills of this cool new machine.
Spoiler Alert! A few Highlights
This machine is a great high performance data analytics and cloud optimized storage server. It’s perfect for use in conjunction with software from our Cloud ISV partners Joyent, Aster Data, Canonical or Greenplum.
The entire front can be jam-packed with disk drives for mega storage. You can get up to 24TB by loading twelve 3.5inch 2TB drives.
While the C2100 has same system board as the C1100, it comes with twice the real estate in a 2U form factor.
There’s even a platform for two additional drives that you can use for a bunch of different purposes such as separating production and non-production traffic.
Last month when I was out in the Bay Area for our launch, I was able to catch up with Rich Wolski, founder and CTO of Eucalyptus. Eucalyptus is one of the key ingredients in the Ubuntu Enterprise Cloud that is being certified to run on Dell’s PowerEdge C systems as part of our cloud ISV program. Here is what Rich had to say:
Some of the topics Rich tackles:
How Eucalyptus started at the University of California at Santa Barbara. They wanted to show how old-style large scale computing (NSF super computer centers) could be combined with new large-scale computing (in the form of Amazon) in the service of science. Wanted to also include 4-6 university data centers.
They put the code out as open source and got deluged by science and commercial industry about potential applications. Grew too big to continue as a research project so they brought it outside.
Working with Canonical and Ubuntu and how the relationship began. UEC and what part Eucalyptus makes up.
How NASA is offering a production Eucalyptus cloud to NASA researchers and other governmental agencies.
Where Rich sees Eucalyptus going in the next two years. The importance of the open source community and their continued focus on private clouds in the enterprise.
If you’re wondering about the funky game show-like setting, I shot this after hours on the day of our launch in the whisper suite. Your guide, as before, is the incomparable Dell Solutions Architect, Rafael Zamora.
A few highlights
The C1100 is a high memory, cluster optimized, compute node
Dont let its slim pizza box looks fool you, upfront you can pack either four 3.5 inch drives or ten 2.5 inch drives.
For high memory optimized compute you can get 18 DIMM sticks for 144GB of RAM.
Comes with your choice of either Intel’s Nehalem or Westmere processors.
Raf also gives a couple of examples of recent customers and how they’ve decide to configure their units.
As a follow on to last week’s PowerEdge C line overview, here is the first individual system overview: the C6100. Click below and let Dell Solutions Architect Rafael Zamora guide your thru the design and features of this densely packed machine targeted at HPC and cloud workloads.
Some of the highlights:
The PowerEdgeC 6100 holds the equivalent of 4 systems which have been packaged into “sleds,” each containing boards, RAM and microprocessors.
Upfront you can put a ton o’ disk drives, either 24 x 2.5″ drives or 12 x 3.5″ drives.
Great for markets like HPC clustering and search engines where compute density is key. (This is not intended for running general purpose apps like Exchange, SQL or Oracle).
It will serve as the compute node in the Ubuntu Enterprise Cloud solution from our partner Canonical.
At last week’s Dell Launch, “Solutions for the Virtual Era,” we unveiled the first three systems in our new PowerEdge C line. These “hyper-scale inspired” systems are based on designs that we have built for our largest scaled-out customers such as Windows Azure, Facebook, Ask.com and Tencent.
The PowerEdge C line is targeted at both Public and Private cloud builders as well as HPC, Web 2.0, gaming and large scaled out web farms. In the video below, Dell solutions architect Rafael Zamora walks us through the PowerEdge C6100, C1100 and C2100.
Upcoming posts
In the days to come I will be posting individual walk-thrus of each of the three systems. I will also be posting interviews I did with executives from our cloud partners Joyent, Aster Data, Greenplum and Eucalyptus (who’s not a partner but provides a key component of our partner Canonical’s Ubuntu Enterprise Cloud).
My favorite cosmonaut-coder Mark Shuttleworth stopped by our offices this morning for a visit. Mark is the founder of both the Linux distribution Ubuntu and its commercial sponsor Canonical. Mark and I sat down in the lobby and caught up. Here is a short interview we recorded.
The 10.4 Ubuntu release Lucid Lynx and what to expect: a strong cloud focus on the enterprise side and a lot of shiny new bling on the desktop as well as making the desktop “social” (e.g. Tweet straight from your desktop)
What Ubuntu is doing in the Netbook space
What excites Mark the most in technology today and why cloud is like HTTP in the early 90′s
Mark Shuttleworth, founder of Ubuntu and the head of Canonical, the commercial entity behind the popular linux distribution, is currently making his rounds in the States. Yesterday he was quite busy, taking the stage at both the Intel Developer Forum in San Francisco as well as at LinuxCon up in Portland Oregon.
Today he popped by Dell here in Austin to chat. I grabbed him for a few minutes right before lunch. Here is the result:
Posted by Barton George 
