Cloudtronics

Amazing Drupal Multisites with AUFS..

2011-03-08T11:28:00.000-08:00

Under the SUN anything possible. Today I'm going to explore the techniques of having a drupal installation with AUFS and produceing multisites using the same codebase everafter..
This would open a new era of Consumer Websites evolution. I knew the phrases are new, but viability do last!

The benefits are manifold:

1. Seemless software updates
2. Plug-n-Play drives using iSCSI Storage
3. Leverage the power of Cloud
4. Easy Failover & maintenance
5. Cost efficient & reliable

Drupal Directory structure for having a multisites:

Now I gonna teach you how to create symlinks in your windows box. It should be in your drupal root folder where index.php resides.

use linkd windows command download & install from here

This linkd software you can simply copy from a installed machine to a new; so you wouldn't need an administrative privileges to install.

one more thing, need to tell apache of this virtual domain; means when you hit apache as http://www.mydomain.com/ where you should landup:



[VirtualHost *:80]

ServerName www.mydomain.com:80
DocumentRoot /srv/www/htdocs/mysite1
[/VirtualHost]

[VirtualHost *:443]
ServerName www.mydomain.com:443
DocumentRoot /srv/www/htdocs/mysite1
SSLEngine On
SSLCertificateFile /etc/apache2/mysite1dev.crt
SSLCertificateKeyFile /etc/apache2/mysite1dev.key
[/VirtualHost]

And that's it !

Getting-a-Lightbox-Alike-Overlays-in-Drupal

2011-01-20T19:43:00.000-08:00

Getting a Lightbox Alike Overlays in Drupal

Cloud Test : No-more a Nightmare..

2010-11-27T08:41:00.000-08:00

A typical BVT(Build Verification Tools) in the cloud:

Cloud will give the advantage to devise your challenges into parts & pieces. You can now think off-the-shelf solutions of what were beyond imaginations a year back.

As selenium becoming more & more powerful by each day , so the Cloud strategy. The legendary “Map-Reduce” concept when applied to realize truly distributed computing bring enormous yielding power.

Basically you will map Tests into parts and collate the results at the end. I will tell how step by step you can achieve this:

In a typical development centre you establish a Local Area Network for daily works, and additively store your daily developments in the Subversion created for the project.
You then engage Hudson to do nightly builds for what your team sum-up throughout the day. Hudson in a hudoop cluster would be the most suitable choice for enterprising your deliverables.

Tips & Trick : part I

2010-05-28T07:02:00.001-07:00

How to Use CXF for Building Quick Service Having Both SOAP and REST

Cloud Usage: Scalable Web Data Mining System

2010-02-17T12:44:00.001-08:00

A typical Data Mashup System can collate data from multiple sources

SAAS embraces "The Cloud" ..

2010-01-20T21:46:00.000-08:00

Introduction

Bringing SAAS Model allow it to outlast an earlier cousin, the "Application Service Provider" leaving a modest room for the marketers how to claim it is. The big question for SaaS may be this: Will it impel the concept of cloud computing, or merely ride on its beckon?

Although the industry acceptance of SAAS is very high, it has its own pros & cons as well..
Recent predictions from Gartner show around 30% of the enterprise revenue being generated by Content & Collaboration applications itself.

The market feedback in brief as per Gartner:

To succefully implement SAAS model for an Enterprise Solution, the strategy has to overcome these recedes.

SAAS-ified

Infact the SAAS & Cloud Computing work in tandem, Cloud enables SAAS at infrastructre level while SAAS provides the way to Agile licensing. SAAS-fication of a solution happens at two stages, viz.

1) Application
2) Infrastructure (that runs the application)

To make applications SAAS enabled, typically application framework has to be modeled with IOC (Inversion of Control). Entire application has to be meta-data driven. Sounds like a purely multi-tenant application.

Contrary saas enablement of Infrastructure happens with leveraging Virtualization, that brings all the characteristics of the classical Cloud Computing architecture.

Benefits

• Tenancy – The ability to distinguish one user from another in the data and execution aspects of a hosted application is a major tenet of SaaS. Generally, the concept of tenancy is void in traditional on-premise installs and can complicate architectures beyond what was traditionally accepted.

• Flexible Monetization & Metering – Being able to charge for your SaaS application independent of the software code, and meter it appropriately, is something that your application shouldn’t have to deal with directly.

• Scalability – The idea that a successful application will buckle under its own popularity is never good. Being able to accommodate your aggregated customer base is a must, and planning for success is a requirement.

• Reliability – What good is a SaaS application that isn’t up?

• Hardware Infrastructure – As a vendor, one of the operational headaches of SaaS applications is dealing with an enterprise-grade hardware infrastructure.

• Value Added Services – A good platform should endow the application it hosts with value beyond what was developed by the vendor. The value should either benefit the vendor or the end user.

• Ecosystem – As the number of vendors that host their applications on a given platform increases, and as the number of users using those applications increases, an ecosystem begins to develop. Ideally, this ecosystem allows all parties the ability to investigate and exercise their right to connections between ecosystem members, deriving value beyond that offered by any single SaaS offering.

• Faster implementation: Customers deploy within hours with Sonian and integrate with their existing systems. ather than weeks or months.

• Intelligent and Infinitely Scalable Computing Resources: Sonian’s use of cloud computing allows our archiving service to automatically throttle up or down computing resources depending on your requirements. This process provides our customers with an amazingly fast search and user experience.

• Infinitely Scalable Storage Resources: With Sonian, you never have to worry about running out of disc space or paying more for storage. Because of our use of cloud infrastructure Sonian provides unlimited storage to our customers.

• Little or no upfront fees: [agile licensing]Avoid paying for the acquisition and licensing of hardware and software

• Lower Total Cost of Ownership: Costing 7 to 10 times less than installed software and hardware.

• Leave it to the experts and Reduce the Burden on IT: Take advantage of experts in email archiving, indexing, search and discovery. Your IT staff have tasks that challenge them and keep them happy.

• Regular upgrades: Receive regular upgrades to enhance your user experience. New functionality is yours when you want it with no additional fees.

• Safe and Secure: DoD encryption and PCI-DSS security standards and unlike other SaaS archiving providers, Sonian provides a private data silo for you and your company.

• Fast and Reliable: Return search results in under 3 seconds regardless of the size of your archiving

• Availability and Redundancy: Sonian’s use of the Cloud allows our service to take advantage of failure resistant and geographically disperse availability zones.

• No Risk – No Obligation trials available: To learn more about the Sonian Archive Service, please complete the contact form and a Sonian Account Manager will contact you promptly.

SLA

An SLA, as its name suggests, is an agreement between the service provider and the consumers, consisting of sections regarding the various commitments to service levels that will be matched or exceeded.
Each section is defined as a Service Level Objective (SLO).

A typical SaaS SLA should have the following SLOs:

Service Availability – define the availability of the service represented in percentage (e.g. 99.95% uptime)

System Response Time – define response time of various transactions represented in seconds. (e.g. login should not take more than 9 seconds)

Customer Service Response Time – a response on customer enquiries should take no more than an allotted time for various services (e.g. enabling a service for a new group should take less than two business days)

Customer Service Availability – hours of availability of customer service represented in a ‘hours per day’ notation. (e.g. 11X5 for regular customers, 24X7 for platinum customers)

Service Outage Resolution Time – the times it takes to restore a service after an outage has been reported. Represented in minutes and hours (e.g. 30 minutes for a full system outage)

Failover Window For Disaster Recovery - how long will it take to restore the service in a disaster recovery site, if disaster disables the main datacenter.

Reclaiming Customer Data – a commitment to transfer all (agreed) data in an agreed format in case the customer leaves the service.

Maintenance Notification – the advance notice that the provider will notify customers of planned service outages, represented in days. (e.g. a planned downtime that will take more than one hour requires 10 business days notification)
Proactive Service Outage Notification - the time it takes for the provider to inform the customer that there are service issues, represented in minutes.

RFO (Reason for Outage) – a report to customers following a service outage explaining the circumstance, the incident and steps taken to remedy the problem. (For more information see the chapter on Incident Management). Some customers require an RFO automatically; in some SLAs it is written that an RFO will be generated only following a specific customer request. Usually the company commits to three business days following the service disruption.

Approach

The Application SaaS-ification allows to reuse same deployed code-base all over along. This has plenty benefits.I know, you are thinking for the applications which are already coded and want to leverage multi-tenancy. In sort, how running businesses would rampup by SAAS-ification.

case study: multi-tenant alfresco

Planning

Lock-in

1. Platform agonistic
2. Proprietary technology usages
3.

Billing Mediation Simplify the Billing Process

Customer Expectations

Not all SaaS companies are created equal. They will vary by maturity, by the vertical they are serving, by the company size they cater for and, of course, by the type of application.
Some applications are core and some are peripheral. Some applications are used around the clock, like metering or call centers and the customers have zero tolerance for downtime. Other applications are rarely used outside of office hours, (e.g. payroll, talent management) and if the system is down, the price is a handful of irritated end-users that will need to take a coffee break earlier than they planned.

Larger customers tend to have more rigorous demands while lower paying customers will usually be more tolerant of the system’s performance and support availability.

Therefore, your SLA should reflect the relative position of your service along the following three vectors:

Customer size (reflecting subscription [potential] size)
Core vs. periphery
Downtime tolerance

So if you are providing a mission critical application to a large customer, whose downtime will cost the customer real dollars, your SLA should be taken very seriously.

Service Level Breaches and Penalties

We have seen the promises that come with the SLAs, but many of these agreements fail to state the consequences to the provider of not meeting the terms.
Each SLO should also define the penalties for breaching the service level commitment.
Penalties are typically specified as a prorated credit for the following month’s subscription fees.
From the customers’ point of view, the penalties should not be flat rated but increase as the service deteriorates, so that the second outage will carry a heavier penalty than the first outage. It is rare that customers insist on this point but those that do will need to negotiate these terms separately.

There is typically a maximum. It is unusual that accumulated penalties will top the monthly subscription costs. There is a catch here. As an extreme example, if your service was down for the duration of the whole month, the customer will be exempt from paying a full month’s service fee – but this is ridiculous of course. The damage to you customers is typically orders of magnitude higher than the subscription costs.
Many SaaS customers commit up front to a year or more of service, for a reduced subscription price. A good SLA will include a section that allows the customer to breach the extended commitment if the provider failed to adhere to the service levels for, say, three consecutive months.

Have a safe flight in the Cloud of multi-tenancy..

2009-11-04T07:08:00.000-08:00

Multi-tenancy at Infra level: sharing resources & computing power of a physical boxTypically Cloud Host are the Landlords provide basic amenities to all the tenants and help them gracefully co-exists. All is good, till you have a bunch of fair tenants. Think one who always peeping thru the wall of others and privacy of neighbors at its helm ..

As you see, virtual host provides a layer of visualization on the resources being shared by the VMs. We will try to figure the risk on the paradigm..

Mix 'N' Match ..

2009-10-04T13:46:00.000-07:00

Lets run a small CRM for your employees, that can be shared between the departments without impacting each others model.

TBD..

Shop As You Like ..

2009-10-02T13:08:00.001-07:00

Do you believe in application shopping ? Days gone when you negotiate to a vendor on time & money, bargain for a little more.. Here you go, with packaged solution tailored & branded in minutes only for you. And these all will happen with a couple of mice click & just swipe of a card..
Nevertheless, need more to scale dont worry just know us. Will scale you before you think strategy, becoz all these a solved problem for industries; we just leverage them the way you want...

Bringing SAAS to Cloud for SMEs ..

2009-09-22T13:05:00.000-07:00

Concepts

The backdrop is cloud and related promising technologies of open source world that venturing across the market. The system is to take care of grossly S/M Enterprises, addressing some of its key areas of “wish lists” that large enterprise still striving for ..

1. Optimization of Computing resources across enterprises
2. Participate in “Go Green” , “Save Earth” kind activities by optimizing resource usage
3. Minimize the Infrastructure maintenance Costs
4. Provide true high availability of Software services , platforms & workbench

The system was divided into three major parts:

1. CWB (the Nucleus – cloud provider)
2. RPM Packager (the cook – the cloud deployer)
3. DB Backup & restore management , Distributed Storage System (the Cloud DATA)

A Midas touch ..

2009-09-10T12:24:00.000-07:00

I have approached to Amazon to allow this kinda usage model, that will relieve data locks as such.. And you start enjoying the delta layer formation..

Protocol..

2009-08-27T05:12:00.000-07:00

National Institute of Standards and Technology, Information Technology Laboratory

Note 1: Cloud computing is still an evolving paradigm. Its definitions, use cases, underlying technologies, issues, risks, and benefits will be refined in a spirited debate by the public and private sectors. These definitions, attributes, and characteristics will evolve and change over time.

Note 2: The cloud computing industry represents a large ecosystem of many models, vendors, and market niches. This definition attempts to encompass all of the various cloud approaches.

Definition of Cloud Computing:

Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. This cloud model promotes availability and is composed of five essential characteristics, three service models, and four deployment models.

Essential Characteristics:

On-demand self-service. A consumer can unilaterally provision computing capabilities, such as server time and network storage, as needed automatically without requiring human interaction with each service’s provider.

Broad network access. Capabilities are available over the network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

Resource pooling. The provider’s computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to consumer demand. There is a sense of location independence in that the customer generally has no control or knowledge over the exact location of the provided resources but may be able to specify location at a higher level of abstraction (e.g., country, state, or datacenter). Examples of resources include storage, processing, memory, network bandwidth, and virtual machines.

Rapid elasticity. Capabilities can be rapidly and elastically provisioned, in some cases automatically, to quickly scale out and rapidly released to quickly scale in. To the consumer, the capabilities available for provisioning often appear to be unlimited and can be purchased in any quantity at any time.

Measured Service. Cloud systems automatically control and optimize resource use by leveraging a metering capability at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported providing transparency for both the provider and consumer of the utilized service.

Service Models:

Cloud Software as a Service (SaaS). The capability provided to the consumer is to use the provider’s applications running on a cloud infrastructure. The applications are accessible from various client devices through a thin client interface such as a web browser (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

Cloud Platform as a Service (PaaS). The capability provided to the consumer is to deploy onto the cloud infrastructure consumer-created or acquired applications created using programming languages and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, or storage, but has control over the deployed applications and possibly application hosting environment configurations.

Cloud Infrastructure as a Service (IaaS). The capability provided to the consumer is to provision processing, storage, networks, and other fundamental computing resources where the consumer is able to deploy and run arbitrary software, which can include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, deployed applications, and possibly limited control of select networking components (e.g., host firewalls).

Deployment Models:

Private cloud. The cloud infrastructure is operated solely for an organization. It may be managed by the organization or a third party and may exist on premise or off premise.

Community cloud. The cloud infrastructure is shared by several organizations and supports a specific community that has shared concerns (e.g., mission, security requirements, policy, and compliance considerations). It may be managed by the organizations or a third party and may exist on premise or off premise.

Public cloud. The cloud infrastructure is made available to the general public or a large industry group and is owned by an organization selling cloud services.

Hybrid cloud. The cloud infrastructure is a composition of two or more clouds (private, community, or public) that remain unique entities but are bound together by standardized or proprietary technology that enables data and application portability (e.g., cloud bursting for load-balancing between clouds).

Note: Cloud software takes full advantage of the cloud paradigm by being service oriented with a focus on statelessness, low coupling, modularity, and semantic interoperability.

How to get Xen instance IP from outside..

2009-08-21T03:33:00.000-07:00

I prepared a script to get IP from Xen instance..

1) Put detIP.sh to Xen box

2) Usage: ./detIP.sh myVm 2> /dev/null

You should return by the IP of myVm instance..

detIP.sh
========

Hope this would be helpful ...

How to configure CAS with LDAP

2009-07-27T22:46:00.001-07:00

The configuration is quite trivial.. still i wrote down:
1. install openldap and add some users:
ldapadd -x -D cn=admin,dc=demo,dc=cloudtronics,dc=com -W -f test.ldif
where test.ldif,

Now how to search, from application.. In tomcat web use deployerConfigContext.xml:

Morphology of Cloud Apps

2009-06-26T06:45:00.000-07:00

Life Cycle of the Cloud Apps:

The figure shows a typical lifecycle of an image running on the Cloud, having non-transient vista. It's evident that persistency of any degree should be made out-of-box.

This spawns another interesting topic. Should admire that Image must undergo at least few changes after instantiation. These changes could be as small as root password to new applications installation..

Just imagine then who could be the version controller for the Cloud Apps -: ) Yes nobody, Cloud don't take responsibility to define such requirements which is real but overlooked !!!

Isn't sound strange??

Amazon says to provide some discrete tool set, but fail to define a solid process that can be automated or having enterprise standard.
You will feel slightly embarrassed, when try to model an object oriented & loosely coupled application deployable to an Enterprise Cloud. Your frustration will mount high, when you really require to maintain & manage multiple copies of the applications alongwith periodic updates & patches..

Listen Gurus, you gone by then - : )

Now, if you do envisage of Amazon public image libraries, see its quite static and stagnant !!
Most of us, don't like to shop around through the expired products, will you ??.. Where specially the IT, changing in each seconds..

If you don't want to pay for "what was" rather pay for "what is" or "what will be" continue with me..

I will tell you how to crack this classic stagnancy..

My first 2 Cents ..

2009-06-16T01:58:00.000-07:00

Enough theory, let's do something handson today. Let's setup a Box for preparing the Cloud Stack.. I carefully picked-up the box by running "egrep '(vmx|svm)' /proc/cpuinfo" command. Because I would like to have a box that has a hypervisor support and can yield optimized results for the instances to run in the Cloud. I took a box with AMD processor and 8GB RAM , 320GB HD.. After thinking twice, I took Centos5.3 64bit OS to install. I took 64bit as this provides the best & versatile support to create both 32 & 64 bit instances on the Cloud.
During installation I chose All the default options, except Selinux & Firewall Off. Since I'm within Corp-Net, I dont want doubly protected!!
Also I created some LVG(logical-vol group) around 50GB, so that I can use it for RAW image creation later.. Hope you appreciate that using LVM image creation is pretty flexible..

This point, letmm explain a bit about what I'm looking for:

If you are familiar with the conventional Grid Computing which unifies the processing power of the multiple machines as surface to collimated usage; inturn you should also appreciate that people will be looking for some ROI out-of-it.. And, that is fulfilled by the Utility Computing..
Cloud Computing goes one step ahead to warp them and present in a SAAS enabled model. Apart from grid & cluster; cloud added a fine grained control as middleware to the architecure. This works a brain for the Cloud building blocks. Typically Client Apps on Cloud use the Middleware to position themselves on the Cloud..

Cloud usage has two facet; the transient Cloud Apps and persistent Cloud Data (used by Apps).. And you also knew, to achieve Cloud Data persistency and failover always includes the cost of redundancy.
There're various interface softwares to provision and govern Cloud & its resources; mostly recalling & acknowledge EC2 as consumable.

Next post I will blog more about the Lifecycle of Cloud Apps and its operational Data..
Till then, stay tuned -: )

Cloud Workbench Featurology

2009-06-04T01:03:00.000-07:00

As I commited yesterday to provide detail of the workbench features, here it goes:

	Amazon	Cloud Workbench
EC2 Functionality
Create an Amazon Machine Image (AMI)	yes	yes
Use pre-configured, templated images to get up and running immediately	yes	yes
Upload the AMI into Amazon S3	yes	yes
Choose the instance type(s) and operating system you want	yes	yes
Start, terminate, and monitor as many instances of your AMI	yes	yes
Static IP endpoints	yes	not sure as hot requirement, but can be alloted by fix Mac
Attach persistent block storage to your instances	yes	yes
Elastic
Increase or decrease Instance capacity	yes	yes
Completely Controlled
root access to instance	yes	yes
Instances can be rebooted remotely	yes	yes
access to console output of your instances	yes	yes
Flexible
The choice of multiple instance types, operating systems, and software packages	yes	yes
select a configuration of memory, CPU, and instance storage	yes	yes
Use with other Amazon Web Services
Supports Amazon Simple Storage Service (Amazon S3)	yes	yes
Amazon SimpleDB	yes	suggestions welcome how to replicate locally
Amazon Simple Queue Service (Amazon SQS)	yes	suggestions welcome how to replicate locally
Reliable
Replacement instances can be rapidly and predictably commissioned	yes	yes
Commitment is 99.95% availability	yes	yes
Secure
Interfaces to configure firewall settings that control network access to and between groups of instances	yes	yes
Inexpensive
Pay for the resources consumed, like instance-hours or data transfer	yes	its free as you own the complete infrastucture
Amazon Elastic Block Store
Off-instance storage that persists independently from the life of an instance	yes	yes
EBS volumes are highly available, highly reliable volumes that can be attached to a running Amazon EC2 instance and are exposed as standard block devices	yes	yes
Amazon EBS volumes are automatically replicated on the backend	yes	possible using a Cron job taking regular backup
Snapshots of your volumes	yes	yes
Multiple Locations
Launching instances in separate Availability Zones to protect your applications from failure of a single location	yes	live migration possible
Elastic IP Addresses
Static IP addresses designed for dynamic cloud computing	yes	possible using fix Mac
Amazon CloudWatch
Resource utilization, operational performance, and overall demand patterns—including metrics such as CPU utilization, disk reads and writes, and network traffic	yes	possible thru commandline tools
Auto Scaling
Automatically scale your Amazon EC2 capacity up or down according to conditions you define	yes	welcome suggestions as how to replicate this feature locally
Elastic Load Balancing
Load Balancing automatically distributes incoming application traffic across multiple Amazon EC2 instances	yes	welcome suggestions as how to replicate locally
Instance Types
Small Instance (Default) 1.7 GB of memory, 1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit), 160 GB of instance storage, 32-bit platform	yes	yes
Large Instance 7.5 GB of memory, 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each), 850 GB of instance storage, 64-bit platform	yes	yes
Extra Large Instance 15 GB of memory, 8 EC2 Compute Units (4 virtual cores with 2 EC2 Compute Units each), 1690 GB of instance storage, 64-bit platform	yes	yes
Amazon S3 Functionality
Write, read, and delete objects containing from 1 byte to 5 gigabytes of data each. The number of objects you can store is unlimited	yes	yes
Each object is stored in a bucket and retrieved via a unique, developer-assigned key	yes	yes
REST and SOAP interfaces	yes	yes
Features of Amazon EBS volumes
Amazon EBS allows you to create storage volumes from 1 GB to 1 TB that can be mounted as devices by Amazon EC2 instances. Multiple volumes can be mounted to the same instance	yes	yes
Each storage volume is automatically replicated	yes	yes
EBS also provides the ability to create point-in-time snapshots of volumes	yes	yes
Create new volumes	yes	yes
Amazon SimpleDB Functionality
CREATE a new domain to house your unique set of structured data	yes	yes
Query your data set	yes	yes
Amazon SQS Functionality
Developers can create an unlimited number of Amazon SQS queues with an unlimited number of messages	yes	not yet thought
Queues can be shared with other AWS accounts and Anonymously	yes	not yet thought
Access to SQS through standards-based SOAP and Query interfaces	yes	not yet thought
Amazon Elastic MapReduce Functionality
Hadoop implementation of the MapReduce framework on Amazon EC2 instances	yes	envisage the feature to replicate locally
Cross Virtual Machine Portability
Instances portable to VMWare,XEN etc	no	yes
Software Update Service to Instances
Instances receive updates seemlessly	no	envisage the feature to implement

The above table will give you an idea how far the other Amazon friendly implemnetaions are..

This also indicates how I'm defining that 60% Gap exists in the opensource world to deal with Cloud ready application development & testing end2end.

Next, I will define how can I really achive this, probably a Development Cloud Stack (just ready to use) for the opensource world..

Cloud Workbench (Building a readymade solution for Cloud Developers)

2009-06-03T08:00:00.000-07:00

I started thinking of a Workbench for the Cloud developers to ease their efforts and save some bucks while working with commercial Cloud providers like Amazon.

In Opensource world how seemless is to develop a Cloud ready application (rather amazon ready) and test with end2end features before actually deploy it on the Public cloud..

Here I want to do pretty serious business of developing an amazon ready application with EBS & S3 supports locally that can move back & forth from private to public workspace.. Are any ready made Developer Stack available now, so I can still run instances overnight without fearing of dollars.. Of course like you guys, I also want a sound sleep at night -: )

There're opensource offerings to make a private cloud as more similiar to Amazon, but are those supports all the features like Amazon provides? Or can I simply clone my Amazon Instance (with EBS & S3 intact) to these opensource offerings ? I would say it's 40% yes, 60% no..

Then how to fill the 60% Gap that still to go..

Initial Thoughts

2009-06-03T05:31:00.000-07:00

Concepts

The backdrop is Cloud and related promising technologies of open source world, venturing across the market. The system is to take care of grossly bigger Enterprises, addressing some of its key areas of “wish lists” that large enterprise still striving for ..

1. Optimization of Computing resources across enterprises
2. Participate in “Go Green” , “Save Earth” kind activities by optimizing resource usage
3. Minimize the Infrastructure maintenance Costs
4. Provide true high availability of Software services , platforms & workbench

The system was divided into three major parts:

1. The Nucleus – cloud provider
2. RPM Packager (the cook – the cloud deployer)
3. DB Backup & restore management , Distributed Storage System (the Cloud DATA)

I have drafted some notes on the Stack as follows:

Infrastructure
=============

Completely customized Cloud Controlled XEN vms, that can be simply powered-up to Go with User data stored in HDFS..
Custom changes in (domU)VMs seamlessly to be written to union filesystem (using unionfs/aufs alike techniques).
this concept be extended to transparently localize Amazon S3 using S3fs fuse..

There may be 3 possible scenarios to use unionfs with Xen VMs:

1) To have a common filesystem image shared by many DomUs, and use unionfs to allow each DomU to have it's own modified version
2) To do this in Dom0 and export the filesystem to the DomUs
3) To export the common filesystem (readonly) and run unionfs in the DomU

And finally, the user modified file-systems made exposed to HDFS (using hadoop/sector DFS)
A private cloud to be built using Eucalyptus and physical Nodes with [Xen vms + Suse]
There are Nagios plug-ins available-ready for Xen-suse , can be used for monitoring the Nodes and probably some auto discovery..

Backup & Recovery
=================

Xen's capability to adopt live migration will be the base for application failover /recovery..
Data recovery to be done for the above user modified filesystem, using a suitable cloud & S3 supported tool (amanda/zamanda/baracula)

Packaging the End2End process
=======================

VMs with delta file changes to be deployed live on Xen based Cloud and that should be powered-on to go..

1) Taken the input from User
2) Prepare application (vm) to be deployed over the Cloud
3) Deploy app-vm & power-on
4) Provide user, the logistic infos & access
5) User’s changes are saved as delta & backed-up

myspace hit counter