We are pleased to write about our new Platform As A Service (PaaS), which we call SimpleHosting. We have been working on this for more than a year, and are integrating the latest versions of component software while maintaining homogeneity with the management framework of our existing Infrastructure As A Service (IaaS) hosting.
Our goal in offering this platform is full abstraction of the system configuration, using our specific knowledge and experience gained from thousands of existing hosting installations. This abstraction then allows you full latitude on the application side, within reasonable limits. We have chosen the limits to be as open as possible by default.
Information about the CentOS repository on Gandi mirror server.
Gandi's Thomas Stocking went to the Cloud Expo 2011 West in Santa Clara this week, and found that marketing is alive and well in the Cloudsphere. Despite the fluff, there were some interesting ideas getting discussed, and companies developing strategies to leverage the evolving cloudscape...
If you read any networking textbook or study guide on the subject of BGP and route stability, very often you will find a mention or even a suggestion to tie your aggregated prefix to null0 to ensure that the prefix is remains in the routing table thereby increasing the stability of your BGP advertisements.
While this is a good thing, to a certain extent, there are situations where such a configuration may lead to service interruption in case of an outage. This quick article talks a little about internet routing using BGP and some "common practices".
For the past few months perhaps you might have made use of the servers in the US. The changes in the storage technology was one of the strong points in the hosting infrastructure. Prior to this, we had to adapt the infrastructure so that it could understand "n" datacenters. The implementation of this new storage platform was not as complicated as it seems since it is completely independent to the architecture in France. With a new datacenter, it was therefore quite a trivial matter to build from scratch, and all of the new servers in the US made use of this new platform from day one.
Our new storage infrastructure is being deployed.
Disk Management is now handled through virtualization... and we are excited to share it with you!
As most of you may have already noticed over the past 18 months, there have been several periods of scheduled maintenance on the Gandi network. Some of these have been fairly intrusive, while others have taken place quietly behind the scenes. I have made a number of innuendos here on the Gandi Kitchen and also on the Gandi Bar over the past year dropping hints about some of the things to come, so I decided to take this opportunity to reveal a little more about what our network operations team here at Gandi is up to and what it means for you over the coming months.
Why are we calling this "Operation Dragonfly" ?? well.. that is for you to try to guess! We will be giving a Gandi T-Shirt to the first five people who find the link/significance of the name of this project, and send us a short one paragraph description of this significance by email to dragonfly AT gandi DOT net :) Contest ends on 14 February 2011 when the first five people to send us a correct answer will be sent their Gandi T-Shirts! (Please include your size and address in your email so that we know where to send it to, and to make sure we don't send you one for a Barbie doll!) Oh .. and before I forget, there are a few subtle hints embedded throughout this article to help you, and after the end of the competition we will also reveal the clues for everyone !
The kernel list on Gandi hosting grows. You have now the choice to use a 2.6.36 or a 2.6.32 with the grsecurity patch.
With the opening of our Baltimore datacenter, we have decided to provide a quick FAQ to respond to a few of the commonly asked questions about the hosting product and how it will work with the two datacenters, as well as some of the other features that we are putting in place.
If you have any questions not addressed here, then please feel free to let us know!
There are many reasons to create a system
image for your servers: to build a custom system with your preferred
applications pre-installed, to create an image of a game server that can be
easily deployed, to simply duplicate a custom server, or simply to backup one's
system...
The procedure is relatively simple and can be performed by anybody, as long as
you pay careful attention to the detail.
What are the modification that Gandi staff makes on standard installed OS to be used on Gandi hosting?
Local modifications
Gandi hosting infrastructure is using Xen virtualization in paravirtualization mode (for the moment). As such we have to build a Linux kernel with specific options to allow it to boot your virtual server. Moreover as we allow customers to dynamically add or remove resources, the hotplug system in the kernel was patched by our team to allow a correct use of this features (mainly correct udev call). All kernel modules are available at each new kernel release on http://mirrors.gandi.net/kernel/. As of the 2.6.32, we now use upstream kernel source for building the xenU kernel and you should find the buildconfig file in /proc/config.gz on your virtual server. We add external patches such as drbd (before upstream integration).
Each Linux base system that we provide on Gandi hosting contains modifications by our team. For example, we removed services based on hardware clock as Xen does not provide direct access to it. On some distribution we had to disable boot features such as ureadhead or plymouth to allow a flawless boot of the virtual server. The main configuration is done during the boot process, especially the first boot process.
On a side note, on x86_32 architecture, to use the hardware capability of Xen, the libc could use the nosegneg hwcap with the correct libc-xen package.
Package gandi-hosting-vm
The idea which triggered this article was the release of a new version of gandi-hosting-vm. The package contains a collection of scripts to setup the local system of your virtual server at each boot and when specific events about hosting resources happen.
Changing hosting resources
When you add or remove resources dynamically to your virtual server, the Linux kernel receives information from Xen - the system managing all the virtualization. Each of these events are passed to the udevd daemon which apply configured rules to these events. It mainly creates files in directory /dev to allow access to the newly discovered resources.
For resources that Gandi hosting allows you to dynamically change, we wrote some udev rules (located in /etc/udev/rules.d/86-gandi.rules) to start a script when a virtual disk, a virtual interface or even a virtual cpu is attached (or removed) to the virtual server.
On a more detailed level, when a virtual interface is attached, the script /etc/gandi/manage_iface.sh is called by udev and a DHCP request is sent for this interface. A couple of other scripts setup the default route (/etc/gandi/dhcp-postconf) and store network configuration (/etc/dchp-hostname) in a tmpfs directory for further configuration at the end of the boot process. When the virtual interface is removed, the script simply removes the local network interface.
When a virtual disk is attached a similar script (/etc/gandi/manage_data_disk.py) is called. It tries to check the file system on the device or in its partitions (only in GandiAI mode) and mounts the file system in a specific mountpoint using the file system label /srv/<FS label>. If no label is setup on the file system, it uses the device or partition name as mount point (/srv/xvdc1 for example). To change the default mount options, please edit the variable mount_options in the beginning of the Python script.
During the boot process
The gandi-hosting-vm package provide two services called on boot : gandi-mount and gandi-config. The first one mounts already attached virtual disks in the local system in the /srv directory (see the description of /etc/gandi/manage_data_disk.py). You can start the service again once your server is booted and it will mount attached disks to the server (if you remove udev packages for example).
The second service starts a couple of specific plugins to setup your local system. Some of these configurations are optional and a configuration file is available for you to choose to setup each of these optional features /etc/default/gandi. Each configuration variable contains a short description in the default config file. These plugins are configuring the default local console for the hosting emergency console, configure the hostname and dns resolver, change the timezone to Europe/Paris, change the hwcap nosegneg according of your kernel version, change the motd to the default and so on.
For example, when the plugin 11-config_ssh is called, it creates SSH key for the local system if the keys are not already present. Then, depending on your configuration, it could add the Gandi SSH management key to the root user keyring (variable CONFIG_SSHMGMT) and reconfigure your sshd server by disabling password access for root, disabling empty password and enabling compression (variable CONFIG_SSHD).
Package gandi-hosting-agent
Gandi agent is used to setup the virtual machine according to customer information. In case of expert mode server, the setup of the local system is limited to setting the root password and creating the administrator user (as chosen by the customer) to avoid ssh-ing the server as root. In case of a GandiAI mode server, the agent uses specific modules to setup applications on the local system.
Once your expert server is setup after creation, you can remove gandi-hosting-agent packages. For example : dpkg -P $(dpkg -l | awk '/gandi-hosting-agent/ { print $2 }' | xargs) in deb based package system or rpm -e gandi-hosting-agent in rpm backed package system.
As you are probably already aware, we have been beavering away to offer you a public API to manage your resources on our Cloud hosting platform. In order to ease the work, several things have been reorganised so as to provide an interface which is easy to use, and allow a real management of your hosting resources at Gandi.
By way of this introduction, this article will be deliberately less technical, and will only succinctly present the elements that will be developed more in-depth when the official release of the API is launched.
We are pleased to announce that you can (finally!) choose the version of your kernel that you want (from a list which will be continually expanded), and associated boot options (cmdline). 2.6.18 and 2.6.27 are "base" versions supplied by Xen (backport of Xen patches for 2.6.27). Version 2.6.32, which is is currently available, uses paravirt_ops and the "Linux" implementation of Xen patches.
We will show you the new kernels here. They can be found in the "advanced mode" within the server's management page:
And you will then be able to access:
Concerning cmdline, you may now deactivate selinux at boot, boot as a single user, change the disk and the boot partition (which is practical for working with "images"), or choose the most appropriate console for your needs. In short, everything that you need to manage your updates in a more friendly environment, or to repair your server in the most autonomous manner.
If you feel that an option is missing, please let us know.
Server hosting by Gandi allow customers to choose from a selection of OS images
available during the creation process of the virtual server. After the creation
of the image by Gandi and internal testing, a new distribution is released to
a specific group of hosting customers called 'alpha'. These clients can create
server using these release candidate images. This allows Gandi to increase the
types of testing and usage, and to find more bugs and problems by working with a small group
of its customers.
Today - May the 21th - the Mandriva 2010.0 image has been released . This new version
of the Mandriva distribution boots with a 2.6.27 kernel by default. It is currently only
available for the 'alpha' customer group but will shortly be available for all customers.
Please contact us if you wish to participate in our alpha testing phase.
16th August 2010 : Image is now available for everybody
(Translator's note: Taskigi Sequoyah would be proud! ;) )
A friend had explained to me that he had just finished installing a web server. Having decided on a change and not to use the mammoth Apache that everyone knows and loves, my friend seemed to have found a decent alternative. Bearing the name Cherokee, it appeared much leaner and with greater performance than Apache. See the benchmarks from the project website at the end of this article.
Personally, I had no knowledge of Cherokee, though I had certainly heard the name pandered around once or twice, but I never really paid it any attention at the time. So to give it a whirl, I decided to install it locally, which to my surprise exceeded my expectations. On a debian (or derived) distribution, installation is achieved by a simple apt-get install cherokee
(or 'aptitude'...)
The web management interface provided is clean and intuitive. The basic idea is quite interesting; It is possible to activate the PHP interpreter with a simple mouse click; same for activating different virtual hosts as well as other options. There are also wizards to assist with the installation of such applications and frameworks as Django, Rails or Wordpress, etc.
I thought that it would be interesting to present this project during a developers meeting at Gandi, especially for those who use GandiAI. The presentation was well received. After a few pow-wow, and a glance through the source code, there was nothing really unusual or risky. This doesn't mean, of course, that there aren't any bugs in it, but at any rate clean code is a good sign! The core is written in C with the administrator interface written in Python.
Thus is the way in which Cherokee made its appearance at Gandi, and a server image was produced. In this way you can rapidly test a server with Cherokee pre-installed.
We have also added an "expert" mode distribution with Cherokee pre-installed... You need only to select this distribution during the steps to create your server. And, just as a reminder, in case you haven't yet tried our service, you can always request a free trial using our online form!
For further information about Cherokee, you can visit the project website. Additionally, you can find a comprehensive article about Cherokee in the Gnu/Linux magazine France, written [in French] by Carl Chenet in issue number 125.
Okay, excuse the corny punch line, but for those who haven't heard the expression, to "go postal" simply means to go crazy. Without delving into the origins of the phrase, suffice it to say that the term these days can also mean to go crazy or emphatic in a positive sense as well.. (or so some would like to believe ;-) )
As many may know by now, the Gandi Mail platform in the past couple of years has seen a few recurring instances of performance degradation every couple of months. As mentioned on the Gandi Bar, our teams have been working to bring you a new and more robust mail platform. This new platform is the culmination of a considerable amount of resource investment over many months.
I am pleased to say that the new platform is fully operational and all Gandi Mail customers are fully migrated to the new system, with the migration itself taking about six weeks or so. We decided to stage the migration gradually over a number of weeks to minimise any impact to our customers, and for the most part (with a very small handful of exceptions), the bulk of the migration was completed without anyone even noticing ;)
Anyway, without further ado, let's take a brief tour of the new mail platform and what has changed in Gandi Mail version 2. Oh.. and before we go on, this article will be at times a little bit technical with the use of some acronyms and other geek-speak. Don't worry, you'll soon get the gist of what we bearded technophiles blabber on about on a daily basis! ;)
To celebrate Gandi's 10th anniversary, this hair-brained idea to give away, in ten days, 55000 domains, raise a very practical question. How, once we open the floodgates on such an operation, to maintain the highest quality of service on the site? The festive spirit could well have transformed into a nightmare for our customers if they were suddenly unable to access their management interface.
So we took the decision to host the event on a dedicated site. This was a hitherto dreamed of occasion to put ourselves into our customers' shoes, and use our hosting infrastructure for this event. We defined the rules of play: Using only the tools provided to our customers, build an architecture which was easily scalable and didn't break the bank, and to demonstrate our renowned flexibility.
As you probably already know, our platform protects you from hardware failures that might occur on your server.
In the event of a problem on the machine, or if we suspect that a problem might occur (abnormal temperature, corrupted memory, etc.), your “server” will automatically be migrated to another machine.
However, if you have an internal problem on your "server" that is not to due to the physical machine, and if it no longer responds, then you will need to take action.
Many websites start with a single server solution, a box acting as a web server and database server all in one. Simply, easy, cheap. The problem comes when traffic gets too high (a victim of their own success!). Many customers want a bigger box, but the answer is actually changing your architecture from "web server" to "web infrastructure". You can duplicate web servers, use the DNS to load balance them and ramp up your capacity very fast and very far.