vmwarez.com
           -Where Virtualization is a Reality!

 

Tuesday, May 09, 2006

Super Fast Server Reboots – Another reason Virtualization rocks.

For some reason, this server goes crazy about once a week. I think it has something to do with the statistics software. We're looking at moving that to its own server one of these days. This software we use has about 3 or 4 different services that run and is not the easiest thing to restart. So, when it goes nuts, we just reboot the server. I think they are planning a simpler restart function for the next version. Besides that, though, I am a firm believer in rebooting servers weekly. I know that will make some of you Linux guys think I'm crazy, but that is just the way I am. I will go into more details as to my reasoning later.

If this were a physical server, it may be worth the 10 minutes or so to wait for the prompts and go through all the trouble of restarting the stats software rather than rebooting. I say 10 minutes because that's how long it seems to take a physical machine to boot... I've not timed one, so I don't have an exact number, but it goes something like this on a Dell:

  • BIOS – 15 to 30 seconds or more depending on the amount of RAM
  • Adaptec BIOS – 15 seconds, easy
  • PERC RAID BIOS – 30 more seconds unless you have to wait for the drives to spin up.. then you're looking at a minute or so.
  • Remote Management BIOS – another 30 seconds
  • Boot OS


On a virtual machine, it goes like this:

  • BIOS – 5 seconds
  • Boot OS


I am in the habit of starting a consistent ping while waiting for a server to reboot. That way I would have an idea of about when to expect it to be operational again. At times, when rebooting physical servers, I get nervous waiting and watching the ping responses. The server room is down stairs and on the other side of the building and often I don't feel like going there just to watch it reboot – unless there are problems. Also, we don't have an IP KVM, so I can't see the console of our physical servers when they reboot – unless I go to the server room.

Enter the virtualized server: We didn't spend the $$ on an IP KVM because we have VMware's Virtual Center. The console for each virtual machine is a window on my PC.... but that's off topic. What I really want to show you is my ping window for rebooting this server:



This is Windows 2003 Server Standard running in a virtual machine on a Dell 2850 Dual Xeon 3.6Ghz running VMware ESX 2.5.2.

This is one of the things that makes me love virtualization more and more every time I experience it.

 

5 Comments:

  • Jim, I have a similar issue with it as well. I also have Dell servers, 2850 and 2850s. They seem to take an incredibly long time to POST and boot. Also like you, I am not sitting in the Lab or Data Center, and I don't have a KVM over IP solution in place. I also have virtualization platforms available, and it is such sweet victory to not only be able to watch the VMs POST and boot, but to also watch them boot within seconds. Long live virtualization!

    Dave

    By Anonymous David Marshall, at May 09, 2006 12:37 PM  

  • Jim,
    Elsewhere you say that you are a "Windows Head", that's fine, I used to be one too.

    I agree with you about the need to reboot Microsoft Windows Operating Systems at regular intervals. This is required because MS OSes do such a poor job of memory managment and program isolation. On MS Windows, many programs are constantly leaking memory, even the OS leaks memory, and it seldom gets reclaimed without a reboot. Also when a program crashes, it usually leaves the entire OS in an unstable state. Frequent ReBoots are required.


    But Linux is different. It's architecture is totally different. Linux is as close to bullet proof as you can get. 99% percent of the time that a program blows up, it does so in total isolation and does not affect the other programs. Even if a program has a severe memory leak, the OS contains it and reclaims all of the memory at termination. This behavior is light years ahead of any OS that Microsoft has ever produced. I have run production Linux servers for over a year without the need to reboot. I only reboot Linux when I am feeling lazy and not wanting to restart the programs by hand.

    Here are a couple of things that you may find helpful: I made a master init.d file to manage my core apps for when I need to do a fast restart of a set of apps in sequence. I have also added a "sighup" option to many of my init.d scripts -- a very nice trick! gives you a consistant way to control the app.

    The only time that you really need to do a restart of Linux is when you want to force a disk check.

    P.S. One of my sites is also using a Dell server with Windows 2003, it has 400 gigs of raid 5 and 4 gigs of RAM. I *have* timed it, and it takes about 6 minutes to shutdown and over 10 minutes to start up. which is totally insane for a 24x7 operation. Note that I have it do an auto logon, no intervention is required.

    Erik

    By Anonymous Erik, at August 29, 2006 2:29 AM  

  • I've heard the weekly reboot espoused by Linux junkies too, as a crude method of change control. If an extended power failure depletes your UPS, you might not be around next time the machine brings itself up. Do your init scripts reflect any changes you've made recently? A weekly reboot forces you to keep valid init and rc scripts while the changes are still fresh in your mind.

    Just imagine if your last reboot was a year ago, and after a power event, something you installed 11 months ago failed to start. Would you even remember where to look?

    By Blogger Myself, at December 19, 2006 2:09 PM  

  • We run multi million dollar production physical Windows servers with uptime for months, and have no such memory leaking problems.

    Get some skilled windows admins in your environment and you will see improvements here.

    I love the speedy boot times of VMs. Even with Linux OS's, boot times can be long. Bottom lines is that Virtualization is here to stay, bringing huge benefits to Windows and Linux OS workloads.

    By Anonymous Anonymous, at December 28, 2007 11:11 AM  

  • "We run multi million dollar production physical Windows servers"

    Ouch! Is that what M$ charges these days for Windows?

    Sorry I couldn't resist.

    By Anonymous Anonymous, at September 10, 2008 12:22 PM  

Post a Comment

<< Home