Hosting on Solaris instead of FreeBSD

Hosting webapps on Solaris instead of FreeBSD? Why not :) Looks like that is exactly what Joyent is going to do. Look at this pdf - how they consolidate on Niagara boxes, use zones, zfs, etc.

Software RAID-5 faster than RAID-10

Software RAID-5 solutions were always much slower in writing data than RAID-10. However ZFS completely changes that picture. Lets say we've got 4 disks - with all the other software RAID solutions when you create RAID-5 from these 5 disks then writing to them will be MUCH slower than if you created RAID-10. But if you create RAID-5 (in ZFS called RAID-Z) from these 5 disks the you will see that writing performance is actually much better then RAID-10.
I did such a test today with 4 disks and sequential writing to different RAID levels using ZFS and 'dd' command.

ZFS RAID                   ##  write in MB/s
#############################################

RAID-10   (mirror+stripe)  ##   117MB/s
RAID-Z    (RAID-5)         ##   175MB/s
RAID-0    (striping)       ##   233MB/s

In theory ZFS in this test (writing) should give us the performance of two disks in case of RAID-10, the performance of 3 disks in case of RAID-Z and the performance of 4 disks in case of RAID-0. And that's exactly what we see above! (117/2 = 58.5000 ; 175/3 = 58.3333 ; 233/4 =58.2500).

MySQL on Solaris faster than on Linux?

Yahoo reports that MySQL on Solaris is faster than on Linux.

SANTA CLARA, Calif., April 21 /PRNewswire-FirstCall/ -- Sun Microsystems, Inc. (Nasdaq: SUNW - News) today announced new benchmark results involving the performance of the open source MySQL database running online transaction processing (OLTP) workload on 8-way Sun Fire(TM) V40z servers. The testing, which measured the performance of both read/write and read-only operations, showed that MySQL 5.0.18 running on the Solaris(TM) 10 Operating System (OS) executed the same functions up to 64 percent faster in read/write mode and up to 91 percent faster in read-only mode than when it ran on the Red Hat Enterprise Linux 4 Advanced Server Edition OS.

"MySQL and Sun have worked to optimize the performance of MySQL Network software certified for the Solaris 10 OS," said Zack Urlocker, vice president of marketing for MySQL AB. "This benchmark demonstrates Sun's continued leadership in delivering a world-class operating environment that provides excellent MySQL performance for our mutual enterprise customers."

I wonder if it means there were some changes to MySQL so it runs faster on Solaris?
In our internal tests MySQL on Linux (however older MySQL versions) was actually slightly faster than on Solaris - maybe we should re-check it. Unfortunately not much info on actual system tuning and disk config is reported (was ZFS used? how much IOs were generated, etc.).

Actual benchmark is here and official Sun announcement here.

T1000 arrived

T1000 has just arrived! You can expect some production performance numbers soon :)
As T1000 uses only two memory controllers out of 4 in US-T1 it could be slower than T2000 with the same CPU - I guess it could depend on application - we'll see how it's here.

Predictive Self Healing

Yesterday system which is E6500 with Solaris 10 reported few times memory errors (ECC corrected) on a board 0 dimm J3300.

Apr 18 17:45:30 server SUNW,UltraSPARC-II: [ID 197328 kern.info] [AFT0] Corrected Memory Error detected by CPU8, errID 0x0002de3b.e98a88fa
Apr 18 17:45:30 server     AFSR 0x00000000.00100000 AFAR 0x00000000.c709d428
Apr 18 17:45:30 server     AFSR.PSYND 0x0000(Score 05) AFSR.ETS 0x00 Fault_PC 0x11e7fe8
Apr 18 17:45:30 server     UDBL Syndrome 0x64 Memory Module Board 0 J3300
Apr 18 17:45:30 server SUNW,UltraSPARC-II: [ID 429688 kern.info] [AFT0] errID 0x0002de3b.e98a88fa Corrected Memory Error on Board 0 J3300 is Intermittent
Apr 18 17:45:30 server SUNW,UltraSPARC-II: [ID 671797 kern.info] [AFT0] errID 0x0002de3b.e98a88fa ECC Data Bit  7 was in error and corrected

As these errors happened few more times system finally removed single page (8kB) from a system memory so problem will not escalate and won't possibly kill system or application. Well, I can live with 8kB less memory - that's real Predictive Self Healing.

Apr 18 19:14:51 server SUNW,UltraSPARC-II: [ID 566906 kern.warning] WARNING: [AFT0] Most recent 3 soft errors from Memory Module Board 0 J3300 exceed threshold (N=2, T=24h:00m) triggering page retire
Apr 18 19:14:51 server unix: [ID 618185 kern.notice] NOTICE: Scheduling removal of page 0x00000000.c709c000
Apr 18 19:15:50 server unix: [ID 693633 kern.notice] NOTICE: Page 0x00000000.c709c000 removed from service

Booting from ZFS

Booting from ZFS was integrated in b37 which is available as SXCR right now. It's not a final solution - it's ugly but it's a beginning. The real fun will begin when GRUB will understand ZFS. You can see instructions on how to boot from ZFS in b37 at Tabriz's weblog.

T2000 scalability

Another benchmark showing T2000 scalability in various microbenchmarks and comparing it to 12-way (24 cores) E2900. Most of the benchmarks are HPC focused.

BrandZ DVD

SystemNews:

"If you are interested in downloading the SolarisTM Containers for Linux Applications functionality, BrandZ DVD 35 for the SolarisTM Operating System (Solaris OS) for x86 platforms is available now as a free download from the Sun Download Center. Based on SolarisTM Express build 35, this download is a full install of Solaris OS and includes all of the modifications added by the BrandZ project.

BrandZ is a framework that extends the SolarisTM Zones infrastructure to create Branded Zones, which are zones that contain non-native operating environments. Each operating environment is provided by a brand that may be as simple as an environment with the standard Solaris utilities replaced by their GNU equivalents, or as complex as a complete Linux user space. The brand plugs into the BrandZ framework.

The combination of BrandZ and the lx brand, which enables Linux binary applications to run unmodified on Solaris OS, is Solaris Containers for Linux Applications. The lx brand is not a Linux distribution and does not contain any Linux software. The lx brand enables user-level Linux software to run on a machine with a Solaris kernel, and includes the tools necessary to install a CentOS or Red Hat Enterprise Linux distribution inside a zone on a Solaris system.

The lx brand will run on x86/x64 systems booted with either a 32-bit or 64-bit kernel. Regardless of the underlying kernel, only 32-bit Linux applications are able to run.

As part of the OpenSolaris community, BrandZ is a hosted project on this open source web site that contains detailed information on BrandZ and the latest information on build 35."

ps. yes, this is continuation of the famous Janus project (LAE)

Solaris Internals & Debugging

Finally new book from authors of the old Solaris Internals book - actually two books. One (over 1000 pages) about Kernel Architecture in Open Solaris and Solaris 10 and the other one about Solaris Performance and Tools (yep, DTrace). This time there are three authors: Richard McDougall, Jim Mauro, Brendan Gregg. Judging from the last book they wrote these two already are a must have books.

More info here.

CPU caps

New way of CPU management - CPU caps. It allows you to set limits on applications (projects and zones) in a percentage of single CPU (although application can still use all CPUs). Right now this is available as a preview on Open Solaris Resource Management Project.

Oracle 10g for Solaris x64

Finally Oracle 10g R2 64bit version for Solaris on x64 is released.

Another non-Sun Niagara test

It's good to see another Niagara tests done by people outside Sun - customers. We found Niagara servers to be really good in many workloads, not just www (both comparing to traditional SPARC servers or x86/x64 servers). Here you can find benchmark of T2000.

So, after a week with the Niagara T2000, I’ve managed to find some time to do some more detailed benchmarks, and the results are very impressive. The T2000 is definitely an impressive piece of equipment, it seems very, very capable, and we may very well end up going with the platform for our mirror server. Bottom line, the T2000 was able to handle over 3 times the number of transactions per-second and about 60% more concurrent downloads than the current ftp.heanet.ie machine can (a dual Itanium with 32Gb of memory) running identical software. Its advantages were even bigger than that again, when compared to a well-specced x86 machine. Not bad!

My patch integrated in Open Solaris

Finally my patch has been integrated into Open Solaris build 37. I must say that a procedure to integrate a patch into Open Solaris is really easy - just send a request for a sponsor (someone to help you) to request-sponsor list then someone will offer his/her self to be your sponsor and that's it. Of course it would be a good manner to first discuss the problem on related Open Solaris list if it involves new functionality, etc. If it's just simple bug you can skip this part.

What is the patch I wrote about? It adds functionality to ZFS so you can list and import previously destroyed pools. It was really simple but I guess it would be useful. The actual RFE is: 6276934. This should be available in Nevada build 37 and it looks like it will make it into Solaris 10 update 2.

FMA for Opteron

Some time age I wrote that FMA enhancements for AMD CPUs are integrated into Open Solaris. Thanks to Gavin Maltby here are some details. Really worth reading.

The Rock and new servers

The Register has some rumors on new Rock processor:

The Rock processor - due out in 2008 - will have four cores or 16 cores, depending on how you slice the product. By that, we mean that Sun has divided the Rock CPU into four, separate cores each with four processing engines. Each core also has four FGUs (floating point/graphics units). Each processing engine will be able to crank two threads giving you - 4 x 4 x 2 - 32 threads per chip.


Sun appears to have a couple flavors of Rock – Pebble and Boulder. Our information on Pebble is pretty thin, although it appears to be the flavor of Rock meant to sit in one-socket servers. Boulder then powers two-socket, four-socket and eight-socket servers. The servers have been code-named "Supernova" and appear impressive indeed. A two-socket box – with 32 cores – will support up to 128 FB-DIMMs. The eight-socket boxes will support a whopping 512 FB-DIMMs. Sun appears to have some fancy shared memory tricks up its sleeve with this kit.