Exadata v2

Improved stat() performance on files on zfs

Bug ID: 6775100 stat() performance on files on zfs should be improved was fixed in snv_119.
I wanted to do a quick comparison between snv_117 and snv_122 on my workstation to see what kind of improvement there is. I wrote a small C program which does a stat() N times in a loop. This is of course a micro-benchmark. Additionally it doesn't cover a case if doing stat() on not cached entries in DNLC has been improved too.

So I run the program several times on each build after a fresh restart. These were the numbers I was getting on average:

snv_117/ZFS# ptime ./stat_loop test 1000000

real        1.941163151
user        0.219955617
sys         1.707997800

snv_122/ZFS# ptime ./stat_loop test 1000000

real        1.089193770
user        0.199055005
sys         0.889680683

snv_122/UFS# ptime ./stat_loop test 1000000

real        0.905696133
user        0.187513753
sys         0.716955921

This is over 40% improvement in performance of stat() on ZFS - nice.
Still stat() on UFS is faster by about 17%.

The fix could also help some very busy NFS servers :)
AFAIK it has not been backported to Solaris 10 so if you think you need it either go for Open Solaris or open a case with Sun's support and ask for a fix for S10.

Oracle: Sun vs. IBM

"Oracle and Sun together are hard to match. Just ask IBM. Its fastest server now runs an impressive 6 million TPC-C transactions, but on October 14 at Oracle OpenWorld, we'll reveal the benchmark numbers that prove that even IBM DB2 running on IBM's fastest hardware can't match the speed and performance of Oracle Database on Sun systems. Check back on October 14 as we demonstrate Oracle's commitment to Sun hardware and Sun SPARC."

FCoE Initiator Integrated into b122

Linux Kernel Development

Recent Changes to SunSSH

SXCE To Be EOL'ed

We all knew it will happen sooner or later. For some people it is probably too soon for others like me it doesn't really matter as they have been deploying Open Solaris distribution instead of SXCE for quite some time. While it might seem like a little bit pre-mature decision I believe that it will allow to better utilize available resources so the open solaris community can focus more on what's in front of us rather then putting their time into SXCE which has no future...

The official announcement:

Sun is announcing the intent to discontinue production of the Solaris Express Community Edition (SXCE) by the end of October time-frame. As we intend to continue on a bi-weekly build schedule, consolidations will move towards producing native Image Packaging System (IPS) packages alongside SVR4 packages and then phase out the latter completely. Technologies such as IPS, Automated Install, Snap Upgrade and the Distribution Constructor will be integrating into a consolidation after following through the established processes including architectural (ARC) review.

We recognize that this transition will require some effort for all members of the OpenSolaris development community, and are committed to working with all of you in making that transition a success. You can expect updated information from us and the communities which manage the consolidations as we further plan the transition schedules.

Questions can be directed to David Comay, Glynn Foster, William Franklin, Stephen Hahn, Dave Miner, Vincent Murphy, or Dan Roberts.

Read or Write Only Process

PSARC/2009/378:

This project proposes two new "basic" privileges.

FILE_READ
Allows a process to read a file or directory whose
permission or ACL allow the process read permission.

FILE_WRITE
Allows a process to write a file or directory whose
permission or ACL allow the process write permission.

The purpose of these privileges is the ability to create a "read-only" (no FILE_WRITE privilege) and a "write-only" (no FILE_READ privilege) process.

The FILE_WRITE basic privilege is required for any modification to a file or directory: open(2), creat(2), link(2), symlink(2), rename(2), unlink(2), mkdir(2), rmdir(2), mknod(2) etc.

The FILE_READ basic privilege is required for opening a file with O_RDONLY or O_RDWR.

Note: a "basic" privilege is a privilege which is part of the default I, P and E privilege set.

ZFS: logbias

PSARC/2009/423:

Summary

Provide zfs with the ability to control the use of resources used for synchronous (eg fsync, O_DSYNC) requests. In particular it enables substantially better performance for Oracle and potentially other applications.

Background

Oracle manages two major types of files, the Data Files and the Redo Log files. Writes to Redo Log files are in the path of all transactions and low latency is a requirement. It is critical for good performance to give high priority to these write requests.

Data Files are also the subject of writes from DB writers as a form of scrubbing dirty blocks to insure DB buffer availability. Write latency is much less an issue. Of more more importance is achieving an acceptable level of throughput. These writes are less critical to delivered performance.

Both types of writes are synchronous (using O_DSYNC), and thus treated equally. They compete for the same resources: separate intent log, memory, and normal pool IO. The Data File writes impede the potential performance of the critical Redo Log writers.

Proposal

Create a new "logbias" property for zfs datasets.

If logbias is set to 'latency' (the default) then there is no change from the current implementation. If the logbias property is set to 'throughput' then intent log blocks will be allocated from the main pool instead of any separate intent log devices (if present). Also data will be written immediately to spread the write load thus making for quicker subsequent transaction group commits to the pool.

To change the property, an admin will use the standard zfs set command:

# zfs set logbias=latency {dataset}
# zfs set logbias=throughput {dataset}

Web Stack 1.5

.

Radiation-Hardened SPARC

Atmel Introduces the AT697F Radiation-Hardened SPARC V8 Processor for Space Missions.

Joerg explains on why SPARC?:

"The reason is not Solaris. It´s a different one. Despite general opinion, SPARC isn´t a proprietary architecture. You get go to SPARC International, get the specification for a 100 bucks or so and you can develop and manufacture your own SPARC CPU. This was used by the European Space Agency to develop a radiation hardened version of a SPARC architecture. The development is called LEON, the proc named in the article uses the LEON2-FT design. The LEON2 and LEON3 design are available under the GNU General Public License respectively under the GNU Lesser General Public License. Being able to get such an architecture essentially for cheap money was the essential reason behind the decision for SPARC (besides of other technical reasons)"

Why Power 4.7GHz is slower than SPARC 1.6GHz?

RAID-Z3

RAID-Z3 (triple-parity RAID) has been implemented and integrated into snv_120 (bug id 6854612) - thanks to Adam Leventhal. Read more on Adam's blog.

Windows 7

Alright Ben you've convinced me to try Windows 7 especially as you can download it and try it for free.

MySQL - 8x Performance Improvement

I came across an interesting problem today. A perl script running a mysql query and it takes too much time to complete. It spends almost all its time while waiting for this mysql query (anonymized):

select a, b, registered, c, d from XXX where date(registered) >= date_sub(date(NOW()),interval 7 day)

The problem is that there are over 70 million rows in the XXX table and the query takes over 7 minutes to complete mostly waiting for a disk I/O.

explain select a, b, registered, c, d from XXX where date(registered) >= date_sub(date(NOW()),interval 7 day)\G

          id: 1
 select_type: SIMPLE
       table: XXX
        type: ALL
possible_keys: NULL
          key: NULL
     key_len: NULL
         ref: NULL
        rows: 72077742
       Extra: Using where
1 row in set (0.02 sec)

So the reason it is so slow is that mysql does not use an index here. It turned out that if you use a function on a column in a where statement then mysql won't use an index!

There is a reason why the statement is using date() and date_sub() functions as it is expected to compare dates where time is 00:00:00 so I can't . But one can cast() functions to timestamp and used registered directly which will allow mysql to use index:

explain select a, b, registered, c, d from XXX where registered >= cast(date_sub(date(NOW()),interval 7 day) as datetime)

          id: 1
 select_type: SIMPLE
       table: XXX
        type: range
possible_keys: YYY
          key: YYY
     key_len: 9
         ref: NULL
        rows: 1413504
       Extra: Using where
1 row in set (0.10 sec)

After the modification the script takes about 50s to execute compared to over 7 minutes which is a very nice 8x performance improvement! Not to mention a much less impact on a database server.