I've been following a number of threads on forums.oracle.com recently - the quality of discussion seems to have improved markedly there since I gave up on it in 2001. Anyway there was a thread about interpreting the AWR report that Oracle provides (at extra cost) and which is very smilar to the Statspack report. The report in question comes from a siebel system and there were some interesting snippets of information from Joe Coffey about the specifics of working with SIEBEL which would likely trap the unwary (like me) when using sqlplus to investigate an application performance problem. The specifics are in this post but are that siebel specifically alters the standard oracle environment on login with 4 ALTER SESSION statements - namely

• alter session set optimizer_mode = first_rows_10;
• alter session set hash_join_enabled = false;
• alter session set "_optimizer_sortmerge_join_enabled" = false;=
• alter session set "_optimizer_join_sel_sanity_check" = true;

Now first_rows_10 is rather sensible for a reporting application where the end user is going to look at pages of results - I'm rather less convinced about the wisdom of avoiding hash_joins and sortmerge joins altogether, but the base point to remember here - for myself as much as any reader - is that when conducting tests using sqlplus (or toad or any other query environment ) it matters that you ensure that your session mimics effectively the one the users will be using.  

I'm just starting an installation of Oracle APEX 3.1.1 into one of our development ebs instances for 2 reasons.

1. To provide a small applet to a project team
2. To provide the infrastructure for the dbas to write their own little applets.

I chose this method for the following reasons (unordered)

• it's fairly cool
• we know pl/sql much better than java
• we don't much like forms.
• you can integrate really easily.
• deployment is pretty simple.

Those of you that have installed apex frequently will know it's a pretty quick install on decent hardware. So the following came as rather a surprise.

timing for: English Dictionary
Elapsed: 00:03:16.71
-- Updating user account expiration. -------
timing for: Upgrade
Elapsed: 00:00:00.43
...End of install if runtime install
...create null.sql
timing for: Development Installation
Elapsed: 01:03:48.57

That's right over an hour. The reason for this is that with 3.1.1 APEX calls DBMS_STATS.GATHER_DICTIONARY_STATS('APEX'); this spend almost 55 minutes performing inappropriate index lookups on the data dictionary for us. This might be solvable, though to be honest it's probably not worth it, but just a heads-up really, the APEX install may take longer than you think in a large environment and with the latest version of the software.

A good day today. I was privileged enough to be at the paper selection day for the UKOUG conference in December 2008. For those who don't know what happens, and perhaps suspect some sort of elite giving themselves presentation slots, here is roughly how it works. 

Firstly a reasonably large group of reviewers from around the world, though naturally UK biased,  score your abstract on a scale from 1 (very poor) to 6 (excellent). They also have an opportunity to comment. These scores are then collated and a small team review the scores and allocate presentations to available slots. The purpose of the 2nd review is twofold - first it allows the team to review the agenda for balance of both topics and target audience - this year for example you will be especially well-served if you are a relatively new dba - and secondly it allows for the moderation of some of the scores - where for example only a very few people have scored a particular abstract. Finally, because some presenters get consistently high scores (Jonathan Lewis being the obvious prime example)  then there is the opportunity to ensure that other, maybe lesser known, presenters also get a look-in as well.

So what did this mean this year, well we had 212 submissions for the Server Technology arena (apps dba submissions are separate). We needed to fill 64 slots. So that means we needed to eliminate 7 out of 10 submissions. The average score for  this stream was 4.5 (that's halfway between good and very good.)  and on average over 18 reviewers would have scored your submission. To get into the Top Quartile (more or less to be guaranteed a place) then you needed to score 4.93 (Very Good) on average. To restate - to stand a good chance of getting in, your abstract needed to show a group of nearly 20 Server Technology specialists that it was either very good or excellent. The bar to present at UKOUG is extremely high. Those who will be receiving congratulations communications should feel justly proud. Those who are planning to go, well you should be spoiled for choice. You can register your interest (and maybe win a laptop) here.  

Richard Foote updated his blog today with a description of the 3 step process for troubleshooting technical problems with business systems. Briefly his 3 steps are

1.  Identify an actual problem that needs addressing, one that’s problematic to the business, not one that only exists in some statistic or in one’s imagination
2.  Determine what’s actually causing the problem as identified in Step 1.
3.  Address the specific issue as identified in Step 2.

This started as a comment, but grew a bit. I suspect that most of the time the 'difficulty' lies in step 1. Identifying a problem that is causing drag on your employers business. This requires at least: 

1. understanding the business in the first place.
2. specifying to a high degree of certainty the issue.
3. quantifying the impact.

IT staff are notoriously bad at 1) and 3) and business staff are notoriously bad at 2) and 3). For example some colleagues of mine went to a meeting with business users of a core system that has historically suffered significant downtime. We identified and made some infrastructure changes that have reduced the downtime by approximately 40 days a year (that's right this system was running at circa 80% availability).  The system has been running in it's new configuration at over 99% availability, and helpdesk calls have all but vanished. The meeting was quite difficult since the business users wanted to complain about the stability of the system. In particular they were upset with the 99% availability statistics because they felt that the stats did not reflect reality, which was that occasionally data was 'lost' or application sessions were apparently hung. The fact that other users could continue to work did not mean that the service was available.

This illustrates particularly well my point 2, the technologists involved had understood a problem statement "the system is often unavailable" in terms of the uptime of the application - i.e Can I log on? The business users on the other hand interpreted the exact same statement as meaning "we often encounter unexpected errors when using the application".