IQ Trainwrecks

RTE news today reports that AIB Bank (one of the largest banks in Ireland) has admitted an error in applying interest rate changes over a 12 month period to so-called ‘tracker’ mortgages.

This has resulted in customers having underpaid their loans in recent months. The bank is, kindly, offering the affected customers the option of paying the shortfall in one go or spread over the remaining period of their mortgage.

‘Manual Error’ is blamed for the problem, which (according to RTE) affects 500 customers.

Tracker mortgages track the base rate of a specific Central Bank (in Ireland this is the European Central Bank), with a fixed margin payable on top of the base rate.

[Update] The Irish Times this morning reports that the Irish Financial Services Regulator is to examine the Bank’s approach to recouping the under-paid mortgage amounts. Quoting a spokesperson for the Regulator:

“In instances where an institution is seeking to recoup money as a result of the failure of its own internal systems we would expect it to be flexible in terms of the arrangements it comes to with consumers”

This story is also carried in the Irish Examiner news paper today

I’m sure most of you are familiar with the concept of the “Butterfly Effect“, that part of Chaos theory that tells us that the small disturbances of the air by a butterfly’s wings in Cleveland can cause a natural disaster on the far side of the world.

A popular myth is that the term ‘bug’ as applied to software glitches was coined by Admiral Grace Hopper after moths (and perhaps butterflies) were removed from the machinery of a Mark II Univac computer which had stopped working.

These two tidbits are related by a recent story in the Financial Times about how ‘coding errors’ (bugs) in computer models has resulted in a debt rating agency having to acknowlede they’d ranked investment vehicles wrongly and as a result increased the risk exposure of investors in those funds.

From the FT:

Moody’s is moving to re-examine the accuracy of all its computer models and place them under a centralised monitoring system after it formally acknowledged earlier this week that a glitch had appeared in one such mathematical model used to rate complex products.

Later in the article:

The moves come after the agency admitted on Tuesday that a “coding bug” had led the agency to incorrectly award top notch triple-A ratings to about $1bn of complex instruments known as constant proportion debt obligations (CPDOs) in 2006. This bug was first revealed in an FT investigation in May.

So, the bugs were found by a newspaper investigation in May 2008.. embarassing. The bugs affected the ratings applied to these CPDOs, and may have distorted the investment strategies of institutional investors. The Financial Times article in May tells us that:

The products were designed for institutional investors. In the recent credit market turmoil, those who still hold the products will have suffered some paper losses while others who have bailed out have lost up to 60 per cent of their investment.

Sheesh… up to 60% losses on investments. And it hit the papers.

The FT (28th April 2008) reports that Ratings agencies are coming under increased pressure to assure the quality of the information used in assigning credit ratings. Ratings agencies are reported to have told Regulators that:

Guaranteeing the quality of information used to assign credit ratings on structured finance securities would be “overly burdensome and redundant”

However, the burden and cost of avoiding bugs in software, processes and controls that impact on the quality of key information used in the Financial markets should be weighed against the risk of financial ‘natural disasters’ where decisions are made on foot of inaccurate information.

The Sunday Business Post newspaper (Ireland’s leading weekly paper dedicated to business news) carried a story this week about errors in calculating mortgage interest due to the 2008 Leap Year which have affected at least one Irish bank. This is an information quality problem we discussed previously on this blog.Of course, the bank’s are simply going to apply the interest missed out to the customer’s accounts retrospectively, with the average additional charge to customers being around €28.50, according to reports. However, the fact that this error has appeared in the national media, and the Halifax’s competitors are all stating that they have had no such problems, suggests that the cost to the bank may be greater in terms of negative PR. In this case the bank made a gross error in defining its information architecture and the core business rules for interest calculations (such as assuming 2008 was not a leap year), and it would seem their systems testing failed to catch the problem until after the event. However, as is often the case, the cost of non-quality information is being passed to the customer, making the immediate ‘problem’ go away (a shortfall in interest payments and profits) while masking the underlying root cause (failure to properly define and test information processing rules for core processes resulting in poor quality information). 

Courtesy of Cambridge News comes this story of a shocking electricity bill.

British Electricity provider N-Power sent a Cambridge woman a bill recently for just over stg£90million (US$ 177million). This was bemusing to her because she has availed of a pre-paid electricity meter. It would seem that for the woman to have run up such a bill would have taken over 1900 years.

As an aside, the Cambridge News article also higlights a recent survey from British website www.moneysupermarket.com:

 A survey last month showed 34 per cent of people have spotted an error in a utility or credit card bill in the past 12 months, while 17 per cent were overcharged during the past quarter.

The survey by financial website moneysupermarket.com reveals half of us don’t bother to check bills.

The Register today reports that British Gas (aka Centrica) is taking legal action against international consulting firm Accenture after a total overhaul of their billing systems costing stg£300m resulted in customers being incorrectly billed (in some instances on multiple accounts - for one customer at the same address). British Gas claims that it has had to hire 2500 extra staff to work on fixing the problems. According to The Register, GB has already written off £200m resulting from customer complaints. This means that the minimum you should estimate for the total cost of non-quality here is a figure to the north of £500m sterling (Cost of implementation + costs of scrap and rework  = ‘First cut Cost of Non-Quality). For readers in the US, that is nearly US$1bn.

And that is BEFORE the lawyers get involved. And before you take into account the cost to British Gas of lost customer revenues when customers switch to a rival supplier to get away from the problems.

British Gas claim that Accenture are responsible for implementing  a system which didn’t work and which had “fundamental errors” in its design and implementation. Accenture, for its part, rejects the British Gas claims saying that:

“Centrica directed the design, build and implementation of the Jupiter system and insisted on many of the features they now find problematic. At their own choice, after extensive testing, in March 2006 Centrica took over total control over all aspects of the system about which they now complain and has operated the system themselves for over two years.”

Accenture will be ‘vigourously defending’ the High Court action. To translate this:

1. It’s in the English High Court (”ka-ching” says the lawyer’s cash register)
2. It will be an aggressive case (meaning lots of lawyers with cash registers going “ka-ching”).
3. Customers (of Accenture and/or British Gas) will wind up paying the price in the end (the ‘Lawyers who like to say “Ka-Ching”‘ will require paying).

So, is this an IQ Trainwreck?

Yes it is. There is a significant cost to British Gas already incurred, with further costs to arise for both British Gas and Accenture. However the real impact has been on British Gas customers, who have wrestled with incorrect billing issues and related frustrations since the system went live.

Andrew Brooks, (one of our IAIDQ members in the UK and a man with some experience in these types of projects it would seem) has a nice post on his blog about the real root cause here. He makes some very valid points about who was (or should have been) driving this particular train when it went off the rails. His short post is well worth a read.

Continue Reading »

Gizmodo.com occasionally features ‘localisation’ problems (they spell it with a ‘z’, yet another localisation problem). As localisation raises challenges of customer expectation, information design, information presentation and a whole heap of other potential tripwires for Information Quality disasters, this is sometimes a good source of quick examples of how poor quality information can screw up processes.

Sometimes, however, there are stories that take it to such an extreme that they move out of the realm of ‘localisation problem’ and into the domain of ‘Information Quality trainwreck’. This following story is just such a tale. (the full Gizmodo article can be found at this link) Continue Reading »

The IAIDQ is holding its annual US conference (the IDQ Conference) in San Antonio this September. As a Director of the Association and a potential speaker at the event I’ve been researching my options for flying to the US as cheaply (but comfortably) as possible.

Imagine my dismay when I spotted that I’d missed an opportunity to take advantage of an IQ Trainwreck to get myself to the US on Business class for less than it costs me to get home from work!

Ireland’s RTE news is today reporting that Aer Lingus has cancelled the bookings made by passengers who tried to avail of a €5 (US$8.00 approx) Business Class fare from Dublin to the US, a flight that normally costs €1,775 (just under US$3,000). [update 2008-04-18]Patrick O’Beirne of Systems Modelling Ltd in Gorey has pointed out to me in an email that the <u>actual</u> cost of the flights was a bit higher when taxes, charges and the relative position of the sun are taken into account[/update]

This fare was available on the aerlingus.com website between 7:30am and 9:00am yesterday after (apparently) ‘promotional’ fares were loaded to the website in error. Around 100 people (some sources have it at 300 people, so I can’t vouch for the accuracy of this figure) managed to book the €5 fare, resulting in a potential loss to the airline of at least€177,700 (just under US$280,000).

The airline sent emails to the customers informing them that they had cancelled the bookings as the price was in error, however the Consumer Association has pointed out that the 100 people had valid contracts. The CEO of the National Consumer Agency has sought a meeting with the airline to discuss the situation.

The airline has argued that the fact that ‘economy class’ seats were advertised on the website at €249 should have suggested that there was an error in the €5.00 price.

However, the Press Release from the National Consumer Agency is quite clear that Aer Lingus would be in breach of contract in cancelling the flights and that “blaming a technical error in their booking system is not good enough”.

[update 2008-04-18] It has been reported this morning that a group of 50 property developers who had booked flights at the €5.00 price are starting legal proceedings to force Aer Lingus to honour their contracts. As we don’t have the concept of a class action suit here in Ireland, unless these property developers are part of a consortium that is a legal entity in its own right, that means that Aer Lingus is potentially facing 50 court cases, each requiring legal representation that will cost a lot more than the €17770 shortfall in the ticket price.

Ah… you just have to love lawyers.

Why is this an IQTrainwreck?

Cost to Aer Lingus of honouring the contracts will be €177700. The cost to the airline of the PR fall out from cancelling the contracts would also be significant. The legal costs that will arise now are also likely to be significant.

Yahoo is today carrying a story from AFP about a 13 year old German school boy who has corrected NASA’s calculations on the probability of ‘planet killer’ asteroid Apophis crashing into earth and causing a global catastrophe. The wunder-kind in question did his analysis as part of a regional science competition.

It seems that NASA forgot to factor in the affect of Apophis hitting one or more of the numerous satellites that orbit the earth in close proximity to the path the Asteroid will take on its next pass past the Earth in 2029. Apparently, if it hits a satellite the odds of Apophis hitting the earth in 2036 drop from a lottery-like 1 in 45,000 to a more troubling 1 in 450.

The IQ issue here is completeness of information. NASA failed to take into account the satellites in its risk model, resulting in a whopping understatement of the risk to the planet.

The short-term IQ Trainwreck comes about because the NASA scientists were corrected by a 13 year old. The long term IQ Trainwreck comes about because the 1 in 45,000 odds are probably firmly fixed in the minds of disaster recover planners around the world giving rise to degree of complacency, whereas a 1 in 450 risk might prompt some consolidated efforts to figure out how to properly manage the risk of Apophis hitting the earth or handling the ensuing global catastrophe.

However, more recent reports this afternoon suggest that the wunderkind may be a blunder-kind and may have based his model on some incorrect assumptions about the path of the asteroid. Oh dear.

In any event, the fact that a teenager is interested in this and that he researched the possiblity of the risks is commendable. Hopefully someone will take the time to reassess his work and determine if he is wrong or just not as right as he thought, improving the accuracy of the prediction models for Apophis.

Good quality information can help save the planet.  Poor quality information can send people unnecessarily into a panic.

Courtesy (yet again) of The Register.co.uk comes this salutory tale that highlights the importance of language in Information Quality, after all it is information that is being transferred when ever we communicate and the expectations of the sender and receiver of any communication can often affect how that message is understood.

 The synopsis of the tale is this…

A young girl from London town was seeking to get a taxi to the airport so she could go on holidays. She rang directory enquiries and asked for a “Joe Baxi” (slang for Taxi apparently). The telephone operator was confused (No Mr. Baxi was listed in the area) and sought a clarification. “It’s a Cab, innit” said the young lady.

So she was put on to a company that specialises in Retail display Cabinets (Cab-inet… Cab innit… you can see how this happened). And she ordered a Cab (abbreviation for Cabinet) for 10am the following morning, price £180. She paid by credit card and the cabinet was delivered the next morning as requested.

The Register quotes the marketing manager of the cabinet company:

“We thought it was a joke at first but the girl was absolutely livid. Because she spoke in ‘Ali G’ style slang, her order was mixed up somewhat. She was absolutely baffled as to why she had a big glass display cabinet delivered outside her house, when all she wanted was a taxi to take her on holiday.”

So the IQ Trainwreck angle:

1. The information was garbled because it was ‘non-standard’ - the use of slang with the directory enquiries company started the chain of events and prevented anyone from catching the confusion earlier… everyone thought they knew what the other person was talking about.
2. The impact on the Cabinet company was they incurred the cost and time of a van and driver and taking a sale that wasn’t a sale…
3. The young girl in question wound up not having transport to the airport and (we must assume) missed her flight and possibly her holiday as a result.

Similar things to this happen every day in organisations, particularly where there are no standards for information formats, standard definitions of what the things that are being managed by the business are (”what’s a customer?”) etc. etc.

A little while ago we shared the story of the Irish Rail train that left the platform at the correct time but somehow forgot 300 passengers, and left with just one visually impaired passenger.

From a reliable source we have recently learned of the findings of the investigation into what happened. It is a salutory lesson in the importance of processes, and the importance of controls and checks on processes.

The normal practice for the Irish Rail service is for passengers requiring assistance in boarding to be boarded last.  Due to some minor operational issues on the platform that evening, it seems that a controller took the decision to have the visually impaired passenger boarded before the other passengers - reversing the normal run of the process.

The guard on the train, having satisfied himself that the visually impaired passenger had been boarded safely proceeded to give the signal for the train to leave - on the assumption that if the last passenger had boarded it was time to go. It would seem that at no time did the presence of 300 people on the platform and the absence of people on the train register with the guard or the controller, so the train left.

A simple process short-cut, taken for a doubtless sound operational reason, gave information to the guard which, in his view of the process, meant it was time to go. At the risk of a bad pun we could say that ‘tunnel’ vision set in. Any number of small checks (such as a random check on carriages to see if there were any people in them) might have prevented the embarrassing problem.

The 300 passengers were accommodated on a different train which made an unscheduled stop on its route to connect with a special shuttle service that was laid on to bring the passengers to their destination. Efficient scrap and rework.

Our source also informs us that the revenue control ticket check statistics for the train that left the passengers behind showed 100% compliance that evening.