…When you have knowledge, skills and experience these are are crucial to make you successful in any area of life.[...]…

Because I wasn’t there, I’ll have to interject here instead…

“Coping without high availability is not a problem for banks”

Untrue as we approach the Teenies (if this is the Noughties…) for two reasons.

Firstly, there is an increasing demand for day+1 or day+0 data, and eliminating the batch window altogether is an aspiration for many financial organisations.

(Or at least it was, before the focus changed to hoarding cash – the perception is unfortunately changing back to seeing “IT” as a pure cost rather than an opportunity to manage other costs and generate value. A cynic might say this was due to blind panic in the boardroom.)

Secondly, there are two issues of scale. Moore’s law brings down cost-per-TB and brings up general capability, but in data warehousing there is a strong tendency for Parkinson’s law to work in the reverse direction, and so the batch window stands still like the Red Queen. We still have 2-4 hours in most organisations and rather than the time shrinking, the window is treated as the constraint and there is a desire to do as much data movement as possible in the time.

The other issue IS one of good old fashioned “backup.” If you have 20TB of data to back up every night, how the hell do you do it?

My answer is to divine what you actually need to back up because it does not exist elsewhere, and which other portions could be recreated from the source within the RTO. HSM and other creative disk-based solutions can also be a partial answer, but banking-proof bandwidth between sites remains expensive. Never underestimate the bandwidth of a lorry full of tapes; but the scale of warehouses is outpacing the growth of tape technology.

So the concept of a backup window remains with us, because divining what NEEDS to be protected at the raw data level, and then backing up only changes, is still in the “too hard” pile for many organisations.

So – the RAQUEL model and similar ideas, such as Kognitio’s technology, are fantastic opportunities, but the operational implications are complex to get right and if we are not careful will kill the concept altogether for the corporate world. Google and Amazon have demonstrated that you can do it right, and in the case of the latter, Amazon have got scalability so right they sell it to other people now as a technology company, rather than as a purveyor of books and various other FMCG.

Many banks, on the other hand, have demonstrated how wrong you can get it and have spent an awful lot of money with IBM, Oracle, Teradata, and co for very little real gain.

Excellent overview – well written