"The most challenging thing at a stock exchange is high availability. The stock exchange operates for three or three-and-a-half hours every day, and the trading system has to be available for every second of these three-and-a-half hours."
So says Engineer Mohammed Khatib, Director of ICT, Amman Stock Exchange. Being at the digital helm of a stock exchange, where a country's economy can either soar or tumble, he's well aware of the problems that can come about when the IT system experiences any downtime whatsoever.
"If you're down for a few seconds, people will know about it," he says. "There's a process that you have to go through, where you have to disclose that you've had a problem, stop the service, and disclose when the service is going to come back again. The publicity is really bad and the reputational cost is extremely high.
Certainly Khatib was facing a large problem, then, given that last year, the stock exchange had issues with the high availability of its website and web services.
"We're running multiple domains -- one is for the official website, and one is for a product that we call MarketWatch, which is where you can look at the prices of the stocks and see where they're going," he says. "All in all, we're running approximately 60 apache servers, so it's quite extensive. "
The Amman Stock Exchange hosts these services over an SDN1 link -- the 155Mbps connection. Speed, Khatib says, was no issue. It was the disaster recovery site that caused headaches for the IT team.
"We had a choice of replicating everything at the DR site, but that would create problems of replicas, and the integrity of the data and switching. We thought of using the traditional cluster technology that we always used for everything else, which is Cluster A at Site A, Cluster B at Site B, and then SAN to SAN mirroring," says Khatib.
The problem with this approach, however, was that the conversion would typically take three to five minutes -- an unacceptable amount of downtime at a stock exchange. Impressively, Khatib came up with a solution using products for something they were never intended for, warranting him a speaker's slot at the recent IDC Middle East CIO Summit.
"I wanted a technology that would replicate while everything is being treated as one cluster stretched across two locations, rather than individual clusters at both locations, and it wouldn't affect the high availability," he explains.
"We started looking at storage virtualisation because any storage virtualisation appliance will give you a reliable replica of the data, while, if you have the fibre infrastructure that we have, you can have all the machines in both locations connected to the same appliance rather than to different appliances. At the same time, you cluster these appliances, so that, if one in one location fails, the other one takes over."
Sign up for CIO Asia eNewsletters.