28 June 2010 Features

 

Co-location: it's a small world

David Bannister

As proximity hosting solutions and co-location services squeeze more and more of the world's trading systems closer together, they are creating isolated pockets. Surely something has to give?

Has anyone started talking about nanotrading yet? If not it's only a matter of time: as the scale of securities trading grows, the technology that the systems run on is shrinking, and that is already down at the nano-scale.

The much-discussed topic of latency is now approaching this, with the tick frequency of data on the OPRA data feeds down at 1 microsecond (µS). Below this, and you're into nanoseconds.

The effect of this has been discussed on many occasions, not least at Intel's fasterCITY series of events. At one of these earlier this year the focus was on how, where, why, and when to measure and report latency, "Indicating a new maturity in the market as opposed to random numbers from isolated benchmark measurements", according to Nigel Woodward, Intel's director for financial services.

Woodward posed the question, "With ever more rapid implementation of trading technologies, and niche new entrants to the market, was there a danger of short cuts in the rigour of technology design and therefore an operational risk - that if it became part of the DNA of market operations could expand to systemic proportions?".

Anthony Warden, global head of algorithmic trading technology at Nomura, stressed that they apply rigorous attention to risk in consultation with clients, and welcomed the controls and prudence of market regulation, while other firms present added that they see that they have a responsibility that they take seriously to provide stable and reliable markets and invest in technology and procedures accordingly.

Chris Pickles, head of marketing for financial markets and wholesale banking at BT, pointed out that regulation such as Basel II and MiFID each specifically addressed operational risk and, via principles-based regulation, required firms to implement effective controls.

Another speaker, Larry Ryan, chief technologist for financial markets at HP, took the debate down to the physical level, outlining the implications that things at the nano level will have for industry structures at the macro level.

Expanding his view of the complete architectural solution stack for high performance trading environments, Ryan pointed out that one of the issues surrounding the latency debate is the fact that while the timescales are moving downwards, some parts of the technology stack are not keeping up. Standard operating system and internet practices are off by "three orders of magnitude", he said.

For instance, all low latency systems are now using solid state disks, which reduces the head seek time of traditional drives (14 milliseconds down to as low as 50 microseconds, a factor of 280), he said.

While this is causing bottlenecks, traders are "evolving into quants" and deploying exotic systems such as pushing Twitter feeds into Complex Event Processors and using them to drive algorithmic trading models.

Inevitably, this is leading to a build-up of systems in data centres close to the heart of the world's major trading centres - Ryan describes them as "epicentres" of trading systems - but there will come a point when the firms may actually be sharing CPU power within the same systems in the co-location data centres.

This, he says, will provide an interesting challenge for the lawyers as service level agreements will be very contentious down at this level.

As IT moves further towards server virtualisation, this can only become more confused. "I'm sure it already happens," says one trading IT architect. "We have a blade in a server rack, you have a blade in the next rack: then they bring out the next generation of blades, and we're both on that - who's stealing clock cycles from who?"

Back up at the human scale, the problems of concentration are creating the possibility of a two-tier market developing - those who can afford front row seats in the dedicated co-lo centres, and those who can't. "It could be argued - and in this world that means that it will be argued - that this contravenes a firm's MiFID obligations to provide best execution," says the architect. "Yes, you'll have a best execution agreement with your clients, and SLAs with the host to ensure that you can match those obligations, but you're hanging out a sign out saying you're in the bargain basement, which is less than marvellous in a market obsessed by speed."

There are, unsurprisingly, technology approaches available such as TimeKeeper from FSMLabs, a software product for "critical applications that helps capital markets and other regulated industries deploy critical applications across clusters, among data centres and over wide-area networks, while meeting requirements for accurate time-stamping and data logging for high-frequency and low-latency trading, market data streams and other transactions".

"High-speed, fragmented markets and cross-asset trading have together focused the spotlight on latency," says Bob Giffords, an independent banking and technology analyst. "Successful low-latency operations demand an ability to synchronise clocks across server clusters in proximity trading hubs. FSMLabs TimeKeeper appears to have significantly raised the bar, increasing confidence in cross-server software metrics in the low microsecond range. Anyone who is serious about high frequency trading should take note."

TimeKeeper is accurate to 10 microseconds or less, which "adds credibility to trade communications among parties and forms a solid foundation for latency measurement", says the company.

Joining the dots

Interconnecting these islands of processing power is also pushing the boundaries of telecoms technology. Many of today's high-speed networks use link speeds of 1Gigabit/s, over which a 1kilobit packet of data will take a microsecond. More and more trading systems are moving to even higher speed interconnect technologies, such as 10 Gigabit/s Ethernet or InfiniBand, which can transmit data up to 40 Gigabit/s.

Now, the venues are starting to look for 100 Gigabit/s. Last year, NYSE Euronext network specialist Ciena announced plans to implement the first 100Gigabit/s network in the US, Europe and globally.

Ciena's 100G technology forms a core infrastructure component to support the growing bandwidth demands of NYSE Euronext's new state-of-the-art data centres in the greater New York and London metropolitan areas, which are going live now as the company's operational hubs for more than one billion daily. (NYSE Euronext hosted a opening celebration at its Basildon centre just last month).

"It is critically important for NYSE Euronext, in our dual role as an exchange operator and technology company, to ensure a technology and operational infrastructure of the highest quality and integrity, and to provide the highest levels of speed and capacity, lowest latency and maximum flexibility," said Stanley Young, chief executive of NYSE Technologies and co-global CIO of NYSE Euronext. "Powering our new data centres and network with Ciena technology will help produce a clear competitive advantage for NYSE Euronext and our customers in terms of executing trades and delivering mission critical information as well as advancing leading edge applications such as state-of-the-art co-location facilities."

NYSE  Technologies' US and European communications engineering teams, led by Andy Bach, senior vice president and global head of communications, as well as global IT procurement and legal support services, worked for more than a year to define an architecture and select a partner to jointly define, create and deploy a solution that would position NYSE Euronext "at the forefront of ultra-low latency executions and market data distribution" before settling on Ciena and its CN 4200 RS FlexSelect Advanced Services Platform.

Steve Alexander, chief technology officer at Ciena, said: "As the traffic demands of enterprises and service providers continue to escalate beyond current network capacities, our solutions provide customers with network scalability from 1 Megabit/s to 100 Gigabit/s, ensuring a flexible, simplified and operationally-efficient network foundation to support a wide variety of applications and services."

But while it may be state-of the art-today, you can bet it will be commonplace very soon - things move fast in this high-frequency, low-latency world. BT

Panel - A question of time: There are orders of magnitude difference in the capabilities of systems.

Microseconds: Millions of events per second

• Time between OPRA message ticks: 1µs
• Time for light to travel 1km: 3.3µs
• 100 bytes of network traffic @ 10Gb/s: 0.1µs (100picoseconds)
• Switch Latency: 500ps-2.5µs 
• 1,000 x86 instructions: 0.3µs
• RDMA App/App: 1µs -500µs

Milliseconds: Thousands of events per second

• Ping to Google: 16ms
• Disk Head Seek: 14ms
• SSD seek: 50µs -100µs
• Operating System Timer Tick: 10ms