Computers Cannot Replace Human Traders, but They Can Help--- A Quick Look at Big Data’s Impact on Trading in Capital Markets

I was watching a SAS industry analyst webinar which was made available via Linkedin the other day. There had been plenty of focus on mega power analytics, big data and various memory computing. One speaker from Gartner, one of the most respectable IT research institutions on the street, spent most of his coverage illustrating the impact of big data on financial services industry -  especially on high frequency trading. He particularly quoted an article in Wired about a new trend in academia research on big data and high frequency trading. The study indicated that there had been more than 1800 flash events in money markets that were traded during the past five years. My boyfriend, who serves in the trading floor of a bulge bracket firm, also commented that there has been a strong economic incentive for all those Wall Street banks to relocate their systems and vendors as near as possible to the exchanges, no matter in New York or in Chicago. By placing and receiving an order a tenth of a second faster, the firm would be able to fish on pop out events in the capital markets. Meanwhile, such events could also massacre the players out of control. For example, last year Citigroup got jerked around so crazily by high frequency trading that there was little correlation coefficient between  the company’s fundamentals and the stock price per se.

VIDEO:http://origin-www.bloomberg.com/video/firms-replacing-derivatives-traders-with-algorithms-An0I9~hBSgaNvTgyuV6FXg.html

Last year research institution SunGard summarized 10 trends shaping big data utilization. Among the ten quotes, the very first one is ‘Companies require larger market data sets and deeper granularity to feed predictive models to forecast profitability and hedge risks in trading throughout the day’. It was not news that sales and trading look huge advantage on data analytics. But my question is how big the push is? Some investment appraisers even doubted traders would not be necessary after foreseeable decades.

Here are some numbers to answer that question. The application of big data has improved the speed and accuracy of trading transactions on average by 13% and that impact will grow to 37% over the next fifteen years, according to Capgemini’s surveys of 600 executive level and senior management from Wall Street. Three-fourths of these managers stated in the study that they consider their trading activities to be “data-driven”. However, it does not necessarily mean that the firm strategies and day-to-day decision making is built on the basis of big data. “Data-driven” here is more referring to data collection and information integration. In fact, even firms like Morgan Stanley which actively seeking improvements by utilizing advanced technologies would suffer some bad times caused by automatic trading. Morgan Stanley’s fixed income trading revenue was $2.04 billion in the first half of 2012, down 60% from 2010, increasing pressure on Chairman and Chief Executive James Gorman to guide the company to steadier financial footing. “They’ve moved into electronic trading better than their peers,” said Robert Kapito, President of investment management house BlackRock, “However, the new trading strategies will not always secure profits and might even lead to potential added-in risks.” Highlighting computer trading is Morgan Stanley’s latest effort to bolster a business that analysts say it has repeatedly misjudged over the years, even as rivals such as Goldman Sachs Group Inc., Deutsche Bank and J.P. Morgan Chase earned billions of dollars in profits. Banks have long viewed electronic trading as a mixed blessing because it has hurt the fees, or “spreads,” that can be charged on each trade.

From my personal perspective, I totally believe the automatic trading will improve the capital markets. However, I do not agree with the opinion that traders will be replaced by computers. This is a lesson that Knight Capital learned last summer as it lost $440,000,000 when its automated trading software went nuts. At a rate of $10 million per minute, Knight Capital burned through almost its entire 2011 profits when its computers decided to buy and sell stocks in the millions. This leaves analysts wondering whether or not Knight Capital would still be a player in the trading game for very long.

This isn’t the first time a software glitch has caused a stock market meltdown. In 2010, a computer glitch caused the stock market to plunge nearly 1,000 points within an hour as automated systems from a number of different companies reacted off of one-another by selling massive amounts of stocks in what was perceived as an inevitable crash. That incident was quickly corrected and reversed, but it could have resulted in a lot of people losing their life savings, or worse.

Intuition is definitely human being’s hugest strength over machines. Computers have the ability to calculate which we probably cannot catch up with for millions of years of evolution. By taking advantage of this capacity, capital markets can be more efficient and diversified. But automated trading can never replace traditional traders because investment is not only about crunching numbers – it absolutely requires a certain type of intuition to crack the ration behind the numbers. I wish one day computers can share this type of intuition though, thus I am more than happy to let one write this little piece of blog for me. J

Revolution in media industry? --- Analysis of Netfix

House of Cards is an American political drama series developed and produced by Netflix. The number of its audience in a week surpasses any other single television show except for Oscars. According to New York Times journalist David Carr, because of the amount of time and money invested in collecting user data, House of Cards was actually a fairly safe bet to be a success.

What does it mean by collecting user data, every time people take an action while watching --- pause, fast-forward, rewind, abandon a show after watching for a few minutes, etc., they create what is known in the world of big data as an "event"--- a discrete action that could be logged, recorded and analyzed. And the data is the power, Netflix, the largest provider of commercial streaming video programming in United States, already stored hundreds of millions of such events. 

In 2012, research showed that Americans watched more movies legally delivered via the Internet than on physical formats like Blu-Ray discs or DVDs, and as we all know increasing new generation use more illegal method, so the audience group should be even larger. While the traditional method can only provide us audience rating which says only good or bad of one show, the new method will tell us what element in a show is attractive.

Using the NoSQL database Apache Cassandra, Matt Pfeil, co-founder and VP of Customer Solutions at big data software company DataStax, which worked with Netflix to implement Cassandra, explained that this is the first time that programming has been developed with the aid of big data algorithms.

But is this telling audience that technology knows all? Andrew Leonard, a Salon reporter considers some of the implications of TV in the era of Big Data:

“The interesting and potentially troubling question is how a reliance on Big Data might funnel craftsmanship in particular directions. What happens when directors approach the editing room armed with the knowledge that a certain subset of subscribers are opposed to jump cuts or get off on gruesome torture scenes or just want to see blow jobs. Is that all we’ll be offered? We’ve seen what happens when news publications specialize in just delivering online content that maximizes page views. It isn’t always the most edifying spectacle. Do we really want creative decisions about how a show looks and feels to be made according to an algorithm counting how many times we’ve bailed out of other shows?”

However, what would weaken Netflix and others is a 2011 report by McKinsey that depict a fundamental disadvantage, although the demand outpacing supply by 50 per cent in North America by 2018, people who are able to efficiently transition Big Data by digging deep and finding out the meaning behind the numbers would have the ability to tailor towards consumers in a particular perspective.

Cashing Out with the Big Data Trend – Analysis of “Hadoop” Business Model

The technology cycles of the 1990s and the following decade set the stage for today’s emerging paradigm shift in Information Technology (IT). Old modes of computing, communication and storage are being replaced with “virtual machines”, loosely assembled as “clouds”, which will soon communicate through “software-defined networks” to “distributed” storage arrays.

Big Data, which is also known as data analytics or data warehousing has been existed for years, and has been widely used by airlines, traders and retailers to mine data sets for actionable trends. But with the growth of social networks, cloud computing, digital surveillance video and digital photos, electronic data now takes many forms, and its volume is growing rapidly. Market researcher IDC estimates that electronic data in all its forms is growing 50% per year and appears to be accelerating. Clearly a 50% growth rate is unsustainable, so the opportunity emerging is to mine this expanding trove and distill insights from disparate sources to deliver both improved levels of service and entirely new applications.

One example that entitles such kind of technologies to obtain fast application is software platforms such as “Hadoop.” 

It was originally created to assist massive software apps search through huge chunk of data from different origins and various backgrounds. These software apps were firstly developed and tested by companies like Google and Yahoo and their initial function was to upgrade the search engines accuracy. However, due to the generalization of big data, these apps are heavily utilized in all walks of life. One example application is to collect data from social networks like Facebook and Weibo for retail industries like Macy’s and Sam’s Club, which will boost the related discount and coupon offers to specific customer groups.

For the time being, Hadoop is still open-source software designed to run on industry-standard, commodity servers running the Linux operating systems. As we all know, open-source is basically free and hard to capitalize from the products. In addition, it is important to unattach the link of Big Data. The opportunity to build a business around Hadoop is with the applications that run on a Hadoop framework, which may hopefully be similar to Apple’s Eco-System emperor. According to Tier1 Research-a D.C. based IT market research firm, the compound growth in the volume of data has rocketed to near 50%. That said, the industry needs a product like Hadoop to boost the market and hit a home run. Based on the analysts’ consensus, early inceptions are very welcomed and the industry buzz is relatively high, particularly among the venture capital community. To conclude, Hadoop, as an example in this blog for million potential businesses that might cash out by capturing the big data trend, clearly is a hidden gold to usher in a new era of large-scale software applications.