• News
  • Columns
  • Interviews
  • BW Communities
  • Events
  • BW TV
  • Subscribe to Print
BW Businessworld

‘Forecasting’s Passé, Nowcasting’s Here’

Photo Credit :

Recently India’s top IT CEOs got together in Bangalore for a two-hour discussion on what Big Data means for India and whether it could be the panacea to a slowing IT services industry. The high level panel included Mohan Jayaraman , MD of Experian Credit Information Company of India Pvt Ltd, Anirban Dey, MD, SAP Labs India, Sandeep Mathur, MD of Oracle India, Arun Kumar, GM of Red Hat india, Rajesh Janey , President of EMC India, Vijaya Kumar Ivaturi, co-founder of Crayon Data and Kaushik Bhaskar, Director, Information Management Software, India Software Lab, IBM. The panelists also debated how the use of Big Data could change citizen services. While some cautioned about the fact that India was yet to start using Big Data, yet others stressed on why Indian companies need to use data and data crunching technologies for their businesses to benefit. Nasscom predicts that the Indian Big Data market will be worth $1 billion --  up five times --  by 2015 while the global market will be $25 billion in the same period.

Pari Natarajan, CEO of Zinnov, was the moderator and he raised a pertinent question about the hype on Big Data and asked if the company representatives in the panel have had meaningful commercial engagements with their clients on the subject. 

Pari Natarajan: Is Big Data a hype or a reality?

Kaushik Bhaskar: It was in the hype curve a few years ago. It is in the mind of every CIO today. 95 per cent of the CIOs have to adopt it in a couple of years or be left behind. At IBM we have done more than 30,000 engagements globally. In the analytics space it is not about tying in a variety of data, it is about insights. I have seen insurance, telecom and banking take up to Big Data. It is a reality today globally.
Mohan Jayaraman: Experian is a data and analytics company. We have an operation in India where the RBI has given us the mandate to run a credit bureau. We have four lines of business; credit bureaus, consumer, analytics and marketing services. Here the commonality is data and our clients hold large amounts of data, which we help them to understand and use meaningfully. In the hype curve of Big Data, there is a good part and a bad part. As long as we can deliver value with technology and analytics we can be assured of the good part. But there is a huge amount that is not reality. A lot of work is still under discussion and a lot of work is in the early stage of deployment.  There is no clear commercialisation yet. We sit on so much data and there is so much more we can do with it, but nothing is being done as yet. The data from the social front can be combined with credit information; this can add value to business and consumers. But we are not there yet.

Sandeep  Mathur: Big Data is a reality. It was not just born out of somewhere because petroleum companies and the government have always been using large volumes of data. The falling cost of storage has given rise to this opportunity. Now companies want to do more with data, they do not want to dump data. It is very exciting when you combine trends like social and business analytics together. India has been very disappointing for us and companies are not innovating enough. CIOs do not want to implement big data and India is mostly watching what is happening in the west. We have several case studies abroad where Airbus has been using all its sensor data to do predictive maintenance. Similarly PayBack, a couponing company in Germany has been using Big Data to make real time offers to consumers. They have 3,50,000 transactions a day and they know what offers the consumers can make use of. Oracle is having conversations around the opportunity and consumers want personalised experience and businesses need to get to what the consumers want. India is a mobile driven market, big data will grow here in time. Any marketing manager will tell you that their marketing has to be very personalised and should focus on consumers. However, in India we are not yet there. Our marketing and loyalty campaign success rates are abysmally low and you see the consumer benchmarking his every other experience with the online world. This is true globally and will soon be common in India, we have seen the growth of Flipkart and how mobiles are changing the way consumers behave.

Rajesh Janey: Over the last two years  a lot of data has been generated. Data was only being stored so far, but now it is about harnessing the data for actionable insights. Converting data into knowledge is important and since storage costs are falling, the consumption of data will grow. This is what Big Data is. Can we use the lower cost of storage, better compute and open source technologies at affordable cost to turn this myth into reality? This is already happening and in India we have large Big Data play. There are three clear use cases for Big Data. The first one is in citizen services, the diversity and spread, the learnings from India cannot be matched by any country. There is clear case for its impact on society in the context of mapping delivery of healthcare and education. Third is around the IT services opportunity, can we get ahead of the curve with data analytics and big data technologies. The question is can we be in the forefront of the Big Data revolution?

Arun Kumar:
The Cloud and Big Data are both possible because they are driven out of commodity architecture. Open platforms have bought costs down and that is the reason you see a boom in entrepreneurship the world over. The small companies do not use legacy architecture to deliver their services. The velocity of innovation is led by small companies. The cost of implementing on open source platforms is so low and small companies piggy back such technology. They have created IP around these open platforms, look at Hadoop, the open source platform that came out of Yahoo and Google helped many companies go and build their services on Hadoop.These small companies that have cloud based business models have billion dollar valuations. Red Hat’s customers are taking open source platforms. Like ‘Mohan’ mentioned; can you take data from a structured format (credit scores) and match it with unstructured social data. I think customers want to know if they can integrate these technologies to understand their consumers. Today it depends on the need of the companies and what they want out of data. If you do not have competition and pressure on bottom line you will not innovate. This continues to be the problem of Big Data implementation in India. Indian companies can no longer ignore the impact of data and will have to look in to it very clearly. Companies must realize that a legacy data base does not support unstructured data.

Anirban Dey: We must cut through the glamour of words and understand what really is changing the software industry and the underlying stacks or the software that powers these stacks. Two clear changes are happening. One is that the atoms are turning into bits. Whatever was mechanical is becoming software? See the mobile phone, it has got rid of the watch, the alarm clock, the music player and what not. The second is that it is empowering the end user. Do we really sit in front of the computer, nowadays we are only with the mobile? Everything should be served to me in a matter of seconds otherwise the game is lost. The wave of software that is taking over is enormous and the crunching behind the software is immaterial to the consumer. He needs it in seconds. It is not about Big Data and it is more about Dark Data. What are companies doing with data? The stack needs to create meaning out of it. Open source and proprietary software is driving innovation. Opportunities are limitless.

Vijaya Kumar Ivaturi:
In the beginning of quantum mechanics, Einstein was once asked “what you say challenges common sense?” His reply was that you have common sense at low speeds, this is the situation of Big Data and it is running at a low scale today. Many conventional methods fail when it comes to Big Data. It is true infrastructure is available. But it only solves computational capabilities and this is available as a service. The second element is mostly the algorithmic complexity which is what you do on top of the infrastructure, which still exists and is part of the cloud system. The third is the visualisation piece which gives a semantic view of the data. Our company, Crayon Data, looks at the problem from a slightly different angle. If you like sea food, does it mean you like a sea facing room in a hotel? If you buy organic milk, will you buy organic vegetables?  There is something that connects this heterogeneous behavior and this is what Crayon uses to make the choice simple in a B2B scenario. You can cross sell or upsell to the consumer. In a wealth management scenario, we can personalise products for customers through social affinity and collaborative filtering. We use graph theory and a platform to execute these complex questions that go beyond structured data. Our work encompasses 40 per cent behavioural science and 60 per cent capital science. You must remember data is more effective with behavioural science. This makes things complex and it is also important that layers of technology are available to use to make sense of this data. So for us Big Data is not a computational problem anymore.

Pari Natarajan:
Let us talk about use cases and how it really impacts clients in terms of bottom lines? Also do let us know what industries use Big Data. Is it FMCG, oil & gas, retail, sports, financial services or government?

Sandeep Mathur: A lot of the use cases cannot be referenced at this summit. I will give you hints. What we reckon is that structured and unstructured data should be combined together for insights. There were no Big Data tools a few years ago and people employed data scientists at expensive cost. One of the projects we as Oracle are working on is for network utilisation with a telecom player. Say there is a concert in a particular area. How do I get information as a consumer about the concert and can the telecom company increase the mobile ‘bands’ within the area to allow social discussions and push offers. Imagine there will be a lot of trending that happens on social data when it comes to a popular concert.  The telecom company by the process of increasing mobile bands in an area increases its digital revenue. In the banking side we work on offers, can I push offers of stores around my area when I use a credit card. The offers are tailor made for me. In customer loyalty we are working with a white goods manufacturer. We help them look at the service data of different appliances with people’s social comments; later we can match the two data sets and take it to the engineers to improve the product. The company can then better their product and warranty programmes. Now look at it this way, Big Data is in every sector. These tools are at the customer’s end and you do not need expensive data scientists. Now there is also a success based criteria as a business model which is evolving. There is an ISV on the telecom side, they do telecom marketing and they assure you a success rate. They take the money out of the campaign that they haveand will take a share of the success.

Mohan Jayaraman: Why do you want to use Big Data technologies? The size of data is becoming large. I was at a conference in Europe and when I said the bank that I was previously working for had 25 million customers they were zapped. This is a reality in India, we sit on so much data. The UIDAI project is large too, you have 600 million customers. This data has to sit in India and can only be made sense of by using Big Data technologies. Now that storage and technologies are cheaper this is easy to do so. The second part is what we need to get out of this data that is crunched. Take for example banks; they need not use Big Data technologies, right now, because they have already set up data warehouses which are 30TB in size. The question is the insight that they can get out of this. You need to align yourself with business objectives if the data has to make sense for you. Information is available and one of the things Experian wanted to do was; if a credit card is swiped and there are 100,000 merchants that you have tied up with, the swipe will tell you the area and the number of merchants. The customer can then be targeted with a personalised message by bringing down the latency. Here you need a business objective to do the data matching of the customers and the merchants in the area. Many people are looking at tech and analytics stack; this is not how you look at data.

You need to align the business objectives that need to be solved and then use analytics. If the businesses ask this question, then they will change the innovation problem. You need to be faster in the data world. One of the big complexities is data matching of customers. In the USA you can use 10 rules and you can triangulate information. In India it is never easy even after eighty rules because the same customer has different addresses and different telephone numbers. Big Data technologies allow you to pinpoint such complexities faster today and analytics should deliver information and convert information at speed. In the telco space, say if there is a mobile prepaid customer whose charge is running out, would you, as the operator, like to cut the call or use analytics to be insightful about his loyalty to the brand. If he is a regular customer then you can give him a credit without him having to cut the call because of the fall in balance. The next time he tops up the mobile you can cut the extra credit given to him previously. This is a tremendous use case. It uses large volumes of data and is predictive analysis.

Rajesh Janey: Given the darkness of the data, data it is like oil. Telecom, finance and citizen services have so much opportunity. In citizen services there socio economic census and targeted subsidy. We are also involved in class census, how do we map it with education and healthcare. Can a telecom provider tell which ring back tone the customer is interested in? In the finance industry can we analyze five million trades that they happen in real-time. ATM analytics is an area of opportunity; say if Citibank’s customers are drawing a lot of cash from a particular bank’s ATM, in a specific location, then we can tell Citibank to set up an ATM in that location because they will have to pay unnecessary services charges to the vendor bank. 

Anirban Dey: There are a couple of examples around fleet management. There are lots of car rentals, taxi companies around the world. They use data intelligently.They use real time positioning from vans and vehicles to connect where they have to be and for the specific purpose. This company in Japan that we work with is a spring water bottling company. They use geo-positioning analytics to figure out where these vans have to reach and at what time. This company delivers water to retailers and makes sure that there is no stock out. Previously they were doing this in a 24 hour window, now they do it in real time.Then there is an oil exploration company that we work with that does off-shore drilling in seabeds. During drilling, the drills should be monitored carefully. We use the vibrations from the boring to understand or predict of the drill is going to break. This is a huge empowerment to all operators.

The value comes from real time streaming of information and predicting failure. In India Adhaar is a very big opportunity. We are working on medical records in Uttarkhand, where every school child’s medical and vaccination records are tracked through an Adhaar ID. We work with milk cooperatives where we track payments when milk is delivered by the farmer. Previously payments were made in a week to the farmer and this was valuable money for him. The payment cycle depended on the quality of the milk and the quantity he had delivered. This process took a lot of time. But by using Adhaar we can credit the farmer immediately because we have all his details and we know what he has been supplying to the cooperative. We were able to shrink the one week payment cycle to real-time.

Sports is a big engagement, there is pre-game and post-game prediction. There is fan based engagement that makes analytics even more engaging. We worked with Kolkatta Knight Riders (the cricket team) on the history of each game of the IPL for the last six years. We created 25 data points per ball to analyse player performance. We can now tell if he is a consistent batsman or bowler. From there on we created an auctioning software where we can bid for the right players. The analytics throws in if the players are good and matches their consistency with other players.  It also shows you who are the other players that are similar and how effective they are for the team to buy. KKR used this live at the auction. The total pool they had for the auction was Rs 60 crore and they saved Rs 2.5 crore.

Arun Kumar: In FMCG we have zillions of consumer data. Can this data be matched to social data and to geo-positioning to do real time analytics to influence traders and consumers. Open source architecture allows companies to make such engagements fasters. Sports businesses have huge customer engagement analytics, look at the predictions along with the fan based social engagementfor NBA. These basketball teams are working on data, not only on the plays, but are also making sure data is used to make consumers be part of the team.

Kaushik Bhaskar: (On a lighter note) One of my nieces comes to me and says to me that she wants to buy a laptop. I give her some advice, but she went after information from so many sources such as friends, reviews and ratings to buy the product before validating my opinion. Companies cannot ignore the way the consumer is behaving. IBM works with a bank where there are silos of information on clients and they are not integrated. They cannot differentiate between a high-value and a low-value client. IBM with their analytics helped them create 150 per cent growth in high value clients and 2 per cent growth in gross profit. We worked with a media company to run predictive models on all the social buzz around the movie Ram Leela. Before the launch we did a sentiment analysis and said that this movie will have a 73 per cent success provided it is launched in the right cities that would accept the movie. We have done it for other movies such as Barfi and Ek Tha Tiger. We primarily use social data to predict if these movies will be successful or not.

Vijaya Kumar Ivaturi: We did some work on music with the Hindi film industry to understand what kind of music should be released before the launch of the movie to create a buzz around the movie to make it successful. Music success is a leading indicator of success of a movie . It is interesting to use data because most of those in the film industry are preset on what sells and what kind of tracks they have to pick from a bank of options. But correlation is not causation. Similarly we worked with some of the IPL teams to do fan engagement where the fan predicts what kind of stroke will be played or what kind of ball will be bowled. When you link this to merchandise sales, this makes a lot of sense for business, there is a business case here. But in India we are a little bit early to introduce such business models.

In classical data analytics the model drives the data. But in Big Data, the data should drive the model because I do not know what the model is. It moves from forecasting to nowcasting and this makes the job complex. For example in retail, connecting the brick and mortar personality to online behaviour is not trivial because we have to figure out if it is the same user or not. This is a very difficult task. In the USA there are studies going on to see if the brick and mortar business trails the online business or the other way around. There is a personality question that you need to solve, in the online world are you the person that you are or are you projecting what you want to be.

Top themes and market attention on: