I attended the annual data analytics and governance conference organized by Informatica, the global leaders in enterprise data governance and data quality, right before writing this. The keynote was as interesting as it was exciting.
The CEO of Informatica Mr. Amit Walia walked us through his vision for a future where every business is empowered by quality data and successful governance. He was followed by a detailed demonstration of an intelligent cloud designed specifically for data management by Jitesh Ghai, the chief product officer.
Amongst all these exuberant revelations, what I could not help noticing was how far we have come in terms of data handling and utility. From struggling to cope with digitization in the nineties, to digital modernization with new software and capabilities, to cloud based data analytics and machine learning, it has been quite a ride. Let us go on a recap-trip, then.
The evolution of storage
From the floppies to the cloud. Tech savvy meme makers find this evolution quite amusing, You may even have seen a little cartoon depicting this. Well, the issue around data storage has more layers than we care to remember.
The cost of storage was the tallest obstacle in the path of the democratization of data. Today, we say that we do not own technology, or science, or code, we just own data – that is the primary and most important currency in the world of technology today. Today, if you accumulated all the data centers around the world it would cover 600 football grounds.
The concepts of multi-cloud and hybrid-cloud have changed the storage scenario for good. More companies can now afford having large databases and data driven executions. Gut-feelings are being replaced by data driven decisions in all fields, and the storage condition is the first miracle you must thank.
Focus on governance and security
If data is the most valuable currency, it has to be governed, managed, and secured. When asked about her top priorities while conducting data migration, Tembi Sibanda, the director of Peloton mentions two things, test, and security. The importance of data security cannot be stressed enough. Security builds trust and trust results into efficiency and accuracy.
With government enforced regulations like GDPR and CCPA in place, cataloging data, tracking lineage, monitoring and managing metadata have become more important than ever. This has put a lot of emphasis on data governance, data architecture, and quality. There is a renewed enthusiasm in terms of hiring data architects among businesses trying to achieve a full scale digital transformation.
Machine assisted data analysis
An all human approach to analyze data in today’s world is set up for failure. The sheer amount and variety of data would not allow any human workforce to achieve viable analytical results in a stipulated time. The insufficient workforce does not help the situation.
Machines are trained to recognize patterns and machine driven analyses are in vogue. There are pitfalls but the gain supersedes the risk. We can notice that a partnership is growing between humans and machines in terms of looking into data and coming to conclusions. It is the crossover we all hoped to see.
Data science in the public sector
India has taken some serious leapes in terms of integrating data analytics and data science in various public sectors. This is one of the most important changes from India’s point of view.
Using real-time analytics to provide farmers with timely and accurate weather data; assisting them in choosing the right crop for the soil in order to maximize the yield, is one example of using analytics for real Indian problems. There are police departments in India that are using facial recognition to tag criminals. Banks are using computer predictive analytics to detect potential fraud.
These developments are fueled by the government’s enthusiasm as well as the great work done by data science training facilities in creating a pipeline of data science professionals. Any good data science institute in Bangalore attracts applicants from around the world.
The ever expanding scope
In the initial days of the data revolution it was pretty hard to convince most business owners about the benefits of having an analytics capability. Things have changed now. The industry has matured now with many successful use cases to show off. Startups and SMBs are benefitting as much as large corporations. With the accessibility of cloud and affordability of data storage, more and more businesses are stepping into the zone, creating more opportunities for the enthusiasts.
