What is the biggest bottleneck for the Internet of Things (IoT) to be successful?
Here’s a clue for you: It’s no different than the biggest bottleneck for “Big Data.” And no, it’s not the infrastructure challenges. Any other guesses?
What if I reframe the question? What is the biggest bottleneck in the process of converting raw data to actionable decisions? Before I give you the answer, consider the life cycle of data.
There are roughly 4 phases, called “the 4 A’s of data” from Jeffrey Stanton’s Introduction to Data Science:
While each phase has its challenges, the problems associated with Acquisition, Architecture (integrating & shaping) and Archival have been (or soon will be) solved by technology. That just leaves Analysis unsolved.
If it wasn’t obvious, the biggest bottleneck is “the human ability to make sense of data.” Our bandwidth for Analysis is limited and is not keeping pace with the explosion of data.
The Internet of Things will further exacerbate this problem. Too many sensors and devices generating way too much data for any human to meaningfully act upon. Solving this problem is a significant piece of the Digital Business puzzle.
There isn’t a silver bullet, but the solution lies somewhere between automating even more of the mundane and giving machines “intelligence” so they can make their own operational decisions. Sounds daunting but thanks to Microsoft Azure, Sparkling Logic and some secret Mariner sauce – we have a solution!
From Data to Decisions: The Smart Way
Our solution is a variation on a system described by Alan K Fish, in his book Knowledge Automation. The technologies used in our example include Microsoft Azure Machine Learning (Ml), Azure Data Lake (or any other appropriate data repository, and Sparkling Logic’s SMARTS decision management service.
We’ve applied our expertise in predictive maintenance (PdM), decision management, analytics, machine learning and cloud to create the conceptual architecture shown below.
- Data is generated by various devices and sensors.
- Azure IoT Suite is used to provide a secure connection and help with the management of devices and collection of data. Once the data is in the cloud it is stored in a repository. There’s an embarrassment of choices for a repository: SQL Server, Azure SQL Database, Azure SQL Data Warehouse, HDInsight, or you can put it into a “data lake” using Azure Data Lake. The choice depends upon many factors.
- Next we apply Azure Machine Learning algorithms to uncover patterns and gain predictive insight. The results of these algorithms enriches the data flowing through the complex event processor, Azure Stream Analytics. The data stream can also be enriched with additional data that can be used to improve the quality of decisions in the next step..
- Finally, the enriched data set is fed into Sparkling Logic’s SMARTS Decision Service which fires off decisions. SMARTS is an ideal tool to convert data into actionable intelligence by giving business analysts and process engineers the ability to easily design, develop and maintain critical “decisioning” logic.
In short, the architecture above streams sensor data into Microsoft’s Azure cloud environment, enriches that data stream, makes sense of it and churns out decisions – with no human involvement! This decision management represents a predictive cloud service that can be re-used for multiple use cases. (Want to see this in action? Check out a Case Study and Demo on Sparkling Logics’ follow-up blog post here.)
Using the pattern described above, or something similar, we can help your business become a more intelligent and scalable digital business. Let’s get you get started with our Quick Start Workshop.