I have been looking at this problem over a few years now<\/div>\n
The IoT industry often speaks of handling both high volumes and high throughputs of data<\/div>\n
However, currently, I find that there are not many use cases for IoT streaming analytics which are unique<\/strong><\/em><\/div>\n
The ‘unique’ and ‘currently’ bits are important i.e. identifying use cases that need the analytics primarily implemented in the stream (and not applied to data when at rest)<\/div>\n
Do you know of companies\/ vendors actually implementing this? (in production today)<\/div>\n
On one hard, every spark\/hadoop vendor could claim so<\/div>\n
But are customers actually deploying these models in stream?<\/div>\n
Just because you are spark or hadoop – your use case may not involve streaming analytics<\/div>\n
There is a cost involved<\/div>\n
and also there needs to be a unique use case to provide benefits<\/div>\n
Ex\u00a0 one could argue that fraud detection needs streaming analytics for large financial institutions<\/div>\n
One example is: HPE Vertica and Kafka<\/a><\/div>\n
Vertica<\/span>\u00a0provides a high-performance mechanism for streaming data both to and from third party message buses.\u00a0<\/span>Because Vertica can both receive and send data to a streaming message bus, you can use it as part of an automated analytics workflow. Vertica can retrieve data from the message bus, perform analytics on it, and then send the results back to the message bus for consumption by other applications.<\/span><\/i><\/div>\n
\u00a0<\/div>\n
So, the object of analytics is the ‘message bus’ i.e. data in motion<\/div>\n
So, my qs is<\/div>\n
a)\u00a0 Do you see other such use cases for IoT? (in production)<\/div>\n
b)\u00a0 Using which platform \/ techniques<\/div>\n
c)\u00a0 i am not convinced that all spark use cases are actually streaming – but happy to explore<\/div>\n
d)\u00a0 confluent\/kafka<\/a> also be an option (outside of the Vertica example)<\/div>\n
e)\u00a0 as also databricks for streaming iot applications<\/a> (as opposed to spark itself)<\/div>\n
f)\u00a0 also newer examples like timescaledb<\/a><\/div>\n
curious to see if anyone is implementing these techniques\u00a0in scale and in production<\/strong><\/div>\n