PubNub Spout for Storm

What is PubNub?

PubNub is a real-time messaging platform which uses publisher / subscriber pattern. You can find out more about PubNub by visiting www.pubnub.com

What is Storm?

Storm is a free and open source distributed realtime computation system. Storm can be used in scenarios like: realtime analytics, online machine learning, continuous computation, distributed RPC, ETL, and more. To learn more about Storm project, visit Storm Project website

PubNub Spout for Storm

Once you get familiar with Storm, you would know that Storm uses a concept called Spouts. A Spout is a source of streams in a Storm topology.   Basically Spout is the one who will read tuples from the external source and emit them in to the Storm topology. Please refer to Storm Wiki to read more about the concept of Spout.

In this post it will show how to use PubNub as the real-time external data source for a Storm topology, by making a PubNub Spout.

The Design

Basically what we do is create a Storm Spout, which is subscribed to a PubNub channel. Once,  some data is been published to that channel, Spout will receive those data and it will emit them in to the Storm topology.

The Code

If you find this useful, Please leave a comment below.

  • Super Deluxe! Now you can subscribe to a topic “channel” and consume the PubNub Real-Time Network as a spout “read input” for Storm Processing goodness.

  • scalabl3

    I was thinking if making a spout, super cool. You can use this also to create data storage of messages in a scalable and easier managed way.