|
| 1 | +--- |
| 2 | +layout: default |
| 3 | +title: Retrieving Storm Cluster Statistics from Nimbus |
| 4 | +--- |
| 5 | + |
| 6 | +# Retrieving Storm Cluster Statistics from Nimbus |
| 7 | + |
| 8 | +This tutorial will show how to obtain basic storm topology statistics from nimbus (similar to the information show in the storm ui dashboard) so you can store that data in a separate ___location -- such as graphite -- for monitoring. Since storm topologies are thrift-based, you can use almost any language to connect to nimbus to download your data. |
| 9 | + |
| 10 | +<div |
| 11 | + markdown="1" |
| 12 | + class="tutorial" |
| 13 | + data-github-author="Whitespace" |
| 14 | + data-license="http://creativecommons.org/licenses/by/3.0/" |
| 15 | + data-facets='{"Operating System": "OS X 10.7", "Language": "Ruby", "Storm Version": "0.7.0", "Thrift Version": "0.7.0", "Package Management": "Homebrew"}'> |
| 16 | + |
| 17 | +## Assumptions |
| 18 | + |
| 19 | +We assume you'll store your data in `~/code`. We also assume that your homebrew is newer enough that the `/usr/local` directory is homebrew's git repo. |
| 20 | + |
| 21 | +## Setup |
| 22 | + |
| 23 | +Let's make a ___location to store our project: |
| 24 | + |
| 25 | +{% highlight sh %} |
| 26 | +mkdir -p ~/code/nimbus-thrift-demo |
| 27 | +{% endhighlight %} |
| 28 | + |
| 29 | +## Download the Storm Source Code |
| 30 | + |
| 31 | +In order to connect to nimbus, we need to get the `storm.thrift` file from the storm source, and use that to generate the thrift bindings in ruby: |
| 32 | + |
| 33 | +{% highlight sh %} |
| 34 | +cd ~/code |
| 35 | +wget https://github.com/downloads/nathanmarz/storm/storm-0.7.0.zip |
| 36 | +unzip storm-0.7.0.zip |
| 37 | +{% endhighlight %} |
| 38 | + |
| 39 | +## Install Thrift 0.7.0 and the Thrift Gem |
| 40 | + |
| 41 | +Now let's install the same version of thrift that storm 0.7.0 uses: |
| 42 | + |
| 43 | +{% highlight sh %} |
| 44 | +cd /usr/local |
| 45 | +brew update |
| 46 | +brew versions thrift |
| 47 | +git checkout 141ddb6 /usr/local/Library/Formula/thrift.rb |
| 48 | +brew install thrift |
| 49 | +{% endhighlight %} |
| 50 | + |
| 51 | +And the thrift gem: |
| 52 | + |
| 53 | +{% highlight sh %} |
| 54 | +gem install thrift |
| 55 | +{% endhighlight %} |
| 56 | + |
| 57 | +## Generate the Ruby Bindings |
| 58 | + |
| 59 | +{% highlight sh %} |
| 60 | +cd ~/code/nimbus-thrift-demo |
| 61 | +cp ~/code/storm-0.7.0/src/storm.thrift . |
| 62 | +thrift --gen rb storm.thrift |
| 63 | +{% endhighlight %} |
| 64 | + |
| 65 | +## Connect to Nimbus |
| 66 | + |
| 67 | +Now that we have the ruby bindings in place, we can create a sample app to connect to our nimbus cluster. Copy the following to `test.rb`, replacing `localhost` with the hostname of your nimbus instance: |
| 68 | + |
| 69 | +{% highlight ruby %} |
| 70 | +$:.push('gen-rb') |
| 71 | + |
| 72 | +require 'rubygems' |
| 73 | +require 'thrift' |
| 74 | +require 'nimbus' |
| 75 | + |
| 76 | +begin |
| 77 | + transport = Thrift::FramedTransport.new(Thrift::Socket.new('localhost', 6627)) |
| 78 | + protocol = Thrift::BinaryProtocol.new(transport) |
| 79 | + client = Nimbus::Client.new(protocol) |
| 80 | + |
| 81 | + transport.open |
| 82 | + summary = client.getClusterInfo |
| 83 | + puts summary.inspect |
| 84 | + transport.close |
| 85 | + |
| 86 | +rescue Thrift::Exception => tx |
| 87 | + print 'Thrift::Exception: ', tx.message, "\n" |
| 88 | +end |
| 89 | +{% endhighlight %} |
| 90 | + |
| 91 | +Now we can run our test app: |
| 92 | + |
| 93 | +{% highlight sh %} |
| 94 | +ruby test.rb |
| 95 | +{% endhighlight %} |
| 96 | + |
| 97 | +Now you can take that data and write it to where you'd like. Enjoy! |
| 98 | + |
| 99 | +</div> |
0 commit comments