Skip to content

Commit 4a5eb51

Browse files
author
Tom Clark
committed
Added python version
1 parent 3cf800f commit 4a5eb51

File tree

1 file changed

+104
-3
lines changed

1 file changed

+104
-3
lines changed

pages/retrieving-storm-data-from-nimbus.md

Lines changed: 104 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -74,13 +74,16 @@ require 'thrift'
7474
require 'nimbus'
7575

7676
begin
77-
transport = Thrift::FramedTransport.new(Thrift::Socket.new('localhost', 6627))
78-
protocol = Thrift::BinaryProtocol.new(transport)
79-
client = Nimbus::Client.new(protocol)
77+
socket = Thrift::Socket.new('localhost', 6627)
78+
transport = Thrift::FramedTransport.new(socket)
79+
protocol = Thrift::BinaryProtocol.new(transport)
80+
client = Nimbus::Client.new(protocol)
8081

8182
transport.open
83+
8284
summary = client.getClusterInfo
8385
puts summary.inspect
86+
8487
transport.close
8588

8689
rescue Thrift::Exception => tx
@@ -95,5 +98,103 @@ ruby test.rb
9598
{% endhighlight %}
9699

97100
Now you can take that data and write it to where you'd like. Enjoy!
101+
</div>
102+
103+
<div
104+
markdown="1"
105+
class="tutorial"
106+
data-github-author="Whitespace"
107+
data-license="http://creativecommons.org/licenses/by/3.0/"
108+
data-facets='{"Operating System": "OS X 10.7", "Language": "Python", "Storm Version": "0.7.0", "Thrift Version": "0.7.0", "Package Management": "PyPI"}'>
98109

110+
## Assumptions
111+
112+
We assume you'll store your data in `~/code`.
113+
114+
## Setup
115+
116+
Let's make a ___location to store our project:
117+
118+
{% highlight sh %}
119+
mkdir -p ~/code/nimbus-thrift-demo
120+
{% endhighlight %}
121+
122+
## Download the Storm Source Code
123+
124+
In order to connect to nimbus, we need to get the `storm.thrift` file from the storm source, and use that to generate the thrift bindings in python:
125+
126+
{% highlight sh %}
127+
cd ~/code
128+
wget https://github.com/downloads/nathanmarz/storm/storm-0.7.0.zip
129+
unzip storm-0.7.0.zip
130+
{% endhighlight %}
131+
132+
## Install Thrift 0.7.0 and the Thrift Python Package
133+
134+
Now let's install the same version of thrift that storm 0.7.0 uses:
135+
136+
{% highlight sh %}
137+
cd /usr/local
138+
brew update
139+
brew versions thrift
140+
git checkout 141ddb6 /usr/local/Library/Formula/thrift.rb
141+
brew install thrift
142+
{% endhighlight %}
143+
144+
And the thrift python package:
145+
146+
{% highlight sh %}
147+
pip install thrift==0.7.0
148+
{% endhighlight %}
149+
150+
## Generate the Python Bindings
151+
152+
{% highlight sh %}
153+
cd ~/code/nimbus-thrift-demo
154+
cp ~/code/storm-0.7.0/src/storm.thrift .
155+
thrift --gen py storm.thrift
156+
{% endhighlight %}
157+
158+
## Connect to Nimbus
159+
160+
Now that we have the python bindings in place, we can create a sample app to connect to our nimbus cluster. Copy the following to `test.py`, replacing `localhost` with the hostname of your nimbus instance:
161+
162+
{% highlight python %}
163+
import sys
164+
165+
sys.path.append('gen-py')
166+
167+
from thrift import Thrift
168+
from thrift.transport import TSocket
169+
from thrift.transport import TTransport
170+
from thrift.protocol import TBinaryProtocol
171+
172+
from storm import Nimbus
173+
from storm.ttypes import *
174+
from storm.constants import *
175+
176+
try:
177+
socket = TSocket.TSocket('localhost', 6627)
178+
transport = TTransport.TFramedTransport(socket)
179+
protocol = TBinaryProtocol.TBinaryProtocol(transport)
180+
client = Nimbus.Client(protocol)
181+
182+
transport.open()
183+
184+
summary = client.getClusterInfo()
185+
print summary
186+
187+
transport.close()
188+
189+
except Thrift.TException, tx:
190+
print "%s" % (tx.message)
191+
{% endhighlight %}
192+
193+
Now we can run our test app:
194+
195+
{% highlight sh %}
196+
python test.py
197+
{% endhighlight %}
198+
199+
Now you can take that data and write it to where you'd like. Enjoy!
99200
</div>

0 commit comments

Comments
 (0)