Sean Gillies (Posts about benchmark)https://sgillies.net/tags/benchmark.atom2023-12-31T01:26:21ZSean GilliesNikolaIs rasterio fast enough?https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html2013-12-13T00:00:00-07:002013-12-13T00:00:00-07:00Sean Gillies<p>I've been working on a Python package named
<a class="reference external" href="https://github.com/sgillies/rasterio">rasterio</a> and have
<a class="reference external" href="http://sgillies.net/blog/2013/11/24/introducing-rasterio.html">blogged</a>
about my goals: more Python idioms, no gotchas, and speed. I'm doing well on
the first two, but now let's look at the last of these. Is rasterio fast
enough?</p>
<p>Here is a raster band read to Numpy ndarray benchmark with a comparison to
osgeo.gdal. The RGB.byte.tif file is a small (about 800 x 800 pixel) GeoTIFF
file. I'm going to open the file, read the entirety of the first band to a
ndarray, and then close the file. The gdal Dataset class has no explicit
closing method, so I'm assigning the variable to None and indirectly freeing
up the GeoTIFF file. In rasterio, you should call an opened file's close()
method when done, or use a <cite>with</cite> statement.</p>
<div class="code"><pre class="code python"><a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-1" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-1" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-1"></a><span class="kn">import</span> <span class="nn">timeit</span>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-2" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-2" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-2"></a>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-3" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-3" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-3"></a><span class="kn">import</span> <span class="nn">rasterio</span>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-4" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-4" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-4"></a><span class="kn">from</span> <span class="nn">osgeo</span> <span class="kn">import</span> <span class="n">gdal</span>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-5" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-5" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-5"></a>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-6" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-6" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-6"></a><span class="n">n</span> <span class="o">=</span> <span class="mi">100</span>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-7" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-7" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-7"></a>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-8" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-8" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-8"></a><span class="c1"># GDAL read as array</span>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-9" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-9" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-9"></a><span class="n">s</span> <span class="o">=</span> <span class="s2">"""</span>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-10" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-10" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-10"></a><span class="s2">src = gdal.Open('rasterio/tests/data/RGB.byte.tif')</span>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-11" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-11" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-11"></a><span class="s2">arr = src.GetRasterBand(1).ReadAsArray()</span>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-12" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-12" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-12"></a><span class="s2">src = None</span>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-13" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-13" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-13"></a><span class="s2">"""</span>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-14" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-14" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-14"></a>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-15" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-15" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-15"></a><span class="n">t</span> <span class="o">=</span> <span class="n">timeit</span><span class="o">.</span><span class="n">timeit</span><span class="p">(</span><span class="n">s</span><span class="p">,</span> <span class="n">setup</span><span class="o">=</span><span class="s1">'from osgeo import gdal'</span><span class="p">,</span> <span class="n">number</span><span class="o">=</span><span class="n">n</span><span class="p">)</span>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-16" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-16" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-16"></a><span class="nb">print</span><span class="p">(</span><span class="s2">"GDAL:"</span><span class="p">)</span>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-17" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-17" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-17"></a><span class="nb">print</span><span class="p">(</span><span class="s2">"</span><span class="si">%f</span><span class="s2"> usec</span><span class="se">\n</span><span class="s2">"</span> <span class="o">%</span> <span class="p">(</span><span class="n">t</span><span class="o">/</span><span class="n">n</span><span class="p">))</span>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-18" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-18" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-18"></a>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-19" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-19" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-19"></a><span class="c1"># Rasterio read band</span>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-20" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-20" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-20"></a><span class="n">s</span> <span class="o">=</span> <span class="s2">"""</span>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-21" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-21" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-21"></a><span class="s2">with rasterio.open('rasterio/tests/data/RGB.byte.tif') as src:</span>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-22" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-22" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-22"></a><span class="s2"> arr = src.read_band(1)</span>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-23" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-23" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-23"></a><span class="s2">"""</span>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-24" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-24" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-24"></a>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-25" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-25" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-25"></a><span class="n">t</span> <span class="o">=</span> <span class="n">timeit</span><span class="o">.</span><span class="n">timeit</span><span class="p">(</span><span class="n">s</span><span class="p">,</span> <span class="n">setup</span><span class="o">=</span><span class="s1">'import rasterio'</span><span class="p">,</span> <span class="n">number</span><span class="o">=</span><span class="n">n</span><span class="p">)</span>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-26" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-26" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-26"></a><span class="nb">print</span><span class="p">(</span><span class="s2">"Rasterio:"</span><span class="p">)</span>
<a id="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-27" name="rest_code_6dc6758e81944e1b880c61cd8d8a3b98-27" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_6dc6758e81944e1b880c61cd8d8a3b98-27"></a><span class="nb">print</span><span class="p">(</span><span class="s2">"</span><span class="si">%f</span><span class="s2"> usec</span><span class="se">\n</span><span class="s2">"</span> <span class="o">%</span> <span class="p">(</span><span class="n">t</span><span class="o">/</span><span class="n">n</span><span class="p">))</span>
</pre></div>
<p>Here are the results on my 1.7 GHz Intel Core i7 MacBook Air:</p>
<div class="code"><pre class="code console"><a id="rest_code_1a65092c94a14ef48126ac30163658e5-1" name="rest_code_1a65092c94a14ef48126ac30163658e5-1" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_1a65092c94a14ef48126ac30163658e5-1"></a><span class="gp">$ </span>python benchmarks/ndarray.py
<a id="rest_code_1a65092c94a14ef48126ac30163658e5-2" name="rest_code_1a65092c94a14ef48126ac30163658e5-2" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_1a65092c94a14ef48126ac30163658e5-2"></a><span class="go">GDAL:</span>
<a id="rest_code_1a65092c94a14ef48126ac30163658e5-3" name="rest_code_1a65092c94a14ef48126ac30163658e5-3" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_1a65092c94a14ef48126ac30163658e5-3"></a><span class="go">2.356141 ms</span>
<a id="rest_code_1a65092c94a14ef48126ac30163658e5-4" name="rest_code_1a65092c94a14ef48126ac30163658e5-4" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_1a65092c94a14ef48126ac30163658e5-4"></a>
<a id="rest_code_1a65092c94a14ef48126ac30163658e5-5" name="rest_code_1a65092c94a14ef48126ac30163658e5-5" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_1a65092c94a14ef48126ac30163658e5-5"></a><span class="go">Rasterio:</span>
<a id="rest_code_1a65092c94a14ef48126ac30163658e5-6" name="rest_code_1a65092c94a14ef48126ac30163658e5-6" href="https://sgillies.net/2013/12/13/is-rasterio-fast-enough.html#rest_code_1a65092c94a14ef48126ac30163658e5-6"></a><span class="go">2.655728 ms</span>
</pre></div>
<p>I'm pleased that Rasterio is only about 15% slower for this benchmark. And
the difference between the two will diminish as files grow larger. Rasterio
does considerably more validation of input and logs quite a bit. Also, rasterio
could be a little more lazy about getting and formatting a file's coordinate
referencing system. Less code and fewer bugs is the main goal of rasterio, so
getting close to gdal.py performance-wise is icing on the cake.</p>