Merge pull request #86 from StratoDem/85-custom-polygon-queries

Update IntersectsFilter to support weighted aggregations

Author: mjclawar

CHANGELOG.md

@@ -2,6 +2,15 @@
 
 All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](http://semver.org/).
 
+## [3.4.0] - 2019-11-06
+### Adds
+- Adds new `detailed_type` option to `IntersectsFilter`. It now supports:
+    - `'intersects'` (simple intersection filtering)
+    - `'intersects_weighted'` (ability to create weighted polygon results)
+
+## Related issues
+- [85](https://github.com/StratoDem/strato-query/issues/85)
+
 ## [3.3.0] - 2019-09-18
 ### Adds
 - Adds new buffer types to the `DrivetimeFilter`. It now supports:

docs/index.html

@@ -258,6 +258,9 @@
                   <li>
                     <a href="#geocoder-queries" class="toc-h2 toc-link" data-title="Geocoder queries">Geocoder queries</a>
                   </li>
+                  <li>
+                    <a href="#custom-polygon-queries" class="toc-h2 toc-link" data-title="Custom polygon queries">Custom polygon queries</a>
+                  </li>
                   <li>
                     <a href="#advanced-queries" class="toc-h2 toc-link" data-title="Advanced queries">Advanced queries</a>
                   </li>
@@ -854,6 +857,66 @@ <h2 id='geocoder-queries'>Geocoder queries</h2>
 <td>Boston, MA</td>
 </tr>
 </tbody></table>
+<h2 id='custom-polygon-queries'>Custom polygon queries</h2>
+<blockquote>
+<p>This query returns the population contained within the custom polygon by year</p>
+</blockquote>
+<pre class="highlight python tab-python"><code><span class="kn">from</span> <span class="nn">strato_query</span> <span class="kn">import</span> <span class="n">SDAPIQuery</span><span class="p">,</span> <span class="n">APIQueryParams</span>
+<span class="kn">from</span> <span class="nn">strato_query.filters</span> <span class="kn">import</span> <span class="n">IntersectsFilter</span><span class="p">,</span> <span class="n">BetweenFilter</span>
+<span class="kn">from</span> <span class="nn">strato_query.aggregations</span> <span class="kn">import</span> <span class="n">SumAggregation</span>
+
+<span class="n">df</span> <span class="o">=</span> <span class="n">SDAPIQuery</span><span class="o">.</span><span class="n">query_api_df</span><span class="p">(</span>
+    <span class="n">APIQueryParams</span><span class="p">(</span>
+        <span class="n">table</span><span class="o">=</span><span class="s">'populationforecast_tract_annual_population'</span><span class="p">,</span>
+        <span class="n">data_filters</span><span class="o">=</span><span class="p">(</span>
+            <span class="n">IntersectsFilter</span><span class="p">(</span>
+                <span class="n">detailed_type</span><span class="o">=</span><span class="s">'intersects_weighted'</span><span class="p">,</span>
+                <span class="n">var</span><span class="o">=</span><span class="s">'geoid11'</span><span class="p">,</span>
+                <span class="n">val</span><span class="o">=</span><span class="p">{</span><span class="s">"type"</span><span class="p">:</span> <span class="s">"Polygon"</span><span class="p">,</span>
+                     <span class="s">"coordinates"</span><span class="p">:</span> <span class="p">[</span>
+                         <span class="p">[[</span><span class="o">-</span><span class="mf">71.17801666259767</span><span class="p">,</span> <span class="mf">42.43321295705304</span><span class="p">],</span>
+                          <span class="p">[</span><span class="o">-</span><span class="mf">71.0145950317383</span><span class="p">,</span> <span class="mf">42.43321295705304</span><span class="p">],</span>
+                          <span class="p">[</span><span class="o">-</span><span class="mf">71.0145950317383</span><span class="p">,</span> <span class="mf">42.3145122534915</span><span class="p">],</span>
+                          <span class="p">[</span><span class="o">-</span><span class="mf">71.17801666259767</span><span class="p">,</span> <span class="mf">42.3145122534915</span><span class="p">],</span>
+                          <span class="p">[</span><span class="o">-</span><span class="mf">71.17801666259767</span><span class="p">,</span> <span class="mf">42.43321295705304</span><span class="p">]]]})</span><span class="o">.</span><span class="n">to_dict</span><span class="p">(),</span>
+            <span class="n">BetweenFilter</span><span class="p">(</span><span class="n">var</span><span class="o">=</span><span class="s">'year'</span><span class="p">,</span> <span class="n">val</span><span class="o">=</span><span class="p">[</span><span class="mi">2010</span><span class="p">,</span> <span class="mi">2019</span><span class="p">])</span><span class="o">.</span><span class="n">to_dict</span><span class="p">()),</span>
+        <span class="n">data_fields</span><span class="o">=</span><span class="p">(</span><span class="s">'year'</span><span class="p">,</span> <span class="s">'population'</span><span class="p">),</span>
+        <span class="n">groupby</span><span class="o">=</span><span class="p">(</span><span class="s">'year'</span><span class="p">,),</span>
+        <span class="n">order</span><span class="o">=</span><span class="p">(</span><span class="s">'year'</span><span class="p">,),</span>
+        <span class="n">aggregations</span><span class="o">=</span><span class="p">(</span><span class="n">SumAggregation</span><span class="p">(</span><span class="s">'population'</span><span class="p">),),</span>
+    <span class="p">))</span>
+</code></pre><pre class="highlight r tab-r"><code><span class="c1"># Not implemented yet
+</span></code></pre><pre class="highlight shell tab-shell"><code><span class="gp">$ </span>curl -X POST <span class="s2">"https://api.stratodem.com/api"</span> <span class="se">\</span>
+    -H <span class="s2">"accept: application/json"</span> <span class="se">\</span>
+    -H <span class="s2">"Content-Type: application/json"</span> <span class="se">\</span>
+    -d <span class="s2">"{ </span><span class="se">\"</span><span class="s2">token</span><span class="se">\"</span><span class="s2">: </span><span class="se">\"</span><span class="s2">my-api-token</span><span class="se">\"</span><span class="s2">, </span><span class="se">\"</span><span class="s2">query</span><span class="se">\"</span><span class="s2">: {</span><span class="se">\"</span><span class="s2">query_type</span><span class="se">\"</span><span class="s2">: </span><span class="se">\"</span><span class="s2">COUNT</span><span class="se">\"</span><span class="s2">, </span><span class="se">\"</span><span class="s2">data_fields</span><span class="se">\"</span><span class="s2">: [</span><span class="se">\"</span><span class="s2">year</span><span class="se">\"</span><span class="s2">, </span><span class="se">\"</span><span class="s2">population</span><span class="se">\"</span><span class="s2">], </span><span class="se">\"</span><span class="s2">table</span><span class="se">\"</span><span class="s2">: </span><span class="se">\"</span><span class="s2">populationforecast_tract_annual_population</span><span class="se">\"</span><span class="s2">, </span><span class="se">\"</span><span class="s2">groupby</span><span class="se">\"</span><span class="s2">: [</span><span class="se">\"</span><span class="s2">year</span><span class="se">\"</span><span class="s2">], </span><span class="se">\"</span><span class="s2">data_filters</span><span class="se">\"</span><span class="s2">: [{</span><span class="se">\"</span><span class="s2">filter_type</span><span class="se">\"</span><span class="s2">: </span><span class="se">\"</span><span class="s2">intersects_weighted</span><span class="se">\"</span><span class="s2">, </span><span class="se">\"</span><span class="s2">filter_value</span><span class="se">\"</span><span class="s2">: {</span><span class="se">\"</span><span class="s2">type</span><span class="se">\"</span><span class="s2">: </span><span class="se">\"</span><span class="s2">Polygon</span><span class="se">\"</span><span class="s2">, </span><span class="se">\"</span><span class="s2">coordinates</span><span class="se">\"</span><span class="s2">: [[[-71.17801666259767, 42.43321295705304], [-71.0145950317383, 42.43321295705304], [-71.0145950317383, 42.3145122534915], [-71.17801666259767, 42.3145122534915], [-71.17801666259767, 42.43321295705304]]]}, </span><span class="se">\"</span><span class="s2">filter_variable</span><span class="se">\"</span><span class="s2">: </span><span class="se">\"</span><span class="s2">geoid11</span><span class="se">\"</span><span class="s2">}, {</span><span class="se">\"</span><span class="s2">filter_type</span><span class="se">\"</span><span class="s2">: </span><span class="se">\"</span><span class="s2">between</span><span class="se">\"</span><span class="s2">, </span><span class="se">\"</span><span class="s2">filter_value</span><span class="se">\"</span><span class="s2">: [2010, 2019], </span><span class="se">\"</span><span class="s2">filter_variable</span><span class="se">\"</span><span class="s2">: </span><span class="se">\"</span><span class="s2">year</span><span class="se">\"</span><span class="s2">}], </span><span class="se">\"</span><span class="s2">aggregations</span><span class="se">\"</span><span class="s2">: [{</span><span class="se">\"</span><span class="s2">aggregation_func</span><span class="se">\"</span><span class="s2">: </span><span class="se">\"</span><span class="s2">sum</span><span class="se">\"</span><span class="s2">, </span><span class="se">\"</span><span class="s2">variable_name</span><span class="se">\"</span><span class="s2">: </span><span class="se">\"</span><span class="s2">population</span><span class="se">\"</span><span class="s2">}], </span><span class="se">\"</span><span class="s2">order</span><span class="se">\"</span><span class="s2">: [</span><span class="se">\"</span><span class="s2">year</span><span class="se">\"</span><span class="s2">]}}"</span>
+</code></pre><pre class="highlight vb tab-vb"><code><span class="c1">' Not implemented yet</span>
+</code></pre>
+<p>To use a provided GeoJSON <code>Polygon</code> as a filter, use the <code>IntersectsFilter</code>.</p>
+
+<p>The example to the right returns the population by year for the area covered by the provided <code>Polygon</code></p>
+
+<p><code>{&quot;type&quot;: &quot;Polygon&quot;, &quot;coordinates&quot;: [[[-71.1326, 42.2981],[-70.9943, 42.2981], [-70.9943, 42.3859],[-71.1326, 42.3859], [-71.1326, 42.2981]]]}</code></p>
+
+<table><thead>
+<tr>
+<th>YEAR</th>
+<th>SUM_POPULATION</th>
+</tr>
+</thead><tbody>
+<tr>
+<td>2010</td>
+<td>852183</td>
+</tr>
+<tr>
+<td>2011</td>
+<td>862990</td>
+</tr>
+<tr>
+<td>....</td>
+<td>......</td>
+</tr>
+</tbody></table>
 <h2 id='advanced-queries'>Advanced queries</h2>
 <blockquote>
 <p>This query returns GDP per capita estimates for counties</p>
@@ -1115,6 +1178,25 @@ <h2 id='intersectsfilter'><code>IntersectsFilter</code></h2>
           <span class="p">]]</span>
       <span class="p">}</span>
     <span class="p">})</span>
+
+<span class="c"># Use this version to get weighted results</span>
+<span class="n">IntersectsFilter</span><span class="p">(</span>
+    <span class="n">var</span><span class="o">=</span><span class="s">'geometry'</span><span class="p">,</span> 
+    <span class="n">detailed_type</span><span class="o">=</span><span class="s">'intersects_weighted'</span><span class="p">,</span>
+    <span class="n">val</span><span class="o">=</span><span class="p">{</span>
+      <span class="s">"type"</span><span class="p">:</span> <span class="s">"Feature"</span><span class="p">,</span>
+      <span class="s">"properties"</span><span class="p">:</span> <span class="p">{},</span>
+      <span class="s">"geometry"</span><span class="p">:</span> <span class="p">{</span>
+        <span class="s">"type"</span><span class="p">:</span> <span class="s">"Polygon"</span><span class="p">,</span>
+        <span class="s">"coordinates"</span><span class="p">:</span> <span class="p">[[</span>
+            <span class="p">[</span><span class="o">-</span><span class="mf">71.13269805908203</span><span class="p">,</span> <span class="mf">42.298135272741206</span><span class="p">],</span>
+            <span class="p">[</span><span class="o">-</span><span class="mf">70.99433898925781</span><span class="p">,</span> <span class="mf">42.298135272741206</span><span class="p">],</span>
+            <span class="p">[</span><span class="o">-</span><span class="mf">70.99433898925781</span><span class="p">,</span> <span class="mf">42.385937107381125</span><span class="p">],</span>
+            <span class="p">[</span><span class="o">-</span><span class="mf">71.13269805908203</span><span class="p">,</span> <span class="mf">42.385937107381125</span><span class="p">],</span>
+            <span class="p">[</span><span class="o">-</span><span class="mf">71.13269805908203</span><span class="p">,</span> <span class="mf">42.298135272741206</span><span class="p">]</span>
+          <span class="p">]]</span>
+      <span class="p">}</span>
+    <span class="p">})</span>
 </code></pre><pre class="highlight r tab-r"><code><span class="c1"># Not implemented yet!
 </span></code></pre><pre class="highlight shell tab-shell"><code><span class="o">{</span><span class="s2">"filter_type"</span>: <span class="s2">"intersects"</span>, <span class="s2">"filter_variable"</span>: <span class="s2">"geometry"</span>, <span class="s2">"filter_value"</span>: <span class="o">{</span><span class="s2">"type"</span>: <span class="s2">"Feature"</span>, <span class="s2">"properties"</span>: <span class="o">{}</span>, <span class="s2">"geometry"</span>: <span class="o">{</span><span class="s2">"type"</span>: <span class="s2">"Polygon"</span>, <span class="s2">"coordinates"</span>: <span class="o">[[[</span>-71.13269805908203, 42.298135272741206], <span class="o">[</span>-70.99433898925781, 42.298135272741206], <span class="o">[</span>-70.99433898925781, 42.385937107381125], <span class="o">[</span>-71.13269805908203, 42.385937107381125], <span class="o">[</span>-71.13269805908203, 42.298135272741206]]]<span class="o">}}</span>
 </code></pre><pre class="highlight vb tab-vb"><code><span class="c1">' Not implemented yet!</span>

docs/source/includes/_filters.md

@@ -348,6 +348,25 @@ IntersectsFilter(
           ]]
       }
     })
+
+# Use this version to get weighted results
+IntersectsFilter(
+    var='geometry', 
+    detailed_type='intersects_weighted',
+    val={
+      "type": "Feature",
+      "properties": {},
+      "geometry": {
+        "type": "Polygon",
+        "coordinates": [[
+            [-71.13269805908203, 42.298135272741206],
+            [-70.99433898925781, 42.298135272741206],
+            [-70.99433898925781, 42.385937107381125],
+            [-71.13269805908203, 42.385937107381125],
+            [-71.13269805908203, 42.298135272741206]
+          ]]
+      }
+    })
 ```
 
 ```r

docs/source/includes/_query-parameters.md

@@ -490,6 +490,64 @@ The second example returns the metro ID for the latitude-longitude pair:
 |----|----|
 |14454|Boston, MA|
 
+## Custom polygon queries
+
+> This query returns the population contained within the custom polygon by year
+
+```python
+from strato_query import SDAPIQuery, APIQueryParams
+from strato_query.filters import IntersectsFilter, BetweenFilter
+from strato_query.aggregations import SumAggregation
+
+df = SDAPIQuery.query_api_df(
+    APIQueryParams(
+        table='populationforecast_tract_annual_population',
+        data_filters=(
+            IntersectsFilter(
+                detailed_type='intersects_weighted',
+                var='geoid11',
+                val={"type": "Polygon",
+                     "coordinates": [
+                         [[-71.17801666259767, 42.43321295705304],
+                          [-71.0145950317383, 42.43321295705304],
+                          [-71.0145950317383, 42.3145122534915],
+                          [-71.17801666259767, 42.3145122534915],
+                          [-71.17801666259767, 42.43321295705304]]]}).to_dict(),
+            BetweenFilter(var='year', val=[2010, 2019]).to_dict()),
+        data_fields=('year', 'population'),
+        groupby=('year',),
+        order=('year',),
+        aggregations=(SumAggregation('population'),),
+    ))
+```
+
+```r
+# Not implemented yet
+```
+
+```shell
+$ curl -X POST "https://api.stratodem.com/api" \
+    -H "accept: application/json" \
+    -H "Content-Type: application/json" \
+    -d "{ \"token\": \"my-api-token\", \"query\": {\"query_type\": \"COUNT\", \"data_fields\": [\"year\", \"population\"], \"table\": \"populationforecast_tract_annual_population\", \"groupby\": [\"year\"], \"data_filters\": [{\"filter_type\": \"intersects_weighted\", \"filter_value\": {\"type\": \"Polygon\", \"coordinates\": [[[-71.17801666259767, 42.43321295705304], [-71.0145950317383, 42.43321295705304], [-71.0145950317383, 42.3145122534915], [-71.17801666259767, 42.3145122534915], [-71.17801666259767, 42.43321295705304]]]}, \"filter_variable\": \"geoid11\"}, {\"filter_type\": \"between\", \"filter_value\": [2010, 2019], \"filter_variable\": \"year\"}], \"aggregations\": [{\"aggregation_func\": \"sum\", \"variable_name\": \"population\"}], \"order\": [\"year\"]}}"
+```
+
+```vb
+' Not implemented yet
+```
+
+To use a provided GeoJSON `Polygon` as a filter, use the `IntersectsFilter`.
+
+The example to the right returns the population by year for the area covered by the provided `Polygon`
+
+`{"type": "Polygon", "coordinates": [[[-71.1326, 42.2981],[-70.9943, 42.2981], [-70.9943, 42.3859],[-71.1326, 42.3859], [-71.1326, 42.2981]]]}`
+
+|YEAR|SUM_POPULATION|
+|-------|----|
+|2010|852183|
+|2011|862990|
+|....|......|
+
 ## Advanced queries
 
 > This query returns GDP per capita estimates for counties

requirements.txt

@@ -1,5 +1,5 @@
 # Core requirements
-Cython==0.29.13
+Cython==0.29.14
 numpy==1.17.3
-pandas==0.25.2
+pandas==0.25.3
 requests

setup.py

@@ -13,7 +13,7 @@
 
 setup(
     name='strato_query',
-    version='3.3.0',
+    version='3.4.0',
     author='Michael Clawar, Raaid Arshad, Eric Linden',
     author_email='tech@stratodem.com',
     packages=[