2019 (old posts, page 6)

Busy

Ruth is out of town this week and the next and on top of solo parenting I am dog sitting. I'll be extra busy through the end of June. Please forgive me if I don't respond immediately to emails or tickets on GitHub.

Arthur's Rock

Arthur's Rock is one of our local landmarks and hiking destinations. It's a big lump of pink Precambrian granite named after Arthur Howard, one of the previous owners of the land that would become Lory State Park in 1967. The Quad Rock and Black Squirrel courses go by it. I've been back twice since Quad Rock, once on a run and once with my family.

https://live.staticflickr.com/65535/48021888767_3473b561ed_b.jpg

View south from Arthur's Rock Trail, May 26, 2019

May was cool and wet and this weather has continued to some degree in June. Lory State Park is as green as I've ever seen it.

https://live.staticflickr.com/65535/48068992632_24cdfafe9f_b.jpg

Arthur's Rock from Howard Trail, June 1, 2019

The ponderosa pines in the second photo were charred during a small lightning-sparked fire in June 2016. I'm glad it wasn't worse. The small forest on the east slope of Lory State Park is priceless.

JavaScript mapping in 2018

It's been five-and-a-half years since I last blogged about making maps in a browser using JavaScript. It's cool that my old Syriac Places demo still works. Yay for web standards. A lot of things have changed in the meanwhile and this is my first chance to acknowledge some of those changes.

I've been writing HTML and JavaScript every other day at work for the past 3 weeks, building a web app to give Sales and sales-supporting teams at Mapbox insight into our imagery basemap. It's HTML and mostly Vanilla JS and has been a fun exercise. Since it's for internal customers only I can use new browser and JavaScript features.

Mapbox GL JS, the JavaScript mapping library that's maintained by my coworkers, is the closest thing to a framework in my app. My current project is the first in which I've used Mapbox GL JS. I'd like to point out some features and aspects of the framework that I find especially notable.

Mapbox GL JS has excellent examples. This is a result of I cribbed freely from https://docs.mapbox.com/mapbox-gl-js/examples/ and everything worked. Not all projects have good examples. John Firebaugh (who has since moved on from Mapbox), Colleen McGinnis, and the GL JS and Documentation teams put a ton of work into these and it shows.

Mapbox GL JS makes a big and important case of dynamic maps simple. I have a GeoJSON Source defined in my app's JavaScript. It is registered with a layer and a map. When I remove features from the source, they disappear from the map. When I add features to the source, they appear on the map. The map is updated fast enough that I can do this on every mousemove event.

Mapbox GL JS Expressions give a developer ample control over styling and filtering while keeping the styling API abstract and simple. In several parts of its API, Mapbox GL JS can evaluate Lisp-like expressions composed of JavaScript arrays. ["+", 4, ["*", 3, 2]] evaluates to 4 + (3 * 2), or 10. In addition to the standard JavaScript operators, the namespace of GL JS expressions operators contains feature property and map attribute getters. An advantages of using JavaScript objects instead of conventional Lisp (as with snuggs and rio-calc) is that developers get syntax highlighting help when writing expressions, and can use JavaScript to build expressions.

Another first for me is that I'm using Turf, the well-known simple features geometry library created by Morgan Herlocker. In my app I'm creating features on the fly derived from the intersections of features in a Mapbox tileset. Turf is fast and its functions take and return GeoJSON objects, which makes it easy for me to use and easy to integrate with Mapbox GL JS and other libraries or plugins. Turf is indispensable for a web app like the one I'm writing.

Lastly, JavaScript itself has changed a lot in the last 5 years. I'm using features of ES6 for the first time ever. You know this, I'll bet, but ES6 arrow functions are like Scheme's lambda expressions and Python's lambda forms. I'm using them exclusively, instead of the old anonymous functions, when filtering and sorting arrays of features. For example, the following sorts features by a date, in descending date order, without needing to type "function" or "return". Handy, and to my eyes, more readable.

features.sort((a, b) => b.properties.render_date - a.properties.render_date)

These are obviously not the only recent advances in the JavaScript mapping field. Or even the most recent, which is why I chose the title "JavaScript mapping in 2018". The app that I'm building could have been written a year ago using the same software. For a glimpse at JavaScript mapping in 2019-2020, I suggest https://observablehq.com/collection/@observablehq/maps.

Still snowing

The season's 19th snowfall is heavier than the 18th. We're now 24 days after the average last snowfall date 🤷.

https://live.staticflickr.com/65535/47900806181_ec73942f64_b.jpg

May 21, 2019

Resting

I haven't run or hiked at all this week. Not a single step.

https://live.staticflickr.com/65535/46947095815_acaa4c9c79_b.jpg

My strained calf is feeling much better after four days of rest and I'm looking forward to getting back out on the trail for an hour this weekend. My plan is to resume running around 25-30 miles a week, starting next week, and then start building up for races in September and October.

Rasterio 1.0.23

I've been busy with a new product at work the past few weeks, writing Javascript and HTML, and hiring new teammates, but found some space this week to knock out some Rasterio bugs and release version 1.0.23. Here's the relevant section of the change log: https://github.com/mapbox/rasterio/blob/master/CHANGES.txt#L4-L20.

I added some control over rio-calc's memory usage yesterday and had fun doing it. In 1.0.22, rio-calc read all your sources into memory, preventing calculations on very large datasets. There are a number of ways to chunk the calculations and output of the results. GDAL's gdal_calc.py script divides the job into chunks that match the storage of the input files. I rewrote rio-calc to chunk the work more like GDAL's warper does: you specify a maximum amount of memory to be used by the program and rio-calc divides the job into chunks that need approximately less than this amount. You, the user, are free to trade memory for speed.

The rio-calc command uses Lisp-like expressions. Some people like them, some people don't. Evaluation of the expressions is done with snuggs. The expression evaluator in gdal_calc.py uses Python's eval(). Snuggs does not use eval() and uses a whitelist of names and keywords. It is much safer. For example, you can't trick snuggs into importing modules that give filesystem access. You could use snuggs on your server to evaluate expressions written by untrusted people on the internet. You would be asking for trouble if you did this with gdal_calc.py. This is not to say that Snuggs is perfect and has no bugs. We found some this week and fixed them. If you're a rio-calc user, make sure you upgrade snuggs to 1.4.6 to get the latest and greatest version.

Thank you for using Rasterio and thank you for the ever-excellent bug reports. Clear writing and examples of failing code make it much easier for a busy programmer to switch context and dispatch bugs efficiently.

Training week twenty-four and Quad Rock recap

This final week's numbers:

  • 15 hours, 8 minutes running and hiking (2nd of 24)
  • 62.8 miles (6th)
  • 11,270 feet D+ (2nd)

Add to that one Quad Rock 50 finish in 13:13:50. My first ultramarathon.

https://live.staticflickr.com/65535/47785140532_7716eaee3c_b.jpg

Ruth says I looked fresher at the finish than I did in 2015 when I finished my first road marathon. It's partly because this race was more fun and partly because of my training. I'm sore today, but not wrecked.

David Bitner ran the first half of this with me and is a 25 mile finisher. He's building toward the Superior 100 in September and this was an early season training run. All the photos below are from his phone.

https://live.staticflickr.com/65535/47782790042_2d6e0c9ba5_b.jpg

Heading up South Ridge Trail, the second of six climbs

https://live.staticflickr.com/65535/47045334154_6d88aacb1e_b.jpg

On Westridge Trail, the top of the third climb

https://live.staticflickr.com/65535/47045348654_60222d0d68_b.jpg

At the finish, just before I went up for another 25 miles

I had so much fun hanging out with Bitner before, during, and after the race. I'm looking forward to the next chance.

Despite the snow that fell 2 days before, the trails were in great shape. The sky was blue for the first half, which made staying in a good mood easy; and cloudy in the second half, so I was never at risk of overheating. For once, it seemed like all factors were in my favor.

Not that there wasn't adversity. At the bottom of the fifth climb I had reached my longest single-day distance ever and I was going into uncharted territory. I had strained my left calf, high up behind my knee, and it hindered me for the rest of the run. My body had reached its VFuel and almond butter & honey sandwich consumption limit. I compensated by hiking more slowly uphill and letting go of the brakes on the downhills, flipping my tactics. I switched to boiled potatoes, Haribo Sour Cubes, and gingerale. I was beginning to suffer a little chaffing as I approached the 40-mile aid station, but had had the foresight to stash my favorite shorts from last year, semi-retired, in a drop bag there. Those old shorts saved my butt.

Too make a long story short: I had a great adventure and am super pleased with myself for finishing. Congratulations to all you other finishers. You who didn't finish have my sympathy. Big thanks to Nick, Brad, Gnar Runners, and all the volunteers who planned and operated a fine event!

T minus fifteen hours

It's a little over 15 hours until the Quad Rock 50 mile race starts. David Bitner is going to be showing up here soon. I can't wait to see his reaction when he sees that there is snow on the course. The weather dried out today but it hasn't been very warm. There may still be some snow on the trail tomorrow morning.

https://live.staticflickr.com/65535/47767955832_3052eb38f5_b.jpg

The Quad Rock course goes to the top of the foothills on the right, multiple times.

GDAL 3.0.0

GDAL 3.0.0, the final fruit of the GDAL Coordinate System Barn Raising project, is out today. Mapbox, my employer, was a financial supporter of the work and I was the shepherd of that support. I'm very pleased at the results and grateful to Howard, Paul, Kristian, and Even for taking the risk. The value Mapbox gets for a small investment is enormous. Are you an engineer at a company that should be supporting GDAL like this and don't know where to start? I'm happy to share my experience.

Here is Even Rouault's release announcement: https://lists.osgeo.org/pipermail/gdal-dev/2019-May/050202.html. Please take note: 3.0.0 has some big and breaking changes. It is not supported by the latest stable releases of Fiona and Rasterio. If you have installed either of those libraries from source distributions, not wheels, and you upgrade your system GDAL to 3.0.0, your Python programs could be broken. If you've installed Fiona and Rasterio wheels, which include their own copy of GDAL 2.x, you're safe.