I've been neglecting my open source projects this summer due to time constraints. At Mapbox we've been onboarding a new President and COO and a raft of new employees, plus a manager for my team at Mapbox. These are hugely positive developments, but have also been a big lift. Moving back to Colorado and bringing our lives here out of suspension has similarly taken all of my personal time. I've been the blocker for new releases of Fiona, Rasterio, and Shapely all summer long and have been feeling rather guilty about it.

Things are looking up now. My kids are back in school and have seen their doctor and dentist. Their schedule of soccer practices and other activities for the season is getting settled quickly. I'm resuming weekly yoga and gym workouts along with my existing running schedule. I like having a weekly routine; it helps me stay relaxed and gives me time for personal projects like writing and computering.

Open source continues to be a big part of my job and as my share of the onboarding lift eases I've been able to increase my time on Fiona, Rasterio, and Shapely. I released the long overdue Shapely 1.6.0 and have supported the GeoPandas team on getting 0.3.0 out. Fiona 1.7.9 was the first bug fix release of that project since June and I'm happy to have that in user hands. This week I'm working on coding and writing about the upcoming Rasterio release. I feel like I'm doing a good job as an open source maintainer and mentor again and am excited about what I'll be able to do in the next few months.

None of these projects would be viable without the help of other developers. To my open source collaborators: thanks for hanging in there and being patient with me!

Last Sunday I ran the first half of the Blue Sky Marathon route to refamiliarize myself with the trails, gauge my fitness, and try out some new gear.

Towers Trail

I don't yet feel completely acclimated to living at 5000 feet. The first mile of Towers Trail (cutting across the image above) at Horsetooth Mountain Park has an average grade of 12% and I had to fast walk much of it to keep my heart rate down.

Horsetooth Rock from Carey Springs Trail

At the top the trail is more gentle and rolling and I had no difficulties. The Tower Trail is a 4WD road, but 90% the race route is singletrack. Rose-colored granite rock and dirt on the heights and brick red sandstone rock and dirt below.

I enjoyed trading my large Camelback pack for a smaller and lighter vest. The marathon has 7 stations and I'm certain that I can get along with less than a liter of water between them. My new shorts with builtin boxer briefs are perfectly comfortable. I tried on a pair of Salomon Speedcross 4 shoes at the store to see what the hype was about but didn't buy them. I liked the fit, but I don't think I'm going to be running any local races that are steep enough or sloppy enough to warrant that kind of traction. They'd be great for French style trails, no doubt about it. My NB Leadville shoes are a good match for the relatively tame Blue Sky and Horsetooth trails.

The drive to the trailhead reminded me of some similarities between Fort Collins and Montpellier. Each city is between plain and mountains. Fort Collins is located where The Great Plain of North America meets the Rocky Mountains and Montpellier is between the Languedoc coastal plain and the Massif Central. Each city has its own local peak: Horsetooth Rock above in Fort Collins and the Pic Saint-Loup at Montpellier. I'd love to host folks from Montpellier here and ask if they have any of the same impressions.

On behalf of the Shapely project, I'm pleased to announce a new minor release.

Shapely 1.6.0 adds several new attributes to existing geometry classes and new split() and polylabel() functions to the shapely.ops module. Exceptions have been consolidated in a shapely.errors module and logging practices have been improved. Shapely's optional features depending on Numpy are now gathered into a requirements set named "vectorized" and these may be installed by running pip install shapely[vectorized].

Much of the work on 1.6.0 was aimed to improve the project's build and packaging scripts and to minimize runtime requirements. Shapely now vendorizes packaging to use during builds only and never attempts to invoke the geos-config utility during import of the module.

Another big change for the project is that the documentation and manual are now hosted at Read the Docs: https://shapely.readthedocs.io/en/latest/.

Thank you all for using, promoting, and contributing (48 of us now!) to the Shapely project. The full change log can be found here.

Share and enjoy.

We spent the first 4 nights of July at 3 refuges in the Alpes and had a stupendously good time. The weather was wet for the first two days, but cleared for good as we left the refuge named after Alfred Wills on the morning of the 3rd.

Climbing away from Refuge Alfred Wills

To the south of Alfred Wills is Lac d'Anterne. It's a beautiful lake bordered by beaches of schist and slate chips perfect for skipping across the water.

Lac d'Anterne and the "cloud city" Fiz

On the morning of the 4th we had amazing views of Mont Blanc from Col d'Anterne and saw a herd of ibex on the cliffs above the pass.

Pormenaz (foreground), Col du Brévent (middle), and Mont Blanc (background)

Mont Blanc

The network of refuges (or "huts" as we say in Colorado) allow hikers to go very light. You don't need any bedding other than a sleeping bag liner and don't need to bring a stove, cookware, or food (other than your favorite chocolate). Alfred Wills was our most comfortable stay and had the best food. We had couscous on the night of the 2nd and roast pork with polenta on the Fourth of July. Our takeaway lunches were couscous salads with bread, saucisson, and thick slices of Tomme de Savoie cheese. Alfred Wills is supplied by helicopter every other week (wines, beer, cheese, carrots, potatoes, and canned goods), but on our way out we met one of the refuge keepers (un guardien) hiking up from Sixt-Fer-à-Cheval with 15 kilograms of fresh baked bread. He said that during the peak season, later in July, they have to bake bread at the refuge to keep up with the demand.

This was our last vacation in France on this trip and was one of our best.

It's time to get serious about returning to my permanent home in Fort Collins, Colorado. We've got our tickets and have begun to pack some bags. Once again, me and the kids are flying on one itinerary and Ruth and our dog are flying on another. The latter is a bit dicey because the airlines may refuse to load our doggy in cargo if the forecasted arrival temperatures are over 80 degrees. We have picked flights to minimize the probability, but some luck would be nice.

A couple weeks ago I finally met the owner of the house that we've rented in Montpellier. A scientist, like Ruth, who has connections with other scientists we know. It's a small world, evolutionary biology, and full of interesting and earnest folks. I wouldn't be surprised if we keep in touch.

I've told people at Coswos, the coworking I've joined in Montpellier, and some of my favorite vendors and shopkeepers – basically anybody who has ever asked where I'm from and what my plans are – that we're leaving. We threw ourselves a party last Sunday night with friends from Ruth's lab and ate Franco-American sliders and drank champagne, cheaper bubbly from Limoux, and margaritas.

The next four weeks will be busy. We've planned a four day hut-to-hut trip in the Alpes. I'm hoping to do some other opportunistic trips to nearby cities. Why I haven't been able to make it to Toulouse or Lyon, I do not know, but I will try to do something about this. We've got a number of boxes of clothes and other personal effects (books and bandes dessinées, aka graphic novels, mostly) to ship. I must drink or give away the bottles of my wine collection that we can't bring back in our luggage. There are a ton of arts and music festivals in the region in July, too, it's really a bad time to be moving.

Friday, the 21st of July, I plan to be writing tests on the deck of my house in Fort Collins and then switching over to grilling burgers, drinking IPAs, watching kids play in the sprinklers, and feeling more than a little bit sentimental about my year in France.

On the weekend of the second round of the French presidential election I, Ruth, and our kids took a three night cruise with another family of friends on the Canal du Midi. It was much more interesting than I expected.

The Canal du Midi connects the city of Toulouse and the Garonne River (and thus the Atlantic Ocean) to the Étang de Thau and the Mediterranean Sea. It was constructed between 1666 and 1681 under the direction of Pierre-Paul Riquet. By a series of 91 locks the canal steps up from 130 meters above sea level in Toulouse to a 190 meter high divide at the Seuil de Naurouze before stepping down to sea level in Sète. There are 57 kilometers of canal on the Atlantic side and 189 kilometers of canal on the Mediterranean side. How to supply such a canal with water was a problem that stumped engineers for many years. Riquet and his team found an overlooked source in the montagne Noire and built a reservoir, the second largest in Europe, by damming the Laudot river, a tributary of the Tarn and the Garonne, about 20 kilometers from the summit at Seuil de Naurouze.

The canal has been expensive to maintain and was never much of a trade route, but has found a second life as a tourist attraction. Boat rental companies, restaurants, and boutiques are found in many of the villages traversed by the canal. I've made a dataset with 36 points of interest along our cruise and used it as a layer in the map below. We put in at Argens, in the Minervois region, and cruised past some of the largest expanses of grape vines in France. The vines on the left bank of the canal (generally north) are of Minervois, those of the right bank, Corbières.

Plane trees were planted along the canal in the early 1800s to stabilize the banks and reduce evaporation by shading the water. Today the trees are dying from a fungal (Ceratocystis platani) disease. VNP, the Voies Navigables de France, intends to cut down every diseased tree (and this may eventually be all of them) and replace them with resistant varieties of plane and other shade trees. We saw signs explaining the replantation project at every long treeless bank. Since 2006, a third of the canal's 42,000 plane trees have been cut down and removed.

Mapbox's imagery of Capestang, a village on the canal, is out of date. We (I work on the team that makes the satellite basemap) will update it, but until we do it affords a look at the way things used to be on the canal. Below you see that the canal in the village was bordered by huge shade trees. You can also see that some of them to the right of Capestang's newer metal bridge are completely dead.

We moored in a different very different Capestang from the one in the map. The trees that lined the banks of the canal have been removed, every one. Saplings have been planted between the stumps of the plane trees, but it will be many years before the canal is completely shaded again.

We were a party of 10: 4 adults, 5 kids, and a dog. Our large pénichette had 4 cabins and slept 12. It was a tight squeeze at a few bridges and windy conditions made the passages extra challenging. The canal is only 2 meters deep at the most, thus these recreational barges displace very little water and blow around easily. I failed at Capestang's narrow and angled old bridge on the second day and we had to get some help from a friendly local boater. Our boat had many patches around the edges, so apparently we weren't the only ones having difficulty at the bridges. If I were to do this again, I'd go for a shorter and less wide boat, especially if the group were smaller.

First night's mooring.

We were on the canal for 3 days and nights and covered a lot of distance. It's 94 kilometers from Argens to Colombiers and back and we probably did 5+ hours of boating on each of the second and third days. This meant passing by some villages and chateaux that would have been fun to visit. 2 hours of cruising per day and 3 hours of walking would be more my speed.

Doing a lot of cruising did let us see a lot of interesting places on and along the canal such as the pont-canals of Répudre and Cesse. The pont-canal over the Répudre river was the first of its kind in Europe. Traversing it was the first time I'd ever been on a bridge in a boat.

Tunnel de Malpas.

Another sight that you really have to see from on deck is the Tunnel de Malpas. 160 meters long, it was the first canal tunnel ever constructed. The Roman Via Domitia came over the hill at this same spot once upon a time and an SNCF tunnel for trains between Béziers and Narbonne passes underneath. I've now passed under this hill by both train (on my family's trip to Barcelona) and by boat.

Vineyards of Corbières.

Boat and crew.

On the return trip, I got off the boat at Malpas and ran the 12 kilometers to Capestang. The canal-side trails are well maintained and we saw many cyclists, some of them with loaded paniers.

Le Somail.

Le Somail is a cute little village with several nice restaurants, an ice cream shop, and a remarkable used book store, le Trouve Tout du Livre. The old stone bridge is typical and a tight squeeze. I had to duck to get under while piloting and we had only a foot or so of clearance on either side.

Waiting for a lock to open.

There's only one lock between the villages of Argens and Colombiers: the 2.4 meter high Argens lock. On the downstream side of it is the canal's longest pound. There's not another lock until Béziers. We went through this lock once on the way out and once on the way back. It took about 15 minutes to go through. I found the process captivating.

Boating is fun and piloting a barge was a new experience for me. There are dozens of new skills to learn and even though I didn't master any of them, it was fun to try. The scenery was great. The canal itself is a fascinating artifact. If you've ever wanted to do this kind of trip, do it in May while the weather is warm but not too hot, and do it before all the plane trees are cut down!

This morning Twitter suggested to me that I might be interested in this tweet and Twitter was right (for once)!

pic.twitter.com/kUaek9rJZ6

— Map/ (@vtcraghead) May 30, 2017

I was pretty sure that PEP 8 was the last word on block comments and told Bill so.

@vtcraghead It looks like you're doing block comments wrong. That's a string. Python uses hash for comments.

— Sean Gillies (@sgillies) May 30, 2017

Then while eating lunch I stumbled onto this tweet by a Python programmer with some cred.

@BSUCSClub Python tip: You can use multi-line strings as multi-line comments. Unless used as docstrings, they generate no code! :-)

— Guido van Rossum (@gvanrossum) September 10, 2011

I'm inclined to believe the author of Python, but I decided to check. It's been a long time since I dug into Python bytecode and I thought I could use a refresher. Here's a small program that contains a script in a triple-quoted string. It compiles the script to bytecode and then uses dis, the Python disassembler, to show the instructions in the script's bytecode. I've never done dis before!

import dis

source = '''
"""script.py:

A script."""

text = """
Some
multi-line
text.
"""

"""
Unused multi-line text
generates
no code.
"""

"""
But if you
use it...
""" + " code!"

print(text)
'''

code = compile(source, '<string>', 'exec')
print(dis.dis(code))

The first anonymous string is stored as the module's __doc__ (this goes for functions, classes, and methods as well). The unused anonymous string beginning with "Unused" does't make it into the bytecode, confirming Guido Van Rossum's tweet. The anonymous string beginning with "But" that is used in the string concatenation does make it into the bytecode.

$ python3.6 script.py
  5           0 LOAD_CONST               0 ('script.py:\n\nA script.\n')
              2 STORE_NAME               0 (__doc__)

 11           4 LOAD_CONST               1 ('\nSome\nmulti-line\ntext.\n')
              6 STORE_NAME               1 (text)

 22           8 LOAD_CONST               2 ('\nBut if you\nuse it...\n')
             10 LOAD_CONST               3 (' code!')
             12 BINARY_ADD
             14 POP_TOP

 24          16 LOAD_NAME                2 (print)
             18 LOAD_NAME                1 (text)
             20 CALL_FUNCTION            1
             22 POP_TOP
             24 LOAD_CONST               4 (None)
             26 RETURN_VALUE
None

I didn't know that unused anonymous strings were ignored like this. I assumed they hung around briefly until garbage collection got them, but Python optimizes them away. Note: you can even optimize the __doc__ away with python -OO.

There are limits to this kind of block comment. They can't be used within brackets, for one thing.

@gvanrossum @BSUCSClub
{
'''
This won't work
as we expect
'''
'foo' : 'bar',
}

{
'''
This will
throw Exception
'''
1: 2,
}

— Vinayak Kaniyarakkal (@vinayakcoder) November 27, 2015

Myself, I'm going to stick to # and Vim block-mode selection, but I'm going to stop telling people using triple-quoted strings as block comments that they're flat wrong.

I did another trail race today. The Festa Trail starts and finishes in Saint-Matthieu-de-Tréviers, a town in the Pic-Saint-Loup vineyard region north of Montpellier. It's a 3-day multi-race event. I ran the 18 km Tour de Pic Saint-Loup with 875 meters of elevation gain in 2:36:10 (unofficially).

Pic Saint-Loup on the left and the Château de Montferrand on the right

The French word pic is related to the English peak. According to Roger Brunet in Trésor du terroir: les noms de lieux de la France, pic does not, despite common characters, have the same origin as the French puy, Catalan puig, or Occitan puech, all of which derive from the Latin podium.

The Château de Montferrand seems to have its origins as a Roman fort and was mentioned for the first time in 1132 as a property of the Count of Toulouse (who I've mentioned before). It passed into the hands of the French crown and remainded a royal fort until Louis XIV authorised its demolition in the 17th century. We stormed the castle during km 4 of the run.

I felt good most of the way, made it through some mild calf cramps around km 15, and managed a bit of a kick at the end. There was one steep kilometer of going up (200 meters) to the ruined castle on the way out and one steep kilometer coming down from the peak on the way back (190 meters), but nothing so sustained as at the Trail Quillan. The trail was completely dry and very rocky (caillouteux, we say here), treacherously so in several spots. There's a layer of limestone that has splintered crazily and traversing it was like running on the threads of a giant screw.

Festa Trail is a fun race and I highly recommend it. With time running out on our séjour, it might have been my last trail run in France this year.

RFC 8142 is the second and final deliverable of the IETF's GeoJSON working group. It standardizes sequences of GeoJSON texts and and a media type you can use to tell receivers "here comes a sequence of GeoJSON Feature objects, not a GeoJSON FeatureCollection." This is useful because a GeoJSON feature collections must be read in its entirety before it can be parsed [1]. It's a blob. A text – not binary – blob, but a blob nonetheless. A FeatureCollection becomes unwieldy as the number of features increases. Dynamic feed-like streams of features (consider a stream of OSM edits or stream of features extracted in real time from imagery) also need a different kind of representation from a static array of Feature objects.

Newline-delimited sequences of GeoJSON objects are being employed by some projects, including a few at Mapbox. In a newline-delimited sequence the individual features must use a compact form. No pretty-printed features are permitted. If you're aggregating features produced by other services, you must parse them and reserialize them in compact form.

RFC 8142 describes a format for sequences of features that may be compact or pretty-printed. Mixed sequences are also possible. The trick is that every sequence item must begin with an ASCII Record Seperator (RS), 0x1E, and end with a newline. Two delimiters. The first allows formatted, pretty-printed texts within a sequence, the latter guards against truncated sequence records. That's it. There's not a lot to RFC 8142 other than this and the definition of a new internet media type to mark this kind of data stream.

Sprinkling RS in your file sort of turns it into a binary file. Python's open() function, for example, does not accept newline=u'\x1e' and can not provide you an iterator over RS-delimited records. You may have to write your own readLine() type of function to get individual items from the stream. It's not the end of the world, but does add some friction. Vladimir Agafonkin tells me that this is the way to do it in JavaScript:

var split = require('binary-split');

fs.createReadStream('data.foo')
.pipe(split('\x1E'))
.on('data', function (buf) {
    var geojson = JSON.parse(buf.toString());
});

There is already support for GeoJSON text sequences in programs that I use often like GNU Parallel, jq, and fio. In parallel's --pipe mode, the --recstart option will split records on RS and --rrs will remove the RS from the output.

parallel --pipe --rrs --recstart '\x1E' cat < data.jsonseq

The current version of jq, 1.5, will read and write RS-delimited sequences if you pass the --seq option.

jq --seq -c '.' data.jsonseq

Fiona's fio-cat will emit RS if you use its --rs option. This is required if you want pretty-printed features. Otherwise fio-cat writes compact GeoJSON delimited only by newlines. The complementary fio-collect and fio-load commands accept either newline-delimited sequences or GeoJSON text sequences.

Note that there's no recommended file extension for GeoJSON text sequences. The format is intended for network protocols and not for files. If you do save them to files it would be best not to use .json or .geojson as an extension because a delimited sequence of GeoJSON (RS or not) isn't valid JSON.

Note also that while the format technically allows mixed sequences containing GeoJSON FeatureCollection, Feature, and Geometry objects, the semantics of these kinds of mixed sequences is unlikely to be understood by consumers. Streams of features seems to me like the best application for this format right now.

Thanks to the following people and organizations: Eric Wilde and Martin Thomson, the WG chairs; Alissa Cooper, Area Director; the RFC Editor and IETF reviewers; Mark Baker, Sean Leonard, and Ned Freed for comments on the media type; WG participants Martin Daly, Stephan Drees, Kevin Wurster, Matthew Perry, ,Allan Doyle, Carl Reed, Jerry Sievert, Peter Vretanos, and Howard Butler; and Mapbox, my employer, for allowing me time to edit the doc.