Python typing mulligan

This is why I've been hesitant to add type hints to Fiona, Rasterio, and Shapely. David Lord on missteps and misgivings:

I want a "start over" tool for type annotating a Python library. I started with Flask as untyped code, then added annotations until mypy stopped complaining. But this didn't mean the annotations were _correct_. Over time I've fixed various reported issues. I feel like if I could start from scratch again, I'd probably get closer to correct with the experience I've gained. But removing all existing annotations and ignores is too time consuming on its own. #python

Let's fucking go

I saw a physical therapist yesterday. I had a virtual visit with my physician today. I had a 2 mile hike in the sun around a local reservoir. Now I'm listening to the Glenn Branca Orchestra on The Frow Show and my take on my health is: let's go!

The expert consensus is that I did not injure my spine, but that muscles in the left side of my hip have clamped down on a nerve. I'm going to proceed as if that is true, foam rolling, walking, and running through the pain, and not worrying about my spine cracking in pieces. I do have a little bit of numbness in my upper left leg and so I will not directly dive into long technical downhill runs. I expect that I'll resolve that soon.

A new CLI for GDAL

Even Rouault has proposed a new, modern, and more coherent command line interface (CLI) for the GDAL/OGR project. I think it's a good idea and a good time to do it. I've wanted a better one for about 15 years. Even credits Rasterio for inspiration, and that's gratifying to see. I started writing Rasterio 10 years ago in part because I wanted a better CLI for GDAL.

What I wanted in a GDAL CLI were the following features:

  • A root command and a few subcommands, one namespace for everything.

  • Uniform arguments and options with predictable ordering and naming.

  • Good documentation of arguments and options.

  • More subcommands with fewer options each. Making gdal_translate into 3-4 commands, for example.

  • Input and output that favor stdin/stdout and JSON.

  • Ease of installation. For example, with pip instead of an OS package manager.

I estimated in 2013-2014 that it was not feasible for me to achieve those goals within the GDAL project itself. GDAL and its community had no funding for this kind of work at the time. I found the GDAL project's tests somewhat inscrutable and frustrating. A hefty legacy of documentation and folk wisdom about the old ways would have to be updated. Mostly by me, certainly. And the GDAL user community largely did not care. Free software that was fast and effective (and, most of all, free!) was already more than most people had dreamed of. A GIS analyst had so many business and organizational problems to deal with already that the rough edges of gdalinfo and gdal_translate didn't even crack her top 20. Software polish wasn't a big concern in the second decade of FOSS4G. I think it's still a hard thing to sell. Individual consumers will pay money for slick, well-designed software that makes them feel good. Organizations value polish less. And neither GDAL nor Rasterio sell anything to individual consumers.

Overhauling gdal_translate, ogr2ogr, and friends within the GDAL project looked like a non-starter to me. Pushing a herd of boulders up a hill, by myself, for free, for a community that was largely content with working around and stepping over these boulders. I think I made the right choice for myself. I got to start from scratch, move fast, and use a modern CLI framework. I made a command line interface for Rasterio that, while not perfect, met most of my goals. And I didn't go broke or burn out while doing it.

Today, thanks to years of fundraising work by Howard Butler, Paul Ramsey, Kristian Evers, and Even, the GDAL project does have funding to overhaul its command line interface as an aspect of overall project health and maintenance. Multiple developers can be paid to work on it. They won't have to donate their time to it as I would have. Rasterio's command line interface can't be adopted by GDAL, or be forked to become GDAL's because it doesn't have all the features of existing GDAL programs (or even of gdal_translate and ogr2ogr for that matter), and my decision to have more subcommands with fewer options is kind of against the grain of GDAL. But the new GDAL CLI can adopt the demonstrably useful features and design of Rasterio's. JSON output, for example, is something that GDAL has already picked up from Rasterio.

Rasterio will certainly fade a little if the new GDAL CLI is designed and executed well. But that's in the nature of software and software communities. Rasterio has always depended on GDAL and benefited from being built on a technically solid and well loved foundation. And I didn't invent CLI subcommands and JSON output, not at all. It's not unfair. If you succeed in open source, if you move the needle, you will be emulated. In this case, I think we can call it progress. I'm content.

In the long run, I stand to get half of what I originally wanted from a GDAL CLI, the first three of the six features I listed above. And there's probably still room for a suite of Unix style programs with different opinions and design decisions, especially if it and GDAL agree on basic concepts, arguments, options, and flags.

Your account is deactivated

I stopped using Twitter a while ago. Then it was rebranded as "X". I dithered about deleting my account. Would it be taken over by crypto scammers or an 18 year-old Russian army private or contractor? I've come to the conclusion that I don't care anymore. They can have it if they want. I've downloaded my archive and clicked "deactivate your account".

If you see a post from an @sgillies on X, it isn't me.

On my feet

Saturday, I reported that I was laid up with a back injury. I'm feeling better today. The pain is manageable and I've been able to spend about half of my time awake upright, standing or sitting on my bed, with regular breaks to touch my toes and do the yoga half lift to stretch my back. I even managed to run a little bit yesterday while helping a guy who was the victim of a hit-and-run incident. Car on car, to be specific. No one was injured, as far as I could tell.

Ban cars. Ban injuries.

It's CyberGIS all over again

In a Mastodon thread about "GeoAI" today, I blurted out:

Uh huh. It's CyberGIS all over again.

We don't talk about CyberGIS anymore. I think it's going to be the same for "GeoAI".

Use pytest's tmp_path fixture

If you're not already using pytest's tmp_path fixture, you really should. The fixture provides a temporary directory for testing use. A directory that you can't reference from other test runs or from other test functions in the same run, but that isn't immediately deleted when your tests finish. The directory is created in your account temporary location and is eventually cleaned up by your computer's operating system. Until then, you can open the directories and their files in other applications.

I've been making assertions on datasets in Rasterio's tests and also dragging them into QGIS for a closer look after the tests finish.

https://live.staticflickr.com/65535/54120218694_cd56609f30_b.jpg

Laid up

Just when my running is getting regular again, I mess up my back with yoga and yardwork. I've got searing pain in my left hip that suggests a herniated disc, and have spent most of yesterday and today in bed. I slept okay last night and hope to again. This is another argument for voting by mail: anybody could find themselves unexpectedly disabled right before election day.

I'm very grateful that I got in a solid mountain run before my mishap. With luck I'll be back out there in a few weeks.

https://live.staticflickr.com/65535/54113272015_c4be540d32_b.jpg

A view over Horsetooth Reservoir and Fort Collins from the top of Arthur's Rock, elevation 6,780 ft (2,066 m).