A new CLI for GDAL

Even Rouault has proposed a new, modern, and more coherent command line interface (CLI) for the GDAL/OGR project. I think it's a good idea and a good time to do it. I've wanted a better one for about 15 years. Even credits Rasterio for inspiration, and that's gratifying to see. I started writing Rasterio 10 years ago in part because I wanted a better CLI for GDAL.

What I wanted in a GDAL CLI were the following features:

  • A root command and a few subcommands, one namespace for everything.

  • Uniform arguments and options with predictable ordering and naming.

  • Good documentation of arguments and options.

  • More subcommands with fewer options each. Making gdal_translate into 3-4 commands, for example.

  • Input and output that favor stdin/stdout and JSON.

  • Ease of installation. For example, with pip instead of an OS package manager.

I estimated in 2013-2014 that it was not feasible for me to achieve those goals within the GDAL project itself. GDAL and its community had no funding for this kind of work at the time. I found the GDAL project's tests somewhat inscrutable and frustrating. A hefty legacy of documentation and folk wisdom about the old ways would have to be updated. Mostly by me, certainly. And the GDAL user community largely did not care. Free software that was fast and effective (and, most of all, free!) was already more than most people had dreamed of. A GIS analyst had so many business and organizational problems to deal with already that the rough edges of gdalinfo and gdal_translate didn't even crack her top 20. Software polish wasn't a big concern in the second decade of FOSS4G. I think it's still a hard thing to sell. Individual consumers will pay money for slick, well-designed software that makes them feel good. Organizations value polish less. And neither GDAL nor Rasterio sell anything to individual consumers.

Overhauling gdal_translate, ogr2ogr, and friends within the GDAL project looked like a non-starter to me. Pushing a herd of boulders up a hill, by myself, for free, for a community that was largely content with working around and stepping over these boulders. I think I made the right choice for myself. I got to start from scratch, move fast, and use a modern CLI framework. I made a command line interface for Rasterio that, while not perfect, met most of my goals. And I didn't go broke or burn out while doing it.

Today, thanks to years of fundraising work by Howard Butler, Paul Ramsey, Kristian Evers, and Even, the GDAL project does have funding to overhaul its command line interface as an aspect of overall project health and maintenance. Multiple developers can be paid to work on it. They won't have to donate their time to it as I would have. Rasterio's command line interface can't be adopted by GDAL, or be forked to become GDAL's because it doesn't have all the features of existing GDAL programs (or even of gdal_translate and ogr2ogr for that matter), and my decision to have more subcommands with fewer options is kind of against the grain of GDAL. But the new GDAL CLI can adopt the demonstrably useful features and design of Rasterio's. JSON output, for example, is something that GDAL has already picked up from Rasterio.

Rasterio will certainly fade a little if the new GDAL CLI is designed and executed well. But that's in the nature of software and software communities. Rasterio has always depended on GDAL and benefited from being built on a technically solid and well loved foundation. And I didn't invent CLI subcommands and JSON output, not at all. It's not unfair. If you succeed in open source, if you move the needle, you will be emulated. In this case, I think we can call it progress. I'm content.

In the long run, I stand to get half of what I originally wanted from a GDAL CLI, the first three of the six features I listed above. And there's probably still room for a suite of Unix style programs with different opinions and design decisions, especially if it and GDAL agree on basic concepts, arguments, options, and flags.