I couldn't resist writing a real matplotlib/Agg mapping engine for PCL to see how it could handle real-world features. In this example I am using the VMAP0-based world borders data available from http://mappinghacks.com. Usually available, that is -- Schuyler's site is offline today. The world borders shapefile has 3784 polygons and roughly 400,000 vertices in all.
My prototype engine is about 9x slower than the existing MapServer-based engine. I actually had expected it to be even slower than this because I am doing some expensive operations in Python rather than C. Turning PCL geometries into a Python list of vertex tuples for matplotlib and transforming from world to image coordinates are costly. This mere factor of 9 gives me hope that a mapping engine using Agg's C++ API could be at least as fast as MapServer's GD renderer.
The result is below. Its color scheme is from Color Brewer, courtesy of Cindy Brewer. The grey blobs around coastlines are polygon stroke noise; the data is too detailed for a map of this scale.
Below, for comparison, is a map using the same data rendered by MapServer (and GD). It renders small features such as the Maldives rather poorly compared to matplotlib and Agg. Indonesia and the Philippines are messy as well.