BostonGIS Blog - foss

Waiting for PostGIS 2.1 - ST_Tile

nospam@example.com (Regina Obe) — Tue, 05 Feb 2013 06:37:00 -0500

In a previous article Waiting for PostGIS 2.1 ST_Resize we demonstrated a new raster function ST_Resize that is available in upcoming 2.1. In this article we'll talk about another function called ST_Tile, which similar to ST_Resize can work with both georeferenced and non-georeferenced rasters. As the name suggests, ST_Tile can be used to cut up rasters right in the database, similar to what the raster2pgsql -t Tilesize commandline switch does when you import. In this article we'll demonstrate the misuse of ST_Tile. ST_Tile can also be used for tiling out of db rasters and what it does when it does that is not to actually tile, but instead mark off the areas where it would cut if it were to actually cut.

Another thing I'd like to mention here which can not be summed up in a single function is that a lot of work has gone into PostGIS 2.1 in improving the performance and robustness of out of database rasters (rasters stored externally but queried as if they were inside the database), and more is planned before 2.1 hits the street. All this work has made me reconsider where out dbs play a role and how their speed profiles compare to in-db rasters. We'll demonstrate the speed profile differences in this article for ST_Tile as well for this small sampling.

If you are not afraid of chewing the fat a little and you are on Windows, there are always fresh windows binaries packaged with all these goodies you can get from: Winnie's fresh baked windows PostGIS binaries corner.

WARNING: This article depicts PostGIS being used in a less than productive manner and shows graphic pictures which some audiences may consider violent and insensistive to the feelings of paintings. Please remember that PostGIS is a tool for doing SERIOUS work and should only be toyed with for demonstration or testing purposes. The fact that PostGIS can be used for amusement, and can become addictive is merely an accident of design. Studies have shown that although PostGIS is a powerful mind-altering drug, continued use and exposure WILL augment intellect and possibly spawn creativity.

Continue reading "Waiting for PostGIS 2.1 - ST_Tile"

Book Review of OpenStreetMap: Be your own Cartographer

nospam@example.com (Regina Obe) — Sun, 08 May 2011 00:15:00 -0400

OpenStreetMap by Jonathan Bennett and published by Packt Publishing is a book that outlines what OpenStreetMap is and how to navigate its various offerings and terminology. Perhaps more importantly, it provides the information you need to become an active contributor to this community mapping project and how to get help beyond what the book offers.

First a brief note about OpenStreetMap. OpenStreetMap is an open source community driven mapping project that comprises data, tools to edit the data, and tiling services and other services to view and reuse the data. It is built on PostgreSQL, our favorite relational database. Unlike other tile services such as Google Maps and Microsoft Bing, the tools and various backup mirrors available allow you to download all the data for your region and setup your own customized tiling service in house. This is actually the main reason we are interested in using it. Many of our clients have networks where because of firewalls, portability, general security concerns, need for further zoom in options for their area of interest, or just need to hide/add more layers to the tiles than what OpenStreetMap or other tile services provide out of the box, they need to be in control of the whole process.

Continue reading "Book Review of OpenStreetMap: Be your own Cartographer"

2011 CGA Conference at Harvard May 6-7 2011 Cambridge MA

nospam@example.com (Regina Obe) — Mon, 02 May 2011 18:34:00 -0400

We apologize for this short-notice. If you happen to be in the Cambridge / Boston area this week, are a big FOSS4G, Web Mapping, or overall GIS fan, you won't want to miss this event happening Friday and Saturday May 6th-7th at Harvard.

Geospatial Collaboration: New Common Ground

Harvard in cooperation with CGA are holding a conference this year May 6th-7th, 2011. Registration is Free and you can register at Geospatial Collaboration: New Common Ground Conference Registration.

There are lots of exciting sessions lined up. Chris Holmes of OpenGeo will be giving the Keynote address.

Lots of sessions lined up on a wide array of GIS topics. A flavor of topics to give you a sense of what is in store:

Community-based mapping
Examples of OpenStreetMap use
Land Managment
GeoDjango, MongoDB, and Solr in one breath
Open Access to Web Mapping Services
Opengeoportal
GIS at MIT
Open Geo Sofware
Mashable Boston
Using Balloons in GIS

Check the event schedule for more details. Event Schedule

PostGIS in Action book sale at Registration Booth

We'll be selling some copies of our book at the registration counter. We'll be selling it for a little less than Amazon or Manning. You may even be able to see us in person, and if you want us to dirty your book with our signature, we'd be happy too. Well Regina will be happy too, Leo still has to overcome his hangup of defacing books with ink.

PostGIS, PL/Pyton, Events, Mapserver XML Mapfiles

nospam@example.com (Regina Obe) — Fri, 20 Mar 2009 03:56:00 -0400

PostGIS 1.4 out soon

PostGIS 1.4 will be out soon, which will be good because it feels like forever we've had this release baking in the oven. The key changes are as follows: Which many are detailed in New in PostGIS 1.4

ST_IsValidReason() -- requires GEOS 3.1 -- will tell you why a geometry is not valid. Its a complement to ST_IsValid() which has existed for as long as I can remember
Improvements in all the aggregate functions ST_Collect, ST_Union, ST_Accum to use new array logic that can better handle collecting of many geometries. You won't notice this benefit much until you start collecting and unioning 1000s of geometries
Name change of hidden st*garray functions to be exposed as ST_Collect(geometry[]), ST_Union(geometry[]) ..etc
Cascade Aggregate Union -- requires GEOS 3.1 -- need I say more?
Prepared geometry - required GEOS 3.1
Better debugging facilities and cleaner code base

Sadly ST_DumpPoints did not make it into 1.4, and we seem to have jumped to 2.0 thinking. Where is 1.5? 2.0 seems offly ambitious (geodetic, better 3d, WKT Raster, restructure to allow support for more types, improved curve) that I feel we really need at least a 1.5 to cushion the path. There are a lot of things we have left on the plate such as ST_DumpPoints, Steve Frost's Tiger Geocoder upgrade for new tiger 2007/2008 format, minor enhancements to distance speed and functionality and dumper that don't require major restructure that I feel those should just come first and that need only be a 3-6 month incubation. Given our new policy of not introducing new functions in minor releases, it would seem almost necessary to have at least a 1.5

Continue reading "PostGIS, PL/Pyton, Events, Mapserver XML Mapfiles"

Mapserver Cheatsheet

nospam@example.com (Regina Obe) — Tue, 30 Dec 2008 16:19:00 -0500

We started to upgrade all our old mapfiles to the new 5.2 standard because first its about time and now that we are starting to do a non-trivial amount of Mapserver consulting, we felt it important to keep informed.

In doing so we started to develop a cheat sheet to help guide us on what's possible in Mapserver Mapfile 5.2. The cheat sheet can be found here -- keep in mind its still a work in progress so please refrain from laughing. In the middle of making this -- all the mapserver document links changed. I guess Mapserver documentors were busy too. I must say the new documenation is much better than the old. It has some nice examples all nicely color coded. So I want to thank Steve Lime, Jeff Mckenna, Jean-François Doyon and the others in Mapserver doc group for a job well done.

In mapping this cheat sheet out, the first thing slapping me on the side of the head is Mapserver really needs an XML defined schema. Yes I know I sound like a hypocrite and one who has been smoking XSLT a tad too long for my own good. Yes I was on the side of the group of people who thought What is it with these XML idiot lovers. What is to love in such a monstrously verbose unforgiving format? Mapfile format is wonderful, easy to read, fast to parse, and write THANK YOU very much.

Now it appears I am switching camps. Wait a minute though - the truth is I could care less if the Mapserver engine can understand XML. I just want to write my map file in XML and am happy to XSLT it to a format Mapserver can deal with. Why? Do I think XML is a beautiful language? On the contrary -- I find the format stinks and I think it is inefficient in many use-cases. It is in fact the most abused language if there ever was one.

Our main frustration with XML is that in our line of work we do a lot of data loading, data scrubbing and so forth, and we are usually on the side of the fence where some goof-ball gives you some blob of XML with weird characters and other junk and just stuffs it in a CDATA tag and thinks that will solve all issues or where some guy just took OO UML 101 and he has spent a great deal of time over-thinking his data model when time would have been better spent getting loaded at the nearest bar. He has this wonderfully elaborate 30-level portable XML data file for his simple list of jobs complete with 200-pages describing every facet of it for you to digest.

XML is bulky, unforgiving, and easily abused so why is it useful? The beauty of XML and XML XSDs is that a lot of software and development tools support them.

If I had an XSD for mapserver, my editor would immediately tell me what values are valid for each type that takes a fixed number of constants
My editor would immediately tell me when I am stuffing something in a layer that is not valid
I can XSLT my way to a standard mapserver file or god forbid some sort of tree thingy with links to documentation of that element. How many times have you looked at the mapserver documentation and thought? Can I stuff this object in a layer group? Have no clue - the documentation doesn't really tell me that unless the writer happened to remember and even then there is way too much reading involved for my 5-minute attention span to deal with.
Nowadays XSLT joins the group of SQL as an embeddable language. Can you think of a modern language that can't deal with XML/XSLT? In PHP/.NET which we program in predominately, both have XSLT classes that can take any xsl file and xml file and transform. My code editor has a built-in XSL processor.
Mapserver mapfile format just lends itself to that - think nested groups, groups that can go in some groups but not others or can have at most x number of item z in it. Mapserver is far from a simple structure with simple rules, so demands something designed for that kind of thing.
I tend to think it would help the documentation effort immensely too, because right now as the documentation stands, there is not enough cross-referencing going on or an easy way to visualize the natural flow of the map file. An XSLT of an XSD would solve that problem nicely I think. Take for example LABEL. Do you have any clue what elements a label can be stuffed in if you were just starting out?

So to me -- or at least with my understanding of things XML -> MapFile -- fairly trivial, XSD fairly trivial but torturous exercise, MapFile --> XML not so trivial but less torturous exercise. I'm perfectly content with an XML -> MapFile route and am actually thinking about creating such a thing for 5.2 if someone hasn't done it already, if only I could ever find the time, but if things keep going as they are, I may just have to do that if nothing but to keep my sanity.

Cross Compare SQL Server 2008 Spatial, PostgreSQL/PostGIS 1.3-1.4, MySQL 5-6

nospam@example.com (Regina Obe) — Sat, 19 Jul 2008 18:25:00 -0400

A few people have been asking us what are the pros and cons of using SQL Server 2008 Spatial and PostGIS and as a Windows user, why would you still consider using PostGIS. Rather than simply providing some hand-waving saying "well if you just care about displaying data, then use whatever you feel comfortable with, but if you want to do real intensive sophisticated spatial analysis and geometric processing without having to purchase a bunch of expensive software, then PostGIS is probably better for you. Hell why must you think in either or propositions - just use both using the strengths of each.", we have tried really hard to quantify the similarities and differences between the 2 and to boot - we have also added in MySQL.

Our analysis can be found at Cross Compare SQL Server 2008 Spatial, PostgreSQL/PostGIS 1.3-1.4, MySQL 5-6

If you have any comments, suggestions of additions, things you felt we got wrong, then please don't hesitate to comment and we'll try to update our survey.

Can Open source Software teach you anything about using Proprietary Software?

nospam@example.com (Regina Obe) — Thu, 04 Oct 2007 08:50:00 -0400

I was reading James Fee's blog recently on Open Source on the beach at Waikiki and as usual James does a good job of stirring up thinking.

One thing I found interesting about the comments was the perception a lot of people have that you choose to understand one tool or another and if you choose open source for one thing, then you are somehow taking away time to learn more useful proprietary techniques. I have found this speculation to be for the most part untrue.

For example I consider myself to be fundamentally a Microsoft SQL Server expert and fairly competent in Microsoft technologies. When I started to get involved in Open source software - such as PostgreSQL, MySQL, Linux, Mapserver, PHP, etc. I discovered something very startling. These Open source folks really care about standards.. Understanding standards is more important than understanding how to use a set piece of software because standards are more aligned with trends in technology.

For example, A long time ago, I used to think COALESCE and NULLIF were functions developed by the PostgreSQL folks. One day forgetting which database I was in, I accidentally used these in SQL Server. And guess what? SQL Server knew what they meant. Similarly SQL concepts like INTERSECT and EXCEPT existed in PostgreSQL long before SQL Server introduced it into SQL Server 2005. So when SQL Server 2005 came out, gosh darn it I already knew how to use these tools. The same holds true for PostGIS by the way. You will find that PostGIS does things the OGC standards way so when SQL Server 2008 finally comes out with OGC Spatial support, I suspect people using PostGIS and similar tools will be way ahead of the curve.

Other example - because of my exposure to open source tools such as PHP, Linux, PostgreSQL, I realized early on that Microsoft's Active Directory was nothing more than a Light-weight Directory Access Protocol (LDAP) service by another name. This had some very interesting consequences. For example I realized that it was much easier to query Microsoft Active Directory with PHP than it was even using Microsoft.NET Framework.

In the GIS world, because of my exposure to Mapserver and other open gis tools, I knew what ESRI's WFSConnector and WMSConnector were for and how to test them long before my GIS brethren knew what those terms even stood for. I knew the foundations of these and found myself explaining the concepts of web services etc. to my GIS friends.

So the point is this - if you are spending your time simply learning how to use proprietary tools instead of really trying to understand the fundamentals that drive these tools, you have totally missed the boat and you will be forever playing catchup.

I have found that Open source really enforces understanding fundamentals more than proprietary does and that is a very good thing because that knowledge has a longer shelf-life.