Microsoft SQL Server 2008 to Have Spatial Data Natively

Now if only I could access that data from ArcMap without also having to go out and buy SDE. Sigh.

Manifold GIS? Haha, that’s kid’s stuff

James: ESRI is helping Microsoft so I’m sure he’s happy

Ed Katibah: SQL Server Spatial is being developed in-house.

Gee James, spreading disinformation as an ESRI evangelist again? Glad to see someone caught you this time…

I was only joking. After I read the comment I realized it could have been construed as a serious complaint against you – should’ve used a smiley face – with all your rants against ESRI, I don’t think people could ever seriously consider you an ESRI evangelist.

Apologies, and keep up the good work!

Google awoke the sleeping giant. Now they will see what a company like MS can do when they put their minds to it

Tell you what, MS is on fire right now. People point to GE and Google, but my money is on MS and the VE/SQL Server combo.

My money as well.

But in all fairness, James, you seem to missing what the implication is of Ed’s comment for all of the legacy GIS vendors. It means that both of the only two spatial DBMS products that appear to have any traction, Oracle and SQL Server (and it is telling that just the announcement gains for Microsoft instant traction), are defining spatial DBMS in a vendor-neutral way.

Microsoft without doubt is the gatekeeper for access to new markets that comprise billions of people. The VE/SQL combo in particular opens the door to a target market numbering in the hundreds of million. That’s the big show, for sure.

The only way you get to play in the big show is if you commit to supporting Microsoft standards. People in the mainstream are far too demanding and far too smart to tolerate any vendor who seeks to interpose their own standards (as ESRI tries with things like “SDE” as opposed to straight Oracle Spatial) against Microsoft. There’s no place in the mainstream for legacy vendors who are bent on interposing their own control over customers and data.

There’s a very funny line in the recent “Borat” movie where someone tells Borat that women in the US have a choice of who they date and Borat replies “Oh, that’s not good for me!” That’s exactly the situation ESRI faces in an ecosystem revolving around a spatial SQL Server, where vendors will have to compete on the basis of transparent interaction with SQL Server instead of trying to trap their users within their own silos. It’s not transparent access if you don’t have a choice, and when people have a choice that is not good for ESRI.

It’s a big part of the benefit of mainstream communities: Microsoft launches an enabling technology and thousands of vendors hop to it in competition, beating each other’s brains out to provide a better product for a lower price. Everyone benefits except those vendors unable to deliver mainstream capabilities for a mainstream price, and to date the legacy GIS vendors have shown a total inability to step up to that competitive imperative. I mean, give me a break: it has been over two years since x64 Windows went into beta and ESRI still doesn’t have 64-bit code? And they still charge minicomputer prices in a personal computer world?

So, yeah, sure I suppose the long-suffering ArcSDE community that is now morphing into a long-suffering ArcGIS server community might hope that the arrival of SQL Server spatial will cause ESRI to carve a few peepholes in the SDE silo from which prisoners can look out onto the mainstream. But what is more likely is that the technorati in the user base will swiftly adopt direct interaction with spatial SQL Server using mainstream software packages that are being readied to support SQL Server, the prisoners will escape the SDE silos and those silos will be abandoned along with all the other relics of 1990’s technology no one uses anymore.

A SQL Server providing vendor-neutral spatial Microsoft standards gives people the choice of using straight spatial SQL Server in a completely transparent way. And that is not good, not good at all, for ESRI.

I see your point James, I just don’t believe that ESRI will ever “support SQL Server natively” anymore than they support Oracle Spatial “natively.” It’s going to come with a bizarre, ESRI-specific armature of metadata tables and other proprietarizations that will function, no doubt about it, to trap people within an ESRI silo. That has been ESRI’s track record so far. And true, ArcSDE is no longer a product name, but the “SDE” stuff lives on as a technical approach in the various incarnations of “ArcGIS geodatabases.” With ESRI the past is very much prologue.

I also respectfully disagree that, compared to the millions in the mainstream, there are any significant numbers of installations interested in simultaneous work with Oracle, DB2, Informix and SQL Server (…don’t forget PostGIS!).

As a result of Manifold shipping DB2, SQL Server and Oracle Express with Locator in it on every Manifold DVD, we are used in more spatial DBMS / GIS installations than all other GIS vendors put together. Granted, those are small installations for the most part but the sheer diversity of installations involved have given us a good look at what the large numbers are in spatial DBMS.

Those large numbers reveal a focus by the great mass of installations on one DBMS at a time. People very strongly tend to choose one DBMS. So you find shops that are Oracle shops and you find shops that are SQL Server shops, but you don’t find very many shops that try to straddle two DBMS horses at the same time, let alone three or four.

Is there a handful of users who are interested in mixing multiple spatial DBMS’s at the same time? Sure, but they don’t amount to a tenth of one percent of the mainstream. If you are suggesting that ESRI’s future is a retreat from the mainstream into becoming a vendor of ever smaller numbers of ever more expensive niche products that numerically no one uses, well I agree with you. But that’s not the big prize of mainstream competition in the huge new market made possible by VE/SQL.

And, by the way, we are talking about this as if one of the GIS vendors, either a legacy vendor like ESRI or one of the modern vendors like Manifold, is going to end up mattering. The message of vast competition made possible by Microsoft enabling technology is that it is often someone completely off the radar that emerges as a winner from those thousands of new vendors who jump into the fray.

We can see that today with SQL Server where there already are vendors relatively unknown in the GIS world who have jumped into the market to provide spatial extensions for SQL Server. It is no accident that MapDotNet chose the spatialdb.com product ahead of ESRI, or, for that matter, Manifold. Virtual Earth and a spatial SQL Server will open up the seams of the sleepy little GIS community to all sorts of new players, providing many new choices. I welcome that, but I don’t believe ESRI does.

either a legacy vendor like ESRI or one of the modern vendors like Manifold

All in the eye of the beholder Dimitri. Many of us view both as dinosaurs.

Does anybody worry about the implications of Developers, IT, Lets say Computer Science taking over GIS vs actual GIS educated people. There are a ton of people running Arview that have no idea of what they are actually doing arlready causing me headaches and developers are really quick to grab something new from microsoft and start building on it. What is that going to do to data integrity as well as anlysis integrity? On the opposite side are we going to be able to do any anlysis with this stuff without esri or just throw it out there on virtual earth so everyone can say COOL!! I would love to be able to let everybody look at my data without my gdb lockingup and being forced back to shapefiles but 10K for 10 connections would be a huge sale for me and Enterpise ya right. But if I move out to say PostGIS I lose much of the analysis I do or want to do now with Spatial Anlysis.

Dimitri if your trying to sell me on Manifolf your too late. We tried it a few years ago as an alternative to esri for some of our customers and it didn’t pan out. I am all over Qgis on a usb stick by the way James your post was the bomb. I didn’t even know Qgis existed before then. I am talking about more in depth anlysis not buffer and intersect. And like it or not years and years of experience using esri for things like raster anlysis. I still knowpeople who won’t leave 3.x because rasters don’t have attributes. Now that they do there are so far behind the curve they won’t change. Problem is they have the best mind for solving the problems. Do you think there is a chance they are going to move to something even more out of there element. Of course I must qualify I do like a lot of esri stuff.

if your trying to sell me on Manifolf your too late. We tried it a few years ago as an alternative to esri for some of our customers and it didn’t pan out.

I intended to praise spatial DBMS in general and Microsoft’s initiative in particular as a superior alternative to being locked into an ESRI silo.

Microsoft initiatives like a spatial SQL Server bring choices to GIS folks they did not have before. That’s very much a Microsoft show, and fixating on any one vendor out of the thousands that any major Microsoft initiative brings to the party is missing the forest for a tree. That’s why my post expressly did not mention Manifold.

In point of fact, Manifold at present does not ship any spatial extenders for SQL Server. It will use SQL Server to store geometry via blobs using the usual types (ESRI SDE geometry, OGC WKB, etc.) but does not yet confer upon SQL Server any native geometry types or server-side spatial operators.

But other people are already extending SQL Server (as I said, you get choices in the Microsoft mainstream), and you can project forward from their current offerings to see where this business will go as many other mainstream players enter the market.

Visit spatialdb.com to learn about their spatial extender for SQL Server, and visit mapdotnet.com to see elegant things being done with SQL Server and spatialdb.com’s stuff.

In fact, both of the above vendors illustrate the richness of Microsoft’s ecosystem. Spatialdb.com could crank out a very elegant spatial extension for SQL Server because Microsoft made it easy for ISVs to extend SQL Server. MapDotNet could utilize spatialdb.com and Virtual Earth because of the great modularity of .NET architectures. Kudos to Microsoft, again. And when SQL Server 2008 comes out MapDotNet can drop in support for them as well.

At the same time, Manifold could easily support both MapDotNet and spatialdb.com even as we support SQL Server 2008, Oracle and other spatial DBMS packages. This sort of free-form, very productive cooperation is a hallmark of the Microsoft ecosystem and part of my message that in the mainstream you get lots of choices.

And, to be fair to all, it is not just a hallmark of Microsoft’s ecosystem – it is also one of the major benefits of “going native” with any major spatial DBMS. Get suckered into SDE or “geodatabases” and you are stuck working through ESRI . Choose native Oracle spatial types and your choices expand to immediate dynamic interchange with hundreds of different vendors. There is no way ESRI will ever assist other vendors in unlocking the doors of the SDE silo, but Oracle’s corporate mission is making it easy and simple for thousands of ISVs to work magic, at will, with data in vendor-neutral Oracle form.

I should remark, since you raised the point, that if you last worked with Manifold “a few years ago” your impression of it is probably over 3000 steps behind current editions, as the product is evolving right now at over 1000 improvements per year. A few years ago Manifold was not an enterprise class GIS, had no spatial DBMS capabilities and had limited spatial SQL. Now it can run with thousands of simultaneous users against terabyte Oracle Spatial data stores with superb spatial SQL and can utterly stomp ArcIMS installations through 64-bit performance. As before, it is still rock-solid reliable as well.

I am talking about more in depth anlysis not buffer and intersect.

Out of curiosity, like what?

It’s pretty amazing what can be done in just one or two lines of spatial SQL.

And, one of the coolest things about using a spatial DBMS is combining the power of server-side spatial analysis in the DBMS with client-side spatial analysis in a GIS.

For example, it’s trivially simple to extract analytic subsets of data out of an Oracle Spatial DBMS by creating a “view” using spatial SQL. This is very easy to learn and to apply and because it uses a standard methodology (SQL) there are, like, a seemingly infinite number of educational resources on it, ranging from “SQL for Idiots” type books to seriously expert resources.

Use the power of the DBMS cluster to get your view, and then link that view into your spatial DBMS-capable GIS. You can then use the analytic power of the GIS, either through client-side spatial SQL or through whatever command, menu or mouse-driven analytic tools your GIS provides to get exactly the analysis you want. It’s very fast, very easy to do and very effective.

If you tell me what you have in mind, I’m sure either I or one of the other participants could provide some concrete examples of how it could be done using the above approach.

weird, the link didn’t show up. anyway, here it is:

http://dspace.library.cornell.edu/handle/1813/2489

I think ArcSDE still got one advantage over standard spatial DBMS.

A standard spatial DBMS got lots of nice feature like spatial SQL and ArcSDE opens the gateway to these querying languages more and more, have a look at the new Spatial Type for Oracle.

There is nothing more frustrating than being stuck in a proprietary ArcSDE binary world, where no one can read your data and you can’t do any spatial querying without going through ArcObjects or the ArcSDE C API.

But if you install ArcSDE as middleware on a spatial DB, you get something very important: VERSIONING AND ARCHIVING out of the box.

This kind of setup allows others to read your spatial data (through views) and you can get all the power of multiuser versioned editing.

If something dies, I will bet it is the ArcSDE Binary Type, but not the ArcSDE Versioning Capabilities.

Prediction, SDE binary will die and Microsoft will scoop up ESRI to get the very robust management interfaces (ArcCatalog) and plop them on top of SQL spatial and ArcMap gets carted off to become MapPoint Uber Professional. All the ArcServer juicyness will just get folded into the current VE efforts.

Wow, this must be the shortest Dimitri comment in history …..

Wow alot happend while I was away. First I will have to look at spatialsql because some of what you mentioned was what I walk talking about and did not believe was possible especially for raster. Nearest Neighbor , Euclidean distance are all things we do for watershed analysis that I didn’t think I could do. My question is are these just sql commands? Other Anlysis I am wondering about then would be things like Geomentric Network Sewer Modeling. We have a ton of time and energy placed into getting exact measurement on this stuff to model it. How would you handle flows, pumps, valves etc. The next question is what to use for an interface. Yes I have no problem opening up something new and going through it. But getting the engineers to use ESRI was difficult enough I hear ( I wasn’t at this company at the time)

artlembo I would love to create cool new map tools. But yes you are right there are way too many GIS people who won’t.

My prefered progression is the ability to use ESRI for analysis, against native sql spatial data types that anybody else can read and the distribute my analysis and or data in any format. But that is because I am comfortable with ESRI and so are my co-workers (The bigger issue) The developer in me would love to dump esri and create cool new stuff, but the analyst in my is afraid of getting the wrong anwers and the GIS Manager in me is worried about how to change an entire system of people who are NOT technologically minded.

Spatial query in ArcGIS in a definition query here: http://webhelp.esri.com/arcgisdesktop/9.2/index.cfm?TopicName=Enhancing_ArcGIS_functionality_using_spatial_types&anchor=sqldefqry

The same logic could be used when the various selection methods.

I’d get on the Microsoft bandwagon once they decide to have a true GIS-capable software. As of now, both Google Earth and Virtual Earth are using Web Mercator projection. They don’t have true spatial reference. There’s more to geographic information science than simply storing, retrieving, and viewing data.

Microsoft should come up with a real GIS product that meets the standards from NMAS, USGS, and similar agencies. Otherwise, GIS would continue to be a mixed bag of ESRI ArcGIS and similar products and PostgreSQL or SQL Server 2008.