ESRI to Support SQL Server 2008 Spatial
Good news for those who want to take advantage of SQL Server 2008 Spatial and ArcGIS.
ESRI’s ArcGIS 9.3 software, the next scheduled release of ESRI’s ArcGIS suite, will take full advantage of the new spatial technology in the upcoming release of SQL Server 2008. With the November SQL Server 2008 community technology preview (CTP), Microsoft Corporation is extending the use and value of spatial technology by integrating it directly within SQL Server at no additional cost.
Our Database Admin and I were just wondering about this yesterday.
So will ArcMap read SQL Server data directly? Or will ArcSDE still be a required part of the architecture, a la Oracle Spatial, ST_Geom, etc.?
My ESRI contact just told me it will read SQL Server directly. Now they could be mistaken, but I think this is how this PR reads.
Maybe James can clarify?
At a conference a few months ago, the MS guy told me that SQL Server could store multiple geometries in each record. I wonder how ArcGIS will handle this or will the concept make Jack’s head explode?
Jack,
It shouldn’t be too hard. PostGIS stores multiple geometries alread, each in a separate column. If SQL Server works similarily, specify the column containing the desired geometry.
What I’d like to see is ESRI support PostGIS directly!
I’m wondering what this announcement implies for current ArcSDE users. Will the enabling of direct access to SQL 2008 mean that we can happily eliminate our $3K a year SDE maintenance and port our data into SQL 08’s data structures?
I highly doubt it’s that simplistic…what’s everyone’s opinion?
Cheers,
-Matt
Well, if ESRI supports SQL Server 2008 natively I’ll have a couple of SDE licenses for sale if anybody wants one.
I’m guessing they’ll throw in some kind of hook that makes you have ArcSDE. I just don’t see them writing the Desktop client to directly send spatial SQL back to the database. In fact, I can almost guarantee it’ll be how they support Oracle Locator today…..you set a switch in the dbtune.sde file to write MS_geom or whatever.
@J Wallis
I suspect you’re right about that. It’s my understanding that the PostgreSQL/PostGIS support at 9.3 will be done that way so I’d be surprised if they do something different for SQL Server.
It´s good for SQL server express users
’bout time. I mean, others have been supporting this concept for a while (spatial database w/o need for middleware), and I seem to recall a post just a while back re. both Manifold and Safe supporting MSSQL2008.
Of course, I’d also be a little ticked that ESRI praticalyl forced me to get SDE into my business in the first place, and *now* they see the light and are making [it] irrelevant themsleves…
Glad to see the support – wish it had come sooner or as a less expesive option.
“What I’d like to see is ESRI support PostGIS directly”
Funny that this one keeps popping up even though it’s been announced loooong ago that 9.3 also will support PostGreSQL.
Fishing arond Dave Bouwman’s blog and ESRI’s support site, it seems that ESRI is moving away from the concept of a separate SDE process on your server. The functionality of the SDE is being built directly into their desktop and server products. I found a paper on linking to Oracle Spatial using Direct Connect instead of a SDE server process. I think the paper said that you have to do a trick with your license file so that ArcMap believes you’re connected to an SDE even if you aren’t. It’s a bit hazy now, but I’ll post the link when I get back to the office. (or is all of this old news?).
The upshot of this, IMO, is that ArcMap will offer Direct Connect to SQL Server 2008 without the need for the SDE. Again, just IMO. Not to do so would be a huge mistake and really leave a door open for Microsoft to develop a low-cost alternative that cuts into ESRI’s client-base.
I suspect (nothing more than a hunch) that you’ll need SDE if you want to continue to use explicit versioning.
If you’re fine with non-versioned, immediate update style of transactions (most if not all of the clients I’ve worked with are) then direct SQL access should be a-ok.
Manifold supports this now. ESRI always following in the rear.
Um, SQL Server 2008 hasn’t even been released yet. Who can use it in production situations? Not I…
But when SQL Server 2008 goes final, ESRI won’t have a final release of ArcGIS ready so then you can be smug. Right now, no…
Funny this post happens today. We have ESRI in our office today to review new infrastructure reqs. They said that ArcGIS will support SQL 2008 but that it will require ArcSDE for communication much like it does with Oracle Spatial.
In fact, much like Oracle Spatial, it will do all computation in SDE – NOT the database – so you are not leveraging the power of the db to do the work… your using SDE unless you go around it…
I wonder if you’ll be able to work with “registered layers” like you can with Oracle. Taking that approach, Oracle Spatial does the heavy lifting and ArcSDE is just a pass-through to ESRI clients.
They said that ArcGIS will support SQL 2008 but that it will require ArcSDE for communication much like it does with Oracle Spatial.
You are being screwed!!! Quick, before those bandits leave your office today stop them! Ask them why they are doing this. They don’t need to do it. Other products like Manifold allow a direct connection. It is simply to sell more ESRI software. SDE is not needed, and this is nothing more than a way to extort money out of you to buy SDE too.
Tell those crooks you’ll take a serious look at Manifold if they don’t change their business approach.
Aaaaagh. This is just annoying!!
Please write us back and tell us what they say..
Hows about supporting OGR drivers. Then I could decide what formats the software supports.
@tom riddle
they told me to go screw myself….and they’d send me the bill.
Warning: If you need something serious, Manifold is NOT the choice!
Now if they could just get ArcExplorer to read SDE.
Hey, this so much fun…
For Lefty:
“Um, SQL Server 2008 hasn’t even been released yet. Who can use it in production situations? Not I… “
Don’t get left in the dust. SQL Server 2008 spatial is out as of the November CTP. Download it and start using it today, either in 32-bit or 64-bit Windows. A Microsoft CTP is, of course, about a thousand times more robust than final ESRI production code so if you are used to ESRI there is no reason to hesitate.
We’ve (I work for Manifold) been working with SQL Server 2008 spatial since, what? May or June? of this year in one of the early pre-prereleases and it is rock solid stable, very fast and awesomely powerful. Visit the Microsoft MSDN forums for SQL Server 2008 and you’ll see the enthusiasm in the SQL Server community for this outstanding new product. Don’t hesitate, dive right in.
For Mel:
“Warning: If you need something serious, Manifold is NOT the choice!” …. like, what did you have in mind by the word “serious,” Mel?
Did you mean 64-bit native code throughout the system? Uh, yeah, Manifold gets a check mark there, ESRI does not. You’re not seriously suggesting that anyone stuck with 32-bit Windows is “serious,” right?
How about multicore processors and massively parallel excution via CUDA? Running 256 processors simultaneously sound “serious?” Manifold does that… how about ESRI? What, that’s a “no” again? Hey… I see a pattern here.
Or, maybe being “serious” enough not to drag feet on Microsoft standards. Vista, to name one. Virtual Earth to name another, … and how about this thread…. SQL Server 2008 spatial support today, out of the box, with rocket fast performance and integrated storage of images and surfaces too as well as vectors. Does that strike you as more “serious” than announcing vaporware?
Guys, please, this SDE thing is an intelligence test. If you stick your head into ESRI’s SDE sucker trap you’ve failed the test. There is no need for SDE in modern times. None.
For that matter, if anyone really says they “support” spatial DBMS there is a level of orthogonality and transparency and automation of function that you get today with Manifold that I don’t think you will ever see from ESRI.
People don’t want to crudely hack data in and out of some spatial DBMS like pulling teeth, they want all the issues dealt with automatically and effortlessly, like projection matching. With Manifold you can pop open multiple windows to visualize data from Oracle Spatial, SQL Server 2008, PostgreSQL and DB2 and cut and paste between them. Manifold automatically converts on the fly geometry types, re-projects on the fly to match native projections available and even automatically converts attribute types.
Let’s take a look at what competent $395 software can do with modern spatial DBMS and see how Arc compares:
There are now four “native” spatial DBMS solutions: Oracle, DB2 with IBM’s Spatial Extender, PostgreSQL/PostGIS and SQL Server 2008. [I don't really count IBM Informix as a separate spatial DBMS since it so similar to IBM's DB2 spatial extender].
Manifold can connect to all of them, simultaneously, using their native connection technologies (such as OCI for Oracle) and their native geometry (such as SDO_GEOMETRY or GEORASTERS in Oracle, ST_GEOMETRY in DB2, GEOGRAPHY or GEOMETRY types in SQL Server 2008, WKB in PostgreSQL) without any middleware nonsense like ArcSDE. It does this with full import/export, read/write, edit for potentially thousands of simultaneous Manifold users working with potentially terabytes of data using Area of Interest windowing. In 64 bits any one of these spatial DBMS solutions with Manifold will make ESRI-anything look like it is halted (which, if it is ESRI software odds-on it might be…).
For that matter, Manifold can also simultaneously connect to ESRI SDE and Personal geodatabases for read/write/edit for simultaneous multiple users without needing no stinkin’ ESRI middleware either.
Manifold will do all the above, displaying and editing data within as many windows as you like in your desktop, automatically re-projecting on the fly, matching projections to a variety of projections available in different target systems, maintaining attributes and translating into available types on the fly as well. It will do that with vectors or rasters, images or surfaces, in 2D or 3D (well, “2.5 D”), and it will automatically run all that through Manifold’s built-in IMS for Internet map server displays using HTML, WMS, WFS-T or image server tiling, if you want to have a web application. All of it works whether you are doing it on the desktop or serving hundreds of thousands of visitors per day on your web site (scales out well to millions of users, if that’s what you need…).
You can overlay GEOGRAPHY and GEOMETRY data from SQL Server into the same window, together with data from Oracle, SDE geodatabases, DB2, Virtual Earth, etc., and move data from one to the other with a mouse point and a click and a simple copy and paste: Manif0ld will automatically manage all the translations, reprojections, recasting of attribute types, etc, on the fly.
Manifold can also confer spatial DBMS capabilities to just about any DBMS system that can handle binary types. Use MySQL if you like. And, it can store images of almost unlimited extent (hundreds of gigabytes) into DBMS products like SQL Server 2008 that do not normally have tiled raster storage capability and then you can fetch and display those images almost instantaneously, pretty much regardless of size.
Want to work with SQL Server 2008? Look at examples of storing a drawing into SQL Server 2008 and then using it:
http://www.manifold.net/doc/example_storing_a_drawing_in_sql_server_2008.htm
http://www.manifold.net/doc/example_linking_a_drawing_from_sql_server_2008.htm
See how easy it is to store a large image into SQL Server 2008:
http://www.manifold.net/doc/example_storing_an_image_in_sql_server_2008.htm
…and then look at an example of tracing over Virtual Earth into SQL Server at
http://www.manifold.net/doc/example_tracing_virtual_earth_into_sql_server_2008.htm
You get all that with Manifold Enterprise Edition for $395 and you also get vastly more capability (like spatial SQL) to boot. And you get it in the convenience of a single, integrated package that has zero maintenance fees. I believe no matter how much you spend with ESRI you will never be able to get anything as robust, simple to use, free of middleware proprietarization, and completely orthogonal and transparent to a wide variety of data sources.
Dimitri: I do have SQL Server 2008 installed and have been using it in our test environments. We can not certify its use until it is in release mode. That is just how things are in my line of work.
At the risk of being dramatic, our software crashes, people die.
@Dimitri
What department did you say you work in again? Marketing?
Impressive Dimitri… I really didn’t know about all your wonders, $395 is way underpriced.
Lately, every single thread is turning on ESRI is a piece of shit and MANIFOLD is so great, don’t be so idiot not purchase Manifold…
I can’t add anything positive to what Kevin found out. I checked my link and the Oracle Spatial direct connect is read-only. ESRI will keep you dependent on the SDE, but, like I said, I think this leaves a big door open for MS for another company to develop a lower-cost program to fill the void. Manifold may do that now, but without some type of a free trial, I’m not diving in.
Really, my interest is in GDAL and MapServer support for SQL Server 2008 Spatial. I’d love for MapServer to offer a native, high-performance dll for access to SSS.
Mel, really…
“Lately, every single thread is turning on ESRI is a piece of shit and MANIFOLD is so great, don’t be so idiot not purchase Manifold…”
Well, you posted your opinion [explicitly saying not to purchase Manifold] so it is fair game to reply with specific examples why what you wrote is not true. I’ll try to keep it shorter next time.
Also, my criticism of SDE given revolutionary spatial DBMS like Katmai in no way suggests the rest of ESRI product is “sh*t”. You’ve got to give credit to Microsoft for what they have done and what they have done in spatial DBMS completely ignores both ESRI and SDE. There’s a message in that.
Look, I happen to like SQL Server 2008 and I think what we do with it defines the meaning of “serious.” But for all that, buying Manifold is not my thesis. Buying SQL Server 2008 is my thesis.
The point of getting into SQL Server 2008 is exactly to take advantage of a more open ecosystem so you don’t have to be tied to any one vendor – not ESRI, not us, not anyone.
Your GIS vendor should be neutral, totally neutral, within SQL Server 2008 so that any moment you can be sure that you can work with hundreds of different applications against your DBMS-stored spatial data and at any moment if your GIS vendor doesn’t do what you need you can dump them and move on to someone who can.
So don’t buy Manifold or ESRI or any other GIS. Buy SQL Server 2008 and master it. Truly understand how you can leverage it and take your pick from all of the thousands of vendors hoping to be a useful accessory to your central choice of SQL Server. Play them off against one another, demand a better price, more technology, faster performance and enjoy being in charge.
You tell me where SDE fits into that picture and then I’ll tell you who is being serious and who is not.
Have a good weekend, everyone!
I’ll say if you are knee deep in ArcGIS, SDE does fit in the picture and is a key part, but your general point about SQL Server 2008 and leveraging it across multiple platforms is key to the future of GIS (I’ll say PostGIS is important as well).
I am a died hard ESRI GIS software user for more than fifteen years and had been associated with Redlands up until May 2006. My comment is Manifold and other such as Mapserver open my eyes on modern GIS technology. If cost-effectiveness, performance and flexibility are your organisation main concerns then it is worth the effor and money (US$395) to evaluate Manifold, especially the way the technology integrate with enterprise database management system (i.e. Oracle, MS Server, IBM DB) and its IMS architecture.
End note:
This is not to condem ESRI’s pioneering efforts and champion of GIS development. Jack Dangermond has my respect as a GIS innovator. There are a lot for me to learn from he. There are also a group of very special people in the Manifold communuty ( http://forum.manifold.net/Site/) we should not missed.
There are also a group of very special people in the Manifold communuty ( http://forum.manifold.net/Site/) we should not missed.
I think those people are mostly hobbiests. I don’t think they have anyone all that “special” as you say. If you want real GIS to get done, get an ESRI guy any day I say.
Jojo:
“We are all born ignorant, but one must work hard to remain stupid.”
Benjamin Franklin
When Manifold improves it’s cartographic output. Then we are cooking!! Otherwise, sorta like Dimitri said, use SQL 08 and slap whatever you want on top of it. Be ESRI, Manifold, etc… Whatever gets the job done and allowed by the powers-that-be.
I won’t be able test any improvements in the future, since there isn’t access to the Manifold anymore.
It should be interesting what shakes out in the coming years.
KoS
@tskam,
“My comment is Manifold and other such as Mapserver open my eyes on modern GIS technology.”
I will certainly echo that in the case of MapServer. I was discussing Image Server with my boss. We pulled up the GSA schedule pricing and almost choked. The base price was $16K, then you had to purchase additional modules if you wanted all the features shown in their brochure. I let my boss know that Image Server isn’t a bad product, but it wouldn’t deliver nearly the value of its price tag for us. USGS has a great WMS aerial service that appears to be MapServer based. If you have some expertise with MapServer, for the cost of Image Server plus a server to run it, you’d be better off just devoting all the money to getting a great server and creating a simple MapServer script to respond to WMS requests.
I know ImageServer has some additional capabilities if you purchase the extra modules, but if you need to create a simple aerial server, I’m going to bet that MapServer is every bit as good as Image Server.
This is the link from ESRI’s site to GSA Advantage:
https://www.gsaadvantage.gov/advgsa/advantage/main/start_page.do
“If cost-effectiveness, performance and flexibility are your organisation main concerns then it is worth the effor and money (US$395) to evaluate Manifold”
Sadly, that really is not the main concern of my organization when it comes to ESRI. It is really more about people maintaining power and control.
@tim.
Not sure what agency you work for. Our agency, even if someone wanted to try Manifold or another software package. If it’s not on the approved list from national. It will never, ever be loaded on a machine, even if it’s just to demo or review the product. Or better yet, save taxpayers some coin.
Which is sad, there are other products out in the market which do a great job. Our agency is ESRI centric, for the good and bad.
Saving money, haha, govt doesn’t seem care. Unless it very close to an election. It’s not their money and all they have to do is “stick a gun” to peoples heads and more money appears.
KoS
@KoS
Sounds familiar. I do a lot of Navy work and am completely free to use whatever tools the NMCI cops give me.
And that’s becoming the norm rather than the exception.
@bill
Let me guess, it takes a year or two for them to “certify” a product?
There have been times where we have been a version of two behind because of the slow “certification” process.
KoS
JoJo, define “hobbyist” for me will you?
I’m an exploration geologist and have been for 35 years. Is that a hobbyist to you? I use Manifold every day. It meets all of my needs.
A software package is just a means to an end. In my case, the end is analyzing data.
I really don’t care what you use to do your job. Just don’t try to insult me for making different choices.
“Jack Dangermond has my respect as a GIS innovator. “
I suggest, he’s more marketor, than an innovator. Or, maybe he’s an innovative marketor.
Think about it, ESRI Press, ArcUser, ArcNews, all those glossy brochures. Spectacular booths at all the trade shows. And let’s not forget our Mecca…UC.
Sure this is just the price of doing business at their level, but ESRI has mastered it, and I would guess that Jack’s been the mind behind it. (We’ll it could be Laura.)
I would also suggest, that Jack’s more in it for the money than the guise of connecting our world, teaching us the geographic language, or the geographic approach. (Or whatever this year’s UC slogan is.)
Making people feel warm and fuzzy about you will get a lot of people on the band wagon. But, it’s interesting to contrast all of ESRI’s environmental lingo with their major revenue sources–fortune 500 and the U.S. military. I sense incongruity…
Scott Morehouse is probably more to credit as the early innovator.
All that said, I do love what I can do with ArcMap, and a file based geodatabase. But I don’t buy the Jack is the innovator line…
@KoS
You’re right on the money. It’s pretty painful. But Lefty’s comment a while back is rather applicable, too. Some environments I work in have tight CM requirements for the same kinds of reasons. They’re just moving to SQL 2005 in those places. SQL 2008 is a distant dream. But that doesn’t mean I’m not using it in my home office!
I wonder how Manifold deals with multilingual support… The software Giants are commited to that issue and we need it (Right to Left etc.)
@P
“Funny that this one keeps popping up even though it’s been announced loooong ago that 9.3 also will support PostGreSQL.”
Sure, it’s been announced, but there are just as many questions as to how the support will be implemented. If I have to use ArcSDE to access a postgis table, then it isn’t direct access in my books.
Just use Manifold….
Funny how Dimitri chimes in with a 50 page essay every chance he gets.
All marketing hype. They are so dying to get bought out by Microsoft it’s sickening.
Well, Microsoft is not going to buy the Manifold Koolaid. ESRI offers a solution that isn’t cobbled together and Microsoft could buy a turn key solution with ESRI and rebrand it easily….just like they did with Great Plains.
‘ESRI offers a solution that isn’t cobbled together’
If ArcWhatever and its add-ons aren’t a prime example of ‘cobbled together’ I don’t know what is.
Show me what other GIS software company offers an end to end integrated solution.
Dang, this is so much fun I’m reluctant to spoil it by talking seriously about reality…
“Funny how Dimitri chimes in with a 50 page essay every chance he gets.”
Not to everyone’s taste, for sure, but serious discussions of complex matters are not conducted in sound bites.
Speaking of sound bites…
—> “All marketing hype.”
Manifold neither announces nor documents anything that is not already shipping. That is exactly the opposite of marketing hype.
—> “They are so dying to get bought out by Microsoft it’s sickening.”
Companies which are on an upward trajectory, like Manifold, are in no hurry to get bought out. They make plenty of money, they like what they are doing and they know perfectly well that as long as they are rocketing upwards their valuation only increases.
It is companies on a downward trajectory like MapInfo and ESRI, who are behind in technology and struggling with new business paradigms, that are eager to sell themselves.
—> “Well, Microsoft is not going to buy the Manifold Koolaid.”
Microsoft doesn’t buy anyone’s Koolaid. Instead, they have won a lot of support from software partners by giving developers a fair chance to do their best. The entire Katmai CTP (Community Technology Preview) process is a good example of that, where Microsoft has been incredibly open and fair about providing full disclosure on this new product and providing free availability to the public even in pre-release form.
Our work with Katmai has been especially positive. We are extremely grateful and deeply flattered Microsoft gave us a chance to work with early internal releases of Katmai spatial, so early in the process that we could support Katmai spatial in our production product in August, three months before the first public Katmai spatial CTP. We had no doubt SQL Server 2008 spatial capability would be massive news in the Microsoft community, so we invested without hesitation the engineering resources required to do a full stack within Manifold for Katmai spatial. FME and others also supported SQL Server 2008 without hesitation and are now reaping the benefits.
ESRI had the same chance to do their best, but instead, they sat on their hands. So now, everyone around the world who is doing GIS with SQL Server 2008 spatial is using Manifold instead of ESRI. No doubt if ESRI weren’t being morons they’d have won a significant share of that business. But as it is we are taking it all. The only Koolaid being drunk is by those folks who are missing the immense benefit of SQL Server 2008 today because they are waiting for ESRI vaporware.
—> “ESRI offers a solution that isn’t cobbled together and Microsoft could buy a turn key solution with ESRI and rebrand it easily….just like they did with Great Plains.”
For starters, anyone with spatial DBMS expertise can see that Microsoft did not select an ESRI approach for Katmai. I’ve heard rumours that ESRI offered their technology free to Microsoft to get the ESRI approach embedded within Katmai, much as ESRI played a role in the spatial extender for Informix and DB2 at IBM, but that Microsoft declined.
Instead, Microsoft clearly designed their own approach and just as clearly designed it very unlike ESRI. There is a message in that: it’s pretty clear that if Microsoft is spending zillions of dollars doing something new in competition to the way ESRI has done it, they’re not likely to want to buy antiquated technology to which they’ve already spurned free access.
For that matter, there is no need for the most handsome, most personable rich guy at a party to have to buy companionship when he already gets to enjoy the company of all the prettiest girls for free.
There’s no need for Microsoft to buy anyone to get the benefit of players like FME or Manifold within their ecosystem. The lure of the riches in the Microsoft ecosystem are enough to draw thousands of players into it with fierce competition, all of which works to Microsoft’s benefit. It’s a party that all the prettiest girls are eagerly going to attend no matter what.
Perhaps a bigger obstacle to an acquisition of ESRI is that it is not just a matter that ESRI’s product line is a Frankenstein quilt of disparate products, technologies and object models. It is the technical difficulty and expense of modernizing all that so it works with Microsoft’s most important technical initiatives, like Vista, 64-bit operation, Katmai, distributed processing, massively-parallel high performance computing and the like, and doing that modernization in a way that can be maintained years into the future. That’s a total re-write from the ground up that would cost hundreds of millions while sucking the career life blood from any executive foolish enough to become responsible for it.
If you don’t think Microsoft promotes Vista with deadly seriousness you don’t know anything about Microsoft. If you think Microsoft is in any way amused by a technological cripple like ESRI failing promptly to support Vista (over two years after betas were widely available!), you likewise don’t know anything about Microsoft. And if you think Microsoft is going to spend hundreds of millions of dollars to modernize ESRI’s spaghetti code and paleoGIS product line when ESRI themselves did not find it economically possible to do so, well, you don’t understand at all that while Microsoft is rich they didn’t get that way by spending money foolishly.
Look, the point of all this is that nobody who has it going on in the GIS industry needs Microsoft to be anything other than the splendid ecosystem and technology framework they provide. When Microsoft creates things like Katmai, they create a pathway to wealth for anyone with the will and the ability to seize the opportunity.
And anybody in the GIS industry who doesn’t have it going on, who cannot seize those opportunities, who is trying their chances with a dull blade against a razor in a knife fight with the likes of Manifold, well, they should not expect Microsoft to intervene to rescue them.
Speaking of dull blades, ESRI’s gross negligence in failing to support SQL Server 2008 spatial months ago is one of those epic blunders that cannot be hidden by a “too little, too late” vaporware announcement about an SDE approach to Katmai. It’s like an aging courtesan hoping the right shade of lipstick will divert attention from all the wrinkles and missing teeth. I think even ESRI partisans privately recognize it as a sign that ESRI’s falling behind in technology may be approaching an irrecoverable turning point.
i guess you missed the post where Microsoft bought Multimap.
“I think even ESRI partisans privately recognize it as a sign that ESRI’s falling behind in technology may be approaching an irrecoverable turning point.”
ok, I have to call the spin alert on that one. ESRI does tons of modern things Manifold is not doing. Like ArcWeb Services GIS in the cloud with heavy duty APIs for Flex , REST, and JavaScript, GIS that’s deployable on other-than-Windows systems, free geobrowser with SDK, managed services, and on and on. Compare making a web application with ArcGIS Server’s Web ADF compared to manifold. Out of the box ArcGIS Server gives you a complete set of tools in Visual Studio, so you can drag and drop, and create slick AJAX web apps sith smooth zooms and pans and tons of tools – in other words very modern looking web applications that your users now demand because they have seen google maps and VE. Manifold IMS gives you “export to web page” that creates a wimpy ASP.net default.aspx that you then are on your own to modify and extend (at great development time expense), out of the box ending up with something that looks totally old school in the browser – how can you possibly claim that the web app you get out of the box from Manifold IMS is better and more modern than the web app you get out of the box with ArcGIS Server? It is its not remotley the same experience, its not apples to apples.
I understand that manifold thinks some of that stuff is dumb, and the corrrect answer is to have nothing but a single windows desktop application, and has built an elegant and sucessful business model to prove it. Don’t get me wrong, I like Manifold a lot, and am a big fan of Manifold IMS, Manifold spatial DBMS GIS, and the price point. But I personally don’t buy “ESRI’s falling behind in technology may be approaching an irrecoverable turning point.”
But my real point of this post – I love Dimitri’s posts, I hope he keeps them coming. I feel smarter after I read them, and feel like I get an inside glimpse into what is going on out there.
Now – I am making an appeal to ESRI -please have a super smart, high up, inside level ESRI corporate guy post here once and a while – you guys are getting totally utterly completley trash talked in this blog by another vendor. Guys from Manifold and even Microsoft post here. So why don’t you post some counter spin?
For starters, tell us what’s different about SDE technology from Manifold’s spatial DBMS approach, I am sure there must be some positive aspect.
I suppose maybe its a “big company” thing not to get sucked into such a debate – whereas, there is an incentive for a small disruptive technology company to do it. Maybe thats just how it is.
Well, I find that companies with nothing to prove tend to not post in places like this. It also takes cultish fervor for the up and comers to gain traction, which is why you see companies like Apple and Manifold in the world.
‘It also takes cultish fervor for the up and comers to gain traction, which is why you see companies like Apple and Manifold in the world.’
Er… Apple’s an up and comer?
6% of the personal computer and OS market share?….yes.
kj makes a fair point in his “spin alert” – I apologize for not clearly stating what I intended. It was especially wrong of me to lapse into a sound bite just after criticizing others for doing so. I’ll try to write at greater length to be more clear.
My argument is not that all ESRI technology is either bad or behind the times. There are many good aspects to ESRI products and their company. For example, it is indisputable that ESRI has a much broader range of industry-specific products than any other GIS company. Want something dedicated to surveyors? Better go to ESRI because you are not going to get that from Manifold.
Even in the case of general desktop GIS products there are areas where ESRI has an indisputable advantage: it is significantly easier to do elite paper cartography with ESRI than with Manifold. ESRI provides metadata and cataloging functions Manifold does not.
As far as the company is concerned while everyone has their favorite “strong arm” story to tell about an occasional pushy sales transaction, overall the story one hears from ESRI’s customers is that they appreciate the professionalism of the ESRI people with whom they work and they have a lot of respect for the company. My own experience bears that out, as in occasional contacts at industry events (most recently at Oracle’s OpenWorld) the ESRI people I have met were without fail the sort of decent and professional people anyone would like to work with. They would be a credit to any company.
But that was also the case with many extinct companies, most notably DEC. DEC, to take a famous example, was well-known both for the decency of its professional staff as well as for the exquisite quality of many of its products. But neither factor saved it from falling behind in core aspects of its business when essential products in essential ways fell behind the times.
The example of ArcGIS Server is a very good one to consider for three reasons: first, it indisuptably does have much to offer and second, it is reversing the nightmarish reputation of ArcIMS and so shows that ESRI does have the ability to recognize and fix problems and third, despite all that it misses the mark where ESRI’s survival is concerned.
Consider first that this thread is really “about” SDE and SQL Server 2008 spatial. For whatever ArcGIS Server does and however good it does it, ArcGIS Server is useless if it cannot get to data inside SQL Server 2008 GEOMETRY or GEOGRAPHY. End of story, there, as far as the Microsoft community is concerned in the non-vaporware here and now.
It is a big factor beyond SQL Server as well because usually when people spend the money to do sophisticated web applications through something like ArcGIS Server they want to do so based upon their corporate DBMS geospatial data warehouse. Setting aside SQL Server for the moment, unless you have bought into SDE you’re going to have a hard time going native against Oracle Spatial and the other DBMS’s. Collectively, there is much greater market intrest in Oracle, SQL Server, PostgreSQL and other “native” spatial DBMS than there is in the infinitesimal numbers of people using SDE. Those great numbers have a weight of influence that ArcGIS Server cannot hold back by itself.
I raise these points to illustrate how one ESRI strategy, to keep people locked into ESRI’s proprietary SDE spatial DBMS technology, works against what could be much wider uptake of ArcGIS Server, motivated by the elegance and value of that product.
“how can you possibly claim that the web app you get out of the box from Manifold IMS is better and more modern than the web app you get out of the box with ArcGIS Server? “
It all depends on what is the most important factor to the majority of customers. You are assuming there is a set of desiderata that drives a customer decision, such as the ability to run on a non-Windows server or the availability of a free geobrowser with SDK, that defines “better” or “more modern.”
In actual fact, most web applications are extremely simple ones where people value things like performance, reliability of integration and cost effectiveness. A simple, brutal strategy of tossing lots of RAM into a server and then using 64-bit Windows with 64-bit code to run many more users than could be accomplished within a 32-bit installation is quite often what webmasters really need to have. Utilizing multicore processors effectively is also highly desired.
If you can run ten times as many page views for a fraction of the cost in hardware, server room space, maintenance and the like while feeling good about being “green” at the same time, well, quite often that is the decisive definition of “better” and “more modern” as far as the customer is concerned.
It is a fairly straightforward marketing strategy as well, since while not so many people know what “GIS in the cloud with heavy duty APIs for Flex , REST” means, everyone instantly can recognize that a software package which cannot run your 64-bit computer in all 64 bits cannot possibly be as “modern” as one that can. Perhaps that’s unfair but that’s also why placing emphasis on indisputably tasty salad dressing won’t save you if you fail to deliver the steak.
So that’s the first phase of how Manifold markets the product in competition. See a typical “knock off” list at
http://www.manifold.net/info/ims_top_ten.shtml
The second phase of how Manifold markets against ArcGIS Server is to focus on the disparate nature of the various ESRI products required to get a real-life web application going. It’s not just ArcGIS Server you’ll, need, it will also be a variety of other packages and technologies like ArcView, ArcInfo, whatever they are calling ArcSDE these days and the like.
Whatever good things you can say about any one of those products and technologies individually, it’s indisputable that collectively they are a lot to get your head around and very, very expensive to buy and maintain. At the same time, using a variety of different products and technologies at different parts of their life cycles tends to make the overall usage much less reliable.
It is much more convenient for someone to have a single, highly integrated package that can be used to create in completely WYSIWIG fashion a web application that spans the desktop to whatever spatial DBMS your organization prefers.
That raises the third phase of marketing, which is that the ESRI products are not only more disparate but they often involve tying yourself to ESRI technologies instead of to vendor technologies. In mainstream markets, nobody wants to get trapped within any one GIS vendor’s silo.
You can see that in obvious ways such as getting trapped within SDE instead of using native spatial DBMS, and in less obvious ways such as the bait in the ArcGIS Server trap in the form of a proprietary “geobrowser.” Let’s see, get hooked into using ESRI’s proprietary browser instead of generic IE or Firefox? No thanks!
For all that, can one use ArcGIS Server to create web applications of great power and exquisite elegance? Yes, of course. I might be even willing to concede that someone who has invested in ESRI expertise and the ESRI product stack and is willing to commit to ESRI standards could do so more easily than creating the same application in Manifold IMS.
I’d also say the ESRI community has Manif0ld to thank for ArcGIS Server, because ESRI would never have invested the effort to improve ArcIMS had not Manifold not been so effective against ArcIMS. In fact, looking at various aspects of ArcGIS Server it almost appears that the product was designed in part with an eye to the “knock off” list Manifold was using against ArcIMS.
But if ESRI does not win the desktop or address mainstream requirements for web servers, even an indisputably respected and elite product like ArcGIS Server still won’t cut it for ESRI’s survival because users for that product are numerically too few to fend off all the newcomers that have something else in mind.
The reason is that by far most web applications are simple applications and the key thing is how rapidly and easily and cost-effectively people can get to the middle ground where all of the volume is. That’s what ends up driving technology, industry relationships and all the rest, and that’s why PCs triumphed over minicomputers.
To take the simplest comparison, a full ESRI product stack to enable a sophisticated ArcGIS Server web application costs, what? $20K? $50K? …. whatever it is, it is up there in the tens of thousands of dollars per server/application.
In contrast, a 64-bit Manifold Universal runtime costs $225, about 100 to 200 times less expensive. But despite the low cost you can hook it up to 64-bit hardware with a quad core processor and 8 to 32 GB of RAM (which is dropping down to around $15 a GB these days) and connect it up to, say, SQL Server or PostgreSQL or Oracle and utterly smash the performance of a 32-bit ArcGIS Server installation connected up to SDE. Not bad for 100 to 200 times less expensive.
The lower cost and higher performance will enable a host of applications that simply would not be economic with ArcGIS Server. I grant you that most users won’t be creating Google Earth-like things with Manifold, but if they really wanted that, well, they could use Google Earth or Microsoft Virtual Earth for free. Or, for that matter, they could embed them within the Manifold IMS application using image server modules.
The lower cost / more potential applications formula translates into better industry relationships as well. It is not lost upon Microsoft, Oracle or any of the other players that when they sow a lot of acorns in the form of much greater volume of applications with someone like Manifold they have a greater likelihood of growing a larger forest of big trees.
That extends to influential hardware vendors like NVIDIA as well. Manifold can use CUDA, so every Manifold server out there is a potential to sell NVIDIA CUDA hardware for each such server. Pretty soon it will not be possible to have IMS of any kind without using CUDA because massively-parallel computation and rendering is so much faster and, compared to the cost of anything else, the NVIDIA hardware is effectively free. Does NVIDIA get more excited at the notion of selling 100 to 200 times more units with Manifold than selling a single unit sold into a single, stratospherically expensive ArcGIS Server application? You bet they do.
But even that part of it is academic because effective usage of massively parallel architectures like CUDA would involve a lot of coding throughout many different ESRI products.
A clarification:
“tell us what’s different about SDE technology from Manifold’s spatial DBMS approach”
Manifold doesn’t really have our own spatial DBMS approach. Our approach is to use whatever is the native approach used by the spatial DBMS vendor, to utilize that within the GIS in as transparent a fashion as possible. Ideally, when you use Manifold to import/export/edit data in SQL Server 2008 GEOMETRY or GEOGRAPHY it would not be possible for a different application working with SQL Server 2008 to even know Manifold had ever been involved. We take the same approach with Oracle Spatial, DB2, PostgreSQL, etc.
So the question for SDE advocates is how SDE compares pairwise to using, say OCI direct to Oracle SDO_GEOMETRY and GEORASTERs, or to Katmai or whatever.
I have to admit that Manifold has introduced our own spatial DBMS technology that enables direct storage of Manifold geometry types within almost any DBMS, complete with spatial indices, etc., and that there is a Manifold spatial extender for SQL Server 2005. It would be fair game to suggest that we are being proprietary jerks in that regard just like SDE.
However, Manifold is extremely careful to always advise every user that the right move is to use the native spatial DBMS technology offered by the DBMS vendor. That is always preferred, if you can do it, because then you get to participate in the broader ecosystem of many applications that can work with that data. We are the first to advise users not to get trapped in anyone’s silo, not even ours. That is in our interest to do because we know darn well that, say, the SQL Server ecosystem is much larger than anything we can create no matter how good our stuff may be so we will sell more product if native spatial DBMS is used.
So why does Manifold have its own spatial DBMS technology? There are several reasons.
The first is that we wanted to provide an entry point for customers who have large data but who cannot afford a commercial spatial DBMS. The “express” versions of commercial spatial DBMS are free, but they are typically limited. Providing an entry point allows individuals and poor organizations to get started. As they grow or get more budget, they can step up (as we recommend) to the mainstream spatial DBMS vendors. DBMS overall has become so affordable or free and so easy to use that even individual hobbyists can benefit from it.
The second reason was defensive, because we were worried ESRI might try something with SDE in the SQL Server world before Microsoft could get Katmai out the door. The emergence of SDE at an early date for Oracle in our view worked against the establishment of a broader ecosystem revolving around Oracle and we did not want to see that repeated with SQL Server.
We had no way of knowing ESRI would be slow off the mark, so just in case they weren’t, we created a free spatial extender for SQL Server to deliver very, very fast spatial DBMS to SQL Server users in a way that would allow easy migration to SQL Server 2008 when it came out.
The third reason is that we feel spatial DBMS is the underpinning for high performance usage of large data, both vectors and rasters. Data is getting much larger very rapidly so the modern task is to be able to handle many gigbytes or even terabytes with instantaneous response no matter what is being done. That is obviously difficult to do, but one part of the solution is clear: it’s not going to happen within dumb file system storage. It’s going to require spatial DBMS in some form or another.
By investing directly into spatial DBMS technology we can learn how to use it well. We can rewrite the system so that instead of having one set of data access routines for spatial DBMS and another set for file system storage we can engineer the lower layers to always use spatial DBMS even if it appears to the user that ordinary files are being used. That improves performance and maintainability.
It’s also the case that if you are doing massively parallel architectures you need to think in terms of transactions as all those hundreds of processors and threads go out to hit that data. In effect, you need a supercomputer/enterprise architecture even if it is just one guy happening to do something with a very large image on his desktop. That too is best done with spatial DBMS and learning how to do it internally enables us to do a better job supporting technology like SQL Server 2008.
But for all that if you are comparing SDE to Manifold’s own spatial DBMS engines you are making a big mistake, because if you have the budget to consider SDE you should be using Oracle Spatial, SQL Server 2008 or DB2 and not get tied to any GIS vendor’s proprietary technology, not even ours.
As far as the ESRI survival thing goes, it could well be that things like ArcGIS Server are in themselves a threat to that. It is a big risk for a company when it neglects the foundation of its market to do ancillary things.
Most GIS still involves day in and day out tasks like maintaining parcel layers, figuring out where flood plains lie and other very necessary, routine tasks. Somebody who wins that market with robust, fast desktop applications that scale well from an individual user to thousands of simultaneous enterprise users working against their choice of enterprise DBMS is going to skim the profit and volume out of the market.
Winning in that market requires great attention to Microsoft standards like 64-bit Vista and direct connections to Katmai, and it also requires massive engineering to keep up with revolutionary changes in hardware like dramatically multicore processors and massively parallel computation. This isn’t science fiction, it is here today and it will be deeply mainstream in the next year or so. Anyone who didn’t start engineering for that near-term future 18 to 24 months ago is not going to have time to do what is required to be competitive when that near-term future inevitably arrives. ESRI’s comments on SDE and SQL Server 2008 indicate to me they’ve not realized 18 to 24 months ago that it is a matter of their survival to focus massive resources on being competitive in that near-term future, hence the comment about turning the corner.
I apologize once more for having lapsed into a sound bite instead of the above more comprehensive discussion.
so what paper tiger are you going to fight in coming years when ArcIMS goes away completely?
J Wallis: Because he can’t compete against the feature set of ArcGIS Server. Sure it is expensive, but it can do more than Manifold can do on the desktop. If you attack ArcGIS Server directly, you just face an Apples and Oranges situation that most folks just won’t buy.
Real GIS shops would never deploy an ArcGIS Server application that could only do what a VE site could do (unless license costs are irrelevant). To just write off ArcGIS Server because it can be expensive seems to be putting your head in the sand.
fortunately my company isn’t poor so costs are not an issue.
ArcIMS is a great piece of software, but even I see that its days are numbered. If ESRI would just give me a way to continue to write nice lean AXL and use in ArcGIS Server i’d be happy.
As a long time user of SQL Server programs I’m looking forward to the release of 2008 edition. It wasn’t to long ago that we upgraded to 2005 but even using that along side some of our newer and more complicated applications it seems like a very dated program. We’ve got several new business applications that need to be introduced into our system but we’ve made the decision to hold off on any SQL Migrations until the new version is released.