I’ve been totally swamped this week so posting has been very light. How about an open thread to keep things rolling?
The crowd is ready to attack
About
James Fee - jfee@weogeo.com
James works for WeoGeo helping people organize, share and monetize their geo-content.
Connect
Subscribe
121 Comments
What is going to happen with FWTools? I saw this message on the email list:
“I have released a new cut of FWTools for Win32 only.
http://home.gdal.org/fwtools/FWTools210.exe
My FWTools Linux system is dead, and whether I produce new FWTools cuts for linux is a bit “up in the air” currently.
I have been working hard on OSGeo4W lately, which I hope will fulfill at least part of the role currently played by FWTools. In particular, it will likely do a better job at being stable, and integrating with more packages than FWTools can. I hope to move FWTools for windows forward in such a way that it is built on packages from OSGeo4W with only a few packages I develop on (ie. GDAL, PROJ, MapServer) tracking trunk and built specially. Potentially I might even move to FWTools just being some add-on “trunk” packages for OSGeo4W and available through it’s installer.”
I had not heard about OSGeo4W at all. I wonder if this is a “good thing” for loyal FTWools users or not.
Did you see the Nokia 6210 Navigator plug on c|net? Mapping Barcelona, all cool and stuff…
http://www.news.com/2300-1041_3-6230616-5.html?tag=ne.gall.pg
KipterUh,
OSGeo4W is intended to be a cross-project win32 installer put out by OSGeo. It will likely supercede MS4W and I hope it will serve many of the people using FWTools now, but who do not really need it’s bleeding edge quality.
http://wiki.osgeo.org/index.php/OSGeo4W
I should stress that OSGeo4W is still in development, and not really ready for broad use. Though it is already possible for brave users to try it out and provide feedback. We are also looking for packagers for more foss4g components.
Frank
Why OSGeo4W over MS4W?
Or why is there a need to create OSGeo4w?
I’m looking for a new, modern GIS package for Windows Vista. Can anyone provide a suggestion?
Manifold, of course!
http://www.manifold.net
For comments from GIS people that have discovered Manifold, see
http://www.manifold.net/info/testimonials.shtml
I’m swamped too – at least I’m not using any ESRI products for my current projects
@Dimitri: TROLL!
@erin Be careful….manifold peeps do have thin-skins and do retaliate.
KoS
Interesting….is there a line down in the Colorado area? I can’t access Sean’s and Matt Ball’s blogs.
KoS
Lefty,
MS4W stands for “MapServer for Windows”. OSGeo4W is intended to grow this to providing any other OSGeo projects or non-OSGeo FOSS4G projects that desire to participate in an integrated fashion. Also, the intention is to move this to being a shared community effort rather than all the work falling on DM Solutions (well, really on Jeff and Howard).
So the name “OSGeo4W” captures the broader set of projects, it’s community orientation.
Why is HDF not available in the windows FWTools anymore?
Some days GIS make me weary. If you want to serve up maps on the internet, you have a never ending list of open source. If you want to pay the man, you’ve got ESRI. If you want to be a zealot, you have manifold. Design patterns, extreme programming, agile programming, ajax, REST, blah, blah, blah…
I want the looks of Google Earth, the speed of World Wind Java, blazing 3d, stunning imagery, complete up-to-date roads, ability to pull in a myriad of external data, open source. Show me that. Anything else and I might as well go into my way-back machine and use paper maps with 3-d glasses.
What’s the last thing you guys have seen that has blown you away?
A discouraged GIS Flunky
mdsummer,
I just checked, and the hdf4 still seems to be available in FWTools Win32. Can you be more specific about the problem you encountered?
Ah, sorry – it’s fine. Silly coincidence with my mistake.
Any thoughts on Where 2.0? Looks like a great conference to me.
Coffee good… work bad…
@RandyH
I totally hear what you’re saying. I’ve felt pretty uninspired by GIS lately as well. Probably the only thing that’s really come close to blowing me away in the GIS field somewhat recently was the first time I opened up Google Earth. Of course, after playing with it for an hour or two, I quickly realized the limitations of it (inability to perform basic overlay-type analysis, really not an appropriate front end to databases, etc., etc.)
Not Huey Long:
In my two years at Where, I felt there was a downward trend in the number of ‘hackers’ there. The first year, it was very hacker oriented from what hear, though I wasn’t there. The second year, it was about 80% hackers, 20% suits. The next year, it was 60% hackers, 40% suits. This year, I’ve observed what appears to be a continuing trend, and backed out.
However, I’m considering flying out specifically for Wherecamp.
@Jesse: Why is Google Earth “really not an appropriate front end to databases”?
Using a keyed session, one can (on Windows, at least) integrate a web-browser based application for managing the KML output, and use Google Earth to actually display the data, all without leaving the single UI (thanks to the GEarth ‘web pane’). This allows you to, for example, update your KML query to be an attribute-based query with parameters you define, and update the KML on the fly.
What part of that isn’t a decent application for interacting with databases?
Here is a fun question.
I am being forced to bring data into an ArcServer on SQL 2005 environment and combine it with services that I pull in from an ArcIMS server. I need a viewer that can view data from ArcServer and ArcIMS and also be customizable enough to allow me to develop a tracking layer that pulls from a separate table in SQL Server and displays lots of moving points on top of the other layers.
Also, it has to look good, be intuitive to use, allow feature queries, and be cost effective.
ArcMap – Obvious choice for functionality but cost is an obvious problem if more than one person needs to look at the data.
ArcExplorer – ArcIMS is a total loss in this viewer. Could it be any slower?
ADF – This isn’t cost effective if we want to serve it from it’s own, or more than one, application server. I anticipate this strategy being something I will want to leverage again and I don’t want to develop myself into an ESRI trap if I need to expand the users of the application. Also, I keep hearing bad things about it.
Open Layers – Extensions for IMS and Server? I know that the tile cache is being, or was being, developed for ArcServer but I haven’t seen it out of SVN yet.
Custom Browser App using SOAP API, ArcXML, and whatever else I want – Sound fun, but time consuming. I have most of this for ArcXML but the SOAP API looks a bit of a pain and I have heard that it is quite limited.
Pushing WMS from ArcServer and IMS and using a cool WMS application – The translators for ESRI to WMS are quite slow compared to the native services. Also, feature querying is a problem.
MapGuide OS? Others? Thoughts?
@CS: So you’re telling me that since I’m 1/2 suit and 1/2 technologist then this is a perfect conference for me? Wherecamp looks awesome and hosted at the Googleplex to boot.
Btw, thanks for the WPServer demo…I’ve shown it around quite a few times.
I don’t know if it’s a perfect conference for you: I know that it’s no longer a perfect conference for me
It might also just be a change in my perspective. During the first one, I was ‘new to the world’: it was my first conference. Since then, I’ve realized that most conferences are just places where people get together to pat themselves on the back. That’s great, but I’d much rather be out building something, not telling people who don’t want to listen how I built something.
@ Dano
Check out our product, MapDotNet Server, it can render tiles and query/insert/update/delete features in ArcGIS Server (ArcSDE), the web services are built on Windows Communication Foundation (WCF), and has a extensive ASP .NET AJAX enabled SDK for developing web apps. Best of all, it is cost effective.
Although MapDotNet Server supports server-side tile caching now, you’ll need to use the Virtual Earth API to serve up your tiles, or write your own Javascript, or extend OpenLayers, etc. However, coming soon, is MapDotNet 7.0 featuring a Windows Presentation Foundation (WPF) control, a Silverlight control and much more.
Our product site is: http://www.mapdotnet.com
Randy:| I totally agree about the current boring state of GIS. Everyone gets all excited about the “script kiddie” Web 2.0 stuff, but none of it is really all that exciting or that functional. Browser applications do have a place, but many the technical limitations for many projects makes it far too limiting. We are actually starting to see more work combing back to the desktop. Why? Because of the robust and non-limiting user experience.
Erin and KoS: “ad hominem” comments don’t advance a discussion. I gave an accurate answer to the question:
“I’m looking for a new, modern GIS package for Windows Vista. Can anyone provide a suggestion?”
If you don’t like my answer, why don’t you state why you think it is not an accurate answer? If you think there is a better alternative, why don’t you tell us about that alternative?
The question is a real, practical one, by the way, that folks who actually need to do productive work with GIS care about. Most folks whose day jobs involve GIS don’t have the time or funding to play integrator or developer with open source.
They need to buy something off the shelf that does everything they want at a reasonable price. That’s a real life requirement that should not be disrespected.
You can’t blame people for not wanting to end up having to make postings like the immediately prior one:
“I am being forced to bring data into an ArcServer on SQL 2005 environment and combine it with services that I pull in from an ArcIMS server. I need a viewer that can view data from ArcServer and ArcIMS and also be customizable enough to allow me to develop a tracking layer that pulls from a separate table in SQL Server and displays lots of moving points on top of the other layers.”
Dano must now play software developer to integrate what ESRI did not. If he had used Manifold he’d have all that available, directly connected to SQL Server (2005 or 2008, spatial or not), as a server, as an IMS or as a client. Yes, sure, he’d still have to do his application but he would not have to engineer his own infrastructure by cobbling together disparate pieces from many different sources.
That talented people can use open source or custom development to solve problems like Dano’s is impressive, but it is not hardly economically efficient compared to buying the desired capabilities off the shelf at a reasonable price, especially considering the cost to the organization of maintaining over time whatever it is that Dano manages to get going as the different Arc-thingies zig and zag through their product life cycle and the open source items drift through their iterations as well.
That is the basic flaw with a lot of this “oh yeah, sure, I can mash up what you want” loose talk, in that it fails to recognize that doing so is signing up to a lifestyle of maintaining a permanent, in-house software development operation. That has a lot of appeal, of course, to those who like to tinker with software development, but it has much less appeal for folks who think of GIS as a tool to accomplish the desired task and not the continued software development of the tools a task in itself.
That latter set is much more interested to straight answers to simple questions, like “can I buy what I need off the shelf for Vista at a reasonable cost?”
I realize that is not a politically correct question at venues like Where 2.0, but that is one reason why conferences like Where 2.0 have near-zero impact on how people are actually doing GIS around the world, because, in part, such conferences are too busy being stylish and politically correct to answer the questions that the great mass of real users are interested in having answered.
Go to any of hundreds of thousands of small towns or other jurisdictions, to name just one class of user, and you’ll find that in their daily need to maintain parcel maps, look at watersheds to deal with floods, deal with mandates from federal agencies, report to the Census intake and so on they have real needs that have to be answered with real, tangible, off-the-shelf software for Windows at a reasonable price.
Speaking of politically-incorrect observations, let me make one more by reminding folks that activities which are crushingly uneconomic have a way of disappearing over time. It is the easiest thing in the world to forget that you have to pay the piper sometimes. Las Vegas is built on the notion that people easily forget that someone has to pay for all those lights, as are built a lot of uses of open source and even things like Google Earth.
Has it escaped everyone’s attention that Google Earth is a phenomenal money-loser for Google? It is possible only because the huge revenues Google makes from ads in its core search engine business enable a subsidy that keeps Earth going. (The situation is similar to Intel, which made so much money from its core x86 monopoly that people more or less didn’t notice that it lost tons of money at just about everything else.)
Does anyone really believe that Internet search and advertising will remain so uncompetitive, so flush with endless excess revenue that companies will be able to afford to continue activities that lose them hundreds of millions of dollars? Has anyone noticed the creeping monetarization of Google Earth and similar offerings like Microsoft’s Virtual Earth?
If you make what you think is a “free” service the linchpin of your GIS work, you’ll eventually find out it is not free at all, it is a commercial product that is as commercial as, say, ArcInfo. It’s just got fewer features and less performance and requires endless mashups with your own time invested in software development if you ever want to do anything with it that significantly approaches GIS. You know, the whole re-inventing the wheel thing. And, if it continues to be crushingly uneconomic for the company that hosts it, as competition inevitably arrives you could find it is no longer around.
For that reason, I say the most effective solutions for GIS will continue to come from those companies that are winning economic rewards directly from their GIS products. Whether you choose a traditional vendor like ESRI (in deference to James’ I’ll avoid the “legacy” word ) or a newer vendor like Manifold, if you want to do GIS work today and reliably into the future that’s the most likely path to get what you need without having to make it yourself.
Dimitri: I was half joking because it wasn’t a “wall of text” that we are used to seeing from you.
Thank you for making my day by dropping the novel on us.
Dimitri:
Amen, my brother! I am not a Manifold user, but may have just become, at least in spirit.
Dimitri: What about the argument…
“I’m being forced to use Windows Vista, Windows Server 2003 or Windows XP”?
Regarding the Where Conference. I have been to all the shows since it started and i agree that it’s on a downward spiral. Last year’s session’s were un-inspiring and the outlook for this year is much more of the same. I honestly think a lot of the presentations lack credibility. Just my 2 cents.
Are we there yet dad? … when will google release libkml?
@Dano Just to comment on: ArcExplorer – ArcIMS is a total loss in this viewer. Could it be any slower? Do you mean ArcGIS Explorer or ArcExplorer? If the first, they made some significant improvements at the latest version, I’d give IMS inside it another try. If the later, thats news to me…I’ve used it as a client for quite awhile without issue.
my missus has been offered a role in Delhi – anyone know much about GIS work out there, rates of pay, etc/
Bit skeptical at the moment…
What about the argument…
“I’m being forced to use Windows Vista, Windows Server 2003 or Windows XP”?
Well, I wouldn’t say “forced” as using that word betrays a political worldview that puts one at odds with 95% of people who use computers, who are darned glad to have Windows and don’t care a lick about vendors too dumb to support Windows.
The reason that 95% of computer users choose Windows is not because someone “forces” them to, it is because they want the benefits of the Windows ecosystem, such as an endless cornucopia of software choices for every budget (or no budget at all…).
Raising the notion of operation under Windows as some sort of negative is exactly the sort of tone-deaf political correctness that marginalizes Where 2.0… they can’t bring themselves to consider the interests of a mass population they see as so unwashed as to choose Windows. Sometimes it seems a point of pride to such folks that what they create has zero applicability within Windows. I think that is self-defeating and completely unnecessary.
Let me pause to ask folks whose heads explode whenever Windows is praised to please calm down. I am, surprisingly to some, a UNIX guy and have some practical reasons to praise Windows.
Some readers of this thread may not realize that I am one of the people who made UNIX free. In the late 1980’s I worked together with Richard Wirt at Intel to make UNIX, real UNIX, free on x86 by allowing distributors like Microport and Bell Technologies to ride on Intel’s distribution license at no charge. I personally managed the distribution of over 100,000 copies of UNIX at darned near zero cost, which at the time represented widest distribution of UNIX ever deployed.
We drove the cost of UNIX on Intel-based PCs down to under $40 and at times zero, well under the prevailing cost of “free” Linux today. Intel’s strategy was to leverage the economies of scale made possible by the PC to shift the reference architectural platform from Motorola to Intel for UNIX, and thereby lessen Motorola’s presence in microprocessor markets. That was Richard’s strategy, and it worked even better than any of us hoped as the subsequent obliteration of Mot shows.
The source code that we handed out far and wide was employed by many to re-engineer AT&T’s product into early Linux distributions. It is no accident that well into the 90’s (and probably even today) there were large chunks of original UNIX source code within Linux. If it wasn’t for the wide availability of real UNIX for essentially free on 386 and similar, I do not believe we would have had Linux as rapidly as it occurred because the reference original that was so diligently copied would not have been so readily available to the folks doing the copying.
I was in charge of promoting that push for UNIX on x86 and I was so successful at it that Bill Gates personally called Andy Grove to complain that if they did not rein me in, the next day Microsoft would port all of its products to Motorola. That same day Andy assigned Ron Whittier, Intel’s worldwide VP of marketing to personally review every marketing piece and ad my group created to make sure we did not ever use the word “UNIX” in preference to a Microsoft product. [Ron was a good sport about it.]
So my personal experience is that I know a lot about UNIX, I like it, I’ve been personally responsible for proliferating it in numbers that even today would be significant and I also was right there when Microsoft surged ahead with Windows. I can assure you from being on the inside with contacts to hundreds of computer system manufacturers that the reason Windows surged ahead was very simple: Microsoft was profoundly better at responding to the needs of mainstream users than the UNIX community was. Nobody forced Windows upon the vendors (either hardware or software) – they all eagerly demanded it.
While the UNIX guys were fighting endless civil wars about Berkleyisms and SUNisms and AT&Tisms and feeling proud about cobbling up impenetrable shell scripts as a way of managing the OS and coming up with endlessly new ways of preventing developers from creating shrink-wrapped applications that could be sold in any sort of volume, Microsoft used what started as a much inferior technical core to nonetheless be far more responsive to mainstream, mass-market needs. Over time they improved their infrastructure so that it was no longer shamefully worse than UNIX and, in some cases like Visual Studio, better than what could be found in UNIX. Not that any of that was important once over 90% of humanity chose Windows for the convenience of having lots of software choices. End of story for UNIX, Windows wins.
There is a lot in that history for would-be open source GIS advocates to learn from, in particular how open source, counter-culture politics and an inability to resist elitism for the sake of elitism crippled UNIX (and later, Linux) in competition with the mainstream.
A good example is Linux, which has to my mind been an appalling waste of a generation or two of programmers in a narcissistic effort to re-engineer an operating system that was already grossly obsolete in the late 1980s, with no greater point, apparently, than substituting some private intellectual property ownership of some core parts of the model OS upon which it is based from AT&T’s ownership into Red Hat ownership. Linux now is much more expensive than UNIX was.
Source code access to UNIX was always easily obtained and effortlessly utilized to create variations, as the endless number of variant UNIX versions has illustrated since the dawn of time. That didn’t prove to be a merit for UNIX, it proved to be a trap because the resultant Balkanization of versions simply caused the UNIX effort to fracture into too many individualized efforts that could never mass together to achieve economy of scale as did Microsoft. It also resulted in a lot of wasted effort as the egos of too many coders caused a reinvention of the wheel at every turn – too many cooks spoil the broth, and all that.
You see that sort of thing again today in all these promiscuous attempts to reinvent the wheel in what passes for open source GIS. People take a cardinal failing, the Balkanization of their operating systems choices and applications infrastructure, and try to spin that failing as a source of virtues: “Ah well, gee whiz, I’m happy to say that interoperability and platform independence is a key priority for me!
” Yeah, sure, because you weren’t smart enough to simply go with Windows you now make a virtue of having to glue together the disparate mess you chose.
For that matter, a lot of what is going on in the “new” paradigm of net-centric development is nothing more than a return to time shared, centralized computing. The only real change is using the web for communications links instead of some cable to a terminal, but what is really going on is the same attempt to roll back the democratization and decentralization of computing out into individual machines and to replace that decentralization with centralized time-sharing.
And why is that happening? It’s going on because the time-shared UNIX guys who were defeated by Microsoft think that by killing the PC they can kill Microsoft. It’s no accident that Microsoft’s enemies (SUN, Oracle, IBM, etc.) have been trying to promote a “network PC” that runs nothing but a thin browser client and doesn’t even need an OS. They figure that if they lost the desktop they can still get back power if they call the shots on the server. It’s a game strategy, I’ll give them that, but it’s not workng and no one should be eager to give up the power of distributed desktops just because some Microsoft competitor wants to use you as cannon fodder.
That’s especially true of GIS usage, because GIS usage is especially bandwidth intensive and so benefits especially well from decentralization out into distributed computing within smart desktops. In fact, you shouldn’t be talking about time sharing a single server out to many users, you should be talking about tossing hundreds of processors into each desktop, all of them slaving away in parallel for the benefit and joy of a single user.
By the way, I refer to all this as “political” questions because choice of OS is not remotely a technical matter. People who are serious professionals in software development can make their application sing regardless of the OS environment.
Talk about whether Windows or Linux or Mac are “better” technically may be interesting talk, but it’s just talk as any of them are perfectly good enough as a host for whatever applications you choose to create. The decision to use one or the other is totally dominated by business issues for vendors (like coding to the largest possible single market) and by similar practical operational questions for users (like access to the broadest choice of software at the highest quality and best value).
The important thing for managers is to not let the kooky political or neurotic tics of unstable staff lead your organization down into grossly uneconomic strategies. Someone who tells you that the reason you should spend tens of thousands of dollars on legacy-ware and then spend hundreds of thousands of dollars on personnel time to stitch together “free” open source products to get a solution that is inferior to what you can get for a few hundred bucks off the shelf in Windows, all because somehow using Windows is “wrong,” well, keep that person away from any decision-making or financial authority.
So do I still use UNIX and open source myself? Sure, of course I do… I think it should be a part of any software person’s professional development, to contribute a richness of experience and fun factor as well. I’m just not going to let any political fantasies prevent me from taking advantage of the immense benefits of the Windows ecosystem.
And last but not least, there is nothing about Windows that prevents you from using open source products if that’s what you prefer and that is what is right for your organization. If anything, one reason we have so much open source is that people have realized that getting it into the Windows ecosystem is the way to proliferate it. For every guy who uses PostgreSQL or MySQL in Linux, there are probably ten who uses it in Windows. It’s just not an OS thing.
Good GIS needs valid geodatabase contents !
Dimitri, we see that Manifold.net is focused on Windows, but you are advocating using a legacy operating system. One can’t have it both ways. Microsoft’s marketshare is decreasing and maybe most of those losses are going to Mac OS X, but with companies such as Dell shipping Linux on their workstations and laptops we are nearing the same tipping point that we saw when Firefox started to push back on IE6.
I still fight every day to keep my company upgrading me to Vista and not because of ArcGIS, but because it is junk.
I don’t expect Manifold.net or ESRI to support other operating systems for their clients, but you can’t dismiss the direction of the marketplace. I know you’ve said again and again you view Manifold.net as the AOL of GIS where you get to everyone, but in doing so you abandon the marketplace to open source solutions, where the really interesting stuff is happening these days.
Wow. I guess I don’t need the Lorem Ipsum generator anymore. Next time I need some text, I’ll come here.
@cs – Where 2.0 #1 was great. #2 was ok. I think that’s all I went to.
I think we to introduce a variation of Godwin’s law for GIS discussion threads.
‘As an online GIS discussion grows, the probability of a comparison of ESRI to legacy software and being dissed by a Manifold user grows to one’
Petr – The OS marketplace may be expanding slightly in the Mac and Linux markets, but they are both, far, far away from ever reaching the overall marketshare that Microsoft offers. If you read Dimitri’s post, you will understand why.
And your comment about Vista being junk is pure nonsense. It has been one of the best operating systems I’ve used going all the way back to the early 80’s. Now I don’t suggest running the OS on older equipment, but with the new procecessors and plenty of ram, the operating system will zing. Heck, I’m even running the 64bit version of Manifold and it is pretty darn impressive.
I wouldn’t put Vista in my top 10 operating systems. It is very disappointingly even on brand new hardware. I’ve downgraded every on of my computers that has shown up to XP or XP64.
I think Petr’s point wasn’t that they’ll ever reach Microsoft, but that he feels that the interesting stuff is happening away from Vista.
I tend to agree to be honest.That isn’t fair, what I mean is interesting stuff isn’t happening on Vista. The fun stuff is happening everywhere and the need for a new OS is almost irrelevant these days. Given that most folks like me have no plans to upgrade to Vista, I’d say Microsoft has some issues to work out.Petr,
I don’t so much advocate an operating system as I simply acknowledge that Windows is the landscape that defines the vast majority (to the 95% level) of what is “computing” for the world’s computer users. It is such a dominant thing it almost does not matter what it is, which is why I don’t get heated up about things like Vista: billions of people use it, which is all that counts and at least hundreds of millions are shifting to Vista so whatever I think of it doesn’t matter. We must, and do, support it. By the way, people I know who have used Vista the most tend to like it a lot (I don’t).
I respectfully disagree that Microsoft’s share is declining, either, as the prime driver for increased Mac sales has been the use of Intel processors in Mac, so that Mac users can run Windows. If anything, that indicates increased share for Windows within the Mac community even if it is only on a part time basis.
I do agree with you, though, that to the extent any alternatives have emerged to Windows it is from Apple. I like Apple and am impressed by their work (before I joined Intel I was one of the earliest Mac developers). But anyone who knows Apple knows that Apple’s focus is a hyper-commercial, hyper-proprietary, hyper-integrated suite of applications and OS. That is why much Apple stuff is so cool: it takes advantage of hermetic Apple control to tie up loose ends. But this is not open source at work, nor, for that matter, is it a return to time-sharing with net-centric applications as the Where 2.0 crowd would have you do.
Don’t make too much of Dell shipping any Linux. They do that to avoid having to pay Microsoft royalties on boxes they ship. They know perfectly well that on the great majority of their “Linux” boxes people will install ripped-off copies of Windows.
As regards supporting other operating systems, I don’t think ESRI or Manifold or any other vendor takes anything other than a purely mercenary view of the matter: what makes the most money and gives us the greatest economic strength and meets the demands of our users is exactly what we will do. The demand overwhelmingly is for Windows, so that is what is done. If the demand were for Linux that would be done instead of Windows. Instead, I see increasing demand for Windows.
That is, of course, a self-fullfilling prophecy for Windows because now Windows has become so extensive that it is really tough for a different OS to break in. It is not just inertia, either, it is the dynamics of the high cost of failing to do a good job of supporting Windows.
The Windows market is extremely demanding, which is something that anti-Windows folks often forget. It is not a sandbox, it is by far the world’s most competitive market where in whatever niche you care to name there are many, at times thousands, of aggressive competitors fighting for every small bit of market share.
Windows users are used to being catered to. If they don’t like one vendor they have a choice of so many others that they have little patience for fools. They darn well expect that every little thing will get done perfectly, and at low cost to boot, to support their investment in the Windows ecosystem. It’s a very demanding environment.
In fact, it is so demanding that unless you shape your entire product development strategy to service Windows you simply will not be competitive against people who do. This has nothing to do with Microsoft pressure, either, it has everything to do with the relentless expectations of Windows buyers, who have been spoiled by so much competition for their favor.
There simply is no way for someone to try to sit on the fence with a multi-platform strategy and do as good a job for Windows buyers as someone who specializes exclusively in Windows operating systems. The multi-platform guy’s costs will be too much higher and his leveraging of common Windows fundamentals will be too constrained by his need for generality and his attention will be too drawn by too many other degrees of freedom in maintaining his software. So he’ll end up being late to the party with Vista support, fail to support 64-bit Windows operation, fail to support SQL Server 2008 Katmai and all that other late stuff that is intolerable to Windows buyers.
I respectfully disagree that really interesting stuff is happening these days in open source, at least I don’t see it happening in GIS except in the form of rather non-central accessories. Frankly, there are some trinkets but nothing fundamental and the implementation is often dreadful.
To me, something “really interesting” is something like applying massively parallel supercomputing processing to GIS for a mere $150, the price of an NVIDIA CUDA card. Being able to increase, say, a complex computation on a surface (aspect, slope, etc.) by a factor of 10 to 1000 in speed for a hundred and change is “really interesting”, and you find that in a form that ordinary mortals can deploy only in commercial products and only in Windows.
For that matter, things like http://www.mapdotnet.com I think are really interesting, but that too is a commercial product. The standards that mapdonet invokes are Windows standards, like Silverlight, so they can be highly beneficial to a lot of people right away.
Is Google Earth interesting? Absolutely, but a) it is a highly commercial, proprietary product that is not open source and b) it is not remotely close to GIS. [I should respond to the earlier suggestion that it is by pointing out that except for the ability to serve a highly proprietary and sloppily rendered image background there is nothing really functional about Google Earth as compared to any of a wide variety of GIS browsers and viewers. Just about any sensible GIS desktop product will be happy to stream in background images from your choice of tiled image servers if that's what you want, and in addition also provide a vast array of real GIS capabilities, all for much less than commercial access to Google Earth costs you.]
If anything, the excitement of one highly commrecial product, Google Earth, is leveraged in GIS only by other commercial products, like Manifold, which can automatically stream georeferenced tiles from image servers for real GIS work [ I trust we are all sophisticated enough here to understand that "real GIS work" is much more than drawing a limited amount of points or other data on someone else's web site]. The OpenStreetMap people get all excited about the possibility of doing that with a limited set of images from Yahoo!, but commercial products like Manifold have been doing that for a long time and from multiple image servers (WorldWind, the Earths, Yahoo, etc) for a long time.
But I’ll turn that question back to you: what do you consider “really interesting stuff” for GIS that is coming from open source solutions that has not already been done by commercial software, perhaps even at a lower cost, or, doesn’t already exist within Windows?
And, so we are talking about something new and not just an attempt to repair a poor procurement decision, let’s agree that by “really interesting stuff” we mean new stuff that has some meaning on its own. That is, I don’t think it is “really interesting” if someone has a massive problem with a hole in the ESRI product line so they are looking for an open source way to plug that hole. That would certainly be “useful” but I don’t think it is particularly new or interesting.
Dimitri:
If you think that the reason that the OpenStreetMap crowd is excited about tracing Yahoo! imagery is because of the technology, you’re insane.
OpenStreetMap has, for years, had a reasonably competent topological data editing interface, with the full ability to load external datasources of many types, including WMS, with a wonderful plugin system that allows you to write extensions for the system that extend well beyond the capabilities of most traditional GIS systems that I’ve seen. This isn’t ‘new’ or exciting, it’s just the everyday tool that OSM users use to manage their data.
The excitement about the Yahoo! imagery is not related to the technology: It’s about the agreement that deriving vector data from Yahoo! imagery allows it to remain “free” as in freedom. That’s a statement that no other large scale provider of free-as-in-beer data has made, to the best of my knowledge.
Now, I understand that this is not a crowd that takes a lot of care for free as in speech (rather than free as in beer): that’s become clear in my interactions with the community over and over. But to those of us who don’t have any budget, who don’t have access to proprietary datasets in the areas we’re working in, OpenStreetMap’s work is extremely important. It allows for a worldwide, high quality basemap built by participants, something that no other group is working to build in public. It allows for the democratization of information beyond the traditional GIS realm, into the hands of the people, someplace that it was never possible to be before.
I don’t really care if you want to live in a world where behemoth desktop machines are likely to grow more and more. I don’t mind if you want to disagree on the evaluation of every piece of software in both the Free Software and proprietary world.
But to claim that OpenStreetMap is excited about the technology that allows them to trace over commercial, proprietary data rather than the legal shared understanding that the data derived in this manner is in line with the Terms of Use of the data, and can be freely licensed as the creator desires, is simply foolishness, and to try to misunderstand the community’s excitement in this way is simply ignorance.
Does Manifold support the use of Yahoo! imagery for the deriving of topological data in a way that falls into line with the Yahoo! terms of service? I’d be interested to see that; Yahoo has stated explicitly that unless it uses one of the existing APIs — Javascript or Flash — that it’s in violation of the terms, and not allowed, and if Manifold really does have this kind of support, I’d love to understand how it is implemented. With that in mind, be careful with how hard you pat yourself on the back there; you might hurt your arm.
The OpenStreetMap project isn’t about technology, it’s about data. The excitement isn’t about technology — it’s about data. And the results of that project are entirely about data — and in my mind, they’re pretty damn incredible results… especially in comparison to everything else out there, since there is no everything else.
/manifold off
/manifold -maybe
/dimitri -off
/manifold +dimitri = null
–dimitri = manifold = true
@Christopher: You can check the following threads for more information about licensing issues in connection with Manifold and Google/Microsoft/Yahoo:
(If you got a day to spare
http://forum.manifold.net/forum/t37999.43
http://forum.manifold.net/forum/t21948.4
http://forum.manifold.net/forum/t22301.44
Petz: Thanks. This confirms my earlier statements, so far as I’m concerned: The ‘image pack’ that is being provided seems to (from what I can tell from the forums; Manifold is designed not to support me) talk directly to the URL scheme of the servers in question, which is a violation of the Terms of Use. Even if displaying these images isn’t a violation of the terms of use, large-scale derivative works based on copyrighted data to which you have an explicit contract which prohibits derivative works (Google) or commercial derivative works (Microsoft) seems shaky ground at best.
I appreciate that the code is not in Manifold itself, but it seems somewhat silly to claim that “Manifold can do this!” while simultaneously explaining to users that Manifold doesn’t do it because it’s unsupportable.
I see no reason, based on the information contained in these posts, to consider the development of derivative works based on digitizing satellite imagery via Manifold to be considered legally sound. I hope that all users who use these image packs are aware of the difficulties they may be attaching themselves to by deriving data from a base which they have a contractual requirement not to use in that way.
Well, my first comment to Christohper [sic] Schmidt would be to calm down. I can see you are heated up, but it would be a more interesting discussion if you raged at things I actually wrote as opposed to something I did not write but which you are mightily annoyed about.
Let me get back to the “reality check” request I made earlier: in good faith I asked Petr, who made a number of thoughtful and sincere comments, to provide some examples of what open source work he found “really interesting” in GIS. Petr may be busy this weekend, but perhaps Christohper [sic] could find it in himself to give an example of something “really interesting.”
I’m still waiting for specific examples, which would be productive to discuss. Some writers have commented that GIS is boring and the only really interesting things are happening in open source. If that’s the case, we shouldn’t have to wait around very long for real examples. Should be plenty of them to discuss.
Regarding OpenStreetMap, I used that as an example in a discussion involving Google, which many people inaccurately think of as open source. Specifically, I commented that commercial products like Google seem to find their greatest utilization in GIS within other commercial products and that even so, the notion of doing that is nothing new.
Let me take a moment to back Christohper [sic] out of exploding head mode and remind him that my posting responded to a technical discussion about really interesting new work. It was not about new things on the legal front.
If you read my comments with a calm mind you’ll see that I was referring to the technology involved since in the sentence prior I had discussed the utilization of Google-served imagery within commercial products and then continued with “The OpenStreetMap people get all excited about the possibility of doing that with a limited set of images from Yahoo!, but commercial products like Manifold have been doing that for a long time and from multiple image servers (WorldWind, the Earths, Yahoo, etc) for a long time.”
Your posting wrongly accuses me of somehow stating that the only thing the OpenStreetMap folks get excited about is the technology. I never wrote that: instead, I just used OSM as yet another example of where open source technology is lagging commercial vendors and so cannot serve as a “reality check” example of a case where open source is doing something “really interesting” and commercial vendors are not.
Let’s switch to what appears to have gotten Christohper [sic] all upset, a consideration of the legalisms involved in a particular usage. Think with a clearer mind, young man, because whether software is used legally or not legally has nothing to do with whether it is open source or whether it is commercial code. Either open source or commercial code can be used legally or illegally.
The legal ramifications of open source work in general as well as specific instances of open source connections to commercial products, and the relative excitement or lack thereof of open source enthusiasts when they find a path through legal landmines was not my theme.
But since you raise the issue, even though you wrongly blew a fuse about it as though I had commented on the matter, I’m a good sport and am happy to comment nonetheless. So let’s talk about the legal ramifications of open source work a bit.
First, let me be clear that the legal ramifications of open source work are not, repeat not, in any way an answer to my “reality check” request made in my earlier post. Whether or not something is “really interesting” as a new technology or product is not the same as whether or not people choose to use it legally.
OK, let’s dive into the legal thing, considering a few examples that take up Christohper’s [sic] themes, where he has repeatedly discussed Manifold features that he obviously does not understand. Let’s clear that up.
Manifold has a variety of means of connecting to image servers. Some, such as WMS client capability that can be used to connect to NASA WorldWind are built-in. Others are open doors in the sense that something like ODBC or OLE DB are open doors, in that they provide a standardized means for people to connect to pretty much whatever they want.
The image server interface, in particular, is an “open” interface that allows anyone to write their own module for whatever image server they like. Hundreds of such have been written by various people. Most users tend to gravitate to modules for Virtual Earth, Google Earth, Yahoo! and similar.
Like many technical things, such capabilities can be used either legally or illegally. That’s nothing new in GIS, since from time immemorial people have been able to use GIS packages to work with data that has been acquired legally or illegally.
Specific examples are, say, Manifold’s ability to use MapPoint as a geocoding data source. It should go without saying that the assumption is that if you have MapPoint on your computer you acquired it legally and you did not rip it off. Likewise, if you acquired MapPoint the presumption is that you are going to honor the license you have with Microsoft. You will not, for example, utilize Manifold to extract the Points of Interest data set from MapPoint and then go off and try to resell it as your own data product, no more than Microsoft expects that by licensing Visual Studio to you it will be used by you to accomplish the same illegal act.
The same goes for Manifold’s built-in ability to connect to Oracle Spatial geocoding, Microsoft Virtual Earth and the like. We assume you did not rip off your access to those products and that you will observe all relevant laws, your obligations under licensing requirements, etc.
Counting up the immensity of connections Manifold offers, the ability to connect using image server modules, geocoding servers, internet map servers, Manifold image servers, WMS, HTML, spatial DBMS and the like, there are thousands of servers out there to which one might connect. The presumption (in fact the requirement of Manifold’s license) is that you will use the capabilities of the product to do so legally, presumbly getting whatever licenses you need, paying for commercial use if that is required, and so forth.
But because Manifold’s image server interface is an open door, it will be used by open source folks to create new modules that they then post for others to use. So are you suggesting that such open source writers cannot be trusted? I expect that open source authors who choose to post their work will be doing so in good faith, and that it is understood they expect their work to be used legally without the need to insert elaborate “idiot sticker” notices to the effect that anyone who downloads their contributions should not use them to break the law. Surely no one disagrees with that.
Although it was unfair for Christohper [sic] to imply there is anything other than an expectation of legal good faith on the part of those folks who choose to use open extensions to Manifold, I can understand why some open source folks might be sensitive about such matters. Clearly, there is a lot of open source technology being misused to violate the terms and conditions of commercial vendors.
Google Earth, for example, has fairly relaxed allowances for noncommercial usage while they put on their Darth Vader face if commercial use occurs. However, there are plenty of mash ups utilizing open source bits and pieces to create a commercial application utilizing Google Earth – are you blaming the open source authors for that?
As far as your excitement that Yahoo! is allowing OpenStreetMap to utilize Yahoo! imagery to trace over to create new vector layers, well I’ve read the commentary on the OSM web sites and the discussion of the matter is pathetic. On the one hand it is so sad to see a bunch of allegedly “open” folks get so excited when a commercial vendor throws them even a highly ambiguous bone, and on the other hand it is depressing to see how yet again the open source community cannot resist being credulous fools when it comes to legalisms.
That Yahoo! personnel have had an exchange by email or have made encouraging comments in a forum or have not actively sued anyone yet does not mean that the intellectual property rights claimed by them or, [this is a big deal] by their suppliers have been waived. It is one of those titanically stupid things like the ownership of UNIX (still being litigated after all these stupid years) that can come back to bite you years later.
I’m not saying that it will stop anyone from using Yahoo! imagery within OSM applications to digitize new vector maps showing streets, but I am saying that you are far from avoiding exposure to the wrath of eager beaver lawyers of the sort routinely deployed in IP disputes. All you need is one supplier to Yahoo! who feels that the terms and conditions of their contract with Yahoo! were not honored and joined into the litigation you will be, as will be anyone who utilized that work product.
My own view is that if OpenStreetMaps had any sense of practicality applied to the legalisms of the matter you would have not failed to pursue all of the bona fide, indisputably public domain sources of vector data. For example, there is the superb VMAP Level 1 data set at 1:250,000 scale that covers much more of the world (about a third) than the current set of Yahoo! imagery, and you don’t even need to digitize it yourself.
By the way, if anyone disagrees with the above it is easy to test: simply produce a writing from Yahoo! that is signed (a real signature) from an executive who is authorized to bind Yahoo! as corporation to such representations that states you can freely digitize, that is, trace over imagery to create new vector maps, the satellite images they serve, that Yahoo! is the sole owner of such images and gives you permission to do so or that if Yahoo! is not the sole owner of such images Yahoo! has the right to grant you such permission and will indemnify you against any claims by the owner, and that you can do with such resultant digitized vector data sets whatever you want.
The above document is not something anyone can produce, and you are kidding yourself to celebrate without it. But that’s not the end of the hypocracy.
I say that when OpenStreetMap goes off on a celebratory binge about teaming up with Yahoo! under Yahoo!’s commercial constraints, on that moment you have ceased being “open” anything. All you are doing is becoming a pawn for a commercial organization – just a different form of advertising. In particular, this mincing happiness that Yahoo! allows your code and only your code to ignore the terms and conditions you say they apply to everyone else is an arrogant slap in the face of every other open source coder. It’s not “open” at all if there is some commercial enforcer calling the shots and if some other developer cannot cobble up something that they think would be better.
A truly “open” situation would be if anyone could connect to Yahoo! to fetch the same imagery you do and then allow users to trace over that imagery to create vector maps. That would allow the innovation and freedom “open” communities are supposed to represent to function. People could create better tools, tools not oppressed with elitist stupidity, tools which could trace stuff directly into useful formats, such as shapefiles or SQL Server 2008 spatial, tools which had better DBMS capabilities and all that good stuff.
But no, the merry folks at OpenStreetMap don’t want to have any of that happen. They are happy to advertise, as Christohper has done, the exclusivity of their Terms and Conditions relationship with the commercial vendor that owns and controls rights in the data they use. If that is the way it is, fine, but I caution you that once you start that dance with commercially-controlled data you must follow the chain all the way back to see who it is that holds the leash at the very end… it might not be Yahoo! but one of their suppliers in the end.
I trust that with the above part of the essay I have demonstrated that OpenStreetMap is not really “open” in the case of Yahoo! utilization, because it is controlled by a commercial vendor and that vendor’s data suppliers.
As for the other work done by OSM enthusiasts, I think that’s fine, but it’s not really “new” from a GIS perspective as folks have been creating new data sets and contributing them into the public domain ever since GIS began.
I think Dmitri should bestow his wisdom upon O.S.M. An honest discussion within the community gets you no where as it’s Guru centric. Open should mean democratic, Commons have already failed, historically, at least for the English.
hmm, how can someone type so much and yet not say anything coherent?
I thought the Yahoo! deal with OSM in using the imagery was to “validate” the maps they create? The streets they are making are from people using GPS and driving and walking around and then uploading into the system for inclusion. The point being for all of this to have a TOTALLY COMMERCIAL FREE set of maps with no license lock-ins or restrictions.
Now then…
“Google Earth, for example, has fairly relaxed allowances for noncommercial usage while they put on their Darth Vader face if commercial use occurs.”
HAH! You funny. Lets remind you of the WorldWind “Plug-in-that-shall-not-be-named”… Google was getting ready to send the lawyers after NASA because of it. And Gaia, that was shut down as soon as it crossed their radar.
I am sure Dimitri makes some interesting points, but a lot of his post is just so much hot air and just doesn’t seem to make sense… but I am sure he thanks James for these threads, as it is the only way he can get any advertising for Manifold.
Dimitri:
I’ll respond in reverse order to the order you addressed the points above, since you’ve taken the one I actually care to discuss and responded to it second.
If you believe that OpenStreetMap’s relationship with Yahoo! is special, you’re wrong. The type of behavior that OpenStreetMap is performing is one that is encouraged by Yahoo!… so long as their terms and conditions are met, and that means not accessing the imagery outside of their API. As the wiki page on the Yahoo imagery usage inside the OSM project, it is “a case of agreeing [on] an interpretation of their Terms of Use.” (Yahoo Imagery Legalities)
So, OpenStreetMap stepped up, discussed the situation, and got Yahoo!’s legal minds to agree that their Terms of Use allow for the development of derivative works under an open license, so long as the data is accessed through Yahoo!’s APIs.
You say: “A truly “open” situation would be if anyone could connect to Yahoo! to fetch the same imagery you do and then allow users to trace over that imagery to create vector maps.”
Okay, so that’s what has been done. What’s your point, again? That that ability only exists if you use the provided APIs to do it? Sure. That’s about protection for Yahoo!, and allows them to meet the requirements of their data provider — especially the ones requiring accurate attribution of the data when it is displayed. How does this make the data and access pattern not ‘open’?
Note also that unlike the other services, Yahoo! essentially only uses one provider for their aerial imagery — at least, they only credit one, and I don’t have much reason to believe otherwise at the moment — so they don’t have the dozens of different agreements to worry about that some other organizations do. There are no ‘dozens of suppliers’ to piss off — only one, who have been extremely open to users of their data for derivative works in a number of ways beyond their involvement through Yahoo! in the OSM project.
Now, as to your point about interesting open source software: I don’t give a crap about interesting software. Software is by always boring, in the same way that a hammer is boring… but a house can be pretty cool. To try and talk about exciting ’software’, Open Source or proprietary, is simply silly. The things that are exciting are the end products that come from using some set of software.
Trying to claim that Potlatch is exciting because it allows for drawing vectors over Yahoo! imagery is silly. However, pointing out that over 5000 different users have contributed to a user-drawn map using a Flash based editor in their browser, and that several hundred people are actively working on that same map every day, is pretty cool: and it’s not cool because of the technology, but because of the way in which it has been advertised and subsequently used.
Your claim was that “The OpenStreetMap people get all excited about the possibility of doing that with a limited set of images from Yahoo!”. This isn’t true. The excitement isn’t about the technology that allows OpenStreetMap to draw maps over Yahoo! imagery. It’s about the fact that people are drawing. Not just a dozen people, but hundreds every day. And going up at an increasing rate.
That’s what’s exciting about this web-based stuff: because you lower the barrier to entry to almost nothing. Not the technology. And your post is treating the OpenStreetMap excitement as if it is about the technology, which is blatantly wrong.
OSM isn’t a software project, it’s a data project. The fact that there is some software which allows that data to be built is not important: the fact that it is being built is.
Chad:
“I thought the Yahoo! deal with OSM in using the imagery was to “validate” the maps they create?”
I don’t know what ‘validate’ means in this context. The Yahoo! ‘deal’ (Which isn’t really a deal at all; just a confirmation that Yahoo!’s legal interpretation of their Terms of Use allows for unrestrictive licenses on derivative works) is just about establishing that deriving vectors from Yahoo!’s satellite imagery is allowed under the terms of service.
Also, note that OSM data is not ‘unrestricted by licensing issues’: the data is ‘copyleft’, under CC-By-SA. This isn’t public domain, and this is an important thing to be aware of. Currently, there are a number of issues with that license. Luckily, OSM is leading discussions on how to generate a copyleft-like license for geographic data, since CC-By-SA is a poor fit, but copyleft of the data itself is not necessarily a bad thing for getting more ‘free’ data into the world.
I would like to make a few observations. One, Dimitri is not very well liked here. Two: Dimitri makes a lot of sense. Three: Why is he not liked if he makes so much sense?
Let me proffer an explanation (or state the obvious): There is a significant gap between the GIS developer community and the GIS user community. As there is in every other industry. Take Hollywood. You have the moviegoers, who pay admission to get entertained, or brought to tears, or scared. You have the studio moguls, who are in it for the money. And you have the haute auteurs, who are in it for the art. It takes a skillful producer, who speaks the language of all three constituencies, to bridge the gaps and make a successful film production happen.
I think Dimitri speaks the different constituents’ languages, is trying to be a disruptive force in our industry, and in the process is getting it from all sides. Imagine a movie producer who (thought he) figured out a way to make a blockbuster at one-tenth of the current cost. He would be attacked just as fiercely from all sides.
For the record, I do not know Dimitri, never met him, and have no relationship, business or otherwise, with him or Manifold.
If you consider what Dimitri says as “making sense” then you’ve got some issues to work on. Look beyond the FUD he’s dropping here and realize what he’s really saying. I’ve given up trying to debate with him, but if you pick his side of the fence then you’ve got some soul searching to do.
Anon, its interesting that you label Dimitri with FUD. If anything, ESRI is guilty of FUD by how aggressive they get when someone is trying to go with another product.
As you know, FUD has its roots in IBM, and I’ve heard ESRI liken their use with the old phrase:
“nobody got fired for buying IBM” (replace IBM with ESRI).
Paco, who said anything about ESRI, IBM or even Manifold. Dimitri is spreading FUD about open source.
I could care less about ESRI so don’t try and deflect the discussion elsewhere. He is trying to confuse the reasons open source is powerful and why people use it. Free isn’t about cost, but the freedom to do what you want with the software.
He is falling back into old habits. How anyone can respect this man’s “opinions” on Open Source after reading this is beyond me.
@anon: ah the old GIS Monitor letter to the editor from Dimitri. Totally forgot about that one.
I followed the link to the Dimitri’s GIS Monitor letter (which is 3 years old now) and while I don’t agree with every last word, I agree with the most of it. In particular, I agree that (citing from the letter, my comments in parens):
(*) open source advocates often put their politically correct instincts ahead of their common sense (and waste time creating code that tries to work on many operating systems, at the expense of features, usability, and everything else),
(*) open source advocates are fond of stating that they have the advantage of parallelization, applying lots of people each doing a small bit here or there, but that’s not how elite software development works (and in general not how software development works, unless it is a mere replication of an existing piece of software or an implementation of an existing set of specs or standards),
(*) open source does not work at all well for applications requiring dense and sophisticated user interfaces, which by their nature require extremely close interaction on a daily basis between the many people who craft them (yes, absolutely, which is why most open source projects experience a shortage of people willing to work on the user interface and most projects try to morph into something that does not need a user interface at all, read: a cop out).
I also agree that 3 years ago there was no open source GIS software worth talking about, and while there have been some progress in this area, the magnitude of this progress is not exactly breathtaking.
To clarify where I am coming from:
I am a professional software developer. During the course of my career I have used more than a hundred of open source tools and have contributed to several open source projects. What Dimitri says corresponds with my experiences and is most definitely not FUD (as in, lies).
My thanks to “Anon” for pointing me to Dimitri’s excellent contribution to GIS Monitor (even if it is 3 years old). Living and working in the UK I was especially pleased to read his comments on the way the Government monopoly known as the Ordnance Survey locks up raw geographic data collected as tax payers expense so that it is almost impossible for UK citizens or small businesses to have access to it. Wish we had such plain speakers over here!
From following this thread, I gather this much:
Dimitri manages to antagonize lots of people, not necessarily with what he says, but with how he says it (kind of like John McCain).
Dimitri: Hire a publicist. You’ll thank me later.
Ah, the good old days, when GIS Monitor was in its prime!
Allan:
The GIS Monitor was in its prime while Adena Schutzberg was running it. Once Matteo Luccio took over, it was all downhill. Dimitri’s letter appeared at about the beginning of the downhill slide. The GIS Monitor ceased publication about a month ago.
@atanas – I was just looking at when that had happened! I was fooled because the format was still very “Adena”-ish for that issue. I agree completely. There was something magic about Adena’s GIS Monitor and that magic did not cross over to Directions or All Points. It’s really too bad.
Drat. I hope Dimitri doesn’t notice that I managed to mis-capitalize my name in my previous post….
@Randy H., Jesse, et al:
You guys need to take a holiday! As a 20-year database veteran just moving into GIS now, I have to say the whole field looks pretty exciting to me. Beats grinding out yet another 3-tier app for yet another bank/insurance company, anyway. Grass, greener, etc. Maybe when I’m a grizzled old GIS hack I’ll see things differently, who knows?
As for web 2.0 mashups etc, I understand your point, but as an outsider to GIS, it seems to me that lots of people don’t actually need much more than this. They certainly don’t need the all-singing, all-dancing, all-consuming leviathan that is ArcGIS, anyway. Whether web-mapping is the same as GIS, I don’t know, but it seems to be what lots of people want.
Just as lots of people would be happier using spreadsheets than an enterprise Oracle RDBMS. Curse them!
@Dano: Have you looked at Safe FME: http://www.safe.com/technology/FMEplatform/overview.php
As an old data-head, this looks to me like a sensible attempt to “do one thing and do it well” i.e. transform your geo-data and leave you to decide which desktop app to use etc. But like I say, I’m Mr Newbie around here and I don’t know anything much about this. Happily, there are lots of GIS people on this thread who might well be ideally placed to offer a more intelligent opinion.
Over to you, guys!
@Merry: I can’t help but feel after reading your comment that we’ve debated before on this blog. I’m sure of it the way you write, but if you feel like you must hide behind new names, go for it. James, can you tell me who it is?
Howard Butler responded very well to this old Dimitri rant. Read his response and then choose to ignore Dimitri and his open source views.
http://hobu.biz/2005/03/02/open-source-gis-has-no-merit/
Erin: I won’t comment on any persons identity.
From the hobu link above: ‘His (Dimitri) complaint about “true” GIS needing to be done on the desktop is wrong as well. MapQuest (and Google Maps) are the largest GIS applications out there in terms of numbers of users.’
Great – I can I throw out my copies of ArcGIS and Manifold and switch over to Google Maps.
If MapQuest and Google Maps are not GIS applications, then what are they Chris?
What is so insightful in the response to Dimitri’s letter on hobu.biz?
The response begins with “I don’t even know how to respond to the comment …” and reads true to that.
“Many of technical advances in GIS and computing come from academia and governments …” – care to present any numbers or otherwise clarify the word “many”?
“Manifold have done this [picked up on the above technical advances and re-sold them as features] (I highly doubt that their much touted support for PNG in their products is their own implementation).” – oh, I am pretty sure these robbers at Manifold use C runtime as well – is that, along with BSD sockets, about the complete list of advances you had in mind?
“Useability … he may have a small point, but the reasoning is all wrong. … if you take an open source project like Firefox, which has millions of users, useability and polish start to become very prominent” – yes, and of course the UI of Firefox fares better than that of most other open source projects NOT because Firefox has a large team of professional developers who are being paid for their work, which is Dimitri’s point, but rather because Firefox has a large number of users? Yes, I am sure Dimitri’s reasoning is all wrong.
And so on and so forth… Who is the one spreading the FUD here?
@James Fee:
Suppose both MapQuest and Google Maps are GIS apps. Are you ready to switch to them?
Are you trying to be funny or just a jerk?
I don’t know about you Straw, but I fire up Google Maps when I’m looking for directions rather than my desktop GIS client.
I meant are you ready to switch to MapQuest / Google Maps for your work. I’d be impressed if you were ready to do this.
I was not trying to be funny and did not mean to offend you.
Who said anything about switching to Google Maps for GIS? I only asked if Google Maps was a GIS application or not. If it is then Howard’s point is correct. If not then we’ve got some work to do on defining a GIS application.
I wouldn’t use Google Maps to perform dynseg, but I also would use a desktop client for finding directions.
“Who said anything about switching to Google Maps for GIS?”
Huh? hobu.biz. I thought we were talking about him.
Quote:
“[Dimitri's] complaint about ‘true’ GIS needing to be done on the desktop is wrong as well. MapQuest (and Google Maps) are the largest GIS applications out there in terms of numbers of users. ESRI now touts ArcServer as their biggest growing product, and everyone and their brother in the GIS software world is/has been jumping into the internet mapping space. The market is clearly going this direction, …”
He is talking about doing true GIS over the Internet. And Dimitri surely talked about that, too, not doing occassional searches for driving directions.
Straw, you are arguing a point that I can’t agree with. This isn’t an all or nothing proposition here.
Can you “do” GIS with Google Maps. Yes.
Can you “do” as much GIS with Google Maps as you can with ESRI? No.
The point Hobu was making was in his last sentence, the market place is going in that direction, unless of course you disagree that Google Maps and their like aren’t GIS applications at all.
I can understand that. But right now, is the market at the point where you can “do” all or even “most” your GIS over the Internet? No. Was it at that point 3 years ago, at the time of Dimitri’s letter? No. Will this happen in the future? Who knows. Then wasn’t Dimitri right about “true” GIS being done – at least for the time being – mostly / only on the desktop, and wasn’t hobu wrong denying that?
Why would anyone want to do all or most of your GIS over the internet? That said, this sure looks like “true” GIS to me.
http://crschmidt.net/mapping/wpserverdemo/
Oh, please, James. You seriously want me to repeat everything I said substituting that new URL you’ve got for Google Maps? OK, I’ll start: Are you ready to switch to that crshmidt thing? Now tell me how you go to crshmidt when all you need is a buffer for your favorite point location. And I will tell you that I meant switching your work. And so on.
Can you do even 50% of what you are doing day in and day out with your desktop GIS software using Google Maps and others? No. Then Dimitri’s point about the desktop still being the place where the real GIS is done is true today and was even more true 3 years ago. And hobu’s denial of that point was wrong.
Straw, pay attention.
Why would I go to the web when I can use my application to do that same thing? That is of course the core to my point that I won’t use a web based GIS.
Christopher who wrote that application isn’t a GIS analyst or even a GIS programmer. I can’t even begin to think of a better example as to who the community at large is moving into GIS and how they have no plans to buy an ESRI product (or any other “GIS Desktop” application either). The point is you can “do” GIS on the web right now and if your workflow calls for buffering or other functions, you can do it.
Hobu didn’t say that web based GIS was replacing desktop GIS at all. He said there was movement toward it. I don’t see how you can deny a movement toward web GIS today or even 3 years ago. We aren’t talking about replacing or even replicating desktop applications on the internet, but adding some GIS functionality to existing web applications where there was none before.
No where do I even foresee desktop GIS going away, but the growth in GIS will be outside the desktop in the coming years.
“True” GIS, “Real” GIS? You guys are arguing over definitions. I do 100% of my directions over the internet and 100% of my reprojecting using ESRI.
Randy:
You still do all of your reprojecting using ESRI? I’ve been doing reprojection in the browser for about 5 months now — http://openlayers.org/dev/examples/spherical-mercator.html is just one example. (The coordinate reference system of Google Maps is mercator, but the units in the position display of that map are in Latitude/Longitude.)
See also: http://proj4js.org/test.html
I am reading Hobu’s words exactly as web GIS replacing desktop GIS. I don’t know how one could read them any other way, given that Dimitri was talking about web GIS not replacing desktop GIS any time soon and Hobu’s calling that “wrong”.
I no doubt agree about the movement of GIS in some limited form to the web. I am sure Dimitri would agree about this as well, especially given that the company he works for has developed their own IMS and continues to improve it (well, for all I know they could just stole that thing from some open source project, but let’s put that aside for the moment).
And, Randy, I agree it doesn’t make sense fighting over definitions. I don’t think I did this. All I ever talk about is GIS as done by GIS professionals.
Not to get in the weeds Straw, but you didn’t say you were talking about GIS being done by GIS professionals when you went down that tangent. True GIS means many things to many folks. James IMHO is right when he shows that you can do geocoding with Google/Yahoo Maps and calls it True GIS.
re: Hobu… I think you are misreading what Hobu said. He disagreed with Dimitri’s point that you needed Windows to have a True GIS. Hobu went down the web as an example that you didn’t need Windows and all he said was Dimitri can’t ignore it. He never said that the web would replace Windows, but maybe I’m not reading the right link.
I think GIS is much wider than being a web or desktop oriented application. In fact, some applications we call GIS are not either.
Example: Airborne systems can now collect lidar data and ortho-rectified imagery together. How many steps does that take out of the geo-processing desktop chain?
Add in automatic feature extraction, and now you have an airborne platform building landscapes in 3D, draping high resolution imagery and plastering vectors on top of it. So where exactly is your chair now at your GIS desk?
Example A robot can move around a building (or neighborhood) collecting GPS data or the dimensions of a room. It can then tabulate that information and probably (not sure on this step yet, build a visualisation and export it in KML and or DWG or whatever format you need.
Automation and linking functions is the next step for GIS I think. We are way past: will that be web or desktop? Already.
If we automate between GIS functions, then we shorten the time to completion – desktop or web based. If the time is shortened, the cost is decreased. The probability of doing more, through automation rises.
We are going to be swamped in data. But, do we know how to use it any better – yet?
Heated debate over what can be boiled down to semantics, resistance to change and advertising.
Chris S. makes an excellent point that GIS (I’ll define this in a minute) is not interesting– rather what you make with it becomes interesting. Shiny tools are nice, and sell like hotcakes, but the fact that they are shiny does not bestow quality or confidence in them. Open source approaches to GIS are an insurance policy on the integrity of the algorithms used in that software– and this should be one of the primary arguments in favor of open source GIS.
There is a already a large movement in the statistics community toward open source implementation of algorithms (R). Emphasis on data integrity and transparency are core drivers in this movement. Accordingly, the R project is lead and supported by some of the most renowned names in statistics.
A similar trend can be expected from the GIS crowd, despite the groans from some of the more dimwitted Arc/Manifold-tards (to use the modern jargon) and associated commercial machinery.
I don’t have any problems with people making money selling software– I have problems with people selling complex software that is used to shape public policy and practice, when the underlying algorithms are not open for inspection.
In the last 5 years I have seen FOSS GIS applications make huge steps in terms of capabilities/user experience. Mapserver, GDAL, PostGIS, GMT, GRASS, Proj — these tools have solid foundations and active development communities, which is not something to be taken lightly.
probly enough for now
Cut and pasted Dimitri’s posts into Word just to see how long it would take to read his responses. Came out to 13 pages of text!
If someone that senior at Manifold has that much time to write on a GIS blog in multiple posts – I don’t think they are that busy at the office.
Where did they fare in the Daratech report this year?
Point taken. Perhaps I’m seeing in your words something that is not there.
@Dimitri
Regarding opensource GIS projects which live up to your expectations. Have you recently tried:
GRASS PostGIS R Mapserver GDAL Proj GMT QGIS
Based on many of your remarks over the years (esp. your widely cited letter to the editor) it is very clear that you have not worked with these products– or if you have, it hasn’t been since the 80’s (GRASS is a raster-based GIS ?).
As a soil scientist I use these tools on a daily basis for real analysis/research. I am not talking about the silly little buffer/intersection -or- “lets make a slope map” examples that everyone does in their Arc/Manifold training classes.
Complex landform classification, route planning in rugged terrain, prediction of soil properties, quantification of microclimate — along with appropriate measures of uncertainty… etc. These type of things are (surprise) accomplished by researchers all over the world on a daily basis with open source software. In fact, most of the “features” found in manifold/Arc* were probably derived from algorithms developed by researchers at public institutions.
Theses packages are also available on Windows.
These packages also come with an excellent support/developer community.
Next time, before you write a 5000 word tirade against open source GIS applications, try some of these applications for a change.
I go away on a long, long weekend…too much catchup.
Dimitri….my comments weren’t directed at you. I don’t have a quibble with your answer, one way or another. But thanks for proving my point.
Might want to take Atanas’s advice.
KoS
Dylan,
First, let me thank you for providing specific examples. That gives us something concret to discuss. Some comments:
“Regarding opensource GIS projects which live up to your expectations. “
Well, they are not my expectations that matter – it is the expectations of mainstream users of GIS, the sort of people who actually do GIS every day such as in the scenario I ventured. That’s the audience that must be pleased.
By “GIS” I mean the expectations of that audience, which in large part means packages like ArcView, ArcInfo, the ArcMap suite, MapInfo, Manifold, Intergraph and the like.
Of the list you offer there is only one package that is at all “GIS” in the sense that the above packages are GIS and that is GRASS. The others are something else or (QGIS) depend upon GRASS for core functionality. Let’s touch base on each of them before returning to GRASS:
PostGIS – This is a spatial extension to PostgreSQL. A more accurate way to refer to it would be as a “spatial DBMS” package, the sort of accessory that is often used together with a GIS package. PostgreSQL/PostGIS is a good example of my thesis, that open source tends to be OK for black box sorts of applications (that is, command line or API applications) but not rich GUIs. But note that I don’t go around stating that “there is lots of commercial GIS” around by citing DB2 with Spatial Extender, Oracle Spatial, Microsoft SQL Server “Katmai” and the like, nor do I cite other accessories like Access or Excel even though these are routinely used, at times very tightly so, together with GIS.
(I am, by the way, personally familiar with PostgreSQL/PostGIS because Manifold is the first and I believe still only commercial GIS that directly connects to PostgreSQL/PostGIS and can directly and dynamically utilize to read/write/edit spatial data within PostgreSQL/PostGIS.)
R – This is not GIS – it is a statistics package. To quote the R page: “R is a language and environment for statistical computing and graphics. ” It is simply another bit of accessory software that is often used with GIS, but to call it “GIS” is as inappropriate as, say, an ArcView person referring to SPSS or SAS statistics as “GIS”.
Mapserver – From the Minnesota Mapserver’s web site: “MapServer is not a full-featured GIS system, nor does it aspire to be. Instead, MapServer excels at rendering spatial data (maps, images, and vector data) for the web.” I happen to like Mapserver (it has improved a lot over the last few years), but it is only one part of what folks would regard as a GIS suite of software and, at that, it is really just a publication and rendering engine. No shame there, but it is only part of the puzzle. You still need a real GIS.
By the way, Mapserver is a great example of what is good and what is not so good with open source. It is primarily a black box publishing appliance, and in that respect plays to the strengths of open source as I wrote in my essay three years ago. But it is classically open source in a way that discourages uptake by the mainstream: for example, the Windows distribution focuses on non-Windows usages (Apache and PHP instead of IIS and ASP or ASP.NET). Even with Jeff McKenna’s generous contribution of MS4W it is a challenge for ordinary people, even web masters to configure and deploy in large volume in the mainstream.
It should also be mentioned that the modern trend is for integration: it is a lot easier and costs vastly less personnel time when your IMS engine is the same as your enterprise and desktop GIS engine. The cost of time to integrate disparate items to cobble up your own solution is very high, so high that it makes sense to pay for a pre-integrated commercial solution.
GDAL – This, too, is not a GIS package. From the GDAL home page: “GDAL is a translator library for raster geospatial data formats” – it is a highly technical tool that, once again, has no significant GUI except for some command line interfaces. Setting aside GDAL’s uselessness for the primary GIS data used these days, vector data, it is not remotely the sort of thing that the average GIS user employs.
Note that my request was not for examples of programming tools. At some level of command line processing, you are really entering the world of scripting and programming more than you are of interactive GIS packages like the ArcGIS suite, Intergraph, MapInfo or Manifold. It is a classic example of a open source blind spot to cite such a thing as “GIS,” somewhat akin to the way UNIX enthusiasts kept citing various shell scripts as a legitimate UI for the masses as compared to Windows!
I don’t want to beat up on GDAL, because it is very clear what sort of black box it aspires to be and it is an outstanding example of doing well what it sets out to do. But as good an answer as it might be to the question “are there any open source black box packages that are useful to talented technical people interested in stringing together the use of open source packages?” that is not the question I posed.
Proj – This is a programming library of projection source code. It is not remotely close to a GIS package as used in the scenario I outlined. You are not seriously suggesting that someone in a county GIS department is going to sit down with Visual Studio and code up their own reprojection application to re-project one drawing into another?
GMT – A collection of around 60 command-line UNIX tools dating to 1988. This would be a good example of the barren user interface and extraordinary difficulty of using such open source stuff for GIS.
QGIS – OK, a real GIS!
However, for much functionality QGIS depends on upon constituent contributions from things like GRASS, so it is more raster oriented than vector for analytics.
The problem with QGIS is that it is very barren of capabilities, has poor support for Windows (32-bit only, for example), and evolves at a glacial pace with but a handful of changes in years. Even if your expectations are no greater than MapInfo or ArcView, I don’t think you’d be able to use it in a production scenario like I outlined. I don’t think it remotely compares to the breadth, depth and meticulous completion and integration of feature sets in something like Manifold. In that respect, it shows how even a reasonably expert, informed, savvy and determined effort to put together a GUI-intensive open source equivalent to commercial GIS packages ends up being very far behind the times and barren in comparison.
GRASS – Everybody cites GRASS when talk of open source GIS comes up, and it is indisputably true that GRASS is a classic example of “real GIS.” But it is also the classic example cited by people who point out the limitations of open-source GIS as something that does not measure up to the expectations of the mainstream.
One of the biggest flaws in GRASS is that it has been a raster package and utterly dismal at vectors. That is changing but is evolving much more slowly than the classic commercial GIS packages (which have long focussed on vector work) are improving at raster operations. Given that mainstream GIS work is overwhelmingly vector, GRASS has never been able to gain popularity with most GIS users.
A further problem is the mind-numbing technicality of doing anything with GRASS. It is a classic example of open source being unable to pull together a sensible, user-friendly GUI. If you have the formidable technical skills to, in essence, program your way through use of GRASS vector modules and the like, well, sure you can get it to do many neat things. But all that assures that only a handful of folks will be able to use it.
I wouldn’t disagree at all that someone with a raster focus might use GRASS, or that recent vector additions to GRASS might not make it possible for a GRASS user who is highly technical to take on the classic vector work normally done with packages like the ArcGIS suite, MapInfo, Manifold or Intergraph. But I don’t remotely believe for an instant that, say, county governments around the US could dump their ESRI environments and do all they are doing today using GRASS with the personnel they have today.
Look, I appreciate the list but if anything, the list you provided just underlines my point that open source GIS to date is a classic example of open source projects not succeeding very well where intense and dense GUI development is required but are best suited for black box appliance applications. GRASS in particular is a nearly perfect example of a set of black box capabilities stitched together by a quasi-GUI that is almost unfathomable to mainstream GIS users.
It is very much like UNIX all over again: some very talented people implementing great ideas in a way that is inconvenient for the masses. I don’t think that is a waste, because it provides encouragement and fertile ground for people who will see those great ideas and ultimately implement them as commercial products for the mainstream.
But, I think it would be great of some of that energy were diverted to assuring that the open source items you’ve enumerated be better integrated with existing commercial, mainstream settings. That should begin with Windows and then continue on to your favorite commercial GIS package. The commercial guys are not at all shy about leveraging open source (Manifold, for example, rather aggressively supports PostgreSQL/PostGIS) if it helps out their users, so it would be great if the open source folks would relax a bit, let their hair down, come to Vegas and party a bit with the commercial applications as well.
Sigh
Dimitri, GDAL does work with vector data via its OGR packages. It has buffering, union, difference, reprojecting, and more. Yes, it is a programming tool, but it does work with vector data. Also, a free-trail for Manifold would help some of us better evaluate its GIS capabilities.
Since this is an Open Thread, I’ll point out that I went to ESRI’s site today looking for directions to the Federal UC. I clicked a “maps” link expecting to see a nice Google Map or Microsoft Map with a pushpin marking the location of the UC.
Instead I get this: http://www.esri.com/events/feduc/map.html Click one of the map links on that page
How the heck can ESRI have this quality map on their website? I’m still looking for the site that makes me want to develop using AGS.
Wow Dimitry, I think you might just win the prize for that last one. Again, your comments reveal just how little you know about any of the applications I listed. I’ll keep my reply short, as I am not payed to advertise.
Some definitions:
GIS: any tool related to the management / analysis / summary / visualization of geographic data. Simple, but good enough for an operational def.
black-box: this is something that performs an operation in such a way that the user has no means of knowing what is going on. all you have is an input and an output. a good example interpolation in Manifold via kriging! nice one. I wouldn’t trust a non-specialist to use this technique, let alone suggest that a “surface via kriging” is just a point and click away!
Open source GIS applications are by definition not black-boxes. (BTW the command windows on windoz is sometimes black, and the window is rectangular– but this is not what “black-box” means). If I am unhappy with / concerned about / interested iin the algorithms employed in say, interpolation by regularized splines with tension in GRASS, I can look it up, and ask the author / read their paper on it. How exactly would I investigate the algorithms in commercial products?
Now onto the packages.
PostGIS: yes, a spatial extension to PostgreSQL– looks like you read the front page, have you seen the manual? It does a lot more than buffers. In fact, it does just about every vector operation (albeit using the simple features model) that one might need. No GUI? No problem.
R: have you ever used this, or seen this comprehensive list of spatial packages? I suppose not. Yes, R is often used for statistics. You can also perform statistics on geographic data (who knew?). If you spend a couple minutes with it, you can even make maps.
Mapserver: data presentation (need I say more?), data management (image server via WMS)… Not a “complete GIS application”– but a nice component in one (I’ll get to that in a bit). I guess you also forgot about all of the languages other than PHP which have native bindings in Mapserver (C#, Java, Python, Perl, …).
GMT: I guess you haven’t used this. Look at the cover of any AGU publication for some examples of what people do with it. sure the syntax is not for everyone– but it is one hell of an automated or even semi-automated tool.
And then GRASS: abysmal vector support? when was the last time you have used GRASS– honestly ? I’ll agree that the vector model (topologically correct) is difficult for some to use at first, but c’mon it isn’t that hard. Yes the GUI still has a ways to go, but there has been very significant improvements in the underlying GUI toolkit. GRASS won’t work as a drop-in replacement for most people, that is true– but you are forgetting what GRASS strives to be; a dedicated geographic/image analysis engine.
I guess GRASS can be difficult if you can’t read, or don’t choose to read instructions. For those that like clicking their way through life GRASS might be too much trouble. Here is one especially difficult procedure in GRASS. 6 lines of “scary code” and you have a nice classification done for you.
Its no wonder that these type of operations scare people– after all typing in commands makes you think slowly about what you are doing, and is a self-documenting approach to transparent work-flow.
By failing to realize that I gave that laundry list of application as a demonstration of small , highly efficient GIS tools your famous affiliation with UNIX should be called into question. Really. Surely you must be familiar with this approach to problem solving.
Its like this. If I want to process text I would use: bash, awk, sed, cut, paste, grep, sort, … etc.
If I want to work with a complex set of geographic data I would use the list of applications from my original post. The point is this– the collection of open source “GIS” applications are not drop-in-replacements for monolithic corporate types. However, combined they form a fully-function set of GIS tools. Note that they are not toys, and can be used to accomplish the same type of end results as one might expect with a COTs GIS.
Hey Dylan, sounds like you’ve got yourself a mighty fine GIS system there. Good for you that you’re able to do serious analysis and predictive modeling without having to shell out megabucks. Good for you that you’ve got a system that you can push and pull to answer your needs without having to wait for a vendor to add the functionality you need.
Wait on a minute. None of those components are a GIS in themselves. Dylan YOU DON’T HAVE A GIS! Just look at how Dimitri framed the question. It must be integrated. I’m sure to you it looks like you’ve managed to integrate all the parts nicely, but /it hasn’t been integrated into a monolithic package by a vendor/, so it doesn’t count! Dimitri’s right. We must stick with the mainstream and only accept change as delivered from on high, rather than being instigators of it. Oops, there goes upstart Manifold! Never mind, the whole world should accept the status quo and you should accept your place in it. It doesn’t matter if you get better bang for your buck by using tools outside the accepted order, you should stick with the mainstream (oops, there goes Manifold again). Fancy wanting do things a little differently so as to better answer your own needs and values. How outrageous!
Strange, I posted a follow-up to #95, but it seems to have been dropped by Wordpress.
The text from that reply is posted here.
dylan, I’ve followed the link (shame it got dropped as your response was honest and on point) but I think the following two quotes encapsulate the direction you take in your response:
“No GUI? No problem. “
[Every real-life GIS user on the planet sinks his or her head and sighs...]
and
“By failing to realize that I gave that laundry list of application as a demonstration of small , highly efficient GIS tools your famous affiliation with UNIX should be called into question. Really. Surely you must be familiar with this approach to problem solving. Its like this. If I want to process text I would use: bash, awk, sed, cut, paste, grep, sort, … etc. If I want to work with a complex set of geographic data I would use the list of applications from my original post. The point is this– the collection of open source “GIS” applications are not drop-in-replacements for the dumbed-down monolithic beasts that are currently in fashion. However, combined they form a fully-functional set of GIS tools. Note that they are not toys, and can be used to accomplish the same type of end results as one might expect with a COTs GIS. “
Well, yeah, sure I get the software tools thing and yeah sure back in the day I could awk and grep with the best of them. My point is that in modern times this sort of thing is exactly what most folks doing GIS want to avoid. That’s why I asked for examples of integrated open source tools that could be dropped into a real-life GIS environment.
I understand you are not shy about stating your approach is different, given your comment:
“The point is this– the collection of open source “GIS” applications are not drop-in-replacements for the dumbed-down monolithic beasts that are currently in fashion.”
We are in total agreement they are not replacements (that was my thesis, so I’m glad you agree), although I am dismayed by what might be disdain for the fashion of the day, your suggestion that it is dumbed-down and your implication that the integration I praise makes for a “monolithic beast”
Making something convenient and integrated and brought to life with a meticulous GUI is not the same as dumbing it down.
Read the spatial SQL chapters in Manifold, the ability to do elaborate computations on surfaces with NVIDIA CUDA, the implementation of Active Columns, support for Iron Python and numerous other .NET and ActiveX languages, a built-in debugger, user interface scripting, Boolean selection modes and much else. None of that is dumbed-down for retards. Read the threads on use of spatial SQL in the various forums and and you’ll see it is not being used by retards, either. Anyone familiar with the ESRI line could easily cite numerous examples of ArcStuff that is not remotely “dumbed down” either. That goes equally well for Intergraph and other serious packages. These are very serious packages developed by very serious developers working nonstop for many years to serve the requirements of sophisticated, expert users.
Quite a few of those folks are also more than expert enough to know the software tools approach quite well, to be able to weigh its advantages and disadvantages from the viewpoint of great experience, and have decided that they strongly prefer an integrated commercial package for a variety of well-known reasons.
But, for that matter, there is a place for “dumbed down” as well. MapPoint is often referred to as a “dumbed down” application [as if Google Earth is not!
], which is of course true and also equally of course part of the great charm and success and joy of MapPoint. It makes a tiny bit of GIS accessible to the masses. It is far from easy to produce a dumbed-down, easy to use package that works as well as MapPoint for so many beginning users.
Regarding “black box,” well, I apologize for not being clear, but I guess that is the price one pays for brevity. I meant that in terms of the user interface in the sense that the module is an appliance and not an elaborate interactive GUI.
Truly elite, fast GUIs are incredibly difficult to accomplish. They are far more difficult than algorithms or a collection of clever, small modules that utilize simple interfaces between each other. What people who try to do them in GIS soon realize is that elite GUIs in GIS are especially difficult because they usually involve exceptionally much computation on the fly with enormous amounts of data in play. So it is not easy to do the very, very many things a sophisticated GUI must do at the same time as doing all those things in real time fast enough to assure snappy performance.
I wish everyone well with the software tools approach if that works for you. It is just that I also would encourage you to think about building more bridges from those tools into the commercial world as well.
Dylan, Tim:
I think integration and lack of tension between different parts of the system along with full-featuredness is exactly what Dimitri is talking about. Noone denies you can do a good portion of what you would normally do with ESRI products or Manifold using a set of existing open source tools. The point is, this requires far more effort on the side of the user who has to connect various pieces of the open source puzzle together. At times, the magnitude of the effort that has to be spent connecting the pieces together rivals that of the effort that has to be spent doing the actual work, although this is usually discovered post-mortem.
Dylan, you talk about the UNIX way and say that if you want to process text you would use bash, awk, sed and other tools. I can do it this way too but I prefer using Microsoft Word. I know I am not alone. Similarly, where you would use the open source tools you mention, other people would use Manifold. However, while there are open source equivalents to Microsoft Word, there is no open source equivalent to Manifold. Which is Dimitri’s point.
Regarding “What qualifies as GIS?”…
If individual components of GIS don’t count as “real GIS”, then I’d argue that no one is “doing GIS”, because no one is ever using every component at once.
But I don’t “do GIS” any more than I “do computers”… although I might do various forms of spatial analysis, or make a map, or get driving directions from Google Maps (or does calculating an optimal route between two points within a road network not count as “GIS”, just because it has a nice easy-to-use web-accessible GUI interface?)
And as for the most interesting things happening in the world of GIS: I’d say OpenLayers (amazing browser-based integration of multiple data sources… I can envision OL becoming the GUI to a network of data sources and web processing services), Manifold (really, a great software package, in spite of the hype)
By the way, I would point out that the reaction to Dimitri’s words by someone named “Fake SteveC” linked at the top of Dylan’s post is a typical example of pompous content-free blabber seen from people who really have nothing to say. True to his type, “Fake SteveC” does not respond to any of the points made by Dimitri, presumably being overwhelmed by the number of holes he could “pick”. By the end of his piece “Fake SteveC” manages to recover enough to throw some juicy personal insults Dimitri’s way, and invite other hotheads to come and share in the fun by doing the same. All in all, very in line with the “insightful” response by hobu, discussed earlier in the thread, although less literate.
Straw: hate to break it to you, but Fake SteveC is a spoof.
But then, I’m starting to suspect that Dimitri is, too.
@Dimitri:
I would take you more seriously if you actually posted prof of your “facts”. As it stands now, they are nothing but someones bloated opinion and are not to be taken seriously.
Also, I would believe you more if you actually tried all these tools which you say are so sub par and “following” commercial apps. I think you looked at the web pages, read a little and then formed an opinion which you have since transformed into the “Holy Truth!”.
@Straw: Miss sarcasm and humor much?
For the work I do, Open source tools fill the need I have. I don’t need ESRI or Manifold. Why should I shell out money when I can do the same for free? My need is to process, orthorectify and combine imagery and serve it to a client (WorldWind). I can do this with several command line tools and a batch file.
Is there a full-blown open source GIS – No. Grass is the only package that closely represents this, but, It is not intuitive and does lack the hands on approach required by most and presented through a usable GUI. I think we all agree that if we are to do detailed spatial analysis we need to purchase a product. I have no arguments about that.
However, there are situations where we have all turned to a open source product to do a GIs operation. As an example, I needed to clip +-2000 images recently and create a vector grid defining their location and attributes. GDAL did this for me with little difficulty and fast. Did I need a commercial product – no. Would a commercial product have done it better – no. The same is true for postgis, which incidently has addons for geocoding and routing, something very topical in GIS today. The list does go on.
What I really wanted to focus on however, is what are the users after, and you may see that is where the open source market is focusing. We are all aware of the GIS triangle, which outlines doers, users and viewers. Doers are at the top and represent a small percentage of the triangle, while viewers represent the majority of people who interact with GIS. Open source “GIS” is heavily weighted in this sector. Currently there are excellent commercial grade products that do just that – enable users to interact with GIS: Openlayers, MapServer, Geoserver, Featureserver,KaMap, Qgis, Udig, Mapwindow (with a highly extensible API to program advanced analytical functions).
let me focus on another real world example. Doing spatial analysis to improve development, service delivery etc is pointless unless you enable the user to interact with the results. How do you do this? You can prepare maps and tables, but that is not representative of where we are in GIS. the best way is to present the user with a Spatial data viewer to analyse and view the data. Sure, you can deploy using a custome built application, but then you need to consider the runtime license. Arcexplorer is also an option, but it does not cut it compared with udig, Qgis or Accuglobe (free, but not OS).
So, the point I am trying to get across is, focus on what OS is doing for GIS, don’t dismiss it because it cannot do complex analysis through a GUI interface. We must all understand that commercial and OS need to operate in parallel not in competition. Is Openlayers going to take over the market share – definately not, but, if more maps get out there (and professional commercial maps already exist) – then thats more focus on GIS and more work for you and me. I can vote for that anyday!
Very appropriate for Andrew to point out the distinction between the different tiers of the GIS pyramid, or GIS triangle, as he calls it.
For GIS to be successful and to have high adoption rates with end users, its operation has to be as simple and as reliable as that of an ATM. The colors have to be right. Yes, the interface matters a great deal. The interface matters a great deal!
To pick up on Andrew’s omments and respond to Chad:
Chad says, “For the work I do, Open source tools fill the need I have. I don’t need ESRI or Manifold. Why should I shell out money when I can do the same for free? My need is to process, orthorectify and combine imagery and serve it to a client (WorldWind). I can do this with several command line tools and a batch file.”
That’s great and no one has any problems with that. Using open source vs. commercial products is not an either/or situation. If the above works for you and you have the talent to do such things and then maintain them and your cost of labor for integration and so forth is sufficiently low that you see the experience as “free,” well by all means do it. But your combination of needs and willingness and ability is very rare.
My experience is that very few people either are able or are willing to operate their daily GIS lives exclusively using open source. The great majority, perhaps to the 99% level of working GIS folks, will base their operations upon some commercial system and only occasionally reach for an open source tool such as GDAL.
Let’s go back and see how this thread got going. It started heating up when Righty asked “I’m looking for a new, modern GIS package for Windows Vista. Can anyone provide a suggestion?” and I responded with a couple of URLs pointing to a commercial product, in this case Manifold. [If anyone can cite some other new, modern GIS package for Windows Vista, shame on you for not responding to Righty's post!
]
After one exploding head over the mention of a commercial product, there emerged a sincere set of comments to the effect that GIS is boring and the really interesting stuff is going on in open source. I disagreed and I asked for examples, giving my own counter-examples. And then we explored a variety of ratholes (to use an Intel term).
Now, my disagreement with those comments does not in any way mean I am against open source. I happen to like open source, I work in its favor and I think it has a lot to offer. But ultimately all I care about is getting the most GIS possible out to the most people at the lowest possible cost with the greatest possible quality and capabilities. I really believe that GIS can have a transformative, positive effect on the world. I can’t think of a major problem confronting the world that would not benefit from sensible, affordable, usable, effective and widespread geospatial tools.
I’d be the first to reach for open source solutions if I thought they had a chance at helping that objective. It turns out they don’t, at least not as effectively as creating a new generation of commercial GIS similar to the revolutionarily transformative effect of mass market personal computing. But no doubt open source tools can and will play a role, part of the pyramid Andrew describes.
The only folks making this an “either/or” proposition are some open source folks, who seem to have a visceral, reflexive antagonism to anything having to do with Microsoft or with commercial products, no matter how much utility or value they offer.
I know plenty of people working with commercial GIS products and I can’t think of any of them who would hesitate to pull an open source arrow out of their quiver if it was the right tool for the job. Conversely, I also know plenty of open source folks and the great majority of them (not all, to be sure, but a great majority nonetheless) seem to have nothing short of an incendiery reaction when it is suggested they might benefit from using a commercial product. It is an astonishing cultural divide.
PostgreSQL/PostGIS is a good example. Commercial people don’t hesitate to use it in conjunction with their GIS, but for some reason the majority of open source folks seem willing to spend a lifetime cobbling up their own front end rather than consider a commercial product, even if that commercial product is here today, elegantly superior in technology to what might be cobbled up, and nearly free to boot.
For that matter, I believe the non-negotiable antipathy towards Microsoft has led some open source developers into poor technical choices. It’s clear to all that a lot of enthusiasm for web applications arises from a competitive hope that by diminishing the role of the PC it will be possible to diminish the influence of Microsoft. But that leads to an over-emphasis on web centralization when at times distributed computing onto the desktop is the more effective technical plan.
In my critique of open source I have enumerated other factors that I think are holding it back, namely sociological and political factors beyond a cultural conviction that Microsoft is Satan. I have cited a sense of elitism which says that folks who prefer commercial software are stupid, I have cited a political element such as a built-in assumption with some open source advocates that capitalism is evil, and so forth. I do believe these are real factors that are holding back the proliferation of open source. I think if you are a real friend of open source you will acknowledge this sort of thing is going on and that it does not help the cause.
Last but not least, as flattered as I am to be the subject of a spoof, it would be immodest of me not to point out that I am not a CEO but simply a very low level staffer. All the same I am grateful to everyone who helps get my writings elevated to higher standings in the search engines.
Hmm.. I think xkcd has been watching this thread..
http://xkcd.com/386/
This is sort of a funny thread. This is not a discussion about commercial vs open source software. This is a discussion about GUI based apps versus command line tools. At my workplace, sadly, we do not have access to Manifold. We do have ArcGIS. As a result, I spend 90% of my time writing AML for Arc Workstation. Every once in a while I check back to see if ArcGIS has analytical capabilities yet. The rest of my time is spent with GRASS, OGR, PostGIS, and FME. Too bad that by some definitions, none of what I do is GIS
Dimitri’s Law: The value of a thread is inversely proportional to the cumulative length of Dimitri’s posts.
May be invoked in any instance of someone posting three or more posts to a thread where each of their posts is greater in length than the average length of all other posts.
@ Chad: A permanent url for xkcd: http://imgs.xkcd.com/comics/duty_calls.png
@David:
“This is a discussion about GUI based apps versus command line tools.”
Not exactly. This is a discussion about the benefits of using tools designed to work together compared to using tools which are “at it” on their own. You said you are spending a lot of your time with GRASS, OGR, PostGIS, and FME. If you describe a typical work scenario involving these tools, we could try coming up with opportunities for improvement which are or could be offered by integrated solutions.
You are right that the words “open source” are in the discussion only because it just so happens that the proponents of using the likes of bash, awk and sed instead of, say, Word for editing text documents, as well as the proponents of using sets of disparate GIS tools instead of integrated suites for doing GIS, also overwhelmingly prefer using free or open source tools to using commercial tools.
As to Tim’s missive, of course, we all know that the value of a thread is directly proportional to the number of posts with links to XKCD. Whether the posts have any thoughts in them is unimportant.
@Straw: Doesn’t have quite the same impact.
A short comment…
“Not exactly. This is a discussion about the benefits of using tools designed to work together compared to using tools which are “at it” on their own.”
I agree with the overall sentiment in that post, but would also like to remind everyone that just because a tool integrates many functions does not mean it cannot also work together with a very wide variety of other tools as well.
In fact, one of the points of creating software for Microsoft ecosystems is the opportunity to create a package that will work together with huge numbers of other tools. People often choose Windows exactly because they value a huge ecosystem within which many different applications can work together, so financial incentives within Windows markets strongly motivate Windows software developers to enable a host of connections to other software using Microsoft standards.
In the world of GIS, any sensible Windows GIS package will do the same. Manifold, for example, can work with just about any DBMS you’ve ever heard of for both regular attribute storage as well as true spatial work utilizing spatial geometry DBMS types and spatial indices within the DBMS. The package actually ships (in the DVD distribution) with four DBMS installations available for users: Express versions of Oracle, SQL Server and DB2 and a PostgreSQL/PostGIS installation as well.
If you want to work in a variety of languages you can script, inside the product, using any .NET language like VB.NET, IronPython or C# or any ActiveX language, including non-Microsoft languages like the Activestate distributions of PERL and Python. I hear there are ActiveX engines now around for a wide variety of languages, like Ruby, so the list probably goes on with many more.
There are endless open doors to enable people to connect to whatever they like. Want to use R or GDAL? Go for it. Nothing about the convenience of integration prevents you from doing that.
Want to run Manifold IMS with PHP and Apache? You can do that as well if you prefer those to IIS and ASP or ASP .NET. [Note that Manifold technical support service products only support Microsoft standards, but that's a pricing decision limitation of a support service product and not any sort of technical or operational limitation of the GIS product itself.]
At the same time there is a lot of internal wiring within a product like Manifold that supports interaction with other Windows applications, such as the Microsoft Office Suite or with any other applications that take the time to be “open” with each other within the Microsoft ecosystem. That’s tens of thousands of products at least and quite likely hundreds of thousands of them.
Last but not least, any modern integrated package will leave plenty of open doors for connections via the web. Manifold, for example, can both publish and consume via WMS, can publish via WFS-T, holds doors open via open geocoding web server interfaces and open image server interfaces and, in the fine detail of spatial DBMS work, natively supports WKB and WKT if your tastes in geometry types run to OGC “open” types.
Bottom line: taking the time to provide the convenience of integration and a rich, meticulous GUI does not mean you have to close doors to interaction with other applications. If anything, given the overwhelmingly larger numbers of applications in Windows, one can assure greater connections to a wider range of applications within Windows than is typical of a non-Windows application.
If that is a short comment, I’d hate to think what a long comment might be from you Dimitri.
I wonder if your comments are why James is running our of disk space?
@dylan @Dimitri
If you put more than two links in your comment, it gets put in Moderation. I’m usually good about checking, but given my travel I’m not always able to jump on it.
If you do ever post something that has multiple comments, just email me a heads up so I can quickly approve it.
Um, at the risk of stirring up a hornet’s nest all over again, can I ask what is “pure GIS” (mentioned above), and why on earth does “purity” matter here?
I’m a newbie studying for an Mc in GIS right now, and I can appreciate all the complex applications of GIS from resource extraction to urban planning, and all the clever stuff that has to happen when you tell ArcMap to re-project a map layer or do fancy analysis on multiple source layers, and so on.
But there’s also a whiff of the GIS priesthood about some of these claims for what constitutes “pure GIS”. Why shouldn’t it also include some of the newer ways in which new groups of users are exploiting geographical information via computer technology – “Geographical Information Systems” in other words?
I’m from a database background, and much as I hate it, I would accept that MS Access is (just about) a “database”, even though it is far less sophisticated than Oracle Enterprise RDBMS, for example (although I have mixed feelings as to whether ESRI’s “geodatabase” is really a “database” – but that’s just my personal prejudice!).
But people pick the “database” they want based on what they want to do with it. I work with Oracle on commercial enterprise systems, MySQL for internet projects, and I’m now looking at Postgres PostGIS for GIS projects. Each of these does different things, but each one is a “database”, and I’m doing real “database” work with them. And I’m not terribly interested in concepts of “purity” when it comes to working out which tools I need to do a particular job. I don’t ask myself if I’m working on “pure” hammering applications when I hammer a nail into a fence, after all.
Why shouldn’t GIS be the same – with a range of GIS tools from the all-embracing all-powerful behemoth of ESRI to the pick’n'mix open-source tools and the mash-up technologies of Web 2.0?
Oh well, I guess my GIS ignorance is showing, eh?
I wonder why Tatukgis does not get any attention in the US? It seems to offer a capable API. One very attractive feature is distributing royalty free applications built from Tatukgis.
One of our handheld programmers found Tatuk in a coder rag. I was really surprised by how functional it is. There’s a cool, free viewer:
http://www.tatukgis.com/products/viewer/viewer.aspx
The company is Polish…
I don’t know how you find the time Dimitri, but thank you for doing so and thereby being the voice of sanity on this post. Elitism has killed far too many good applications, and leaves us ‘great unwashed’ who just need an app to get the job done (a commerce or science or medical or whatever job).
I know one can do some pretty amazing analysis with the various open source apps; what their proponents haven’t noticed is that someone trying to map an epidemic in the middle of the Congo Basin (for example) does NOT want to muck around with fiddly stuff. They need a ‘quick and dirty’ that works and that integrates with other people’s software.
GIS is a TOOL: its form should always be driven by its function. Victorinox would never have achieved recognition for their (Swiss Army) knives if they hadn’t been simultaneously integrated, functional and high-quality TOOLS.
I’d love to see a free version of Windows. Open source developers? I do not mean Wine, I mean an opensource OS that would take the place of Windows.
You can either keep up your elitist pretence that Bill Gates never happened, or you can take the fight to his turf. Your choice. Whoever comes up with the best tool is who we will choose.
David Franklin:
All tools should be designed to fit the purpose. In the same way that Windows is not the best operating system for all purposes (Manifold’s website is, after all, hosted on Linux), closed source GIS tools don’t fit the needs of all people all of the time.
Open Source GIS tools meet the needs of some people that can’t be met in any other way. Proprietary GIS tools meet the needs of some people that can’t be met in any other way. Trying to pretend that every problem is a nail is silly.
@Christopher Schmidt: Trying to pretend that every problem is a nail is silly. Believing in earnest that every problem is a nail is scary and dangerous.
I have a flash site i’m looking for the script who shows google adsense with flash. where can i get it?