First off, I had an absolute blast. The city, the venue and the people have all been just wonderful to experience. Right now especially it is very interesting because the GeoWeb is finally being implemented in larger scales and we are beginning to see the results of those who work hard at trying to realize the promise of what the GeoWeb is. I think Ron Lake puts it best when he says the GeoWeb is the Internet, not some of little corner of it. If we think that, for the most part, the Internet can be used by anyone, anywhere, on any platform device. Simply put I can collaborate while sitting next to my pool in Tempe, AZ on my iPhone with colleagues using Linux workstations in Abu Dhabi sitting in high rise office buildings. The internet doesn’t care that I have a iPhone any more than it does that they have Firefox on Linux. The same is the case for the GeoWeb, my use of ESRI Servers should not limit someone using FOSS to access and use those services.
Now of course in practice it rarely works out that way. Most ESRI Server implementations doesn’t enable OGC standards even though ESRI has worked really hard at implementing them. And even FOSS servers don’t necessarily publish OGC formats that the GeoWeb wants to use. The technical limitations of the GeoWeb have been removed and now the problem is cultural. We need to start thinking about how these systems are going to come together and how we’ll be able to collaborate without having to all be on the same platform or language. People always use ESRI as an example of a company that is limiting the GeoWeb by 
not supporting OGC very well and they’ve probably earned that reputation. But to be fair, there are plenty of FOSS users who want to limit their products or services to only other FOSS systems. While ESRI’s might have been technical in nature (though I can see how people might have taken their stance as cultural), the limit of not allowing your products and services to be used by all because of some cultural or personal feelings about the spirit of Microsoft, Apple, ESRI, Oracle, etc is just as bad. Those who want to take part in this new open environment will grow quickly and leave those who put up artificial impediments to their participation will be left behind.
So what does this mean for those who want to see how they can take part in the GeoWeb. Well first off, make sure you are implementing solutions that aren’t closed. That doesn’t mean that you can’t use things such as Oracle, .NET or ESRI. Make sure those solutions offer up information and data in formats that people can use and build upon what you’ve done. I see great potential for government agencies that allow 
their data and information to be part of everything from mashups created by some neighborhood group to global companies who want to see new marketplaces and areas for expansion. This should be done through services, not FTP sites or zipped up shapefiles. I can’t be sure my applications are using the latest data if I have to manually browse an FTP site and somehow reconcile my data with yours. A simple service where I can subscribe to information is much simpler for all. Second, end users of the data should begin to recognize that their output shouldn’t be only paper map or even a PDF. KML, GML, GeoRSS and many of the other standards work very well when accompanying a paper or PDF map.
Making your data discoverable is also very important. That I would spend time making my data easily usable and not take the time to make it discoverable hurts my implementation. Making sure Google and Microsoft (assuming Live Search ever gets fixed) are crawling your information is critical to its acceptance. We will begin seeing spatial results start showing up in Google very soon and when that happens, those services will become extremely popular. If I’m the County of Maricopa, I don’t want my data hidden behind some old MapGuide Active X control, but as discoverable services that people can subscribe to and use. Think of it simply, if your data isn’t discoverable via Google search, someone else’s will and the parcel information that shows up will not be under your control. You can choose to ignore spatial search, but someone else is sure to step into your space and offer such services.
The time spend with everyone in Vancouver was well spent and I’ll continue to post what I saw an learned over this next week. Seeing real world implementation that take advantage of what the GeoWeb offers and seeing how successful those are, really validates the vision. It is still early enough in the process to be on the ground floor and there are still huge hurdles as far as data standards and security that need to be addressed so getting involved now can only help everyone. The idea that 10 people can use 10 different software packages and collaborate on geospatial products is very powerful. And of course the added benefit is that you can choose the software platform that best meets your needs and not worry about matching your clients or consumers platforms. That saves everyone time and money, just like the Internet itself has done.
You don't want to be this guy, do you?
Vancouver Photo Credit: jahdakine
19 Comments
James good article. I agreed with you up to the point where you said “This should be done through services, not FTP sites or zipped up shapefiles.”
Now services are great – don’t get me wrong, but there is a point where you need raw data to do raw analysis on or integrate seamlessly with your stuff and you don’t want to fiddle with authenticating with some service and cherry picking your area of interest especially when you are trying to pull this via a command line.
Services (or at least the state of current services) just don’t work well for that kind of thing.
I tend to like MassGIS a lot because while they provide a ton of web services which make mashups easy, they also provide raw data.
Fair enough Regina, I can agree that offering up raw data is probably still a good idea for those who aren’t ready to take advantage of the GeoWeb or need the files for analytical reasons. But I still think not offering up services that users can subscribe to really puts the brakes on the ability to access up to date data. At least if organizations would offer some GeoRSS feeds to tell us (or the search engines) when data is updated on the FTP, that would go a long way to addressing the problems.
Wonderful summary James!
The problem If find with services right now is that they are slow, especially the OGC ones (WMS, WCS, WFS). Both vector and raster datasets perform better when they are stored locally. And in situations where pixel values matter the compression used by most services is unacceptable for quality manual or automated feature extraction from imagery.
I agree with Regina, services are great, but let’s always have the option to grab the data behind the services.
Love the photo at the bottom:
Regina: try a service-free web approach — data via HTTP (!FTP) fronted by an Atom or RSS feed that provides metadata (add some GeoRSS too) and download links to the zipped shapefiles. Google, for one, will treat such feeds as sitemaps, and they are usable in many other applications.
@Sean,
Good point. I was thinking about that too which is why I kind of said (The state of current services). A REST based interface would be the ideal solution to replace the FTP/simple directory listing HTTP approach. Actually directory listing HTTP is kind of nice and sort of follows REST in a very brain-dead but efficient and simple to implement way.
I just haven’t seen that many implementations of REST based services/GeoRSS that achieve that nice end. I think the standard (if I could even call it is still not that well supported by products that would actually be outputting this stuff so still seems like a more manual process than dropping the files in a directory browse folder and making them available).
I’m thinking about cases such as Wget and pulling tiger data (pointing Wget at the tiger listing is trivial the way it is set up and while its old school technology – it works pretty well) and has the benefit that even a fairly non-techie government can set up such a thing.
Its a first step to making data available and then you add services to add cream to the pudding.
the geoweb and online, shared, open data access and data services should be the norm, however, as long as users continue to stay in the 90s by downloading or even worse, buying, free data in archaic formats, sans meta data yadayada the GeoWeb as it is meant to be is likely not just around the corner. Ironically, it’s the neo-geographers, mobile Internet practitioners, and the new-age geo spatial technology practitioners that are leading the way. Hopefully the old-school GISers will catch up soon. I like that quote… The GeoWeb is the Internet. Actually, without the GeoWeb, the Web2.0 would not exist in the way it does
Yes, would be nice to have a ubiquitous wget-like program that could traverse entries of a feed and get the data at the other end of links.
IMHO….data should never be “locked up” by a service. There should always be a option to download the raw data.
Myself as a user, I don’t always want a “canned” presentation of the data. I want to create my own from time to time.
It’s similar to reporting the news, if I chose to, I should have access to the raw information that went into the news report.
KoS
Maybe KoS, but as a providor of data, I’m begining to think that our users would rather subscribe to a service (restful or not) and have that data pushed to them as needed. They just can’t be bothered downloading and integrating the data in their applications.
I’m sure there are users that want/need this kind of data, but we’ll probably start charging more for FTP downloads than we would offering up service based data. The marketplace wants it this way… Brave new world….
The marketplace dictates you charge more for FTP downloads vs. building and maintaining a service to push out the data in a useable format? How so? I’m guessing it’s y’alls own particualr business plan?
Danny, you may be right concering private data. And they (private enterprises) can do what they will, with their own data. Public data on the other hand, if that happened, locking the data up in a service. I would be one of the first to request who’s nuts should be squashed!!
KoS
@DannyV I agree with KoS. Maintaining an ftp is much less effort than maintaining services with all users hitting you at once. For ftp you just need band-width and a consistent way of updating via automated ftp upload, for services you need band-width, processor speed, db backend, another service running within your web server or as separate and a lot more infrastructure. Both can coexist and should.
I’m really quite tired of these people thinking lets abandon everything we’ve been doing and jump on the next new bandwagon that comes along. These people obviously don’t have a sense of the varied ways people use and need to use data.
Its like Radio – sure internet is great and TV is great, but please don’t take away my Prairie Home Companion on my dumb radio device. Just because its old doesn’t mean we should get rid of it. Its tried and true technology.
Look, I’m not going to tell you that you should want or expect FTP downloads. I’m not in a position to talk about my company (probably for the best as I’m not a politically correct guy), but we are much focused. We have FTP downloads right now as well as DVDs we ship out to our clients, all via subscription. You cannot use our data without paying for a subscription.
Now we are focused very much on what some might call a niche area of GIS, and our clients are willing to pay big bucks for our datasets. That said, most of them do not have the infrastructure to download and process our data and have been asking for service based data. I can’t talk price specifics because the marketing guys would flip out, but our plan is to decrease the cost of the data via subscription a couple hundred a month and increase the price of FTP downloads by a couple thousand. This has been very well received by our clients and most of them have been very happy with our restful “database” services that we’ve offered to them in the last year. The idea is that they can integrate directly with their applications and not have to pay consultants or GIS staff to just import the data into their web services. Plus the services will be updated as changes happen, not on a monthly/quarterly basis as the FTP data is/was updated.
Look, if people need the raw data, it will still be available. As a GIS scrounger myself, I’m not naive enough to not understand the implications here. BUT in our marketplace, with our clients, this makes total sense. In fact we expect our sales to increase as people will be able to take our data and put it in their WS clients (yea I know how much this crowd likes WS, but we match what our clients want), Google Earth or even subscribe to our tile server.
@Regina: We won’t be getting rid of FTP; we’ll just be offering cheaper services that the consumers of our products want. For those who still want FTP, they’ll still get the same datasets (though now we’ll just offer shapefiles) and can continue doing what they wish with the data.
@DannyV
Sorry for having been so snippy. Your point is well taken.
I guess a larger part of the population does want just services they can mash into their site and without great need of a consultant to support and maintain the infrastructure.
Just wanted to make sure the other 25% of the population that needs raw data is not forgotten in the GeoWeb hype. Not to mention those consultants making services from raw data to make the other 75% of the world happy.
Eventually I think the lines will be pretty blurred, because a smart service will be able to look at the client coming to it and its request – and look like a plain http/ftp service or a rich web map service whichever is most appropriate. I just don’t think we are quite there yet was my main point and for a minimalist low budget organization producing useful data – throwing together a simple ftp/directory http interface is probably an easy start.
@Regina: This could enable what you are thinking about.
http://www.spatiallyadjusted.com/2008/07/18/weogeo-helps-safe-software-into-the-cloud/
The idea that FME can translate and offer up whatever data in any way you wish could allow companies to spend more time developing data and not worrying about how to either server them up or what formats they should be in.
Yap I was thinking along that line and that would be the ideal service. But as you see with weogeo – they rely on a lot of power to make that happen.
I think the whole cloud computing thing is still in its infancy though that I’m not about to throw all my eggs in that basket. Maybe a few to test the waters.
It concerns me a little that everyone is jumping on the Amazon/Google cloud wagon and there seems to me to be shall we say a single point of failure.
I’ll be more comfortable when I see more clouds forming.
@Regina – I think there may be some confusion between cloud infrastructure and cloud services.
We provide a Marketplace service on the internet. When used in conjunction with our hosting services, it looks like other cloud services, e.g. Google Mail, that provide storage and processing for a fee, rather than you having to process and store information on your own computers.
Like any internet service it requires infrastructure. Google Mail is built upon Google’s massive infrastructure of computers, storage, and bandwidth. We, being much smaller than Google (massive understatement), have chosen to use a cloud vendor of storage and processing, i.e. AWS, for our infrastructure. We have extracted our cloud marketplace service from the infrastructure required to offer than service to an internet user community.
Moving forward, one could imagine that if we decided to offer more cloud services around the geo-transformation capabilities built into Safe Software’s FME Server, we would need more processing and storage infrastructure. How we build that infrastructure will depend on the opportunities available to us at that time.
However, you as a user of our cloud services are only involved with the cloud infrastructure vendors to the extent that we use them for our infrastructure. If we chose to build our infrastructure ourselves in a co-location facility, our cloud services would have nothing to do with the cloud infrastructure vendors, e.g. Amazon and Google.
I know this may seem like I am splitting hairs, but I think it important to consider the abstraction of internet services, aka cloud services, from internet infrastructure. Users of services are accepting the risks of the services providers infrastructure, regardless of whether it is built on home grown computers or developed on AWS.
PB
Paul,
Thanks for the elaboration. I guess I was mixing terms a bit.
In the end though as a user – I am concerned that if every service provider I use uses the same cloud provider, then when the cloud provider goes down (e.g. recent Amazon and Google clousd failings), I’ll be tweedling my thumbs so to speak because all the services I depend on will be down at the same time. Sort of when my internet goes down – (there is a lot of data I maintain locally intentionally so that I have something to do when my interface to the world goes down). I suppose its a bit paranoid thinking.
I guess I’m just making the point that there are not enough cloud providers out there and everyone jumping on the same clouds gives me cause for concern.
Regina,
no worries. Just saw a place where I hoped to create value-add to the discussion.
I agree with your concerns, and trust me it is a concern of ours as well. At the end of the day, we all have to trust some system somewhere for communication, storage, and processing.
You have chosen to tier your trust such that you always data local to maintain peak work efficiency during periods of communication outages.
For us, it is a question of who is more suited to running the “metal” infrastructure for a user-facing web application – Amazon or us. In addition, even if we are better, are we sufficiently better to overcome the higher costs associated with running it ourselves.
Every application (and user) will have to make up their own mind on this cost/benefit analysis depending on their unique application. That said, I not concerned about the fact that others are using the same cloud vendor, so much as I am concerned that the lack of competition to AWS might make them lazy.
best, PB