Multi-Cores and 64-Bit GIS
We’ve hit on this discussion before and with the Developer Summit coming up maybe it is a good time to think about the direction processors and their movement to multi-cores and 64-bit processors.
At 9.3, ArcGIS Server Enterprise (or whatever ArcSDE is called these days) will move to 64-bit. This is a huge improvement as I would guess most new database deployments are built on 64-bit servers. But clearly ArcGIS Server itself is not going to be 64-bit at 9.3. When spec’ing out new servers, it is impossible not to go down the 64-bit route and servers going multi-core only in the next year ArcGIS Server will only get slower because it cannot take advantage of new technology. With the focus moving from clock speeds to cores, ArcGIS Server users run the risk of being stuck at a level of performance that is going to be unacceptable in the future.
I’ve been told that ArcGIS 10 will support multi-core/64-bit, but given that it probably won’t go final until at least late 2009/early 2010 that means we’ll be running into trouble way before we can even deploy beta version of ArcGIS 10. But is this really a concern for users? Generally speaking, most folks I’ve talked to don’t seem to really be bothered by this and maybe that is why ESRI is waiting until v10 to ship multi-core support.
Is the lack of 64-bit ArcGIS Server going to impact your business moving forward?
Moony only runs 64-bit servers, do you?
Oh my, the sooner 64-bit support happens the happier we’ll be. This should be the top priority of ESRI moving forward.
I wish ArcInfo would support 64-bit systems.
Sooner or later I’m gonna upgrade my workstation. I’m sorely tempted to run Vista Ultimate 64 with 8GB of RAM and host my dev environment and ArcGIS Enterprise and SQL Server on VMWare or other virtual OS. Wondering if anyone has taken a stab at this?
Greg – Yes, I’ve setup my development machine just like you mentioned. I know we have a bunch of Vista naysayers around these parts, but the OS is rock solid. Make sure you have newer hardware and you will be fine. VMWare works very well in this environment.
I have no idea what Lupin has to do with 64bit GIS, but I’m laughing my ass off yet once again.
what exactly do you hope to gain from more bits?
I agree with Lefty. The desire of my organization to publish data and create sophisticated web-based tools is only increasing. I find it shocking that it could be two more years before AGS (outside of the “SDE”) will take advantage of the the hardware advances that are already a year or more old today. I’m not saying that ESRI is going to go bankrupt tomorrow, but this is the way organizations look up one day find that their status as the industry leader has gone by the wayside.
What ever it takes to improve the performance of dynamic display data in AGS should be a PRIORITY! Caching is great, but it is an investment which cannot always be made. 64-bit and or improved rendering times “may” shut me up
We recently refreshed our database server to 64 bit SQL 2005 on a 4 processor, 8 core machine. The performance improvement was amazing. Now we would like to refresh our map servers, but the clock speed options are actually slower these days then they were when we bought them a few years ago.
Intel is going to standardize on quad core processors starting next year, and we are stuck with ArcGIS which can only take advantage of one at a time. Right now we compensate by running many map servers on each machine to give us more concurrency, but still the time it takes to render a single map is basically linked to the clock speed of the processor.
Given that Intel has shifted focus from clock speeds to cores, we have been stuck in the mud performance wise for years now and likely years to come.
http://www.portlandmaps.com
It is always reassuring to see ESRI users debating the the benefits of utilising modern technology. Really, ESRI product users should have been having this discussion and lobbying ESRI four or five years ago.
But all is not lost…
As legacy GIS users, you too can have the benefits of GIS that utilises HPC parallel processing technology, provided by the combination of 64-bit Windows, multicore CPUs/GGPUs and huge amounts of RAM to deliver teraflop performance, now!
Here is how:
Ditch your legacy GIS software
Purchase a licence or licenses to an x64bit version of Manifold System 8.x
3.Using the money you have now saved from not upgrading/purchasing legacy GIS products, build a 64-bit 2u or 4u quadcore CPU workstation, pack it with up to with 64Gb or 128Gb or RAM, RAID arrays of fast HDDs holding data, page and temp files.
Into your workstation motherboard install the lasted CUDA-enabled high-end or ultra-highend grapahics controller from nVidia (eg a Quadroplex system , or Quadro FX5600 or FX4700 graphics cards). Then splash out on a dedicated nVidia GGPU system (ie Tesla C870, D870) to deliver teraflops of parallel processing performance to handle continental size DEMs and remote sensing datasets.
Now use the balance of your savings to treat yourself and family to that holiday you have been meaning to take somewhere nice, satisfied and reasurred you have now bought the most advanced GIS from a company with foresight and which is committed to harnessing the latest advances in IT hardware and operating system technology to the benefit of its user community.
Of course I dont expect many legacy GIS users will take such advice., but as a satisfied user of the most modern GIS, that gives me a commercial edge.
Cheers
What the fuck is Manifold?
Windows has been 64-bit for over two years now, and for almost that long it has been darned near impossible to buy an office computer that does not use a 64-bit multicore processor. You can buy RAM at about $100 for 4GB. If you run 32-bit GIS products in 32-bit Windows you are throwing away years of technical advances to emulate 1990′s machines that run single core, 32-bit processors limited to 1 or 2 GB of RAM.
That makes no more sense than buying an expensive 8 cylinder car automobile to get hot performance and then disconnecting the ignition to all but four of the cylinders because your mechanic finds it too intellectually challenging to work on eight cylinder cars. Get a new mechanic!
In fact, I’ll bet that just about everyone posting on this blog who has acquired a computer in the last year or so is actually writing on a computer that contains a 64-bit processor, probably at least a dual core processor to boot. If you are running a 64-bit processor you should also be running a 64-bit operating system and 64-bit applications. In the mainstream, this is totally routine.
64-bit operation has huge benefits:
64-bit Windows is much more reliable than 32-bit Windows. Unless you enjoy cracking out your “what to do when it crashes” cheatsheet, you will go 64-bits just for greater reliability.
64-bit applications likewise tend to be much more reliable than 32-bit applications. The main reason is that the application is not confined to the very small 1GB/2GB process spaces of 32-bit Windows, so that application errors which cause crashes in limited 32-bit spaces get the benefit of enough slack not to crash in 64-bit applications.
64-bit GIS applications tend to run much faster, often a factor of four or more even for routine desktop work, because GIS data sets in modern times routinely are big enough to require paging to disk to deal with the small memory spaces allowed by 32-bit Windows. Whenever you page out to disk you trade microsecond RAM response for millisecond disk response, a thousand times slower. Using 64-bit GIS applications will often avoid that paging for pheonomenally faster response. I have yet to meet a GIS user who complains about snappier response.
This effect is especially important if you are doing analytics, because many modern GIS algorithms are recursive, so the ability to actually utilize lots of RAM translates into dramatically faster operation.
Modern RAM is so cheap that it is dumb, dumb, dumb not to pop 4GB or 8GB of RAM into your machine. Keep in mind that Windows itself is getting bigger and bigger, and the amount of crud loaded into your computer gets bigger and bigger (ever look at your processes tab recently?) so it is often the case that the limited memory space you have available within 32-bit Windows is not fully available for your GIS process. Pop 4 GB or 8 GB of memory into a 32-bit Windows system and you are wasting it, since Windows won’t be able to use more than 1GB or 2GB of it effectively. Pop 4 GB or 8 GB into a 64-bit Windows system and it can all be seamlessly used. To take advantage of lots of cheap RAM, you must run 64-bit Windows.
That memory effect is especially profound in IMS servers, where the least expensive way of dramatically increasing the number of visitors a web server can host is to simply go 64-bit Windows, 64-bit IMS and toss tons of RAM into a multicore server. Servers with two sockets are now routine and cheap: install two quad-core processors and you get eight cores with lots of RAM that will eat a 32-bit ArcGIS Server installation alive at a tenth of the price. Multiprocessor access to spatial DBMS is especially essential to performance, so not having that also kills ESRI performance on top of not being able to use 64-bit memory spaces.
No doubt the phenomenal disparity between 32-bit glacial response and 32-bit unreliability compared to 64-bit, multicore speed and total reliability is one reason ESRI seeks to go 64-bits with their IMS / ArcGIS Server products first.
And yes, they are getting their heads kicked in by vendors who offer 64-bit, multicore IMS / server products – many users who wouldn’t even think of looking at someone other than ESRI have made the decision to try an alternative because they feel forced into it by the lack of 64-bit and multi-core support in the ESRI product line.
Those customer losses are especially painful to ESRI because it is exactly those customers who need 64-bit performance who are normally the most willing to pay excessive ESRI prices and who are normally the most insulated from poaching by ESRI competitors. They are the last people ESRI wants to get a taste of life with modern price/performance, because once they see for themselves the power and reliability of 64-bit operation in IMS or DBMS apps, they realize that they are probably being failed by ESRI in other GIS product areas as well. It has been often the case that a single customer transition to non-ESRI, 64-bit IMS has ended up costing ESRI millions in lost GIS business overall as the rest of that customer’s GIS usage also transitions. It therefore has not escaped ESRI’s attention that for their GIS competitors, selling 64-bit product against ESRI is like shooting fish in a bucket.
ESRI is caught between a rock and a hard place, because even once they finally introduce some 64-bit server products, all they will succeed in doing is emphasizing how obsolete the rest of their product line is. No sensible person will be happy with 32-bit ArcInfo once they start getting a taste of 64-bit life with, say, a quasi-64-bit ArcGIS Server offering. What’s ESRI going to say… Go 64-bits with ArcGIS Server because it is essential but stick to being a 32-bit retard with ArcInfo?
If you don’t use 64-bit GIS products in 64-bit Windows you are making a decision to give up the benefits of modern hardware, modern price/performance and modern reliability. That is especially wasteful given that your hardware is almost certainly 64-bit already. 64-bit hardware and 64-bit Windows have been mainstream for two years now. Don’t buy any excuses from a vendor who is so incompetent at software development that they are years behind what even your kids now think is old hat.
If your vendor is so senile they cannot keep up with mainstream progress, go find one that can, and don’t wait until 2009 or 2010! (…. 2010? … they have got to be kidding…)
I love it: everytime Manifold makes $495, ESRI loses $22,000!!!!
I’m sure there are things that keep ESRI up at night, but I’m darn sure Manifold is not one of them.
@mnfd: $22,000 is nothing. You think too small.
“Keep in mind that Windows itself is getting bigger and bigger, and the amount of crud loaded into your computer gets bigger and bigger (ever look at your processes tab recently?)…”
I think that disparaging comment is as close to trashing Microsoft as Dimitri has ever come.
I cannot resist…
mnfd: “everytime Manifold makes $495, ESRI loses $22,000!!!!”
… it is even worse than that, as most server installations of Manifold use the Universal Runtime x64 package, which is only $225. For that matter, Enterprise Edition x64 is only $445 (roughly equivalent to ArcInfo plus ArcSDE plus ArcIMS plus Arc Objects plus parts of 3D Analyst, etc.).
Imagine…. everything ESRI wants to sell you in ArcGIS Server plus a whole lot more, all running in true multiprocessor, fully 64-bit code today (…no waiting until 2010!) for only $225 per web server no matter how many processors or applications are running on that web server.
James: “I’m sure there are things that keep ESRI up at night, but I’m darn sure Manifold is not one of them.”
James, you just might be right about that. I sure hope you are!
The sort of company that thinks it is OK to take five years (!) to partially support x64 Windows [shot clock started three years ago with the first x64 Windows releases, and add to that 2 years to the beginning of 2010...] is probably so totally clueless about modern technology they don’t recognize a mortal threat when it appears.
That’s fine by me. I hope they stay asleep at the wheel until it is way too late for them to realize a race has been on and they’ve lost it. That’s how these legacy guys disappear: they get so fat and happy with what they think are captive markets that they forget what a real competitive tempo is all about.
Even if we used the release of Vista 64-bit as the start point, 2-3 years is still an eternity in software.
Maybe pushing server out the door first is all part of their Server-as-the-solution-t0-everything strategy. Need 64-bit analysis? Don’t bother with ArcInfo, get a server license and run the analysis there. Your 32-bit ArcInfo is just the “client”.
I seem to recall it being blogged/commented here that ESRI seems to push Server solutions to customers even when its not really necessary…
We’ve been using Windows XP PRO 64bit SP2 with ArcGIS since the inception of SP2 and ArcGIS runs better emulating on a 64bit machine (yes ArcGIS runs in 32bit mode so no one send me an e-mail commenting on this). Consequently, we experience less ArcGIS crashes.
As for Manifold, can anyone critique Manifold vs. ArcGIS with screenshots please? It would be good to do a comparison critiquing ArcGIS and Manifold instead of filling us with empty comments about how ArcGIS is worse. Where’s the evidence? Back it up with proof. Make me a believer!
@Luke:
These links have been passed around numerous times, but for your benefit, and proof:
Spatial SQL Instruction Video – Gives you a great introduction to the capabilities of Manifold Spatial SQL
http://dspace.library.cornell.edu/bitstream/1813/2489/2/spatialSQL.wmv
How do I do that in Manifold/ArcGIS – Side by side comparisons of common GIS operations
http://dspace.library.cornell.edu/handle/1813/165
Another great screencast of basic Manifold functionality
http://www.scancontrol.com/Tutorials/ManifoldGISReports/tabid/102/Default.aspx
A user’s first impressions working with Manifold, contrasting digitising functionality with ArcGIS and Mapinfo
http://www.quartex.co.za/Blog/2008/01/28/ManifoldGISFirstImpressions.aspx
Also, look at these other links
http://www.manipedia.eu/index.php?title=Relevant_Links
There is loads of material out there from which you can get a good understanding of Manifold capabilities, if you really want to.
It’s not easy to summarize as both products are very large. The definitive specification (with many screenshots) for Manifold is the online edition of the user manual:
http://www.manifold.net/doc/manifold.htm
When printed out in Letter size paper and 10 pt font that runs to over 5000 pages. I suppose if you printed out all the documentation for all products involved in the ArcGIS suite, that also would run to thousands of pages. So a detailed comparison tends to be unrealistic.
I think that’s why folks turn to summaries. A useful summary will not be an empty summary. For example, there is the oft-repeated statement that Manifold Enterprise Edition ($395) is more or less equivalent to getting ArcInfo plus ArcSDE plus ArcIMS plus Arc Objects plus parts of 3D Analyst plus miscellaneous parts of other Arc thingies – that’s actually a very useful comparison of the two.
To get beyond that into specific details you have to discuss specific interests. For example, this thread is about 64-bit operation and the ability to utilize multiple core processors or multiple processors on motherboards. It is useful to note the benefits of both 64-bit operation and multiple processor capability, that Manifold has those capabilities and no ESRI product has them, and also to note the practical strategies to increase performance and to lower cost made possible by 64-bit, multi processor capabilities.
Usually when folks want a comparison they don’t want an outline a hundred pages long with thousands of features compared. They usually want an impressionistic guide as one colleague might offer another.
The User Comments page on the manifold.net home page offers such impressions from people familiar with both ESRI products and Manifold products. I’ve also spoken with hundreds of people who have used both, many of whom are not shy about expressing their opinions pro and con for either.
If you compare the latest ESRI products with the latest Manifold products, what you usually hear first (after the obvious difference in price) is the difference in product packaging.
Manifold packages virtually all functionality into a single product, the base Manifold license, usually Enterprise. In contrast, ESRI packages functionality into several different products. The idea that one product provides four different modes of work is a difference: one Manifold license gives you typical desktop workflow, workflow as both a client and a server in enterprise settings, an objects library, and/or an internet map server. These would be different products in the ESRI line.
In terms of raw functionality if you compare an equivalent basket of ESRI products with Manifold, the usual impression is that Manifold provides both a broader and deeper set of features and that in general the features tend to work better (with one big exception) than ESRI products. “Better” here means work faster, are easier to use, enable simpler work flows, can handle more complex situations, etc..
A good example is that topology overlays within ESRI products have restrictions the Manifold equivalent does not: you have to have non-overlapping areas that have non-pathological topology. Manifold will merrily cookie cut through points, lines and areas simultaneously and doesn’t care if areas overlap or have bizarrely pathological topology.
I realize the above is difficult to believe from a $395 product if you’ve just spent a few tens of thousands, and it is routinely dismissed as puffery, but I’ve not met any serious ESRI person who has taken the time to learn a current Manifold release who has not come to the same conclusion.
Experienced ESRI users will remark that the details of workflow in Manifold are very different from ESRI, but usually report they had no difficulty adapting. Almost unanimously ESRI users are astonished by what they report to be the much greater reliability of Manifold in things like no more crashes, especially if they are running 64-bit product.
Other conclusions people report is that Manifold is significantly less silo-oriented than ESRI. A good example is support for many spatial DBMS products using the DBMS vendor’s own native standards, not requiring some proprietary middleware to do spatial DBMS. Another example is Manifold’s fanatic emphasis on Microsoft nomenclature and standards. ESRI is getting better about that but the difference in style is very clear.
Last but not least, Manifold tends to be much more open to interactions with other programs and data sources. Good examples are the very wide range of formats supported out of the box and the commitment to interaction in an open way with image servers either as a client or as a server.
Let’s turn to ESRI’s advantages against Manifold. There are two principle advantages, a big one and a little one.
First, the big one. ESRI has a very broad product line that includes numerous sub-products or feature sets catering to specific industries. Manifold takes the approach that supporting general tools ends up getting broader industry support. However good the Manifold approach may be in the long run, in the short run it is indisputable that if you want to buy a third party tool for, say, metes and bounds applications, you will much more likely find it for ESRI products.
The small advantage ESRI has is that it is significantly easier to do elite paper cartography with ESRI products than with Manifold, and that won’t change for six months or a year or so.
Most people seem to think that simple cartography is easier with Manifold and creating electronic displays for viewing onscreen or in web sites is easier in Manifold. It is also true that Manifold provides many image effects only possible with something like PhotoShop (see the Gallery), which is a different approach than classic cartography.
But for all that if you want to create a truly elite paper map such as might be created by National Geographic or USGS, you will find it substantially easier to do so with ESRI products. You can still create such things with Manifold, but it will take you longer with much more aggravation and “tinker time.”
I do believe, by the way, that the very best such maps are created using a GIS together with Adobe Illustrator and a tool like Avenza. Our objective at Manifold is to ultimately grow the print layout “paper space” composition and editing console within Manifold to incorporate all of the truly required features made possible by Illustrator. That will take longer than a year, but probably not as long as, say, three years. Since ESRI right now is not equal to Avenza plus Illustrator, I only figure a year or so to significantly exceed ESRI in the paper cartography workflow area.
I should say a few words about the “knock off” papers ESRI has circulated about Manifold, as folks sometimes get the wrong impression from those. The big issue is not ESRI insincerity, it is failing to recognize that Manifold evolves so fast that knock offs written more than six months back are way out of date and no longer accurate. ESRI honestly does not understand what it means to evolve a product at the rate of 1000 improvements per year as Manifold does.
ESRI originally started by suggesting that Manifold is a “toy.” Manifold 4.00 as a GIS was certainly a toy compared to either ArcView or ArcInfo. But within six months 4.50 came out which was roughly comparable as a vector GIS to ArcView, albeit more analytical.
ESRI eventually realized that the “toy” thing was annoying customers who soon learned better, so they switched gears and then commented that Manifold was but a low end desktop product with no web capability. That was certainly true until Manifold in 5.50 introduced IMS.
The next slam was that Manifold had no significant Enterprise capability, nothing at all like ESRI’s SDE. That was also true until Manifold introduced enterprise capability in two steps.
First came Manifold’s own Enterprise model, which shared items stored in a central server but not at the atomic level like SDE. That was very popular, but it led to ESRI’s accurate criticism that it was not a true multiuser system with object-level atomicity. Fair enough.
That changed when Manifold introduced full support for Oracle Spatial with Oracle’s help a few years ago. Since then support for spatial DBMS has extended to every spatial DBMS known, native, plus connectivity to SDE geodatabases and personal geodatabases. Now that Manifold can support enterprises with thousands of users simultaneously editing data (through areas of interest windows) that can be literally terabytes in size, ESRI does not usually refer to Manifold as a “toy” anymore.
In fact, we’ve somewhat changed positions in the enterprise space with the mainstream spatial DBMS vendors relying on us to provide native support first, often as a way of fending off any ESRI distractions in trying to push ArcSDE over the spatial DBMS vendor’s own standards.
ESRI for a while was trying to promote knockoffs against Manifold IMS in small things until they realized that drawing any attention to IMS was against their interests. For at least two years Manifold has been substantially ahead of ESRI products in that area. That’s especially true when 64-bit, multicore operation is concerned so I don’t think they are eager to encourage any more head-to-head comparisons in the IMS arena.
There did linger criticism that Manifold was much slower than ArcView or ArcInfo to render and navigate in large data sets. That pretty much went away with the much faster rendering that came out in Release 8 in 2007. But you still hear some of that.
I think the current slam against Manifold by ESRI is that it still can be slower than ESRI products when working with very large data, say drawings that are gigabytes in size. That’s true to a degree, but here we are getting into implementation details such as sensible storage within spatial DBMS.
No matter. This summer’s campaign will roll out enhanced Manifold products that will be able to handle very large data, effectively of unlimited size, with lightning speed without users having to pay attention to how they choose to store it. It will also start a recurrent focus on cartography that I think will take a couple of releases, say a year, to get ahead of ESRI.
ESRI does have the problem that quite a few of their sales and marketing guys are still thinking of Manifold as it was three years ago. So without any intention on their part to mislead their customers they are saying stuff that just isn’t true anymore.
Hope this helps!
Can someone post some high user traffic sites that use Manifold IMS? I’ve seen a few examples, but nothing that really shouts big user base.
My attempt at Manifold is highlighted below:
Creating a simple map in Manifold
Fighting Manifold (or fighting the way I’ve learned GIS)
To be fair, I wasn’t looking at replacing ArcGIS so I didn’t spend the hours required to learn the new GIS system. So take my findings with a grain of salt. I found the GUI confusing and not very intuitive.
Is there any examples of manifold maps that use complex rendering like this?
http://www.portlandmaps.com/detail.cfm?action=Explorer&activeTheme=Property&ZoomLevel=3&propertyid=R332563
Also it would be good to see execute times for Manifold vs. ArcGIS maps.
Besides all that, for those of us who are heavily invested in the ESRI platform, I’d be interested to hear back from some of the ESRI folks for when we could expect multi-core and 64 bit support. Even better would be the move to using DirectX or OpenGL for rendering so that hardware acceleration could be used.
Till then I guess we will stick with our older 6 server cluster. =oP
-Phillip
Some demos where offered here from a previous poster:
http://www.mapserving.com/publicmaps.asp?M=PublicMaps
None of them are very fast or look that great. There must be better ones out there, just like ArcIMS sites look bad by default. I’m willing to keep an open mind here at this point.
@Dimitri…Good to hear about the cartographic upgrading.
When that happens, one major gripe will be gone.
Now, if only the interface…
I do hope y’all have continued success and keep moving the bar higher and farther.
Someone has too.
KoS
That depends what you mean by “high user traffic.”
General Motors (GM) is a very large auto company and they bundle their OnStar service with many of the cars they sell. The national site that shows OnStar coverage throughout the US probably gets more user traffic than 99.99% of IMS sites regardless of which vendor’s IMS product you are measuring.
Visit http://www.manifold.net/info/ims.shtml for the IMS home page – there’s a thumbnail there that teleports you into the GM Onstar coverage site.
The reality is that most IMS sites (again, of any kind) get very little traffic. Even a midsized city’s site will typically only get a few thousand to well under one hundred thousand hits a day. That is very easy to serve, even with a relatively slow server and even if you only use 32-bit products like Arc.
So why all the noise about 64-bits and multicore, when for the most part even a slow, 32-bit IMS works in most cases? There are several reasons:
Reliability. Can’t have crashes on a public web site even if it doesn’t get too many hits.
Performance with large data. Even if you don’t get many hits, as data sizes expand you take much less of a hit when using 64-bit product and lots of cheap RAM.
Headroom. Everyone dreams of the day when their website hits the “big time”. As everyone who has managed a sophisticated web farm can tell you, the job changes in fundamental ways when you scale up from just one server to managing many. If you can keep the application within a single box, albeit one with two sockets hosting eight cores, you win a lot of economies, not the least of which is fewer worries for the person in charge. People like the idea that if they start with 64-bits they could (and some do) start with a single dual core processor in one socket and then as their needs grow and the prices come down jump all the way to having two quad core processors and more RAM, all with zero changes in their software. Quite literally, just plug and play.
By the way, for all our pride in getting more and more performance and capacity I really think that the bigger challenge and what will end up being more decisive in terms of winning share in IMS markets is making IMS easier to use [more standard templates, more examples of connections to other tools], easier to customize [drag and drop interaction with Visual Studio, etc], easier to interact with other tools [Silverlight, etc], more architecturally flexible [more examples using OpenLayers, pre-built clients for WFS, etc.] and, of course, lower in cost.
Numerically speaking there are far more “small scale” applications than there are applications like OnStar that span the entire United States. The small applications also tend to have fewer technical resources to support them than, say, a web company with hundreds of programmers and a supporting cast really able to handle a big server farm.
But if your cost is only $225 for really limitless IMS capability, well, that opens up a lot of strategies even for the little guy. For example, most people in organizations that use GIS don’t actually do GIS. They just need to see the results of someone else having done GIS. For them, IMS is perfect.
Consider a case where a state government is creating maps in, say a spatial DBMS (could even be an SDE geodatabase). They have a need for county governments in remote counties to use those maps, to make them available for local consumers within their government to view those maps.
You could drive the cost of GIS down effectively to zero by having the counties run IMS on their intranet (or within the state’s own WAN or Internet). That gives everyone in the county the ability to view those maps using a free browser. It’s going to be an insignificant number of hits, but it will save the county a lot of money.
To take another example, when the cost of IMS drops to zero suddenly it makes sense for even individual users to expose their own maps or data, should they so choose, through the web so that they can get to it remotely, show clients what they’ve done, etc. You wouldn’t do that at $20,000 but you sure would for $100. This is another case of very low number of hits but very high convenience and very important to the user and, ultimately, very many more licenses put into play.
You gotta have a better site than that Dimitri. I hope that web client code isn’t your and is something GM forced upon Manifold.
Lefty: That site is not something Manifold created (there is a big ecosystem of consultants who do such things), but it is a real OnStar site so no doubt the consultant did whatever was desired by the customer.
You mentioned: “There must be better ones out there, just like ArcIMS sites look bad by default. “
I agree 100%. It is almost axiomatic that the more people like a site the farther and more custom it is from a default template.
People tend to forget that the innards of what any IMS serves out is just a very small part of what can be accomplished by a gifted web designer who also has good taste.
For all that there is a role for simple templates that are easy to use, do the job out of the box and are difficult to mess up.
Philip: There are lots of city sites. The way to find them is through the forum at
http://forum.manifold.net
where lots of IMS developers have posted their sites. I like the look of the site you cited and that could certainly be done in Manifold.
You commented on DirectX and OpenGL – that’s not going to have a significant impact on rendering speed because the hardware acceleration done for such stuff does not address the bottlenecks in 2D GIS. It helps for 3D work, but not the 2D stuff.
The greatest bottlenecks in rendering large data in 2D GIS work tend to arise from a combination of data access and simplification tasks. For example, suppose you have a 2 GB drawing with zillions of objects, each made of up very many coordinates.
Zoom out, pan, zoom in, change the window size, jump to a different location and your rendering system has to be able to get enough data to figure out what is in view and how it should be simplified to give a good rendition at whatever happens to be the right zoom level and viewport.
If you have a dynamic system like Manifold that can re-project on the fly it is also possible that the viewport through which you are looking at the data uses an entirely different coordinate system than the one in which the coordinates of the geometry data are expressed.
In fact, you could even have multiple different layers consisting of radically different coordinate systems or even data (raster data with different sized pixels, different shaped pixels implied by different coordinate systems, different vector layers in different coordinate systems, different visual effects like hill shading or coloration of 3D surfaces shown in 2D, what overlaps what, what is exposed or not exposed, different algorithms for rendering complex line formatting or area formatting… all that).
There are many different strategies for dealing with data access, computation and rendering, but they are not usually helped in any significant way by DirectX or OpenGL.
They can be helped by massively parallel computation using things like CUDA. For example, it sure helps to have a few hundred processors working in parallel to do re-projection on the fly so you know what is visible and what is not in any particular window given the coordinate system that window uses.
Dimitri: Does Manifold support map tiles?
Lefty: Yes, in several ways. To name a few:
example in the IMS examples page at
http://www.manifold.net/info/ims_examples.shtml
…it’s the first example in the “More Advanced…” section. Note that these are deliberately simple examples which are tutorials intended to teach techniques.
As a WMS Server.
As an image server (like the tiled delivery from Google Earth or Virtual Earth servers).
As a client onsumer of image servers… you can layer in a tiled layer from WMS, image servers like Google Earth or Virtual Earth, etc.
Automatically tiling as a client from image libraries.
Automatically tiling large images or other rasters for storage into spatial DBMS, and then re-assembling tiles when images are linked from the spatial DBMS. This can be done using the vendor’s native raster DBMS type when available (Oracle Spatial’s GeoRaster) or using Manifold’s own storage if no native raster type is available. This is astonishingly fast, a real credit to the DBMS vendors, and much faster than image libraries taken from the file system.
The latter two scenarios also go faster with multiple processors, since then multiple threads can be launched to fetch and render tiles from either the image library or from the spatial DBMS.
@Lefty: yes, I believe the sites posted below use the “map tile” technique.
@Phillip: I’m not sure that comparing execution times between Manifold IMS and ArcIMS would actually show much. The reason being that the two applications work quite differently, so the user coding of the site would probably have an effect. Here are a couple of sites though that I though looked pretty good:
http://gis.welland.ca/wims/default.asp
http://map.niagarafalls.ca/
Ah! That just reminded me of another area where ESRI is ahead of Manifold. (gotta be fair…)
ESRI has really good cataloging while Manifold expects you will use whatever other tools you already use to keep your Windows file system organized. I think we’re better at browsing stuff stored in spatial DBMS, but when you want to catalog your GIS holdings on disk through your GIS, ESRI clearly outshines Manifold. Gotta fix that.
“I’m sure there are things that keep ESRI up at night, but I’m darn sure Manifold is not one of them. “
James, actually, I wouldn’t be 100% sure of that. Losing users to Manifold may not be “keeping ESRI up at night”, but I’m positive they have taken notice of Manifold (personal experience), and that it is happening more often.
“$22,000 is nothing. You think too small.”
This is the exact reason that I use Manifold, I don’t have the funding to even think about a full blown ESRI GIS solution. We NEED GIS, but we can’t afford to go with a full ESRI implementation. It would easily cost more than the $22,000 mentioned to do the ESRI implementation we need. Sure, there are things that we could do with ArcGIS Desktop and ArcGIS Server that we can’t do with Manifold, but there are also things that we can do with Manifold that can’t be done with ESRI.
The bottom line here is that Manifold serves the needs of our users that are currently using it, at a fraction of the cost to do the same thing with ESRI. I currently have some internal IMS sites that meet the needs of many employees (with more planned once I get the new dual quad core processor, 24GB RAM, 64bit server up and running), and a pilot project with Manifold running on 4 desktops with some very basic application specific customizations to the standard Manifold GUI interface. The people involved in the pilot project love Manifold, and it is exceeding their expactations (not GIS people, 3 are engineers, and one is a planner). My software investment for the IMS, and the pilot project was less than $2,000.
Now back to the general subject of 64bit computing – it has HUGE advantages over 32bit. I think ESRI is definitely hurting themselves (and the users) not moving forward to 64bit at a faster pace. The simple fact that you are no longer limited to 2, possibly 3Gb of RAM makes a dramatic improvement in performance.
Look at the 3D Viz/Movie industry. All of the top tier applications (3DS, Maya, XSI, C4D, etc.) already have 64bit versions of their software. It has dramatically increased the quality and quantity of work that effects studios can do. Sure, GIS isn’t Hollywood, but we have been hobbled by the same 32 bit hardware/software constraints that many of the studios were (especially the smaller companies without inhouse software production). But, there is a lot of competition in the 3D world. If a company didn’t move to 64bit as fast as they could, they stood to lose a large portion of their market share (time IS money in this indusrty).
So, 64bit, who needs it? We do! Turn on several vector layers, a bunch a labels, and a highres orthophoto in ArcGIS (no SDE), then be prepared to sit back and waste time waiting for the screen to refresh. What about doing a very complex spatial analysis on several large data sets? 64bit will make your life much better!
ESRI needs to wake up and smell the roses, if they don’t push to multithreaded 64bit code rather quickly, the will lose a large number of sales to the smaller vendors that are multithreaded and 64bit, or even to OS software running multithreaded and 64bit.
Why is Manifold’s success contingent on ESRI noticing them? Why such concern about ESRI? I just don’t get it.
If you’ve got the right product at the right price, success will come. The focus on what ESRI is doing seems unhealthy to me.
But what do I know about selling software?
Didn’t complete my edit in time!!!
When talking about the 3D Industry in my last post, I wanted to finish up with:
But ESRI doesn’t have any real competitors in the GIS world like the big boys in 3D do to push them forward…….or do they?
Where are these ESRI “knock off” papers?
James,
Well, there are three related angles being discussed here: reaction to a comment you made, a general concern by GIS users about shifting markets, and to a minor extent whether anyone at Manifold cares what ESRI does.
Your comment
…in your usual pithy way stirred up the mix and prompted some comments. People respect your observations so you are going to get a reaction.
But I think the real discussion is a very serious awareness by many ESRI users that this 64-bit, multiprocessor thing is a significant watershed.
It’s not just some dumb, small thing that doesn’t mean anything. Missing the sea change to 64-bit by five years is a really big deal that cannot be easily explained away. People sense that and want to hash it out, and it’s natural to discuss the possible merits of 64-bit operation using the only benchmark there is for fully 64-bit GIS (if anyone knows others I’d be grateful to hear reports of any new developments).
From a purely technical perspective, Manifold is useful for this because of the existance of both 32-bit and 64-bit versions that are identical otherwise, so you can really see what the benefits may be of 64-bit operation in a true apples-to-apples comparison. It’s a huge deal that never fails to impress folks who have made the jump to 64-bits… and that’s praise for Intel, AMD, Microsoft and the RAM industry that have made 64-bit operation so compelling. As mainly ex-Intel guys we knew the 64-bit thing was essential but couldn’t do anything about it until the processor vendors and Microsoft did the heavy lifting to make it a practical reality.
I don’t mean any disrespect but there is not much thought about ESRI in terms of Manifold planning for the future, because bigger players dominate what happens. ESRI just cannot compete with the likes of Microsoft, Oracle, NVIDIA and Intel when it comes to defining either the near or long term future. There is just too much really new stuff coming down the pipeline and you have to look at the road ahead when you move fast.
All that new stuff from NVIDIA, Microsoft, Oracle and so on represents billions of dollars invested by the smartest people on the planet to implement wonderful new things. That’s very fast company and we have to work very hard to keep up so when we go to Microsoft and they ask “So,… how are you coming along on implementing…” we have more to report than lame excuses for not getting it done.
So, sure, we respect ESRI but there’s a lot more on the plate to deal with that is a higher priority.
I think everyone makes a good point on where we are in the industry. I agree with James that we need to stop focusing our attention on ESRI and what they can and cannot do but use whatever tools that are available from other GIS software initiatives. Open source is becoming more mature such as SharpMap and new proprietary GIS programming tools and engines for the web such as Mapdotnet. With these initiatives, the advancement of GIS technology speeds up and better ideas are implemented. The GIS industry is better because of these initiatives. Personally, I would like to see more. It helps diversify and strengthen the GIS DNA.
As for 64-bit and multi-core software, many software vendor’s are faced with retooling their core code, which costs tons of time and money. However, as developers we know this to be better for our GIS future. But placed in the accountant hands, this vision and future development of newer technologies and ideas die. Thus, we as developers need to be more forceful in our ideologies when justifying to administrators/CEOs or we stagnate. For example, many of Symantec’s code are very old as they keep building newer functionality on top of the old delphi, cobal, Turbo C code without retooling. Consequently, the core software becomes very expensive to retool overtime. Software companies that don’t retool their software tend to wither and die or be faced with rearchitecting their core solution; thus, creating space for new ideas and concepts to emerge. If Manifold is the wave of the new future for GIS, it will naturally occur. However, when I see people looking backwards at older technologies to try to prove that their solution is better seems to lead me to ask the question, “why?” If your solution is futuristic, it will emerge naturally. Your comments about ESRI is more political than comparative.
I checked out the websites for IMS and they are fairly quick but I wonder why they are so small. Do you have a website with IMS that increases the extent of the map frame. The test of quickness is determined by how it handles multitude of large datasets displaying them in an efficient way. How will IMS hold up in a rich vector/raster environment (i.e. 60 detailed GIS layers with high resolution orthophotos (1/2 foot pixel resolution)? With these specifications, can the map window be scaled to fit on a normal 1200×768 or 800×600 resolution map frame and still perform well? Being multicore and 64-bit enabled only helps when the solution can handle these types of specifications eloquently. Please don’t take this question the wrong way as I’m trying to figure out the advantages of building on top of Manifold in a real-world environment.
Dimitry – Re: City Web Site Traffic
Our map servers get between 15,000-30,000 requests per hour during business days. We have 6 machines running 12 instances with our own load balancing/fail-over scheme. The servers are monitored constantly and automatically restarted when they fail.
Serving lots of maps is no problem for us, its the speed in which they render. I’d like more speed but don’t want to drop the quality of the maps we have now.
It’d be interesting to see screenshots of the rendering options for Manifold. ArcGIS has so many layer rendering options that it makes you dizzy, but it also makes great looking maps (even if they are slow to render.)
We’ve played around with anti-aliased maps in ArcGIS and they look fantastic, its just too bad they are much too slow for what we need.
@Philip,
Why don’t you use tiling to improve speed? Is it not practical based on your circumstances?
Tim – we’ve got some tile maps but because we have over 30 different types of maps, it would take a lot of tiling to replace everything.
And even if we went to a tile based map, we still have to generate the tiles… the longer it takes to render a map the longer it takes to make tiles.
Luke:
I don’t know what IMS sites you are looking at, but if you are looking at the IMS examples page on the manifold.net web site those are all tutorial examples. The data is deliberately kept small so that the zip files are small and easily downloaded. Plus, you never know what machine a new user is employing to try out sites so it pays to keep examples small in case it is a small, slow machine.
All of the manifold hosted demo sites tend to use smaller frame sizes because some of our visitors connect via slow Internet links (think middle of Africa, etc. ) so they prefer smaller sized web content.
Sure. You can make it whatever size you want. There are lots of sites (see the forum, as well as citations in this thread by contributors) of much larger map frames.
Note that webmasters often prefer smaller sizes for the maps they show for a variety of reasons.
Probably the most frequent is that it takes no thought at all to use a smaller size, like 600 x 400, to assure the display will almost always more easily fit the browser window no matter the size of the monitor/laptop/window than it is to code a variety of sizes to fit different devices.
Probably the next most frequent is that it is almost a law of nature that webmasters, like everyone else, always like more bandwidth than they have purchased.
They toss smaller images down the pipe so they can get snappier action for more visitors with a cheaper pipe.
Third is being considerate of users who may have slow Internet connections, a real factor in poorer areas in the US where many folks still use dial-up and absolutely a real factor in many parts of the world where slow Internet connections are the best you can get.
If you have an intranet application over a fast LAN where you know the monitors in use you don’t care about the above, of course, so you’ll likely use larger sizes.
The number of layers or images really doesn’t matter as much as the methodology used.
That depends upon a lot of factors which are set forth in the Performance Tips topic in the user manual. If you want to do complex things with large data on a small machine you will need more expertise to pull that off than someone who is doing simple things with small data on a rocket fast machine. (The usual case with much software).
Let’s consider an example: Manifold gives you very many options with how to store, link and consume images. Choose a fast method (spatial DBMS storage, linked ECW image, etc.) and use the same projection as your map window (so no reprojection on the fly need occur) and it is instantaneous. Choose a slow storage method and force it to reproject on the fly and it will be painfully slow – but those are gross negligence sorts of mistakes that webmasters should not be making.
Likewise, it usually doesn’t pay to try to display 60 layers full of complex data at the same time since the display will be illegible. The more usual situation is that there are 60 layers in the project but only some intelligible subset of them is displayed at any one time. Users (or the application) chooses the layers that are “on” at any given instant, using methods like zoom ranges whereby layers are turned on or off for display automatically as users zoom into or out of the display.
I believe some of the sample sites or those given by users illustrate the use of the above techniques. Zoom ranges, for example, are really easy to use.
Sure.
More accurately, being multicore and 64-bit are essential technological advantages which help the solution handle those types of specifications eloquently. One of the most important benefits of 64-bit, multicore operation is providing faster rendering performance.
Let’s take an example: many web applications are intrinsically linked to use of DBMS with most sophisticated spatial web applications utilizing spatial DBMS (in fact, now that using spatial DBMS has become effectively free, we see even small users taking advantage of spatial DBMS within the manifold community). It’s a really big deal for performance if you can fetch data from that spatial DBMS using multiple processors to launch multiple threads. If you have eight cores in your server and you run Manifold to link to a georaster stored in Oracle Spatial, Manifold will use all those cores to grab georaster image tiles from the spatial DBMS using multiple threads.
That’s an example where absent multicore performance someone might say “oh, that renders slow… toss another machine at it” but the faster rendering that comes about from multicore performance is not a simple matter of multiple cores each rendering a portion of a small display, it is in speeding up the data access portion of the task, which quite often is indeed the bottleneck. Sure, there are cases where multiple processors can render an image faster when each takes over the rendering of a portion of that image. Manifold does that with image libraries. But I just wanted to use this example to show how multiple processors can improve rendering performance by improving something other than the final rendering task, that is, by improving data access.
If there is a lot going on then likewise you’ll want to have 64-bit operation because then you will not be limited to what in actual practise in 32-bit Windows quite often ends up being a mere 1 GB process space. You’ll want to be able to load up your server with 8, 16, 24 or 32 GB of RAM and actually be able to use that RAM.
In the example given by Phillip, I wouldn’t be surprised that one of the reasons why he needs to use multiple boxes to deal with a relatively small volume of traffic is that each of his boxes is emulating a 32-bit processor running 32-bit Windows. So, instead of being able to use a single contiguous 16 GB space within a single 64-bit box to avoid paging to disk, to manage delays caused by paging caused by the relatively small memory space supported by 32-bit Windows he has to divide up his user services to run on multiple boxes.
That’s just a guess and no doubt there are other factors at play, but going 64-bit often makes a big difference if you have the sorts of complexity issues that make a process just large enough to trigger paging within a 32-bit Windows system. That is a huge deal.
The moment that happens, you are no longer working within the microsecond access times typical of RAM but instead are dealing with the millisecond access times typical of hard disks – a thousand times slower. You don’t have to slow down data access a thousandfold too often to end up making your application dramatically slower than it could be if paging were not in the picture.
By the way, a lot of folks may wrongly think that “oh, I’m concerned about rendering and a 1280 x 1024 frame to render is way under a gigabyte, so that can’t be the case”
That’s not how it works since GIS packages in general have a lot to think about in order to decide exactly what has to be displayed within that 1280 x 1023 frame. So it is not the raw number of bytes for that many pixels, it is the size of the processes involved with working with the data that may or may not be displayed, considering, possibly, the entire size of the data, memory consumed by data access methods, by analytics (as are necessary for things like thematic formatting, deciding how something is to be rendered, visability of what’s visible given on what’s on top or not, etc.) . Also, keep in mind that programs which use Windows will naturally want to leverage a wide variety of Windows facilities and related Microsoft technologies to avoid re-inventing the wheel. It is the most natural thing in the world to include some handy capability without realizing that you’ve just increase the size of your process dramatically to utilize that capability.
[Folks are sometime shocked at the "huge" amount of memory that can be consumed for what seem to be simple things. I'm not bad-mouthing the phenomenon, either, by the way, because human time is far more expensive than RAM. The correct thing to do in modern times is to take advantage of the ability to re-cycle effective, well-debugged code. That's much wiser than re-inventing the wheel to make it fit into what are now obsolete and too-small memory spaces.]
If you are emulating a processor that tops out at only a gigabyte or two you are going to be paging a lot more than if you have the seamless ability to use larger RAM. And it must be emphasized that what used to be astronomically expensive, say, 8, 16, 24 or 32 GB of RAM is now very affordable, probably some of the least expensive parts of the web server.
Phillip:
That’s only about 150,000 to 300,000 requests per 10 hour day – more than most sites get but in an absolute frame of reference not all that much. I would not be surprised if you could get all that running on a single dual-socket (eight core) server with, say, 16GB of RAM, running x64 Windows and x64 Manifold and either match or exceed the rendering performance you have today with six machines.
The way to test this without spending or risking a lot of money is to procure a quad-core 64-bit computer with, say, 8GB of RAM… that’s a relatively inexpensive machine that would be just fine as an upgrade for your personal development desktop machine after you’ve finished the test.
Use that machine to get some expertise in Manifold so you can configure your application optimally and then give it a try. Initially, you can do a quick and dirty web site that accesses and displays the big data in a realistic test that avoids any performance killers but which is not totally equipped with all the refinements you’ll eventually add.
If it pans out OK, well, then you can make the bigger jump to a dual socket, big memory server (which tend to cost significantly more than single socket “desktop” machines).
If it doesn’t pan out OK, you will almost certainly discover some new possibilities that you can put into play with future projects that could end up saving you a lot of time and money. Worst case, you just upgraded your personal development machine to be ready for whatever comes down the 64-bit pike.
One point of advice on the machine: get something that can host NVIDIA CUDA cards, ideally with two full-speed sockets so you can do SLI with two CUDA cards. CUDA performance is so intensely addictive and so inexpensive that you’ll never forgive yourself if at the same price you could have sourced a box that later on will let you plug in two full-speed cards but failed to do so.
[Your initial web application almost certainly won't be able to use the CUDA cards but as summer turns into fall the next generation of Mainfold product will quite likely give you some eye-popping performance benefits if you have a CUDA card in the box.]
You’ll have, to some degree, the same issue with Manifold as clearly if you are working with very large, very complex data to provide a lot of detail there is more computation the system must do. But 64-bit operation wins you a lot and even though it is a more complex matter, multicore operation will be a resource you’ll want to utilize too.
Rendering options in Manifold are disperesed among many different features throughout the system. For example, layer opacity is a property of a layer whether it is displayed on the desktop console or through IMS. Likewise zoom ranges, anti-aliasing, various sorts of thematic formatting, label clipping, and what I guess would be hundreds of individual features involved in rendering. Some of these are specific to different data types so they are not in just one part of the user manual. You’d have to visit the drawings, layers, images, surfaces and terrains and so forth chapters.
On the plus side, since these rendering capabilities and options are the same whether you are looking at your map through IMS or through the regular desktop console (akin to using say, ArcView or whatever), that makes it easy to create, edit and see in a WYSIWYG way what will appear in the IMS display.
By the way, everyone reading this who is running 32-bit Windows owes it to their organizations to upgrade their personal machines to 64-bit Windows.
Let me help you justify the expense by assuring you that whatever GIS strategy you end up pursuing, now is the right time to go 64-bit. Prices have dropped, drivers are everywhere, RAM is cheap, new motherboards support the new generation of quad-core processors… it’s a real sweet spot to start getting that hands-on experience upon which your organization will depend when it comes time to move to 64-bits. Rome wasn’t built in a day, and all that.
Considering that 64-bit operation could easily save your organization huge sums of money, now is the time for you to procure a 64-bit box for your desk. Would be irresponsible not to!
Dmitri,
Our peak IMS performance with Manifold is about 20,000 renders/hour on a 3.2GHz quad core server. Full North American geography simplified to 1:50,000 in the MAP file and more detailed geography loaded in from a PostGIS database at lower zoom levels. We’ve ditched the MapServer object as we need more flexibility, but we’re still using Manifold’s objects as the underlying rendering engine.
Phillip,
I’ve been doing some experimenting with MapGuide recently and am finding it to be a pretty well-developed product. They have a samples page at:
http://mapguide.osgeo.org/livegallery.html
One particular demo that caught my eye regarding nice labeling was:
http://216.103.100.65/mapguide/sftrees/index2.aspx
I can’t tell you about performance, but maybe MG is another option if you’re looking for alternatives.
Instead of 64 bit, 128 bit, dual-core, 256-core, and so on, why not use distributed computing on the net like GStock Virtual Supercomputer for Stock Picks ? If it works for the community, it will work for the individual…
After all, the “net” is the computer… certainly in the future…
You certainly could use distributed computing (which I will refer to as “cluster” computing here) – the main questions being whether doing so is more performant, less costly/convenient, etc.
No. The “net” is a communications medium, not the computer. Whether you are linking multiple processors (the real computers) together using local RAM bus bandwidth or whether you are linking multiple processors together using network bandwidth makes a big difference, because networks are vastly slower than local bus bandwidth and because there are practical issues involved in coordinating resources not directly under your control.
From a software perspective, there is not much algorithmic difference between parallelizing a task to run on multiple local processors or parallelizing that task to run on multiple non-local clustered processors. The main differences are the very non-trivial aspects of discovering which resources are available in foreign machines that are much less directly under your control and dealing with the highly asynchronous nature of dispatching part of a task to a remote processor and then collecting and integrating any results when, and if, the dispatched task ever gets serviced. That adds to the cost and complexity of the application as compared to a purely local one.
The main hit to performance is that networks run far, far slower than local parallelization. This is for two reasons:
Networks are much slower than processor to RAM bandwidth. WANs are even slower than LANs and because of regulatory bottlenecks are falling even further behind on a proportional basis.
The speed of light limits performance as physical distance increases between processors. This is already a factor within architectures seeking multi-teraflop performance.
But setting the above two issues aside, what counts for most people is simple convenience. If you can plug 256 lightning-fast processors into your computer for a mere $550 (as can be done with NVIDIA’s 9800 GX2 board) or 512 processors for about $1100, well, unless your time is worth no more than that of a burger joint’s employee you are simply going to plug in one or two such boards and enjoy the immense power of hundreds of processors locally.
So, yeah, clusters were a great idea in the 1990′s but have become a second or third tier idea in modern times given the rapid progress of vendors like NVIDIA in packing hundreds of processors into local bandwidth for what are nearly free prices.
I think that’s why GIS vendors most advanced in technology (everyone knows we are not talking about ESRI here…) have come out with massively parallel computing using NVIDIA CUDA and are not pushing clusters.
Tim/Phillip,
We’ve developed on the new AutoDesk MapGuide 2007 and 2008 version and it is well developed for a viewer. However, the raster engine is less than adequate. We’ve tried tiling, compressing the data in different increments (60%, 40%, and 10%). Since we’re an aerial mapping company, we were able to modify the data in many different ways. The end result was the same. Also, you could not use both map and base layering systems mixed. This was a problem when rendering raster layers. For this, I give MapGuide a “Good” status as the cost is low for a well architected product. Hands down, the vector engine is the fastest renderer that we’ve used. The modifications in the WYSIWYG environment are much better than its competitors. It was much easier to modify and manipulate the API. ESRI’s WebADF controls are less than desirable to modify. Overall, I was impressed by AutoDesk and if they speed up the raster engine they could be the IMS of choice for GIS consultants.
Luke,
Thanks for the information. I’m not surprised about the raster issue since I’ve had a hard time even getting them to load. Also, do you use it with .NET or PHP? It seems to be a breeze to get it to work with Apache/PHP, but I can’t get it to work with IIS/ASP.NET. I really like the Studio product – it seems like great way to build maps.
Tim,
We use IIS 6.0/ASP.net 2.0. It is really easy to install and configure on the IIS side. Click on my link and I’ll send you the install directions. By the way, there is no upgrade package developed for MapGuide.
I thought readers of this thread would be interested in a report of actual multiprocessor performance gain with time to do a task reduced from 20 minutes to 33 seconds, as reported in the georeference forum:
(From http://forum.manifold.net/forum/t62427.2 )
To cut the time required for a GIS task from 20 minutes to 33 seconds by using modern hardware at a fraction of the cost of legacy GIS software is revolutionary. Many GIS tasks will see such gains even with a much less expensive card than the Quadro. For example, toss in a 9800 GX2 for a mere $550 and you get 256 processors hammering away at your job.
Getting factors of sixty speed improvements for such low costs is totally revolutionary. If you are buying new computers, make sure you get NVIDIA for your GPU and make sure your system vendor supports NVIDIA. In fact, this is so much bang for the buck you should probably spec out a larger power supply in your computer so you can easily plug in two or three CUDA enabled cards for even more performance. I guess it goes without saying that 64-bits also is the only way to go.