Wednesday, April 30, 2008

Google Earth and Charts Tutorial Part 2

Thanks to an anonymous poster below it was brought to my attention that the chart in the description info window does not load in Google Earth 4.3. At first I thought it was just a very slow load time, but then it doesn't ever load. After some experimenting, I discovered for some reason when I add the labels to the chart, the image never loads. If I remove the following line from the url then it loads fine: &chl=MaleFemale. I haven't the slightest idea why it does this, and afterall GE 4.3 is still in beta (what of Google's isn't still in beta?)



If you want to see this in action, create an empty text file with .kml instead of .txt for the extension. Paste the following kml in the file and then save it. The image shouldn't load in the info window. Then remove the &chl=MaleFemale, and save. Now the image should load.



<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://earth.google.com/kml/2.2">
<Placemark>
<name>080690013023044</name>
<description><![CDATA[<img src="http://chart.apis.google.com/chart?chs=250x100&chd=t:59,41&cht=p3&chf=bg,s,65432100&chl=Male|Female"> ]]></description>
<Style>
<IconStyle>
<Icon>
<href>http://chart.apis.google.com/chart?chs=250x100&chd=t:59,41&cht=p3&chf=bg,s,65432100</href>
</Icon>
</IconStyle>
</Style>
<Point>
<coordinates>-105.040382,40.615846,0</coordinates>
</Point>
</Placemark>
</kml>

Tuesday, April 29, 2008

“Trust in Allah, but tie your camel”

I was doing some AJAX today, trying to dynamically load markers from points stored in a geodatabase. Basically, just an XML response formatted specifically for my data, and parsed using Javascript. It works, surprisingly given my programming skills :). Anyway, probably wouldn't have gotten finished if it weren't for FireBug. A handy little Javascript debugger that lets me know when I have errors in my code. I use Visual Web Developer express for ASP.NET programming, and the 2005 version doesn't really have anything like this for Javascript (I think I read version 2008 does). FireBug works with Mozilla Firefox, but I think there is an alternative version for Internet Explorer. Be sure to disable FireBug when not debugging because it will report errors on any website, and slows down Gmail...

So I've started work on another project. I am a little unsure of confidentiality, so I won't list all that are involved. I'm sure the project will be made public, or at least the publication will be. It probably wouldn't be too difficult to guess who this work is for if you are in Australia. I'll be modeling camel management plans...while it is tempting (and easier) to just turn in a set of photographs of myself shooting camels, I'm of course referring to GIS-based models. Since the turn around time on this is pretty quick (2 months!), the model is a fairly simple Multi-Criteria Evaluation. I'm planning on using IDRISI (Andes?) to do this, but in the back of my head I'm thinking of writing a plug in for MapWindow. Two months isn't long, and IDRISI has a number of built-in tools for performing a Multi-Criteria Evaluation, as well processing rasters (distance surfaces, friction surfaces, etc...). The reason I want an open source solution is to create a user interface that could be used by anybody. That way a land manager could come in, and given a set of criteria (most-likely predefined), they could spit out a map showing potential locations for different camel management plans.

As idyllic as camels look in the Australian desert, they aren't native species. They were brought in as pack animals, and in many cases the train routes and road routes actually follow the same route as the old camel trains. Now their population is approaching one million, and they can be quite destructive to infrastructure, and biodiversity (or vice versa). Probably the unique part of this model is it is trying to identify locations for management plans based on a perspective, and also including perspectives in the model. Obviously, when a question is asked it comes from a certain perspective, and to answer that question certain criteria will be relevant. On top of that, there will be a layer that explicitly shows where certain management plans cannot be implemented based on the community, land owner, etc... This will most like be a constraint factor...but also could be a distance surface I suppose...Have to think about that one.

I'll keep you up-to-date on it.

Friday, April 25, 2008

GIS Recognition

I recently moved to Australia a few months ago, from the United States. My goal was always to continue in my GIS career, but in case that wouldn't be practical I was prepared to go another route (like work in a bookstore). Luckily, though, finding GIS work has not been a problem.

Of course finding work in Darwin isn't that difficult. If you can't find a job here, you're not looking, or you don't actually want one. Actually, you probably don't even need to look, and a job will fall into your lap. This probably refers mostly to "skilled" labor.

I'm sure the job market has something to do with all my new found work, but I think there is a deeper issue going on. I believe that Australians are more open to the spatial side of life. Spatial awareness seems to be much more prevalent here. Even on the bus the other night, I overheard a mobile phone conversation (not that you cannot overhear someone talking on their phone in public) and he said, "Do you want me to do a presentation on the geographic study?" In the states, I felt I constantly had to insist on the value of GIS and Geography. Here it is more like: "Oh, you do GIS, would like a job?"; instead of "You study rocks?[glassy eye look]" It just seems that Australians are already thinking spatially. I've worked for a company where I tried to implement GIS, and it was a struggle (despite already using CAD extensively). Actually, it really never happened. Then I've worked for a company where GIS was already established, and continued to invade every aspect of the firm's services. Once GIS is there, then it is there to stay. In the least, most everyone in Australia has heard of GIS, if they are not able to explain it, and are definitely open to the value of maps. Some people are skeptical though. As in one case, I was told that it is silly Google Earth (I believe "flashy technology" was used) is the motivator for data collection initiatives. In other words, this flashy technology was what they were using to push for more data. I resisted the urge to state that this person had just slapped an entire industry in the face by calling GE and GIS by extension "flashy technology", and calmly explained that the true value lies in the capability to analyze and display said data geographically. I asked, "Wouldn't it be useful to consolidate thousands of pages of financial data and display it on a map, where one can quickly identify where the resources are and are not?" There is probably a paper in there somewhere about the impact GIS has had on policy and policy/makers.


I think Australia, and probably all of the commonwealth, gives geography a higher place in education than Geography has in the United States. Let's face it, we Americans really aren't geographic people. The Geographic Literacy survey shows that. I think the downfall of Geography is well documented in the American Geography community. From the loss of Geography in Ivy League Universities to the advent of social studies, Geography lost its ground in the 20th century. Take a look at Why Geography Matters for a good read. It's coming back though; the American Association of Geographers is reporting record membership and conference attendance numbers, Harvard has even made steps to reintroduce Geography. I'm sure we all know that this is partially due to the Google Maps and Google Earths. These technology have made Geography extremely accessible.

Moving too Australia has definitely been a good career move for me.

Thursday, April 10, 2008

Multi-What?

I was tasked with creating a Google Earth file for a presentation. The main goal was to demonstrate change in enrolment from one year to the next (two years). I can't go into data specifics due to confidentiality. There were a handful of variables they wanted to show, mainly enrolment, and indigeneity (indigenous status), at two scales (cluster (similar to school district in scale), and individual school). If you try to approach this traditionally, cartographically speaking, who knows how many maps you would end up with. Some sort of bivariate or multivariate symbol would probably have to be created. With GE though, there are different possibilities. The obvious benefit is being able to show data at multiple scales using regions. This allows the presenter to zoom into specific areas of interest and as you get closer, the cluster symbols disappear and schools pop up. To show change, I used the timespan tag and GE's animation capabilities. Basically, I used the total enrolment as the altitude and animated between the two years. So you could see the rise and fall of the points in "3D." I also had the symbol set to different colors indicating positive, negative, or no change. Essentially, I took the route of "dynamic multivariate symbols" (I'm citing myself, b/c I think I just made that term up :)). This obviously wouldn't have been possible without Google Earth. Then again, this wouldn't have been possible without modern-day presentation technology.

Well, I think I'm off Google Earth for a while. I was focused pretty heavily on it for this presentation. In my spare time I'm working on a Box Shaped World tool set. The focus is mostly on tools that I need, weighing heavily on spatial statistics. I have a Ripley's K that is functioning quite well, and I think I've got a basic kernel density tool built. All the tools are built in VB.NET, for MapWindow 4.4 (and presumably 4.5 when that is released). I chose MapWindow for my project for a variety of reasons: supports .NET (the only programming language I know), has an editing environment for shapefiles (unfortunately it only supports shapefiles for vectors, but shapefiles are a universal format), extensive raster/grid support, and an apparent plan for future directions. Plus it is free! My tools will be free as well, but I don't think I'll release source code, at least not initially. I thought about sharpmap, but didn't go with it. I like sharpmap and hope it continues along its path. I didn't want to program a GIS interface, and there isn't much editing support from what I could gather. I guess (this may or may not be true) my impression is the MapWindow folk seemed to have it a little more together, but I'm not involved in the development process and I appreciate their efforts and time. Sharpmap seems to be completely volunteer based, too, where MapWindow is University based...Again, I like sharpmap and hope to use it in the future. Anyway, my tools will advance as I have time for them...

Wednesday, April 9, 2008

Google Earth and Charts Tutorial

This isn't really a tutorial in the proper sense of the term, but just a posted example of using a combination of KML, NetworkLink, ASP.NET generic handler (ashx), and Google Charts to create a dynamic symbolized map for Google Earth.

I started by downloading some 2000 census data in census block format, along with SF1 data, from ESRI's free download area. Then I brought it into Manifold selecting just a few census blocks for all of Fort Collins, Colorado, and created their inner centroids. I filtered some of the data that had 0 population from the SF1 table. Clearly, when/if you open this up you will find that I should have weeded a few more points, but on the other hand it shows how quick this runs. My server is pretty slow, and I tested this on a wireless internet connection that didn't have a very strong connection (plus I was streaming both Google Earth and some music). In Manifold, I related the two tables and exported it back out to a shapefile. On export I added two columns for lat and long.

In Visual Web Developer Express I created a generic handler (ashx). Instead of going into each line of code, I'll just summariz(s)e. Below is a link to a zip file where you can download both the KML file and the ASHX file. I used Sharpmap 0.9 to connect to the shapefile and read the table's contents. I just took the percentage of Males and Females (number of males / population, number of females / population) for my working statistic (truly groundbreaking stuff :)). You can see from the code that it basically just writes out the KML file, creating a Google Chart api url as it goes.

You can view the final product by opening this KML file in Google Earth. This KML file contains very little KML, and basically just has a Networklink back to the address of the handler.

If you want the files used, they are available here.

As always, any suggestions, comments, questions are much appreciated.

Here are some other examples I've found:
http://googlemapsapi.blogspot.com/2007/12/using-chart-api-in-kml-for-quick-data.html
http://www.barnabu.co.uk/interactive-spiders-and-charts/

EDIT:
Just noticed the Thematic Mapping Blog also has a post on using charts in Google Earth. Here is the link: http://blog.thematicmapping.org/2008/04/using-google-charts-with-kml.html
or with open layers http://blog.thematicmapping.org/2008/04/openlayers-and-google-chart-mashup.html

Tuesday, April 8, 2008

More Fun With Goolge Earth

This is a bit of a continuation with the previous blog entry. Instead of using the networklink I wanted to create something dynamic but portable. So basically, it consists of the same process of generating a kml file and images of pie charts, only this time the kml isn't written as a response but to the server. Then the whole folder is zipped up to a kmz including all the images. To do the zipping I used an opensource solution called #ziplib (sharpziplib) which worked very easily and only required a few lines of code (below).

Of course I'm not the first or last person to do this dynamic kmz creation with ASP.NET, and a brief search will produce essentially the same process using #ziplib. Although, I don't think many people or dynamically creating graphs at the same time (could be wrong, if so please correct). I really want to use Google charts to do this, because they have some nice looking charts, but due to some proxy server issues this became more hastle than it was worth. I think if I do a tutorial or something, I would use Google charts instead. Again this has probably been done somewhere.

Here is a fix to a problem I was running into. You need to set the size of the file before adding it as an entry: http://community.sharpdevelop.net/forums/p/6986/19897.aspx#19897

On a side note. I keep delving more and more into programming. I do think it is a necessity. I can handle working with vb.net, vba, javascript, and python. Definitely more of dabler than developer. I keep telling myself that I should start to learn C# to take things to a more advanced level. A lot of these language structures look similar to c#: javascript, python, java, j#, etc. At least to me anyway. But what can I say - I love VB. As I've read elsewhere, it is such a verbose language you can read it like a book. I just think it is so easy to use, and in a lot of ways so portable with VBA, VBScript (I've used this with dos to repeat creating a directory structure over and over again), VB.NET, VB6. I can use it in so many different environments. It's hard to resist the temptation to come back to it...and so C# will have to wait some more.

Here is a copy of the code I am using:

Private Sub ZipFolder(ByVal currentFolder As String)
Dim Filenames() As String = System.IO.Directory.GetFiles(OutputPath & currentFolder & "\")
Dim s As New ZipOutputStream(System.IO.File.Create(OutputPath & currentFolder & ".kmz"))
s.SetLevel(9)
Dim buffer() As Byte
ReDim buffer(4096)
For Each file As String In Filenames
Dim entry As ZipEntry
Dim fileInfo As New System.IO.FileInfo(file)
entry = New ZipEntry(fileInfo.Name)
entry.Size = fileInfo.Length
s.PutNextEntry(entry)
Dim fs As System.IO.FileStream = fileInfo.OpenRead
ReDim buffer(fileInfo.Length)
Dim sourceBytes As Integer = 1
Do Until (sourceBytes <= 0)
sourceBytes = fs.Read(buffer, 0, fileInfo.Length)
s.Write(buffer, 0, sourceBytes)
Loop
fs.Close()
fs = Nothing
sourceBytes = Nothing
entry = Nothing
Next
s.Finish()
s.Close()
Filenames = Nothing
s = Nothing
buffer = Nothing

End Sub

Monday, March 31, 2008

Mmmmmm Goooooogle Eaarrtth....

I have to admit, I didn't much bother with Google Earth when it first started making headlines (however many years ago that was). Sure like any Geonerd I downloaded and goofed around. "Oooh spinning globe (Wheeee!)." The imagery, of course, was stunning. It wasn't until recently that I've really found a use for it and started to delve into it. Now, I'm quite enamored with Google Earth.

The current project I'm working on is for the Department of Employment, Education, and Training....and water sports. The NT government has some good department names. I'm under the education department and there is currently a push to be able geolocate all the schools and associate this with their excellent data warehouse. The Northern Territory has some very remote communities (some still are lacking in power/electricity). I'm sure to fellow GISers it is obvious the need to be able to locate where your resources are going, and not going. A map is a simple way to do this. I'm the only GIS user working there and only on a part time basis at that, so it is important for everyone to be able to view the data when I'm not available to create a map. Their data warehouse is already running on SQL Server, and so I've been building my system on top of that (or next to). MapInfo was chosen as the GIS, and I'll describe my woes with this program later. I got into Google Earth, because another department had set up their on Google Earth server and had a site license for GE Enterprise. That's when I started to explore what we could do with it.

This is what I've come up with. I'm sure I'm not the first person to do this, but I thought I would at least share my methodology.

KML has a special section called network link. This allows the KML creator to instead of populating a file with the coordinates and geometry, to tie it to a web based server side script. This was extremely easy to do. I created a generic handler (ashx) in asp.net, and wrote a small class that created the response in the kml/xml format. The class pulled all of the coordinate information directly from the database server. Pretty cool to have the dynamic capability built into Google Earth. and only requiring a little server side scripting. Maybe 50 lines of code if that.

That's good, but it is just the location of schools. Obviously the description for the info window that pops up when the location is clicked can be populated with various data from the warehouse, but this might become a bit tedious to code. Instead I just placed a link to a profile page I had already set up. Unfortunately, you can't place an iframe in the window, because it isn't sophisticated enough to handle every scenario. No big deal.

What I really want to do is be able to display the data as a symbol. I could easily classify the symbols using some sort of nominal data stored in the database, or even some continuous data. I had originally created some maps with pie chart symbols in MapInfo showing the break-up of enrollment. I was asked if there was a way to click on the map and see what the numbers actually were. Well, this was a PDF, so the answer in that case was "no." I started to think if there was a way to do this with Google Earth, that would allow for the creation of a much more interactive environment. Using GDI+ and .NET, I was able to create a png image of a pie chart from the data in the warehouse, while programatically creating the kml file. Originally, I manually took this data and created a KMZ file. Then I started to use network links and tied my pie chart code in with my server side script. Now I have a dynamic kml file that creates pie charts on the fly as it is shown in Google Earth (images stored on the webserver). Now whenever it is opened up, it will reflect the current changes in the data warehouse. I plan on achieving the same effect with a proportional symbol. .NET has assemblies that allow me to compress data, so I could easily create a fixed kmz file based on this same method.

I would also like to do something similar using 3D. I attempted to create an extruded polygon based on the data. Unfortunately, these do not change their scale based on zoom. I looked around, but did not find anything. I'm thinking I could do this using regions. But at this point, I'm a little confused how regions work.

Anyway, it isn't difficult with a little programing to create a highly interactive map with Google Earth and ASP.NET.