Conference

I will be presenting at the American Association of Geographers on Sunday 22 March 2009.

Move

I've started to post at the new blog location.

My apologies for any inconveniences. This site will remain up for the timebeing, and I don't have any plans to delete it.

http://www.boxshapedworld.com/blog

What is Twitter?

I'm not quite sure what twitter is, but I signed up for it, because everyone is doing it.

http://twitter.com/boxshapedworld

I don't really say anything useful, but for those who have read the entries on this blog, you already knew this. Anyway, I'm always interested in what people have to say about the spatial world (is that redundant?) in 140 characters or less.

For the Skiers

Being from Colorado originally, I always get asked if I ski. Well, yes and no. I used to downhill ski, but before I left I tried and really enjoyed cross country skiing. Either way, I still find these interactive maps pretty cool.






*Can't seem to get anything but the Park City map to load. Probably a conspiracy by Utah to steal Colorado's glory, but maybe you'll have more luck :).

Speaking of software...

That never changes its interface.

Just received an announcement from Clark Labs. They just released the latest incarnation of IDRISI software called Taiga. If you are looking for a low cost Image processing software, or raster based GIS, this should be one that you look at.

More APIs

So, when will we see the first Google Maps/New York Times Article mashup?

http://open.blogs.nytimes.com/2009/02/04/announcing-the-article-search-api/
http://www.readwriteweb.com/archives/nytimes_exposes_huge_api.php

GIS Jobs and Blog News

While I hope that you are not one of the thousands/tens of thousands of people that have lost their job in the last year, I thought I would post this on here. It's hard to tell what it is really like in the U.S. being in Australia, and Darwin for that matter (which, cross fingers, doesn't seem to be hit too hard at the moment). If you are one of the unfortunate few, or perhaps wanting to find another job, this might interest you.

There are a handful of GIS specific job websites out there, but I think the best one is actually a general job site called indeed.com. You can search for all the U.S. or by location. The feature I like best about it is that you can subscribe to a search and get updated automatically in your favorite reader (e.g. google reader).

=============================

On a different note. I've decided to revamp the blog and tutorials section by migrating over to my main web site. I've been neglecting that site quite a bit, and it really just became a testing ground for web projects and learning projects. And I'll face it, I'm a bit of a google fan'person' and they made it really easy to blog and create webpages without having to set up html and css and the like. On the other hand I don't want to be dependent on google (for example they are getting rid of google pages), and they don't make it easy to get my work back. I think it will be better to have everything on one site, though. Luckily I set up the feeds for this blog through feedburner and hopefully I can just switch the rss over and anyone subscribed won't notice a difference.

Venturing deeper in to the murky waters of the Geoweb...

or Neogeography...or Web 2.0...or whatever you want to call it...

Basically to keep up with the ever changing world of the GIS Analyst/Geographer, I've set upon myself the tiny minor little task of learning how to create Rich Internet Applications (RIA). I already know a little about AJAX, and have used it for small things. For those that don't know what AJAX is, it stands for Asynchronous Javascript and XML. Basically it is a way for a webpage to stay put, and the content to change. Whereas before to change the content you had to go to another webpage, this really slows things down, and isn't a good user experience. So AJAX talks to the server via Javascript and the server sends back information in XML format, and Javascript parses that info. The advantage of this process is that doesn't require any third-party extensions to get a Flashy (Adobe Flash that is) experience. I don't particularly like programming in Javascript. And in some instances you have to set up your code depending on browser the user is using, because not each browser is the necessarily the same.

So along comes Microsoft and their new product called Silverlight. The so-called "Flash-Killer." Well, it's not going to kill Flash, but some healthy competition doesn't hurt anyone. The appeal was that I could program in VB.NET to create the RIA, and it could all be developed using the Web Developer 2008 Express Edition. Develop Flash-like apps for free? Sounds better than 600$ for the cost of Adobe Flash software. The disadvantage is that Silverlight has 1% of the market to Flash's 98% (and an extra 1% for all the people still using Netscape Navigator 4), so no one really has the plugin to view the application. Forcing people to download a plug-in leaves me with a bad taste in my mouth (hence the appeal of AJAX). Granted by the time everyone upgrades to Windows 7, Silverlight will be pre-installed.

Now I'm not just doing this for fun, but it is related to an actual project. The project lead has a Mac, and Silverlight plugins aren't available for the mac (they are now). So I switched my attention to Adobe Flash, and learned about Adobe Flex. Flex is for developers (or dabblers like me) and Flash is for designers, to put it succinctly. There is a lot of overlap between them, such as the main language - ActionScript. Also, Flex 3.0 SDK is free, so you can, in theory, create Flex apps for free. They also sell Flex Builder (an IDE), or as a 60 day trial. Since I'm at an academic institution, they also provide it for free.

I was a little daunted at trying to learn another programming language (or half-learning like I usually do), but I've been pleasantly surprised by the whole experience. Flex Builder is just ok as an IDE, I much prefer working in Web Developer Express (it has much better intellisense), but I'm also used to working in it. I think that in general Flex is easier to work with than Silverlight because it has the documentation and community support/examples that Silverlight doesn't (yet?). I have a copy of Adobe Flex 3 Bible by Gassner and that is helping quite a bit. Flex also has a lot of built-in functions to make typical tasks much easier, like fading in or out. I found Silverlight's animation less intuitive. An extra bonus is that Google released a Google Maps control for flash/flex. There is a Virtual Earth wrapper for Silverlight that is really good to work with, but Google Maps has better imagery in the project area.

Anyway, if you've wanted to start working with RIA's, I think I would recommend Flex first, then AJAX, then Silverlight.

Here is a beta version of the project I've been working on. I attempted to set up the Flex front end to be generic so that all the content is received via RSS, so in theory I could just switch urls to a different RSS that read a different database. Of course, the code is a little sloppy as it was a learner project, but I should be able to clean it up for the final incarnation. There are little buggy problems with this, but I wanted to get a test up on the web. All the markers are stored as geometry in a SQL Server 2008 database and put into a GeoRSS so that it's readable by Flex. The "media" is stored in a separate table with each item using a foreign key reference to the spatial data, so that they are "geotagged." This way, as the database is updated so is the little Flex program. In the beta version of the blogspot engine, they have geotagging, so hopefully I'll be able to add blog entries as media content as well. I could try that, but then would need to add the entries to the media content table. So far images and videos are stored in a bucket on my host's server, with the link stored in the table. Any feedback or suggestions would be appreciated.

Google Analytics

Hasn't really been a year since this blog was started, but the year has ended. Shortly after starting it I tracked it with Google Analytics which is free and pretty easy to embed in the blog. I've had 1,700 unique visitors and 2,100 visits from 108 countries. The top country is the U.S.A followed by Australia (which is probably me logging into the site :)), then the UK. Which makes sense since this is an English-language blog. I wouldn't mind posting in another language, but I don't know any sufficiently to do that. Anyway, thanks to all that read this blog. I hope it is of some use to you, because I mainly post here to give back to the Internet community. I have learned much from that community, so if I can contribute even a little that makes me happy.

Minard

This is perhaps my favorite illustration. It is by Charles Joseph Minard and shows Napoleon's invasion of Moscow, Russia and the subsequent retreat to Poland. It is a space-time illustration, using multivariate data. Edward R. Tufte (author the Visual Display of Quantitative Data) suggests it is the greatest visualization ever made. It is very engaging, and simple to read (sorry but this one is in French, there are English variants).

Hello

Been traveling in Southeast Asia for the holidays. Will hopefully post something here in a couple of days when I get settled in again.

Camels

Well, the final report on the Camel project has officially been released. Here is a link where you can download it. There is a separate report (54) for the stuff I worked on. THis was incorporated into chapter 11 of the main report. Essentially it is a multi-criteria evaluation using weighted linear combination. There is python code in the appendix for performing (modularly) a weighted linear combination, including calculating weights. To calculate the weights you need NumPy.

Google Earth In Marketing

Was in Brisbane last week and noticed a new marketing campaign for Qantas. Here is a picture of one their posters. Map pushpins have been around since the seventies at least, that is according to old cop movies. But I think the notion of a pushpin being put into 3-dimensional space is really influenced by Google Earth. Perhaps I can even coin a term - ge-marketing. :)

Cool

I do have interests outside of GIS and Geography. One is movies...It's great when the two meet. Here is a really beautiful information visualization of movie revenues for the past 30 years. To me it looks the way oil paint can layer together. Very well made.

Correction Manifold and SQL Server 2008

I misunderstood about Manifold and SQL Server 2008. It sounds like you actually need the Enterprise edition of the product to use the native spatial data type...A point for MapInfo on this one, minus 1 for Manifold....If anyone is keeping a tally.

Tutorials

My tutorials page is fairly dormant, which I apologize for. I'd like to add to it, but I've been busy. Also, the page is hosted on Google Pages, and it has been announced that Google Pages will be discontinued and migrated to Google Sites. I think having the tutorials on Google Sites will actually be better, but I'm not sure when this "migration" will actually happen. I just hope that I don't lose any of that work...

You've Got the Points - Now What?

So in the last post I showed how I got some points into SQL Server 2008. Not the most elegant bit of code (I'm a dabbler not a developer), but it worked. Now what can you do with it. Besides the many Transact SQL (Microsoft own brand of SQL) methods available for interacting with spatial data (see OGC ones here), you could create your own GeoRSS feed. I'm fairly new to creating RSS feeds, so this might not be exactly correct, but you may find the query useful. The key here is adding a reference to Microsoft.SqlServer.Types. Then you can actually use their geometry type and the methods associated with it to retrieve your results.

This is all in an generic handler (*.ashx). The Imports:

Imports System
Imports System.Web
Imports System.Xml
Imports System.Data
Imports System.Configuration
Imports System.Collections
Imports System.Web.Security
Imports System.Web.UI
Imports System.Web.UI.WebControls
Imports System.Web.UI.WebControls.WebParts
Imports System.Web.UI.HtmlControls
Imports System.Data.SqlClient
Imports System.Net
Imports System.Threading
Imports Microsoft.VisualBasic
Imports System.IO
Imports Microsoft.SqlServer.Types

And the Class:



Public Class geo_content : Implements IHttpHandler
  
   Public Sub ProcessRequest(ByVal context As HttpContext) Implements IHttpHandler.ProcessRequest

      
       context.Response.ClearHeaders()
       context.Response.Clear()
       context.Response.ContentType = "text/xml"
       context.Response.ContentEncoding = System.Text.Encoding.UTF8
       context.Response.AddHeader("ContentType", "text/xml; charset=utf-8")
      
       Dim xmlWriter As New XmlTextWriter(context.Response.Output)
       xmlWriter.WriteStartDocument()
       xmlWriter.WriteStartElement("rss")
       xmlWriter.WriteAttributeString("version", "2.0")
       'xmlWriter.WriteStartElement("rdf:RDF", "http://www.w3.org/1999/02/22-rdf-syntax-ns#")
       'xmlWriter.WriteAttributeString("xmlns", "rdf", Nothing, "http://www.w3.org/1999/02/22-rdf-syntax-ns#")
       xmlWriter.WriteAttributeString("xmlns", "geo", Nothing, "http://www.w3.org/2003/01/geo/wgs84_pos#")
       xmlWriter.WriteStartElement("channel")
       xmlWriter.WriteElementString("title", "geoContentRSS")
       xmlWriter.WriteElementString("description", "latitude and longitude from sql server 2008")
       xmlWriter.WriteElementString("language", "en-US")

       Dim con As SqlConnection = Create_Connection()

       If con.State = ConnectionState.Closed Then
           con.Open()
       End If

       Dim geoReader As SqlDataReader = Create_Geo_reader(con)
       Try
           Dim q As String = """"
           If Not IsNothing(geoReader) Then
               If geoReader.HasRows Then
                   Do While geoReader.Read
                       xmlWriter.WriteStartElement("item")
                  
                       Dim geom As New SqlGeometry
                       geom = CType(geoReader("Geom_Data"), SqlGeometry)
                       Dim lat As Double = CType(geom.STY, Double)
                       Dim lng As Double = CType(geom.STX, Double)
                       Dim id As Integer = CType(geoReader("GeoID"), Integer)
                       Dim name As String = CType(geoReader("Name"), String)
                  
                       xmlWriter.WriteStartElement("title")
                       xmlWriter.WriteCData(name)
                       xmlWriter.WriteEndElement()

                       xmlWriter.WriteStartElement("description")
                       xmlWriter.WriteCData(Name)
                       xmlWriter.WriteEndElement()

                       xmlWriter.WriteElementString("link", "http://www.boxshapedworld.com")
                       xmlWriter.WriteElementString("id", id)

                       xmlWriter.WriteStartElement("source")
                       xmlWriter.WriteAttributeString("url", "http://www.boxshapedworld.com")
                       xmlWriter.WriteString(name)
                       xmlWriter.WriteEndElement()
                      
                       xmlWriter.WriteElementString("geo:lat", CStr(lat))
                       xmlWriter.WriteElementString("geo:lng", CStr(lng))
                      
                       'close item element
                       xmlWriter.WriteEndElement()
                   Loop
               End If
           End If

           If con.State = ConnectionState.Open Then
               con.Close()
           End If

       Catch ex As Exception
           MsgBox(ex.ToString)
           con.Close()
       End Try

       xmlWriter.WriteEndElement()
       xmlWriter.WriteEndElement()
       xmlWriter.WriteEndDocument()
       xmlWriter.Close()
   End Sub

   Public ReadOnly Property IsReusable() As Boolean Implements IHttpHandler.IsReusable
       Get
           Return False
       End Get
   End Property

   Private Function Create_Connection() As SqlConnection
       Try
           Dim Connection As New SqlConnection(System.Configuration.ConfigurationManager.ConnectionStrings("baseBSWstr").ConnectionString)
           Return Connection
       Catch ex As Exception
           MsgBox(ex.ToString)
           Return Nothing
       End Try
   End Function
  
   Private Function Create_Geo_reader(ByVal Connection As SqlConnection) As SqlDataReader
       Dim programReader As SqlDataReader
       Dim sqlStatement As String
       Dim command As New SqlCommand
       Dim dt As New DataTable("geograph_data")
      
      
       sqlStatement = "SELECT * FROM geograph_data WHERE [GeoID] <> @negGeoID ORDER BY [Name]"
      
      
       Try
           If Connection.State = ConnectionState.Closed Then
               Connection.Open()
           End If

           With command
               .CommandText = sqlStatement
               .Parameters.AddWithValue("@negGeoID", -9999)
               .CommandType = CommandType.Text
               .Connection = Connection
           End With

           programReader = command.ExecuteReader
           Return programReader
       Catch ex As Exception
           MsgBox(ex.ToString)
           Connection.Close()
           Return Nothing
       End Try
      
   End Function
End Class

This starts by creating an xmltextwriter that is set to the context output stream. I add the xml namespace for geo. There is a separate function to create the connection string, and a separate function to create the sqldatareader. In this function you see a simple SELECT statement with parameters for the where value. This is returned and the geoReader starts to cycle through each row. A variable called geom is used to house the geometry, and you can see that it is of a type SqlGeometry. Using this type gave access to the STY and STX methods. Again, not the most elegant bit of code, but it works.

SQL Server 2008 Express

I've finally gotten a chance to play around with SQL Server 2008 express edition. In case you haven't heard 2008 introduced native spatial support. I'm not sure how the internals store spatial data but you can add geometry (or geography) data types as Well-Known Text or Well-Known Binary. Well-Known Text is pretty easy to understand and work with. It's pretty easy to work with SQL Server using the Management Studio Express Edition. If you have MapInfo 9.5, or Manifold version 8 you can directly connect to express edition and edit geometry that way. I don't think there is support for this in ArcGIS 9.3 without SDE (personal or otherwise) which means you probably need an ArcEditor license. I happened to have an XML file with latitude and longitude locations in it that I used to store locations for a mini (personal) google maps project. Using VB.NET 3.5 I could easily connect to both the xml file and sql server to add the geometry as point locations.

Here is the code that I used. It is server side code, with the connection string stored in the web.config file.



    Dim xDoc As New XmlDocument
    xDoc.Load(Server.MapPath("cities.xml"))
    Dim cNodes As XmlNodeList
    cNodes = xDoc.GetElementsByTagName("marker")
    Dim i As Integer = 0
    Dim con As Data.SqlClient.SqlConnection
    con = New Data.SqlClient.SqlConnection(ConfigurationManager.ConnectionStrings _
("baseBSWstr").ConnectionString)
    For Each XNode As XmlNode In cNodes
        Dim ac As XmlAttributeCollection = XNode.Attributes

        Dim lat As Double = CDbl(ac.Item(0).InnerText.ToString)
        Dim lng As Double = CDbl(ac.Item(1).InnerText.ToString)
        Dim name As String = CStr(ac.Item(2).InnerText.ToString)

        con.Open()
        Dim cmd As New SqlCommand("INSERT INTO geograph_data
VALUES (" & i & "," & "geometry::STGeomFromText('POINT
(" & lng & " " & lat & ")', 0), '" & name & "')", con)
        'cmd.CommandText = "INSERT INTO geograph_data
VALUES (" & i & "," & "geometry::STGeomFromText('POINT
(" & lat & " " & lng & ")', 0), '" & name & "')"
        cmd.ExecuteNonQuery()

        con.Close()
        i += 1

    Next

This was just run on my hard drive with a local instance, so I wasn't too concerned about security. From what I understand using parameters is meant to be more secure. Here is an example.

Now if you want to play around with sql server 2008 with just management studio here are some tutorials.
Boston GIS
JasonFollas - this describes the difference between geometry and geography...pretty good series.
Developer Fusion
MSDN

Book Recommendation

A colleague let me glance over this book by Chang. I have to say it is one of the better introductory GIS texts I have seen. I've owned a few. The one for my undergrad courses was weak, but in fairness it was one of the least expensive textbooks I had to purchase. Longley et al. produced a nice one as well. I would go with Chang over Longely, though. Chang's book is ESRI-biased, but I think it is of value to everyone. There are a number of worked examples that show how the little black box works, e.g. affine transformations, and ordinary kriging. I'm not talking about point and click, I mean actual mathematics. My colleague also said the section on modeling is excellent, but I didn't look through it in great detail.

Unfortunately the book budget is a little low right now, as I've made a number of purchases recently so I don't own a copy. Speaking of which, I also recommend Bivand et al.'s spatial statistic book for R. This fills a void in Spatial Statistic books that has been growing. Most spatial stat books focus heavily on theoretical, which is fine, but for someone like me that is not in a course there is a lack of worked examples. Waller and Gotway's book is theoretical and has exercises, but now answer key so who knows if I produced the "correct" results....Andy Mitchell's book is a great start, but doesn't go into much depth (doesn't talk much about first and second order effects, doesn't talk about inhomgeneous k-functions, etc...)

Find values in one table that are missing in another table MapInfo

Took me a little while to find an example of this, so I thought I would add one to the "interweb" to help other people.

One part of MapInfo that is extremely powerful is the SQL functionality. It isn't comprehensive, but there are a number of things you can do with it. Where I'm currently living/working (The Northern Territory) has a number of communities that could have several different names for the same location. This comes from communities named by European settlers in the area, and Indigenous names. Basically, you can have a spreadsheet of data that you want to tie to a geographic location via the name of an organisation, but the spreadsheet names might not necessarily match the database names that have the latitude and longitude. So I want to find out which values in the spreadsheet are "missing" from the database. In other words, which values did not join.

Two Tables: sdss_geography and tutorial_sample_2
Two Columns: sdss_geography.Organisation_Name and tutorial_sample_2.School

The first step is to perform a Join on the two tables. Using Query --> SQL Select

Select Columns: *
From Tables: sdss_geography, Tutorial_Sample_2
Where: sdss_geography.Organisation_Name = Tutorial_Sample_2.School
Ordered By: School
into Table Named: InitialJoin

Next go to File --> Save Query and save the InitialJoin query as a table. Then close the query table and load the InitialJoin.tab table.

Once the table is loaded, go back to Query --> SQL Select.

Select Columns: *
From Tables: Tutorial_Sample_2
Where: not School in (select School from InitialJoin)
Ordered By: School
into Table Named:MissingJoin

Click OK.

This produces a table of missing values. I take this table and make sure the name and spelling matches the central database.

Hopefully that helps someone else.