Compile GDAL with GeoTiff support

I've been running Ubuntu on my personal machine for a while now and just realized that the gdal synaptic package does not enable support for GeoTiffs. This is where Gentoo has one up, but its not the end of the world. You can 'simply' build GDAL from source with the correct flags.

Here's what I did:
1. Install the Tiff, GeotTiff, Png and OpenJpeg dev packages and their dependencies (libtiff4-dev, libgeotiff-dev, libpng12-dev and libopenjpeg-dev). Also be sure to install g++ and subversion.
2. Fire up a terminal and checkout the GDAL subversion repository anywhere in your home directory:

svn co http://svn.osgeo.org/gdal/trunk/gdal

3. Go into the GDAL directory and run configure:

cd gdal
./configure --with-geotiff=/usr --with-jpeg=/usr --with-libtiff=/usr --without-libtool

4. In theory you should be able to run make now, but the configure script missed the geotiff header files directory so I had to set these manually. This is the error I received:

make -C gtiff install-obj
make[2]: Entering directory `/home/andreh/workspace/project-ossim_dev/gdal/frmts/gtiff'
/bin/sh /home/andreh/workspace/project-ossim_dev/gdal/libtool --mode=compile --tag=CXX g++ -g -O2  -Wall  -I/home/andreh/workspace/project-ossim_dev/gdal/port -I/home/andreh/workspace/project-ossim_dev/gdal/gcore -I/home/andreh/workspace/project-ossim_dev/gdal/alg -I/home/andreh/workspace/project-ossim_dev/gdal/ogr -I/home/andreh/workspace/project-ossim_dev/gdal/ogr/ogrsf_frmts   -DOGR_ENABLED -I/home/andreh/workspace/project-ossim_dev/gdal/port -I/usr/lib/include -I -I/include  -c -o ../o/geotiff.lo geotiff.cpp
libtool: compile:  g++ -g -O2 -Wall -I/home/andreh/workspace/project-ossim_dev/gdal/port -I/home/andreh/workspace/project-ossim_dev/gdal/gcore -I/home/andreh/workspace/project-ossim_dev/gdal/alg -I/home/andreh/workspace/project-ossim_dev/gdal/ogr -I/home/andreh/workspace/project-ossim_dev/gdal/ogr/ogrsf_frmts -DOGR_ENABLED -I/home/andreh/workspace/project-ossim_dev/gdal/port -I/usr/lib/include -I -I/include -c geotiff.cpp  -fPIC -DPIC -o ../o/.libs/geotiff.o
geotiff.cpp:34:21: error: xtiffio.h: No such file or directory
geotiff.cpp:35:21: error: geotiff.h: No such file or directory
geotiff.cpp:36:27: error: geo_normalize.h: No such file or directory
geotiff.cpp:37:23: error: geovalues.h: No such file or directory
geotiff.cpp:46: error: 'GTIF' was not declared in this scope
geotiff.cpp:46: error: expected primary-expression before ',' token
geotiff.cpp:46: error: 'GTIFDefn' was not declared in this scope
geotiff.cpp:46: error: expected primary-expression before ')' token
geotiff.cpp:46: error: initializer expression list treated as compound expression
geotiff.cpp:47: error: 'GTIF' was not declared in this scope
geotiff.cpp:47: error: expected primary-expression before ',' token
geotiff.cpp:47: error: expected primary-expression before 'const'
geotiff.cpp:47: error: initializer expression list treated as compound expression
geotiff.cpp: In destructor 'virtual GTiffDataset::~GTiffDataset()':
geotiff.cpp:2249: error: 'XTIFFClose' was not declared in this scope
geotiff.cpp: In member function 'void GTiffDataset::WriteGeoTIFFInfo()':
geotiff.cpp:3336: error: 'TIFFTAG_GEOPIXELSCALE' was not declared in this scope
geotiff.cpp:3346: error: 'TIFFTAG_GEOTIEPOINTS' was not declared in this scope
geotiff.cpp:3363: error: 'TIFFTAG_GEOTRANSMATRIX' was not declared in this scope
geotiff.cpp:3393: error: 'TIFFTAG_GEOTIEPOINTS' was not declared in this scope
geotiff.cpp:3404: error: 'GTIF' was not declared in this scope
geotiff.cpp:3404: error: 'psGTIF' was not declared in this scope
geotiff.cpp:3413: error: 'TIFFTAG_GEOKEYDIRECTORY' was not declared in this scope
geotiff.cpp:3420: error: 'TIFFTAG_GEODOUBLEPARAMS' was not declared in this scope
geotiff.cpp:3422: error: 'TIFFTAG_GEOASCIIPARAMS' was not declared in this scope
geotiff.cpp:3425: error: 'GTIFNew' was not declared in this scope
geotiff.cpp:3428: error: 'GTIFSetFromOGISDefn' cannot be used as a function
geotiff.cpp:3434: error: 'GTRasterTypeGeoKey' was not declared in this scope
geotiff.cpp:3434: error: 'TYPE_SHORT' was not declared in this scope
geotiff.cpp:3435: error: 'RasterPixelIsPoint' was not declared in this scope
geotiff.cpp:3435: error: 'GTIFKeySet' was not declared in this scope
geotiff.cpp:3438: error: 'GTIFWriteKeys' was not declared in this scope
geotiff.cpp:3439: error: 'GTIFFree' was not declared in this scope
geotiff.cpp: In static member function 'static GDALDataset* GTiffDataset::OpenDir(GDALOpenInfo*)':
geotiff.cpp:4285: error: 'XTIFFClose' was not declared in this scope
geotiff.cpp: In member function 'CPLErr GTiffDataset::OpenOffset(TIFF*, GTiffDataset**, toff_t, int, GDALAccess, int)':
geotiff.cpp:4640: error: 'TIFFTAG_GEOPIXELSCALE' was not declared in this scope
geotiff.cpp:4647: error: 'TIFFTAG_GEOTIEPOINTS' was not declared in this scope
geotiff.cpp:4659: error: 'TIFFTAG_GEOTRANSMATRIX' was not declared in this scope
geotiff.cpp:4700: error: 'TIFFTAG_GEOTIEPOINTS' was not declared in this scope
geotiff.cpp:4724: error: 'GTIF' was not declared in this scope
geotiff.cpp:4724: error: 'hGTIF' was not declared in this scope
geotiff.cpp:4725: error: 'GTIFDefn' was not declared in this scope
geotiff.cpp:4725: error: expected `;' before 'sGTIFDefn'
geotiff.cpp:4730: error: 'GTIFNew' was not declared in this scope
geotiff.cpp:4739: error: 'sGTIFDefn' was not declared in this scope
geotiff.cpp:4739: error: 'GTIFGetDefn' was not declared in this scope
geotiff.cpp:4741: error: 'GTIFGetOGISDefn' cannot be used as a function
geotiff.cpp:4748: error: 'GTRasterTypeGeoKey' was not declared in this scope
geotiff.cpp:4749: error: 'GTIFKeyGet' was not declared in this scope
geotiff.cpp:4751: error: 'RasterPixelIsPoint' was not declared in this scope
geotiff.cpp:4757: error: 'GTIFFree' was not declared in this scope
geotiff.cpp: In static member function 'static TIFF* GTiffDataset::CreateLL(const char*, int, int, int, GDALDataType, char**)':
geotiff.cpp:5707: error: 'XTIFFClose' was not declared in this scope
geotiff.cpp: In static member function 'static GDALDataset* GTiffDataset::CreateCopy(const char*, GDALDataset*, int, char**, int (*)(double, const char*, void*), void*)':
geotiff.cpp:6217: error: 'TIFFTAG_GEOPIXELSCALE' was not declared in this scope
geotiff.cpp:6226: error: 'TIFFTAG_GEOTIEPOINTS' was not declared in this scope
geotiff.cpp:6242: error: 'TIFFTAG_GEOTRANSMATRIX' was not declared in this scope
geotiff.cpp:6279: error: 'TIFFTAG_GEOTIEPOINTS' was not declared in this scope
geotiff.cpp:6301: error: 'GTIF' was not declared in this scope
geotiff.cpp:6301: error: 'psGTIF' was not declared in this scope
geotiff.cpp:6303: error: 'GTIFNew' was not declared in this scope
geotiff.cpp:6304: error: 'GTIFSetFromOGISDefn' cannot be used as a function
geotiff.cpp:6310: error: 'GTRasterTypeGeoKey' was not declared in this scope
geotiff.cpp:6310: error: 'TYPE_SHORT' was not declared in this scope
geotiff.cpp:6311: error: 'RasterPixelIsPoint' was not declared in this scope
geotiff.cpp:6311: error: 'GTIFKeySet' was not declared in this scope
geotiff.cpp:6314: error: 'GTIFWriteKeys' was not declared in this scope
geotiff.cpp:6315: error: 'GTIFFree' was not declared in this scope
geotiff.cpp:6325: error: 'XTIFFClose' was not declared in this scope
make[2]: *** [../o/geotiff.lo] Error 1
make[2]: Leaving directory `/home/andreh/workspace/project-ossim_dev/gdal/frmts/gtiff'
make[1]: *** [gtiff-install-obj] Error 2
make[1]: Leaving directory `/home/andreh/workspace/project-ossim_dev/gdal/frmts'
make: *** [frmts-target] Error 2

And this is how to get rid of it:

# open the GDALmake.opt file with an editor
gedit GDALmake.opt

# locate the CPPFLAGS line and add the include path at the end "-I/usr/include/geotiff"
CPPFLAGS    =  -I$(GDAL_ROOT)/port -I/usr/lib/include -I -I/include -I/usr/include/geotiff

5. One can now run make and make install:

make
sudo make install

Limited Linq to Sql

I've been playing around (ie pulling my hair out) in my spare time over the last few weeks, trying to make Linq more loosely coupled with a UI and to extend its capabilities to allow for many-to-many table updates. I find it completely rediculous how intricate Ms has made all of this. Perhaps I'm missing the plot, but unless you're trying to do some really simple little application that requires little to no flexibility on the DB side of things, STAY AWAY FROM LINQ.

I wrote my own Linq a few years ago, but decided to give Ms Linq a chance and it hasn't really been smooth sailing. The most enjoyable part of the experience was dragging my tables into the dbml window and seeing VS create all the classes for me. The rest is a friggin nightmare.

I'm not about to give up the fight though, but I'm pretty distraught at the moment.

TexnicCenter DDE Commands for Acrobat Reader

Not really a dev related post, but I had to re-install Vista due to an epic fail on MS's part. My pc went into sleep mode and wrote the files from the C:\ProgramData\Microsoft directory to the C:\ProgramData\Microso~1 dir.

If this ever happens to you, I'm sorry to say you'll have to re-install your OS. I tried everything to get the files back to where they were supposed to be, but it looks like MS tightened security on FS and its impossible to move some of the Crypto files to where they belong. Even if you boot up in Linux...

Anyhow, I have to get latex running again and I'm using TexnicCenter as an editor. I forgot the DDE commands and had to figure it out again. Joy.

Here's the basic setup:

1. In TexnicCenter, hit Alt+F7 and click on the viewer tab.
2. In the "View project output" and "Forward search" options, select the "DDE Command" radio button and fill in the following: Command = [DocOpen("%bm.pdf")][FileOpen("%bm.pdf")], Server = acroview, Topic = control
3. For the "Close document before running Latex" , select DDE with the same settings as above, but change the command to be [DocClose("%bm.pdf")]

One can also fool around with some other commands. Here's the snippet on DDE from the Adobe Interapplication Communication document:

Acrobat Application DDE Messages
This section lists all DDE messages. For complete descriptions of the parameters associated with DDE messages, see the DDE sections of the Acrobat Interapplication Communication Reference.

Application Configuration

• AppExit — Exits Acrobat.
• AppHide — Iconifies or hides Acrobat.
• AppShow — Shows Acrobat.
• CloseAllDocs — Closes all open documents.
• HideToolbar — Hides the toolbar.
• MenuitemExecute — Invokes a menu item, given its language-independent name.
• ShowToolbar — Shows the toolbar.

Document Manipulation

• DocClose — Closes the file without saving it and without prompting the user to save the document if it has been modified.
• DocDeletePages — Deletes a specified range of pages in a document. It cannot delete all pages in a document.
• DocInsertPages — Inserts specified pages from one file into another.
• DocOpen — Opens a document and adds it to the list of documents known to DDE, allowing it to be manipulated by other DDE messages (for example, FileOpen).
• DocReplacePages — Replaces specified pages using pages from another file.
• DocSave — Saves the specified file.
• DocSaveAs — Saves an open file into a new file, without warning the user if there is a problem saving.
• DocSetViewMode — Controls whether bookmarks or thumbnail images are shown in addition to the document content.
• FileOpen — Opens and displays a file, making it the current document and bringing it to the front if it is already open.
• FileOpenEx — Opens and displays a file, making it the current document and bringing it to the front if it is already open. The file is opened during an idle loop to allow DDE messages to continue flowing during the opening of large documents.

Document Printing

• DocPrint — Prints a specified range of pages from a document, without displaying a modal Print dialog box to the user.
• FilePrint — Prints all pages in a document, displaying a modal Print dialog box to the user.
• FilePrintEx — Prints all pages in a document, displaying a modal Print dialog box to the user. Only PostScript Level 1 operators are used for PostScript printing. Printing is performed during an idle loop to allow DDE messages to continue flowing during the printing of large documents.
• FilePrintSilent — Prints all pages in a document, displaying no print dialog box to the user.
• FilePrintSilentEx — Prints all pages in a document, displaying no print dialog box to the user. Only PostScript Level 1 operators are used for PostScript printing. Printing is performed during an idle loop to allow DDE messages to continue flowing during the printing of large documents.
• FilePrintTo — Prints all pages in a document to a specified printer, using a specified driver and port, displaying a modal Print dialog box to the user.
• FilePrintToEx — Prints all pages in a document to a specified printer, using a specified driver and port, displaying a modal Print dialog box to the user. Only PostScript Level 1 operators are used for PostScript printing. Printing is performed during an idle loop to allow DDE messages to continue flowing during the printing of large documents.

View Manipulation

• DocGoTo — Goes to the specified page.
• DocGoToNameDest — Goes to the specified name destination within the document.
• DocPageDown — Scrolls forward through the document by one screen area.
• DocPageLeft — Scrolls to the left by a small amount.
• DocPageRight — Scrolls to the right by a small amount.
• DocPageUp — Scrolls backward through the document by one screen area.
• DocScrollTo — Scrolls the view of the current page to a specified location.
• DocZoomTo — Sets the zoom for a specified document.

Search-related

• DocFind — Finds a string in a specified file.

Python, GDAL: Set JPEG quality values and other att

Here I shall show how to set the quality values of the JPEG format in GDAL. This post will build upon this previous article which describes how to create a basic JPEG.

It is pretty trivial to set the quality (and other) JPEG values. Here is a list of parameters one can set for the JPEG format. These parameters are simply passed to the CreateCopy() method as a list of options. Here's an example:

# ... Some other code ... #

if (cmp(output_file_ext.lower(),".jpg") == 0 or cmp(output_file_ext.lower(),".jpeg") == 0):
  # Add options
  options = []
  options.append("QUALITY=90")

  jpg_driver = gdal.GetDriverByName("JPEG")

  # Pass options as parameter
  jpg_driver.CreateCopy( output_file, dst_ds, options=options )
  os.remove(output_file_tmp)

Python, GDAL: Adding GeoTiff Meta Data

I'm pretty much a noob when it comes coordinate systems and projections. I only know the standard GPS coordinates which are in the latitude/longitude (lat/long) format. It turns out that this system is known as the WGS84 (World Geodetic System) format.

The project I'm busy with at the moment, generates GeoTiff images and I only had a raster with the bottom lower left corner coordinates and the pixel sizes in km. It also happened that this is the only information I needed to generate the GeoTiff.

So this is the info I have:

ncols         174
nrows         115
xllcorner     14.97
yllcorner     -34.54
cellsize      0.11

It's just the number of x and y pixels, the lower left corner and the pixel size.

Now we can go ahead and generate the GeoTiff like so:

if __name__ == '__main__':
# Import libs
import numpy, os
from osgeo import osr, gdal

# Set file vars
output_file = "out.tif"

# Create gtif
driver = gdal.GetDriverByName("GTiff")
dst_ds = driver.Create(output_file, 174, 115, 1, gdal.GDT_Byte )
raster = numpy.zeros( (174, 115) )

# top left x, w-e pixel resolution, rotation, top left y, rotation, n-s pixel resolution
dst_ds.SetGeoTransform( [ 14.97, 0.11, 0, -34.54, 0, 0.11 ] )
  
# set the reference info 
srs = osr.SpatialReference()
srs.SetWellKnownGeogCS("WGS84")
dst_ds.SetProjection( srs.ExportToWkt() )

# write the band
dst_ds.GetRasterBand(1).WriteArray(raster)

Easy as pie with GDAL.

Grep and punctuation marks

I've often tried to use grep to filter search results for locate and several times I've had to search for a string with locate:

locate svn

This search would obviously include all the '.svn' folders too. Trying to get rid with them with

locate svn | grep -v '.svn'

does not work, since grep uses basic regular expressions, and the period mark will simply match any character.

To explicitly indicate the usage of a punctuation mark, one has to put it in brackets. The statement will now look something like:

locate svn | grep -v '[.]svn'

I just scanned through the grep man pages, and everything is explained pretty clearly there. I'd recommend reading the bit on regular expressions.

MySQL and the joys of security

I've been running my local MySQL server on Gentoo without hassles for a while and recently tried to connect to it from a remote machine. Obviously there was bound to be some issues and here's how I solved them.

When you first try to connect remotely to the machine, you'll be greeted by this lovely message:

# mysql --user=root --password=some_password --host=192.168.0.1 MYDB
ERROR 2003 (HY000): Can't connect to MySQL server on '192.168.0.1' (111)

To get this sucker out of the way, edit your /etc/mysql/my.cnf file and comment the following line:

# bind-address = 127.0.0.1

Note to all the security girls: Your server will now be accessible from any other networked machine. Oooohh nooooo!!

Anyhow, now we try to connect again, and whooopa!! Another error:

mysql --user=root --password=some_password --host=192.168.0.1 MYDB
ERROR 1130 (00000): Host 'remote_pc_hostname' is not allowed to connect to this MySQL server

Ah bite me. Ok, so you have to grant his Lordship permissions to connect to thy server. In order to do that logon to sql on the sql server itself:

# mysql --user=root --password=some_password
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 11
Server version: 5.0.70-log Gentoo Linux mysql-5.0.70-r1

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> GRANT ALL ON *.* TO 'root'@'remote_pc_hostname';
Query OK, 0 rows affected (0.00 sec)

Now all should be well in fairyland, but alas. The code securo freaks over at Sun have presented ye with another trial.

# mysql --user=root --password=some_password --host=192.168.0.1 MYDB
ERROR 1045 (28000): Access denied for user 'root'@'remote_pc_hostname' (using password: YES)

Ok I'm taking the fairytale rubbish too far. Here's what you should've done on the sql server in the first place:

# mysql --user=root --password=some_password
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 11
Server version: 5.0.70-log Gentoo Linux mysql-5.0.70-r1

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> GRANT ALL ON *.* TO 'root'@'remote_pc_hostname' IDENTIFIED BY 'some_password';
Query OK, 0 rows affected (0.00 sec)

Sheesh! You little girls are wasting my time. Take a que from MS SQL Server.