Python, GDAL: Set JPEG quality values and other att

Here I shall show how to set the quality values of the JPEG format in GDAL. This post will build upon this previous article which describes how to create a basic JPEG.

It is pretty trivial to set the quality (and other) JPEG values. Here is a list of parameters one can set for the JPEG format. These parameters are simply passed to the CreateCopy() method as a list of options. Here's an example:

# ... Some other code ... #

if (cmp(output_file_ext.lower(),".jpg") == 0 or cmp(output_file_ext.lower(),".jpeg") == 0):
  # Add options
  options = []
  options.append("QUALITY=90")

  jpg_driver = gdal.GetDriverByName("JPEG")

  # Pass options as parameter
  jpg_driver.CreateCopy( output_file, dst_ds, options=options )
  os.remove(output_file_tmp)

Python, GDAL: Adding GeoTiff Meta Data

I'm pretty much a noob when it comes coordinate systems and projections. I only know the standard GPS coordinates which are in the latitude/longitude (lat/long) format. It turns out that this system is known as the WGS84 (World Geodetic System) format.

The project I'm busy with at the moment, generates GeoTiff images and I only had a raster with the bottom lower left corner coordinates and the pixel sizes in km. It also happened that this is the only information I needed to generate the GeoTiff.

So this is the info I have:

ncols         174
nrows         115
xllcorner     14.97
yllcorner     -34.54
cellsize      0.11

It's just the number of x and y pixels, the lower left corner and the pixel size.

Now we can go ahead and generate the GeoTiff like so:

if __name__ == '__main__':
# Import libs
import numpy, os
from osgeo import osr, gdal

# Set file vars
output_file = "out.tif"

# Create gtif
driver = gdal.GetDriverByName("GTiff")
dst_ds = driver.Create(output_file, 174, 115, 1, gdal.GDT_Byte )
raster = numpy.zeros( (174, 115) )

# top left x, w-e pixel resolution, rotation, top left y, rotation, n-s pixel resolution
dst_ds.SetGeoTransform( [ 14.97, 0.11, 0, -34.54, 0, 0.11 ] )
  
# set the reference info 
srs = osr.SpatialReference()
srs.SetWellKnownGeogCS("WGS84")
dst_ds.SetProjection( srs.ExportToWkt() )

# write the band
dst_ds.GetRasterBand(1).WriteArray(raster)

Easy as pie with GDAL.

Grep and punctuation marks

I've often tried to use grep to filter search results for locate and several times I've had to search for a string with locate:

locate svn

This search would obviously include all the '.svn' folders too. Trying to get rid with them with

locate svn | grep -v '.svn'

does not work, since grep uses basic regular expressions, and the period mark will simply match any character.

To explicitly indicate the usage of a punctuation mark, one has to put it in brackets. The statement will now look something like:

locate svn | grep -v '[.]svn'

I just scanned through the grep man pages, and everything is explained pretty clearly there. I'd recommend reading the bit on regular expressions.

MySQL and the joys of security

I've been running my local MySQL server on Gentoo without hassles for a while and recently tried to connect to it from a remote machine. Obviously there was bound to be some issues and here's how I solved them.

When you first try to connect remotely to the machine, you'll be greeted by this lovely message:

# mysql --user=root --password=some_password --host=192.168.0.1 MYDB
ERROR 2003 (HY000): Can't connect to MySQL server on '192.168.0.1' (111)

To get this sucker out of the way, edit your /etc/mysql/my.cnf file and comment the following line:

# bind-address = 127.0.0.1

Note to all the security girls: Your server will now be accessible from any other networked machine. Oooohh nooooo!!

Anyhow, now we try to connect again, and whooopa!! Another error:

mysql --user=root --password=some_password --host=192.168.0.1 MYDB
ERROR 1130 (00000): Host 'remote_pc_hostname' is not allowed to connect to this MySQL server

Ah bite me. Ok, so you have to grant his Lordship permissions to connect to thy server. In order to do that logon to sql on the sql server itself:

# mysql --user=root --password=some_password
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 11
Server version: 5.0.70-log Gentoo Linux mysql-5.0.70-r1

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> GRANT ALL ON *.* TO 'root'@'remote_pc_hostname';
Query OK, 0 rows affected (0.00 sec)

Now all should be well in fairyland, but alas. The code securo freaks over at Sun have presented ye with another trial.

# mysql --user=root --password=some_password --host=192.168.0.1 MYDB
ERROR 1045 (28000): Access denied for user 'root'@'remote_pc_hostname' (using password: YES)

Ok I'm taking the fairytale rubbish too far. Here's what you should've done on the sql server in the first place:

# mysql --user=root --password=some_password
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 11
Server version: 5.0.70-log Gentoo Linux mysql-5.0.70-r1

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> GRANT ALL ON *.* TO 'root'@'remote_pc_hostname' IDENTIFIED BY 'some_password';
Query OK, 0 rows affected (0.00 sec)

Sheesh! You little girls are wasting my time. Take a que from MS SQL Server.

Creating JPG's and other formats with Python and GDAL

Ok for some daft reason the driver.Create() method is not supported by all image drivers, but the driver.CreateCopy() is. The main difference being that Create() actually creates the format from scratch whereas CreateCopy() makes a copy of another existing image. I'm sure they have their reasons, but I don't understand why everything isn't created in a generic image type and then converted to the destintation image format through CreateCopy().

Anyhow, this is how I created a JPEG with GDAL. The same approach can be followed for multiple image formats. See this link for a complete list of supported formats.

# Import libs
import numpy, os
from osgeo import osr, gdal

# Set file vars
output_file = "out.jpg"
output_file_root = os.path.splitext(output_file)[0]
output_file_ext = os.path.splitext(output_file)[1]
output_file_tmp = output_file_root + ".tmp"

# Create tmp gtif
driver = gdal.GetDriverByName("GTiff")
dst_ds = driver.Create(output_file_tmp, 512, 512, 1, gdal.GDT_Byte )
raster = numpy.zeros( (512, 512) )
dst_ds.GetRasterBand(1).WriteArray(raster)

# Create jpeg or rename tmp file
if (cmp(output_file_ext.lower(),"jpg" ) == 0 or cmp(output_file_ext.lower(),"jpeg") == 0):
 jpg_driver = gdal.GetDriverByName("JPEG")
 jpg_driver.CreateCopy( output_file, dst_ds, 0 )
 os.remove(output_file_tmp)
else:
 os.rename(output_file_tmp, output_file)

Create a GeoTiff with Python and GDAL

It looks like the GDAL tutorial on creating GeoTiff's is slightly outdated (2008/05). I received this error with Python 2.5.2, GDAL 1.53 and NumPy 1.0.4:

Traceback (most recent call last):
File "test.py", line 9, in <module>
dst_ds.GetRasterBand(1).WriteArray(raster)
File "/usr/lib/python2.5/site-packages/gdal.py", line 796, in WriteArray

File "usr/lib/python2.5/site-packages/osgeo/gdal_array.py", line 154, in BandWriteArray
AttributeError: dtype

As far as I can tell this occurs because the matrix raster doesn't have the attribute (property) dtype. Something must have changed, but I managed to get it going by using the numpy lib rather than Numeric.

The code that worked for me looks like this:

import numpy
from osgeo import osr, gdal
format = "GTiff"
driver = gdal.GetDriverByName( format )
dst_ds = driver.Create("out.tif", 512, 512, 1, gdal.GDT_Byte )
raster = numpy.zeros( (512, 512) )
dst_ds.GetRasterBand(1).WriteArray(raster)

One can also explicitly state the matrix's data type by adding the dtype attribute:

raster = numpy.zeros( (512, 512), dtype = numpy.uint8)

PHP: move_uploaded_file, Permission denied

If you ever receive the failed to open stream: Permission denied error in Linux on the move_uploaded_file function, try changing the upload folder's permissions through the chmod command:

chmod 0777 upload_dir

Nesting floating DIV overflow problem

I had an issue with non-IE browsers where a floating div would oveflow its parent container.

Here's an example:

<div id="Container" style="background-color: Blue; width: 400px; padding: 10px;">
 <div id="Image" style="float: left;">
     <img src="http://www.google.com/images/logo.gif" />
 </div>
 Some text yada yada. Some more text yada dada.
</div>

One can solve this issue by simply adding a div that places a break after the floating div:

<div style="clear: both; font: 1px/1px sans-serif;">&nbsp;</div>

So in the end, it should look something like this:

<div id="Container" style="background-color: Blue; width: 400px; padding: 10px;">
 <div id="Image" style="float: left;">
     <img src="http://www.google.com/images/logo.gif" />
 </div>
 Some text yada yada. Some more text yada dada.
 <div style="clear: both; font: 1px/1px sans-serif;">&nbsp;</div>
</div>

PHP Web Application Root URL

I wrote this little function to return a PHP applicaiton's root url. It is very quick and dirty, but I thought I'd still share it:

function get_app_root_url()
{
    $app_root_dir_name = getcwd();
    $idx = strripos($app_root_dir_name, '/');
    
    if($idx >= 0)
        $app_root_dir_name = substr($app_root_dir_name, $idx + 1);

    $app_root_url = get_page_url();
    
    $idx = strpos($app_root_url, $app_root_dir_name);
    
    if($idx >= 0)
        $app_root_url = substr($app_root_url, 0, $idx + strlen($app_root_dir_name));
        
    return $app_root_url;
}

// Curtosy of http://www.webcheatsheet.com/PHP/get_current_page_url.php
function get_page_url() 
{
    $page_url = 'http';
    
    if ($_SERVER["HTTPS"] == "on") 
        $page_url .= "s";
    
    $page_url .= "://";
    
    if ($_SERVER["SERVER_PORT"] != "80") 
        $page_url .= $_SERVER["SERVER_NAME"].":".$_SERVER["SERVER_PORT"].$_SERVER["REQUEST_URI"];
    else 
        $page_url .= $_SERVER["SERVER_NAME"].$_SERVER["REQUEST_URI"];
    
    return $page_url;
}

So long Java!

I've been struggling on and off for over two weeks to get my development environment up and running for Java web apps. I went with the JBoss Seam framework, since it was supposedly lightweight; Eclipse because it had a lot of plugins and seamed to be the most popular free IDE; Tomcat, since its what our web server is running.

In the end I had to ditch Eclipse because it has major issues with plugins cross versions. It's near impossible to get the right jar files and versions for all the Tomcat, Seam and Debugging plugins to play together. So I went with NetBeans, which seemed to be miles simpler to use, but with less functionality and a smaller community.

Even with the simpler NetBeans it was still a mission to get everything working. Finally I was ready to start doing some real coding, but oh my, what a mess!! Perhaps I'm just an idiot, but Seam doesn't have any logical (/www/myproject/...) structure and there's hundreds of XML files all over the shop. Phew... I tried to follow a few tutorials, but I was unable to find a good one. There isn't a single one that just says 1. Here is all the crazy config files, 2. Here is the code, 3. This is how it it all is structured, 4. This is how you deploy... bla bla. Come on people! Keep it simple!

After getting the examples going I read somewhere that I would need JBoss AS in order to run Enterprise Java Beans and it seemed like Seam used EJB for session management or something. Aaargh!!!! The Seam website explicitly states that you can run Seam on Tomcat, but fails to mention that you wouldn't have all the functionalities. What a load of rubbish!! Our web server only runs Apache 2 and Tomcat, so this is a bit of an issue.

I could see that ease of web development in Java is still VERY mush inferior to the likes of Asp.Net or RoR. It's almost impossible for a noob to create a simple website that queries a db with an IDE that supports debugging.

I had a quick look at PHP and it took me two days (which is also far too long for my liking) to get a full dev environment setup with Apache 2, MySql, Eclipse, PDT and Zend. After that my first application was up and running within minutes. PHP is logical, very well documented and has a huge dev community. It seems brilliant!

So that was the final blow to the head for Java. I can't understand why Java is so popular in big enterprises. Well at least for the web side of things. It's like using C++ to run a web app. It simply isn't geared for that purpose. Or perhaps I'm just missing something.

Facelets Support in NetBeans 6.1

After getting Seam working by manually deploying a .war, I moved onto the next step of integrating Seam into an IDE. Eclipse is a clunky heap of rubbish. I'd say its problem is that it is too flexible. There's a million ways to do one simple thing, but no single way to do one simple thing properly.

After serveral hours wasted on Eclipse, I moved over to NetBeans (6.1). There's a great little tutorial on getting your NetBeans environment setup over at Edem Morny's tech blog. Everything went smoothely, up to the part where one installs Facelets Support, where I get this nasty little message:

Missing required modules for Plugin Facelets Support:
JSP Parser [module org.netbeans.modules.web.jspparser/3 = 200805300101]

This can be avoided by not updating NetBeans. Fortunately there is a bit of a hack to get around this issue:
1. Go to your netbeans-6.1/enterprise5/modules directory
2. Unjar the org-netbeans-modules-web-jspparser.jar file

jar xf org-netbeans-modules-web-jspparser.jar

3. Create a backup of the orignal org-netbeans-modules-web-jspparser.jar just in case.
4. Edit the META-INF/MANIFEST.MF file.
5. Change the OpenIDE-Module-Implementation-Version: to match the verion the facelet is complaining about. In this instance 200805300101.
6. Save the file and jar the whole thing again (by explicitly adding the modified manifest file)

jar cfm org-netbeans-modules-web-jspparser.jar META-INF/MANIFEST.MF org

Manaully clean your Gentoo world file

I've been in a love-hate relationship with Gentoo for over 3 years now. Luckily there's a lot more love than hate these days. Well 3 years is a long time and it's time to clean this bad boy up...

There's a lot of articles on-line on how to remove cruft and deprecated packages etc. But I struggled to find one that will help you clean up your world file. Being a bit of a quick fix kind of guy, I hammered this little python script together, which will loop through your world file and find the package description for each entry. The script isn't perfect, but it gets the job done for the most part. I still have to manually edit the world file, but at least now I get a description of each entry as I step through the file.

Here's what you need to do:

Log in as root and fire up a text editor:

nano print_world_file_details.py

And paste the following:

import os
import string
import sys

if __name__ == '__main__':
 f_in = open("/var/lib/portage/world")

 lines = f_in.readlines()

 for line in lines:
     index = line.find("/")
     length = len(line) - 1
  
     if(index > 0):
         print ""
         print "--------------------------------------"
         print line[index+1:length]
         print ""
         cmd = "emerge -s " + line[index+1:length] + " | grep -A7 '" + line[:length] + "' "
         os.system(cmd)
      
     sys.stdin.readline()

 f_in.close()

Exit and save (Ctrl+x, Enter). Run the script:

python print_world_file_details.py

Just hit enter after each lookup and manually remove the entry from your world file. Since the script uses grep to filter the ermerge -s results you might get multiple descriptions for some packages. Like dev-java/java-config will list dev-java/java-config-wrapper too. A slight nuisance, but who cares?

Note: It is not considered safe to update your world file manually, so don't mess around if you don't know what you're doing. Please make a backup before editing.

Get JBoss Seam going on Gentoo Linux with Tomcat

I've been given a web app to do for an open source driven organization. I like the ideology behind open source, but I must say that the complexities involved in just getting started, must be the single biggest hurdle in the adoption of OSS.

Most Java and Linux fanboys might rupture a vain for me saying this, but to get any sort of Java web app going is massive pain in butt. To get Asp.Net going, one merely install visual studio and off you go. You don't even need IIS these days. To get a Java web app (dev environment) going you have to install a web server, an application server, a java framework, an IDE, IDE plugins to support the java framework and perhaps additional frameworks / tools which will allow you to work in a "code-behind" fashion. Then you have to decide on a web frameworks and the choices are endless: Spring MVC, Struts, Seam, JSF, Tapestry, Wicket, Hibernate, Stripes, etc. You can Google until you're blue in the face, but every single framework claims to be the best and all have their own following of groupies. To be honest, it is a big geeky mess!

Unfortunately I don't have the luxury of cleaning up this steaming pile of rubbish, so I'll have to get my hands dirty and figure out what the best solution is through trial and error (very scientific).

My client already has Tomcat running, so luckily I don't have to bother about figuring out which application server is the best. A further requirement is that the web framework should be quite lightweight. It seems as if JBoss' Seam framework isn't too much of a brute, so I'll go with that and probably get burned half-way in.

Even though I detest overcomplicated systems, I've grown quite fond of Gentoo. Once you've mastered this beast, it is quite pleasant to work with. (Perhaps I'll have the same feelings towards Java web apps once I've finished here, but for now it's not looking good.) So I'll run through the steps of getting Seam going on Gentoo.

To start off with, we'll need JDK (or JRE). Login as root, emerge sun-jdk (in my case version 1.6.0.07) and logout. After that we need to get Tomcat.

Note: You can try to get Tomcat going through portage, but you'll end up pulling your hair out, because Seam uses the naming-factory-dbcp.jar, which is not included in the build for some bizarre reason.

cd ~
mkdir applications
cd applications
wget http://apache.mirrors.hoobly.com/tomcat/tomcat-6/v6.0.18/bin/apache-tomcat-6.0.18.tar.gz
tar -xf apache-tomcat-6.0.18.tar.gz

Now let's set the environmental variables needed:

nano ~/.bashrc

and add these lines:

export JRE_HOME=/opt/sun-jdk-1.6.0.07
export TOMCAT_HOME=/home/yourname/applications/apache-tomcat-6.0.18
export CATALINA_HOME=$TOMCAT_HOME
export CATALINA_BASE=$TOMCAT_HOME

Note: You might want to emerge the tomcat-native package, since it will add the Apache Tomcat Native library. I'm not sure if Seam needs this, but I noticed Tomcat having a bit of a fit when it couldn't find it. The message in the $TOMCAT_HOME/logs/catalina.out was "INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path"

One can also configure an admin user, but this is not necessarily required. Edit the tomcat-users.xml file like this:

nano $TOMCAT_HOME/conf/tomcat-users.xml

<tomcat-users>
<role rolename="manager">
<user username="root" password="irock" roles="manager">
</user>

One can now start Tomcat:

cd $TOMCAT_HOME/bin
./startup.sh

You should now be able to access Tomcat on http://localhost:8080

Tomcat will display $TOMCAT_HOME/webapps/ROOT/index.html

So we can now follow Seam's Tomcat guide, which I had to modify a bit:

1. Go into the jboss-seam/examples/jpa directory
2. Copy the lib/hsqldb.jar into $TOMCAT_HOME/lib
3. Run ant tomcat6 to build the application
4. Stop Tomcat $TOMCAT_HOME/bin/shutdown.sh
5. Remove previous any exiting deployments: rm -r $TOMCAT_HOME/webapps/jboss-seam-jpa
6. Copy war file: cp dist-tomcat6/jboss-seam-jpa.war $TOMCAT_HOME/webapps
7. Start Tomcat $TOMCAT_HOME/bin/starup.sh
8. Access the app at http://localhost:8080/jboss-seam-jpa

And Bob's you're uncle!

Linq Context.Add Renamded To

A number of Linq to Sql tutorials online are based on intial "Orcas" (now 3.5) release of the .Net Framework. A number of namechanges were made and I picked one up whilst learning Linq. Here's the error I received:

'System.Data.Linq.Table<Namespace>' does not contain a definition for 'Add' and no extension method
'Add' accepting a first argument of type 'System.Data.Linq.Table<Namespace>' could be found
(are you missing a using directive or an assembly reference?)

The context.Add methods was changed to context.InsertOnUpdate.

Apply themes to Images in Asp.Net

The Asp.Net Themes approach is somewhat lacking when it comes to images. It is easy enough to switch between CSS and Skin files, but switching between images aren't as trivial. Luckily there is a relatively simple way to do this.

My thinking was to set the image folder dynamically. First, we'll have to figure out what our image path is. Say one use the following structure for your image folders:

App_Themes/Theme1/Images
App_Themes/Theme2/Images
...

The following ImagePath accessor can be used to determine the current theme's image path (I would recommend you to put this in a Base class for this to be accessible to all pages/controls):

public string ImagePath
{
  get
  {
      return string.Format("{0}://{1}{2}/App_Themes/{3}", Request.Url.Scheme, Request.Url.Authority, Request.ApplicationPath, Page.Theme);
  }
}

Now that we know where the currently selected theme's images are, we need to set the path for the controls in the aspx page. This is easy enough if you using normal HTML controls as follows:

<img src='<% Response.Write(ImagePath); %>/Image.jpg' />

But this approach does not work for server controls. If you use the following line, you'll receive an error:

<asp:Image ImageUrl="<% Response.Write(ImagePath); %>/Image.jpg" runat="server" />

Error:
Server tags cannot contain <% ... %> constructs.

One could set the asp:image's ImageUrl in the codebehind, but I find it to be a bit clunky. In order to use the ImagePath in the aspx file, one can use an approach by Dave Reed.

Create a class in the App_Code called CodeExpressionBuilder:

using System;
using System.CodeDom;
using System.Web.Compilation;
using System.Web.UI;

[ExpressionPrefix("Code")]
class CodeExpressionBuilder : ExpressionBuilder
{
  public override object ParseExpression(string expression, Type propertyType, ExpressionBuilderContext context)
    {
      return expression;
    }

    public override CodeExpression GetCodeExpression(BoundPropertyEntry entry, object parsedData, ExpressionBuilderContext context)
    {
      return new CodeSnippetExpression(parsedData.ToString());
    }
}

One can now set the ImageUrl in the aspx file as follows:

<asp:Image ImageUrl='<%$ Code: ImagePath + "/Image.jpg" %>' runat="server" />

AJAX ProgressUpdate issue

I had an issue with the UpdatePanel and ProgressUpdate AJAX extensions controls in ASP.NET.

The following error occurred in the Javascript:

if (this._dynamicLayout) this.get_element().style.display = 'block';

I solved the problem by moving my ProgressUpdate control out of the UpdatePanel's tags. I guess I was looking for trouble when I did that.

Having the ProgressUpdate within the UpdatePanel work in certain instances, but I also had another UpdatePanel in the same UserControl, so it got a bit hairy.

I'd like to verify this, but as always, the deadline looms.

Constraint cannot be special class 'object'

I was busy creating a generic method to convert a List to a DataTable when I received the error:

Constraint cannot be special class 'object'

on this statement:

public DataTable ListToDataTable(IList list) where T : Object

Blegh... why would this be? Anyhow, I did a workaround by changing the statement to the following:

public DataTable ListToDataTable(T list) where T : IList

A bit annoying, since now one has to send a List

Python: Multi-dimensional dynamic arrays

Another very poorly documented topic is how to create dynamic arrays of type array. Every second website will show you how to create static 2D arrays, which is pretty much useless for 90% of the practical applications out there.

I 'solved' it to a degree by creating a list of array items, but this is still not ideal. I would like to use a pure multi-deminsional array of type array.

Here's how I did it:

multi_arr = []

for i in range(some_range):
   arr = array('f') # array of type float
  
   # ... populate arr ...

   multi_arr.append(arr)

Julian Day in Python

I found very little documentation on obtaining the Juilian day with the time module in Python. So this is how it's done:

import time

print time.strftime("%j")

As easy as that!

Linux: shutdown: you must be root to do that!

In order to use the console commands shutdown, poweroff, halt or reboot, one has to be a super user. One might see messages such as:

shutdown: you must be root to do that!
poweroff: must be superuser.
reboot: must be superuser.
halt: must be superuser.

In some cases the user can simply be added to the wheel group. This should give the user enough privileges to shut the machine down. A more reliable approach is to make use of Sudo, which was specifically designed to allow ordinary users to execute certain super-user-only commands.

The following section will explain how to setup sudo to allow other users to the machine down through the command line under the Gentoo distro.

Install Sudo
Login as su:

su -

Emerge Sudo:

emerge app-admin/sudo

Configure Sudo
Run visudo:

visudo

Enter the following:

# Runas alias specification
YourUserName  ALL=(root) NOPASSWD: /sbin/reboot
YourUserName  ALL=(root) NOPASSWD: /sbin/halt
YourUserName  ALL=(root) NOPASSWD: /sbin/poweroff
YourUserName  ALL=(root) NOPASSWD: /sbin/shutdown

Replace YourUserName with the user name which requires shutdown privileges.

See this guide on how to create a group which allows all its users to shut the machine down.

Named pipes issue: WaitForConnection hangs around indefinitely

I recently created an IPC client and server based on the newish NamedPipeServerStream and NamedPipeClientStream classes in .NET 3.x.

It worked great, but when trying to stop the server, the main thread would hang aound waiting for the WaitForConnection() statement to return. I tried to kill the thread in a number of ways, but to no avail.

In the end, I created an easy workaround by doing a "dummy" client connection to the server. This will cause WaitForConnection() method to continue and the thread can then be stopped gracefully.

Named pipes issue: System.UnauthorizedAccessException: Access to the path is denied.

After yesterday's triumphant discovery I had a nightmare of a day trying to figure out why my client couldn't connect to the server. Time and time again, the following exception was raised:

System.UnauthorizedAccessException: Access to the path is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.Pipes.NamedPipeClientStream.Connect(Int32 timeout)
at System.IO.Pipes.NamedPipeClientStream.Connect()

And in the debug output window:

A first chance exception of type 'System.UnauthorizedAccessException' occurred in System.Core.dll

My best googling skills couldn't find an answer for this one and I ended up trying to change the permission for the NamedPipeServerStream and venturing down the wrong path by changing the PipeAuditRule and PipeAccessRule settings.

In the end turned out my PipeDirection had to be set to InOut, even though I was only streaming information from the Server, and not receiving information.

Here's the winning line of code:

NamedPipeServerStream Pipe = new NamedPipeServerStream(PipeName, PipeDirection.InOut, 1,
   PipeTransmissionMode.Message, PipeOptions.None);

.NET IPC through Named Pipe Servers

I've been struggling to get my own IPC server and client going. The work was based on this great post on The Code Project. Even though the article was very useful, I found the code to be a bit clunky.

I restructured a lot of it into very neat client / server structure. During debugging, I stumbled onto this "new" addition in the .Net 3.5 Framework. Named pipes are neatly integrated, so no need for my client and server anymore. Doh!

Oh well, I only lost a day's worth of coding, and at least I can rest assured that this part of my project will be quite stable.

Windows Service uninstall failed

Whilst developing a Windows Service, an exception occurred. I opted not to debug and when I exited the VS instance the service naturally terminated.

When I got back into the Services MMC, my service was marked as Disabled. When trying to uninstall the service through InstallUtil myservice.exe /u, the following exception was raised:

An exception occurred during the Install phase.
System.ComponentModel.Win32Exception: The specified service has been marked for deletion

This problem is easy enough to solve. Just quit the MMC and uninstall the service again.

Visual Studio editor annoyances

I have the nasty habit of copying code and then accidentally hitting Crtl+C again without selecting any text. VS clears your current clipboard item and replaces it with the line you're on.

I find this quite annoying, but luckily there's a way to get the previous items in the clipboard back. Just keep hitting Ctrl+Shitf+V to cycle through all the previous items.

UI updates and threading

I've been working on a small Windows Forms app which has a thread running in the background. This read needs to send info back to the UI during certain intervals. It's a pretty easy problem to solve and there are numerous sources on the web explaining this.

I did come across weird behaviour when my app went into a deadlock. It hanged at the Invoke method in the following code:

// This updates some control safely
protected void UpdateSomeControl()
{
    // Check if an Invoke is required
    if (InvokeRequired)
    {
        // Update the UI through delegation
        Invoke((ThreadStart)delegate()
            {
                // ... Update the UI ...
            });
    }
    else
    {
        // ... Update the UI ...
    }
}

This is upposed to work! So what happened?

Turns out there are two types of calls to modify a control in .Net. The first is a SendMessage and the other PostMessage. Invoke uses SendMessage, which causes the update message to be sent, but hangs around until the message has been processed. BeginInvoke uses PostMessage which is basically a fire-and-and-forget approach; the message is added to the queue and the process goes of doing other work without waiting for it to processed.

So my Invoke was obviously hanging around waiting for the control update to be completed, while the control update was waiting for some other process further back in the queue... at least that's how I understand it.

To solve this issue, simply use BeginInvoke rather than Invoke. I'm sure this fix is not appropriate for all problems, but in my case it did the trick.

For a more detailed explenation, see this CodeProject page.

The infamous "\\computer is not accessible. You might not have permission to use this network" error

I must have come across this stupid error a dosen times and it is often solved by simply changing the Windows machine's Network ID or by running the Network Connection Wizard thingy in the Control Panel -> Connections page.

This time though, we had a computer playing hardball and was only solved by doing the following (from PChuck's Network):

Download and install the Windows 2003 Server Resource Kit Tools.

Click Start | All Programs | Windows Resource Kit Tools | Command Shell.

Type these three lines, one at a time, at the command prompt. The ntrights
commands are case-sensitive, so type them exactly as shown. Note the "+r" in
the second one and the "-r" in the third one:

net user guest /active:yes
ntrights +r SeNetworkLogonRight -u Guest
ntrights -r SeDenyNetworkLogonRight -u Guest

And that solved our problems. Well for most guys. For some reason my Vista still did not want to connect. I changed my Network ID to be same, and whala! It worked! This is a bit peculiar considering that I was able to connect across different Network ID's on my previous XP machine. I don't have time to double check and validate that now though.

More issues moving over to new system

Ok so I don't have this dll on my system.

Could not load file or assembly 'System.Web.Extensions, Version=1.0.61025.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.

Just download and install the ASP.NET 2.0 AJAX Extensions package. Problems solved!

Vista administrator woes

Ok if you're a Vista user you're undoubtely also sick and tired of being bombarded with security pop-ups, which are aimed at keeping the average user from destroying the system with malware.

So you want to be a real administor? Follow these steps:
First, turn off UAC warnings.
1. Go into Control Panel.
2. Open up User Accounts (in classic view) and User Accounts and Family Safety -> User Accounts (in standard view).
3. Click Turn User Account Control on or off and follow the rest of the instructions.

Ok now you got rid of most of the issues, but Vista still with give you a hard time with Services and applications such as SQL Server and Visual Studio. It still won't think you are a full blown administrator and prompt you with the odd warning. Quite annoying.

Finally, Turn off "Run all administrators in Admin Approval Mode"
1. Open a run window, and type in secpol.msc and hit enter.
2. Expans Local Policies -> Security Options.
3. Scroll down to User Account Control: Run all administrators in Admin Approval Mode and disable the damned thing.
4. Restart and all should be well.

Visual Studio 2005 and IIS 7: Unable to open the Web

I recently upgraded to a new workstation, which came with Vista Ultra SP1. I tried to open my previous web apps and got this message:

"Unable to open the Web 'http://localhost/...'. The Web server does not appear to have the FrontPage server extensions installed."

A quick google search came up with this article on the MS Knoweldge Base:
http://support.microsoft.com/kb/936857

I followed the instructions and enabled the IIS Metabase and IIS 6 configuration compatibility component. I went back into VS Studio, clicked Reload Project, but no luck.

Next I went into IIS 7's management console. I noticed that the web apps were not added to an application pool yet. Just right-click the folder and choose Convert to Application. Reloaded the project again, and nothing...

Trying to solve the issue, I got a bit side-tracked and sorted out some annoying administrator issues.

I rebooted my system and my web app is working!! Whoop whoop!