Tuesday, December 25, 2007

HTC Touch WIFI Connectivity

This article will briefly go through the the important steps of setting up your HTC Touch for WIFI connectivity.

There are two basic things to consider - connecting to an unsecured network and to a secured one. I hope you already know the access details (network key) for the network you're trying to connect to!
  1. Go to Start > Settings > Connections > WIFI (the Wireless LAN icon will provide you with details on a selected WIFI network. Useful perhaps when the network is hidden)
  2. In the list of networks select the desired one and click connect or tap on the network name (notice whether it says Available or Unavailable and aim for the former)
  3. Leave Network name as is (actually you can't change it anyway I think) and leave Connects to to The Internet, click Next
  4. On the next screen for most practical cases select for Authentication: SHARED (most folks leave it at OPEN and just waste time trying to figure out why the connection is not working)
  5. Select WEP for Data Encryption (this is a very common setting)
  6. Uncheck the The key is automatically provided checkbox (you'd need to enter it) and enter the network key (that you hopefully know) into the Network key box (note: this should usually be a 5-digit key)
  7. Leave Key Index to 1 (these are the common settings I'm discussing. If they're different in your particular case its probably because you already know more or have someone that does so you should be talking to your network admin if this is the case to get the exact details)
  8. Leave the rest at what it is, click Next
  9. Leave everything at default (Use IEEE 802.1x ... checkbox and EAP type)
  10. Click Finish
If all went well, your HTC Touch will soon enough connect itself to the network. If it does not get connected within the next 10-15 seconds then you should probably review one or more of the connection properties used above.

The funny thing with it is that when things go wrong it does not say much. Even worse - the network would sometimes disappear completely from the list of available networks or become unavailable. Whenever you see this chances are that you've misconfigured the network type (perhaps you put OPEN instead of SHARED or the other way around for the Authentication property) or the encryption type is not right (most of them do a WPA but sometimes you may have a different one).

Hope this helps.

Friday, December 21, 2007

Working With en Existing App in Visual Studio .NET

I have recently started work on an existing ASP.NET application. The first challenge was to get the application properly installed, built and working in my development environment. If you're lucky references to sub-projects in your solution will be through relative paths. That's half of your problem gone if that is the case. If you're not lucky, I'd suggest you do that.

Step 1: make all sub-projects referenced through relative paths.

This will require that some of the project files/folders are moved around a bit, but they should be somehow made a logical structure. (hm, my last sentence makes me think about Maven and the way in which project dependencies are handled there... I ought to lookup somethings on good practices with VS dependencies).

Step 2: sort out web projects and their virtual directories and applications.

Next, check for web projects. Each one referenced in the visual solution should have a virtual diretory created for it in the IIS root directory (unless your placing your code directly into the IIS root). In addition, chances are that you'd have to create an application for each virtual directory mentioned in the VS solution file. To create an application for a virtual directory in IIS 5.0 do this: Settings > Control Panel > Administrative Tools > Internet Information Services > Local Computer > Web Sites > Default Website > Properties (right click the virtual folder in question) and click Create next to the text box with the label Application Name.

Keep opening your Visual Studio project until you get it to stop complaining about not being able to open one of its sub projects. Once this step is done, we can move on to building things.

Step 3: do a clean build of all projects

Once you're here, you should build a project. In my case this was, to my great surprise, successful as well. However when I tried running the application the stack trace kept on reporting errors which occur in controls referenced by paths that are not on my machine! I figured this was really weird as I just built the whole solution successfully and there must be a place where paths to controls were hard-coded. Again, I was wrong.

Two things you should pay attention to:
1. DLLs used in the bin folder of the application
2. Your build configuration

I think the final solution is really related to your build configuration, but its worth talking about the first one as well.

Checking the /bin folder of your project (and any of the sub-projects) you should be able to see all required DLLs being inside it. Look at the dates. If you have been rebuilding your solution these dates should be very recent, if not a few minutes ago. However, if you're seeing that some of the DLLs are much older then chances are that you're not rebuilding them properly. Obviously if there are 3rd party DLLs in your bin folder these are not applicable for this discussion - look at only the DLLs that are built through one of the sub projects of your solution.

It turns out that lots of these DLLs in the bin/ folder of many sub-projects were not rebuilt and carried a timestamp of a few years back. That suggests that the build configuration either skipped this completely or did not copied the relevant DLLs where they were supposed to be.

So, what to do? Simple: open your VS Configuration Manager and make sure that all the projects have their build checkbox checked. Build again. It is supposed to build the entire set of projects that go into the solution.

Quick check into the bin folder should verify that those timestamps are showing that most of the DLLs are up to date.

Unfortunately, its still possible that this will not properly rebuild your solution. I found that my "main" project which depended on all these sub-projects still had references to some really old DLLs from a few of them. If you manually copy the affected DLLs you're 99% of the time going to be OK - the application should start running properly. But why were these DLLs not in place? I would have thought that VS would take care of these dependencies for me.

Well, as usual, the error is not with VS, but with the programmer. Its all about project dependencies. Right-click your VS solution and choose Project Dependencies.... It reveals a small dialog box that shows for a selected project what are its dependencies. Similarly, it shows a an overall build order (derived from the Configuration Manager setting).

Looking at my problematic "main" project it showed that it did not depend on a few of the sub projects it should have. I'm yet to figure out how is this information generated but my best guess is that the way in which project references are handled is the reason why this configuration would not be properly done. Ticking the dependency properly causes the solution to start being built the way its supposed to.

Final Notes

All these problems really suggest a deeper need for properly dealing with the way your projects are built. Most of us take it for granted and not pay too much attention to this. However, builds should be automated (as we all know, right?). Some effort should be spent to get the project in a state that allows it to be downloaded and built in a few minutes so that a development environment is setup very quickly. I guess its still sad that we must waste time on these things. I'll hopefully figure something more useful then this comment and do another post on that. Its either some "proper" building of projects in VS usage of something like NANT.


Typical Problems

If your project is perhaps suffering from the above build configuration what happens is that when you try running the application you get all sorts of weird stack traces. But a common theme is that the stack trace will show paths (on the HDD, not URLs) referring to paths that are not on your machine but instead residing most likely on the machine of the previous developer. This suggests that the runtime application is picking up DLLs that were built by someone else on another machine.

Wednesday, December 19, 2007

IIS 5.0 serving pages slow

I had recently encountered a really weird behavior while doing some development. I'm running IIS 5.0 on an XP Pro and for some strange reason it started to serve pages really, really slow. Serving images, for example, was super-slow. Typically this wasn't the case so it took me a while to figure out that my ColdFusion service was off. Once you turn it back on everything goes back to normal.

Thursday, December 13, 2007

Sitemap for a Large Site

If you're running a site with a large number of articles it is advisable to use sitemaps and submit them to Google for a better coverage of your content. In theory you should be able to submit your sitemaps to Yahoo! and MSN as well (more on this later).

What is a sitemap?
Taken from www.sitemaps.org:
Sitemaps are an easy way for webmasters to inform search engines about pages on their sites that are available for crawling. In its simplest form, a Sitemap is an XML file that lists URLs for a site along with additional metadata about each URL (when it was last updated, how often it usually changes, and how important it is, relative to other URLs in the site) so that search engines can more intelligently crawl the site
A typical sitemap file that contains links you wish to submit to a search engine will look like:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>http://www.example.com/dir/p1.html</loc>
<lastmod>2004-06-07</lastmod>
<changefreq>monthly</changefreq>
<priority>1</priority>
</url>
<url>
<loc>http://www.example.com/dir/p2.html</loc>
<lastmod>2004-06-07</lastmod>
<changefreq>monthly</changefreq>
<priority>1</priority>
</url>
... up to 50,000 links like these in a single file ...
</urlset>
Please read through the official section on how do you go about submitting your sitemap to search engines.

How to submit a very large number of URLs?
You'll have to submit multiple sitemap files. The trick is simple: build individual sitemap files full of links and then have a main "directory" (or sitemap index) stating what are all the files and their names. Then submit the index file to the search engine. Obviously XML for the sitemap index is slightly different. Below is a sample of what a sitemap index file might look like:

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<sitemap>
<loc>http://www.example.com/sitemap-1.xml.gz</loc>
<lastmod>2007-12-21</lastmod>
</sitemap>
<sitemap>
<loc>http://www.example.com/sitemap-2.xml.gz</loc>
<lastmod>2007-12-21</lastmod>
</sitemap>
</sitemapindex>
Notice that the sitemap index refers to 2 separate sitemap files which are the ones that contain the actual links. In the above example, these are called sitemap-1.xml.gz and sitemap-2.xml.gz. Also notice that the reference is made to the full URL that points to the gzipped sitemap file and not the actual sitemap file itself (sitemap-1.xml).

Google officially accepts up to 50,000 URLs per single sitemap file OR (and this is quite important) the size of the sitemap file (once unzipped) cannot be bigger then 10MB. The second condition was added a few months back.

Finally, you should compress these files prior to submitting them. This means that your sitemap index file should have references to zipped files. Google works with gzip so find a way to zip the sitemap individual files containing all those links. My testing showed that compression makes a 50,000 URLs file (fairly long URLs I'd say) to about 400KB. Otherwise its about 10MB in raw format. This was done with a CFMX 7.1 on Win Server 2003 with the help of the excellent Zip CFC.

Notes
Few things to remember: Google seems to be the only one that happily is trying to work its way through a submitted sitemap file. Yahoo! and MSN are not. At least that is what I've noticed so far. Truth be said - Google is the only one that provides useful webmaster tools that you can use to see how your site fares with the search engine.

It may be useful to monitor whether google is picking up your sitemap. First things to check is the webmaster tools (click Sitemap on the main menu once you login) and submit your sitemap. Its quite quick to pickup your change so check shortly and it will report the number of URLs it picked up from the submitted sitemaps. Somewhere inside the webmaster tools is a page that says what's the crawl rate. From my experience it takes about 7 to 10 days for google to start seriously reporting something into that graph. However, by the time you can see a graph there your web server log should have shown you quite a lively interaction with the google bot.

Useful Links for Submitting Your Website to Search Engines
  1. Sitemap main site (http://www.sitemaps.org/)
  2. Search engines unite on sitemap discovery (article available at http://searchengineland.com/070411-080716.php)
  3. Google sitemap-related pages
  4. Yahoo Site Explorer Search Documentation (http://developer.yahoo.com/search/siteexplorer/V1/updateNotification.html)
  5. Google sitemap ping url:

  6. MSN seems to work through MOREOVER (I couldn't find info on submitting directly to MSN but have found some reference to MOREOVER in that context instead so I'm providing it here too). Their sitemap ping url is:
    http://api.moreover.com/ping?u=http://www.example.com/sitemap.xml
  7. Yahoo! sitemap ping URL:
    http://search.yahooapis.com/SiteExplorerService/V1/updateNotification?appid=YahooDemo&url=http://www.example.com/sitemap.xml
    (You'd need to get a Yahoo! AppID before you can submit this)
  8. ASK.com sitemap ping url:
    http://submissions.ask.com/ping?sitemap=http%3A//www.example.com/sitemap.xml
(for all of the URLs above please change "www.example.com" to the domain you're trying to manage).

Final note: I have not really gotten great results with MSN and Yahoo! so perhaps I'm missing something in this post.

Sunday, December 9, 2007

GTalk with HTC Touch

Now that you have your HTC Touch working with GMail, you're probably looking into other useful ways to spend its battery. One winner goes with being able to chat with your GTalk, MSN Messenger, Yahoo! Messenger buddies while you're on the go.

One nifty gadget you may want to try is the Agile Messenger. It works very well and supports GTalk and Yahoo! The only trouble is that its not free. Too bad. The trial version expires after about 2 weeks I think. I've used it without any problems for GTalk chats. Really great!

There are two other that I've tried with little success - neither one worked actually. Check out AniMessanger (which is really a Jabber client) and GoTalkMobile - perhaps you'll have more luck. I couldn't get much out of either one (although GoTalkMobile was showing up its menus but never got my GTalk friends retrieved). After some further reading it seems to be that the thing to look for is really a good Jabber client for Windows mobile since GTalk supports XMPP protocol that all Jabber clients talk well. I'll be updating this as I find something more useful...

Finally, good-old Skype. Its surprisingly not straight-forward to find the download for the mobile device. Oh, and accessing it through HTC Touch is not that great. Its not optimized for the viewer as you're better off downloading things through your PC. So anyways, you can get SKYPE for your mobile here.

Note that I did not try doing this without a WIFI connection, so I wouldn't exactly know what's the experience like if you're using your telco's GPRS (or whatever other) service to access the net.

Updated 17 Dec 2007...
Well seem to have found one that's working nicely with GTalk! Check out OctroTalk.

Friday, December 7, 2007

Drupal White Screen of Death

Just when I thought that there's smooth sailing all the way I encounter a scarry problem with Drupal 5.3.

Being a newbie, I've probably been adding too many modules into the installation, but that's another story. So far, the Drupal was happily working and there was nothing weird happening. Until yesterday.

I added Embedded Video Field module (part of Embedded Media Field). Initially it worked fine - I was able to embed a YouTube video without any problems. However, after I've added this new module into my version control (I'm using Subversion), things started behaving strangely.

First symptom was after submitting a form on page (I was playing around with defining a user role) I was presented with a white screen. Some google-ing led me to posts talking about WSOD (White Screen of Death) on Drupal's main site. After reading through what to check first I realize that there was no memory shortage as my PHP.ini has a variable memory_limit set to 128M.

I turned on error reporting in PHP.ini as well by setting display_errors = On so that my apache error.log finally started showing up something. To my surprise it had something like:

[Fri Dec 07 10:01:49 2007] [error] [client 127.0.0.1] PHP Fatal error:  Cannot redeclare video_cck_bliptv_info()
(previously declared in ...sites\\all\\modules\\emfield\\contrib\\video_cck\\providers\\bliptv.inc:7)
in ...\\sites\\all\\modules\\emfield\\contrib\\video_cck\\providers\\_svn\\text-base\\bliptv.inc.svn-base on line 20
OK. Now we're getting somewhere I thought. From this I figured that Drupal was actually including SVN control files during execution. Now I have to be honest and admit that it says _svn and not .svn is not without a reason. There was a post somewhere out there that suggested modifying the default .svn extension for SVN to _svn so that Visual Studio .NET (!) can happily work with files under SVN. That fix truly did get my .NET work to move on but I suppose along the lines you end-up having this.

Disabling Drupal Module Directly

What to do now? Since the site was completely dead I figured to disable the offending module directly in the database. Modify table SYSTEM and set Status=0 for row with name cck_video. Remove everything from cache_content table as well and voila - the site is back online.

CCK_Video Module

Further reasearch lead me to a pending issue on the mentioned module. It seems its not playing nice with Subversion, so I suppose my bad that I did not catch that before. But who reads a list of pending issues prior to installing a module in Drupal? Too bad for this bug with Subversion. I guess one workaround is to have that particular module out of SVN which would not cause that error above.

Closing ?> Symbol in PHP Files

Another possible error for WSOD in Drupal could be related to Drupal coding standard. It is apparently recommended that the closing ?> symbol be omitted from all php files (read on that here, under Invisible Errors). I looked at my template.php and found that thing at the bottom (I'm using the wonderful Liquid Aberdeen theme). Promptly removing that thing crashes my site again. WSOD. Great. Apache says:
[Fri Dec 07 10:04:51 2007] [error] [client 127.0.0.1] PHP Parse error:
syntax error, unexpected '>' in ...Apache aberdeen\\template.php on line 223
Nothing else to do but to put back the closing ?> into my template.php and move on with my life. There's a lengthy discussion on this available here. From what I gather, most folks tend to suggest you should keep it there. Note that I'm not exactly sure whether you're supposed to have it in the template.php as compared to, say, an include file. I'm guessing that for template.php under Drupal it is supposed to be there

So my 2c: be on the lookout for possible issues with SVN messing-up your Drupal installation due to its control files, and as a workaround, move only those offending modules out of version control if possible. Oh, and you DO have version control on your Drupal site, rite?

Thursday, December 6, 2007

Importing Content into Drupal

There is a useful Drupal module for importing data into your Drupal site. That is the Node Import project which provides support for importing content from CSV files (comma and tab-delimited).

My first impressions are mixed: it does manage to import nodes (primarily into my custom defined content types) but for some strange reason the CCK fields which reference other nodes do not get populated as per the specification during the wizard process.

In addition, when trying to upload a large CSV file the first screen where the file selection takes place just gets refreshed with a red warning that a file must be selected for import without any hint as to what may be wrong with the CSV file. When that happens, try to break-down your CSV into smaller chunks as there may be a wrongly formatted entry in the file that breaks the whole thing.

When on the topic of CSV, the file that I'm using is generated with a very useful CSV tag library by good people from RedBalloon Lab as I was moving things through a Cold Fusion file. However, note that it's not perfect when you're dealing with exports of large text fields which contain HTML markup. I have not yet figured out what exactly gets messed-up but every few hundred entries I get an entry that completely breaks the file.

Final notes on the importing process into Drupal: CCK node references do not get populated (despite the fact that they're supposed to) and when I edited manually the CCK table content_field_yourfieldname which stores the node reference ID I was further surprised that the front end still did not show the updated data (author name was still blank after I've updated the node ID that refers to the author). Turns out that the caching mechanism plays a role and deleting contents from cache_content table forces Drupal to show the last snapshot of data.

[Updated 11 Apr 2008]
Few months later I'm back to importing content. I just figured an important aspect of importing data: you'll probably need to adjust script execution time before importing will take some time to process the files, especially if you have lots of data.

One way to adjust your global setting of how long a PHP script can run is to edit the php.ini file and change the following line:

;;;;;;;;;;;;;;;;;;;
; Resource Limits ;
;;;;;;;;;;;;;;;;;;;

max_execution_time = 120 ; Maximum execution time of each script, in seconds

By default its set to 30 seconds (evidenced by my node import behavior showing a blank white screen after that time passes by). So I suppose you'd want to change that value only while you're working on the import and then put it back in.

Now that this is de-mistified, I recall seeing my Drupal modules page go blank once in a while and this must have been the reason for it.

Tuesday, December 4, 2007

Handling whitespace in ColdFusion

ColdFusion contains a number of tags to help prevent whitespace being returned to the browser. There is a nice blog on this here.

I tried applying those recommendations above, but kept on getting whitespace produced. To complicate my problem, processingdirective for suppresswhitespace=true seem not to do much within CFCs at all. Tried putting them at various places (within and outside methods in a CFC to no avail). Even worse - if you put a processingdirective before your CFC CFCOMPONENT on top it completely breaks. I suppose this is more of a problem on my end as there's I'm sure a reference point somewhere saying you can't do that.

Anyways, the weird and irritating thing was that the MSSQL Server kept on getting data with some funny characters in front of it (sort of a box). I had a call to do a trim and then a cleaning method that replaced some characters in a string before its fed into a CFQUERY. Turns out that the only way to get rid of those "boxes" (as well as the whitespace) was to use CFSILENT in my CFC. One is placed after the CFCOMPONENT tag on top and the other way at the end. This seems to do the trick nicely!