With every new release of Mac OS X, the need to re-install Perl's MySQL driver "DBD::mysql" arises again. And every single time it does not work out of the box (i.e. using cpan). Since I'm giving up using Perl and write my new scripts (yes, also the Shell scripts) in PHP, partially because of problems like this, I was able to postpone dealing with this problem, which arised after installing Snow Leopard - up until today.

If you have the problem it is most likely (and has always been) due to wrong paths to MySQL's header files. The fix is simple, and if you google for it people were complaining about the same problem even under OS X 10.2. Here's the solution: Build the thing manually, giving the right paths to Makefile.PL:

I needed to backup the contents of a file sharing directory of a server in our LAN to the shiny new mirrored 1 TB RAID inside my Mac Pro. Having a simple backup is nice, but it's even better to have snapshots, with the possibility to jump back to any date and have the exact representation of the disk as it was back then (TimeMachine also does this pretty well for Macs).

Always creating a complete backup would be a waste of disk space par excellence and in my case - backup via LAN - also pretty hard on the network. Fortunately there is rsync, and there's our friends ssh and launchd. rsync takes advantage of *NIX file system hardlinks and when called with the right arguments only backs up new files since the last backup, pointing to the older copy of a file if the file has not changed in the meantime. Mike Rubel has written up a nice article covering these aspects of rsync, check it out.

Thanks to rsync, backing up remote content to a local disk (or vice versa) is pretty easy, but there are some things to consider when automating the whole process. I've written a script that currently runs successfully on my Mac; it's called once daily at night (maybe I should say "every 24 hours") by launchd and uses ssh's shared keys feature to authenticate the machine.

Grab the script right here.

Breakdown

Here's a little breakdown of the process. Since we use launchd on this one, the script is run as the root user, keep that in mind!

The script takes two arguments, first argument is the SOURCE directory and the second argument the TARGET directory. The script then creates a folder with the current date inside the TARGET directory and performs a backup of SOURCE into this directory using rsync.

At the top of the script you can set some options, then follows a check whether you really provided the two arguments, the TARGET directory is created if it does not yet exist, and the name for the new snapshot is created using the date function. This code is all pretty straightforward, no need to show it here.

Since we want to use incremental backups, we're going to tell rsync in which directory it can find the latest backup, so that rsync only needs to transfer the new files. I use ls -U to get the latest directory first - what puzzles me here is that the list I get is reversed, means that the newest directory appears at the bottom. According to the man page it should be the other way round, now I don't know whether I didn't understand the man page or whether this is a Bug in Snow Leopard. If you're going to use this script verify this behavior and use head -n1 instead of tail -n1!

SSH Authentification

Then comes the tricky part with SSH. In order to not have to type the SSH password each time the backup runs, we use SSH public key authentification, which is always a good idea anyway. (If you don't know what this is, you might want to read a little).
But since we want to use this script from launchd, and the environment of launchd is not aware of ssh-agent, ssh won't know our SSH key and will ask for the password, nevertheless. So we need to ssh-add our key to the ssh-agent every time launchd performs our script. This works - as long as you have no passphrase for your ssh key set (which you should have!).
If you have a passphrase, we take advantage of SSH_ASKPASS: If there is no display where the script could ask for a password, it launches the program given as SSH_ASKPASS - this program should return the pass - typically this is a password input window. But we don't want a window, so we provide a program that simply returns our passkey. I have a script that simply echo-es my password, named 'catp', in root's home directory, chmodded to 0700. THIS IS A SECURITY RISK, however small, but I've found no other solution to this so far.

So we check whether SSH is needed (we assume this when an @ is present in either directory path), set DISPLAY to none and give our password returning script to SSH_ASKPASS, then setup ssh-agent and ssh-add our key:

Calling rsync

Finally we can call rsync. We first check whether there actually is an older backup, if not we just create a new one. If there already is a backup, we tell rsync to first look in the directory --link-dest for the files before transferring them again.

Assume a file has not changed and is already present on the backup. rsync now sees this file, sees that it's still the same, and instead of copying it again creates a file system hardlink to this file. You now have two filenames (one in the new and one in the older directory), that point to the same file, and you can still "delete" the old file - as long as another name points to the file, the file is not deleted. And as you now see, this way you can delete any backup, the other existing backups will always be unaffected, as if the file had been copied for every backup. Really nice!

Note I use --numeric-ids since I backup to another machine than where the data is stored.

Deleting old backups

This part is actually pretty basic, one could certainly expand this code. You can set a max lifetime of the backups, which I set to 180 days. Snapshots older than 180 days will be deleted by the following code.
Note I got some errors using find ... -exec rm -r {} \;, so I used a simple for loop. :)

Automating with launchd

Now it's time to tell launchd to start this backup regularly. It's easiest to use Lingon to do this. Lingon is no longer being developed, but still works for me.
Create a new Users Daemons entry, give it a name under (1) and provide what it has to do under (2). For me this is:

Then under (3) you can set to have it run whenever you like. I run it every night at 23:45 - it would probably be even better to let it run at around 3:00 in the night, but then the date of the backup folders would always be one day off. Minor issue, though.

Here is the complete XML, to be placed inside /Library/LaunchDaemons. If you do this manually, don't forget to sudo launchctl load -w /Library/LaunchDaemons/ch.my_new_backup_deamon.plist.


Have fun!

For my Eponyms iPhone App I wanted to bless SQLite with the ability to handle special chars - especially German Umlaute - like they should be handled, which e.g. is treating ä like a when sorting. Currently these Umlauts appear after Z, which is not what we want.

SQLite offers the ability to compile with complete Unicode support, provided by the ICU Project, but that will bloat the library and is not too easy to cross compile to the iPhone platform. This is also what Ioannis Epaminonda thought and he took action and provides a solution including "only" the desired features of the Unicode package, resulting in a much smaller footprint.

Here's how to get this to work for your SQLite-using iPhone App, implemented as a static library:

Create your own SQLite static library

1. Create a new Project in Xcode, use the iPhone » Library » Cocoa Touch Static Library Template. Pick a good name, the library will be called libYourName.a in the end
2. Download the SQLite Amalgamation source - that is all of SQLite in one (well, two with the additional header) file. Add the Header and the main file to the currently empty project
3. Download Ioannis source code files, and add the two files (.h and .c) to your project. Your project should now look like this:

4. Add SQLITE_CORE and SQLITE_ENABLE_UNICODE preprocessor definitions:
Call up the project's information window, select the "Build" tab and click the gogwheel icon, then select "Add User-Defined Setting". Add the two statements under the keyword GCC_PREPROCESSOR_DEFINITIONS.

5. By default the library template will enable Objective C code, but we don't need this. Remove the -ObjC flag from the Project's Other Linker Flags in the project's information window and remove the Foundation Framework from the frameworks:


6. At the very last, the unicode c file still contains the line #include <sqlite3.h>. This will include the system-wide header file, but we want it to include the header file we've just downloaded. So we add "." to our header search paths (don't check the recursive checkbox when asked):

At this point you can hit the Build Button and Xcode will already generate the libX.a static library file which you could already use. But we're taking advantage of Xcode's ability to cross reference projects, this way you'll only have to build your original project and the library will be built using your current build settings; for each build you'll have the latest version of your static library for your current settings, which is quite convenient.

Add the library to your iPhone project

1. Open your iPhone project in Xcode, the SQLite project still open
2. Drag and drop the SQLite project into the Groups & Files Section of your iPhone project:


In the sheet that pops out, do NOT check the copy items into...; we want a reference to that project, not a copy:

3. Now we tell Xcode to use our own SQLite library, not the general one we've used so far. Remove the SQLite library from Frameworks in case it's still there. No need to add our own there, it's already known to Xcode since we added a reference to our project. But we need to tell Xcode that it must copy this library to the application bundle, otherwise your users will not get that library and the app will of course not run. Our library appears right where we inserted the reference to the library project, you can simply drag it from there down to the Copy Bundle Resources target group. In case Xcode did not do this automatically, also make sure that it appears in the Link Binary With Libraries group.


4. Xcode will use the build settings of the main project also for linked projects, so make sure you add the GCC_PREPROCESSOR_DEFINITIONS also to the settings of your iPhone project (See step 4 of Create your own SQLite static library)
5. We need to load these Unicode extensions on App launch and unload them before App quit; I've included the corresponding functions in the function that initially opens a database connection and the function that closes it again:
• On launch: sqlite3_unicode_load();
• Before quit: sqlite3_unicode_free();
6. That should be all, compile and Run!

I hope this helps, in case I missed something leave a comment. In case you want to do cross referencing more often, there's a more dedicated way in doing this by adding constants to Xcode and define a shared build directory. See Clint Harris Blog for how to do this.

This one bugged me a long time, and over the weekend I decided that it's finally time to roll it: I created a simple QuickLook Plugin for CSV files.
The nice thing is - the Finder also uses this Plugin to create the Finder Icons. For speed reasons I decided to limit the display to the first 200 rows maximum, but since this thing is Open Source, you can change that whenever you like. The project website is here: http://code.google.com/p/quicklook-csv/. There you can also find a DMG containing a pre-compiled binary ready to use after download.

One thing that I'm not so sure is the UTI (Uniform Type Identifier) for CSV files. The plugin currently registers for the types listed in the Wiki, but OS X handles CSV files somewhat strange. I think the UTI for CSV files should be public.csv, analog to public.html or public.plain-text. But (at least on my Macs) OS X uses dyn.ah62d4rv4ge80g650. Well, whatever. If you discover other UTIs, please tell me so I can include these in the plugin. Otherwise it won't work for you.



Update Sept 8. 2009
Since Mac OS X 10.6 Snow Leopard, Apple uses public.comma-separated-values-text as UTI. Fine with me. :)

Update March 10, 2010
Released a small update, it should now also display previews for FileMaker exports, files ending in ".tsv" and set the badge item accordingly to "tab" instead of "csv".

Open Source Software is a great thing, developers have a chance to work together on a project and improve the product, parts of the product can be used to create or improve other apps and you can build your own version of the app.

Of course there's always some lunatics out there that want to profit from the work of others. This is exactly what happened to Mathias and me, authors of MedCalc.
The full source code to MedCalc is released under the Apache 2.0 license on Google Code, and we've already gotten some requests from people wanting to use our source. But what happened yesterday/today just blows my mind. The German guy named Sandro Stricker takes our source and puts it on the App Store as a 1$ app! He does not bother to comply to the license and all he changes is the name (from MedCalc to "X MedCalc"), the Icon, the loading screen and - the copyright and development notices! While that alone is an absolute cockiness, he also copy & pasted the contents of our support site to his website!

We've already contacted Apple and wrote him an email telling him to withdraw the app from the store momentarily. You can check out his website praising his pirated MedCalc at software.sandrostricker.de or just take a look at the comparison screenshots:

Left: Original MedCalc info screen
Right: X MedCalc info screen


Left: Original MedCalc support website
Right: X MedCalc copy & pasted support website


It's really sad to see people like him exploiting the idea behind Open Source. And I really wonder where he took the source of "his" other apps from...

Update 13.3.2009
X MedCalc as well as X Binary Clock have been pulled from the App Store. I don't know whether he did pull it himself or whether Apple did.

Another software release just became reality - today Mathias Tschopp and I submitted MedCalc for iPhone to the iTunes store, it should be available for download within a week or two (hopefully).

Mathias is the original developer of MedCalc for the Palm and WinMobile, and in the last few months we brought MedCalc to the iPhone. We've put a lot of work into this project and it was our goal to use the techniques that have made the iPhone so popular to make the best MedCalc so far. We're already very proud of the 1.0 release, but stay tuned, this is not the end!

MedCalc is Open Source Software, released under the Apache License 2.0 and will be a free download from iTunes. For more information, screenshots and movies visit the original website, and if you're missing a formula or have other suggestions, be sure to submit the contact form found on the website.

While we're at it, thank you Mathias for your hard work!

Update:
It's already available in the iTunes Store