Do It Yourself .Mac

From The Network People, Inc. - Wiki
Jump to: navigation, search

How I created my own .mac replacement

by Matt Simerson
2006.02.08 - misc updates.
2005.07.11 - v2 updated for use with Mac OS X 10.4 (Tiger).
version 1 available

Is .mac worth it to me?

Many of the reasons I do not find .mac useful are the same reasons I encourage others to use it. My needs are different than the average computer user. This is not an "I hate .mac" article but rather an explanation of the motivation and methods I used to provide myself with comparable services that are more useful to me. I publish it so that others may benefit from what I have learned.

Please don't expect free support from the author. Support requests that arrive without monetary compensation are typically ignored. Instead, try using the support forums.

You can also read about my use of .mac

Project Goals

Retain the useful features: Regardless of whether or not I renew my subscription, I want the features I have found most useful (iDisk, iSync (between computers), iCal sharing, and Backup).
Enhance the useful features: The most value can be found in enhancing each feature. For iDisk, speed and disk space are impediments to its usefulness. iSync works quite well. iCal sharing works well but publish and subscribe updates are terribly slow. Backup is hamstrung by the disk space issue.

Plan of Attack

Since most of the the .mac services revolve around the use of of webdav, the first step seems rather obvious: set up a webdav server. There are numerous documents published on this subject so I only give a brief summary of the steps I took. Then things get more complicated as I convince my desktop to use my webdav server instead of Apple's. That creates a number of problems which I successively tackle and attempt to beat into submission.

Getting it Done

Set up webdav

The first order of business is to get webdav installed and configured. I run Apache 2 on FreeBSD, so I already had mod_dav installed. It was just a matter of configuring it. I did so by creating two directories for use by WebDav. The /home/idisk directory is used by webdav clients, /var/run is used internally by Apache.

mkdir /home/idisk/html /var/run/webdav
chmod 755 /var/run/webdav
chown www:www /home/idisk/html /var/run/webdav

Note that my Apache server runs as the user www. If you run Apache as some other user (nobody is common), then make sure to alter the chown command to suit.

I then added these config lines to the Apache config files:

DavLockDB /var/run/webdav/DavLock

<VirtualHost *:80>
    ServerName idisk.cadillac.net
    DocumentRoot "/home/idisk/html"
</VirtualHost>

<Directory "/home/idisk/html">
    Dav on
    AuthType Digest
    AuthName iTools
    AuthDigestDomain "/"
    AuthDigestFile /home/idisk/WebDavUsers
    AuthGroupFile /home/idisk/WebDavGroups
    Options None
    AllowOverride None

    <LimitExcept GET HEAD OPTIONS>
         require valid-user
    </LimitExcept>

    Order allow,deny
    Allow from All
</Directory>

The next task is creating the password digest file that Apache will use for authentication.

htdigest -c /home/idisk/WebDavUsers iTools mattsimerson

Finally, I created a DNS record for idisk.cadillac.net and restarted Apache. I was able to use Finder.app's "Connect To Server" menu item and connected directly to http://idisk.cadillac.net.

Idisk1.jpg

Now I have something resembling my own iDisk. Since my web server is also on my home LAN, I am connected to it via gigabit Ethernet, which is unbelievably faster than .mac. Apple's iDisk is painfully slow, even when I had an extremely fast T3 internet connection.

Add Quota Support

One thing you may notice is that KB available is empty. The Apache 2 dav module does not include working quota support.

Andreas Amann wrote to point me at a mod_dav patch by William Carrel with quota support. The patch is available here and I've mirrored it as well. I've tested the patch and it reports back your free disk space on the server. When I enabled quotas on my FreeBSD server, it does not honor them but does report them. This is not quite ideal but better than nothing. To apply this patch you will have to rebuild Apache and apply the patch to the dav module.

Idisk2.jpg

Once you have installed Apache with the patch, reconnect and you should have quota results shown.

Configuring WebDav accounts

Now that webdav is enabled, we want to allow others to connect as well. In the following example, I set up two user accounts for myself and my wife.

cd /home/idisk/html
mkdir mattsimerson jen
chdir mattsimerson
mkdir Backup Documents Library Movies Music Pictures Public Sites Sites/.calendars
chdir ../jen
mkdir Backup Documents Library Movies Music Pictures Public Sites Sites/.calendars
chown -R www:www /home/idisk/html

Repeat the previous steps for each additional user you want to create.

Restrict access to "personal" directories by addding the following to your Apache config:

<Directory "/home/idisk/html/*/Public">
    Options +Indexes
</Directory>

<Directory "/home/idisk/html/mattsimerson">
    <LimitExcept GET HEAD OPTIONS>
       require user mattsimerson matt
    </LimitExcept>
</Directory>

<Directory "/usr/home/idisk/html/jen">
    <LimitExcept GET HEAD OPTIONS>
        require user jen
    </LimitExcept>
</Directory>

Do the same for each user that you add.

Publish iCal calendars on webdav

My wife and I enjoy being able to share each others calendars. The next step was to get iCal to publish to my new server. For consistency, I recommend publishing your calendars to the same location that Apple uses for .mac: Sites/.calendars. You can test now by setting up a test iCal calendar:

ICal-publish.jpg

After clicking publish, you should get a success notice that looks like this:

ICal-published.jpg

After updating our calendars, I also modified them so that they auto-publish after each change. When using .mac, I had them update every hour, as updating would take quite a number of seconds to complete. Now the publishing of changes is transparent. Once our calendars were all published, it was a trivial exercise to get PHP iCalendar to publish our calendars.

Emulate www.mac.com

Now things get more complicated. If you do not have experience with using Server Apache and configuring DNS servers, you may want to find someone with these skills to help you out.

The first step was to convince my mac that my new webdav server was Apple's iDisk server. I began by using tcpdump on my firewall and see where the connections where going. As of Mac OS 10.4 (Tiger), when I access the .mac control panel, it retrieves the configuration and settings from four different servers:

configuration.apple.com
syncmgmt.mac.com
idisk.mac.com
www.mac.com

Backup.app connects to both idisk.mac.com and www.mac.com. The next step is to determine exactly what the clients are looking for from the server and provide it (to the extent possible). The simplest way to redirect the traffic to my web server is to map the DNS for those four hostnames to my own server.

Configure DNS

I manage my own DNS servers so it is quite easy for me to tell them that they are authoritative for mac.com, and resolve the addresses to my servers IP. If you only need this to work on one system, simply add an entry to /etc/hosts to do the same thing. After adding the entries to /etc/hosts, run "lookupd -flushcache". You can test by running a network command like "ping www.mac.com" and you should see the traffic going to your host.

Configure Apache

To get Apache to act like idisk.apple.com, I considerably expanded the Apache directives as shown in ["http://www.tnpi.net/computing/mac/tips/idisk/idisk.cadillac.net.txt this apache include file]. I also created another include file to simulate www.mac.com. Finally, I generated a SSL certificate for www.mac.com. I did it the same way I always do, RTFM on the Apache web site. I also signed the new SSL cert with my CA key which is already trusted by all the macs on my LAN. You can also use the Mac OS Cerficate Assistant which you will find in the menu of Keychain Access on OS X 10.4.

Create emulation scripts

Next up was creating the scripts that respond as Apple's do. This was quite easy and you can read all about it on Otto's site if you're interested. The nuts and bolts are pretty simple. Since you cannot packet sniff the https connection, get the info from your Apache logs to see what URLs are being asked for. Once you have that, install a script there that dumps the POST info to a temp file. You end up with something like this in that temp file:

SERVER_SOFTWARE = Apache/2.0.55 (FreeBSD)
SERVER_NAME = www.mac.com
GATEWAY_INTERFACE = CGI/1.1
SERVER_PROTOCOL = HTTP/1.1
SERVER_PORT = 443
REQUEST_METHOD = POST
SCRIPT_NAME = /WebObjects/Info.woa/wa/Query/retrieveDiskConfiguration
REMOTE_ADDR = 10.0.1.218
CONTENT_LENGTH = 160
{
   body = {relativePath = Public; }; 
   function = retrieveDiskConfiguration; 
   header = { 
      password = ******; username = mattsimerson; version = 1; 
   }; 
}

I copied and pasted the {} info into the file foo and then used lynx to see what Apple's server returns. When I sent this request:

lynx -source -post_data -useragent="InternetPref/Version-10.2" \
https://www.mac.com/WebObjects/Info.woa/wa/Query/retrieveDiskConfiguration < foo

I got back this response:

{
   payload = {
     guestReadEnabled = Y;
     guestWriteEnabled = N;
     hasGeneralPassword = N;
     iDiskQuotaInBytes = 104857600;
     iDiskUsedBytes = 11383808;
     relativePath = Public;
   };
   statusCode = success;
}


Knowing what the script needs to return, create the script:

mkdir -p /home/www.mac.com/WebObjects/Info.woa/wa/Query
vi /home/www.mac.com/WebObjects/Info.woa/wa/Query/retrieveDiskConfiguration

The contents of that file should be something like what follows:

#!/bin/sh

echo Content-type: text/plain
echo

cat << EOT
{
   payload = {
       guestReadEnabled = Y;
       guestWriteEnabled = N;
       hasGeneralPassword = N;
       iDiskQuotaInBytes = 1048576000;
       iDiskUsedBytes = 339338752;
       relativePath = Public;
   };
   statusCode = success;
}
EOT

Now when you access the iDisk tab in the .mac control panel, it gets its values from the script you just created.

Dominic Rivera contributed a script that returns the disk free output instead of using a static value for disk space.

Idisk-cp.jpg

Notice that I now have 1GB of space available. It is, in effect, infinite because no matter how much data I upload, I will always have 663 MB free. If I were a service provider, I would rewrite the script in Perl or C, have it parse the POST data, verify the authentication parameters, and return actual disk quota values. Since I am only doing this for myself and my wife, that is not important.

I can connect to my iDisk using Finder.app "Go->iDisk->My iDisk" menu, or the Cmd-Shift-I keyboard shortcut. When I connect, it is wicked fast and I have ample disk space available.


Idisk-simulated.jpg


Get Backup.app working

Upon running Backup.app, a check of my Apache log files revealed that it was checking the URL https://www.mac.com/WebObjects/Info.woa/wa/Query/accountInfo. As before, we install a script there to capture the POST data