The focus will continue to be on scripting and automations, but a lot has changed since the last post and so I might find new things to post about. There will be RSS feeds for each category and tag so if some things don't interest you, just subscribe to the things that you like.

Happy New Year!

Not too long ago, Omnifocus added the ability to import tasks in Taskpaper format. The import initially seemed very basic and only allowed you to dump them into Omnifocus' Inbox for processing. Digging into the documentation shows that you can be more specific about what project you want those tasks to go into as well as context, defer and due dates.

Up until now, if I wanted to create multiple tasks for tonight, I would have to add each one individually in Drafts and use my "Due Tonight" action, return to Drafts and repeat. Using the Taskpaper syntax import, I can now add multiple lines as tasks in Drafts and import. Each line would look something like this:

- Take out the garbage @flagged(true) @context(Admin & Routines) @defer(6pm) @due(8pm)
- Finish blog post @flagged(true) @context(Admin & Routines) @defer(6pm) @due(8pm)

Since Drafts supports JavaScript, I can create a single script to support my tonight, weekend or today tasks.

targetTask = 'Todo';
defaultTaskString = '@flagged(true) @context(Admin & Routines)';

dueDateStrings = {
    today: '@defer(now) @due(5pm)',
    tonight: '@defer(6pm) @due(8pm)',
    weekend: '@defer(next saturday @ 10am) @due(next sunday @ 5pm)'
};

function generateOmnifocusUrl(params = {}){
    let {contentLines = [], target = targetTask, dueTime = dueDateStrings.today} = params;

    contentLines.forEach((line, index, theArray) =>
        theArray[index] = `- ${line} ${dueTime} ${defaultTaskString}`
    );
    content = encodeURIComponent(contentLines.join('\n'));
    return `omnifocus:///paste?target=/task/${target}&content=${content}`;
}

params = {
    contentLines: draft.content.split('\n'),
    dueTime: dueDateStrings[draft.getTag('taskWhen')],
}

draft.defineTag("OFTasks", generateOmnifocusUrl(params));

The action will first prompt me asking whether these lines are for tonight, this weekend or today and will then create the TaskPaper lines and open Omnifocus.

You can download the action for Drafts here.

I've been spending a considerable amount of my time lately writing Apex code (Salesforce's proprietary language similar to Java and C#) and have come to appreciate it's ability to serialize different objects. Python isn't particularly good at this, and so I initially struggled with coming up with a clean way of implementing the generator. The new JSON feed spec has many nested objects and so representing these as separate classes made sense. Let's look at an author

class Author(Object):
    def __init__(self, name, url=None, avatar=None):
        self.name = name
        self.url = url
        self.avatar = avatar

This is a basic representation of an author based on JSON feed's spec. If we were to simply try to serialize this in Python using the json library, we'd come across this exception

TypeError: <__main__.Author object at 0x107d23e10> is not JSON serializable

The json library allows you to pass in your own custom parser, and so I created a base class for all my my objects that would contain one method that the parser would look for as a way to tell it how to serialize each class.

import json

class JSONEncoder(json.JSONEncoder):
    def default(self, obj):
        if hasattr(obj, 'as_json'):
            return obj.as_json()
        else:
            return json.JSONEncoder.default(self, obj)

class Base(object):
    def as_json(self):
        return self.__dict__

class Author(Base):
    def __init__(self, name, url=None, avatar=None):
        self.name = name
        self.url = url
        self.avatar = avatar

Now we can call the same method, while passing in our custom JSON encoder to serialize our class

a = Author('Ryan M')
json.dumps(a, cls=JSONEncoder)
# '{"url": null, "name": "Ryan M", "avatar": null}'

Now it's just a matter of building a class for each object in the JSON feed top-level object

class Item(Base):
    pass

# a list of Item classes, since there are many
class Items(list):
      # list type doesn't have a __dict__ accessor, so we just return the list to be serialized
    def as_json(self):
        return self

# The top-level JSON feed object containing all child objects
class JsonFeed(Base):
    pass

I've left out the implementation details for generating each of these objects for brevity, but the idea is all there. Each class now knows how to tell the json encoder how to be serialized, so it's just a matter of implementing the Pelican plugin and writing the output.

class JsonFeedGenerator(object):
    def __init__(self, article_generator):
        self.articles = article_generator.articles
        self.settings = article_generator.settings
        self.context = article_generator.context
        self.generator = article_generator

        self.path = 'feed.json'

        self.site_url = article_generator.context.get('SITEURL',
                                                      path_to_url(get_relative_path(self.path)))

        self.feed_domain = self.context.get('FEED_DOMAIN')
        self.feed_url = '{}/{}'.format(self.feed_domain, self.path)

    def write_feed(self):
        complete_path = os.path.join(self.generator.output_path, self.path)
        try:
            os.makedirs(os.path.dirname(complete_path))
        except Exception:
            pass

        with open(complete_path, 'w') as f:
            json.dump(JsonFeed.from_generator(self), f, cls=JSONEncoder)

def get_generators(article_generator):
    json_feed_generator = JsonFeedGenerator(article_generator)
    json_feed_generator.write_feed()

def register():
    signals.article_generator_finalized.connect(get_generators)

You can see the feed for this blog here. The source for the entire plugin can be found on Github here. The plugin should work for all sites right now. I chose not to implement multiple languages into the feed since it doesn't seem like the spec supports this. Hopefully they consider this as they improve the format.

Over the years, I've built version of a workflow that lets me share files with Dropbox. They've always been very specific to me and never reliable or secure enough to share with others. The Dropbox API has come a long ways and now gives me the ability to share this workflow without exposing my app secret.

This is my primary way of searching for files. I have Alfred configured so that I can hit the right arrow to take me to the actions menu.

You'll see that I have two actions set up. One for simply creating a link and another for creating a link that expires in a week. The latter is only possible if you have a Dropbox Pro account.

Once you've selected one of the options, a notification will appear telling you that the link was created and the link will then be in your clipboard.

You can download the workflow by clicking the Alfred icon below. Instructions for setting up the workflow can be found by clicking on the [x] in the top-right of the workflow once it's installed. Alfred doesn't do a great job making it easy to find the set-up instructions.

Features

• Supports multiple accounts if you have a personal and business account
• You can create any number of expiring links by creating another action and modifying the Alfred Workflow JSON expires key to a number of days
• Do to all of the different permissions that Dropbox offers for their business product, if a link already exists for the file you're trying to share with more restrictive permissions, a link won't be created. It's too difficult to expose what permissions that exist in a notification bubble.

The easy solution is to simply point iTunes to your photo library in Dropbox and sync your photos over. This has a few downsides:

1. If your photo library is large, you can't sync everything over
2. iTunes doesn't let you sync multiple folders, so its all or nothing
3. iTunes creates a thumbnail cache in the folder of photos, which means you have a large folder constantly syncing to Dropbox, which isn't ideal

I already use Hazel to sort and organize my photos, so I figured adding another workflow would be fairly easy. On the computer where I sync my iPhone to iTunes, I set up a workflow that looks like this

Since my photo folder structure looks like [year]/[month]/[event], I need to look at each of the photos and then continue the workflow if it matches a matches a shell command

As long as what Hazel processes is a folder, and matches this regular expression, we can continue on to process the photos. This regular expression looks for a folder path that contains the the numbers 2015 through 2019¹. You can change the 5 to be whatever range you need, but I didn't need to go back to photos older than that.

The next step is creating a hard link to the photo to a new folder I keep in ~/Pictures called "Photos for iPhone."

A hard link is nice here since it simply references the original file and doesn't take up space on your hard drive.

Now I can point iTunes to my newly created folder. Each time a new photo gets added to my Dropbox photos folder, a new hard link is created and then synced to my iPhone the next time I plug it in.

Setting keyboard shortcuts on Mac is actually fairly easy, but it requires a lot of clicking around. Fortunately there's a way to do this from the terminal that's faster and easier.

The defaults command in MacOS is nothing short of a mystery. It does some powerful things, but the documentation is sparse and half of the time I don't know what I'm doing. That being said, I've had a script written for a long time called new_computer.sh where I set all of my favorite global and application-specific shortcuts when getting a new computer.

Let's take an example of a shortcut everyone should have: Print As PDF from within a print dialog. I've always set it to ⌘ ⇧ P. To do this within System Preferences, the steps are:

1. Open the Keyboard preference Pane
2. Click the Shortcuts tab
3. Click App Shortcuts
4. Click the + symbol
5. Fill out the prompts
   • Leave All Applications Selected
   • Menu Title is "Save as PDF…" (it's an elipsis, not three periods. Type option ; to get the symbol)
   • Choose your shortcut

It's almost too many steps for one shortcut, let alone multiple. Let's try this in Terminal:

defaults write -globalDomain NSUserKeyEquivalents  -dict-add "Save as PDF\\U2026" "@\$p";

Easy, right? Sort of. The syntax for writing global shortcuts is fairly straight forward. If you're not creating a shortcut for a specific application, you can use the command above and simply change the title and shortcut. Here are how to represent all of the modifier keys:

• @ is command
• ^ is control
• ~ is option
• $ is shift

So command-shift p becomes "@\$p".

The reason this came up was that Omnifocus recently added tabs. This is great except that there's no shortcut for cycling through the tabs. This makes the feature almost pointless for me. So to add shortcuts, I ended up using the command above, but I need to target Omnifocus only.

defaults write com.omnigroup.OmniFocus2 NSUserKeyEquivalents -dict-add "Show Next Tab" "^\\U005D"
defaults write com.omnigroup.OmniFocus2 NSUserKeyEquivalents -dict-add "Show Previous Tab" "^\\U005B"

Here I'm setting show next/previous tab to control [ and control ]. Once you've set your keyboard shortcuts, you'll need to quit and re-launch the application in order for the new preferences to be read.

I've not been a heavy user of Day One, and with the new version, I've stopped entirely until they provide end-to-end encryption with their proprietary sync service. Journaling my bike trip isn't anything I'm worried about being out in the open, and so I'll use it to keep a log of my days on the trip. At the same time, I want to keep my friends and family up-to-date on my trip. Since I use Pelican for this site, it seemed like a reasonable choice to use it for this trip and use Github Pages as an easy, free place to host it.

The first step was getting the Pelican site set up. I used the basic quickstart and put in a custom theme that I found online. The only modifications I made was using the photos plugin to make it easier to add galleries if I want in the future. Publishing to Github Pages is trivial. You can follow the steps here.

Now the fun part. Day One lets you export a journal entry as Markdown. When exported, it's compressed into a zip file which includes a folder of photos if you've included any in the journal entry. For each post, I use the export action and then upload to a folder I've created in Dropbox. I have Hazel watching this folder which will do the following:

1. Unarchive any file that appears
2. Move the unarchived contents into a new folder I unoriginally name "decompressed"

I then have a separate rule watching "decompressed" which will

1. Move any image file type into my blog's images folder
2. Move any text file into the content folder

Step 2 here requires a little bit of extra work. Day One has some weird formatting issues and I also need to update the image urls in the entry to match what Pelican expects. The script isn't my finest, but it takes care of everything

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

#!/usr/bin/python

import codecs
import re
import sys
from datetime import datetime

input_file = sys.argv[1]

f = codecs.open(input_file,
                mode='r',
                encoding='utf-8').read()

# Get rid of the tabs that DayOne inserts
f = f.replace(u'\tDate:', 'Date:')
f = f.replace(u'\tWeather:', 'Weather:')
f = f.replace(u'\tLocation:', 'Location:')


# Replace default Markdown image syntax with Pelican's syntax + photos plugin
f = f.replace('![](photos/', '![]({photo}/')


title_re = re.compile(r'\n\n#\s+(.*)\n')
title_search = title_re.search(f)
now_datestring = datetime.now().strftime('%B %d, %Y at %H:%M:%S %Z')

# We need a title: header for Pelican
if title_search:
    f = 'Title: %s\n' % title_search.group(1) + f
    f = title_re.sub('', f) + "\n"
else:
    f = 'Title: Update %s\n' % now_datestring + f

if "Date:" not in f:
    f = "Date: %s\n" % now_datestring + f

with codecs.open(input_file, mode='w', encoding='utf-8') as new_file:
    new_file.write(f)

Now the file is cleaned up and in the right place. We can now publish and push to Github.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

#!/bin/bash

cd ~/Dropbox/blogs/biketour/pelican_site

make publish

git add ..

git commit -am 'update blog'

/Users/rjames/dev/pelican/bin/ghp-import output

git push git@github.com:rjames86/rjames86.github.io.git gh-pages:master

That's it! You can see the posts and follow my bike tour at http://rjames86.github.io

Nearly every day I come home from work to a slew of notifications from my day.

One way to close all these is to open up the Notification Center panel and click all of the X's, which will also clear out your notifications. That's still janky and I'd rather have this done without having to click a bunch of times.

I first wrote the script using Applescript. I ran into some annoying issues when trying to ignore certain notification windows, such as System Updates. I wanted to ignore that window entirely, and since Applescript doesn't have a continue in for-loops I opted to use Javascript which ended up being easier.

var app = Application("System Events")

notificationCenter = app.processes.byName('NotificationCenter')

function closeWindow(window){
    window.buttons.whose({
        _or: [
            {name: "Close"},
            {name: "OK"}
        ]
    })().forEach(function(button){button.click()})
    delay(0.1); // The UI can't always keep up, so we introduce a short delay
}

notificationCenter.windows().forEach(closeWindow)

I'm currently running this in Keyboard Maestro, but it could just as easily be run with Alfred. I've made very basic versions using both for download:

• Alfred
• Keyboard Maestro

I had been thinking this weekend whether it would be worth the time to try to migrate all my snippets to Keyboard Maestro. Browsing my Twitter feed, it looked as though Dr. Drang had beat me to it. Unfortunately he didn't do the work I was hoping I wouldn't have to do, and so I sat down to see how hard it would be to convert snippets to macros. Turns out...not that hard.

Here are the requirements for running this script:

• Python 2.7 (I didn't test Python 3.x). If you're running an older/newer version of Python, you should be able to replace python with /usr/bin/python2.7 when running the script.
• TextExpander 5.x. If you are running version 4, your settings will be named Settings.textexpander instead of Settings.textexpandersettings.

I've made some decisions as to how I want the snippets to work. Notably

• Pasting instead of typing. Typing is too slow.
• Delete the last clipboard item, since it was the text that was just pasted.
• Groups remain the same. If you used groups in TextExpander, they show up as "Snippets - " in Keyboard Maestro.

There are a few things that I haven't yet solved. Some I might in the future, others maybe not:

• I didn't test Applescript since I didn't have any. Please let me know if that one breaks.
• Placeholders and variables from TextExpander won't work. This means if you had a "today's date" snippet, you'll need to rewrite that one¹.
• Custom delimiters. I haven't really figured out if there's a way to do this. I've tried changing Keyboard Maestro to only fire on delimiters, but it doesn't seem to work. If anyone figured this out, please let me know.

Before running, be sure to update the variable TEXTEXPANDER_PATH to wherever your TextExpander settings file lives. To run, it's as simple as navigating to the location where the script lives in Terminal.app and entering

python TE.py

You'll now have a folder on your Desktop named 'TextExpander_to_KeyboardMaestro' with all of your groups.

Update 2016-04-12

Big thanks to NW in the comments for helping me debug a few things.

I've removed the import of enum. I had forgotten that wasn't a standard library in Python 2.7. I also added a list of requirements above for running the script.

Update 2016-04-14

Also thanks to Dr Drang for posting about the script! I've made some updates and also put up a repo for those who would like to make edits and pull requests.

Updates are

• Optional prefix if you want to change that up when moving to Keyboard Maestro
• Insert text by typing OR pasting
• Added some instruction on how to edit the variables to have the script do what you want

The repo is now hosted at https://github.com/rjames86/textexpander_to_keyboardmaestro

You can download the script on Github here

import plistlib
import os
import glob

'''
This script will parse through all group_*.xml files within your TextExpander folder.
Anything marked as Plain Text, Shell Script or JavaScript should be converted into
Keyboard Maestro groups with the same title and abbreviation.

All new KM Macro files will be saved to the Desktop.

'''

# Modify this area to customize how the script will run

# Change this path to where ever your TextExander Settings live
HOME = os.path.expanduser('~')
TEXTEXPANDER_PATH = HOME + '/Dropbox/TextExpander/Settings.textexpandersettings'
SAVE_PATH = HOME + '/Desktop/TextExpander_to_KeyboardMaestro'

# Change this if you'd like to change your snippets when importing to Keyboard Maestro
# If your snippet is ttest, you can make it ;;ttest by changing the variable to ';;'
OPTIONAL_NEW_PREFIX = ''

# Change this if you want the snippet to inserted by typing or pasting
# Remember it MUST be 'paste' or 'type' or the script will fail
PASTE_OR_TYPE = 'paste' # 'type'




############

# Edit below at your own risk

############

snippet_types = {
    'plaintext': 0,
    'applescript': 2,
    'shell': 3,
    'javascript': 4,
}

snippet_types_to_values = dict((value, key) for key, value in snippet_types.iteritems())


class KeyboardMaestroMacros(object):
    @classmethod
    def macro_by_name(cls, macro_name, group_name, name, text, abbreviation):
        return getattr(cls, macro_name)(group_name, name, text, abbreviation)

    @staticmethod
    def javascript(group_name, name, text, abbreviation):
        return {
            'Activate': 'Normal',
            'CreationDate': 0.0,
            'IsActive': True,
            'Macros': [
                {'Actions': [
                    {'DisplayKind': KeyboardMaestroMacros._paste_or_type(),
                     'IncludeStdErr': True,
                     'IsActive': True,
                     'IsDisclosed': True,
                     'MacroActionType': 'ExecuteJavaScriptForAutomation',
                     'Path': '',
                     'Text': text,
                     'TimeOutAbortsMacro': True,
                     'TrimResults': True,
                     'TrimResultsNew': True,
                     'UseText': True}, {
                        'IsActive': True,
                        'IsDisclosed': True,
                        'MacroActionType': 'DeletePastClipboard',
                        'PastExpression': '0'}
                    ],
                 'CreationDate': 482018934.65354,
                 'IsActive': True,
                 'ModificationDate': 482018953.856014,
                 'Name': name,
                 'Triggers': [{
                    'Case': 'Exact',
                    'DiacriticalsMatter': True,
                    'MacroTriggerType': 'TypedString',
                    'OnlyAfterWordBreak': False,
                    'SimulateDeletes': True,
                    'TypedString': KeyboardMaestroMacros._abbreviation(abbreviation)}]}
            ],
            'Name': 'Snippet - %s' % group_name,
        }

    @staticmethod
    def applescript(group_name, name, text, abbreviation):
        return {
            'Activate': 'Normal',
            'CreationDate': 0.0,
            'IsActive': True,
            'Macros': [
                {'Actions': [
                    {'DisplayKind': KeyboardMaestroMacros._paste_or_type(),
                     'IncludeStdErr': True,
                     'IsActive': True,
                     'IsDisclosed': True,
                     'MacroActionType': 'ExecuteAppleScript',
                     'Path': '',
                     'Text': text,
                     'TimeOutAbortsMacro': True,
                     'TrimResults': True,
                     'TrimResultsNew': True,
                     'UseText': True}, {
                        'IsActive': True,
                        'IsDisclosed': True,
                        'MacroActionType': 'DeletePastClipboard',
                        'PastExpression': '0'}
                    ],
                 'CreationDate': 482018934.65354,
                 'IsActive': True,
                 'ModificationDate': 482018953.856014,
                 'Name': name,
                 'Triggers': [{
                    'Case': 'Exact',
                    'DiacriticalsMatter': True,
                    'MacroTriggerType': 'TypedString',
                    'OnlyAfterWordBreak': False,
                    'SimulateDeletes': True,
                    'TypedString': KeyboardMaestroMacros._abbreviation(abbreviation)}]}
            ],
            'Name': 'Snippet - %s' % group_name,
        }

    @staticmethod
    def plaintext(group_name, name, text, abbreviation):
        return {
            'Activate': 'Normal',
            'CreationDate': 0.0,
            'IsActive': True,
            'Macros': [{'Actions': [
                {
                    'Action': KeyboardMaestroMacros._paste_or_type('plaintext'),
                    'IsActive': True,
                    'IsDisclosed': True,
                    'MacroActionType': 'InsertText',
                    'Paste': True,
                    'Text': text}, {
                        'IsActive': True,
                        'IsDisclosed': True,
                        'MacroActionType': 'DeletePastClipboard',
                        'PastExpression': '0'
                    }],
                'CreationDate': 0.0,
                'IsActive': True,
                'ModificationDate': 482031702.132113,
                'Name': name,
                'Triggers': [{
                    'Case': 'Exact',
                    'DiacriticalsMatter': True,
                    'MacroTriggerType': 'TypedString',
                    'OnlyAfterWordBreak': False,
                    'SimulateDeletes': True,
                    'TypedString': KeyboardMaestroMacros._abbreviation(abbreviation)}],
            }],
            'Name': 'Snippet - %s' % group_name,
        }

    @staticmethod
    def shell(group_name, name, text, abbreviation):
        return {
            'Activate': 'Normal',
            'CreationDate': 0.0,
            'IsActive': True,
            'Macros': [{
                'Actions': [{
                    'DisplayKind': KeyboardMaestroMacros._paste_or_type(),
                    'IncludeStdErr': True,
                    'IsActive': True,
                    'IsDisclosed': True,
                    'MacroActionType': 'ExecuteShellScript',
                    'Path': '',
                    'Text': text,
                    'TimeOutAbortsMacro': True,
                    'TrimResults': True,
                    'TrimResultsNew': True,
                    'UseText': True},
                 {'IsActive': True,
                  'IsDisclosed': True,
                  'MacroActionType': 'DeletePastClipboard',
                  'PastExpression': '0'}],
                'CreationDate': 482018896.698121,
                'IsActive': True,
                'ModificationDate': 482020783.300151,
                'Name': name,
                'Triggers': [{
                    'Case': 'Exact',
                    'DiacriticalsMatter': True,
                    'MacroTriggerType': 'TypedString',
                    'OnlyAfterWordBreak': False,
                    'SimulateDeletes': True,
                    'TypedString': KeyboardMaestroMacros._abbreviation(abbreviation)}],
                }],
            'Name': 'Snippet - %s' % group_name,
        }

    @staticmethod
    def _abbreviation(name):
        return OPTIONAL_NEW_PREFIX + name

    @staticmethod
    def _paste_or_type(snippet_type=None):
        value = {
            'paste': "Pasting",
            'type': "Typing"
        }
        if snippet_type == 'plaintext':
            return "By%s" % value[PASTE_OR_TYPE]
        else:
            return value[PASTE_OR_TYPE]


def parse_textexpander():
    '''
    Each TextExpander group is its own file starting with the file name 'group_'.

    Example snippet dictionary
    {
        'abbreviation': '.bimg',
        'abbreviationMode': 0,
        'creationDate': datetime.datetime(2013, 5, 19, 19, 42, 16),
        'label': '',
        'modificationDate': datetime.datetime(2015, 1, 10, 20, 19, 59),
        'plainText': 'some text,
        'snippetType': 3,
        'uuidString': '100F8D1F-A2D1-4313-8B55-EFD504AE7894'
    }

    Return a list of dictionaries where the keys are the name of the group
    '''
    to_ret = {}

    # Let's get all the xml group files in the directory
    xml_files = [f for f in glob.glob(TEXTEXPANDER_PATH + "/*.xml")
                 if f.startswith(TEXTEXPANDER_PATH + "/group_")]

    for xml_file in xml_files:
        pl = plistlib.readPlist(xml_file)
        if pl['name'] not in to_ret:
            to_ret[pl['name']] = []
        for snippet in pl['snippetPlists']:
            if snippet['snippetType'] in snippet_types.values():
                to_ret[pl['name']].append(snippet)
    return to_ret


def main():
    text_expanders = parse_textexpander()
    for group, text_expander in text_expanders.iteritems():
        macros_to_create = []
        for snippet in text_expander:
            macros_to_create.append(
                KeyboardMaestroMacros.macro_by_name(snippet_types_to_values[snippet['snippetType']],
                                                    group,
                                                    snippet['label'],
                                                    snippet['plainText'],
                                                    snippet['abbreviation'])
                )

        # Create a new folder on the desktop to put the macros
        if not os.path.exists(SAVE_PATH):
            os.mkdir(SAVE_PATH)
        # Save the macros
        with open(SAVE_PATH + '/%s.kmmacros' % group, 'w') as f:
            f.write(plistlib.writePlistToString(macros_to_create))

if __name__ == '__main__':
    main()

Feedback and pull requests welcome. I'll continue to update the script and this post if I make any substantial iterations.

Last week at work I was working on a fairly large refactor of our front-end. Large pieces of code were being moved around and others re-written to be cleaner and more understandable. Throughout this process, I was leaving myself todo's so that I'd remember to fix something later. Problem is, I would rarely ever go back to them. That was until someone on my team shared some bash functions they had written to make following up on those todo's much easier

It's fairly common practice to leave yourself todo's as comments in code such as

# TODO(ryan) fix this later.

That way if someone comes across it in the future, they'll know that whatever is below may not be perfect and that I plan on fixing it at some point. Finding all your todo's later is a different story. That's where some fancy bash functions come in handy.

function ga_code_search() {
    # alias todo='ga_code_search "TODO\(`whoami`\)"'
    SCREEN_WIDTH=`stty size | awk '{print $2}'`
    SCREEN_WIDTH=$((SCREEN_WIDTH-4))
    # Given a spooky name so you can alias to whatever you want. 
    # (cs for codesearch)
    # AG is WAY faster but requires a binary 
    # (try brew install the_silver_searcher)
    AG_SEARCH='ag "$1" | sort -k1 | cat -n | cut -c 1-$SCREEN_WIDTH'

    # egrep is installed everywhere and is the default.
    GREP_SEARCH='egrep -nR "$1" * | sort -k1 | cat -n | cut -c 1-$SCREEN_WIDTH'

    SEARCH=$AG_SEARCH

    if [ $# -eq 0 ]; then

        echo "Usage: ga_code_search <search> <index_to_edit>"
        echo ""
        echo "Examples:"
        echo "    ga_code_search TODO"
        echo "    ga_code_search TODO 1"
        echo "    ga_code_search \"TODO\\(graham\\)\""
        echo "    ga_code_search \"TODO\\(graham\\)\" 4"
        echo ""        
        return
    fi

    if [ $# -eq 1 ]; then
        # There are no command line argumnets.
        eval $SEARCH
    else
        # arg one should be a line from the output of above.
        LINE="$SEARCH | sed '$2q;d' | awk -F':' '{print +\$2 \" \" \$1}' | awk -F' ' '{print \$1 \" \" \$3}'"
        # Modify with your editor here.
        emacs \+`eval $LINE`
    fi    
}

If you read through the comments, the_silver_searcher is far faster than grep for searching contents of files. If you don't have it already, I'd highly suggest installing it with brew install the_silver_searcher. If you don't want to, be sure to change SEARCH=$AG_SEARCH to SEARCH=$GREP_SEARCH.

The function itself isn't that interesting. It's when you assign aliases to use this function that things become interesting. Here are the three that were given to me:

# Find todo items that are assigned to me. TODO(ryan)
# You can change `whoami` to whatever you want.
alias todo='ga_code_search "TODO\(`whoami`\)"'

# Find merge conflicts that need to be resolved.
alias conflicts='ga_code_search "<<<<<<<<<"'

# Find anything below your CWD.
# You can now type `cs some_piece_of_code`
alias cs='ga_code_search'

My favorite by far is the first alias todo. Here is some example output when running this command:

> my_project (master): todo
 1  app/models/strava.py:102: # TODO(ryan) probably should memoize this at some point so its faster.
 2  app/models/strava.py:148: # TODO(ryan) make this line prettier
 3  app/templates/strava/index.html:50: <!-- TODO(ryan) move this into its own template file at some point -->

Notice how there are numbers next to each result? That's because you can also open the file right to that todo item by typing todo 1! As the function is written, it will open in emacs. If that's your editor of choice, you'll be set. I'm personally a fan of Sublime Text. There's a way to also open a file in Sublime Text to a specific line number. Simply change the text in red with that in green:

- LINE="$SEARCH | sed '$2q;d' | awk -F':' '{print +\$2 \" \" \$1}' | awk -F' ' '{print \$1 \" \" \$3}'"
+ LINE="$SEARCH | sed '$2q;d' | awk -F':' '{print +\$2 \" \" \$1}' | awk -F' ' '{print \$3 \":\" \$1}'"

- emacs \+`eval $LINE`
+ subl `eval $LINE`

I've only used the functions for a few days now, but it's greatly improved my workflow for getting old todo's done in code. If you'd like to download these scripts, here is the Sublime Text version and the emacs version.

Up until OS X 10.11 El Capitan, Server Favorites were stored in a plist file called com.apple.sidebarlists.plist. I finally got around to upgrading my computers at home only to realize that my "server" workflow stopped working. After inspecting the plist file, I found that those favorites were gone and were hiding elsewhere. After a bunch of searching, and the help of Houdaspot, I found them in ~/Library/Application Support/com.apple.sharedfilelist/com.apple.LSSharedFileList.FavoriteServers.sfl. What is this sfl extension? Still not really sure, but after some poking around, this was the only resource I could find that helped me get started.

I don't necessarily like the CoreFoundation stuff in Python, and since I'm on a OS X JavaScript automation roll right now, I decided to give it a try. Turns out, it's really easy.

items = $.NSKeyedUnarchiver.unarchiveObjectWithFile('/Users/username/Library/Application Support/com.apple.sharedfilelist/com.apple.LSSharedFileList.FavoriteServers.sfl')

items = items.objectForKey('items')
itemsCount = items.count

to_ret = []
while (itemsCount--){
      item = items.objectAtIndex(itemsCount)
    to_ret.push(
        {
            name: item.name, 
            url: item.URL.absoluteString
        }
    )
}

to_ret

This returns a nice object of the name and url for the servers in your favorites.

You can download the workflow here.

The documentation is just as sparse in the developer documentation, however this article by Tyler Gaw helped get me started in understanding how to represent Objective-C methods in Javascript. It's probably easiest to just show the script and explain what's going on.

ObjC.import("AddressBook");
sAB = $.ABAddressBook.sharedAddressBook
meRecord = sAB.me

var propertyToObjCType = {
    'email': $.kABEmailProperty,
    'address': $.kABAddressProperty,
    'phone': $.kABPhoneProperty
}

var labelToObjCType = {
    'work': $.kABWorkLabel,
    'home': $.kABHomeLabel,
    'iPhone': $.kABPhoneiPhoneLabel,
}

function valueForProperty(property){
    return meRecord.valueForProperty(propertyToObjCType[property])
}

function getEmailByLabel(inputLabel){
    emails = valueForProperty('email')
    label = labelToObjCType[inputLabel]
    for (var i = 0; i < emails.count; i++){
        if ($.CFEqual(emails.labelAtIndex(i), label)){
            return emails.valueAtIndex(i)
        }
    }

}

function getAddressByLabel(inputLabel){
    addresses = valueForProperty('address')
    label = labelToObjCType[inputLabel]
    for (var i = 0; i < addresses.count; i++){
        if ($.CFEqual(addresses.labelAtIndex(i), label)){
            return sAB.formattedAddressFromDictionary(addresses.valueAtIndex(i)).string
        }
    }

}

function getPhoneByLabel(inputLabel){
    phone = valueForProperty('phone')
    label = labelToObjCType[inputLabel]
    for (var i = 0; i < phone.count; i++){
        if ($.CFEqual(phone.labelAtIndex(i), label)){
            return phone.valueAtIndex(i)
        }
    }

}

The biggest thing to point out is that if you have a method called in Objective-C like [ABAddressBook sharedAddressBook];, this gets converted to dot notation $.ABAddressBook.sharedAddressBook. The Obj-C bridge is always called with either ObjC. or $. followed by the method.

You can find a nice list of different properties and values for the address book here. For labels, the most common will be $.kABHomeLabel and $.kABWorkLabel for home and work respectively. If you've created a custom label (let's call it XXX), you can reference it by calling $.kABXXXLabel.

As with any other JavaScript snippet in TextExpander, you can call any of these functions to expand the contact information that you'd like.

getEmailByLabel('home')  // returns your home phone number

getAddressByLabel('work')  // returns your work address

getAddressByLabel('iPhone')  // returns your phone number labeled iPhone

For those of you who don't like copy/pasting the same code over and over, there's a nice little hack that you can do in TextExpander.

First, create a new Plain Text snippet with the code from the top of the post. I called the snippet "getInfoFromContacts". Once that's done, you can create new snippets that take advantage of this code by creating new JavaScript snippets with the following:

%snippet:getInfoFromContacts%
getEmailByLabel('home')

This way, if you update something from the main part of the code, you don't have to update all of your TextExpanders.

The first step was to generate a separate css file that only contained what was needed when you first load and see the page. I use Less as my pre-processor, and created a very small Less file that looked like this:

@import (inline) '../tipuesearch/tipuesearch.css';

@import 'default_mobile.less';
@import 'largescreens.less';

Once compiled and minimized¹, I needed to add it to my base.html template.

<style type="text/css">
{% include 'critical.css' %}
</style>

Here is where the problem when generating my site.

WARNING: Caught exception "TemplateSyntaxError: Missing end of comment tag". Reloading.

Since my minimized CSS contained '{#', Jinja was interpreting this as a comment and raised an exception. While this is an easy fix by changing the Jinja environment variables within Pelican's generators.py, I didn't want to go this route since I would need to update this² every time there was an update to Pelican. Instead, I wrote a Jinja extension which Pelican supports natively.

# in pelicanconf.py
from jinja2.ext import Extension

class CustomCommentStrings(Extension):
    def __init__(self, environment):
        super(CustomCommentStrings, self).__init__(environment)

        environment.comment_start_string = '###'
        environment.comment_end_string = '/###'

JINJA_EXTENSIONS = [CustomCommentStrings]

Update 2017-01-05

If you're using Pelican version 3.7+, you don't have to write the custom extension shown above, you can simply update the JINJA_ENVIRONMENT settings variable:

JINJA_ENVIRONMENT = {
    'comment_start_string': '###', 
    'comment_end_string': '/###'
}

One thing to note here is that if you are using {# ... #} as comment strings in Jinja, you'll need to update them to whatever new start and end strings you define.

And success! The critical.css file was successfully imported and I now my critical CSS is included on page load. With this, Google now gives me a 100/100 speed score for mobile and 98/100 on desktop.

When I initially ran the test, this site came back with a score of around 41/100 for both desktop and mobile. I would have been fine leaving it, but that was pretty bad. Google does a great job telling you what things need to be improved, even down to the specific files causing problems.

My first task was "eliminating render-blocking JavaScript and CSS." I was lazily loading all of my JavaScript in the <head> tag, so this was as simple as moving that to the bottom of the page. Google also suggested using the async attribute.

<!-- Initial setup -->
<!DOCTYPE html>
<html lang="{{ DEFAULT_LANG }}">
<head>
    <script async src="{{ SITEURL }}/theme/js/main.js" type="text/javascript"></script>
  ...
</head>

<!-- New setup -->
...
    <script async src="{{ SITEURL }}/theme/js/main.js" type="text/javascript"></script>
</body>
</html>

Eliminating render-blocking CSS was a little more tricky. Google suggests inlining critical CSS. I haven't taken the time to figure out which CSS that would require and how this would change my workflow. For now, I've taken their suggestion and load my CSS at the bottom of the page using a <script> tag.

[Update 2015-09-14]: I figured it out. You can read how I added the critical css here

<script>
  var cb = function() {
    var l = document.createElement('link'); l.rel = 'stylesheet';
    l.href = "{{ SITEURL }}/theme/css/style.css";
    var h = document.getElementsByTagName('head')[0]; h.parentNode.insertBefore(l, h);
  };
  var raf = requestAnimationFrame || mozRequestAnimationFrame ||
      webkitRequestAnimationFrame || msRequestAnimationFrame;
  if (raf) raf(cb);
  else window.addEventListener('load', cb);
</script>

Ok. Easy stuff done. This put my site up into the 50's range. Good, but still not great. Let's tackle the one that's lowering my score the most: gzip compression and caching.

I host my blog at the wonderful macminicolo.net, which means I control the server and have to do my own optimizations. Turns out, this really wasn't that hard to do. Here's how to easily enable compression in Apache.

# Always back up your config file before changing a bunch of stuff
sudo emacs /etc/apache2/httpd.conf

# Make sure this is enabled
LoadModule deflate_module libexec/apache2/mod_deflate.so

<IfModule mod_deflate.c>

    # Restrict compression to these MIME types
    AddOutputFilterByType DEFLATE text/plain
    AddOutputFilterByType DEFLATE text/html
    AddOutputFilterByType DEFLATE application/xhtml+xml
    AddOutputFilterByType DEFLATE text/xml
    AddOutputFilterByType DEFLATE application/xml
    AddOutputFilterByType DEFLATE application/x-javascript
    AddOutputFilterByType DEFLATE text/javascript
    AddOutputFilterByType DEFLATE application/javascript
    AddOutputFilterByType DEFLATE application/json
    AddOutputFilterByType DEFLATE text/css

    # Level of compression (Highest 9 - Lowest 1)
    DeflateCompressionLevel 9

    # Netscape 4.x has some problems.
    BrowserMatch ^Mozilla/4 gzip-only-text/html

    # Netscape 4.06-4.08 have some more problems
    BrowserMatch ^Mozilla/4\.0[678] no-gzip

    # MSIE masquerades as Netscape, but it is fine
    BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html

    <IfModule mod_headers.c>
        # Make sure proxies don't deliver the wrong content
        Header append Vary User-Agent env=!dont-vary
    </IfModule>
</IfModule>

Save and restart Apache

sudo /usr/sbin/apachectl restart

You can test to be sure it's working by using curl on a file that matches any of the above content types. You should see Content-Encoding: gzip in the response headers.

curl -I -H 'Accept-Encoding: gzip,deflate' https://ryanmo.co/theme/js/main.js


HTTP/1.1 200 OK
Date: Sat, 12 Sep 2015 20:38:24 GMT
Server: Apache/2.4.10 (Unix) PHP/5.5.20 OpenSSL/0.9.8zd
Last-Modified: Sat, 12 Sep 2015 02:47:24 GMT
ETag: "38169-51f83da1c0700-gzip"
Accept-Ranges: bytes
Vary: Accept-Encoding,User-Agent
Content-Encoding: gzip
Cache-Control: max-age=2592000
Expires: Mon, 12 Oct 2015 20:38:24 GMT
Content-Type: application/javascript

Next is content caching. This is also something to edit in the httpd.conf file for Apache. Google suggests at least 7 days for default caching, and up to a year for content that doesn't change often.

# Make sure this is uncommented
LoadModule expires_module libexec/apache2/mod_expires.so

## EXPIRES CACHING ##
<IfModule mod_expires.c>
    ExpiresActive On
    ExpiresByType image/jpg "access plus 1 year"
    ExpiresByType image/jpeg "access plus 1 year"
    ExpiresByType image/gif "access plus 1 year"
    ExpiresByType image/png "access plus 1 year"
    ExpiresByType application/x-font-ttf "access plus 1 year"
    ExpiresByType text/css "access plus 1 month"
    ExpiresByType application/javascript "access plus 1 month"
    ExpiresByType image/x-icon "access plus 1 year"
    ExpiresDefault "access plus 7 days"
</IfModule>
## EXPIRES CACHING ##

At this point, I was getting into the mid to high 80s for my score. Awesome. At this point, I could probably stop and be satisfied with the results. The remaining suggestions were easy, so I kept going. The first was to minimize all of my CSS and JavaScript. Easy. I just hit that checkbox in CodeKit and moved along. I did take one additional step here and took advantage of CodeKit's ability to combine JavaScript files into a single file by adding a header to my main JavaScript file.

# @codekit-prepend "../js/jquery-1.10.1.min.js", "../js/bigfoot.min.js";

Lastly, Google suggested that I compress my images. They were even nice enough to provide a zipped file of all your CSS, JavaScript and images compressed for you. If I don't have to do the work, then I won't. I downloaded the file and replaced all my images with the ones they gave me. In the future, I'll be using ImageOptim to optimize the images on my site.

My final score check: 91/100 on mobile and 97/100 on desktop! I think I can call that a success. Honestly, when trying to load my site on different devices, I didn't notice a significant increase. That being said, at least I'll be in Google's good graces for being a good web citizen, and I'll avoid any risk of them down-ranking my site for doing things incorrectly. I still want to take advantage of the critical CSS at some point, but I can leave that for another day.

I've been meaning to write about backing up email for a few months now. With Dr. Drang's recent post about archiving email using formail, I figured this a good enough time as ever to post my solution.

To install getmail, you can use Homebrew, or manually if you're in to that sort of thing.

brew install getmail

For the configuration files, I first created a directory in my Home folder

mkdir .getmail

The config files for both Fastmail and Gmail are similar, but I'll include both for completeness

Gmail

[retriever]
type = SimpleIMAPSSLRetriever
server = imap.gmail.com
username = <username>@gmail.com
mailboxes = ("[Gmail]/All Mail",) # To pull all emails
port = 993

[destination]
type = Maildir
path = ~/gmail-archive/

[options]
# print messages about each action (verbose = 2)
# Other options:
# 0 prints only warnings and errors
# 1 prints messages about retrieving and deleting messages only
verbose = 2
message_log = ~/Dropbox/Scripts/logs/gmail.log
receieved = false
delivered_to = false
# dont re-read messages its already pulled down
read_all = false

Fastmail

[retriever]
type = SimpleIMAPSSLRetriever
server = mail.messagingengine.com
username = <username>@fastmail.com
mailboxes = ("INBOX.Archive",)

[destination]
type = Maildir
path = ~/fastmail-archive/

[options]
verbose = 2
read_all = false
message_log = ~/Dropbox/Scripts/logs/fastmail.log
delivered_to = false
received = false

You'll notice that I don't include the password in either config. If password is omitted from a config on Mac OS, the first time you run getmail, you'll be prompted to enter your password and it will then be stored securely in KeyChain. Since I'm using Maildir as my type, you'll need to create special folders, with 3 specific subfolders

mkdir ~/fastmail-archive
mkdir !$/{cur,tmp,new} # creates 3 new folders in ~/fastmail-archive

The same will need to be done for ~/gmail-archive.

Once this is set up, you'll need to run the script once to enter in your password

/usr/local/bin/getmail -q -r ~/.getmail/getmail.gmail

If I ever need to view the emails, I do the same as Dr. Drang and use mutt, which can also be installed with Homebrew.

mutt -R -f ~/gmail-archive

In TextExpander 5, you can now use Javascript in snippets. Since I had already created some scripts using Contacts, I re-used some of the bits to create TextExpander snippets to expand information directly from Contacts.app.

Contacts = Application('/Applications/Contacts.app')

// ::attr is the attribute you want pull (emails, phones, etc.)
// ::accountType is what you see to the left of the info in Contacts.app
function getMyInfo(attr, accountType){
    method = attr == 'addresses' ? 'formattedAddress' : 'value'
    search = Contacts.myCard()[attr].where({label: accountType})
    results = search.length ? search().map(function(a){return a[method]()}) : []

    if (results.length == 1){
        return results[0]
    } else if (results.length > 1){
        return results.join(", ")
    } else return ""

}

// Grab my 'blog' email address 
getMyInfo('emails', 'blog') // returns email address

The getMyInfo function is fairly generalized to allow you to get any basic information like emails or phone numbers from your contacts. The first argument is the type¹, and the second is the label you want, such as Home, Work, etc. To get a phone number, just change the function called to getMyInfo('phones', 'iPhone). To get an address, change to getMyInfo('addresses', 'home'), which will return a formatted address.

Update 2015-08-23: Updated the getMyInfo function to support addresses.

Update 2015-09-20: For a while, I couldn't figure out why occasionally expanding things from Contacts would take a little while. Then I realized, for AppleScript to work, the application you're calling needs to be open. If you do want to use this as a way to expand your information from Contacts, just know that you'll need to be running the application for the TextExpander snippets to work.

Update 2015-09-20: I've figured out a way to do this without Contacts.app running! Check out my post here for more info.

Every month I need to fill out four receipts in a PDF form that I made with PDFPenPro. I never liked doing it and I felt like there had to be a way to do this better. I did some searching, and of course came across an old post by Dr. Drang where he was adding rich text to PDFs. This was close to what I wanted, but since I had taken the time to create the form fields myself, I figured I should take advantage of them if I could.

After some painful reading of the PDFPen Javascript library and playing around, I found that I was able to set the values of form fields if they had assigned names. I first went through the process of naming all of my form fields.

First we start with creating our app variable for PDFPen

app = Application("PdfPenPro")
app.includeStandardAdditions = true;

After that, I created a few functions to easily set the values for text fields and buttons

function getFormField(doc, field) {
    formField = doc.pages()[0].imprints.whose({fieldName: field})()

    if (formField.length) {
        return formField[0]
    }
    return
}

function setFormField(doc, field, value) {
    theField = getFormField(doc, field)
    if (theField.class() == "button") {
        theField.checked = value
    } else {
        theField.value = value
    }
}

I've found that with PDFPen (or maybe more generally), keeping track of windows in applications with Javascript can get tricky. The order in which they appear can change, and so I've written myself a few helper functions to keep track of documents as I work through scripts.

var getByName = function(fileName){
    return app.documents.byName(fileName);
}
var getDocPage = function(fileName, pageNum) {
    return getByName(fileName).pages[pageNum-1];
}

These are helpful when I want to duplicate my template receipt that I've created and not run the risk of overwriting the original.

function createNewReceipt() { 
    /*
        Opens the Receipt template
        Creates a new document and duplicates
        the template into the new doc
    */
    app.open(templatePath)
    var currentDoc = app.windows[0].document
    var currentDocName = currentDoc.name()
    doc = app.Document().make()
    app.duplicate(getDocPage(currentDocName, 1), {to:doc})
    return doc
}

At this point, it's simply a matter of filling in the fields however¹ I like. I chose to have a few input fields and optional lists. Here are examples of both of those and how they can be implemented in Javascript.

/*
    Creates a list to choose from where the options
    are an array called dateOptions.
    I always take the 0 element since I don't allow
    empty selections and multiples aren't allowed
*/
dateChoice = app.chooseFromList(
                        dateOptions,
                        {withTitle: "Start Date",
                        withPrompt: "Choose Start Date", 
                        defaultItems: dateOptions[1],
                        multipleSelectionsAllowed: false,
                        emptySelectionAllowed: false}
            )[0]

/* 
    Basic dialog box. 
    defaultAnswer has to exist, otherwise there
    won't be a text box to type into
*/
receivedFrom = app.displayDialog("Who sent the check?", {defaultAnswer: ""})

I then created an array of all the fields that I wanted to be filled.

fieldsToFill = [
{ value: receivedFrom.textReturned, fieldName: "ReceivedFrom" },
    ...
{ value: paymentType == "Check" ? true : false, fieldName: "CheckCheckBox" }
]

The very last lines are where the magic happens and all the fields are filled out. It was pretty fun to watch the form fill almost instantly and then let me save it.

newDoc = createNewReceipt()
fieldsToFill.map(function(obj){setFormField(newDoc, obj.fieldName, obj.value)})

saveLocation = app.chooseFileName({defaultName: rentReceiptName(dateChoice), defaultLocation: savePath})
newDoc.save({in: saveLocation})

That's it! This will take a lot of the slow process out of filling out my receipts each month. The next step will probably be generating the email and attaching the PDF. PDFPen already includes a script to do this very thing, so no more work needed for me. You can see the entire script here

After searching around, I found a nice way to automatically create a session each time I ssh into a machine, or re-attach if it already exists.

if [[ "$TMUX" == "" ]] &&
        [[ "$SSH_CONNECTION" != "" ]]; then
    # Attempt to discover a detached session and attach
    # it, else create a new session
    WHOAMI=$(whoami)
    if tmux has-session -t $WHOAMI 2>/dev/null; then
    tmux -2 attach-session -t $WHOAMI
    else
        tmux -2 new-session -s $WHOAMI
    fi
fi

I first check to be sure I'm not in a screen session and also that I'm using ssh and not local to my machine. After that, it's a simple check to see if a session exists. If so, re-attach it, otherwise create a new one. This can be simple added to the bottom of your ~/.bashrc file. Now every time I ssh in to any machine, my previous session is sitting there waiting for me.

First Impressions

Starting the workout on the watch is nice. Two-three taps and you're starting a workout. What I didn't find ideal is that there wasn't a screen that gave you an overview of elapsed time, speed and distance all in one. There were individual pages for each section that you'd have to swipe through, which isn't very safe while riding a bike. I ended up leaving it on the heart rate monitor since the one I usually wear isn't working.

Speed

Speed was within 0.3 - 0.5 mph of the Garmin the entire time

(I know, I know. This was not safe to be taking this photo while moving.)

Distance

Distance was nearly spot on. I accidentally forgot to start my Garmin, so the watch was ahead by about 0.5 miles. By the end, it was still within the same range

Time

This is the one area that I wasn't happy with on the watch. The watch does let you pause the ride, but it won't ever pause it automatically. If you stop for lunch, which I did, I had to manually pause the ride. I forgot to when I stopped to get an espresso along the way.

My elapsed time (top-right on the Garmin) was the true amount of time I spend on the ride and time (top-left) is the time on the bike. With the watch, it all depends on whether you pause your ride or not

Battery

Workouts destroy Watch's battery. By the end, Apple Watch was at 29%, while the Garmin was at 79% after around 3.5 hours.

Location

I assume that Apple Watch is using GPS to calculate distance and speed. However, no where does it show you a map of where you went. Not a deal breaker, but it's fun to look back on your previous route, or be able to ride the same route again to improve your times.

Siri

I tried using Siri while riding. I don't know if I wasn't being loud enough or the road noise was making it difficult to hear me, but I couldn't get it to ever work. Plus needing your other hand for some actions makes using Apple Watch while riding nearly impossible (yes, I tried nose touching, but it wasn't great either).

The Finish

The ride summary on Apple Watch is very nice. It gives you a summary before you save the workout

Final Thoughts

If you don't want to invest in a dedicated cycling GPS, Apple Watch is a decent option, granted you don't go on rides over 5-6 hours. You won't get all the cool features, but if distance, speed and calories is all you care about, it's a pretty good option. I'll be sticking to my Garmin 810 for rides.

Next will be to try out Strava, which will use the iPhone for the heavy lifting rather than the watch.

[Update 2015-05-03 15:45 ] As @ismh points on out Twitter, another good test would be the Garmin vs. the Apple Watch without the iPhone since I was basically comparing my iPhone GPS to Garmin's.

I started playing around with writing a script to download them using Python, but decided I should check Github to see if anyone had already done this. Sure enough, the first result was a script that someone had written that would download all your paychecks.

The script worked perfectly, but there were a few small changes I wanted to make. First, I added a new method that let me easily set the destination path. I also didn't want to have to remember to run this every two weeks, nor did I want a cron job running this script with my password in plain text. I forked the repo, and started incorporating OS X's Keychain to let me store my password securely and running the script once a day to pull down any new paychecks. I borrowed the Keychain library from the alfred-workflow. This lets me easily save and retrieve passwords from Keychain in python. To set my for ADP in Keychain, I used the quick script (you could also do this manually in Keychain.app):

from keychain import KeyChain
import getpass

my_password = getpass.getpass("Enter pw: ")

KeyChain().save_password("my_username", my_password, 'adp')

The script is smart enough to not re-download paychecks that you've already saved, so now it was as simple as adding a new line to my daily cron jobs

python adp.py "my_username"

Lastly, just so that I know when a new paycheck has been added, I set up a small Hazel rule to send me a notification through Pushover whenever a new file is added

You can see my fork of ADP-paychecks here. You can download a zip of the project here.

If you haven't already read it, Macminicolo has already posted their own [50 ways to use your server][50ways]. I thought it would be worth sharing some of the ways that I use my Mac Mini. Some of these things I've already shared in the past and I'll be sure to post more details on any of the other things in the future.

[50ways]: http://blog.macminicolo.net/post/47038825502/50-ways-to-use-your-server

Syncing and Backup

Dropbox

All of my Dropbox files are synced to this computer. My MacBook Air doesn't have enough space to store all my files and so the Mac Mini is the place where I store all my Dropbox files locally so that I can run workflows and have a local backup.

Off-site Backup

Since I have so much space, I use it as an offsite backup for my laptop using Arq over sftp. Nothing too fancy or special here, but it's a nice alternative to Time Machine as an offsite backup.

Hazel

Hazel may be my favorite reason for having an always-on Mac. Hazel watches multiple folders in my Dropbox folder and keeps my Dropbox much more organized than I ever would manually. Some of my favorites are

Organizing my photos

I've talked about this one a bit in the past. I love that Carousel will automatically upload my photos to Dropbox, but the Camera Uploads folder becomes a wasteland of files if you don't organize them on a regular basis. I move all of my photos into a photos folder organized by year. I've written about this in more detail here. If you use Carousel and have ever saved photos that someone else shared with you, you'll know that a completely different folder is created in Dropbox called Carousel. In this folder, more folders are created with the email address of the person who shared the photos with you. I want these photos in my normal photos folder and so I run the same set of rules as my Camera Uploads to reorganize these photos. The only exception is that I add a "carousel" tag to these photos so that I know they were added from Carousel.

I take a selfie every day (620 days and counting) and am far too lazy to move that photo to its own special photo every day. I've made sure to always use Camera+ to take these photos. Hazel looks at the metadata of the photos in Camera Uploads and if the photo was taken by the front camera and the app used to create the file was Camera+, it's moved to its own special folder and renamed to just YYYY-MM-DD.

Publishing my blog

I use a static blog generator, Pelican, which means that I can store the entire project, including the Python code in my Dropbox account. While I'm on my Mac, it's easy to run a shell script to publish my blog. On iOS, it's not quite as easy and so I use Hazel to watch my blog folder for a file called 'publish.blog'. If that file exists, the shell script is run and the file is then deleted. Since my girlfriend runs her blog over at keepitlit.co with the same static blog generator, it's much simpler for her to create a file just like this when she wants to publish her blog.

IFTTT → Dropbox → Flickr → AppleTV

I have a rule set up in IFTTT that will append to a text file each time my girlfriend or I post a photo to Instagram. Each time this file runs, I have a script that uploads the photo to a private Flickr album. My AppleTV is then set to that album so that we have an updated list of photos as a screen saver. I realize I could do this directly in IFTTT, but I don't like that you can't make the album private.

Download the script here

Time sensitive Dropbox shared links

If you have a Dropbox Pro account, this is now a feature built right in. I have two folders named "One day" and "One week". Files that I want to share temporarily are copied to that folder. After the set amount of time, the files will be deleted and I'm sent a push notification. For the one week folder, I also get a notification the day before to remind me that it'll get deleted.

You can download the 1 Day rule here. Be sure to add your own Pushover key and secret, or remove it if you don't need notifications.

Scanned files

This folder is for files added from my Fujitsu ScanSnap or Scanbot for iOS. If the file hasn't been OCR'd already, a script will run to launch PDFPenPro and OCR the file. I then have a series of rules set up to move the files based on their names.

Work Receipts is my favorite. When I scan a receipt in Scanbot, I have a snippet "wwr" that expands to "Work receipt". Hazel watches for any new PDFs with that string in the filename. Files are then moved to my expenses folder, organized by date. It then creates a new task in Due.app with a due date of one week in the future so that I'll remember to do my expenses for the file. I no longer have to keep all my receipts and I'll never forget to actually do the expenses¹.

Business cards obviously moves any business cards to a special folder. Hazel watches for a string match in the filename to know to file these as well. Finally, personal receipts moves the files to my own receipts folder for archiving.

Web Server

I use this Mac Mini as a web server since it has more than enough bandwidth and speed. I had never set up an server before, and so this was a fun learning experience to do it all myself. I run a very basic Apache, MySQL, PHP stack for my web server.

Blogs

I host this blog from my Mac Mini as well as a couple of others, most notably my girlfriend's.

Site Analytics

I don't want to use Google Analytics. They know enough about me already and so I use an open source version called Piwik. I've been fairly happy with it so far.

URL Shortening

I don't like long urls and will shorten them whenever I can. When I publish my blog, I always shorten the URL. I like having full control over that and so I'm using yourls to shorten and track URLs.

VPN

I was running OS X Server and used the Mac Mini as a VPN server. Since upgrading to Yosemite, I haven't gotten around to upgrading server, but it's on the todo list. Check out Macminicolo's blog for some great instructions on setting up a VPN here.

Tapiriik

When I'm out cycling, I use a Garmin GPS. Most of my friends use RunKeeper, and I prefer Strava over all of the services. Tapiriik is a great service that lets you keep your fitness services all in sync, including syncing TCX files to your Dropbox account. It's open source, so you can run a local version on your own computer.

WebDAV

When I was using Omnifocus, I didn't want to sync my database through their servers. I could be wrong, but I don't believe it's encrypted on their servers. I feel much better knowing that it's on my machine and I have completely control of it. I have set up my own WebDAV server so that I can sync my database. It's been extremely fast and reliable.

Automation and Scripts

I have crons running on an hourly, daily and weekly bases. I don't want to bore you with all of them, but here are a few of the better ones.

getmail

I use getmail for archiving my Gmail daily (they've been known to lose data once in a while). I've never needed to use it, but if I ever decide to change providers or Gmail just hits the delete key someday, I'll have a complete backup of my email. A great introduction to getmail can be found here

Slogger

I love Slogger and Day One. I've customized a lot of the current plugins and even wrote my own for Instagram. You can read more about it here

Download Pinboard

My updated version of Brett Terpstra's pinboard → webloc file script to have tagged webloc files locally. You can read more about this project here.

Face detection → Finder tags

I don't want to use iPhoto, Aperture or Picasa as a photo management application. Instead, I use Picasa to harvest the facial recognition data, and then have a script that applies Finder tags of the person's name to the photo. I haven't shared this one, because it's not done yet, but it's functional. It's a lot of fun to be able to get all the photos of a person with a simple Spotlight search. Hopefully I can share this in the near future.

Dropbox Deletions

I like to keep tabs on my shared folders and any scripts that might be running in my Dropbox account. I parse my Dropbox RSS feeds for deletions of more than 50 files and send myself a push notification with Pushover.

You can download the script here

Dropbox inbox

Throughout the week, I'll add files to an "Inbox" in my Dropbox folder. On the weekends, I send myself a push notification if there have been any files added so that I can deal with them.

Media

I don't have a lot of media. I've never been attached to the idea of owning my music or video and stream whatever I can. For any content that I've ripped over the years, I have Plex running on my Mac Mini. Again, since the connection is so fast, there's little to no lag when streaming something from home or on my phone.

I'm fine with reminding my mom to put the PDF in Dropbox, but I then have to check back every-so-often to see if she's actually done it. I've created a rule now that will check for any files in our shared Verizon Bill folder and if Hazel hasn't seen it before, it will send me a push notification with Pushover.

I then wrote a handy little Applescript based on PDFPenPro's default script called Split PDFs that will take the first page of a PDF and save it to a new file. I differentiate the files by just adding "SINGLE PAGE" to the filename

set basePath to "/path/to/verizon/folder"

tell application "PDFpenPro"
    open theFile as alias
    set originalDoc to document 1
    set docName to name of originalDoc

    if docName ends with ".pdf" then
        set docNameLength to (length of docName)
        set docName to (characters 1 thru (docNameLength - 4) of docName as string)
    end if


    set newDoc to make new document
    set savePath to ((basePath as rich text) & docName & " SINGLE PAGE" & ".pdf")

    copy page 1 of originalDoc to end of pages of newDoc

    save newDoc in POSIX file savePath

    quit
end tell

Finally, so that I don't forget to submit the expense, I have one final Applescript that creates a todo item in Omnifocus based on David Spark's post here

set theDate to current date
set deferDate to (current date)
set dueDate to (current date) + (1 * days)
set theTask to "Expense Verizon Bill"
set theNote to theFile

tell application "OmniFocus"
    tell front document
        set theContext to first flattened context where its name = "A Context"
        set theProject to first flattened project where its name = "Expenses"
        tell theProject to make new task with properties {name:theTask, note:theNote, context:theContext, defer date:deferDate, due date:dueDate}
    end tell
end tell

In Marked's preferences under Advanced, there is an option to add your own preprocessor. This gives you the ability to format the text in the file before Marked renders the markdown.

The script simply looks for any occurrence of the {static} and replaces it with the path to my content folder in Pelican.

#!/usr/bin/python
import sys
import re
import os

home = os.path.expanduser('~')


class PelicanFormat:
    def __init__(self):
        self.blog_path = home + '/Dropbox/blog/content'
        self.content = sys.stdin.read()

    def __repr__(self):
        return sys.stdout.write(self.__str__())

    def __str__(self):
        return self.content

    def replace_filenames(self):
        self.content = re.sub(r'{static}', self.blog_path, self.content)

    def change_codeblocks(self):
        """
        TODO Pelican uses ':::language' to override syntax highlighting.
        """
        pass

if __name__ == '__main__':
    p = PelicanFormat()
    p.replace_filenames()
    print p

Now I can preview images for my blog posts instead of broken images.

Bonus!

This is a Text Expander snippet I use to create image urls for Pelican. It looks for the last file that was added to my images folder and then creates the url

#!/bin/bash

DROPBOX_PERSONAL=$(python -c "import json;f=open('$HOME/.dropbox/info.json', 'r').read();data=json.loads(f);print data.get('personal', {}).get('path', '')")

BASE_PATH="$DROPBOX_PERSONAL/blog/content"
IMAGE_PATH="images"
SEARCH_PATH="$BASE_PATH/$IMAGE_PATH"

LAST_ADDED=$(mdfind \
    -onlyin "$SEARCH_PATH" \
    'kMDItemDateAdded >= $time.today(-1)' \
    -attr 'kMDItemDateAdded' | \
awk -F"kMDItemDateAdded =" '{print $2 "|" $1}' |
sort -r | \
cut -d'|' -f2 | \
head -n1 | \
sed -e 's/^ *//' -e 's/ *$//' -e "s:$BASE_PATH::")

echo -n "![]({static}$LAST_ADDED)"

The script is a slightly simpler version of the original, but the core functionality is the same. Each bookmark is saved as a webloc file and apply any OS X tags to the file. This can be paired with an Alfred workflow to easily search by title or tag.

You can download the download-pinboard project here. Feel free to check out the Github project here.

Setup

Create a settings file

cp settings.py{.example,}

with the following information

_PINBOARD_TOKEN = 'YOUR TOKEN HERE'
_SAVE_PATH = HOME + '/Bookmarks/'

In settings.py set your Pinboard token and the path where you want your bookmarks to be saved. Your token can be found at https://pinboard.in/settings/password. The path must exist where you save your bookmarks and must end with a trailing /.

Running the Script

To start the script, you can simply do

python main.py

Optional arguments

-v, --verbose Shows output as stdout
-t Filters the bookmarks you want to download by tag. You can pass multiple -t tags, but no more than 3. Multiple tags are AND not OR
--reset [optional num of days] Resets your last updated time. If you don't specifiy a number, it will reset to 999 days.
--skip-update Lets you bypass the last downloaded time. Nice for redownloading everything.

Notes and Todo

I have a very small number of bookmarks (~150) and so I don't know if there will be any issues with a really large library. If you have one, and run into problems, please let me know and I'll happily look into it.

My first attempt at the new JSX Automation was a script to back up my contacts, which would allow me to run this automatically on my Mac Mini server. Here is what the script looks like

var app = Application.currentApplication()
app.includeStandardAdditions = true

now = new Date()
nowString = now.getFullYear() + "-" + (now.getMonth() + 1) + "-" + now.getDate()

// Replace outputFile with this if you want to automatically set the path
// var outputFile = Path('pick your path')  
var outputFile = app.chooseFileName({
    withPrompt: "Pick where to save your vCard backup.",
    defaultName: nowString + "_backup.vcf"
})

var a = app.openForAccess(outputFile, {writePermission: true})
Contacts = Application('/Applications/Contacts.app')
contacts = Contacts.people()
outputString = ''

for (var i = 0; i < contacts.length; i++){
  outputString += contacts[i].vcard()
}

app.write(outputString,{
    to: a,
    startingAt: 0,
    as:'text'
})
app.closeAccess(outputFile)

app.displayNotification("Backup finished!",{
    withTitle: "Backup Contacts",
    subtitle: contacts.length + " contacts backed up."
})

The current script lets you choose the path to save the file. You can change this to have it be the same path every time if you want (see the instructions in the comments above).

You can download the script here to try it for yourself.

Big thank you to Alex Guyot at MacStories for his introduction to Javascript Automation.

I realize now that simply revealing the file isn't as useful as performing actions on the file. There are also occasions when I want to see the last 5 files added, not just the most recent. I've converted the workfow to now be a script filter.

I've updated the download link below to be the latest Alfred workflow. The old version is still available with an empty keyword.

Enjoy!

To continue on yesterday's post, revealing files in the Finder can be very useful. One thing that I find myself doing daily is moving into a particular folder in my Dropbox account once I've used the Alfred "move" action or when a new file has been added to my account.

How many times have you see this notification and wondered what file it was and where on earth it is in your Dropbox account?

Similar to revealing the last file added to my Dropbox folder, I can show the file last added in my Dropbox account. The only difference here is that my two Dropbox folders combined (work and personal) amount to about 150,000 files. Listing off all those files and sorting them by Date Added would take far too long. Instead, I can take advantage of mdfind, which is the command line version of Spotlight.

DROPBOX_WORK=$(python -c "import json;f=open('$HOME/.dropbox/info.json', 'r').read();data=json.loads(f);print data.get('business', {}).get('path', '')")
DROPBOX_PERSONAL=$(python -c "import json;f=open('$HOME/.dropbox/info.json', 'r').read();data=json.loads(f);print data.get('personal', {}).get('path', '')")
DROPBOX="$HOME/Dropbox"

LAST_ADDED=$(mdfind \
    -onlyin "$DROPBOX_PERSONAL" \
    -onlyin "$DROPBOX_WORK" \
    -onlyin "$DROPBOX" \
    'kMDItemDateAdded >= $time.today(-1)' \
    -attr 'kMDItemDateAdded' | \
awk -F"kMDItemDateAdded =" '{print $2 "|" $1}' |
sort -r | \
cut -d'|' -f2 | \
head -n1 | \
sed -e 's/^ *//' -e 's/ *$//')

if [ ! -z "$LAST_ADDED" ]
then
  echo "$LAST_ADDED"
  open -R "$LAST_ADDED"
fi

I include the paths to both the work and personal Dropbox folders if you have them (it doesn't matter if you don't) as well as the regularly named "Dropbox" folder. From there, it's a matter of getting the name and date last added for files that have been added within the last day. The result shows up almost instantly with having over 150,000 files in my Dropbox folders.

You can download the workflow below.

My original workflow simply looked like this

Unfortunately, listing files or using the find command doesn't give you the file last added. You can get away with using ctime, but not in every case. Turns out Date Added is an attribute that Mac OS X adds to every file, which meant that I could use mdfind to get the file that was last added. All that's left to do is print out a list of file name and date last added, sort them, and get the most recently added file from the Downloads folder. From there, its just a matter of using the open -R command to reveal the file

DOWNLOADS="$HOME/Downloads"

RECENT=$(mdls -name kMDItemFSName -name kMDItemDateAdded $DOWNLOADS/* | \
sed 'N;s/\n//' | \
awk '{print $3 " " $4 " " substr($0,index($0,$7))}' | \
sort -r | \
cut -d'"' -f2 | \
head -n1)

open -R "$DOWNLOADS/$RECENT"

mdls -name kMDItemFSName -name kMDItemDateAdded ~/Downloads/*

Lists the name and date added for all the files in the Downloads folder

sed 'N;s/\n//'

Looks at the next line and removes any newlines, which puts the name and date added all on one line¹

awk '{print $3 " " $4 " " substr($0,index($0,$7))}'

Returns the name and date added in a nice format like "2014-11-15 19:36:28 "Arq.zip""

sort -r

Sorts the lines

cut -d'"' -f2

Splits the lines on a quotation mark and returns the second result (the filename)

head -n1

Gives the top item in the list, which is the most recently added file

open -R

Reveals the file instead of opening it in OS X.

You can download this workflow to reveal the last added file in your Downloads folder below.

Pythonista offers a great library which gives you access to your contacts on iOS. With a short script, I can back up my contacts to a folder in my Dropbox account. This will add a vCard file to my Dropbox account with the date the script was run.

Note: You'll need the Dropbox login script for this to work. Visit this link to get it set up. I keep mine in a folder called "lib" in Pythonista.

You can download my Contacts Back up script here.

import contacts
import sys, os
import console
sys.path += [os.path.join(os.path.dirname(os.path.abspath(__file__)), 'lib')]
from dropboxlogin import get_client
from datetime import datetime

# Update this path here for the backup
# location in your Dropbox account.
BACKUP_PATH = '/Backups/Contacts'

TODAY = datetime.today().strftime('%Y-%m-%d')

dropbox_client = get_client()

VCARD = "".join(person.vcard for person in contacts.get_all_people())

console.clear()
dropbox_client.put_file(BACKUP_PATH + '/contacts {}.vcf'.format(TODAY), VCARD)
print 'Backup complete!'

If you're like me, you're going to forget to do this on a regular basis. I hadn't yet found a reason to use the IFTTT Launch Center Pro triggers, but this turned about to be a great reason to use it. I have a trigger that goes off on the first of every month that will launch the back up script.

If you want to get reminders to back up your contacts using IFTTT, you can use the recipe below.

Current Podcasting App: Overcast

• 99% Invisible
• Accidental Tech Podcast
• The Black Tapes
• Connected
• Criminal
• Hello Internet
• Lifeoff
• NPR TED Radio Hour
• Radiolab
• Small Town Horror
• Invisibilia
• Under the Radar
• Ungeniused
• Upgrade

Update

As of June 2016, Instagram has changed their API and no longer allows this script to work. Sorry :(

I didn't know ruby and quickly learned how bad some API documentation can be, but I wanted this plugin more than all the others. After fiddling with it for an evening, I was able to log Instagram posts with more than just a photo, including:

• number of likes
• comments
• date of post, not the date Slogger ran
• location data (including place name if you used Foursquare checkin)

The last point is by far my favorite. I can look at a map over the last year and see all the Instagram photos I've taken and where I took them

I also wanted to import photos that I had already taken. The plugin now will let you set backdate: true and will log the last 20 photos that you had posted on Instagram. Once it's finished, it'll automatically set itself to false to prevent duplicate posts¹.

Setup is fairly straight forward. I create a local server, which runs you through the Instagram OAuth flow. After you've finished, you simply paste in the access token, and it'll run from there

> ./slogger -o instagram
Initializing Slogger v2 (2.1.14)...
08:01:18      InstagramLogger: Instagram requires configuration, please run from the command line and follow the prompts

------------- Instagram Configuration --------------

Slogger will now open an authorization page in your default web browser. Copy the code you receive and return here.

Press Enter to continue...

Last night I finally did a pull request and it went live this morning. You can check out and download the latest Slogger on Github. If you find any issues or bugs, please send them my way. Enjoy!

In most cases, setting a variable to your Dropbox path is relatively easy. You could set your .bashrc to look something like this

DROPBOX_PERSONAL=$HOME/Dropbox

But this fails in a few situations, all of which apply to me on one or more of my computers

• Multiple Dropbox accounts on one computer (Personal and Business accounts)
• Dropbox isn't located in my home folder

If you're running Dropbox version 2.8 or higher (you should be anyways), there's a json file that tells you where your Dropbox folders are located. The json looks like this:

{
    "personal": {
        "path": "/Users/username/Dropbox (Personal)",
        "host": 1234
    },
    "business": {
        "path": "/Users/username/Dropbox (Business)", 
        "host": 5678
    }
}

What this means is that you can set global variables using this information in your .bashrc or .bash_profile so that you always know where your Dropbox folder is

DROPBOX_WORK=$(python -c "import json;f=open('$HOME/.dropbox/info.json', 'r').read();data=json.loads(f);print data.get('business', {}).get('path', '')")
DROPBOX_PERSONAL=$(python -c "import json;f=open('$HOME/.dropbox/info.json', 'r').read();data=json.loads(f);print data.get('personal', {}).get('path', '')")

Now all you have to do is reference your Dropbox folders with $DROPBOX_PERSONAL or $DROPBOX_WORK.

On Monday, 2014-08-25, I signed up for Comcast online. There was a relatively nice deal where I would get up to 50mbps download for $34.99/mo for the first 12 months. Seemed pretty reasonable and I so I chose the plan and started through the sign-up process. Towards the end, they gave me some good news. My apartment was already ready for Comcast and I could order the self-install kit (SIK) and not worry about having someone come out to the house. I wanted my internet NOW, so I chose the overnight option, which was an extra $30.

The next day, the modem shows up at around 8:00am and so I quickly pull it out of the box and hook it up. It doesn't work. I immediately call Comcast and tell them that my modem isn't getting a connection and I believe that my apartment doesn't have whatever hookups it needs (even though the last tenant had Comcast too). I had to go through the basic "unplug it, wait, plug it back in" b.s. and then they told me I would need to have a tech come out and there may be additional charges. A few hours later, I get a confirmation email that the tech would be coming between 7:30-8:00am on Friday. Perfect, since now I don't have to miss work. But now there's a new problem. The email has an order summary, saying that my monthly bill will be $80 + $30 for the overnight shipping + a new $50 for a failed SIK install. Again, I call Comcast, and this is where the real annoyance begins.

The first person I talked to was pleasant enough, but told me that they couldn't see online deals and so they couldn't and wouldn't honor the deal I claimed to have signed up for (you don't have a browser to look??). They basically made me swear on my life that I signed up online and not in a store before they would let me go any further. I was then transferred to someone else, only to re-confirm my name, phone number, address and last 4 of my social and then told the exact same thing. Repeat these steps 3 more times before getting disconnected during the hold process. I call back and immediately tell the person I've been transferred 5 times and disconnected once. I need someone to help me or I would like to talk to someone about getting a full refund for my service. I was immediately transferred (again) to a customer retention specialist. She was nice, and at least seemed to have a soul. She understood my situation and told me that she will get me to someone who can honor the deal I signed up for. She was nice enough to also do a live call transfer in which she explained to the other rep while I was on the line what my situation was. She, also, was very nice, and was very explicit about what she was doing and that she left detailed notes about everything that happened. She waved my overnight fee since a tech was coming out and that if I got charged for anything else, call back, give her agent ID and tell the rep to look at the ticket notes. At this point, I was happy and assumed it was smooth sailing from there. Nope.

It's now Friday morning, I get up early and eagerly sit and wait for the Comcast tech to arrive and give me my internets. Comcast let me down...again... Here's the timeline so far (remember, the scheduled appointment was 7:30-8:00am):

07:30am - No show. I'll give him a few more minutes.
8:20am - Called Comcast to see if someone was coming out. Their automated system tells me the tech is late (as if I didn't know) and that he'll be arriving between 7:55-8:25am. I can press 4 to talk to someone. Pressing this button then says "this entry is invalid" and immediately disconnects
08:23am - I call again, but this time it says that my tech missed the appointment and immediately tries to transfer me, and then disconnects due to an invalid entry. I'm now stuck and cannot talk to anyone at Comcast.
08:36am - I call one last time, claim to not have an account and finally get to someone. He's actually friendly and helpful. He gives me the $20 credit I'm due for the tech not being on time, but promises that dispatch would be calling me within 15 minutes to let me know what's going on.
09:04am - Sitting and waiting for either the tech to show up, or Comcast to call me back. So much for that 15 minutes.
09:22am - Still no call or tech. I think I'm going to just go to work at this point. I really don't want to go the whole weekend without internet.
09:33am - Decided to call one more time. They told me the tech just straight up isn't coming now. The next appointment is Sunday from 12:00-2:00pm. Guess we'll see what happens then.

[Updated 2014-08-31]

Saturday 2014-08-30

8:00pm - I receive my Comcast bill, once again saying I owe $110, of which I'll be paying $80/mo instead of $39.99. Comcast's site says they're available 24/7 on the phone, but when I call, it says they're closed and won't be open again until Tuesday since it's a holiday. Luckily their chat was available. I explained in detail my situation, the agent who left notes for my account, and that my bill is incorrect. They verified that the bill was in fact correct and that a new one will be issued in a few days. The person was very nice and answered my question in detail.

Sunday 2014-08-31

12:45pm - The tech arrived (on time). He was a nice guy and got everything set up. He was overly chatty and it ended up taking almost an hour and a half to get everything set up. He was nice enough to say that my modem was "bad" which waves the $50 service fee since there was a problem with my service.

All in all, my experience was pretty poor with Comcast. My last few interactions were plesant, but I fear they weren't the common experience for most people. I hope that I don't have to contact them in the future, but I'm sure I will to resolve my bill.

I did some poking around, and found that there were a couple of different reasons why my photos in Dropbox weren't displaying dates:

• The DateTimeOriginal exif tag was missing entirely
• The DateTimeOriginal was set to 0000:00:00 00:00:00

With the magic of exiftool, I found a way to find all the photos in my Dropbox folder that were missing dates and output the results to a CSV.

exiftool -filename -r -if '(not $datetimeoriginal or ($datetimeoriginal eq "0000:00:00 00:00:00")) and ($filetype eq "JPEG")' -common -csv > ~/Dropbox/nodates.csv

This will give you a CSV with all of the common file information for the images.

At this point, you'll need to decide how you'll want to fix these photos. From what I have seen so far, the best exif tag to go on is -filemodifydate, but you'll probably need to figure that out on your own. If you want to fix any photo that matches the above criteria, you can do something like this

exiftool `-datetimeoriginal<filemodifydate` -r -if '(not $datetimeoriginal or ($datetimeoriginal eq "0000:00:00 00:00:00")) and ($filetype eq "JPEG")' ~/Dropbox

I like MultiMarkdown tables. So that my Foursquare checkins looked nice, I first created a file in my Dropbox account with a heading

| Date |  VenueName  | VenueUrl | Shout | MapURL |  City | State | Country |
| :---: | :---: | :---: | :---: | :---: | :---: |

In IFTTT, I then created a recipe which matches my table headers

You may have noticed that I added an additional "Address" column that isn't getting filled out. IFTTT doesn't explicitly give the address of the venue you visited. However, the link to the Google Maps image contains GPS coordinates that I can use. Dr. Drang's post gave me the idea to parse out the coordinates and then use them how I'd like. This script, which I'm using with Hazel each time the file is updated, reverse geolocates the coordinates and returns the full address using OpenStreetMap. After that, it appends that address to each line in the markdown file.

#!/bin/bash

FILE="$HOME/Dropbox/IFTTT/foursquare/foursquare.txt"

START=1
index=$START
IFS=$'\n'     # new field separator, the end of line
for line in $(cat $FILE)
do
    mapsurl=$(echo $line | sed -n 's/.*(\(http.*\)).*/\1/p');

    existingaddress=$(echo $line | grep -E '^.*\(http.*\)(.*\|){2,}$');

    if [[ ! $mapsurl || $existingaddress ]]; then
        (( index = index + 1 ))
        continue
    fi

    coords=$(echo $mapsurl | sed -E 's/^.+\?center=([0-9.,-]+).+/\1/');
    lat=$(echo $coords | cut -f1 -d,)
    long=$(echo $coords | cut -f2 -d,)

    address=$(curl -s "http://nominatim.openstreetmap.org/reverse?format=json&lat=${lat}&lon=${long}&zoom=18&addressdetails=1")

    country=$(echo $address | sed -e 's/.*\("country":".*"\),.*/\1/' | awk -F'"' '/country/ {print $4}')
    city=$(echo $address | sed -e 's/.*\("city":".*"\),.*/\1/' | awk -F'"' '/city/ {print $4}')
    state=$(echo $address | grep "\bstate\b" | sed -e 's/.*\("state":".*"\),.*/\1/' | awk -F'"' '/state/ {print $4}')

    # update the line of text
    sed -i '' -e "${index}s/\(.*\)/\1 $city | $state | $country |/" "$FILE";
    (( index = index + 1 ))
done

In the end, the table then ends up looking something like this:

There are tons of tools out there that show you what time it is in other parts of the world. One thing that isn't as readily available is a quick way to tell me what time it would be in California if it's 3:00pm in Dublin. I decided to write a quick TextExpander snippet that would let me pick the time and then it would output the time in all of my chosen time zones.

The first step is to choose the time zones that you want to appear. In my case, I chose the following since we have offices in these locations:

• Europe/Dublin
• America/Los_Angeles
• America/Chicago

Now I need to convert a chosen time to all of these time zones. This can be done using the date command in bash. Here's a quick example to try in the Terminal:

TZ=Europe/Dublin date -jf "%H:%M %z" "$(date "+%H:%M %z")" "+%H:%M %Z"

• TZ lets you choose the time zone for the date command
• -f tells date the format to expect for the input
• -j tells date to not change the date allowing the -f flag to convert a time
• "$(date "+%H:%M %z")" just gives the current date that looks like HH:MM +0100
• "+%H:%M %Z" is the output format

This gives you the following result:

04:52 IST

Now to do this for multiple time zones:

timezones=( "America/Los_Angeles" "America/Chicago" "Europe/Dublin")

for zone in ${timezones[@]}
do
    TZ=$zone date -jf "%H:%M %z" "$(date "+%H:%M %z")" "+%H:%M %Z";
done

Which gives:

08:55 PDT
10:55 CDT
16:55 IST

Lastly, let's add in some TextExpander input methods, and we have a way to use this with whatever time we want:

#! /bin/bash

/* 
Enter a time using 24H. 1:30pm is 13:30
*/
ENTERTIME="%filltext:name=Hour:width=2%:%filltext:name=Minute:width=2%"

timezones=( "America/Los_Angeles" "America/Chicago" "Europe/Dublin" )

for zone in ${timezones[@]}
do
        TZ=$zone date -jf "%H:%M %z" "$ENTERTIME $(date "+%z")" "+%H:%M %Z";
done

Hazel

Similar to my previous post, I'm using Hazel to detect special types of photos. I decided for 100 Happy Days I would always take the photos using the default Camera in square mode.

Hazel makes this really simple. Each time a photo that matches the criteria comes into my Dropbox Camera Uploads folder, it gets resorted and renamed to YYYY-mm-dd.jpg.

This simply takes care of the photos themselves. But now I want them to also appear on my blog. I have a separate rule that watches this new folder of photos and moves them into my Pelican project folder.

The key to this one is that I name them with sequential numbers, starting with 1.jpg. This will be useful later for my blog.

Pelican Blog

I decided to set up a hidden page on my blog to host these images. I created a custom template since it's fairly unique and different from the rest of my blog. The meat of the template is just this:

<article>
    <h3><a href="{{ SITEURL }}/{{ page.url }}">{{ page.title }}</a></h3>
    <div id="two-columns" class="grid-container" style="display:block;">
        <ul class="rig columns-2">
        </ul>
    </div>
</article>

I'm using the CSS for the gallery from this post by Ali Jafarian.

This is where my Hazel photo naming comes in handy. I'm using a simply JavaScript function to embed these images on page load.

function createImages() {
    start_date = new Date(2014, 03, 03) // April 3, 2014
    days_passed = Math.floor((new Date() - start_date) / 1000 / 86400);
    extension = '.jpg';

    for (var i = 1; i <= days_passed + 1; i++) {
        var li = document.createElement('li');
        var img = document.createElement('img');
        img.src = '/images/100daysofhappiness/' + i + extension;
        img.setAttribute("onError", "this.onerror=null;this.src='/images/imagenotfound.jpg'");
        var h3 = document.createElement('h3');
        h3.textContent = "Day " + i;
        li.appendChild(img);
        li.appendChild(h3);
        jQuery('.rig').append(li);
    }
}

jQuery(document).ready(function() {
    createImages();
});

I can easily compute the number of days that have passed and safely assume that an image exists for each of those days. I learned today that if you add the attribute onError to an image, you can create a fallback image in case the real image source doesn't exist.

I take a lot of quick photos that I never plan to keep around. In most cases, it's just to send to someone quickly. iMessage is easy, but the images aren't compressed nearly enough and can take a while to upload. I've now started uploading the images to Dropbox and sharing the link. The upload speed is reduced since Launch Center Pro will take care of reducing the quality before uploading. The message sends almost instantly because there isn't an attachment. Here are a few of workflows I use with Dropbox:

Upload last photo taken and get the link

This is if I simply need a quick link to share anywhere. The image uploads at 50% quality. I have a folder called Temp/_Destrctable Folder where I keep all my throwaway images. I'm using the TextExpander snippet ..ttimestamp to name the files like 14-03-08-19.42.45.jpg

launchpro-dropbox://addlastphoto?path=/Temp/_Destructable Folder&name=<..ttimestamp>.jpg&quality=50&getlink=1

Upload last photo and put the link in an in-app message body

Quick sharing with iMessage. Settings are the same as above.

launch://x-callback-url/dropbox/addphoto?attach=photo&path=/Temp/_Destructable Folder&name=<..ttimestamp>.jpg&quality=50&getlink=1&x-success=launch%3A//messaging%3Fbody%3D%5Bclipboard%5D

Upload from any source to Dropbox

Nice if you haven't taken the photo yet

launch://dropbox/addphoto?attach=photo&path=&name=&quality=&getlink=1

Going Digital

I was fortunate that when I decided to take on this project, my mom already had two large Epson flatbed scanners with transparency adapters. What was even better was that my mom was highly organized over the years and archived all of the negatives of every photo she had ever taken. This only left us with one thing to do: scan the photos. Since I was living in California and my mom in Montana, the work of pulling the negatives and scanning them was going to be done by my mom. We also needed to have a way to transfer the files from her computer to mine. Dropbox was an obvious choice in this case, but there was one problem that would complicate everything: hard drive space.

My mom was still using her PowerMac G5 at the time and hard drive space was pretty limited. It wouldn't have taken long before she wouldn't have enough space to even scan the photos. As a simple solution, once the scans were uploaded and synced to my computer, I could have manually removed the files and place them on my own computer, but I was lazy and didn't want to check constantly whether new files had been added. At the time, I was just learning how to code and thought this would be good practice ¹. I ended up writing a script that would mirror the folder structure for the scans in our Dropbox shared folder on my local Desktop and then remove the original file in Dropbox. The old folder structure was maintained so that if any new files were added to the same folder, my mom wouldn't have to recreate them. I then set this up as a cron job to run once a day and then send me an email digest of all the files that were transferred.

After a couple of weeks, I had nearly 15 years of photos in folders organized by year totaling around 85GB. Each Photoshop file was around 1.GB each and the photos were scanned at 300dpi. Now the hardest and longest part of the project was about to begin.

Cropping, Resizing, and Renaming

I wasn't entirely sure how I was going to do this part efficiently. My mom hadn't laid out the photos in a symmetric grid and there wasn't a reliable way to detect photo borders. I also decided beforehand that I wanted to preserve the original files and so I would save an individual Photoshop file for each photo that was cropped. I then wanted to have a separate folder that was simply for viewing the files.

Starting off, I wanted to try manually doing everything and automate things over time. Cropping the files using the marquee tool was always going to be manual. I would select the file, copy it, create a new file with the dimensions from the clipboard and then paste the photo into the new file. After I had gone through the entire file, I would save all the files at once with random names (you'll see later why the naming here didn't matter). This part immediately became tedious. I did some research on how I could make this easier or faster and discovered Photoshop actions². What was great about this was I was able to record every step I was taking into one single keyboard shortcut. This broke down the process to simply selecting the photo and hitting shift-F1. This one keyboard shortcut took care of copying the file, creating a new file with the dimensions of the clipboard, pasting and then finally selecting the previous file. That last step was key. Instead of a final control-tab to move back to the original file, the action took care of it for me. You can download the Photoshop action here.

At this point, I had an original Photoshop file and a folder called Cropped where all the new photos lived. I now needed a way to rename these files to something meaningful. Automator and Alfred made this simple. After I finished cropping, I would select all of the newly created files, run my Alfred extension "Rename Scans" which would trigger an automator script, prompt me to name the files, and then each file would be renamed from something like Untitled1.psd to November 1987_1.psd.

At this point, the final step for each of these files was to create a viewable JPG for every photo. Turns out, Photoshop has a great feature called Image Processor. After the files were neatly renamed, I would open up the Image Processor, select the folder, and hit go. My settings were always saved so there wasn't much else to be done each time I ran this. I would take the Photoshop files, create a new JPG at 5 quality in a new folder called Low Res Images with the same naming convention.

I was then able to share this folder back with my mom and the rest of my family. They enjoyed watching the photos get added over the last year or so as I casually worked on the files.

Once I had done all of the steps for each file, I would move the folder of PSD files into a folder called Done. This simply gave me a better idea of how many folders I had left to work on.

Viewing

In late January 2014, I finally finished cropping all of the photos. I never intended on it taking quite this long, but it was never something that needed to have a deadline. It felt great to know that I was finally done and could just sit back and look at all of the old photos from what I was little. I was using Lyn to view all the photos and realized that something was a little off. All of the photos were out of order. All of the folders had been named as Month Year, and even if I was viewing all of the photos at the same time, they were sorted in the order that I had created the files, not the time they were actually taken. I couldn't sort them in a photo viewer, Dropbox's photo tab would sort them by file creation and not EXIF date taken, and using Spotlight search was more-or-less pointless. There was no way that I was going to manually date 3,300 photos by hand. I had used the command line tool exiftools a few times, and I started looking into whether this would be a possibility for dating the files. It turned out that the command was really straightforward for naming a folder

exiftool "-AllDates=1999:12:31 12:00:00" foldername/

Even though I could have done it by hand, I didn't really want to have to type this in for 80 or so folders of photos. I quickly wrote up a Python script that would parse out the date from the folder names and prompt me to confirm whether this was correct or not. I was fine hitting Enter 80 times.

Some of the folders were called things like January-March 1995. For these cases, I would just assume the first month for the date. I wasn't going for perfection, but rather a good estimate for the time the photos were taken.

You can take a look at the script here. Do note that the script is really specific to my folder structure so it might not work perfectly for you, but it'll be a good start if you need to do something like this.

What I Learned

Epson now makes a scanner that eliminates a lot of the hard work around cropping the photos. It's expensive, but it would have saved me a lot of work.

I've made this comment before, but I still would love to have a way to embed facial recognition into the metadata of photos. My perfect world would be having the ability to do something like search for all of the photos of my brother before 1995.

Ultimately, this was a long, but satisfying project. I sleep better at night knowing that all of our family photos are backed up and not be lost forever if there were ever to be a disaster.

You can search either by the 3-character airport code or by the city name. You can make your search specific enough to return one result, such as "dublin, ireland"

or simple so that you can see multiple results, like "ireland"

You can download the Airport Codes workflow here:

Now that Launch Center Pro has the launchpro-messaging:// action with x-callback-url support, I can chain SMS messages together. This is something I've been wanting for a long time for this specific use case. When I began writing the action, I found one hiccup in which is I would need to write the message to each individual person unless I wanted to first copy it to my clipboard before running the action. I didn't have luck with the launchpro-clipboard:// action while calling the clipboard from the same action even though it is supposed to work in theory¹.

I decided to venture out and use Pythonista to generate the url scheme for me and then pass it back into Launch Center Pro's in-app messaging. The nice thing here is that I can cleanly list out all of the contacts to whom I'd like to send the message from with in the script and change the action much more quickly.

contacts = [
    'friend1@dropbox.com',
    'mom@gmail.com',
    '+1-555-867-5309'
]

The script is written in such a way that I can put as many contacts in as I want and the url scheme will still get generated correctly with the url-escaping.

Now, I have a nice list of message options in Launch Center Pro that will then send the same message to all of my contacts:

If the "Just Landed" option is chosen, a new prompt will be given to type in the place where I landed. Once I'm brought back into the app, I just need to hit send for each message.

You can download the Python script here and the Launch Center Pro action can be found here

[Update] 2014-02-15

I made some updates to the script so that you can send custom notifications for each individual person. It also uses Pythonista's location services to automatically put in the city name to make it easier.

contacts = [
     {
        'address': 'person1@gmail.com',
        'landed': 'Hi, Mom. Just landed in ',
        'boarding': 'Boarding now!',
        'shuttingdown': 'Shutting down. I\'ll text when I land.'
     },
     {
        'address': '+1 555 867 5309',
        'landed': 'Hi, John. Just landed in ',
        'boarding': 'Boarding now.',
        'shuttingdown': 'Shutting down. See you soon!'
     },
]

Here's a link to the updated version.

Taking Photos

My iPhone is one of the main ways that I take photos. Since it's always in my pocket and takes great quality photos, it's by far the easiest way to take photos no matter where I am. I've had a lot of fun with the iPhone 5S and the burst and slo-mo modes.

I've never considered myself a photographer. For a long time, I had my mom's hand-me-down Olympus E-500. It was a great camera, but I had no idea how to use it and it was bigger than I preferred. Before moving to Ireland, I decided that I wanted to learn the basics of photography and have a camera that would grow with me as I learned more. The Olympus PEN E-P5 had just started pre-order and I decided that this would be my first "real" camera. I only had a few requirements, and it fortunately satisfied both of them: GPS tagging and small/lightweight. I've now had this the E-P5 for a little over 6 months and couldn't be happier.

Importing

I only have one main way to upload my photos - Dropbox Camera Uploads. Whether I use the Dropbox app for iOS or the desktop application, my photos end up in the same place to get processed (more on that later in Organization).

Any photos that are taken on my iPhone are quickly uploaded via the Dropbox app. When I use my E-P5, I will first turn on the built-in Wifi to sync GPS data from my phone to the camera¹. Once that is all taken care of, I plug the camera into my laptop and Dropbox grabs the new photos and imports them.

Organization

I'm still pretty new to Hazel, but dealing with my photos was the reason I decided to bite the bullet and buy it. My Dropbox Camera Uploads folder was nearing 900 photos and I hadn't taken the time to organize them in over a year.

Before Camera Uploads, I was suffering through iPhoto. It always bothered me that my photos were obfuscated from view. I always found myself wasting time trying to find the original or using the export option. When Camera Uploads was released, I searched for a way to cleanly export my photos into a Year-Month-Event folder structure. I discovered this script that gave me more than what I wanted and solved my problem perfectly. For anyone who wants to use this, the command I used was

# -x deconflict export directories of same name
# -d stop use date prefix in folder name
# -y add year directory to output
python exporti_photo.py -x -d -y

I've been using the Year-Month-Event structure for a few years now and have starting running into a slight annoyance. I find myself constantly flipping between months trying to remember when a certain event happened. I finally came to the conclusion that the month directory was pretty unnecessary. What I decided on was the folder structure Year-MM.YY Event Name.

This gives me a much easier way to visualize my photos by event name rather than poking through folders by month.

My Hazel workflow is fairly simple, but takes care of everything in one rule. I've set up a few exceptions for photo types that don't need to be sorted, such as screenshots or other PNG files. I also have rules set up for fun projects like my "photo a day".

Finally once photos are sorted, I will manually go in and individually name all of the events that were created. This makes it much easier to search for events in the future. The next step in my process here is to tag photos. The one feature I do miss about iPhoto was the facial recognition. Since I haven't found a way to do facial recognition outside of Aperture or iPhoto, I will manually go in and tag photos with the names of those in the photos. This has been very useful when I want to find photos of people in certain contexts. For example, the tags me, office, dublin will give me photos of myself in the Dublin office, but not San Francisco.

Consumption and Sharing

In Mac OS X, I have three ways that I view my photos. The first, and most basic is Finder. The Cover Flow view in Finder is actually a great way to quickly go through photos and get the ones that you want. When I'm wanting to share my photos with others, I use the Dropbox Photos page. As a quick way to share a select number of photos quickly, I've still found this to be the best way. For general viewing and pruning of photos I don't want, I've been using a not so well known app called Lyn. It has some nice features for sharing to multiple services, but what I really like about it is that it'll just watch a folder and display the photos in that folder. Lyn will also let me see all of the metadata about the photos, including a map if there is GPS information. Lastly, on the rare occasion that I want to edit my photo, I will import the photo into Aperture. For the same reasons I dislike iPhoto, I dislike Aperture. I will typically import the photo, edit it, and then export back into Dropbox.

On iOS, I have two primary apps that I use to view my photos. The first, unsurprisingly, is the Dropbox app. For quick viewing and sharing, I will use Dropbox since that's where all of my photos live. As a Photos app replacement, I use Unbound. What's great about Unbound is that it treats folders in your Dropbox account like albums. Since my photos are organized this way anyways, I get perfectly created albums that I can view and even cache to my phone for offline viewing.

The Future

Dropbox has been doing a great job improving the photo experience. Photo organization is a very personal thing and trying to solve this for the majority is not an easy task. Many companies are trying to do this, and so far there has been no clear winner. As much as I love my folder organization, I would really like to get to a point where I don't even have to worry about where my photos are. The metadata of the photos should be enough for an application or website to organize the photos for me.

I mentioned this earlier, but one other thing I would really like to see is a 3rd-party app that does facial recognition and applies tags or some other bit of metadata to the file. Tagging my photos with peoples' names is by far the most manual part of my photos workflow, but also one of the most important to me.

This may not come in handy to anyone, but I thought it would be worth showing some of the creative ways that Hazel can be used to organize your files based on more than just creation/modification time or file type.

In late May 2013, I decided I wanted to do one of those time lapse videos where you take a picture of yourself in front of the camera every day. At first, the hardest part was just remembering to take the picture each day. Once I was in the routine, I started to find the task monotonous to pull the photo from my Dropbox folder, rename it to YYYY-MM-DD.jpg and then move it into a special folder I had creatively named "Picture a Day." Hazel was already taking care of my general photo organization, but I wanted to ensure that these photos got organized specifically so I started digging into the special traits of these photos. I quickly found a few default options in Hazel that would help me do this:

• Device make
• Pixel width/height
• Content creator

I was always using Camera+ for these photos because of the grid and level features. It allowed me to align my face in the same place in the photos. Since I always used the front camera, the dimensions of the photos remained the same. After playing around, here is the Hazel rule I came up with

Another key piece here is the datestamp token. The rule watches for Dropbox's Camera Uploads filename format YYYY-MM-DD HH.MM.SS.jpg. This wouldn't be necessary except for that this token then becomes useful in the actions portion. I can take that token and rename the file based on the token to simply YYYY-MM-DD.jpg since I don't care about the hour the photo was taken. What's great about the token is that this will prevent accidental naming of the file if I happen to upload it the next day or I'm flying between Ireland and the US and date times get messed up.

While this rule is fairly specific, it's saved me a lot of time having to organize the photos manually.

Customization

I haven't done much yet in terms of customization quite yet, but I'm adding little bits every day.

Original Files

I recently updated Pelican to the newest version 3.3. The part that was new to me here was that you have the option to keep the original file in your output directory.

# Set to True if you want to copy the articles and pages in their original format (e.g. Markdown or reStructuredText) to the specified OUTPUT_PATH.
OUTPUT_SOURCES = True

Update 2014-02-25: Turns out this was a bug. It's been since fixed. See the thread here on github.

I'm not entirely sure if it's a bug or something I was doing wrong, but I noticed that instead of creating an index.txt for every index.md file, it would create a directory called index.txt and then place the original markdown file within it. I did some poking around in the source code and found a slight issue with the copy function within the util.py file. It was checking if any destination existed, and if not, it would create a new directory.

if not os.path.exists(destination_):
    os.makedirs(destination_)

I made a couple of changes to prevent this from happening. The first was that I added an additional argument to the function called is_file and then added this to the destination check

def copy(path, source, destination, destination_path=None, is_file=False):
...
if not os.path.exists(destination_) and not is_file:
    os.makedirs(destination_)

Finally, in generators.py, I added the argument where the copy function is called in _create_source in the SourceFileGenerator class.

copy('', obj.source_path, dest, is_file=True)

Now that the files are being generated correctly, I used the tip by Gabe Weatherhead over at Macdrifter to add a link to the original file for every post. You can see an example of this post at the bottom of the page.

Automatic Posting to App.net

App.net's new Broadcast platform is pretty cool. I've subscribed to a few people already and I like the idea of having a way to broadcast each post that's made. Pelican doesn't have a great way to detect new posts, so I'm playing with my own solution by keeping track of every post and comparing.

In my Fabric file, I created a function to check for new posts and then use the App.net Broadcast API to make a post

def adn():
    current_posts = util.current_posts()
    post_history = pickler.load_old_results('lib/posts.pkl')
    new_posts = list(set(current_posts) - set(post_history))

if new_posts:
    for post in new_posts:
        get_adn = util.ADN(POST_PATH + post)
        get_adn.post()
    pickler.store_results('lib/posts.pkl', current_posts)

I get the current posts by simply listing the contents of the posts directory and then compare to what was previously stored the last time a new post was made. I keep this is a file called lib/util.py, which explains why I have to call os.path.dirname twice.

def current_posts():
    post_path = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), 'content', 'posts')
    return [f for f in os.listdir(post_path) if not f.startswith('.')]

This seems to be the most reliable solution since it won't send broadcasts if I edit a file. Finally, when the publish function is called from my fabfile, I call adn().

Automation

I'm traveling a lot these days, which means that sometimes I only have my iPad or iPhone with me. I'd still like to easily create posts without having to write up the post, log in via Prompt, commit and push. I went with a setup fairly similar to Evan Lovely and use Hazel to watch for new posts within a directory.

My Hazel workflow relies on an additional piece of metadata in my posts instead of just the file itself. This prevents any accidental posts and also lets me put whatever file I want in the folder. The file needs to pass the following script:

import markdown
import sys
import codecs

f = codecs.open(sys.argv[1], mode='r', encoding='utf-8').read()
md = markdown.Markdown(extensions = ['meta'])
md.convert(f)

if md.Meta.get('hazel'):
    sys.exit(0)
else:
    sys.exit(1)

As long as the piece of metadata "hazel" exists in any of my files, Hazel moves the file into my Pelican project folder and my publish script takes over.

That's it for now! I'll keep iterating on the process and make things better.

I created a very simple workflow that takes the contents of my clipboard and opens a file in a text editor of my choice. The Alfred workflow can be triggered in one of two ways. The first is with the keyboard shortcut "hyper"-s¹. The second way is simply typing "scratchpad" into Alfred.

I have it set to save a file in my Home folder called ".scratchpad" and then open the file in Sublime Text. Right now, the workflow will check for Sublime Text version 3, then version 2 and if neither exist, it will open the scratchpad in Text Edit.

You'll need the Alfred Powerpack to use this workflow. If you already have it, you can download it with the link below.

I was poking around in Launch Center Pro for iOS the other day to just see what kinds of things I could do, and I noticed one of the options was Soulver. My main use case for Soulver has always been one-off conversions or keeping score while playing Farkle with my girlfriend. It occurred to me that this would make a really nice way to do quick conversions.

I didn't want to have to create a mess of different actions for each unit. The way around this was to create a variable within Soulver and always reference back to it with multiple conversions. I started out simple with a quick US Dollar to Euro and Pounds and vice-versa.

To achieve this, Launch Center Pro uses x-callback-urls which allows apps to send data to another app and perform actions. The following url requests a number and then sends over this number as a variable to Soulver

soulver://new?text=x%20%3D%20[prompt-num:Text]%0Ax%20euro%20to%20usd%0Ax%20gbp%20to%20usd%0A----------------%0Ax%20usd%20to%20euro%0Ax%20usd%20to%20gbp%0A----------------&title=Currency%20Conversion

This worked perfectly and solved the two biggest requirements that I had: to be quick and to not rely on a data connection. I then wanted to go a little further and so I did some very basic unit conversions.

The url for the action is very similar, but using different conversions

soulver://new?text=x%20%3D%20[prompt-num:Text]%0AFahrenheit%3A%20x%20C%20to%20F%0ACelsius%3A%20x%20F%20to%20C%0A-%20-%20-%20-%20-%20-%20-%20-%20-%20-%20-%20-%20%0AMiles%3A%20x%20km%20to%20mi%0AKilometers%3A%20x%20mi%20to%20km%0A-%20-%20-%20-%20-%20-%20-%20-%20-%20-%20-%20-%20%0AFeet%3A%20x%20m%20to%20feet%0AMeters%3A%20x%20feet%20to%20m%0A----------------&title=Temperature

If you have both Launch Center Pro and Soulver, you can download both of these actions here:

Currency Conversion
Unit Conversions

When I first received the watch, I found the overall design and feel of the watch to be great. Putting aside the fact that the watch is meant to be an extension of your phone, it was a really nice watch to wear. It felt nice, it wasn't super flashy and I wasn't constantly worried that I was going to scratch or break it. What I found disappointing about the watch was that the notifications were inconsistent and took a lot of work to function at all. I ended up getting frustrated and put the watch in a drawer for nearly 4 months.

With the most recent update to Pebble's firmware, it's now really simple to get all notifications on the watch. Previously you couldn't only receive SMS , phone calls and email notifications. Now, any notification that is set to "banner" will show up.

The only downside I've found so far is that even if you have your notifications set to not appear in the lock screen, you still receive the notification on the watch. Since I receive so many emails, I like to only get the notification on my phone when I unlock the screen. Now every time I get an email, my Pebble watch is buzzing on my wrist. It creates a bit of unnecessary anxiety, so I'll have to see what I want to do with that.

Overall, it's a great update. I do find myself wearing the watch more often and am excited to see what other new stuff they have in store in the future.

In the early days, Dropbox offered a Public folder where you could easily serve webpages, images or anything else you want to share to the world. The risk there is that the links to the files were formulaic and anyone could crawl your Public folder looking for things they maybe shouldn't have. This formula looked like this:

www.dropbox.com/u/<number>/<name of file>

To add a level of security to the shared links, Dropbox now has a hashed value so that someone would then need to know the unique hash as well as the file name. The chances that someone is able to guess both of these within the next 10,000 years is pretty low. The second thing that was added was a preview to your shared links. If you have images, you see a nice gallery in your links and Office documents now have a preview. The downside here is that simple file hosting doesn't work by pasting in the link.

To solve this, you just need to change the actual shared link with the link to the file itself. To do this, you just need to replace www.dropbox.com with dl.dropboxusercontent.com. This will serve the true file instead of the file wrapped in a preview. For those of you using snippet software like TextExpander, you can make this a lot faster by making a shell script snippet with the following:

1
2
3
4
5

#!/bin/bash

url=$(echo %clipboard | sed 's/www.dropbox.com/dl.dropboxusercontent.com/g' | tr -d '\n')

echo "$url"

Now an image like

https://www.dropbox.com/s/kyjm1pr79g2irfj/Guinness%20Storehouse%20top.jpg

turns into this:

I very rarely write in a physical notebook. All my notes are written in either Byword, MultiMarkdown Composer or NVAlt and synced with Dropbox. Plain text allows me to have my notes everywhere, whether it's online, on my phone or my computer. I've had this recent desire to start writing things down in a notebook. I'm traveling a lot more lately and sometimes want to conserve phone battery or just entirely too lazy to pull out my phone.

To go all out, I did some research on what was considered the "best mechanical pencil." For the most part, rOtring pencils kept coming up as the best pencils out there. I decided to give it a try and go with the rOtring 600 0.5 mm mechanical pencil.

This is an entirely new world for me and I don't claim to be expert when it comes to pencils, but what I can say is that I love this pencil. The first thing that I noticed was the weight. It's definitely heavier than the average pencil (or pen for that matter). No part of the pen is made of plastic, which is also a huge plus and obviously contributes to the weight. So far it's been great to write with. The only complaint I could come up with is that I wish it were a little thicker. Sometimes I feel like I'm stressing my hand to hold on to it.

If anyone has any pencils or pens that they can't live without, I'd love to hear about them. My next test will be on my new Field Notes Expedition series, which are water, tear, burn proof.

I put together a really short video to show the pencil in use.

Since moving to Dublin, my girlfriend and I have wanted to keep our friends and family up-to-date on everything we've been doing. I recently bought the new Olympus E-P5 and have been taking a lot of pictures. So that everyone knows what we're doing, we decided to share a Scriptogr.am blog and post pictures of our adventures.

Sometimes it's quick and easy to snap a picture on Instagram and share with all your friends, but my parents and family aren't on Instagram, but they know to follow my blog for updates. Instead of having to manually pull the pics down, write up a post and publish it, I used a combination of IFTTT, Dropbox and my server at Macminicolo.net to do all the work for me.

The magic starts at IFTTT. I have a recipe that watched for a particular tag when I post to Instagram. If that tag exists, a text file is saved to my Dropbox account. I have a cron running once an hour¹ to run the script and check for any new files.

One of the only complaints about Scriptogr.am has been that I have to manually hit a publish button before posts will go live. But with their API, the posts are immediate². Now, all of my Instagram adventures (and my girlfriend's) can be posted to our blog for friends and family to follow. Once the post is made, I get a Pushover notification letting me know that a post was made by either me or my girlfriend.

If you're interested in the script, it can be found on Github here. An example of the posts being made can be found at our travel blog keephouseadventures.com

Since moving to Ireland, my contacts wouldn't show up correctly since I hadn't prepended all of contacts with '+1'. I wasn't about to manually change all 700 contacts in my phone and fortunately came across a nice post that had the following AppleScript:

tell application "Address Book"
    repeat with eachPerson in people
        repeat with eachNumber in phones of eachPerson
            set theNum to (get value of eachNumber)
            if (theNum does not start with "+" and theNum does not start with "1" and theNum does not start with "0") then
                set value of eachNumber to "+1" & theNum
            end if
        end repeat
    end repeat
    save
end tell

Before running this, I highly recommend backing up your contacts. This can be run easily by just launching AppleScript Editor and pasting in the code above. Enjoy!

They first told me that I was just going to have to be driven to the finish line. Luckily someone decided to get creative and boot the tire. We placed a piece of cut tire unbetween the tube and the tire. Once the tire part was fixed, we wrapped the tire in duct tape to keep it from possibly blowing again.

All in all it was a great day.

Now I was left with another problem. I was annoyed having to command-tab over to NVAlt, command-D to go to the search field and type just to find the note. With Alfred 2's new File Filter, I can now search specifically for my notes within Alfred. I now just launch Alfred, type 'note' and any keywords I want and am immediately taken to my note in NVAlt.

This last week I've spent a lot of time writing up plans and documents. I wouldn't put NVAlt in the category of great text editors and so I've been using MultiMarkdown Composer, but I'm still saving to the same notes folder. It seemed only obvious to add more functionality to my Alfred script. Now I can optionally hold down command or control to open my note in MultiMarkdown Composer or Byword respectively.

You can download the Alfred Extension here:

Be sure to set your path to your notes folder in the File Filter under Search Scope. If you're using Notational Velocity, you can change which application is opened in the action script.

Update 2016-06-23

As of June 2016, Instagram has changed their API and no longer allows this script to work. Sorry :(

Update 2014-09-04

I recently submitted a new plugin that now comes with Slogger which uses the Instagram API. You can check out my post with more information here.

I've received a few questions about this IFTTT recipe which logs my Instagram posts to Day One. There are a few others floating out there, but there are a couple of things that I wanted to have:

• The Day One entry date is the date the picture was taken
• The caption is saved in the journal entry
• Ignore duplicate posts if I also posted to Twitter

The last point assumes that I'm also using the default Twitter logger. If you want to ignore all of your Instagram tweets, add the following to be on line 112 in the twitterlogger plugin:

break if tweet_text.include? 'instagram'

You can download the Instagram IFTTT Slogger extension here. Simply add it to your plugins directory and run the following once to set up the slogger_config file:

./slogger -o instagram_ifttt

You'll need to set the location of your IFTTT slogger directory. The plugin will check for any text files and then automatically move them into a "logged" folder once they've been added to Day One.

To use, navigate to any file/folder from within Alfred and trigger the Actions panel. I have a file filter set up to search specifically my Dropbox folder

and trigger Actions to browse on Dropbox

Remember, you’ll need the Alfred Powerpack for these extensions to work. Click the following icon to download the Alfred script to your computer:

The first thing to do is figure out where you want to keep your .bash_profile in your Dropbox account. I keep mine in a folder called Sync that's shared between my work and personal Dropbox accounts. To move your .bash_profile, use the following command in Terminal:

mv ~/.bash_profile ~/path/to/Dropbox/.bash_profile

Once you've moved it here, create a new .bash_profile in your home directory and add the single line:

source ~/that/path/to/.bash_profile

Thats it! From now on, just point the local .bash_profile to the one location in your Dropbox folder.