Service announcement: solving macOS KeyChain issues caused by Passenger

One of the major features introduced in Passenger 5.1 was the security update checker. This (optional) feature allows users to be notified in case there are any important Passenger-related security updates so that they can take timely action to keep their systems secure.

At the time of writing (Passenger 5.1.2), this security checker has a macOS-specific issue that could prevent networked Time Machine backups from working, and could prevent Macs from auto-joining known wifi networks. We are working on a fix, but until then we would like to announce a workaround.

This issue only affects macOS users, so all other users can simply ignore this announcement.

Problem description

In Passenger version 5.1.1 and on macOS only, loading Passenger into the system Apache would cause the security update checker to fail and print an error in the log. We attempted to address this issue in Passenger 5.1.2.

The approach we took to was to create a private keychain on Passenger startup, and set it as the default keychain. Then when Passenger shuts down it would revert this change. A more detailed technical description of our approach can be found on this Gist document.

However there was a difficult to detect side effect to this solution. Even when the change was reverted, the plist in which this preference was stored was not completely reset to its initial state. Though it should have been an equivalent and valid state, nevertheless the system would become unable to read the system keychain. This in turn prevented networked time machine backups from working and prevented affected Macs from auto-joining known wifi networks.

Workaround and future solution

We are working hard to address this and believe that we have a potential fix worked out, however this time we are performing much more testing and verification in order to try and ensure that there are no unintended side-effects to this patch.

In the meantime if you are affected by this bug here is a workaround to get your system working again:

sudo apachectl stop
sudo defaults delete /Library/Preferences/com.apple.security.plist DLDBSearchList
sudo defaults delete /Library/Preferences/com.apple.security.plist DefaultKeychain

Then edit your Apache configuration to include PassengerDisableSecurityUpdateCheck on. Then you can restart Apache and not experience this issue.