Firefox 4: libsmime3.dylib in use error during Mac OS X install

I have been using Firefox 4 betas on my Windows box for a while now and decided to upgrade my Mac OSX version. After downloading and opening up the dmg I tried to replace my existing Firefox 3.6 copy (after Quitting of course) and got an error saying that it couldn’t replace Firefox because file libsmime3.dylib was in use and pretty much out of luck.

After a few quick searches on Google I found out that it a common error even when upgrading to 3.6. One of the nice things about OS X is that you’ve got a BSD system and plenty of tools at your disposal. Here’s what you do to find which application is using a specific file and how to kill it.

Update: Before trying the following commands quit Evernote and Dropbox and try to replace Firefox again (thanks to Sebastian and Kim Mullins).

First open up a Terminal (Located under Applications > Utilities) and at the command prompt type the following command to find the application locking the file

sudo lsof -V | grep libsmime3.dylib

In my case this is the output I got indicating that vpnagentd was the application with the file in use.

vpnagentd 136 root txt REG 14,2 267408 28103066 /Applications/Firefox.app/Contents/MacOS/libsmime3.dylib

vpnagentd is a component of the Cisco Anywhere Connect VPN client (which I use to remote into work). Anyway, in order to do your upgrade you’ll need kill the app with either of the following commands

1. Using the process name (first column in the output)

sudo killall vpnagentd

2. Using the Process ID (second column in the output). You'll need to use this if you get a message like "No matching processes belonging to you were found" which means that probably the process name was truncated in the output.

sudo kill -9 136

Replace 136 with the number in the second column of the output.

After that you should be able to replace your existing Firefox application.

In some cases vpnagentd is set up to autostart which will respawn after you kill the process. In this case you'll need to enter a few more commands

cd /Library/LaunchDaemons/
sudo launchctl stop com.cisco.anyconnect.vpnagentd    
sudo launchctl unload com.cisco.anyconnect.vpnagentd.plist    
sudo killall vpnagentd

To start vpnagentd execute the following command:

cd /Library/LaunchDaemons/
sudo launchctl load com.cisco.anyconnect.vpnagentd.plist

References and further reading

• Download Firefox Betas
• lsof - list open files
• grep - print lines matching a pattern
• killall - kill processes by name
• sudo - execute a command as another user
• Eric's Agile Answers - for the tip on stopping and starting the VPN agent with launchctl