More Detail

While we continue working through our response to the compromise we reported yesterday, we wanted to provide some more insight into the question of how this happened and what we have done to ensure security going forward.

On May 8, the Bitly security team learned of the potential compromise of Bitly user credentials from the security team of another technology company. We immediately began operating under the assumption that we had a breach and started the search for all possible compromise vectors.

Over the course of the next few hours, the Security Team determined with a high degree of confidence that there had been no external connections to our production user database or any unauthorized access of our production network or servers. They observed that we had an unusually high amount of traffic originating from our offsite database backup storage that was not initiated by Bitly. At this point, it was clear that the best path forward was to assume the user database was compromised and immediately initiate our response plan, which included steps to protect our users’ connected Facebook and Twitter accounts.

We audited the security history for our hosted source code repository that contains the credentials for access to the offsite database backup storage and discovered an unauthorized access on an employee’s account.  We immediately enabled two-factor authentication for all Bitly accounts on the source code repository and began the process of securing the system against any additional vulnerabilities.

We have a number of projects remaining to continue to add layers of security, but here are some of the things we have done since the breach and are continuing to work on:

  • Invalidated all Twitter and Facebook credentials

  • Rotated all credentials for our offsite storage systems

  • Enabled detailed logging on our offsite storage systems

  • Rotated all SSL certificates

  • Reset credentials used for code deployment

  • GPG encryption of all sensitive credentials

  • Enforced two-factor authentication on all 3rd party services company-wide

  • Accelerated development of our work to support two-factor authentication for bitly.com

  • Accelerated development for email confirmation of password changes

  • Added additional audit details to user security pages
  • Enabled detailed logging on our offsite storage systems

  • Updated iPhone App to support updated OAuth tokens

In addition, we’ve posted the answers to some frequently asked questions below.  We’ll continue to update you here as needed.

Rob Platzer

CTO, Bitly

FAQ

Were passwords exposed?

Hashed passwords were exposed but plain text passwords were not.  All passwords are salted and hashed.  If you registered, logged in or changed your password after January 8th, 2014, your password was converted to be hashed with BCrypt and HMAC using a unique salt.  Before that, it was salted MD5.  

Were any of my Bitlinks affected or changed?

No.  The production database was never compromised nor was there any unauthorized access to our production network or environment.  The data was from an offsite static backup.  There was no risk of any data, including redirects, being changed.