We want to start by saying thanks to the incredible creators on Roll that we now consider family after the past few weeks. I want to personally thank Sascha Bailey ($BAEPAY), Didi Taihuttu ($FAMILY), Tijo ($BEAR), Joel and Travis ($HERO), Skeenee ($SKULL), Julien Bouteloup ($JULIEN), Richard Kim and Sam Engelbardt ($RNG), Alex Masmej ($ALEX), Harrison ($FIRST) Connie ($HUE) and many other Roll creators specifically for their time, input and feedback on how Roll can better serve creators. Trevor, Cherry and Andrew: respect you guys immensely and looking forward to what you do next. You guys are trailblazers (you already know that) and we thank you.
We’d like to give another personal thank you to our friend Whaleshark ($WHALE), who’s been a guiding light for many creators after the hack in March. They’ve built an extraordinary community and are a hallmark of what’s possible on Roll.
Additionally, we’ve spent a large portion of the last two weeks listening not only to the 42 creators that were affected, but hundreds of Roll community members that have gathered in our discord, emailed us, been vocal on twitter, hopped on a call, provided support, criticism and suggestions on how Roll can be better not only for creators, the the millions of users we wish to serve in the coming year. We see you and we hear you.
Update
Security has been and always will be paramount for us. The rest of this post serves as update on the security hack at Roll on March 14, 2021. An attacker was able to gain access to the private keys of Roll’s hot wallet and steal social money stored in that address. The attacker then sold all the tokens on Uniswap for ETH and transferred the ETH to Tornado Cash.
We are working with a blockchain forensics investigator and law enforcement to understand how the attacker might have gained access to the key, and to try to identify the attacker.
A total of 42 tokens were affected in this attack. This is the list of tokens that were affected: WHALE, FWB, KARMA, JULIEN, 1337, MORK, CHERRY, FAMILY, BEAR, SKULL, LADZ, RARE, ALEX, PICA, BAEPAY, SWAGG, KERMAN, CAMI, HUE, OSINA, ATS, GOB, ARKE, SCOTT, JAMM, FIRST, PAUL, DSGN, JOON, CALVIN, WGM, BPC, ALXO, YUMI, PIXEL, RDR, BONES, GCASH, FORCER, PYGOZ, TING, HERO.
These were the 3 transactions – 1, 2, 3.
As soon as we became aware of the attack, our first priority was secure all the remaining tokens. We transferred all the remaining social money into our multisig and disabled all external withdraw transactions to ETH addresses. Beyond the 42 tokens above, the over 300+ tokens have not been affected. Those remain safe in our multisig. There are no additional tokens in the compromised hot wallet.
All the tokens that the attacker was able to steal have already been sold for ETH. The attacker no longer holds any social money.
We created a $750,000 fund internally to help creators and their communities affected by this. There is no single way to make this fair to all the affected parties – creators, their community, and the Uniswap LPs. We deployed the ETH to help as many communities as we could by directly buying the social money from the Uniswap pools. This was essentially a counter-trade to the attacker.
We’re also deploying Roll’s allocation of social money, combined the the tokens that we bought in from the pools after the hack to further compensate creators. There are a few creators we were unable to make whole at this stage and are working with them directly.
Regular Users of Roll
- All of your balances of the 300+ tokens that were not affected in the attack are safe. You can always check your balance by logging into your account on the Roll app.
- Your balances for the 42 tokens that were affected will be compensated. Your balance on the Roll app for these tokens is currently up to date and accurate.
- You can continue to be LPs of the social money on Uniswap. The attacker does not hold any of the 42 tokens anymore.
- You will be able to withdraw the social money on Roll to your personal ETH addresses soon. More on that below.
- Feel free to reach out to us at [email protected] for any questions.
Creators on Roll
- If you have issued social money via Roll that isn’t one of the 42 tokens affected, then you are not affected by this incident.
- If you have issued social money via Roll and are one of the 42 creators affected, we’re working with you directly to find the best path forward. We are immensely grateful to all the communities that have been really supportive of our efforts to recover from this.
- Your community can continue to be LPs of your social money on Uniswap. The attacker does not hold any social money anymore.
- You and your community will be able to withdraw social money you currently hold at Roll to your personal ETH addresses soon. More on that below.
- Feel free to reach out to us at [email protected] for any questions.
New Creators
- If you are waiting to mint your social money via Roll, that process will be delayed by a few weeks.
- Your token name and symbol will still be available when it is minted.
- More details on the changes to our infrastructure and smart contracts are outlined below.
- Feel free to reach out to us at [email protected] for any questions.
We are working on re-enabling withdraws in our system after a comprehensive overhaul of our key signing infrastructure as outlined below. The current timeline is 1-2 weeks. We’ll keep you all updated on the timeline as we move closer to opening up withdraws.
Investigation
We are conducting parallel investigations on the incident currently. We want to understand how this happened and how to avoid something like this from happening in the future.
We are working with a specialized blockchain forensics investigator that also provides cybersecurity services and investigation. We are still in the middle of this investigation and they are following certain leads related to the transactions the attacker performed.
They are also closely looking into our infrastructure and operational procedures to narrow down the probable ways the key was compromised. We will be revamping our infrastructure around key signing and management going forward.
Simultaneously, we are also working with law enforcement agencies around the incident. We will share any results of the investigation with the community. Due to the sensitive nature of the investigation and the fact that it is ongoing, we cannot share any intermediate updates yet. If you have any leads on the attacker, please contact us. We’ll have a generous bounty if the information you provide is valuable and directly leads to the apprehension of the attacker
Security Enhancements
After the incident, we immediately performed an informal security audit from a top security practitioner of all of our infrastructure. This helped us identify any weaknesses in our infrastructure security, access control, and operational security. We are revamping all of these currently.
In addition to that, we will have an independent, third-party security audit of our entire infrastructure, especially around the hot wallet management and any transaction signing and private key management on our infrastructure. We will share the results of the audit with our community.
Here are the specific steps we are taking –
We are significantly enhancing the security around the hot wallet key management. We are leveraging AWS provided Hardware Security Modules (HSMs) to hold the private keys to our hot wallet. The keys will not leave the HSM module but can be used to sign and send the transactions back to our blockchain services, which will then submit the transaction to the blockchain. The code that interacts with the HSM signing will be locked down both from an access control point of view and any code updates as well to provide extra security. The service will authenticate all requests via a certificate to make sure the caller is legitimate. In addition, it will only sign certain type of whitelisted transaction types such as withdraw of an ERC20 token.
In the intermediate term, we are exploring moving to a multi-party computation (MPC) framework to sign any transactions. In this framework, a set of independent parties hold a piece of private data. This prevents a single point of compromise. The private keys are not held in one single place.
This will also let us set up highly customized alerts and requirements around key signing. For instance, we can institute a four-eyes rule for large withdrawals (at least 2 independent approvals) but let the smaller withdrawals go through without any manual intervention required. We can set rules around time-based withdraw limits and approvals as well. All of these can happen without the full private key being stored in a single place.
We are also creating a state of “pending withdraw” on our web app. This will be triggered based on certain rules such as daily limits and transaction amounts. If this is triggered, then the withdraw will be in a pending state and an admin will manually review and approve the transaction. At that stage, it will get submitted to the blockchain for confirmation. This will show up in the users’ history section.
On the infrastructure end, we are creating more granular access control to each part of the infrastructure and enhanced logging to better alert us of any suspicious behavior. All of these will be audited by an external third-party security auditor.
Finally, we will hold only a limited amount of social money in our hot wallet going forward. For large withdrawals, we will rely on the “pending transactions” flow outlined above. We are currently aiming to process all pending withdraw transactions within 48 hours and improve our operational procedure here over time.
Smart Contract Enhancements
We are deeply rethinking some of the aspects of the Roll smart contracts. While we had plans and a roadmap for a V2, this security incident has made us think hard about some of our initial choices. To that end, we are going to deploy a new “V1.5” version of our smart contracts in the coming weeks.
This version of our smart contracts will be able to give complete custody of the vesting funds and initial allocation to the creators themselves. The creators can then make an independent decision on how much social money to keep on Roll depending on their usage of the tools that Roll has built and will build in the future.
We will also be adding more flexibility around the initial allocation to the creator, the Roll allocation, and vesting details at the time the social money is created. We originally sought to standardize this for all creators on the platform, but the concept and usage of social money have grown beyond that initial step and we want this to be flexible for a much wider set of creators.
We will continue to keep the maximum supply at 10MM for all social money issued via the Roll smart contracts in V1.5.
More details on smart contract enhancements separately when we are closer to launch.
Thanks again to the endless list of people that have reached out to support, thanks to those that want to see us do better, and thanks to those who have just started embracing this new medium. We love you and will be back extremely soon with some news.
– Team Roll