It is probably safe to say that Salesforce and Zoom are two of the most high-profile software companies operating today, and outside of an IT context, two of the best known to the public. And it’s certainly safe to say that thanks to their prominence, both are targets for threat actors and cyber criminals who would love to get their hands on the companies’ crown jewels.
Founded by ex-Oracle executives, customer relationship management powerhouse Salesforce weathered the dotcom crash to become the first cloud software company to cross the billion-dollar revenue threshold. Later, after buying Slack in 2020, it became the biggest enterprise software supplier in the market when its quarterly sales surpassed those of SAP.
These days, it is a global software behemoth that attracts the likes of U2 frontman Bono to chat at its annual Dreamforce jamboree.
Zoom, while no Salesforce in scale, has come to be just as much a critical tool as Salesforce for many. Founded in 2011 out of Cisco Webex, the company attracted little attention until early 2020, when the Covid-19 pandemic locked down the world and millions of isolated people turned to its video conferencing service for work and play.
But at the same time, a litany of data privacy and security issues surfaced in the then little-known platform, and it has since moved mountains to address them.
For both organisations, ethical hackers have come to play a vital role in helping seek out and squash bugs and security vulnerabilities in their core products. Salesforce set up its first bug bounty programme in 2015, eight years ago, and these days pays out over $2m in bug bounties every year, working through HackerOne.
“We very heavily rely on HackerOne as one of our security partners, and we entrust them to help us vet researchers for our bug bounty programme. It’s just one of our many security efforts,” says Lindsey Swartz, senior manager for security technical programme management at Salesforce.
“We really utilise the bug bounty programme and have increasingly utilised the differing and unique perspectives that hackers bring, in addition to the very intelligent security engineers that we have internally.”
“We rely on HackerOne as one of our security partners, and we entrust them to help us vet researchers for our bug bounty programme. It’s just one of our many security efforts”
Lindsey Swartz, Salesforce
Zoom chief information security officer (CISO) Michael Adams, who arrived at the company in August 2020, witnessed the company’s transition first-hand during the pandemic.
“I have been privileged to be ingrained in all the different aspects of the growth and the [security] commitment we’ve made. I think we’ve made tremendous strides … we’re very proud of the comprehensive security programme we’ve established,” he says.
Zoom started its first bug bounty programme in 2019, later enlisting the support of HackerOne to run things. Since its inception, it has paid out over $2.4m across more than 400 reports.
“In early 2021, we were about a year into building out our security programme, and so as we were ramping up … the reality was there were moments when we had to get out in front of potential threats and vulnerabilities, but you can only scale so much at any moment in time, particularly if you want to scale with strong talent and expertise,” explains Adams.
“HackerOne helped us leverage leading security researchers and the best ethical hackers from around the world, in ways that if we had tried to do it all ourselves we wouldn’t have been nearly as successful…. It was a profoundly important time for Zoom as an emerging platform, and that drew in some of the talent that might have been harder to do at other moments.”
H1-4420: hacking live and in-person
Ethical hacking is a job that lends itself to remote working – all you really need besides your skills is a laptop and an internet connection – and the stereotype of a lonely figure sitting in the dark in a basement apartment is not entirely untruthful.
But all hackers like to get out sometimes, and this summer, a group of 90 travelled from over 41 countries to come together in person at London’s CodeNode event space for a live, day-long hackathon.
Participants at HackerOne’s H1-4420 hackathon in London
This was HackerOne’s H1-4420 event, an annual fixture for a few years now, and at this year’s show, both Salesforce and Zoom put their platforms under the microscope.
For Salesforce’s Swartz, participating in the event was an opportunity to connect with and learn from the hacking community, increase the exposure of the Salesforce product set to ethical hackers, and seek feedback on what elements of its bug bounty programme were working well and which needed improvement.
“Something that was super valuable was being able to sit down and pick the brains of these hackers in terms of their methodology,” she says.
“As the threat landscape continues to evolve, I think it’s so helpful to not only work internally on evolving what that looks like in terms of security measures, but also to really understand what hackers are thinking about and what new strategies they are bringing.
“We started the day with intros and a welcome ceremony. And then the hackers just started working. There was a silent room where hackers could just focus on their work, there were common areas where hackers could collaborate, and a separate space for our triage team to intake those vulnerabilities, take quick action, review reports, coordinate on meetings with internal teams, and make sure that we can find, verify and fix, all within the same day,” she says.
Under normal circumstances, Salesforce coordinates with hackers from all over the world, so Swartz found being able to talk directly with them, ask them questions, and make sure her internal teams had all the needed details – or could revert to a hacker to ask them a question without having to wait for them to wake up in their time zone – exceptionally helpful.
“It was so cool to be able to look into a room and see five hackers sitting at a table and working on the same thing and learning from one another,” she says.
“At the end of the day, we had a ‘show and tell’, and it was great to see hackers looking at one another’s findings. That’s another value point of running a live event – it’s not only our team learning and evolving the way we think about bugs, but also seeing other hackers learn from the best,” she adds.
For the event, Swartz also provided a series of focus areas and target products that Salesforce wanted addressed – these included Commerce Cloud, Field Service Lightning, MuleSoft Anypoint Platform and Slack.
Zoom’s team rolled up to the venue near Liverpool Street Station off the back of a similar event in Las Vegas, with similar goals to Salesforce’s. “For us, it was an opportunity to put the full platform out there in front of security researchers and have them take a run at it,” says Adams.
“As we look to expand our impact in the market, it’s not about being first, it’s about being best, and this is a critical component of being best to market in my mind; security and privacy being part of the fabric of the company”
Michael Adams, Zoom
“I emphasise the full platform piece because that’s really important to us as we grow and diversify. It’s not just a focus on video comms or team chats, we spread out and present the whole play, which now includes things like Zoom Mail and Calendar, or Zoom IQ, which is our AI offering.
“As we look to expand our impact in the market, it’s not about being first, it’s about being best, and this is a critical component of being best to market in my mind; security and privacy being part of the fabric of the company,” he says.
Adams also wanted to test out a new, in-house Vulnerability Impact Scoring System (VISS). Zoom’s proprietary VISS is designed to complement the general Common Vulnerability Scoring System (CVSS) used by the security community at large.
“VISS allows us to tailor the security researchers’ work so that we’re focused on the things that are most important, that have the highest level of impact, and then our payouts are oriented accordingly,” he says. “So not only did we get the concentrated focus of the security researchers, but we got them to use VISS for the first time.”
Adams went into the hackathon fully prepared to receive all sorts of feedback and action changes to VISS based on that. This mentality paid off, and Zoom has since worked with one of the participants to weave more detailed examples of how severe a vulnerability may be in the real world into VISS. Adams hopes this will help hackers understand where they may want to focus their fire and what kind of payout they can expect.
“I don’t want security researchers to waste their time. I want them to know what is most impactful to us. I want that to be very clear and I want them to be able to focus their work in those areas,” explains Adams.
“That isn’t to say that if there’s a low or medium [impact] vulnerability I don’t want to know about it, but I’d much rather know about a high or critical [impact] vulnerability,” he adds.
“Not only do I want to pay accordingly, but I want to incentivise accordingly, and if I can’t provide clarity within my scoring system, it’s hard for them to understand. We’re now able to provide a certain degree of clarity – we iterated through the events and now we have some takeaways to move forward with and continue to refine the VISS,” says Adams.
Participants at HackerOne’s H1-4420 hackathon in London
Basic operational security best practice means Computer Weekly cannot disclose any of the specific vulnerabilities found in Salesforce’s or Zoom’s products that day, but we can reveal that some big issues were found, so the message for security teams at end users is, as ever, to make sure you are fully patched.
“Every time you focus a team of high-level security researchers you’re going to find some interesting things … [and] some areas where you didn’t have the resources to commit the time to,” says Adams.
“There were some findings that were helpful and interesting, but what’s exciting for us about those pieces is that they give us the opportunity – and we take advantage of that – to remediate issues before they become real-world problems.”
Why you should embrace hackers as well
Despite the best efforts of ethical hackers to reclaim the word hacker, the negative tropes, stereotypes and misleading narratives all stubbornly persist, and for all the good ethical hackers do, it can be hard for IT leaders to surmount these obstacles and convince the board they should connect with the community.
So, how can we convince the reticent that hackers are a benefit to the organisation? For Salesforce’s Swartz, one of the benefits of running a bug bounty programme through a supplier such as HackerOne is that it not only helps the community demonstrate it is ethical, professional and reliable, but that it hands control to the customer. For example, Salesforce’s invite-only programme means Swartz can vet who the business works with.
“We take into account a number of factors: professionalism, performance on similar programmes and assets, what their particular focus areas are depending on what we’re putting out there. We also look at what hackers our peers are partnering with, to ensure we’re partnering with hackers who are respectable,” says Swartz. And no, Salesforce does not work with those who have criminal records.
“There’s so much value to be had by embracing hackers rather than being fearful of the potential connotations, [and] there’s so much good to be had from partnering and collaborating and valuing their perspectives and treating them as professionals” Lindsey Swartz, Salesforce
“I know it sounds cheesy, but security is a team sport, and we value the perspective hackers bring,” says Swartz. “There’s so much value to be had by embracing hackers rather than being fearful of the potential connotations, [and] there’s so much good to be had from partnering and collaborating and valuing their perspectives and treating them as professionals.”
Salesforce has also realised the value from ethical hacking during live, real-world incidents, especially the “five alarm fires” of cyber security incidents, like WannaCry or NotPetya, that will go down in history.
“In December 2021, when the Log4j vulnerability was announced, we immediately kicked off patching efforts internally,” says Swartz. “While patching efforts were well underway, we enlisted the support of 30 of our most trusted bug bounty hackers to bolster our existing efforts to make sure there were no unmitigated incidences of the Log4j vulnerability.
“That was a really strong proof of the value we put on hackers and making sure we enlist their support in addition to what we do internally.”
Zoom’s Adams says the value of working with ethical hackers comes in part because the researchers act as a force multiplier in general, and in part because it enhances the sophistication and experience that Zoom can bring to bear on its internal security programme and development team.
This has been a significant investment for Zoom over the years, but fortunately, it was one that the C-suite was quite willing to make once it fully understood the challenges Zoom faced and the requirements it had, and the investment is certainly paying off.
Indeed, if you’re daunted by the prospect of enlisting ethical hackers, considering the value that bug bounty programmes can add to the balance sheet may be the deciding factor for the C-suite when it comes to getting the project over the line.
Reflecting on his own experiences evangelising the idea, Adams says: “There were a couple of different groups that came to our live hacking event that I had the opportunity to sit down and talk with. I probably spent at least an hour with one of them, talking about the nature of the event, talking about our bug bounty programme, why we believe in it and the value we have gained from it.
“At the end of the day, it’s not for me to make a decision on whether they should or shouldn’t do it, but if you want to see value, you can look at Zoom’s bug bounty programme and see it very, very clearly.”
