How to bake backup into your Salesforce releases

Share with

Transcript

So if everyone's ready to get started, I shall hand over to you.

Perfect. Thank you very much, Amy. It's always lovely to contribute to Gearset Snow, a broader mission. Thank you very much. And thank you very much for everybody who has joined and take out their time from extremely, extremely busy schedule. We know I know everybody in the Salesforce ecosystem is extremely busy, so really appreciate that for your time.

So today, as as Amy mentioned, we are going to cover, backup, how how backup can be taken not just as a afterthought, but in fact, we'll try to bake it into our releases to make sure that if anything goes wrong, you always have a time machine to go back and restore. And I'll share my, you know, my journey, how we started from day one when it when backup popped up in in our team as a as a as a as a a acronym.

So a little bit about me. I am Nadeem. As Amy mentioned, I am into Salesforce DevOps.

It's been almost ten years now. I have been dealing with all things DevOps and Salesforce.

I know DevOps or Salesforce is little bit weird when you compare with traditional software development. I have been deeply involved in a in our release management for Salesforce governance, building environment strategies, and especially contributing to the large scale Salesforce programs. And as as Amy mentioned today, we are going to see one of the most underrated topic in DevOps for Salesforce, hack ups. Especially, why we should not take it as an afterthought. In fact, we should bake it into your releases.

And my my my aim is to share my own experience and also uncover some of the most common misconceptions when it comes to backup and definitely share my own journey, how we convinced our management, how we get the buy in to get a backup solution and gain our confidence in our releases.

Okay. So most important question. Why why backup matters in Salesforce? Right? I I have seen this many times. I get this question asked every time whenever I am involved in any engagement or I get DMs on my on my LinkedIn, you know, account that.

And what I found is one of the biggest misconception in Salesforce ecosystem is, like, why do we need a backup? Salesforce does have a backup for for my data and metadata. Right?

That is true, but not at the larger, extent.

And that helps us to accept the fact that Salesforce is not something which which runs on its own. There are there is a concept exist, and we all know that as shared responsibility model where Salesforce owns some part of your system, but you also, as a team, is responsible to protect your data because the biggest, you know, threat to your data does not come from a system failure or system problems.

It comes from your business issues.

And that's that's the fact, and that's what we have seen on the real world.

Especially, if when you when you deploy a for example, we know I know that we we we advocate the agile practices, but the matter of fact is there are teams who still rely on big bang deployments. I call it a nuclear deployment. And I can virtually guarantee whenever there is a nuclear deployment, there is always a a sense of concern within the team. If anything goes bad, what are their strategies to bring it back?

What are the steps to be taken to make sure that everything is back up and running. And it it it it motivated us to think about, like, how much appetite we have to suffer a loss, whether it's data or metadata. So and also, apart from all this data issue, we also know the compliance angle is increasingly, spreading across the ecosystem. The more regulated industry you are, then you need to demonstrate that you just not take backups, but you can also restore quickly and reliably be because you need to try gain trust of your customers and your stakeholders.

And most importantly, you need to gain the peace of mind so that developer can innovate faster, knowing that if if anything goes wrong, they always have a time machine to go back and restore.

But having said that, let me ask you, where are you? As a team, where do you stand as of today?

If I ask you that, okay, how frequently you back up? For example, what you see on on on my screen is is a report from Salesforce DevOps state report for twenty twenty five, which tells us that forty four percent of the teams do the backup, daily.

So this is something which is increasingly gaining adoption within the Salesforce ecosystem.

Okay. So as I mentioned, one of the biggest misconception is Salesforce keeps data safe so I don't need backup. That that's the biggest misconception I have seen in the ecosystem. And one one one one argument I see on on day to day is why do we need a backup?

We can always go back and refresh. Right? Ask yourself how easy for you is to refresh your sandbox, especially in a very, very fast paced environment where you deploy it on a for example, if I take my example, my current team setup, we deploy it on a daily basis. There is always a heavy traffic in a lower environment, especially in SIT and UAT.

So you ask yourself, how frequently you are free to refresh? And I'm I'm hundred percent confident that majority of the team is not ready to refresh their sandboxes because it comes with in of its own pain points. Right? If you refresh your full copy sandbox, it comes with so many strings attached.

You need to mask your PII records. You need to reconfigure all your, integration endpoints, many other things.

And then, again, coordinating with the team, communicating, making sure that your instance is on the right instance that you want, whether it's preview or non preview. So many strings attached to a refresh. What we see on paper is not what we see in the real ground.

And, obviously, if if any deployment happen, for example, as I mentioned earlier, the nuclear deployment, it I can virtually guarantee it will fail. And I have seen in the real world where mass data update or something goes wrong, any field deletion, it can have a knock on. I I really like how Lauren said here said says that if if your metadata is broken, it can have a severe knock on impact on your data as well. I I I have a very good example here what I have seen in the past. One of our developers accidentally deleted all the Apex classes from the lower environment, and it was a shared sandbox. Imagine the chaos.

Everybody was started shouting what is my my my functionality is failing. I don't see my workflow working. When when I when I was involved with this, I found that more than nine hundred Apex classes were deleted. And it's the fact it happens.

That's where the backup the the rule of backup comes in. You your developers are all always having the peace of mind that, okay. No worries. We always have a mechanism to bring it back.

There was also incident when one of the nuclear deployments, corrupted all our record type assignments, and it was a it was a bad night for all the team members to fix it. And it happened before we had the backup, by the way. So it it it took almost eight hours for us to rebuild all the assignments and all the all the systems back in place. And imagine the downtime and all the reputation loss to the stakeholders for the production environment. And also, as I mentioned, this compliance requirement getting more and more into mainstream also motivate us to, you know, have a backup solution that you can have more confidence within your customers and also in the all the stakeholders.

Most important, our learning journey, how we started. So it all started with so we we already have GFS. One of the most important factor what I have realized is when you if you see the DORA metrics of DORA survey of the last year, it was found that teams who have one single tech stack to cover all their ecosystem tend to have better Dora metrics compared to those who have a scattered tech tech stack because you need more different set of knowledge, skills, management, so many other things you need to take care of. That's where one of the biggest advantage of having a tool like Gearset is it it gives you a ecosystem, not just a solution. It gives you a ecosystem which covers end to end your DevOps, your life cycle all the way from deployment, monitoring, automation, and backup as well.

We started our learning journey from one of the amazing platform, Gears at DevOps Launchpad. What I love about this platform is it's democratized. And that's the beauty of this ecosystem is most crucial knowledge is democratized. So it's not something like limited to a team, limited to a company.

Everybody is free to learn and challenge their current understanding. So I would highly recommend if you're thinking you're in a journey to start backup solution in your team, start with DevOps launchpad. There is a great certificate which covers the end to end modules from basics to advance. You should definitely check it out.

Wonderful.

One of the pivotal, instance in our learning journey was when when we we as a team connected the theories which we learned from this, platform to the real world prob problems, the end to end implementation flow that, you know, if if, let's say, if any, flow deployment went sideways, then we realize that, okay, we need a solution like backup to, make sure that developers have the peace of mind. And most importantly, not just having the backup, If having a backup is important, then having a robust system to restore is also as as important as having a backup.

That's what we realized once once we started the journey.

What we learned well, one of the most important learning which we found during this journey was the mindset shift.

That the when you have the safety net, your it changes the psychology of your team. Right? What do we mean by that is it gives more headroom for your team to experiment because your team cannot innovate in a constant fear to lose something. You have to give them a safety net so that they they can be creative. That's what we found after we implemented backup.

And we all ultimately, we realized that Salesforce is secured by default, but secure is Salesforce is too big to fail. That's, you know, that's that's the reality.

And we also realized that it's a shared responsibility model to also and especially for any fast paced team, especially I mean, I I belong to a fast paced team and I have seen that when whenever you deploy faster, you comes it it comes with a risk, especially when when there is a gap in your QA QA resourcing, you you you are fixing a hard fix. You are skipping the testing cycle.

Anything related to that, that comes with a risk. And if you have a risk, it's indirectly proportional to a data loss or metadata loss.

What I have seen what I have also seen is we treat metadata something not linked to your data. But the fact is in Salesforce, your metadata is in these increasingly linked to your data. So if your metadata loss can have a impact on your data as well.

That's where we started the awareness for backup for Salesforce. We started to spread the word within the team that, hey, we really need something like backup so that if anything goes wrong, we always go back and restore. And there are some nuances to Gearset specifically, which I will cover later. We we did a a very thorough POC across many different systems, including Gearset and other systems.

What what we found unique about Gearset is its its ability to control the automations. In real world, when you do a restore or, you know, restore of a backup, your automation can act as a blocker because you have validations, triggers written on your object. That's where you need you need to have something which can switch off your automations.

And the fact is you need some some tool which gives you control to control the automation. What do we mean by that? When we did the POC for other systems, it it it did not give us the control. So when you when you attempt to restore, it switched off and switched it on on itself.

Don't have the control. Whereas in Gearset, you have the control. So let's say I'm attempting a restore and it failed at the first attempt. So Gearset is not enforcing me to switch on the automation again.

So let's say if I am attempting five attempts to restore my data, I don't have to switch on and off my automations five times.

That this is what I found unique about Gearset, and that was one of the reasons we we gone with Gearset as a solution for backup.

These are the nuances which you only see when you give your get get your hands dirty in in a solution like like backup.

So as I mentioned, once the backup was in place, our day to day culture drastically changed.

And, obviously, the developers were more confident because they knew that they could recover if something went wrong. And the re most important, the release managers now have more confidence in their releases because whenever there is a big bang release or nuclear release, they they they follow the pre, post deployment snapshot concept that really help them to compare what was the shape of the metadata before the deployment and what it it is after the deployment. Because you can't you you can roll back your metadata, but there is no mechanism for you to roll back the data.

Now, most importantly, the wider team, all the wider team members saw that your Salesforce releases are safer and more reliable.

And most importantly, this implementation was absolutely aligned with our broader DevOps goal, and that was security, resilience, and continuous improvement. I still remember the quote from one of the Dora, articles.

It was mentioned that we should not look at the elite teams. In fact, we should look at the teams who is getting better at getting better. That's what we're trying to achieve here.

And as I mentioned in my pre I should have mentioned in this slide, but I mentioned already one of the biggest win for us was the comparison of pre post deployment snapshot.

And this is what we have implemented already in our releases. Whenever there is a big bang release, we always try to, take a backup. That's one of the freshest backup before that the release engineer hit the deployment and for the big bang releases.

And it it significantly gained confidence for the release engineers to deploy safely and have the pre, post deployment snapshots to compare what went and how it looks like after the deployment. And most importantly, getting the peace of mind. Right?

One of the best practices which I want to share with all of you here is start your backup strategy with production and, you know and start small.

When you start small, you will have more room to experiment.

And that's the key key decision maker for you to gain more confidence across the other other other environments. And then you can slowly, gradually go lower to the lower environments. In my in my in my experience, what I found is initially when we started, we started with production only. But when we realized that after that incident, when our developer deleted all the classes, we realized that it should be expanded across the environment, whether it's one developer whatever strategy you have, whether it's a shared sandbox strategy or it's a for a developer sandbox strategy, you should have backup for all the environments. It it def definitely creates that safety net.

And one of the things which really pushed us to think about something like backup solution is increasingly increasing release frequency because the demand of business was increasing. There was huge number of deployments coming in. Our team exponentially grew. We were just twenty people in January this year, and we are in October. We are almost forty five people. So you can imagine how fast we grew. And when you grow faster than usual, that's where you you you miss the caps.

You miss what what really what what what what should we really focus on.

That's where if you have a bug backup solution from day one, you don't have to worry if anything goes bad.

Because when you grow at a unprecedented rate, it comes with a risk. It and that's where you should definitely have a safety net in place before you increase your, expand your team exponentially.

And and and and as as you can see, you know, before we had the backup, if something broke, we didn't always have a quick way to recover. We always have to think and take the traditional steps.

Take take the code from repository, then again build it locally, deploy it to the environment. That's again a painful approach altogether.

And we also implemented the artificial restore approach. Right? Why why why it is important is, as I mentioned, if taking backup is important, then same way it is most more important that you know how to restore and what are the necessary steps to restore, who will own the restore. So these things must be taken care from day one. Like, you should build the strategy such a way that you are you are very clear on the nuances of taking the restore. If anything goes bad, what are the key steps you will take to make sure that how to overcome that problems?

And most importantly, I as I mentioned, that one of the key learnings we got is that metadata rollback and data rollback are not the same. You need strategies for both. Right? Because though both comes with its own, nature.

And it's not just backups. Right? You're taking snapshot. Backups are not just taking snapshot. You have to actually test restores.

Otherwise, you are just building false confidence. And that's what we realized in the in the very beginning. That's where we we we started to know build this culture of no. We should not just take the backup and leave it.

Let's have a formal disaster recovery plan and define the RTO, recovery time objective, how much time it actually takes us for the for the restore. And it on paper, it looks good, but in real when you come to the real world, it all depends on what type of failure you have, what type of data corruption you have. So having a strategy gives us good starter for you to, you know, gain confidence.

And lesson learned, the biggest the biggest change which we acquired was the day to day culture changed, and the developers started feeling more confident because they knew that if they can always recover if something went wrong.

And this pre post deployment snapshot allowed the release managers to put so so the release manager could focus more on planning and less on fixing the issue and saving production environment.

And, ultimately, the wider team saw that Salesforce releases are safer and more reliable.

And as I mentioned earlier, it aligned with our broader DevOps goal, which which is a a complete ecosystem that consists security, resilience, and continuous improvement.

So what we we saw, it was not just cultural. In fact, we could measure it. I mean, if you rely your team on data, that's where the the true decision making comes in.

There was a complete reduction in rollback times When we did not have a backup solution, sometimes it takes days for us to recover a data loss. All data loader activity and we did it up, rebuilding, it takes sometime it takes days. But when we onboarded a backup solution and we did the, disaster recovery, implemented this as disaster recovery, it comes it came down to ours.

And most importantly, we implemented the automated daily and weekly backups across the environments. So it's not just you you need to actively take the backup. DSIT allow you to take the backups proactively so you don't have to worry about whether it's running or not running. It it it it takes care of that.

And, obviously, this audit team was extremely happy with us. Whenever they ask something, we were always in a position to share it with them.

And most and and what we what we should think about is the real value is not just in having a safety net. In fact, it is in giving your developers and admins the headroom so that they can be creative without compromise. It's very important.

And one of the best practices checklist is go start small, focus on production first. Make sure you test your restores before you rely on them. Don't think that I have the backup. Let me rest assured. Make sure you test it, grill it, break something in production environment. Go and do the manual backup restore and see where is the gap and try to build the knowledge because that knowledge will act as a as a the the the knowledge will allow you to build the autonomy, not to depend on some specific team member. That's very important.

Now build back backup checks checks be into your releases readiness process.

And, obviously, the buying backup protects business continuity and reduce risk.

And make sure the implementation is just the beginning. That's where you start. It's not the end.

And removing your policies regularly and updating your runbooks, keep testing your restore. This is very important.

Okay. Now the most important question.

Which one is your team?

There are only two types of team in the Salesforce ecosystem. One with backup and one without backup. So where where do you fall?

In fact, we need more voices to challenge the traditional notion of backup and recovery in Salesforce. Why? Because the true innovation does not happen when you you when your team is operating under the constant fear of losing their data or metadata.

In Salesforce DevOps, you cannot innovate at speed unless you can recover at speed. This is very important. And every day, we are becoming in more and more fast paced. So make sure, you know, you have all the tooling in place so that if anything goes wrong, you always have a time machine to go back and restore.

That's it from me. I would love to hear from you. What's your biggest challenge when it comes to backups today?

And if if your team goes sideways in production deployment, how much time take? How how much time will it take for you to restore?

Thank you so much, Nadeem. Sorry. I was just trying to get a link for the site in the chat there. We that's been such a great and helpful session, and we've had a ton of questions come through. And the first one I'd like to ask actually has come through in the chat. This is from Rajat who asks a very good question about aligning your backup schedule with your deployment cycle.

I wonder, Nadim, if you have any best practice advice on how to align those two Absolutely.

That's a great question. So it all comes on what your deployment is touching. So not all all object is same. Right?

Not all object in your business is equivalent. For example, your invoice is not your invoice is more important compared to your scheduled, object. So you need to make sure you build a strategy. If you're touching your core business objects or core business metadata, you always take a snapshot or run your backup before you hit the release, deploy.

And that's option number one. And what gives it allows you to have a high frequency backups where it allows you up to I'm not sure the exact number, I think, but I think it's up to ten objects which can run on high frequency time, for example, every one hour or so. So that's something which you can leverage.

Amazing. Thank you, Nadim.

Sorry. Sites auto corrected my message there, but I've popped a link in the chat for you and your question about deployments and merging. And there's a great we have a great course there on how Gearset compares and deploys, and that will take you through that. But if you do have any super specific questions on how the Gearset how Gearset's developed solution, no matter what element it is, backup, compare, deploy automation, how that works, do feel free to reach out to Gearset's live chat.

You can access that straight from our homepage, gearset dot com.

I'll just pop a super quick link in the chat there.

Yeah.

One of the one of the great thing one of the great thing about Gearset is it allows you to have thirty day free trial, everything included. You just play with it, grill with grill it. It's all yours for one month.

Exactly. Exactly.

We've had a few people asking if they can test backup. So as Nadim says, that's the free thirty day trial gizzard is the perfect place to head if you're curious to try things out yourself.

I'm gonna we're gonna run over a couple of minutes because I think we've got a couple more awesome questions that have come through in the q and a.

Yeah. Definitely.

So I just wondered this question is maybe it's quite a tricky question, Nadine, but I wondered if you had any thoughts on this one. Lauren's asked, how do you see backup best practices evolving in twenty twenty five and beyond, especially in terms of AI and automation?

That's a great question. Right?

You know, when it comes to AI, it's it all it all it all boils down to data. Right? The quality of data you have. So it's very important that you have a safety data around your data.

If it if data corruption is happening, do you have the visibility or not? So the answer to your question is it will go in I mean, will go to the j curve. The adoption of backup will be in j curve. If you draw drive adoption rate, it will be in J curve because it's all around data.

Right? AI you cannot build the AI system without quality data.

I don't know if I answered your question correctly.

No. I think that's bang on. I yeah. I love your point about it's all about the quality of data.

I think it's so accurate. But, Nadeem, I wonder if you had any advice on you spoke a little bit about the differences between data and metadata and the importance of backing up both. But we have got some questions around the strategies of how to get started with both metadata and back backing up both metadata and data. Do you have any advice around what you should start with?

What should the end goal be? Any strategy advice there?

Yeah. Yeah. Definitely. So as I mentioned in my talk as as I mentioned in my talk as well that in Salesforce, it's a unique system. Say, your metadata and your data are are in is in singly, linked. So if there is any impact on your data or the metadata, it can have a knock on impact on the data as well.

So don't treat it two separate entities. Treat it like a single entity, and that's that's where you should start.

Because you you cannot have a robust data backup without having a robust metadata backup because it both are interconnected.

For example, let's say, you you are you are having a backup, and after backup, your metadata is corrupted. For example, your pick list got changed or anything anything happened in your metadata level, so you cannot be able to restore until your metadata and data are aligned. So treat it as a single entity. Do not treat it as a two different entities when it comes to strategizing the backup solution.

And that's where Gearset plays a very good role. Right? It it gives you the same UI for you to do the restore for data and metadata both.

But it will not feel alienated when you when you get your hands on on this platform.

Awesome. Yeah. Thank you so much, Adeel. That's fantastic advice. That is all we have time for today, folks.

We've run slightly over anyway, but we've had so many fantastic questions. If we didn't get around to answering your question today, don't worry. We will reach out to you. I'll reach out to the team and get, like, confirmation on any specific questions that have been asked in the chat, and we'll get back to you.

As I mentioned, we will be sending out an email tomorrow, which will have today's webinar recording, the slides, all the links to the resources we've had in today's session in there. So keep an eye out for that roundabout this time tomorrow in your inboxes. In the meantime, if you do have any Gearset specific questions, as I mentioned, just head over to gearset dot com. We have a live web chat there.

They can answer all of your specific questions on Gearset. Or if you have any questions for DevOps Launchpad, any of our courses and certifications, if you need any recommendations of where to look for certain information, just pop me an email at team at dev ops launch pad dot com, or you can reply to that email I'll be sending out tomorrow as well with any of your questions.

But that is all we have time for today. A huge, huge thank you to Nadim for presenting today's session and answering all your awesome questions. So thank you so much, Nadim, and thank you everyone for attending. We hope you found it really insightful, and we shall see you again very soon. So thanks so much, everyone, and take care.

Thank you very much. Have a lovely rest of the day.

Thanks, folks. Bye.