CI/CD Pipelines for long term projects

Share with

Description

Discover how you can supercharge your CI/CD pipeline with Gearset’s functionality, supporting slower-moving, longer projects. Long-term projects in Pipelines allow you to use CI/CD over extended periods and prevent long-lasting feature branches from impacting your releases. In this breakout session, Richard Owen, Senior Project Manager at Gearset, and Jonny Harris, Salesforce DevOps Engineer at Zurich Insurance Ltd, showcase:

  • What long-term projects in Pipelines are
  • The pain points that long-term projects in Pipelines resolve
  • How long-term projects in Pipelines work and the benefits they bring to teams
  • A live demo of long-term projects in Pipelines

Learn more:

Relevant videos:

Transcript

Hi, everybody. And thanks so much for coming on to this breakout session on CITV pipelines for long term projects. Please do feel free to drop any questions as you go in the chat. We do have some time outside at the end for q and a. I'll try to get through to many of those questions as possible.

So first, some instructions. I'm Richard. I've been the the team at PM for three years now, and I lead a large teams group focused on making workflows easy and efficient for salespeople to support and sizes, but especially for the largest teams we work with today. And in this role, I've been product manager for our post production policy since its inception, working with over four hundred teams ranging from teams of two to three, up to two teams of more than fifty to streamline and improve their end to end processes.

And I'm obviously delighted to be joined here today by Johnny. So, I've worked with Johnny over the last three years. He's been managing and elevating the Salesforce development process in Zurich, going from admin to developer to developer engineer. And over this time, we've worked together on a bunch of initiatives across gearset pipelines, and it's helped give us a much better understanding of what teams need as they scale and grow to be successful.

So what are we gonna talk about today? So I'll start by giving an overview of pipeline's projects. Where did it originate? What were the pain points that we've seen the whole of the holiday team's experience today? Then I'll highlight how it works and some of the benefits that it brings to Teams. I'll then pass over to Johnny to talk about how he's implemented this, the problems it solves, and we'll see a live example of it in action as well.

So with that, let's get started.

So most Salesforce teams that we work with today have multiple different streams of work, which have different cadences. Some of the small teams may be able to push everything through in their business as usual stream, but that is very much the exception rather than the rule. Most teams have a busy LU stream, and they may also have longer term projects. And these long term projects can last from a few weeks up to a year in duration.

Teams may have multiple projects ongoing at any one time. The most I've seen with any team is about twelve parallel projects.

Only, these projects exist because there's functionality which needs to be held back from going to production until this proves, and then it needs to be released either in slices or as a single floor block, a single product release going straight to production in one go.

So what are the teams what are the challenges these things face today?

So first off, they they need to be able to isolate product work from their business to digital pipeline. Those changes shouldn't be allowed to go to production until they're ready to go. So being able to isolate them, keep them aside, and then move them through when they're ready is really, really important.

Secondly, they need to be able to iterate on existing work. So if you've got a product which lasts several months, they need to be able to work on work which is done earlier in the projects. And because if you're running from Maine, the project changes haven't made it back to main yet. They haven't been released to production.

So we think about this in the context of the pipeline's branching model. So where all features were brought from main or master, if you've got project work, which isn't getting released for a while, and it's just sitting there in your staging or UAT environments, you've created the features. It's the earlier environment. It hasn't gone through domain and on push out to products yet. It won't be there to build on when you're carrying all work in a later sprint when you're creating that next feature branch. Carry that on for several months, and it becomes really, really hard to build on the product release effectively. Plus, you've just got a whole bunch of stuff sitting in early environments, which isn't doing very much.

And then the flip side, this is true as well. If you're working in an isolated project stream, the branch you're working on will get really, really way behind the main branch, and environment just becomes really bad. You're likely to get a whole bunch of merge conflicts when you try and merge that together and release the project, and dealing with that close to a production release is no fun at all.

And then finally, simple PR volume is gonna be an issue here as well. So imagine a six month project with hundreds of PRs in progress. Having those hanging around will get in the way of other PRs. It makes it harder to see, what's in your VA stream. It creates a lot of noise for you. And on top of that, it just starts slowing down in the Git repo that you're working with.

So how do teams handle this before we developed pipeline projects as a solution? So before this bug strategy was developed, teams have two main ways of dealing with long running projects. So firstly, and we see this mostly with smaller teams managing fairly simple projects, they could just load all those changes into a single sandbox we got there, the long term project branch, paired with sandbox, and keep building on top of this, building more and more stuff in there, and then release it when ready. Now this has a couple of main drawbacks.

Most notably, it's hard for the team to manage problems or things that that need reversing if you've got no test environment to validate changes or you'll just got that one box because it's really a one it's a one shot process.

It's also hard to manage any Jira or any other which you're working with associated with this because you need to reassociate them as as soon as you create that PR going towards production.

Alternatively, you could have a second pipeline. It's about the project work, which ends in a branch which is referenced here. So this is similar to the case which we had previously, but instead of just being a single dev sandbox linked to a project branch here, then it that is linked to a separate sub pipeline, which can include other testing environments, other dev time boxes to be able to iterate on our work. And then, once you're ready to release it, you could create that, PR from that project box into your main pipeline and then release it through UAT afterwards production. Now the main drawbacks here, again, you got lack of ticket tracking, so you need to reassociate tickets once you create that PR. But, also, if you're using the reporting API, you're likely to miss metrics on project work because reporting metrics are linked to an individual pipeline, and this doesn't work very well. It's quite a clunky process.

So what was our solution here? So we decided to create first class support for this sub pipeline process, but within a single pipeline. So we've created a new project's objects in pipelines, which has several key benefits. So first off, it allows you to create features from a product branch rather than the main branch.

So that way you can easily. And any changes going to production in the business as usual stream or from another project will get that propagated directly into your project environments. So this way, they stay closer to the BAU. It makes the eventual project release cleaner and easier.

And on top of that, you deal with most conflicts little and often as you go. This allows you to shift left. It prevents those conflicts from being a problem later on closer to production.

And all tickets that you work on, the features and products, are gonna be tracked in the system as you as you're associating them with the features within the products, and then those tickets are associated with the project release that you create. And And this makes tracking changes so much easier, and it ensures that all appropriate tickets are covered as you push a release out. So this is quite similar to the release functionality that we have in pipelines. We're just managing it at earlier stages of the process.

And then finally, the overall PR volume is greatly reduced here. So rather than dealing with hundreds of PRs for for projects, we consolidate them into the project branch, and then we release our project in slices. So this reduces the PR volume. It makes the overhead far lower and easier to deal with. It's much easier to see what's going on in the pipeline as a whole.

And here's what it looks like. So as well as our boxes here on the left and our main setting environments, the the integration, the production, graphics environment, we've got this new project of environment, which shows what branches link to. It shows the number of environments you have contained within that project. It's been one of those. It's one of these super easy, strategies we can show in a few minutes time.

And this can this product can be joined to whichever stage of the pipeline it needs to. So if it needs to go straight in towards production, if you do that, if it needs to go through the next stage or even to integration, it can be done that way as well. And changes going into production through the VAU stream will be back propagated to the product, which will help keep it up to date while you're working on it. So now let's dive into the product itself.

So we here, we can see the product. This looks similar to that multi pipeline case that we're seeing a little bit earlier on, but this is now all within the same pipeline. So it's connected to the UAD branch, as you can see on the right. And then any features which developed in here, you develop a part of the sandbox.

You then push it through, and then those features get collected up back into the project a t branch ready to be released towards production. And this helps you iterate your new existing work. If you've got a long project, you can build up on on top of that without those changes needing to go to prod first.

Then when a project release is ready to go, we can come back out to the main pipeline and see the tickets associated with that project, which have built up. So here's where we can then create a release slice of the projects and start the process of pushing it towards production. So if you highlight the projects, you can see these are all the tickets which are associated with it, and that's where you can create the portal to move it through.

And once we created that, then here's how it looks. So you can name the release. We show you what project it's coming from, and then this has got all the tickets associated with it. You got validation, which is done as a block.

It's a single PR validated as a block, almost like an early release branch. So the project name, along with any name you're gonna release, all tickets are attached. And if you need to release a second slice of that project, you can create an EPR from it, and it'll just contain that extra work that you've done. So creating and releasing of the chicken slices becomes super straightforward.

And with this, I'll now hand over to Johnny to talk a bit more about how he's implemented this at Zurich.

Where the project would be trying to push through, say, something like a new object. And where because the new feature branches were coming from Maine, the project branches wouldn't have those new objects in. So we had constantly just deploy the new objects even though there might have been just a new field being added or a small change. We were constantly deploying objects, which caused quite a lot of pain points for the developers in the team.

And then for for me, pushing a project to go live with with a lot of work to get it back into the BAU chain.

As Richard mentioned, you get come across all the merge conflicts. You got outdated sandboxes. You outdated branches to try and work out what the right stuff is to deploy at the right time.

So our kind of midterm solution, which you can go to the next slide, as as we're working and discussing through, we we we needed to find a way to to make it so when we merged project work to a feature branch, it had all the project work in. So it had it at an earlier point. We needed to make sure that the project, when it was going forward into the BAU chain, it it went in as one deployment rather than lots of individual little bits going spend all my hours and my work day working through merge conflicts to try and make it get into the BAU pipeline. So we want it so it come across in one block nice and easy, nice and clean-cut.

So while those discussions are going on and Richard and team were scrolling ideas away and trying to work on a solution, that what we had to at Zurich to go next time, Richard, we just for the interim, as Richard mentioned, we worked on having a split pipeline. So that means as I've Richard showed earlier, we had a BAU pipeline, and then we had a project pipeline was connected for a sandbox.

And the sandbox and the BAU would have the last branch of the project.

Now it worked to an extent. It it covered off some of the issues, especially around bringing through those objects at an earlier point in time. However, for myself, that meant I had extra pipelines to maintain. So working across zero five orgs, suddenly, you're doubling that up, tripling that up. You know, we're up to ten to potentially ten pipelines at one time, which, you know, it's hard to maintain. There's lots of manual steps to transfer across into the pipeline.

And, also, we're unable to carry Jira stories across. So when you were, you know, one of our projects, you're talking suddenly a hundred Jira stories might be attached. We weren't gonna sit there and manually attach them all again.

So we lose that tracking bit going across.

Go to the next slide.

So once we've got long term projects in there, actually, all those challenges melted away magically as if overnight.

So we now have I'm back to having one single pipeline to manage both the projects and.

So that's now less management for me to try and keep on top of all the different CI jobs and pipelines and keep them in tune with each other.

The project teams can now pull feature branches from their highest environment at an earlier rate.

And, actually, another big win there is we're seeing the benefits of the delta comparison that gear set allows you to do for the Apex changes. So instead of having to deploy the whole Apex class at times, we're seeing just lines being deployed.

So that also then brings in that less risk of moving through those changes. So we're not looking at a massive, you know, fifty line APIP class. We're looking at one line that's changing so we can trust the test. We know it's what's going on. We know it's changing.

And we're facing less merge conflicts.

And, actually, touch wood, since we've had long term projects in for the last project, I only think I saw one single merge conflict, which is a massive win for me.

Again, it's less focused on fixing merge conflicts and more focused on iterating and improving our processes.

And, also, just that last little win as well, the Jira stories are now being carried over. So it means now we we're actually tracking the Jira story all the way through end to end, and we can keep an update with the business then on where that story is at all times.

Okay. So what I'll do now, I'll just share my screen.

I'll just walk you through the kind of what we've got set up currently is Zurich.

Just in a pipeline.

Cool. Richard, just checking and see my screen.

Yep. Because you're fine.

Cool. Great. Okay. So this is as you saw on Richard's slide, this is how it looks at the moment, and this is what we're utilizing at Zurich. So what we've got here is just a standard BAU pipeline that's working through just free environments. We got our dev environment into our UAT environment into our production.

And off the side here, you can see our project work.

So if I just click through into the project, you'll see here that I've got a currently UX dev environment, and it's got one of its own project environments inside. So this is its own sandbox that it works on.

Against his sandbox, bit of blue Peter magic. I've got a change ready to go. So what I'll do is I'll promote this change in.

And what this will do now, deploy it to my branch, guess at summit, deploy it to the org.

And what that will allow me to do is off the back of that, pull through that change into the BAE pipeline.

So say if you had five or six changes here, you deploy them all into the UAT environment of your project, you get testing you all the sign off.

As Richard mentioned earlier, that is one and one PR into the BAE pipeline.

Now go across a little refresh.

And What you'll see then is that project work ready to be merged into the BAU.

Where's she there?

Do you see if now project's going through? It's running some checks. We've got some approvals to be ticked. That'll then pull through those changes.

And what is really great about the project work as well is just how easy it is to spin that up. So, again, in your normal pipeline, all it is is go up to ads. It's just a great project.

And from here, you can give it your project name, and then you can either select the branch or create a new one. And, again, the great thing here is coming from main seat. You're keeping that common ancestor throughout the whole of the pipeline, so it helps with those merge conflicts.

Yes. Come to create that branch.

And then it's just case of create project, and it's as easy as that. So now that that project shell has been created, ready to add in your CI jobs like you normally would in a in a pipeline.

That's already a go.

Richard, back to you.

Oh, well, thanks very much. Thanks very much, honey. And, you've got about about ten minutes left or so in the breakout session here. So, happy to open the floor for but, if you got any questions about this, or any anything about, about projects, please do drop it in the chat.

Let us know.

In the meantime so, Johnny, how many how many year products do you anticipate using this for over the course of the next few months, over the course of the year?

How many how many parallel do you have?

So for us at Zurich, I imagine what what we've got at the moment, like like you see here, we've we've got our normal BAU chain of environments.

And, actually, if what we're gonna start to do is apart from outside of incidents, we're then gonna have a project for each of the changes going through. So they'll be kept separate until they are ready to come as the BA pipeline. And then what that allows us to do is to keep a direct route to production for quick fixes, for hot fixes, and for those incident changes without being clogged up by project work, which might sit halfway through and then stop it and stop moving forward.

So I'd imagine us having well, we could end up having five or six within one org at one time, to be able to manage those changes as they go through. And and, like you said earlier, have those slices going in, when they're ready, when they're tested, when they're signed off rather than waiting the six months to a product to an end in one big bang to try and iterate in and keep those full chunks going through.

Yep. That makes sense.

I see a question coming in from, from Dylan. Yes. So, do you feel as though this leads to big bang releases versus iterative updates?

Tony, what's your what's your view on that?

Yeah. So as I just said, I think you you can do both ways. Right? Because I know that in some industries, small literature changes is easier to do. In others, like insurance, sometimes those issues changes are harder because of the sign offs you need. You bit more red tape around it. So but both for Boswell with the pipelines long term pipelines.

I've I've done one project where it was smaller changes going through at a quicker rate, and we've done one where it's a big bang going live. So both work just as well as each other.

Yeah. That makes sense. I think the goal is to try and make those big bang releases where they are necessary as smooth and as pain as possible to try and reduce the drift that you have so that you're just dealing with that block of work as it as it is. And it's one of the one of the interesting things that we've seen as we're dealing as we're as we're working with a lot of larger teams now is that best practice stops being so monolithic.

Like, what quite often when we're dealing with we're working with teams of, like, five to ten, it was quite easy to say, like, this is what good looks like. But as you do with a higher volume of changes going through, they all need to go into the common environment, slightly going into integration, going into UAT. There is a bottleneck which can come out if you're dealing with individual features. And so being able to consolidate elements into a small to a smaller number of elements, which you can validate in blocks more effectively, They may not need to stay there for a long time. But if if you can push those through, effectively and quickly, then it'll it'll put you in a much better state in terms of being able to get those changes to production as easily as possible and speeding things up overall.

Just just very quickly, Richard. Sorry. Yes. I was gonna say, just as a point of proof for that, that change is now merged into that environment, into that branch. I can now create my second release that will go alongside this the first deployment going in. So you can see how that you can cut up those deployments into bigger section or smaller chunks to move through.

Yeah. That's cool.

And then, of course, if you did create multiple chunks of this and then have bring it to production, they can also be incorporated into a pipeline's release in the same way as you would, a group of features if they needed to go that way.

Then we got a question through from Vijay, here saying, how can hotfixes be handled in long term pipeline scenarios? As in fact, that's a very relevant question. It's one that we've been, we've been thinking about at the moment.

Johnny, how are you dealing with the have you had to hotfix anything which is coming out of a project so far, or has it so far been not not a case that you would you'd be looking at?

We we haven't had the case for that yet, just because we have the the testing in place. Like I said earlier, that stops the project going forward until certain testing is met and require and there's certain requirement.

But Mhmm.

You know, for for for me using it, if you got a hotfix that come from a project that you need to get back into that project pipeline, then going through the project itself, you can move it for a quicker rate. So it's into that end environment and pull it through into especially if there's nothing if there's nothing else in that higher branch, then you could just pull through that hotfix into BAU through to the production to fix that issue.

But as I said earlier, what we're trying to do is keep that BAU pipelines as free as possible so that a hotfix can go through there. And then you just once it goes past the pipe the projects, you can back propagate it into the project and go back down inside the project shell itself. So there's, in my eyes, there's multiple ways to do it, that both come to the same outcome.

Yeah. I would I would agree with that. I think that's certainly certainly possible to be able to to create changes to the through the BAU workflow to be able to create hotfixes. It would also be possible, and this is something which we're looking as a future enhancement here, to be able to more easily expose the ability to create a hotfix directly off a project release branch.

So this would be similar to what you might do in a kind of more of a Git flow based workflow. We've created a release, and then you can create a hotfix off that to then merge back into it to be able to hotfix that up. That's something which can be done manually today. It's something which we're looking to add improved support for in later releases of this of this functionality.

And in fact, for some teams who are who have started adopting, long term projects effectively as a way of implementing more of a Gitflow based workflow, that is what they're doing today. So it's a very relevant question. It's one that we're thinking about how to present the best user experience. You're right.

It is the the approach is very similar to Gitflow, and it's effectively so the the choice of about which branching strategy to adopt effectively it it's determined by what you value most most in terms of the process. Because if you want to maintain feature independence as much as possible, then keeping on branching from your master branch and being able to to to put put that through is is is the most effective way of doing it. If if you wanna be able to, run from that earlier place, be able to create to do the project from also, like, a developer environment and then bring that back in and then push like, create release branch off that, then that's effectively the gameplay workflow.

It's very similar to what we're doing here in projects, and that that workflow can work for a bunch of teams as well. And we're intending to enhance this to to bring to allow better support for that.

Address Salesforce development integration with SAP development taking place in different swim lanes with codependencies.

Now one we may have to take away because I'm not sure I fully get the context of the question, in terms of it. If you can put a few more details about what SAP developments is referring to there, then we can, Yeah. So it depends on the on how that how that's linked in with the dependencies.

Typically, it's typically, at this point, it's something which you'd need to which I think you need to manage side by side, in in terms of the in terms of being able to to make sure those changes go through effectively. But, maybe that's one which you need to take away and and, I think I'm back I'm back to you with an answer.

Then we got a couple minutes. We got time for maybe one more question. Yes. Just one now.

Okay.

We'll we'll we'll, we'll still be on here for another couple of minutes until we get to the end of the session. But, otherwise, just wanna say, yeah, thank you very much, Johnny, for joining me today. Been, obviously, a pleasure. Great to great to see see this in action, and then you'll be able to go, everybody here. Thank you for moving on to this breakout session. We can head back to the main room ready for the ready for the panel to start.

Thank you.