Future-proof your DevOps strategy with testing and observability

Share with

Description

This live webinar covers best practices for implementing testing and introduces observability principles to help you monitor deployments and delivery.

Learn more:

Transcript

So I think if everyone's ready to get started, Andy, I shall hand over to you.

Well, thank you very much, Amy. Welcome along, everybody. It's great to have you here, and it's great to see all the interest in testing and observability.

What we're gonna do over the course of this webinar is to explore these concepts with a view to sort of giving you guidance on actions that you can take to increase the resilience of your orgs and applications.

Before we dive into that content, though, it's probably worth just taking a step back and ensuring that we understand the concepts at hand here to make sure that we're all on the same page. Now testing is probably the easiest one. The easier of the two, at least. This should be fairly fairly common, knowledge, I suspect.

Just to make sure. So testing, we've obviously defined this as the in ensuring that the application works as it's specified. Great great testing will provide early feedback and visibility of any variations in the application from how it should be expected to work. We can, of course, automate testing.

We can do manual testing. We can do both at the same time in certain cases.

And it's definitely a form of QA. I think this is quite important. It's sometimes those terms are interchangeable. Right? It's QA.

Quality, I think, is probably at a something that's the responsibility across the team. Right? I think, it's not difficult to imagine building something that works perfectly well, but actually doesn't work at all. Yeah. It doesn't do what the users want it to, as much as you can imagine something that doesn't work despite what the users want it to do.

Observability, on the other hand, is probably less of a common commonly encountered concept on Salesforce at least.

If you come into Salesforce from different development environments and practices, then you may have encountered it. But in case you haven't, let's define it with a bit of an example, I guess. Something like a dashboard on a car, if you're familiar with that. You get warning lights.

You get gauges. It show you what's going on. It gives you advanced warning of things that might not be in a condition that's that's perfect. Right?

You don't wanna drive a car until ultimately it just stops one day, and nobody knows what the problem is. You wanna get ahead of those problems and get repairs in when you need to. And this is really the the principle of observability.

You get a view into the state of the workings that you don't that you wouldn't get necessarily from viewing the app the application or the car from outside.

So observability is, to a a large degree, proactive. That's the that's the that's the theory there. There's always gonna be reactive elements if we think of things like government limit exceptions, which, you know, we'll talk about as we go on. But, ultimately, the aim is to get ahead of these critical business standstill issues.

Problem is, though, there's very little information out of the box on Salesforce historically.

What there is is scattered across different UIs and APIs, and it's very difficult to get a a single comprehensive view of the state of the application. And the data that is there doesn't really give you that comprehensive view. It gives you information to corners of it, but not that full dashboard view.

So with that in mind, let's dive into the topics at hand, and we'll take a trip through testing into observability.

And we'll start with what I'm calling here undifferentiated testing, which is, you know, may may not, become immediately obvious what's going on, but let's let's explore that a bit and start by having a look at this thing, the testing triangle.

If you search for it, you might see it referred to as testing pyramid, but, I'm quite a fan of, Euclidean rigor. It's it's very definitely a a triangle and not a pyramid. Anyway, it's a it's a well known concept in software engineering. The the contents are potentially a little bit dynamic.

You'll very always very invariably see unit tests at the bottom, some form of integration tests above that. As you go up, maybe the terms change a little bit, end to end test you might see. This example here has got UAT in a cloud or or beta testing up in a cloud at the top, as you can see, suggesting it's it's, optional. We'll dig into that a little bit more.

But despite the regardless of the terminology, the principle remains the same. And that's as your test complexity and scope increase as you go up the pyramid, the quantity of those tests should decrease.

If we overlay some Salesforce some typical Salesforce org structure onto the pyramid, I guess the point to take away here is that there's not typically a sort of one to one testing phase to org pairing.

Unit tests, yeah, they're responsibility of developers, but you might want to execute the unit tests such as they are in the integrated environment as well. You you are UAT testing, if there's a UAT org, you might have automated testing. You might have manual testing. On the right hand side there, we've got the type of, language or the what facilitates the the execution of those tests, let's say.

So, certainly, at unit test level JavaScript, unit test, Jest test for Lightning Web Components, Apex. You can write integration tests in Apex. But as you go up towards the bigger end to end tests, Apex can't cope with this sort of thing. The question marks are there simply because you've got a choice here.

You might use a UI testing platform. You might use People. You might write your own Selenium based web tests.

Ultimately, though, the point here is that each band is a different type of testing.

Undifferentiated testing, which was the the the phrase we said at the start that we want to get away from, would would be something like testing the same acceptance criteria on the story throughout each of those orgs. That's not really adding value as you go through.

So how do we get to a point where we can we can create these different types of tests, and what what are they? Well, I think a fundamental thing to to get your head around here is something called the single responsibility principle. Again, it's a it's a standard object oriented programming concept.

There's nothing magic about it in terms of Salesforce.

But if we if we take an example of something like an opportunity discount function, your business processes ultimately will be composed of these individual actions.

At unit testing level, you might think of it if we keep that car analogy like a light bulb that you used to put into the into the light fittings, you know, the the brake lights or the headlights. You can test the bulb quite happily outside the the the car. You don't have to plug it in and press the pedals or turn the lights on within the car to know if you're the manufacturer of the bulbs.

So the unit testing is very similar here. If we've got an op if we've got a function to discount an opportunity, let's disconnect that from the business process because we may there may be multiple business processes where that needs to happen. What what we want to test here is that this can cope with almost anything that gets thrown at it. We would test this with multiple opportunities, one opportunity.

An empty list, null, a discount percentage that's positive, one that's negative, zero percent, greater than a hundred. How should this function work under all those conditions? We don't necessarily know at this point what the business process now or three years in the future is going to be, but we want to be certain that that component works as we specify it. So there are your unit tests.

As we step up the triangle, though, if we're getting towards the integration tests, ultimately, your business processes are going to be a combination of these functions put together in the sync in into a specific process.

Carrying on that example from before, if we think about adding an opportunity line item, might trigger some sort of discount potentially on an opportunity.

But that's a multistage function. We might need to decrement some stock because we put the opportune the item into the opportunity line item. We maybe want to check if the discount applies. If it does, apply it, and equally then maybe send a notification to the customer. So there's multiple processes. And we want to test that these functions actually work together in this this cohesive unit, especially where the output of one becomes the input for the next stage.

That's that's where we get into the integration step, and that's what integration's getting at. But we can't do that if we don't have these separate functions in the first place.

If we go on a level further towards the end to end test, which might be might be referred to as UI test, This is where you're chaining a number of those individual business processes put together into a larger life cycle.

Again, sticking with the example, this might be the opportunity the whole opportunity life cycle.

Creation of the opportunity, addition of the items, the quote generation acceptance, opportunity closure, you know, with that original discount function is now just simply one part of one stage.

These are operations that in reality are gonna take potentially months to achieve end to end. So it's not possible to do this in Apex. You just don't have the scope within the boundaries of a single Apex transaction to set all this data up and execute all these processes, which is why we're coming out of Apex here and potentially using UI driven tests whether they're automated or manual.

But this is very much up at the top end. This is the UAT space. These are the processes ultimately that the users are working through on a daily basis.

You're not necessarily, at this point, testing the acceptance criteria on a feature if you compare that back to the unit test or maybe the integration test.

So we started off with undifferentiated testing as the thing we wanted to avoid. So what is differentiated testing then? Well, how do you how should we achieve that? Well, the critical points are to understand the different types of testing. First of all, the bands of the testing triangle or pyramid to define what's needed for each item, to understand what sort of test that means for your application.

Then you can start to identify your business processes and gradually introduce that composability if that's not something that you have at the moment. It's very difficult to say you're gonna come to a complete business standstill and spend three months breaking the application apart and putting it back together in in a composable manner. Then this probably has to be worked on iteratively opportunities opportunistically Then since you're working in a particular area, see if the refraction can be done, or potentially assign a set amount of each development iteration potentially for this sort of work.

The next point would be to think about shifting UAT left.

If we go back to that triangle that we saw at the start where we had UAT as a sort of bubble cloud at the top of it, what was that all about?

It's it's potentially a bit controversial in some scenarios to suggest that UAT is optional.

But in a world of continuous delivery, u a UAT, as we understand it, in a lot of cases, could be viewed as a bit of an anti pan because this is at a point where a lot of work can slow down and stop, and you can get into quite a a long lengthy cycle of continuous refinement of features. And that's not what continuous delivery is about.

So what are the problems with UAT? Well, it's there's nothing fundamentally wrong, but there's certain types of UAT or certain forms of it that do become problematic.

Unstructured UAT. What do we mean by that? Well, if it's executed without a clear set of test goals, it can become opinionated.

You know, sometimes this is what you want. Ultimately, these features, when we go back to the triangle, this is the top. These are the end to end business processes. This is what the users are doing. So you need their feedback. You need their rubber stamp that this is actually gonna work for them, of course. But are you getting a collated shared opinion across the user base, or you're just getting individual opinions from whoever seems to be testing it at the time?

UAT, if you get into that sort of cycle, it it can block work or change priorities.

It's a really late stage if UAT is very close to production to be changing changing functionality.

There's a risk of regressions, unexpected consequences of doing that. Of course, the opportunity cost of the development that you've already done could have been best spent better spent doing something else instead or building it right the first time, let's say. So where UAT becomes this moving target, delay and release and integration that can keep branches open longer. We've mentioned in previous webinars how that particular activity is something that can really, impact on the, quantity of merge conflicts and your ability to deliver.

So it can become problematic. Can we avoid it? Well, in a lot of cases, potentially, we can. We don't necessarily want to eliminate UAT because it's part of the testing triangle as we saw. But how can we make it more efficient?

While these sorts of steps definitely have have proven successful for in a lot of cases, really, the fundamental point is that the team should be well versed in the application that they're building, being able to challenge requirements.

Users are great at providing solutions, maybe not always the requirements.

Make sure that whoever's responsible for delivering the functionality of the team are empowered to challenge the requirements to get to the ultimately, to the job to be done, as we call it.

Therefore, you can actually build these requirements in. If you've gone to that previous stage and worked out the various testing layers and you've got into composability, then you can get to a point where for any given set of changes, you can really define in advance what needs to happen in each of those layers of the testing triangle. What does this mean for the UAT test? What does this mean for the integration test?

If you don't have that domain knowledge across the team, persevere.

Do I've worked in the very first job I had in Salesforce. We were all new to Salesforce. We were all new to the product. We knew nothing about it, and we couldn't do this.

We were terrible at it. Our estimates were all all off, and we didn't know the product whatsoever. But we kept doing it, and we got better at it. And we were great at delivering by the end.

But if we'd given up and stopped because we weren't sure of it, we'd everything would have definitely been far less efficient overall.

We keep that regular release cadence up as well. Don't get blocked in UAT.

If we cross over from the testing concepts in towards, observability now, we'll go on a little journey backwards, I think, here from reactivity towards proactivity, which is what we said observability ultimately is. There is nothing more reactive than unhandled exceptions.

Things like governor limits, as we said at the start, are unhandleable. So these are always gonna be a feature. These are always gonna be possible no matter how proficient your application is at catching and handling exceptions. So we need to work out what they are and what we're gonna do with them.

You may well be very familiar with something like this. This is the sort of email that you get sent when some Apex encounters a non core exception.

Similarly, now these days, if the same happens with the flow, you'll get something like this. There's more at the bottom about the specifics.

But these are pretty you know, these aren't great as a starting point. They're a starting point for sure, but this is this is what we get from Salesforce.

So when you get one of these emails, if you get one, what what's this saying? Well, ultimately, something is not completed properly. You know? There needs to be this needs two parallel ideally parallel activities.

There's got to be of this. There's maybe bad data in the database, incomplete process. There's a user at least sat there saying or an asynchronous process potentially sat there saying, I haven't been able to do what I wanted to. And then ultimately, there's potentially a fix that needs to be applied to stop this happening again.

Maybe it is a book. Maybe the business process needs changing. Who knows? But, also, Salesforce doesn't really know what the process your error handling process is, and it shouldn't really be able to either.

It's the best it can do is send us this email to who it thinks might be interested parties.

So that's a way of doing it, but it doesn't really feel like the best possible way. What might be better? Well, this is where things start getting exciting.

We're starting to push at the edge. We're starting to push even beyond the edge of what Salesforce is doing today. This is why we're talking in this webinar really about longer term things for twenty twenty five, for example. There's not necessarily a quick fix solution because the platform functionality is exactly that, but it feels maybe a little bit inadequate, especially in twenty twenty five.

So if we think about the sort of experience you might want in this scenario, well, we wanna get the information to the right people rather than just the last person who modified the, component, which is where the email goes. If that's some generic service account who's responsible for deployments, that's not great. It might not even be a monitored inbox. So get the right information to the right people.

You want better interpretation of the reported problems. Big stack traces or emails with lots and lots of technical data and maybe aren't really the best starting point.

If you had access to potentially the information around things like components that are changing recent deployments, that's an interesting way of being able to layer on intelligence and look at potentially tying back the error to changes in that recent component.

And, ultimately, quick access to recent changes in the effective area, as we said. That's such that that would be such great intelligence to be able to have when you're trying to get to the root cause of an issue.

There there's a bit of a lack of solutions in this area, to be honest, at the moment. The the call to action is more about getting familiar with your current usage, your consumption of these emails. Are you getting them? Who's getting them? What are your processes when those occur? Once you know what those are, then you can look to improve what you've got, whether that's internally or using solutions that may come onto the market.

Data is a huge part of observability.

Login is your key to to all of observability, ultimately. This is this cannot be underestimated. Let's have a look at what this means.

These are probably familiar to you. The Salesforce debug logs, they're not, they do a job. Like the emails, They're they're there for a purpose.

But, you know, if if a cost if a user says they've encountered some sort of issue in production, then what's your what's your process from then? Well, you've got to you've got to try and replicate it. Switch on the logs, see if they can replicate it, or try and replicate it in an org.

Is the org high enough fidelity product to production for you to be able to replicate it? The orgs themselves, they're you've gotta switch them on. There's limits to them. They're, you know, complex to to pour through.

You know? They're difficult to reverse engineer when there's asynchronous stuff going on. You know? The the the log is, chronological rather than related to transactions.

So whilst the information's there, it's not necessarily the easiest to find.

Now the community have gone out and provided layers more intelligence on that. This is one example of it. This is Nebula Logger. There are there are certainly other options. So but it's the principles really that are important here.

So with a solution like Nebula, you're in control of what's being logged. You can set the data. You can choose exactly what gets logged and what doesn't get logged. You can define the login levels, as you can see on the right hand side there, the very familiar ones that we're used to from Salesforce.

You can log variable information, limits information, exceptions. It's not just error logging. This is all data that you can drive and get a view through any particular transaction. And, ultimately, this ends up as data in the org. So you can use Salesforce functionality. We can re create reports, dashboards, process automations. It's just great visibility.

You can re you can parent and child logs. You can connect them together so that child asynchronous jobs from a parent transaction can all be seen together.

It's a really powerful way of being able to capture information about everything that's going on across your transactions.

So what does this mean in terms of observability?

Well, first of all, when we think about that original example about the user who's having the issue in production, the experience now is completely different. Right? You can just go in and have a look at the data. You can see what they did. You don't have to go switch logs or get somebody to replicate it all. It's all there already. You can build process automations.

They get the notifications to the right people at the right time.

You know, you can, benchmark test. So if you've gone into that composability model and define these business processes, you can actually start locking out the limits, the transaction limits at the start and at the end of these processes and work out the delta. And you can say that to discount a bucket of fifteen opportunities took one point seven seconds, and this is known data. And you can start to track this and trend it and see if things see if changes are increasing the time, reducing the time.

The future is, once you have access to this data, the limit of it is your imagination, ultimately.

And with this, we move on to the last point, which is sort of prevention, which is where we started out by talking at the goal of observability being ultimately prevent prevention getting ahead of these issues. So let's say you've got data, you've got a composed application with known business processes, your login data, you've got all of that set up. Well, at this point now, as we mentioned, you can do these measured execution times for common business processes. Trend analysis, you can see very quickly what, impact any changes made if you've refracted something. Has execution time gone up, down? Are you using more SOQL, less SOQL?

Are you getting to a threshold that you need to be worried about? The old dashboard, is there a warning light on? Because now we're using ninety SOQL in a particular business process.

The the worst time to find out that you're using a hundred and one is when you're using a hundred and one. If you knew when you were using ninety, then that's great information to have. Performance testing is very easy to do because you've got these separate measured components.

Nonfunctional requirements can come in. You can have barriers about how much you expect your application to be able to process.

You can create dashboards showing this. You've then got the ability to have this single place where you've got that holistic view of the dashboard, as we said. You know, not just transaction limits, as we mentioned, your org wide limits. Are are we approaching storage limits, a p those twenty four hour limits of API calls, platform events?

You can even break it down by process and how much they've used if you got a particular operation that's very expensive and consuming a lot of these. It's it's it's incontrovertible tangible evidence that can drive optimization work. You can't just say make this faster. You can say this takes this long. We need to cut it down. You've got the evidence there.

Smart exception handling. This is really great. If you think of a payment gateway, let's say, if users are putting in or aren't putting in the right information, there's the exceptions are being thrown from it.

You don't need to do much about that in terms of notification flagging. The users will just put the right information in. If the payment gateway is down for five seconds and the next operation is fine again, then maybe that's a little bit more important. There's an ephemeral issue with it.

Maybe you need to keep an eye on that. If it's been down for five, ten minutes or something, you can now take a a dedicated action to go and tell whoever needs to know about this that the payment gateway seems to be down. Now the users may well tell themselves, but, actually, you can get ahead of all of this. There was unhandled exception emails about the errors if the exception wasn't being handled.

I'll just go into some unmonitored inbox.

So to wrap this all up in summary, what are the key points across those two topics? Well, only run the tests which are needed. There's a lot involved in that, as we said, about the composability and understanding the business process. But, ultimately, remember that testing triangle. Each layer should be a different set of tests, fewer in scope and greater in scope, fewer in number as you go up, but to make sure that it's production worthy.

Build UAT in. See if you can bring any of those user requirements or as many as possible into the original development work, which is much easier once you've got that composable application.

Make sure you've got a process around these unhandled exceptions.

Whatever it is at the moment, get on top of it because this is important information that's maybe going going astray because of the the weird email process that Salesforce implement.

We I hopefully, we've demonstrated fully how much visibility and potential logging data brings you. We didn't really talk about the, the implementation effort. That's undeniable. There is some but the benefits on the other side of doing that should be huge. And the bottom line, I think the key to all of observability is don't wait for your application to tell you that it's failed fallen over.

Keep an eye on it and make it tell you when it's approaching that stage. You'll have a much more serene experience, I think, with your application if that's the case.

So thank you very much for your attention and your attendance. I'd like to point out the, the QR code. If you attempt to, have a look at what Gearset offers in testing, observability, or Salesforce DevOps as a whole, QR code will give you take you to a page where you can sign up for a fully featured thirty day free trial. We'll keep that up on the screen, but, that's the end of my, content. So I shall hand back over to you, Amy.

Thanks so much, Andy. Awesome. So we do have a couple minutes, to squeeze in a few questions.

So if anyone wants to pop any more in the q and a or chat, you've got a couple minutes to do that too. We do have one here that's come up from Sam. And Sam has written, my code isn't necessarily broken down into functions and business processes.

How do I go about creating all the separate functions to unit test? And I think Sam's referring to one of your slides at the beginning there where you had a nice image of all the different functions.

Oh, yes. Yes. Okay. Well, that's interesting because that's, it's very tempting to go into the code first for that sort of thing, but that's it's not not really how it should be done. I think the understanding of the business processes is critical. It's almost you've got this big block. Let's say you've got this big mass of an application that's quite, unstructured and everything sort of talks to each other.

Really get it down into the sort of major chunks first, maybe three or four separate business lines of business that may be going on. Then you can look at the specific types of activity that are going on within there, and you can keep this iteration going smaller and smaller and smaller until you get down to these functional items. And then once they're identified, the creation of them and the validation of them via unit test, you should find much easier than trying to tackle this huge mountain of an application to start with.

Perfect. Yes. Great advice. And we have one time time for one more question.

And this one is, what are the problems with the unhandled exception emails?

Okay. Yes. So we alluded to them a little bit there. So the the way that Salesforce defines who they go to is is a little bit sketchy. It's either the person who last modified the, the component, or it can be, the, a a set of users who are defined in a in a configuration in the setup menu.

People go, you know, people come and go through organizations.

Are these in boxes being monitored?

Is that information appropriate to the people who are getting it?

There's lots of I think the time it takes from that email being sent to this work that's needed starting, can be short circuited a lot by getting the right information to the right people. I think they're okay, but there's more that could be done.

That is all we have time for today. So thanks so much to all of our attendees for joining.

If if as I say, if we didn't get round to answering your question today, you can reach out to us, via the gear, live chat on gearset dot com.

Or if you're a customer, you can reach out to us in the app as well.

And I will be sending out an email tomorrow with today's recording, some bonus resources, lots of good stuff for you to learn more about testing and observability.

And you can reply to that email, as I say, with any feedback from today's webinar or any questions you have, and we'll get back to you there.

So, yeah, thanks again, Andy, for presenting such a fantastic session. We hope everyone got some really great advice from this, and we hope to see you all again soon. Thanks so much, everyone.

Thank you.