Episode 40
· 59:39
Richard Bradshaw (00:01)
Hello everybody and welcome to the Vernon Richard show.
Vernon (00:05)
Yay.
Richard Bradshaw (00:07)
How are you then?
Vernon (00:10)
my my goodness. Well I I well relevant to this this conversation we're about to have. ⁓ there's just lots going on man. Just there's so much for trying to figure out how to do. Where should the focus be? What even is this job anymore? It's just I feel like I am fr frankly, I feel like I'm constantly failing. ⁓ yeah, but ⁓
But I've got a smile on my face, which is interesting. It's an interesting kind of ⁓ it's an interesting experience to be going through, to be honest. It's definitely it feels like
High stakes, it feels stressful, intense, important. And there's this element of this is fun. Like this is challenging in a good way. ⁓ yeah, so that's where I'm at. What about you, man? How are you feeling?
Richard Bradshaw (01:09)
I feel the same. I've just come back from HusDef, a testing conference in Budapest, Hungary. Absolutely fantastic, but so many conversations, so much chat, so many ideas. And then I've gone back to work where I've got a new job. I don't think people know that since we started this, but I've got a new job. Building a platform that you can build agents on. So it's very meta right now.
Vernon (01:13)
yes.
yeah. Yay.
Right. Yep.
Richard Bradshaw (01:35)
So yeah, I'm the bit same as you. There's so many things from the past, but also it's so exciting about what's going on. But this episode, we have a guest. We mentioned about having guests, and an opportunity came up. So we thought, you know what? Let's experiment and have a guest. And our guest, talking about hosteth, is this chap who some of you will know. It's Chris Armstrong. Hello, Chris.
Vernon (01:45)
We do. First time ever.
Yay.
Chris Armstrong (01:57)
Hello friends, how are you?
Richard Bradshaw (01:59)
We are good. Very, very good. So.
Vernon (02:01)
I feel I feel like this is bringing balance to the force and equilibrium 'cause I'm constantly invading Christopher's ⁓ podcast called Testing Peers. If you're not listen a listener, please go and listen. so this is this is nice.
Richard Bradshaw (02:16)
So we don't want to get too much into football but Chris is also a football person so we're just going to keep it really simple right now. Thumbs up or thumbs down for how you're currently feeling about football.
Vernon and Richard his thumbs down and Chris his obviously thumbs up because he is an Arsenal fan and that is the end of that. Right, okay. So ⁓ Chris, you've just been to Husteth as well. How was Husteth for you?
Vernon (02:36)
Cambridge, Cambridge.
Chris Armstrong (02:40)
incredibly tiring, my voice has only just recovered, but it was a really exciting event to meet a lot of people that we see online in one place and I think for me, know, we'd be talking about AI stuff.
People aren't talking about AI as like this weird little thing that we now need to become friends with anymore. They're now saying, we've got a job to do, we've got things to do, we're being much more proactive. And it felt a lot more invigorating than maybe what we saw in the previous couple of years, when people are a bit like, no, you can't replace my job and you can't do this. And people are really much more proactive and that feels like we've got a better foundation to move forward than we did before.
Richard Bradshaw (03:26)
think I feel the same. and Verne were just having a chat during the intro and I almost felt like some people very much like AI is here, it's coming. And then a few people were like, yeah, you know, it's kind of on the edge. Maybe I'll dabble, maybe I won't. But like it really is here now. Like it's very hard to have any talk without mentioning AI at all. And I think there's levels to it, but it is now here and now it has become, how do we thrive? Like we've always found a way to thrive.
How do we now thrive? What needs to happen? How do we keep software quality high and working while this change is ongoing and taking what we can from the past, but also adapting now and evolving and changing to feel comfortable in this space because there is a need for testing. Obviously, there's never not a need, right? But that things are changing, especially the way we do it.
right, Chris, what do you do? What's your job, mate?
Chris Armstrong (04:29)
What do I do? Yeah, I'm the Developer Relations Manager at SmartBear.
Vernon (04:31)
Introduced itself.
Richard Bradshaw (04:36)
A what now? A who, where, what? That doesn't sound like a testing job. What are you doing on my podcast? ⁓
Vernon (04:40)
Does that even mean?
Chris Armstrong (04:41)
Well, so there's a couple things, right? So what is SmartBear and what is Developer Relations? Which one do want to start with?
Richard Bradshaw (04:49)
Let's start with SmartBear.
Chris Armstrong (04:51)
So, SmartBear have been a maker of tools since about 2008, and currently our portfolio sits at around 25. We have tools that are used for API design, API test, we have open source, and we have commercial. We also have many testing tools from your classic automation tests, visual tests, load ninjas, soap UI.
We have test management solutions and then we have things that as you start to think about a world of completely autonomous and completely human led, we have kind of a spectrum of the tools that go all the way from design through to observability.
Richard Bradshaw (05:39)
find it interesting, obviously I don't know if either of you read this, but there's a big thread yesterday on LinkedIn about tools versus automation. So you just said 25 tools and this whole thread was taking basically saying AI is not a tool, right? AI is automation. And I've obviously done a lot of work in this space, but I've never really made the connection this person was making. But in summary, they were like, a tool is something you wield, right? It's something that you're in control of.
Chris Armstrong (05:49)
Hmm.
Richard Bradshaw (06:08)
And then there's automation, which is kind of doing the job for you where you don't need to be there. And I found it really good distinction. I'll find the link for you. But like I know a lot of the tools in SmartBear's case used to be tools that you would sit and use. Like you would be the one using, but I've also known in recent times, a lot of these tools are becoming more what this person would lean into the automation space where they're doing stuff without you having to.
prompt it or trigger it or do anything. It's sat there just running. And I found that interesting. The distinction between like an LLM is not a tool you use. It goes off and does a task for you and brings it back. Whereas a tool is something normally you're constantly like you're holding as you're doing it. But anyway, I'm not sure if that's a useful distinction, but it's definitely worth reading. No one else has read it.
Vernon (06:59)
I'm gonna go look into that. Sounds interesting.
Richard Bradshaw (07:02)
Yeah, it was this, I think it's just that we use it, the word like wield and that got me thinking of some of the tools I've used in the past, even SmartBear's tools. know, were at post-death last week, we were talking about soap UI and ready API and these other tools that I've used a long time ago and they, they're hands on tools, right? They're tools that you're using as you're doing your task. Like they weren't so much things that you would like be doing stuff on your behalf. Like.
without you being in front of it. Whereas now I know there's things like BearQ, which is doing things on your behalf when you're not told it to. It's set up and it will go, I think there's something for me to do. Let's go off and do it. So it's interesting just that distinction. I'm finding it interesting. It's got me thinking anyway. Yeah. What about your role then, Chris? So like, I know you've got into DevRel and I think me and Vern have dabbled in
Chris Armstrong (07:50)
You
Richard Bradshaw (08:00)
DevRel, not official roles I guess, but like, no, you don't feel like you have. I feel like in some of my previous roles I've dabbled in that kind of domain, but never official title.
Vernon (08:04)
Absolutely not.
No no I'm sorry.
Yeah. Yeah. No, I d yeah, I was agreeing with you. I d I definitely feel like I've dabbled, but I definitely don't feel like an actual DevRel. But you're a DevRel manager on top. So it's so yeah, that's gonna need some explaining, I think, my friend.
Richard Bradshaw (08:25)
Oof.
Chris Armstrong (08:28)
Well, the manager part is we were in a team, we didn't have a leader and they said, you're basically doing the job already, Chris, would you like to take on more responsibility? And I said, cool, yeah, I like working with these people. They've given me their thumbs up, let's go for it. That's nice and easy, lovely to get a promotion. As for DevRel, I mean, it isn't identical everywhere you go, but to give a really high level sort of explanation for DevRel.
I kind of look at it as bi-directional consulting. So you spend a lot of time out with the community at events like Herstef in webinars and podcasts and speaking to really good humans in really cool spaces and you sort of learn what's going on in the world and then you go back to your product teams, your engineering teams and you go, this is what's happening, we should be making.
solutions that fit these requirements, these needs. We speak to salespeople to make sure that they understand that you have to go a bit deeper than the surface buzzwords. And we speak to marketing people to say, hey, this would be a really cool way to shake this story. This would be well received. This wouldn't be well received. And we say, hey, this is a cool person that we could bring in to come and play with our tools. Maybe make some videos, Rich?
But in terms of what whatever it is, I like to reference a book. We're in book club now. There's a book called Developer Relations here by Caroline Labco and James Parton. And they have this beautiful picture here that says the Developer Relations framework you see. So I'm gonna read backwards because I'm showing you there. You've got Developer Experience. You've got Developer Marketing. You've got other words that I can't read at the moment and then Community Sat at the bottom. Yeah.
Richard Bradshaw (10:02)
Yeah, perfect title.
Okay.
Developer education, developer success.
Chris Armstrong (10:23)
So if you think about like some of the external signals that you'd see from, it's a sort of Venn diagram, but some of the external signals that you'd see coming out of developer relations would be things like a community platform. At SmartBear, have a community platform. You have community managers, you have people who.
create a space where people can maybe see like a front door to the company, but also see where are the conversations, where's the support, where are all these sorts of things that I can get involved in to learn about various tools and opportunities. Then there's also content, people thinking about ideas and trying to articulate them to make things applicable to the real world.
where you're speaking on behalf, being ambassadorial for your company and doing a lot of that stuff. And then there's also empowering your team within your own company to do the right things. We all know the likes of dog fooding and that word integrity, also credibility comes from.
you do the things that you say that we should be doing. If we talk about good practices, are we doing those things? We talk about, this is what quality engineering is. Are we doing those sorts of things? So you can back yourself up and you're trying to add a bit of uniformity to your storytelling as well as layers of depth and credibility in that space.
For someone with ADHD, it's a wonderful role to be within because you are in all of the places, all of the time, you get to know and be known with a lot of different people, you get to help shape the future. Like the reason I was interested in SmartBear when the role came my way was SmartBear is on this path towards something, something AI being involved in quality engineering today. And I...
passionately believe that if we are just going to clamp AI onto everything that we've always done, all those frustrations we've had all of these times are going to just be worse and we're all going to burn out and everything is going to be trash and homogeneous and rubbish. And so really look at it and go, we can change everything. We've got this opportunity, all this stuff that we've hated for 20 plus years. What would we do if we had the opportunity to do it better? It's a real time of disruption and
a tool vendor who makes a lot of tools and has a voice to speak to a lot of people has an opportunity to say, scratch that, here's the stuff that's really important. Here's the solutions to real problems that we can help that go beyond just the granular, it looks at more of the holistic opportunity. And so one of the words I used in my interview, which obviously worked, was I want to be on the right side of history in terms of AI adoption.
going on in our industry right now. I don't wanna be sat here going, I told you so, or I wish we hadn't done this. This is an opportunity, that is a platform, and I want to actually push forward to do better things more appropriately and ⁓ less abstracted, but in a way that is still informed.
Richard Bradshaw (13:38)
Nice. There's a lot of things I want to go back to there. So before we dive into the world of AI and how it changes it, right, at the start of that, you mentioned DevRel is kind of this bi-directional consult, I can't remember the exact words you used, but it's something like that. Yeah. So you get to play, well, you don't get to play. You kind of have to be aware of what the industry is up to, aware of what people like me and Vernon on the ground are doing and saying.
And then you're also feeding that back into SmartBear, right, to try and lead direction, like shape the direction of some of the products. But at the same time, you're also pushing the direction of SmartBear, which may always not be in line with what you're seeing the other way. So how do you juggle that balance between like SmartBear is saying we're going in a direction, but you're observing that people are doing B and a little bit of A.
So how do you find that balance? So how do you manage that balance?
Chris Armstrong (14:42)
⁓ Well, as testers we have a gut reflex to a lot of things, right? There are things that we see that we don't like or we're uncomfortable with. Don't outright trust your gut, ever. But, lean into that, what's that signal telling me? And I sort of pressure check, sense check, speak to some other humans that I trust and go, am I crazy or is this a real thing? Does this sound right? Does this feel right?
and then being very respectful in challenging people but directly. So for example, I've seen, you know, go-to-market decks that have said various words and had various presentations and I've had to just go, no chance, that can't go. That's the wrong message for the wrong people. It doesn't look right. You have to be...
very proactive, but also, like, that has to be backed up with other things. Because if I just sat here and said, you can't do that, what's anyone got anything to go with? So the productivity, I hate the thing that my old manager told me, which was don't just bring me problems, bring me solutions, but you have to qualify your feedback. So just by saying, that's wrong, that's bad, isn't enough, or this I like,
It's this I like because this is good. We should do more of that thing because I've heard over here, this is a really important opportunity for us to do those things. We look for connections that we can draw upon. Also see what everyone else is doing. So being a communicator who kind of puts their shame kind of just over in the corner because you appreciate that it's bigger than just you and it's a privileged position, but
people won't listen to you if you just sit there and let it slip your shoulders. You lose respect for people. Respect is probably like the last bastion of something that we have in the workplace.
Richard Bradshaw (16:53)
So, ⁓ where do I go from here? So...
⁓ I did have a question that's completely left my head. Let's put a little mark. Let's put a marker here. Yeah, of course you can.
Vernon (17:04)
Can I can I can I dive in? Can I can I dive in? Yeah, so
What is it like when you say bi directional consulting, right?
That so I've been a consultant, worked at ⁓ a few consultancies. That's where, you know, that's the firm eight style of firm that Rich just left. And I know you, Chris, have worked at a lot of consultancies bloody hell.
I always I always find that being a consultant is a tricky balance because you want people to give you feedback and give you intel and insight so you can help them. But also sometimes the expectations are that the person who has ⁓ pulled in the consulting just wants you to tell them stuff. Just immediately tell them stuff. So how does how does that work being a dev rel? Because I
I my understanding of what a devrail was is that ⁓ you're kind of part of the community that you're serving. And so it's less about you know you I would be surprised if you ran in ran into Hustev ⁓ last week and says everyone must start using SmartBear and BearQ because it's incredible and blah blah blah. ⁓ so how does that how does that consulting
That traditional consulting stuff kind of play out in bi directional consulting, like you said, because that sounds really interesting to me.
Chris Armstrong (18:40)
So, I mean, in both directions, the key is context. We don't just get context. You uncover context, you interact with people, you chat to people, you have discovery, and for a lot of time, you're biting your tongue.
Vernon (18:49)
Mm.
Chris Armstrong (18:58)
You're switching your brain off from putting your own experiences and projecting them onto other people. You're listening, you're understanding. You're looking for cues, verbal and non-verbal and other sorts of patterns that can inform the way that your discussion works. At SmartBear, one of our things that we're leaning more into is this thing called the testing front door. And the concept of that is...
We just said we've got a buttload of tools over here. We're not trying to sell all of the tools and so people are getting forced into some sort of workflow and pattern that isn't gonna work for them, because everybody has a different context. Everyone has a different reason they are where they're at and everyone has a different destination that they're trying to get to carrying whatever passengers they have with them. And so the concept is very much that the comers you are, it doesn't matter where you're at. What matters is you're here.
We listen to what you've got and we then present solutions. It's the same in consulting. You are a useless consultant, in my humble opinion. If you turn up and you take something that you've done everywhere else that you've worked before and you just go, that's what we're going to do now. might get paid today, but maybe tomorrow, not so much. And people get very frustrated when they don't feel respected and there isn't something that's bespoke to their.
Richard Bradshaw (20:04)
Gosh.
Vernon (20:09)
Yeah. Yeah.
Richard Bradshaw (20:11)
Yeah, but it worked at the last place.
Chris Armstrong (20:23)
scenario, like you can tell them to go, why the hell are you using that tool? There might well be a really good reason why you're using that tool, but at surface level it doesn't appear. So taking the time to understand the art of consultancy is very much, you're listening for your discovery. You shouldn't be projecting a lot of stuff early doors. You should be listening to understand, which not only informs how we can move forwards, but also where else. that...
point then speaks to one of my sort of mantras of the now, which is decisive humility. There might not be a solution yet. We might not know enough to do anything with it. But the point about being decisive is A, you're owning it, you're not BSing anybody right at the beginning, but also you're actually saying, but I'm gonna own this, I'm gonna find out. We're gonna find a way and a path.
to be able to achieve those things. The future isn't written yet and quite frankly, in six months time, some of the technology we're working with today is gonna be redundant anyway. So understanding that whole picture is actually more important to then be able to apply and flex and change as we go forwards.
Richard Bradshaw (21:26)
Absolutely.
It would be redundant a lot quicker than it's ever been before as Go on,
Chris Armstrong (21:32)
Mm.
Vernon (21:36)
I was gonna say is is the bi dire is an example of the bi-directional nature of that when you talked earlier about looking at a I think it was a go-to-market strategy saying ⁓ the like the wording of this is not that's that's that's the community that's you listening to the market the community and saying hey we need to communicate like this because interpretation
Chris Armstrong (21:46)
Mmm.
Yeah.
Yeah.
Yeah, but also going to your product teams and they say, hey, these are the features we're gonna work with, and you're like, no. Or, do know these guys are doing that? Or, actually there's an opportunity to pivot into a new direction because actually this is now more important. So you are team smart, but you're also team quality engineering and you're saying, I want this whole thing to be better.
Vernon (22:09)
Right. Right, right, right.
Mm. Mm-hmm.
Richard Bradshaw (22:25)
So you're saying that you go to the testing front door. I'm picturing you as a of a bouncer right now, Chris. It's like SmartBear are about to knock on my front door, and Chris is stood there going, what message are you trying to deliver to Richard? Actually, no, you'd be better off. Let me see your credentials.
Chris Armstrong (22:39)
No, it's, the door's open, the come as you are thing is legit, like, so, you shouldn't need to qualify yourself to someone that you're asking for help and solutions. You just go, this is where we're at, here's the reality, let's face it, like, LLMs these days tell us all the positivity, they're always gonna give us an answer, we actually just want to get down to the reality of the situation as early as possible to then proactively move on.
Richard Bradshaw (22:44)
Yeah, yeah.
So everything you just said to me very much feels like, it's not the same as testing, but it feels like a lot of the skills that you would have picked up, right? Because we're often this bi-directional role, right? It's like, Debs are like, this works, or that works, or we're building this because they've said so. And then we're over here going, well, I don't think the customer wants that. The stats don't say this. If you do that, you're going to break this. But then the way that you're, and I want to ask you in a second, is kind of the way you go about doing this, right?
Chris Armstrong (23:23)
Hmm.
Richard Bradshaw (23:37)
We're kind of that random. We do experiments, So we often have to, as I was listening to you, was like, he sounds like he's talking about your testing story, right? But we have to do these experiments. We have to try and validate our opinions or our insights or our beliefs, and then take all that evidence along to whoever it is that you're wanting to not so much educate, but inform, whatever the right term is.
So off the bat of that, what are you finding is a good way for you to do that? And when I mean do that, I mean what techniques, approaches work well for you? Because I've seen recent surveys from SmartBear. I see polls. I know you've got this community platform that you mentioned building content. There's webinars. I don't want know everything that you do, but what do you like the most? What do you prefer as your way of doing
Chris Armstrong (24:31)
You
Richard Bradshaw (24:37)
that bi-directional work.
Vernon (24:39)
be a long podcast that would be
Chris Armstrong (24:40)
It's, well, I mean, high level is it's doing what you perceive to be the most important thing. I enjoy talking to people and understanding where they're at, finding people's pain points. Like in a previous life, I remember putting a post out asking why people hate test management tools. Because I was genuinely interested in why people get so frustrated in that space. There's no point in me just
making a survey and doing these sorts of bits, you want to cut through that and get to the bottom of it. enjoy reading, I enjoy writing, I enjoy experimenting. A lot of what we talked about is very theoretical and we're just being social busybodies, which is fine and that can be quite a visible thing, but one of things you talked about there was experiments.
actually demonstrating like here's an actual thing, here's a real life thing. So we've got playgrounds ⁓ and things that people can actually adapt and change and we can say actually wouldn't it be cool if, rather than just stopping there, then doing stuff and having the ability to lead by demonstration and example is really important as well. ⁓ We did a webinar a while ago talking about the updates to the open API spec and we sort of showed.
where those standards and how they've evolved still apply in our tool sets. We showed people as well. The art of being able to tell a story to people that means that it is applicable to as many people as you can do is a difficult thing in and of itself because we've just discussed everyone has a different context. And so that's a really fun bit when you're out. And you have to be cautious to Verland's point. I'm not going around selling all of the things.
I'm not a salesman. And so, you you hopefully have got into a position in DevRel where you have some credibility, you have some knowledge, you have some understanding, some reading, you're able to connect things, and you're able to use that to articulate shape, make ⁓ great contributions to product roadmaps, and then conversely say, that's an interesting topic, insert external forum here.
this is how I would do it, this is what I would do and giving people the opportunity to sort of play and participate.
Richard Bradshaw (27:11)
Excellent. Well, off the back of that, one of the things that I found like the main thing with DevRel, right, is a lot of companies make a lot of promises or they use a lot of terminology, right, that may not always understand. So I like, for example, for me, I can, I can go onto a tool website and I can read, you know, the taglines, the heroes, and you know, the promises.
And for me, I always kind of turn to the DevRel, as you just said, to see that in action. Like, yes, I believe your tool may do that, but before I commit to maybe playing with it and trying it, I'd rather try and find something from someone that's kind of validating those promises. So I've seen a lot of DevRel work from that, and I really appreciate that kind of stuff. And I've also seen that, for example, I'm currently doing it with SmartBear. So I'm making some videos for BearQ.
where I've read what BearQ says it does. And now I'm like, well, does it do it? And I want to see if it does it. And at the same time, I'm going to share what it does do. But for example, one of the things at the moment that SmartBear is shaping a lot of their framing around is what's literally on your t-shirt. I did not know this would be on his t-shirt. He's very much on brand. Well done. But application integrity, right? Go on.
Vernon (28:34)
⁓ yeah, he's doing the right thing. Okay, give him that pay rise.
Richard Bradshaw (28:38)
Yeah, by the way, just in case there's any doubt, absolutely was running around Husteth just screaming SmartBear constantly. I didn't manage to get it on video, but like, no, I'm joking. But yeah, I can see, I can read the words, right? Application integrity, right? But what is it, Chris? Like, what is the, because it feels like this overarching philosophy of where things might be heading, but like, can you explain it to me? Like, what is it?
Chris Armstrong (29:01)
Mm.
So application integrity is an outcome that you achieve with good sound software quality engineering practices. That's the high level. How you achieve that is gonna be different depending on your context. Like at who Stefanie Kohl van der Hoeven spoke about how there's this huge black box and so the way that we understand through the abstractural testing around that black box, that's how we understand what the outcome is compared to what we.
intended at the beginning, but in other circumstances, we might actually want to look under the bonnet and see what's going on. And we want to see that we're following those good practices, we're following those illities, we're doing all those sorts of things. It's sound, good engineering practice that leads to an outcome that we can actually put our name on. You know, I worked with a lot of computer games.
and some of them I knew very, very early were bad and I did not want my name on those credits. Because people will look at that and go, who tested this? We've all heard that one. It's a painful thing to be in when you already know that it isn't very good. What we want to be able to achieve is something that we're proud of. Is this actually good? Is this as intended? Does this...
Vernon (30:08)
Yeah.
Chris Armstrong (30:29)
meet the needs of the end user. That means a lot of good practice like solid requirements, things that actually speak to who that end user is and what problems we're trying to solve and all those sorts of things that we've talked about for many, many years. But as I mentioned earlier, like the whole amplification of a broken system, which I think most of us can agree we've operated within for a terrifically long time, there are problems like, did you meet DOR? No.
But it's fine, we'll just YOLO it later, we'll add up a few comments, it'll be fine. That's a very human problem, and if you're amplifying a very human broken problem with lot of inferences that maybe we have, how are you going to know? can't test your way out of bad quality design and bad quality implementation. So a lot of this is about regular feedback, things that you talk about, that we understand where we're at.
Has it done the right thing? Has it met the expectations? Application integrity is a goal that we should be working towards because an application without integrity is a massive waste of time, is AI slop. It's something that people would be really ready to pick up and throw away if they don't think it's any good. You want software that does a good job. You want an application that solves a problem and that people want to carry on using because they believe in it.
Vernon (31:53)
So I feel like that definition isn't specifically about excuse me, man. I feel like that definition isn't explicitly about AI or, you know, systems with AI, but has it has it been impacted quite a lot by the introduction of this thing?
Chris Armstrong (32:19)
The speed and scale that AI has come has happened and also a lot of people who don't know how software is built and are building software. I had a ⁓ CTO at one company go, I've made a new onboarding platform for all of our customers. H5 coded it and it looked really nice on his computer.
Vernon (32:27)
Yeah.
Richard Bradshaw (32:39)
You
Chris Armstrong (32:40)
put it in for a PR, there are quite a lot of reviews that took place on that thing. Non-compliances with security practices, with good practices you have as your company. Stuff on the surface there that looks like it works often doesn't. And the thing that we're quite passionate about is building good software that we're proud of. We're still human, we still wanna feel like the Thomas the Tank engines. We want to feel like we're really useful engines.
Vernon (32:47)
Okay.
Yeah.
Chris Armstrong (33:07)
We want to feel like what we're doing is worthwhile and the outcomes that we're creating are good quality things that have integrity.
Vernon (33:18)
Okay, okay, okay. Got it.
Richard Bradshaw (33:20)
So off the back of that, right? That's like, I get the goal, right? And I kind of have a better understanding, but there's no one tool, right? That's going to help me do that. And also it sounds like a lot of it can't be done by a tool. So your DevRel role and the kind of position, it sounds to me like you're going to have to do a lot of extra education, let's say, knowledge sharing.
to help people get the most out of the tools that SmartBear are building. So they're being built with this in mind, but I just can't take one of your tools, dump it in my ways of working, and we magically achieve application integrity, right? So, yeah, true. So how are you and SmartBear approaching some of that challenge?
Chris Armstrong (33:55)
Mm.
Neither should you.
So if we go back to the testing front door, that whole thing is all about, I mean, actually, we could even wind ourselves back. You remember that lovely one page test plan? think Claire Reckless wrote about it, was informed a lot about by Lisa and Janet's work in agile testing. A lot of this stuff is about specificity. Yeah, yeah, yeah, exactly.
Vernon (34:32)
Mm.
And Mike talks as well.
Chris Armstrong (34:43)
And there's lots of specificity that is involved around it. There's the scope, there's boundaries, there's environments, there's people. There's all of these things that we need to know about. Just the same as with the consultancy, the same as with creating solutions. The more that you know upfront, the more applicable, the more appropriate, and the more value you can glean from the right solutions. It might be in a context that somebody has said to you, you need to find a tool.
that has AI to do something for you and you're like, okay. But what are the problems I need to solve first? I shouldn't be looking for where I can bolt AI into that bit. Someone might have said to you, you need to automate more of your testing. Okay, but why? It's always going that extra layer of understanding the why. So a lot of the discussions that we have about the tools is the tools are here, we know their capabilities, we also know where they might be going with their roadmaps and.
where they can interact with other things in your tech stack. But as with all things, you need to start agnostic of tech stack. You need to understand culture, context, portfolio, goals, they're near or far, ⁓ goals that you need to try and achieve to then be able to work out what is the right thing for me. You can speak, say that we have Zephyr as a test management solution.
for Atlassian. Do you work in Atlassian tech stack? No, don't use Zephyr, not gonna help you. And the same is true, if you've got more of an automated pipeline, do you even need a test management tool? No, there's no point in forcing people saying this is the explicit SDLC for 2026 that includes AI for everybody. That doesn't exist. We've all looked at a million different SDLCs. ⁓
Vernon (36:19)
Mm-hmm.
Chris Armstrong (36:40)
You've got to understand where everything sits first. It's a lot more work
But it also means you've got a lot more opportunity to be successful and helpful and applicable in order to do that.
Richard Bradshaw (36:57)
I've just landed in the middle of what you just said, though. I don't need a full blown TMS, but there has become a need and a requirement to store all the results from the automated test executions in a queryable manner. So it's not like I just saved the reports from the CI-CD and dumped them somewhere. I've been told we need a goal to have a queryable system, essentially put them all in a database, and be able to say, against this version on this day,
these automated tests ran, and this was the results. Don't need a whole test case. Don't need any of that. But I do need this bit in the middle. But to go back to the plethora of tools that you mentioned at the start, like 25 odd tools, it? It sounds like you almost need something that big, which is kind of unique to SmartBear. That is a lot of tools. But as what you've just said, if the goal is application integrity and there's so much context involved, you've used the word good a lot.
Good is very contextual. And you're going to different front doors and understanding their problems. So I feel like that puts SmartBrain a pretty unique position. But again, not going to go into it, but must make your job hard because there's so many contexts out there. There's so much flavor, right? But let's focus solely now on AI. So for yourself, Fern, I'm not going to let you frame this and introduce this. How is AI changing?
Chris Armstrong (37:56)
Hmm.
Hmm.
Richard Bradshaw (38:23)
your day to day and kind of where do you think it's going to head and then we'll see what Chris and SmartBear are up to in the same space.
Vernon (38:35)
That is very easy. So
Richard Bradshaw (38:38)
You
Vernon (38:42)
Seeing this in my company, but I'm seeing it at large. Is we're still solving that that fundamental problem. Now, is there a problem here? Like we're still trying to make the systems rebust bug-free, etc. But
Instead of us doing the testing as humans, what we're trying to do is build systems, things, entities, robots that will do that testing. So you drop something into that system and it comes out the other side of that black box that Nicole talks about. Tested. I'm doing air quotes for folks who can't see what doing. ⁓ and so
I I I am going to have a meeting directly after this call with my team about this very topic. So it's very important of mine. Because I I I think that we are going to be forced to operate more like product teams and delivery teams and maybe not even teams, but product people. Like we're we are building testing products for the market. And that market is your dev team, your engineers, for sure.
Richard Bradshaw (39:32)
You
Vernon (39:54)
probably your number one product segment market segment but also maybe your product folks, maybe your design folks, et cetera. So it's about being able to identify where are the where are people having testing and quality challenges? Where is it or to use ⁓ smart bare time terminology, like where is the application integrity under threat? And how can I what can I build to to remove the threat, mitigate the threat, manage the threat, etc.
Whereas before it'd be just like, what am I going to go and create? It's more like, okay, how do I need to modify the system so that thing disappears?
Richard Bradshaw (40:28)
Hmm
If you just made tens of thousands of competitions to SmartBear with that description Vernon, now every, every QE team. No, no, but I get, I get the point. I get the point.
Vernon (40:43)
Well well this is this is that that that whole SAS question, isn't it? Yeah, it's that whole SAS question. Do you really want to go and build twenty five tools and manage them all yourself? I mean you might, but you probably don't.
Richard Bradshaw (40:58)
Well, not going down that tool direction, I just mean the notion where you said, look, it's not as a QE, you'd be like, how can I do this work? How can I do that work for them to help them with their integrity slash quality doing good work? Whereas now you're saying it's more, what can I build and shape and offer them that's going to help them do the work? Almost like we've always said about QE, right? The goal is trying to do yourself out of a job, right? But in a way,
Chris Armstrong (41:25)
Yeah.
Richard Bradshaw (41:28)
It's now accelerating to producing tool sets, bringing in tool sets that fit for purpose to help those teams.
Vernon (41:37)
Ex
Not to go too much of a tangent, and I know that you know we're supposed to be hearing from Chris, but I'll try and be quick. What I'm learning about entreprene entrepreneurship is there's three flavors of service that you can give to customers. you can help them do it yourself, you can do it with them, or you can do it for them. And I think up until this point, we've been doing a particular version of
⁓ like do it yourself or maybe do it with you. Whereas the stuff that we're building now is definitely we're going to build something that does something for you. So it's the difference between saying, I'm going to teach you how to understand what the risk is of the thing you're building so that you can decide how you're going to build it and how you're going to test it. Whereas now they just want something that just does that for them. They don't they don't want to learn necessarily, right?
And I don't there's a whole argument about whether that's right or wrong, but that's definitely what people want, I feel, anyway. ⁓
Richard Bradshaw (42:43)
That feels very much though back to that tool automation conversation I said. They don't want to sit and use the tool. They want it to be done for them. But on that note though, I know SmartBear are doing something like this, right Chris?
Vernon (42:47)
Yeah yeah. No.
Chris Armstrong (42:54)
Yeah, well there's there is I mean you're right I mean there is a there's a really weird balance of appetite though you know one of the things that ⁓ that people are using these days to talk about is is is the spectrum of autonomy taken from the ⁓ the autonomous vehicles because there are some people that want just everything done for them and there are folks that want to do literally everything themselves
because there's risk appetite, there's also time, there's also commercial pressures to keep up with all of these other things. And so we've also tried to map our tools along that sort of various different levels of ⁓ the spectrum because of different people's contexts across that. So we've got this BearQ tool that Rich has been playing with. It's much more on the full autonomous kind of space. It can keep a human in the loop, be directed, it can be full agentic.
all the way down to ones where people are still writing manually everything that they want to do and executing it themselves for whatever reason, instead of all those different parts in between. My take was always quality doesn't belong to the tester. Good quality engineering is engineering designed through release, done quality. And what that looks like, what quality looks like to different people will be different, but it involves.
everyone and and the the balance is around Authority ownership accountability responsibility and what does that look like it's easy to say qualities everyone's responsibility but what those sorts of things look like in real life and How that dovetails into something something? Agenda can autonomous is different as well because people do want stuff to go and be done for them But I guarantee you they want to know when something's gone wrong
that it's been taken care of. They want to know who they can point at and go, what's happened here? Are you responsible? Are they responsible? Who is responsible for those things? And that is one of those questions that people are wrestling with.
Richard Bradshaw (44:53)
Yeah.
Vernon (45:07)
100% and I and I think that's you know my experience of that is that's that's where we are you know leaning into more is because because we have that those reps of doing that kind of work we're being asked to to to really lean into that part of it because you're right, you know, it's no use driving your autonomous vehicle and
some accident happens and you go, well that's not my fault. That's some bloke called Musk. Like he was that was his fault. Like, nah, you were at the wheel. It's your responsibility. And I think that I think that's I think that's very, very true.
Richard Bradshaw (45:40)
You
Chris Armstrong (45:52)
That's where a lot of education needs to come from, right? We need to let folks know that by taking on this board, we've all scrolled through terms and conditions and accepted stuff before without reading them. But there's a bunch that people need to appreciate, cause and effect. And as we go more agentic and we empower more things, like we've all had people reporting to us before. We don't know everything that they do. There are levels of trust that we have in the work that people do. And Rich, you'll have seen this as you've been playing with with Berq now.
Vernon (46:02)
Yeah.
Chris Armstrong (46:22)
you're gonna send things off and get a surprise sometimes. Stuff's gonna happen that you didn't expect to happen. It won't have done everything that you've done because the direction you gave it wasn't full of exact specificity. You were saying, go off and do this sort of thing. And then it will go and do stuff. And sometimes you're gonna be annoyed and go, you shouldn't do that. Stop wasting my time, my money, my tokens. Go and do something more exact. You're learning how to better lead your team.
Richard Bradshaw (46:25)
Yeah, yeah.
Yep.
Chris Armstrong (46:49)
And that's the same thing we're, we've talked a lot about wanting more agency as testers for a long time. And now we're going to have to give up some of that agency into agentic spaces. And it's going to surprise us, it's going to scare us, but hopefully it's also going to excite us. This whole art of the possible thing is very exciting. And being pleasantly surprised is wonderful.
Richard Bradshaw (47:16)
Yeah, couldn't, given what you both said, I try and summarize it that a lot of testing work doesn't require an awful lot of agency. Some of it is quite simple and a lot of that is being wrapped up into agentic tooling. You mentioned I have been playing with BearQ and sometimes it hasn't done what I wanted it to do. But also, am yet, because I'm building up the series, I am yet to really tailor BearQ. BearQ doesn't know much about me.
It doesn't know how I like to test. It doesn't know the kind of thing I'm testing. I've not really given it anything about my application because I wanted to start by just seeing what it would do. But that feels very much like what you're both saying about we're giving these tools to people who may not have a testing background. They may not understand all the steps needed to achieve application integrity, right? But we're going to try and codify a lot of that into skills, agents, processes.
Vernon (48:08)
Exactly.
Richard Bradshaw (48:13)
I can't see the under hood of BearQ, right? But my guess is a lot of its eight system prompts are going to be very tailored to how SmartBear believe that tester should operate. BearQ has three roles that I've seen, right? It's got the QA lead, it's got the automator, and it's got the tester. And I'm sure there'll be more to come. But under the hood, someone will have given that a bit of a flavor, a bit of a persona. However, that might not match me. It might not match.
my context, but I know I can change it. But I feel the way we're going is definitely wrapping a lot more of this into Agente. Because a lot of the time, let's say you're testing a ticket and you work in a good way, good practices. The basic acceptance of criteria for that ticket is probably in the ticket. So the very low hanging fruit of does this thing actually do what it's meant to do, I believe, can be picked up by an agent.
Does it do it in the way that's consistent with the way we operate, or the way it was asked, or do we have metrics to know whether the customer's going to know? that's extra on top. But we can train more stuff in the future to even do some of that. So I'm thinking that we're going to definitely see a lot more of people needing the likes of BearCube. But also, I think the other thing that Vernon said is, and you both said, sometimes it doesn't go the way you expect. So when we're
take an autonomous car analogy, Very rare the roads, the rules of the roads have been set for a long time, right? Like it's almost like a fixed set of rules. Yes, you get idiots breaking those rules, right? And the car should learn how to adapt with them. But in our ways of working, you can probably model most of the ways that we're currently working. But now that's changed a lot, right? Because of AI and we're all changing what we're doing. So I still think there's this...
We can definitely delegate more to these types of tools than we should be, but there's still going to be all that need, I feel, on the outside of when these processes, autonomous flows, pieces of automation don't do what we expected to do. And those results end up in front of someone who doesn't know any better. How are they going to adapt and change? So I still feel there's that need for the specialism, and in turn, that continuous cycle that Chris mentioned at the start around the role of the dev rel.
is pumping that back into the process to how we keep it helping us provide good insights and achieving that integrity. Go on, Vinh.
Vernon (50:48)
What Chris, do you wanna
Richard Bradshaw (50:50)
⁓ you're just nodding, you're just agreeing with me. It's very rare that you nod along to Richard, so I was like, all right, I'll take it, I'll take it. Yeah, go on, Chris.
Chris Armstrong (50:52)
It's just these just three things to it.
Vernon (50:54)
I wanna hear
Chris Armstrong (50:58)
I mean...
For me, mean, a lot of what you're talking about is the value of trusted abstraction at this moment in time. We're going to have more and more abstraction. But the further we abstract ourselves away from what's actually being done, that's the danger. it's the knowing, the knowing what's going on and understanding that part and still knowing the pathways through. You talked about having dashboards, Rich. A dashboard that has a pie chart that I can't then click through and see, well, what were the failures and how I can do that thing is a useless
Richard Bradshaw (51:27)
Yeah,
Chris Armstrong (51:29)
useless dashboard. Something that says I've got 85 % coverage but doesn't tell me what I'm covering. What I'm not covering is useless. It's a number. And so that level of being able to apply actual abstraction with value that I can drill down into if I need to. And having the art of our jobs in the future being exploring the abstractions so that I can get that information. I can't hold everything in my head. But if I know how to get that information.
there is value in there. if we get to the point where we just take everything at surface value and we're going to get a lot of positive reinforcement from a lot of AI because that's what it does, you need to be able to learn to drill down. And a lot of the discussion, the discourse that we're gonna be having going forward is going to be around how do I do this?
Richard Bradshaw (52:21)
Yep.
Chris Armstrong (52:21)
What does good look like in terms of abstraction? What does good look like in terms of challenging your agents in terms of saying, show me the real data? Because you can see so much stuff superficially created that people just go fine, whatever. I use the example of at school, I signed my own homework diary every single week. It was a control check that the school had.
Vernon (52:41)
Yeah.
Chris Armstrong (52:45)
that made them believe that my parents were checking my homework diary every single week to show that I was doing my homework. But from week one, I signed it myself. They never saw another signature. As far as the school was concerned, that control check was taking place. That was as deep a surface layer as they went. It looked real. They checked their books. If we don't actually challenge what really exists, where does that really come from? How does that look? And go back like a bit deeper.
then we're going to have a lot of not application integrity.
Vernon (53:20)
Chris, ⁓ your parents are avid listeners of the podcast, so you've just you're in massive trouble now when you go home. Just wanna let you know. You've blown
Richard Bradshaw (53:24)
You
Chris Armstrong (53:27)
You know, my history teacher called me Billy Nowe Work.
Richard Bradshaw (53:32)
and your kids are going to start doing the same hack.
Chris Armstrong (53:35)
My wife's teacher, she's well on top of all this stuff, don't worry.
Vernon (53:36)
Right.
Richard Bradshaw (53:41)
I think to frame up a future conversation and maybe for even some of the work you do, you're doing that smart way, Chris. And I spoke about this at Husdeth. The whole thing around this integrity and doing what is right and doing the right thing, right? As humans, we often do take shortcuts. And a lot of the time we rely on things being tacit knowledge. And I think we've been able to in recent years, I think we've been able to do that. And when people ask, how's it work, why, we can just defend it and people will be like, okay. But now that we're delegating more to
AI and agents and our applications have AI in them, which can make a decision on its own, right? The way you code it could categorize something, ⁓ you know, the way it wants to, it could decide whether someone can have a loan or not. Suddenly that need for traceability and that digging down in production, yes, but also how we tested it, I think is going to be more crucial than we've ever cared for before. I think we've been able to take a lot of shortcuts in recent years and I don't think we can anymore.
And it's one of those things that I loved about BearCube because in the last video I did, I was asking it to retest issues for me. And the thing is it did. But it also, at the same time, without any full effort, it recorded all the videos for me, all the screenshots, put it all into the issue, tied it all together. And I didn't ask it to do that. It's one of those rules, like you said, that's been coded into it because it provides that digging down.
And I think that is something that we want to need a lot more of and to see SmartBear going in that direction already with the tooling that's around BearQ. And I fully expect to see BearQ integrated with a lot of the other tools that you have, right? The API side, the contract testing side, the links into Jira. I already know it's connected to Jira, right? It was one of the first agents connected to Jira, testing agents.
Chris Armstrong (55:32)
first assignable QA agent in Atlassian.
Richard Bradshaw (55:34)
Exactly. And I do think that whole traceability results direction is something that we're going to need a lot more of. if you look at, mentioned test case management systems at the start, right? And you know, I'm sure me and Vern have got episodes absolutely slating test cases, right? But I mentioned the word test case so much in my keynote last week because it was kind of the original framing of going from requirement to results.
But now with the tooling we have, need to push the evidence now. I used to make jokes about tick, green, green, green, like Richard, where's the proof? Well, the proof is it's Richard. Richard said he tested it. Richard has integrity. We trust Richard. And now it's, well, agent A, BearQ, said it tested it. But what did it do? How did it do it? And we need to see it.
Chris Armstrong (56:14)
Yeah.
But that's, there's trust, there's experience, exactly.
Vernon (56:29)
Mm.
Richard Bradshaw (56:32)
To have that built in from the off, I think, is really, really cool. And I really do think that is a big part of where tooling is going to head. The whole requirement intent, showing you've done it, showing the evidence. And it's clear SmartBear have got their finger on the pulse with that already.
Excellent.
Chris Armstrong (56:56)
we've got an event coming up. ⁓
Richard Bradshaw (56:56)
So ⁓ I was going to say, it's closing remarks. Chris, is there anything you want to plug, you want to share, you want to promote?
Chris Armstrong (57:05)
Well, SmartBear have got, I don't quite know how it's been pulled off, but I'm quite excited. We've got Andy Weir doing an event with us on October 22nd. Andy Weir, the author of The Martian and Project Hail Mary is doing an event, which is insane. And from my perspective, we talked a lot today about storytelling and evidence and backing things up. And I watched him on Neil deGrasse Tyson's podcast.
really going through some of the science and the logic behind how he was shaping the story for Project Hail Mary and they were like, let's talk about how you landed on this particular element for this to be made out of. And they're like, yeah, I can see the logic behind that. I liked it because he showed his work, he didn't just make something up and go, yeah, gamma rays, we're fine. There was some depth and some science behind it. And that's why it makes an amazing, deep storyteller who does a lot of research.
a really exciting person to have for this event that we've got. It's 11.30 a.m. Eastern time. Now we're five hours removed from Eastern time, you know, so it'll be our afternoon. There'll be a lot of things that'll be shared on that front. But beyond that, you know, if you're using JIRA, if you're using Atlassian tools, BearQ is on marketplace.
There's a self-start trial, give it a go. Look at some of the stuff Rich has done. You can have a go with some of the tuning. We have many of the other tools. But check it out, give us feedback. We've always taught as testers, we thrive on feedback. Stuff you like, tell us you like it, tell us why you like it, tell us what the application was, stuff you don't like.
telling me don't like something, I can't do a lot with that, but if you tell me I don't like it because it doesn't work in my context, then I can do something with it. I've got the phone numbers of a hell of a lot of important people making tools here. Do you know what I mean? When I said I came here because I wanted us to make a difference and be on the right side of stuff, I'm not kidding. We can only do that because we get useful, tangible feedback from people. And otherwise, check out Testing Piers.
Richard Bradshaw (59:22)
Anything to promote Vernon? ⁓ I know you've got a little mission on the go at the moment. I've seen it.
Can't I hear you anymore?
Chris Armstrong (59:39)
Silent Vernon. I cannot hear Vernon, but it was, I'm sure it was exciting. There you are.
Richard Bradshaw (59:39)
Can you hear him Chris?
⁓ go on, you're back.
Chris Armstrong (59:49)
You were back and then you weren't. There you are.
Richard Bradshaw (59:50)
He's gone again!
It's like it's switched microphone or something mate, I don't know what's going on.
No. When you face that way, it works. No, the other way. No, it's gone. Whatever's going on is gone. Doesn't matter. Vernon's got nothing to promote. Vernon needs to go buy some new microphones. No.
Chris Armstrong (1:00:08)
No. No.
We'll be at Aero Dev together, we, in November then?
Richard Bradshaw (1:00:24)
Have you got crisp on? Is crisp like killing your background noise or has the background noise increased or something?
Anyway, if you can hear me, was going to, I know you're doing this writing challenge. I was trying to promote your writing challenge that I've seen. Yeah. No, some it's happened to your audio, but you've gone.
That's very weird, isn't it, Chris?
Vernon (1:00:51)
Morning.
Richard Bradshaw (1:00:55)
Something's cutting you out. Anyway, give up, Vernon. I'll promote it for you. We've lost Vernon. Vernon's microphone's trash. Vernon is currently doing a 30-day writing challenge, I believe, trying to post every day to get back into consistent writing. So that's really cool. And Chris has just mentioned that they're both going to be at Oredev, ⁓ an event I've been to twice. Fantastic event in Malmo. So I hope you both enjoy that. I've just given the automation and testing website a bit of a revamp. ⁓
new principles on there, a bit more reasoning of what I'm trying to do with it, and the history of AIT as well. Because there's a lot changing. So yeah, Chris, absolutely pleasure to chat with you. I'm very much enjoying the work that you're personally doing at SmartBear. And then also, as people know, I've been doing some work with some of the videos and playing with BearQ. And I am really enjoying kind of where it's going. So keep up the good work.
Chris Armstrong (1:01:51)
Thank you.
Richard Bradshaw (1:01:52)
And yeah, thank you for being on the Vernon Richards show. go on then.
Vernon (1:01:56)
Can you hear me now?
Chris Armstrong (1:01:58)
Yes.
Richard Bradshaw (1:01:58)
Yes.
Vernon (1:02:00)
Okay, the only thing I wanted to add with all of us being football fans, right? There's another famous, there's another famous Richards who will ask a question at this point whenever him and his colleagues are interviewing somebody. And that is, who's your favorite presenter, Chris? Like who's your favorite?
Chris Armstrong (1:02:23)
Does it have to be on this show?
Richard Bradshaw (1:02:25)
Yeah, he means this show. He's copying CBS, right? He was the best. Go on, Chris. Come on.
Vernon (1:02:26)
Ideally yes. Ideally. Come on. Come on.
Chris Armstrong (1:02:32)
Well, Vernon, in my best testing vernacular, it depends on the context.
Vernon (1:02:38)
Aaaaah! Well done!
Richard Bradshaw (1:02:40)
Chop out.
I had dinner with him last week and chatted for hours. I bought him a beer and I still don't win. Outrageous. All right. Thank you everyone for listening. I hope you enjoyed it. The same as we always say, we're interested in feedback. We've got Chris on as a guest. If you like us having guests, let us know and we'll do more of them. ⁓
Vernon (1:02:46)
dear.
But let us know who you would like us to have as a guest as well. That would be cool.
Richard Bradshaw (1:03:08)
Absolutely. Sweet, right. Enjoy your days, everyone. Bye bye.
Vernon (1:03:14)
Cheers Chris, bye everyone.
Listen to The Vernon Richard Show using one of many popular podcasting apps or directories.