AI Autonomy and Application Integrity | Chris Armstrong

The chaps break with routine and bring a guest on the pod!

Be sure to register for SmartBear’s upcoming webinar with Andy Weir, author of The Martian, where he’ll be talking about ‘From AI Chaos to Agentic QA’ 

Register here - https://smartbear.com/lp/from-ai-chaos-to-connected-software-quality/?utm_source=r-bradshaw&utm_medium=content-creator&utm_campaign=CP-26-q4-npi&utm_content

Chris Armstrong runs developer relations at SmartBear, which means part of his job is telling a tool vendor what testers really think, and part of it is telling testers what that vendor is really up to. Sometimes it means saying "no chance" to both. That puts him in an interesting seat for the question hanging over this whole conversation. More and more testing is being handed to agents that go off and do the work while nobody is watching. So what happens to the people who used to do it, how do you know the agent did what it says it did, and who carries the can when it didn't? Vern opens by admitting he's no longer sure what his job even is. Rich has been putting SmartBear's own testing agent through its paces on camera, and it hasn't always behaved. And Chris has a confession about a school homework diary that might change how you read every green dashboard you see from now on.

00:00 - Intro
01:22 - "What even is this job anymore?" Why Vern feels like he's failing, and is smiling anyway
03:45 - What changed in how testers talked about AI at HUSTEF this year
05:27 - A DevRel (manager no less!) on a testing podcast?!
06:38 - If AI isn’t a tool, what is it?
09:42 - DevRel in two words, and neither of them is "sales"
12:52 - What happens if we clamp AI onto everything we've hated for 20 years
14:29 - The company says A, the community is doing B. Whose side is DevRel on?
17:35 - Useless consulting and what Chris calls decisive humility
24:47 - Why Chris asked people why they hate test management tools, rather than run a survey
28:19 - Application integrity is on Chris's T-shirt. What does it mean off it?
29:57 - "Who tested this?" and the game credits Chris didn't want his name on
31:47 - The CTO who vibe coded a customer onboarding platform, and what happened at review
33:05 - Can you buy application integrity off the shelf?
37:48 - Why Vern thinks test teams are about to start behaving like product teams
40:59 - DIY, DWY, DFY? Vern explains
42:11 - Everyone wants it done for them, right up until…
44:58 - Testers asked for agency for years. Now we have to hand some of it over
46:20 - BearQ hasn't always done what Rich wanted, and he isn't sure that's BearQ's fault
50:00 - The dashboard Chris calls useless
51:35 - The weak control check Chris beat at school, every single week
52:36 - "Richard said he tested it" used to be proof. What counts when an agent says it?
55:49 - What Andy Weir has to do with showing your work
59:07 - The Richards Question

Call to action
Chris says a tool vendor can't do much with "I don't like it", and neither can we. So tell us why.

If you agree, let us know. If you disagree, let us know. If you violently agree or violently disagree, definitely let us know. Guest episodes are an experiment for us, so we especially want to hear whether you'd like more of them, and who you'd like us to invite next. Questions, comments and concerns are all welcome.

And don’t forget: Like, subscribe, and share 😊.