When employees use AI to get better answers, customers have better experiences | Engageware

When employees use AI to get better answers, customers have better experiences

AI in CX summit | 2026

When your employees can’t find what they need, your members feel it. In this session, OnPoint Credit Union’s Aimee Ten Eyck-Schaffer shares how a 600,000-member credit union transformed internal knowledge search and what it meant for employee confidence, content quality, and member experience.

See what it covers:

The AI suite built for real business outcomes

Explore the Aivo Suite

Transcript

SpeakersAuto-scroll

Keyboard Shortcuts

Alice Milligan

Having been in customer experience and brand marketing for a significant portion of my career, one of the things that I've always said is that your employees are the biggest advocates for your brand and really where the rubber meets the road in terms of delivering on your brand's promise. That interaction and how your employees serve and address client needs is really important.

And so the next session is really going to talk about how OnPoint Credit Union employees use AI to better answer customers' questions and give them a better experience.

Justin Smith

Thank you, Alice. Amy, thanks so much for taking time today to chat with us and to share your story. But before we dive into that search transformation story, why don't you set the stage for us a little bit? Tell us a little bit about OnPoint.

Who do you serve? What is your approach?

Give us the lay of the land as far as you and your organization.

Aimee Ten Eyck-Schaffer

Sure. So OnPoint is a local credit union. We started as Portland teachers and have slowly expanded to serve Oregon and Southwest Washington. We have roughly six hundred thousand members, nine point five billion dollars in assets, which is a really big deal when you start to think about knowledge management. We're about to cross the line where we're going to be federally regulated. So having our information accessible and inline is super important.

We're serving about thirteen hundred employees, just a little bit over thirteen hundred. We've got fifty-nine branches and a lot. We've been with Engageware for a little over ten years, which has been really fun to grow with you guys.

Justin Smith

Yeah. And thank you for that. It's been a great relationship. You and I go way back. And so, you know, again, happy that you're able to share your story.

So, you know, you had mentioned going into this that traditional search left employees frustrated, hunting through documents.

Can you unpack what that experience was like? And sort of what the overall impact was of that?

Aimee Ten Eyck-Schaffer

So yes, traditional search, when we first started using Engageware produced a list of findings for folks, and we had to be really careful on how we curated what would end up in those search results.

Otherwise, you'd have just a super long list and lots of pages to navigate through to find what you needed. So we leveraged keywords and really prioritizing the top level of a procedure in order to get folks in the door.

But that left them having to sift through the rest of the information in the procedure to find what they needed.

Justin Smith

Which is time consuming, frustrating. Let's go a little bit deeper here. Let's go to the next slide. And, you know, tell us a little bit about what made, you know, the old way of searching challenging for your teams and, you know, what were some of the limitations and impacts of that?

Aimee Ten Eyck-Schaffer

On this slide, we're focusing just on our retail procedures. We actually use your system for our entire organization. But for retail alone, we have around one hundred and forty-five procedures. So imagine searching and having that list come up.

We couldn't reveal all of the steps within a procedure because that would be unwieldy to sort through. So that left folks trying to find the procedure they needed and then trying to figure out which step within the procedure they needed to access. And if they didn't know which procedure a step lived in, that made it a bit more challenging. So keywords really was our tool at the time, but it still left folks clicking around quite a bit and spending time and then maybe even giving up and phoning a friend for the information rather than leveraging the knowledge management system.

Justin Smith

It's typical traditional search challenges. You know, we hear this, I guarantee you a lot of people listening today have that problem.

And so when we're thinking about the solution to this, I think you saw an opportunity to leverage something new.

What made you kind of say, "look, now's the time for AI enhanced search" and what was different about this particular approach?

Aimee Ten Eyck-Schaffer

While we were hearing from our employees, it was frustrating trying to find what they need. It was frustrating for subject matter experts because they knew they'd produce the documentation. And there was just that gap that we were trying to solve for for them to get from what they were looking for to finding it. And the solution that you provide for AI really solves for that. Not only does it get them into the right procedure regardless of really how they're searching for it, but it also takes them to the specific section or step of the procedure that they were needing. And so as soon as we engaged with the AI for Engageware, folks were finding what they needed directly. It's pretty exciting.

Justin Smith

Yeah, and I think it's neat that, you know, you have this large knowledge base of curated content that you guys have spent a lot of time making sure that the information is updated, validated.

And using the AI to surface up those answers, right, to be able to look through that knowledge base and quickly find the answer. And then like you said, being able to source that back to that grounded, curated content, you have the ability for your employees to drill down even deeper and if they want to look at the full procedure. So it's that blend of leveraging AI to really make the process more efficient, but then grounded in your knowledge base and that curated content.

Aimee Ten Eyck-Schaffer

Exactly, they get that quick hit information answer at the top. And a lot of times that solves what they were looking for without even having to deep dive. But when the need arises to deep dive, it's as easy as a click into that section and reading further.

Justin Smith

Yeah. And so if we got to the next slide, I think some of the metrics that came out of this, this is sort of, I think, really telling here, but it sort of backs up what we just said, right? More searches, fewer clicks, rising engagement is sort of the headline here. But talk us through a little bit about these numbers and what changed for your team post after you guys launched this.

Aimee Ten Eyck-Schaffer

Yeah. So this snapshot is three weeks after we before and after we launched AI. And so you can see a dramatic drop in the portal interactions three weeks after turning AI on, including content views, also a pretty significant drop.

That indicated to us that meant folks have to click around a lot less. They are finding what they need faster. They're not having to click through the different sections of the procedure to hopefully find the answer that they need. The other piece I found fascinating was that total searches actually increased, which meant folks were trusting the system more and engaging with it more. The other fun piece I kind of like to highlight in this data is three weeks after we launched, we started at about seventy-five percent answer rate for receiving an AI response. We've been able to increase that to about eighty percent now by analyzing the reporting that you provide us.

We can take a look and see what folks are trying to find, what AI is telling them, and also where they're not having success in getting an AI answer and start to look at our content and see what we can do to start to get them answers.

Justin Smith

It sounds like this is a constantly evolving refinement, continuous improvement, right? Where, you know, you were able to take a leap forward with the AI, but really that ongoing visibility into what your folks are searching for, how they're using the product, where are the gaps, maybe where are things unclear, where do you need to either create new content or update existing content to continue to just make that employee experience better. Is that fair that that's just an ongoing thing? I would guess it's kind of critical in the organization to have somebody that believes in that and is doing that on a daily basis.

Aimee Ten Eyck-Schaffer

It's absolutely important. And I'd even take it a step further to say, with what we can see in the reporting you provide, we can actually see how folks are talking about process and what they are calling things. And it didn't always match what we were calling it in the documentation. And so it led to really informed updates on our end to make sure we're talking about it in the same way that the users are and really helping to facilitate that match and what they're seeking to find.

Justin Smith

Right. Excellent.

Okay. This is an interesting one where, you know, I think this was maybe a benefit that you got out of using the AI that maybe you didn't predict, maybe you didn't know was going to happen and was going to come. But tell us about some of the hidden knowledge that got discovered through this process.

Aimee Ten Eyck-Schaffer

This one was actually kind of funny to me.

So we turned AI on, to your point, you've said this service is knowledge. We have a very deep knowledge base. And so there's lots of stuff that gets added, that gets communicated to folks, but our days get busy. And as soon as we turned AI on, we found this one nugget around, like how can a member change their PIN? And we have this internal call center that answers calls for folks, gives them information. And they got so excited that this popped up, they didn't even know.

And it had been out there for about six months. So it's a process that we uncovered that folks were excited to socialize once they could find it.

Justin Smith

Yeah, sometimes these tools are smarter than we realize. Yeah.

And then, going to the next slide. So there's a story here around your SME collaboration and your use of short codes. Tell us a little bit about that.

Aimee Ten Eyck-Schaffer

Yeah, so I was just talking about earlier leveraging your reporting. I really like to take a deep dive. It's fun to see what AI answers, but it's even more fun to see what it's not answering.

And one of the things that I noticed was we kept getting these five to six digit searches in there and I couldn't figure out what the purpose of them was. And so as I was analyzing the data along with Copilot, it suggested that potentially they were short codes. And after doing some research, I realized it was. Our employees would have members show up in a branch or call in and say, "I got this message, was this from you?" And so we were able then to take that data, implore our sneeze to capture what short codes they were using so we could put them into the system and employees could validate that those were real and true numbers from OnPoint and not necessarily fraudsters trying to trick them.

Justin Smith

Excellent, cool.

And then Amy, you took another step. So after your team started using the tooling, you did a survey out to those employees to kind of gauge, what's their feedback and are they finding the tool helpful? It sounds like you got some really good results there. Why don't you talk us through some of these? What did you hear? What was the general you know, ten or that you got back from your employees? And you can click through a couple of these bullet points on here, but you could walk us through that, Amy.

Aimee Ten Eyck-Schaffer

Yeah. So I always worry that the number data you see, it's really easy to turn the story into what you want it to be. And so something I really wanted to do when we launched it was to see if the qualitative data matched the quantitative data. So we launched a survey with our organization and the results came back matching what we saw in the numbers, which was folks were super happy that they were able to find their information very quickly. That was a hundred percent score on that.

Eighty percent reported time savings. When we did launch, there is a bit of getting used to, like, searching for more than just a keyword, which is what we trained them to do before. So they were still kind of getting used to that behavior.

They really enjoyed that source usage. So again, you get the wonderful AI answer at the top, but being able to go straight into the section of the procedure you were hoping to find was a huge win.

And then the behavior change, if they didn't find what they wanted using those search words, we were finding that they were trying again with a little bit more detail rather than just giving up and phoning a friend.

Justin Smith

Right. Right. I think that behavioral change is an interesting callout. I mean, it's we're all our behaviors are all changing as far as using ChatGPT, using Claude, and understanding how to interact with these tools and leverage them.

Same thing here, right, where users had a traditional search experience expectation, then you turn on this AI assist tool, they have to adapt on how to use it.

You're not doing a keyword search, you're asking a question.

And the better question you ask, more than likely the better answer you're going to get because you're giving the technology the ability to understand what is this person looking for, right? And then helping them, helping craft that answer. So I think that behavioral change is an interesting callout because I think, you know, we're all going through that just as a society, really over the course of the last year and beyond.

So before we wrap up, what you just walked through and presented to us is really more than a technology implementation. It's kind of a shift in how OnPoint and your employees use these tools, surface critical knowledge, evolve the content and the knowledge base over time.

And so, these key takeaways summarize a good bit of that, but from your perspective, what were the things that really jumped out as far as the most impactful things here?

Aimee Ten Eyck-Schaffer

Probably the two biggest impactful pieces for us were number one, getting employees to information faster. And then number two, we didn't really expect it when we were choosing to add AI, but the data that we get from having it in our system, and the ability that it gives us to enhance and improve our documentation has been invaluable.

Justin Smith

Awesome. Great.

Amy, so thank you so much for sharing your use case and your experience with us. Always great when, you know, we can bring customers to share their experience in their words. And this is a great example of how leveraging AI on top of that controlled curated knowledge source, which again, I think is so key here that, you know, the backbone that the technology is using, that content on the backend is so important that it's curated, that it's up to date, that it's controlled. And I know that you and your team have put a lot of time into that side of it, which then enables the tooling to give great results. So Alice, I think maybe we have time for a few questions.

Alice Milligan

Yeah, I think we have time for a few questions and there are a couple put into the Q and A box. Box. Amy, thanks so much, very insightful.

One of the things I know is true typically of most organizations is that change and implementing something of the nature and scale that you did requires a good amount of buy-in and alignment. So can you tell us a little bit about how you got internal buy-in and alignment, especially from senior leadership, to move forward with what you did? Absolutely.

Well, I

Aimee Ten Eyck-Schaffer

was lucky enough. There is an imperative out there for folks trying to find how to get AI implemented in their organization somehow. And I knew that this was available. So I was able to bring my senior leadership together, demonstrate what this AI tool could offer.

And also given what Justin just said, share how we're still able to provide solid, accurate, reliable information. And just in demonstrating that alone, it was a really easy sell on my part to be able to get permission to launch this. We did do a pilot ahead of signing, which helped. And so that really helped give the confidence that the tool was reliable and successful.

And then we were able to move forward from there.

Alice Milligan

Just an additional question that ties nicely into what you talked about, but you just mentioned, you know, accuracy and reliability.

So how do you make sure the AI-generated answers are accurate? And what does your review process look like?

Aimee Ten Eyck-Schaffer

We were really anxious that AI might get creative or hallucinate or things like that. And so the pilot for us was super important. And effectively what I did was I pulled together our subject matter experts and the folks that oversee our risk. And we went ahead and ran that pilot. I provided them weekly reporting of the results. So it asked them to dedicate fifteen minutes a day just to search and see what came through. And then they were able to evaluate and give feedback on the responses.

We did make some updates to content, but actually what was surprising was that a lot of it was already where we wanted it to be. I think there's always that fear that your content might produce a funny answer or be incorrect. You'll find some weird cobwebs in the corner.

But really it validated for us that we had a solid knowledge base and that we could trust the AI product. The last two weeks, we rolled it out to some targeted locations.

I specifically chose ones that let us know that they were struggling to find stuff because I'd know they'd be the ones to let me know if they didn't like the AI experience. And I had nothing but great responses from them as well. And that gave us the confidence to be able to launch, by I think it was March last year.

Great.

Justin Smith

That's a smart, smart approach, know the crawl, walk, run. You know, you had some internal, like a tight team just sort of test it.

Then you did a launch to a few locations first, get that feedback. So you didn't try to bite it all off at once, you took it in steps and that probably allowed you to kind of iterate as you go. So that's

Aimee Ten Eyck-Schaffer

We're continuing to exercise that approach. It targeted our retail group and maybe like the main stream HR rate resources. Our goal this year is to roll it out through the rest of our internal back office areas that have their own info centers is what we call them. And so they can have their own dedicated AI resource as well. So they're going through that same iteration of test and launch.

Alice Milligan

Thank you so much for your willingness to share your experience and your learnings.

Justin Smith

Yeah. Thanks, Amy, for being such a great partner.

Aimee Ten Eyck-Schaffer

I gotcha.

Alice Milligan

All right. I can feel the love.