… This is the seventh in a series of posts about how I ended up where I am today.
One of the most hated jobs in Level 2 support is managing the queues. The department is made up of teams who specialise in certain products or areas. I worked in the Application Development support team, but tickets came in for everything.
While on the queue your job is to read every incoming support ticket and figure out where it should go. Which department should handle it. Whether Level 1 (if that’s where it came from) did their due diligence, and if not, follow up. Whether the customer is premium, because only certain people could handle those. Whether the customer has an ongoing relationship with a specific engineer. And then, out of whoever is left, who in that department is the best fit based on workload and specialities.
It was time-consuming, thankless work. When tickets got routed to the wrong group or the wrong person, there were arguments. One person on the team was exceptional at it, could read a ticket and know exactly where it needed to go almost instantly. But for the most part nobody wanted queue duty. People wanted to solve customer problems, not do paperwork.
“You Can’t Automate That”
During a lunchtime conversation about all of this with one of my colleagues, I said why can’t we just automate it. He said it wasn’t possible. You’d need to understand the technology behind each ticket, and simple keyword matching would never be good enough. Too much nuance.
So I wrote out a specification to prove him wrong.
I detailed how we could use NLP trained on existing support tickets to find the best route for incoming ones, combined with an algorithm that could pick the best engineer for the job based on workload, speciality, and past performance.
What I didn’t know when I handed it to him is that he sent it to senior management. They saw merit in what I’d designed and gave me time and a couple of people to help build it out.
93%
Using LanguageWare and the backlog of previous tickets, I created an application to build the NLP dictionary. The team helped test and curate the results. We ran it against real routing decisions to see how it compared.
Your average engineer doing queue duty was about 68% accurate in routing tickets to the right place. The one expert we had, the person everyone wished was on queue permanently, ran at 94%. The routing engine came in at 93%.
Couldn’t beat the expert. But it was close enough to show real promise, and it was consistent. It didn’t have bad days, didn’t get distracted, didn’t resent being on queue duty.
The Side Effects
One of the more interesting things that came out of the engine wasn’t in the original plan. When we took a support ticket and cross-referenced it against the support knowledge base, the system could surface the most relevant help documents. And when we pointed it at source code, it could identify the files most likely to contain the area where a fix should happen.
That was a genuine surprise. We’d built it to route tickets, and it turned out the underlying technology could do something much broader. It could connect a customer’s problem description to the actual knowledge and code that would solve it.
What Came Next
The project didn’t go further. What I didn’t know at the time was that IBM Research were already working on something called Watson, which would be considerably more powerful than anything I could build with LanguageWare and custom algorithms.
But the work wasn’t wasted. Building that system taught me how to think about the gap between what humans do well and what machines can take over. The expert running at 94% wasn’t doing keyword matching. They were drawing on years of context, relationships, and instinct. Getting a machine to 93% meant understanding what they were actually doing and finding a way to approximate it. That’s a problem I’d spend the next decade working on in different forms.
It also caught the attention of the people building Watson. And that’s how I ended up in the conversation that would define the next chapter of my career.
This is the seventh in a series of posts about how I ended up where I am today. Next: the pivot from conversational AI to agentic systems, and why everything before this was preparation for what came after.