What are large language models? A guide to LLMs in CX

What started as a fun tool to generate text and images, large language models (LLMs) have quickly evolved into so much more. According to Sam Altman, OpenAI’s CEO, different generations use these AI tools in different ways.

• Older generations treat LLMs as a Google replacement.

• Millennials lean on LLMs as a life advisor.

• Gen Z uses them like an operating system. 

But in the business world, where exceptional customer experiences can set you apart from competitors,AI agents are the next generation of customer service bots. These chatbots, powered by LLMs, are essential for keeping pace as AI advances, not an optional luxury feature.

In this guide, we’ll break down how large language models work, advantages and limitations to consider, and how they’re reshaping the future of customer service.

More in this guide:

• How do large language models work?
• Advantages and challenges of LLMs
• Large language model use cases
• How are LLMs trained?
• The future of large language models
• Frequently asked questions
• Build your own LLM-based bot in minutes

How do large language models work?

Large language models (LLMs) are built on transformer-based deep learning architecture, a structure designed specifically to handle sequential data like language. 

Within this architecture is a mechanism called self-attention, which allows the model to evaluate every word in a sentence relative to the others. Instead of just looking at the last few words, self-attention weighs the importance of all words in a sentence, no matter where they appear, giving the model a broader and more accurate understanding of meaning, context, and relationships.

Here’s an example of how it works in practice:

As LLMs like GPT-4 are trained on massive datasets pulled from books, websites, articles, and more, the self-attention mechanism helps the model learn how language behaves in real-world use.

It sees how tone shifts, how word order changes meaning, and how to keep ideas coherent over long stretches of text. The result is a model with billions of parameters capable of understanding and generating responses that mimic human writing with surprising accuracy.

This structure is what powers tools like Zendesk AI agents. When you interact with an LLM through a conversational interface, the model:

1. Processes your input

2. Assesses context through self-attention

3. Generates a highly relevant, fluid reply

That’s why modern LLMs engage rather than just respond. They can summarize information, answer follow-up questions, adapt tone, and even clarify confusion in a way that feels natural.

Through conversational AI, Zendesk AI agents tap into these models to understand nuanced customer requests and handle queries that used to demand a human agent. From the very first interaction, they interpret customer intent with context and clarity. 

Unlike outdated chatbot systems that rely on rigid scripts, these agents can follow complex dialogue, stay on topic, and resolve customer issues without sounding robotic or frustrating the user.

Advantages and challenges of LLMs

With every new technology comes benefits and pain points. With the right guardrails and security certifications in place, the benefits of AI and LLMs outweigh the challenges.

The flexibility of large language models makes them valuable across industries. In addition to customer service, LLMs are being used for fraud detection in finance, clinical documentation in healthcare, legal contract analysis, marketing content generation, and more. But in CX, their benefits are especially tangible in the following ways:

• AI agents: Businesses can automatically and autonomously respond to all customer interactions, no matter the time of day or how complex a request is.
• Knowledge base creation: Using Zendesk generative AI tools and proprietary systems, teams can identify missing support articles and use LLM-powered tools to instantly populate AI-powered knowledge base articles from basic information.
• Agent assistance tools: Tools that leverage OpenAI’s LLMs help agents create macros, summarize tickets, make personal recommendations, expand reply options, and more.
• Voice tools: AI solutions trained on large language models enable teams to automatically summarize calls and generate transcripts.

To effectively leverage the benefits of large language models, it’s crucial to choose the right AI provider. Keep these considerations in mind when selecting a solution to help you navigate common challenges:

• Security:LLMs can unintentionally expose sensitive data if they’re not deployed with proper protections. Zendesk offers enterprise-grade controls, like Advanced Data Privacy and Protection, to keep interactions secure.
• Hallucinations: If left unchecked, LLMs are prone to producing inaccurate outputs. Guarantee your AI as a service (AIaaS)provider can set parameters and choose a bot designed for your specific use case. For example, AI agents are purpose-built for CX, so they can provide accurate responses to complex questions, unlike other options.
• Time to value: Deploying an LLM-powered bot requires a transformer, hardware, software, and expertise. Invest in a solution like Zendesk to take your deployment from days to minutes without needing a technical team to implement and launch LLM technology.

LLMs aren’t perfect, but when paired with purpose-built design and transparent oversight, they unlock more reliable automation that improves customer satisfaction and reduces operational strain.

Large language model use cases

The real-world applications of large language models extend well beyond basic chatbot functionality, like speaking conversationally without training and adapting to brand tone and voice. AI in customer service shows LLM-powered bots evolving into autonomous agents capable of end-to-end support resolution.

According to the Zendesk Customer Experience Trends Report 2025, 87 percent of CX Trendsetters will design an experience for consumers to use AI assistants across the customer journey by 2027.

It’s no small wonder, then, how many organizations are jumping on the LLM-powered bot bandwagon. Zendesk AI agents offer a clear example of what’s possible with a conversational interface built on large language models:

• Scale your support while lowering the cost per interaction: AI agents allow teams to process and answer more support tickets and handle unexpected service spikes without increasing headcount or straining budget or staffing resources.
• Build a bot in minutes: By connecting your knowledge base to a pre-trained generative AI solution like Zendesk, your bot can immediately start helping customers naturally and conversationally with little to no technical training required.
• Offer customers 24/7 support: Bots don’t need to sleep, eat, or take vacations. They can provide round-the-clock customer care and instant resolutions for queries such as checking the status of an order, requesting transaction info, or changing a password. Autonomous bots like AI agents can even resolve more complex interactions from start to finish.
• Provide multilingual support: LLM-based bots and AI agents can instantly translate the contents of your knowledge base and converse with native-level proficiency, providing the same level of customer service in all interactions, no matter your customer’s preferred language. They do this without sifting through FAQ pages or producing disjointed, confusing dialogue like an earlier-generation chatbot might.
• Make your human agents’ jobs easier (and more rewarding): Automation elevates the role of your human support agents into AI managers and editors. With LLMs handling routine interactions, support teams can focus on work that requires problem-solving, nuance, and empathy. This reduces employee turnover, opens up space for upskilling, and powers more rewarding careers.

While this is not an exhaustive list of LLM use cases, these are the most common and rewarding ways AI and automation can be implemented.

How are LLMs trained?

Large language models can have billions of parameters with nodes, layers, weights, and biases working together to make one model. Using vast datasets, LLMs apply self-learning techniques to predict the next token in a sequence.

When a prediction is incorrect, the model adjusts its internal parameters to improve future predictions. Over time, this results in a model capable of generating coherent, context-aware text.

The training process for LLMs is complex, involving several distinct stages:

• Pre-training: The model learns from massive datasets pulled from books, articles, forums, and code repositories. This helps it develop a general understanding of grammar, tone, and structure.
• Fine-tuning: After pre-training, LLMs can be customized with industry- or task-specific data. This narrows the model’s focus, improving accuracy for targeted applications like customer support or legal analysis.
• Zero-shot learning: Base LLMs respond to requests without specific examples. This type of learning typically leads to varied response accuracy.
• Few-shot learning: LLMs receive a few examples, which allow them to improve their responses in specific situations based on new data.
• Chain-of-thought prompting: This approach guides the model through step-by-step reasoning processes to solve complex problems with greater clarity.
• Multimodal training: Newer large language models are being trained not just on text, but on images, audio, and even video, enabling them to understand and generate content across formats.

This is why generative AI tools based on large language models are so adaptable. They’re not rule-based systems. They’re probabilistic, flexible, and continuously improving as new data and prompting techniques are introduced.

The future of large language models

While it may not have the same ring as LLMs, we expect that “reasonably sized” language models will overtake the current model in the years to come. Using generative AI, these language models can run on tens of billions of parameters instead of hundreds of billions.

Medium and small language models will be cheaper to run, making it easier to see and reap the benefits of CX automation.

As we look to the future, we also anticipate:

• Verticalized models trained on domain-specific datasets for finance, legal, healthcare, and other regulated industries

• Multimodal capabilities allow models to work with images, video, and voice alongside text

• Smarter assistants that go beyond simple Q&A to handle tasks, schedule actions, and integrate with third-party systems

• Faster fine-tuning of workflows, making it easier for companies to adapt LLMs to their tone, brand, and use case

• The continued sophistication of AI assistants, including Alexa and Siri

LLMs are constantly evolving in size and ability, but the focus is shifting toward precision, speed, and integration. That’s why it’s important for teams using this technology to understand their current and future capabilities.

Frequently asked questions

Build your own LLM-based bot in minutes

Large language models and AI chatbotscan be complicated to figure out, but Zendesk AI makes it easy. Our solution is pre-trained on large language models and over 18 million real customer service interactions. Within minutes, you can launch Zendesk AI agents that handle up to 80 percent of interactions automatically, without sacrificing quality or customer trust.

Zendesk delivers a clear path to value. Automate more, stress less, and deliver accurate, conversational AI experiences your customers will actually enjoy. Try Zendesk AI agents today.