Understanding LLMs: What Large Language Models Actually Do

What is an LLM?

A large language model is a type of AI that's been trained to read and generate human language. GPT-4, Claude, Llama, and Gemini are all LLMs. They power chatbots, writing tools, code assistants, and an increasingly long list of business applications.

The "large" part refers to the scale of training. These models have seen billions of web pages, books, articles, and code repositories. They've learned patterns in how language works: grammar, facts, reasoning styles, even tone.

But they don't "know" things the way a person does. They predict the most likely next word based on patterns in their training data. That's a crucial distinction.

How LLMs generate text

At the most basic level, an LLM works by predicting the next token (roughly a word or part of a word) in a sequence. You give it a prompt, and it generates a response one token at a time, each choice influenced by everything that came before it.

This is why they're so fluent. They've seen enough language to know what "sounds right." But it's also why they sometimes make things up with complete confidence. The model doesn't check facts. It generates the most statistically likely continuation.

What LLMs know (and don't)

LLMs have broad general knowledge. They can write about Australian tax law, explain engineering concepts, or draft marketing copy. But their knowledge has limits:

• Training cutoff: They don't know about events after their training data was collected
• No access to your data: They've never seen your internal documents, customer records, or business processes
• No real-time information: They can't check current prices, stock levels, or live systems
• Confidence without accuracy: They'll state incorrect things with the same confidence as correct ones

This is exactly why patterns like RAG exist. They give the model access to your specific, current data at query time.

Business applications

The most practical business uses for LLMs right now fall into a few categories:

• Content generation: Drafting emails, reports, documentation, marketing copy
• Data extraction: Reading invoices, contracts, forms, and pulling structured data out
• Classification: Sorting support tickets, categorising feedback, routing emails
• Summarisation: Condensing long documents, meeting transcripts, research papers
• Knowledge Q&A: Answering questions about internal data (when combined with RAG)
• Code assistance: Writing, reviewing, and debugging code

Key limitations

LLMs are powerful, but they're not magic. Be aware of:

• Hallucinations: They make things up. For business-critical applications, you need guardrails.
• Cost: API calls to frontier models (GPT-4, Claude Opus) cost real money at scale.
• Latency: Complex queries can take seconds, which matters for real-time applications.
• Privacy: Sending data to cloud-hosted models means your data leaves your control, unless you use private deployments.
• Context windows: Models can only process a limited amount of text at once (though this is growing rapidly).

Choosing a model

The model you choose matters less than how you integrate it into your workflows. That said, here's a rough guide:

For most Australian business applications, we find that GPT-4o or Claude work well and integrate easily into RAG systems deployed on AWS. The model can always be swapped later. The architecture matters more.

Kasun Wijayamanna Founder & Lead Developer

Postgraduate Researcher (AI & RAG), Curtin University - Western Australia

View profile →