ai-ml

Function Calling: What It Is and How It Applies to AI Products

An LLM capability to invoke external functions or APIs by generating structured JSON arguments, enabling reliable tool use in agentic workflows.

Function calling is an LLM capability that allows the model to invoke external functions or APIs by generating structured JSON arguments, enabling reliable tool use in agentic workflows. Rather than asking an LLM to produce a free-text answer, function calling instructs the model to select from a set of defined functions and output the arguments needed to call them in a structured, machine-parseable format. The calling application then executes the function and feeds the result back into the conversation. This seemingly simple capability is what makes AI agents practically reliable. Without function calling, getting an LLM to invoke tools required parsing natural language output that might be formatted inconsistently. With function calling, the LLM produces a deterministic JSON structure that the application can parse, validate, and execute confidently. Understanding function calling, how it works across different model providers, and how to design function schemas effectively is foundational for anyone building agentic AI products. For UK product teams, the security boundary function calling creates is directly relevant to FCA explainability requirements and ICO accountability guidance, since every tool invocation is mediated through application code that can enforce access controls and log the full audit trail. Prompt injection attacks, where malicious content in retrieved documents attempts to hijack function calls, are the primary security risk to mitigate in production agentic systems. SpeedMVPs builds function-calling AI agents for UK and EU founders, delivering secure and auditable automation in 2-3 weeks from GBP 8,000 with full code ownership on handover.

What Is Function Calling: A Plain-English Definition

Function calling is a feature offered by major LLM providers, including OpenAI, Anthropic (as tool use), and Google, that allows developers to define a set of functions the model can invoke during a conversation. These functions are described in a structured schema, typically JSON Schema format, that tells the model the function's name, what it does, and what arguments it takes. When the LLM determines that invoking a function would help answer the user's request, rather than generating a natural language response, it generates a structured JSON object specifying which function to call and what arguments to pass. The application receives this output, executes the actual function (whether that is a database query, an API call, a calculation, or any other operation), and sends the result back to the model as part of the conversation. The model then uses the function result to generate a final response. The term 'function calling' is used by OpenAI. Anthropic uses the term 'tool use' for the equivalent capability. The underlying mechanism is the same: the model outputs structured arguments for an externally defined action rather than free text. Crucially, the LLM does not actually execute the function. It only decides which function to call and what arguments to provide. The execution happens in the calling application, which means the developer retains full control over what code runs. The LLM is making a structured decision, not taking direct action on external systems. Function calling is the technical foundation for AI agents, copilots that can perform actions on a user's behalf, and any AI product that needs to interact with external APIs, databases, or services rather than just generating text responses.

How Function Calling Works

The function calling lifecycle has four steps. First, the developer defines one or more function schemas in the API request. Each schema includes the function name, a description of what the function does, and a JSON Schema definition of its parameters with descriptions and types. Second, the LLM processes the user message and the available function definitions and decides whether to call a function or respond with text directly. If it decides to call a function, it outputs a structured response containing the function name and the arguments it wants to pass. Third, the calling application receives the function call specification, validates the arguments, and executes the actual function. This is where real work happens: an API is called, a database is queried, a calculation is performed. Fourth, the function result is sent back to the LLM as a message in the conversation, and the model generates a final natural language response incorporating that result. A concrete example: a fintech startup built an AI assistant for expense management. The assistant has access to three functions: get_transactions (which queries the database for transactions matching criteria), get_budget_status (which returns current spend against budget categories), and categorise_transaction (which updates a transaction's category). When a user asks 'How much did we spend on travel last month versus budget?', the LLM calls get_transactions with date and category parameters, then calls get_budget_status for the travel category. The results are returned and the model produces a natural language summary comparing actual spend against budget. The design keeps all data access in the application layer, ensuring that only data the application authorises to expose is accessible to the model, an important control from a GDPR data minimisation perspective.

Why Function Calling Matters for AI Product Development

Before function calling existed, connecting LLMs to external systems required instructing the model to output text in a specific format (such as 'To call a function, write CALL: function_name(args)') and then parsing that text output with regular expressions or string matching. This was brittle. Models occasionally deviated from the expected format, the parsing code needed to handle many edge cases, and the overall reliability was significantly lower than what users expected from a production product. Function calling changes this fundamentally. The model is trained to produce valid JSON against the provided schema. Modern models are reliable at this, and providers offer 'strict' mode options that guarantee the output conforms to the schema. This reliability is what makes production-grade AI agents possible. For product teams, function calling enables several valuable capabilities. AI copilots that take actions on behalf of users, updating records, sending messages, creating calendar entries, become practical. AI customer service agents that can look up order status, initiate returns, or check inventory move from fragile demos to dependable production systems. Financial AI tools that query live data rather than relying on the LLM's potentially stale training knowledge produce accurate, real-time responses. Function calling also provides a natural security boundary. Because the application controls which functions are defined and executes them itself, it can enforce authorisation checks, rate limits, and audit logging around every action the AI takes. This is directly relevant for FCA-regulated fintech products and any application where AI actions need to be traceable.

Common Use Cases for Function Calling in AI Products

Database queries are one of the most common function calling use cases. Rather than training a model to write SQL directly, developers define functions like search_records, get_customer, or filter_orders, and the model calls them with natural language-derived parameters. The application executes validated queries and returns results. API integrations enable AI assistants to interact with third-party services. A travel booking assistant might have functions for search_flights, check_availability, and get_pricing. A business intelligence tool might have functions for get_metric, filter_by_date, and compare_periods. Calculations and data transformations that require precision beyond what LLMs are reliable for, such as financial calculations, date arithmetic, or statistical operations, are handled cleanly via function calling. The LLM decides what to calculate and with what inputs; the application performs the actual computation. Workflow automation products use function calling to let an AI orchestrate multi-step business processes: create_task, assign_to_user, update_status, send_notification. Each action is a defined function the model can call in sequence. Search and retrieval within RAG applications is increasingly implemented via function calling, where the LLM can decide when and with what query to call a search function, rather than always retrieving at the start of a request. For regulated sectors, function calling is particularly valuable because it creates a clear audit trail. Every action the AI takes is mediated through a defined function call that the application logs. In FCA-regulated contexts, this supports the requirement to explain and audit automated decisions.

Related Concepts

Tool use is the term Anthropic uses for the same capability in Claude models. The concept is identical: define tools the model can invoke, receive structured invocation outputs, execute them in the application, and return results. The implementation details differ slightly across providers, but the pattern is the same. AI agents depend on function calling as their primary mechanism for acting on the world. Without reliable function calling, building agents that take meaningful actions requires fragile output parsing. Function calling is what moved agents from research demonstrations to production-viable products. Agentic workflows use function calling extensively. A multi-step workflow where an agent plans, executes, evaluates, and iterates is fundamentally a series of function calls mediated by the LLM's reasoning. Structured output is a related capability. Where function calling focuses on invoking external functions, structured output constrains the LLM to return a specific JSON schema as its final response, without implying external execution. Both capabilities use JSON Schema definitions and improve reliability over free-text output. AI orchestration frameworks like LangChain and LlamaIndex provide abstractions for registering tools and handling function calling loops, making it easier to build applications that use many functions without writing the dispatch logic manually. For UK compliance, function calling's explicit audit trail is directly relevant to EU AI Act transparency requirements and UK GDPR obligations around automated decision-making. When an AI takes an action via function calling, the full context, which function was called, with what arguments, and what result was returned, can be logged for compliance purposes.

Frequently Asked Questions

What is the difference between function calling and structured output?+

Function calling instructs the LLM to produce structured arguments for an externally defined function that the application will execute. The intent is to trigger real-world actions or data retrieval. Structured output constrains the LLM's final response to conform to a specific JSON schema, without implying any external execution. Use function calling when you need the LLM to decide which actions to take and what data to retrieve; use structured output when you want the LLM's answer to be reliably parseable by your application code.

Can I use function calling to let the LLM query my database directly?+

Not directly. The LLM generates the function arguments but does not execute anything itself. You define a function like search_customers with parameters for name and date range, and when the LLM calls it, your application code runs the actual database query. This is intentional and beneficial: your application validates the arguments, enforces access controls, and executes only safe, parameterised queries. Letting an LLM generate raw SQL directly would create SQL injection risks and make authorisation enforcement very difficult.

How do I write good function descriptions for LLMs?+

Function descriptions are prompts for the model's function selection behaviour. Be explicit about what the function does, when to use it, and what the parameters represent. Include example values in parameter descriptions where helpful. Make the distinction clear when two functions have overlapping purposes. Avoid vague names; a function called get_data is harder for the model to use correctly than get_customer_orders_by_date_range. Test your function schemas with realistic queries to identify cases where the model selects the wrong function or produces invalid arguments.

How reliable is function calling in production?+

With current top-tier models and strict mode enabled, function calling is highly reliable for well-defined schemas. Errors are most likely to occur when function descriptions are ambiguous, when two functions are too similar and the model selects the wrong one, when the user's request maps ambiguously to function parameters, or when the required information for a parameter is not present in the conversation. Production systems should validate all function arguments before execution and handle cases where the model produces a function call that cannot be executed cleanly.

Does function calling have security implications I should be aware of?+

Yes. Prompt injection attacks attempt to manipulate an LLM into calling functions with attacker-specified arguments by embedding instructions in user-supplied or external content. For example, a malicious document in a RAG pipeline might instruct the model to call delete_record with specific IDs. Mitigations include validating all function arguments server-side, requiring explicit user confirmation before irreversible actions, limiting the functions exposed to what is strictly necessary for the use case, and treating the LLM's function call outputs as untrusted input that your application code validates before execution.

SpeedMVPs builds production AI products that use function calling to integrate securely with your existing systems and APIs, delivered in 2-3 weeks from GBP 8,000 with full code ownership. Get a free consultation at speedmvps.co.uk

Get a Free Quote