When Two Things Seem the Same, But Aren’t: Functions Calling vs. MCP

18.04.2025

With the rapid adoption of large language models (LLMs), it became clear early on that real-world enterprise applications require something more than general AI knowledge. To be genuinely useful, especially within companies, models need to work not just with their own training data, but with internal company data—data that lives behind firewalls, inside intranets, databases, Jira, SharePoint, and other tools.

Moreover, companies have proprietary knowledge, workflows, and methods that aren't publicly available and thus unknown to any LLM out of the box.

To bridge this gap, developers have created techniques to extend LLMs with external tools, data, and logic. In this article, we'll explore two such methods:

🧩 Function Calling – great for lightweight and flexible integrations.
🔗 Model Context Protocol (MCP) – a structured, standardized way to manage tools, state, and long-running interactions.

🛠️ Method 1: Function Calling

Function Calling is available through the OpenAI Assistants API and was discussed in this previous article. It's a simple way to tell the model: "Hey, you can call this function whenever it's relevant."

📋 Defining a Function

You describe your function in a JSON schema that includes:

{
  "FunctionName": "get_eu",
  "Description": "Retrieve fields of JSON data from external vector DB",
  "Parameters": {
    "type": "object",
    "required": ["text", "limit"],
    "properties": {
      "text": {
        "type": "string",
        "description": "The input text to retrieve data"
      },
      "limit": {
        "type": "string",
        "description": "How many similar chunks to retrieve"
      }
    },
    "additionalProperties": false
  },
  "StrictParameterSchemaEnabled": true
}

Description tells the LLM when it should call this function.
Parameters define the required inputs and help the model construct valid calls.

⚙️ Implementing the Function

Here's an example using FastAPI and a vector database:

@app.post("/eu")
async def eu(request: Request):
    data = await request.json()
    limit = int(data.get("limit", 5))
    text = data.get("text")

    conn = psycopg2.connect(...)  # connect to PG vector DB
    cursor = conn.cursor()

    embedding = get_embedding2(client, normalize_text(text), "text-embedding-3-small")
    embedding_str = np.array(embedding).tolist()

    query = f"""
        SELECT id, file, page, text_chunk, ...
        FROM embeddings
        ORDER BY embedding <#> '{embedding_str}'::vector
        LIMIT {limit}
    """
    cursor.execute(query)
    results = [dict(zip([desc[0] for desc in cursor.description], row))
               for row in cursor.fetchall()]
    return json.dumps(results, ensure_ascii=False, indent=4)

This function returns a list of semantically similar chunks from a vector store based on the input prompt.

🤖 Smart Behavior by the LLM

Here's the magic: the LLM can intelligently decide to call the function multiple times, adjust parameters dynamically, and compile results into a coherent response. For instance, it might:

Ask for more documents if the results seem sparse.
Choose how many "similarities" to retrieve.
Combine results into a refined final answer.

🔄 Flow Diagram

User
  │
  ▼
OpenAI Assistant (LLM)
  │
  ▼
If function match found → Call external function via API
Else → Proceed with default response

✅ Pros

Easy to implement and use
The model generates arguments automatically
Highly flexible

❌ Cons

Functions must be managed manually
No built-in support for persistent sessions or state
No standard – implementations may vary

🔌 Method 2: Model Context Protocol (MCP)

MCP is a new approach aiming to standardize tool integration and context management across agents and LLMs. Unlike function calling (which is ad hoc), MCP introduces a formal, persistent connection between the LLM and its tools.

It's already being adopted by major players in the AI space—including Anthropic (Claude) and others.

🔄 How MCP Works

Rather than ad hoc JSON and REST, MCP introduces:

A structured, long-lived protocol
Persistent connections (not just per-request)
Built-in context awareness, memory, and task management

🧠 Block Architecture

User
  │
  ▼
MCP Router / Orchestrator
  ├── Context: User input, preferences
  ├── History: Dialogue state
  ▼
Task Decomposition & Parallel Agent Assignment
  ├── Travel Agent
  ├── Booking Agent
  └── Knowledge Agent
      │
      └── External API Calls (hotels, flights, data)
  ▼
Context Merge & Final Answer
  ▼
User Response

📌 Real-World Example

You can find official examples here: modelcontextprotocol.io/examples

Note: the examples use npx for transport and require Node.js ≥ v22. Older versions (e.g., v14) may not work correctly.

✅ Pros

Standardized protocol across tools and agents
Deep contextual awareness (not just prompt-level)
Enables complex workflows, chaining, and memory
Scales well for multi-agent architectures

❌ Cons

Slightly steeper learning curve
Still emerging—limited support outside leading vendors

🆚 Summary: Function Calling vs. MCP

Feature	Function Calling	MCP (Model Context Protocol)
Definition	JSON-based function schema	Structured, session-based protocol
Focus	Individual API calls	Full context-aware interactions
Context Awareness	Prompt-level only	Session + memory + tools
Flexibility	High (manual)	Higher (but standardized)
Ecosystem	OpenAI, LangChain, etc.	Claude, open orchestrators
Output	Function call responses	Stateful dialog + actions

🎯 When Should You Use What?

Use Function Calling when:
- Building a simple assistant or MVP
- Quickly testing integrations
- You need full control over function behavior
Use MCP when:
- Developing a full-scale AI agent
- You need persistent state or memory
- You're orchestrating multiple tools or agents
- You want to future-proof your architecture

Have you tried either of these in your projects? Drop a comment or share your thoughts—I'd love to hear how others are integrating LLMs into real-world systems. 🚀

👉 Bonus: Check out the full vector DB example in the blog post here