High-Performance Go Implementation with Multi-Provider LLM Support
4.5K
High-Performance Go Implementation with Multi-Provider LLM Support
Universal LangChain server supporting OpenAI, Claude, Ollama, Llama.cpp, and more with MCP tools integration
Featuresā ⢠Quick Startā ⢠API Documentationā ⢠Examplesā ⢠Dockerā
ā š Multiple LLM Providers
|
ā š§ Advanced Features
|
Pull the pre-built image from Docker Hub:
# Pull the latest image
docker pull jefriherditriyanto/langchain-mcp-api:latest
# Run container
docker run -d \
--name langchain-mcp-api \
-p 6000:6000 \
jefriherditriyanto/langchain-mcp-api:latest
Using Docker Compose:
version: '3.8'
services:
langchain-mcp-api:
image: jefriherditriyanto/langchain-mcp-api:latest
container_name: langchain-mcp-api
ports:
- "6000:6000"
environment:
- PORT=6000
restart: unless-stopped
Run with:
docker-compose up -d
http://localhost:6000
GET /
Response:
{
"message": "š¤ LangChain MCP API is running",
"version": "1.0.0"
}
GET /health
Response:
{
"status": "ok"
}
POST /chat
Request Body:
{
"credential": {
"provider": "openai",
"api_key": "sk-...",
"model": "gpt-4o-mini",
"set": {
"temperature": 0.7,
"max_tokens": 2000
}
},
"system_prompt": "You are a helpful assistant",
"input": "tolong cari dns lookup dari www.example.com, sajikan dalam bahasa indonesia",
"servers": ["http://host.docker.internal:4050"] // this tool available on "MCP Server: Python"
}
Response:
{
"messages": [
{
"role": "user",
"content": "tolong cari dns lookup dari www.example.com, sajikan dalam bahasa indonesia"
},
{
"role": "assistant",
"content": "{\"tool_name\":\"network_dns_lookup\",\"tool_args\":{\"hostname\":\"www.example.com\"}}",
"tool_calls": [
{
"id": "manual_1770445792732616000",
"name": "network_dns_lookup",
"args": {
"hostname": "www.example.com"
},
"type": "tool_call"
}
],
"response_metadata": {
"finish_reason": "stop",
"model_provider": "openai",
"model_name": "gpt-4o-mini",
"usage": null,
"system_fingerprint": ""
},
"usage_metadata": {
"output_tokens": 120,
"input_tokens": 1382,
"total_tokens": 1502
}
},
{
"role": "tool",
"content": "Tool 'network_dns_lookup' SUCCESS: {\"hostname\":\"www.example.com\",\"ip\":\"104.18.26.120\"}",
"tool_call_id": "manual_1770445792732616000",
"name": "network_dns_lookup"
},
{
"role": "assistant",
"content": "Berikut hasil **DNS lookup** untuk `www.example.com`:\n\n- **Nama Domain**: www.example.com \n- **Alamat IP (IPv4)**: 104.18.26.120 \n\n### Penjelasan:\nDNS (Domain Name System) berfungsi untuk mengubah nama domain (seperti `www.example.com`) menjadi alamat IP (seperti `104.18.26.120`) yang digunakan oleh komputer untuk mengakses situs web.",
"response_metadata": {
"finish_reason": "stop",
"model_provider": "openai",
"model_name": "gpt-4o-mini",
"usage": null,
"system_fingerprint": ""
},
"usage_metadata": {
"output_tokens": 611,
"input_tokens": 84,
"total_tokens": 695
}
}
],
"message": "Berikut hasil **DNS lookup** untuk `www.example.com`:\n\n- **Nama Domain**: www.example.com \n- **Alamat IP (IPv4)**: 104.18.26.120 \n\n### Penjelasan:\nDNS (Domain Name System) berfungsi untuk mengubah nama domain (seperti `www.example.com`) menjadi alamat IP (seperti `104.18.26.120`) yang digunakan oleh komputer untuk mengakses situs web.",
"usage_metadata": {
"output_tokens": 731,
"input_tokens": 1466,
"total_tokens": 2197
},
"model_provider": "openai",
"model_name": "gpt-4o-mini",
"finish_reason": "stop",
"total_iterations": 2,
"tool_calls_count": 1,
"execution_time_ms": 26818,
"execution_time_sec": 26.818,
"tokens_per_second": 81.92
}
POST /chat/stream
Request Body: (same as /chat)
Response: Server-Sent Events stream
data: {"type":"start","timestamp":"2024-02-04T09:00:00Z","input":"What is the weather?"}
data: {"type":"servers_checked","available_servers":["http://host.docker.internal:4000"],"total_servers":1}
data: {"type":"thinking_start","timestamp":"2024-02-04T09:00:01Z"}
data: {"type":"thinking_chunk","chunk":"I need to check the weather...","is_final":false}
data: {"type":"message_start","timestamp":"2024-02-04T09:00:02Z"}
data: {"type":"message_chunk","chunk":"The weather is ","is_final":false}
data: {"type":"message_chunk","chunk":"sunny, 28°C","is_final":true}
data: {"type":"done","done":true,"total_steps":3,"timestamp":"2024-02-04T09:00:03Z"}
| Provider | Key | Required Fields |
|---|---|---|
| OpenAI | openai | api_key, model |
| Claude | claude | api_key, model |
| OpenRouter | openrouter | api_key, model |
| Ollama | ollama | url, model |
| Llama.cpp | llama_cpp | url, model |
| vLLM | vllm | url, model |
{
"set": {
"temperature": 0.7, // Creativity (0.0 - 2.0)
"max_tokens": 1000, // Max response length
"top_p": 0.9, // Nucleus sampling
"frequency_penalty": 0.0, // Repetition penalty
"presence_penalty": 0.0, // Topic diversity
"max_context_messages": 4 // History window size
}
}
Build your own MCP (Model Context Protocol) server to provide custom tools for the LangChain MCP API.
Full working example available at:
# Clone MCP server example
git clone https://github.com/jefripunza/langchain-mcp-api/tree/master/mcp-server-bunts-express
# Navigate to MCP server example
cd mcp-server-bunts-express
# Install dependencies
bun install
# Run the server
bun run dev
Server will start at http://localhost:4000 š
mcp-server-bunts-express/
āāā src/
ā āāā index.ts # Main server
ā āāā registry.ts # Tool registry
ā āāā tools/
ā āāā math.ts # Math tools
ā āāā weather.ts # Weather tools
āāā package.json
āāā tsconfig.json
src/index.ts)import express from "express";
import cors from "cors";
import helmet from "helmet";
import morgan from "morgan";
import { tools, findTool } from "./registry";
const app = express();
app.use(express.json());
app.use(cors());
app.use(helmet());
app.listen(4000, () => {
console.log("š§ MCP Server running on http://localhost:4000");
});
app.use(morgan("dev"));
// REQUIRED!
app.get("/health", (_req, res) => res.json({ status: "ok" }));
// MCP-style: list tools, REQUIRED!
app.get("/mcp/tools", (_req, res) => {
res.json(
tools.map((t) => ({
name: t.name,
description: t.description,
parameters: t.parameters,
})),
);
});
// MCP-style: invoke tool, REQUIRED!
app.post("/mcp/invoke", async (req, res) => {
const { name, arguments: args } = req.body;
const tool = findTool(name);
if (!tool) {
return res.status(404).json({ error: "Tool not found" });
}
if (!tool.handler) {
return res.status(400).json({ error: "Tool handler not found" });
}
const result = await tool.handler(args);
res.json(result);
});
src/registry.ts)import type { Tool } from "./types/tool";
import { mathTools } from "./tools/math";
import { weatherTools } from "./tools/weather";
export const tools: Tool[] = [...mathTools, ...weatherTools];
export function findTool(name: string) {
return tools.find((t) => t.name === name);
}
src/tools/math.ts)import type { Tool } from "../types/tool";
export const mathTools: Tool[] = [
{
name: "add",
description: "Add two numbers together", // add to main prompt, please detail!
// add to main prompt, please detail!
parameters: {
type: "object",
properties: {
a: { type: "number" },
b: { type: "number" },
},
required: ["a", "b"],
},
// as controller / logic base
handler: async ({ a, b }: { a: number; b: number }) => {
console.log(`ā
MCP1 Math: ${a}+${b}=${a + b}`);
return { result: a + b };
},
},
];
src/tools/weather.ts)import { fetchWeatherApi } from "openmeteo";
import type { Tool } from "../types/tool";
// advance tool version
export const weatherTools: Tool[] = [
{
name: "getWeather",
description: "Get weather data by coordinates",
parameters: {
type: "object",
properties: {
latitude: { type: "number" },
longitude: { type: "number" },
},
required: ["latitude", "longitude"],
},
handler: async ({
latitude,
longitude,
}: {
latitude: number;
longitude: number;
}) => {
const params = {
latitude,
longitude,
hourly: ["temperature_2m", "relative_humidity_2m", "rain"],
timezone: "auto",
};
const responses = await fetchWeatherApi(
"https://api.open-meteo.com/v1/forecast",
params
);
const response = responses[0];
const hourly = response.hourly()!;
console.log(`ā
MCP1 Weather: ${latitude}, ${longitude}`);
return {
latitude,
longitude,
temperature: hourly.variables(0)!.valuesArray(),
humidity: hourly.variables(1)!.valuesArray(),
rain: hourly.variables(2)!.valuesArray(),
};
},
},
];
Your MCP server must implement these three endpoints very required:
| Endpoint | Method | Description |
|---|---|---|
/health | GET | Health check |
/mcp/tools | GET | List all available tools |
/mcp/invoke | POST | Execute a specific tool |
# List available tools
curl http://localhost:4000/mcp/tools
# Invoke math tool
curl -X POST http://localhost:4000/mcp/invoke \
-H "Content-Type: application/json" \
-d '{
"name": "add",
"arguments": {"a": 5, "b": 3}
}'
# Invoke weather tool
curl -X POST http://localhost:4000/mcp/invoke \
-H "Content-Type: application/json" \
-d '{
"name": "getWeather",
"arguments": {"latitude": -6.2, "longitude": 106.8}
}'
Once your MCP server is running, use it with the LangChain MCP API:
curl -X POST http://localhost:6000/chat \
-H "Content-Type: application/json" \
-d '{
"credential": {
"provider": "openai",
"api_key": "sk-...",
"model": "gpt-4o-mini"
},
"input": "What is 25 + 37?",
"servers": ["http://host.docker.internal:4000"]
}'
The LangChain MCP API will automatically:
Control detailed execution logs with the VERBOSE environment variable:
# Enable verbose logging (shows all requestID-prefixed logs)
export VERBOSE=true
go run main.go
# Or with Docker
docker run -d \
-p 6000:6000 \
-e VERBOSE=true \
jefriherditriyanto/langchain-mcp-api:latest
# Disable verbose logging (hides requestID-prefixed logs)
export VERBOSE=false
go run main.go
# Or simply don't set the variable (defaults to false)
go run main.go
When VERBOSE=true, you'll see detailed execution traces:
[1kqlh2PxHZZvLVLiBbdbWfdXQ9] [START REQUEST]
[1kqlh2PxHZZvLVLiBbdbWfdXQ9]š¦ [AGENT] Creating LangChain Agent...
[1kqlh2PxHZZvLVLiBbdbWfdXQ9] Provider: llama_cpp
[1kqlh2PxHZZvLVLiBbdbWfdXQ9] Model: gpt-oss-20b.gguf
[1kqlh2PxHZZvLVLiBbdbWfdXQ9] ā
Loaded 22 tools from MCP servers
[1kqlh2PxHZZvLVLiBbdbWfdXQ9]š [INVOKE] Starting agent invocation...
[1kqlh2PxHZZvLVLiBbdbWfdXQ9] Input: What is the weather?
[1kqlh2PxHZZvLVLiBbdbWfdXQ9] š [ITERATION 1/10]
[1kqlh2PxHZZvLVLiBbdbWfdXQ9] š Built 2 messages for LLM
[1kqlh2PxHZZvLVLiBbdbWfdXQ9] š¤ Calling LLM...
[1kqlh2PxHZZvLVLiBbdbWfdXQ9] ā
LLM Response (245 chars)
[1kqlh2PxHZZvLVLiBbdbWfdXQ9] š§ Detected 1 tool call(s)
[1kqlh2PxHZZvLVLiBbdbWfdXQ9] 1. getWeather({"lat": -7.7, "lon": 109.0})
[1kqlh2PxHZZvLVLiBbdbWfdXQ9] āļø Executing tools...
[1kqlh2PxHZZvLVLiBbdbWfdXQ9] [1/1] Executing: getWeather
[1kqlh2PxHZZvLVLiBbdbWfdXQ9] ā
Success from http://host.docker.internal:4000
[1kqlh2PxHZZvLVLiBbdbWfdXQ9] ā
Tools executed successfully (1 results)
[1kqlh2PxHZZvLVLiBbdbWfdXQ9] ā
[INVOKE] Agent invocation completed
curl http://localhost:4000/healthmax_context_messages (default: 4)-N flag with curl for streaming| Metric | Value |
|---|---|
| Startup Time | < 1s |
| Memory Usage | ~50MB (idle) |
| Concurrent Requests | 1000+ |
| Response Time | < 100ms (without LLM) |
MIT License - see LICENSEā file for details
ā
Jefri Herdi Triyanto (@jefripunzaā )
Content type
Image
Digest
sha256:d0943948cā¦
Size
38.7 MB
Last updated
8 months ago
docker pull jefriherditriyanto/langchain-mcp-api