Connect an LLM to a Live Web API

A standard language model is like a snapshot of the internet at a certain point in time. It doesn’t know about today’s stock prices, current system alerts, or code updates from a few minutes ago. To unlock the real potential of AI, you need to connect LLMs to the outside world. In this article, I’ll show you how to connect an LLM to a live web API to fetch trending GitHub repositories and use the LLM to analyze which ones are worth learning for AI.

Connect an LLM to a Live Web API: Getting Started

What we’re building has two main parts. First, we’ll get real-time data from GitHub using their public API. This gives us structured and current information.

Next, we’ll send that data to an LLM, which will turn the raw information into useful insights.

This approach; retrieve, reason, and generate, is the basis for many real-world systems, like RAG pipelines and AI agents.

Before we start coding, you’ll need two main dependencies:

requests for calling APIs
transformers for running a local LLM

Install them with:

pip install requests transformers torch

Step 1: Fetching Live Data from GitHub

Let’s begin with the step that makes this system dynamic; pulling in real-time data:

import requests

def get_trending_repos():
    url = "https://api.github.com/search/repositories"

    params = {
        "q": "machine learning",
        "sort": "stars",
        "order": "desc",
        "per_page": 5
    }

    response = requests.get(url, params=params)

    if response.status_code != 200:
        raise Exception(f"GitHub API Error: {response.status_code} → {response.text}")

    data = response.json()

    repos = []
    for repo in data["items"]:
        repos.append({
            "name": repo["name"],
            "description": repo["description"],
            "stars": repo["stargazers_count"]
        })

    return repos

import requests

def get_trending_repos():
    url = "https://api.github.com/search/repositories"

    params = {
        "q": "machine learning",
        "sort": "stars",
        "order": "desc",
        "per_page": 5
    }

    response = requests.get(url, params=params)

    if response.status_code != 200:
        raise Exception(f"GitHub API Error: {response.status_code} → {response.text}")

    data = response.json()

    repos = []
    for repo in data["items"]:
        repos.append({
            "name": repo["name"],
            "description": repo["description"],
            "stars": repo["stargazers_count"]
        })

    return repos

We are using GitHub’s search API to find machine learning repositories, sorted by popularity.

It’s important to note that we don’t send raw JSON to the model. Instead, we pull out only the key details:

Repository name
Description
Star count

Structuring API data before sending it to an LLM is crucial. If you skip this, your prompts get messy and the results aren’t as good.

Step 2: Loading a Local Language Model

Next, we’ll set up a basic text-generation model using Hugging Face:

from transformers import pipeline

llm = pipeline("text-generation", model="gpt2")

from transformers import pipeline

llm = pipeline("text-generation", model="gpt2")

This sets up a simple LLM pipeline that can generate text from a prompt.

In this example, GPT-2 isn’t instruction-tuned, so it doesn’t follow instructions as well as newer chat models. That’s fine for this tutorial, but it means prompt design matters more.

Step 3: Turning Data into Insights

Now for the interesting part, we will feed structured data into the model:

def generate_insights(repos):
    repo_text = "\n".join([
        f"{r['name']} ({r['stars']}⭐): {r['description']}"
        for r in repos
    ])

    prompt = f"""
    Analyze these GitHub repositories and suggest which ones are most useful for learning AI:

    {repo_text}
    """

    result = llm(prompt, max_length=200, num_return_sequences=1)

    return result[0]["generated_text"]

def generate_insights(repos):
    repo_text = "\n".join([
        f"{r['name']} ({r['stars']}⭐): {r['description']}"
        for r in repos
    ])

    prompt = f"""
    Analyze these GitHub repositories and suggest which ones are most useful for learning AI:

    {repo_text}
    """

    result = llm(prompt, max_length=200, num_return_sequences=1)

    return result[0]["generated_text"]

Here, we converted the structured data into a block of natural language. This is a common technique in LLM systems.

Instead of making the model read JSON, we turn the data into plain text, which it understands best.

Step 4: Orchestrating Everything Like an Agent

Now, let’s put everything together into a simple pipeline:

def ai_github_agent():
    print("🔍 Fetching trending repositories...")
    repos = get_trending_repos()

    print("\nTop Repos:")
    for r in repos:
        print(f"- {r['name']} ({r['stars']}⭐)")

    print("\n🤖 Generating AI insights...")
    response = generate_insights(repos)

    print("\nAI Output:\n")
    print(response)

if __name__ == "__main__":
    ai_github_agent()

def ai_github_agent():
    print("🔍 Fetching trending repositories...")
    repos = get_trending_repos()

    print("\nTop Repos:")
    for r in repos:
        print(f"- {r['name']} ({r['stars']}⭐)")

    print("\n🤖 Generating AI insights...")
    response = generate_insights(repos)

    print("\nAI Output:\n")
    print(response)

if __name__ == "__main__":
    ai_github_agent()

When you run the script, three things happen in order:

Live data is fetched from GitHub.
The data is formatted into a prompt.
The LLM generates insights.

This setup is basically a simple AI agent. It observes using the API, thinks with the LLM, and responds with the output.

Closing Thoughts

That’s how you connect an LLM to a live web API. This is the point where AI moves from being just a demo to becoming a real system.

If you start thinking in terms of data pipelines and LLM reasoning, you’ll naturally build more useful, production-ready AI apps.

I hope you enjoyed the article! Follow me on Instagram for more AI and machine learning tips. You can also check out my book, Hands-On GenAI, LLMs & AI Agents, to get career-ready in AI.