Full Stack & AI · Day 04 · Chapter 05

LLM
Integration

We use a small LBS College knowledge assistant to make integration patterns concrete.

Retrieve evidence. Generate an answer. Verify citations. Ask for human help when evidence is missing.

The problem statement

Question
Which undergraduate programmes are offered at LBS College?
Evidence
Curated chunks from official LBS College pages.
Permitted result
Answer + retrieved source citations + help flag.
Not permitted
Inventing facts or citing sources that were not retrieved.
One component, not the whole system

The application owns
the authority.

The model interprets language. The backend owns identity, permissions, approved evidence, validation, actions, and the user experience.

flowchart LR
  U[Visitor question] --> B[Application backend]
  S[(Curated LBS knowledge base)] --> B
  B --> C[Retrieve authorised context]
  C --> G[Groq LLM]
  G --> V[Parse and validate]
  V --> R[Recommendation or safe failure]
The browser never receives the Groq API key. The backend chooses the chunks and owns the model call.
Start narrow and make it testable

One question. One trusted source.
One measurable contract.

A response is usable only if it has the required fields and cites LBS source chunks retrieved for this request.

The application result

{
  "answer": "The programmes page lists ...",
  "citations": ["programmes-2026"],
  "needs_human_help": false
}

Human help is a valid result when the curated knowledge base cannot answer the question.

Retrieval augmented generation

Current, scoped evidence
before generation.

Retrieval selects LBS source chunks before the model sees them. The curated knowledge base—not model memory—remains the source of truth.

flowchart LR
  D[Curated official LBS pages] --> F[Chunk + source ID + URL]
  Q[Question] --> F
  F --> E[Ranked relevant chunks]
  E --> P[Prompt assembly]
  P --> G[Generate answer]
  G --> V[Verify cited IDs]
RAG does not require a vector database. This demo uses visible lexical scoring first; the same boundary later works with embeddings, FAISS, or hybrid search.
Hosted model call

Groq through the
OpenAI wrapper.

One familiar Python client. One server-side API key. No local model download or inference server for participants.

The OpenAI SDK is the client library; Groq is the hosted provider. The application still owns its own request and result contract.

Configure the client

from openai import OpenAI

client = OpenAI(
    api_key=os.environ["GROQ_API_KEY"],
    base_url="https://api.groq.com/openai/v1",
)

response = client.chat.completions.create(
    model="openai/gpt-oss-20b",
    messages=messages,
    temperature=0.1,
    response_format=ANSWER_SCHEMA,
)
Make the output useful to code

The output we are
looking for.

Strict structured output constrains the generated JSON. Application code must still verify that every citation is among the retrieved chunks.

JSON schema boundary

ANSWER_SCHEMA = {
  "type": "json_schema",
  "json_schema": {
    "name": "lbs_college_answer",
    "strict": True,
    "schema": {
      "type": "object",
      "required": ["answer", "citations",
                   "needs_human_help"],
      "additionalProperties": False
    }
  }
}

Schema-valid JSON is not proof that a recommendation is correct.

Tool calling

The model can request.
Only code can act.

A model-generated tool call is a proposal with untrusted JSON arguments—not an executed action.

sequenceDiagram
  participant M as LLM
  participant A as Application code
  participant T as Approved service
  M-->>A: tool request + JSON arguments
  A->>A: validate schema, user, permission, policy
  alt permitted
    A->>T: execute with server credentials
    T-->>A: trusted result
    A-->>M: tool result
  else denied or invalid
    A-->>M: typed error or no result
  end
Our LBS College Assistant is deliberately read-only. A future tool still needs code-side validation, authorisation, execution, and audit logging.
Live coding route

Walk through the RAG boundary.
Then vibe-code it.

Read the implementation from the curated knowledge base outward, not from the API call inward.

1
lbs_knowledge_base.json — official LBS chunks with source IDs and URLs.
2
retrieve() — rank and select relevant chunks before generation.
3
build_messages() — system instructions, selected evidence, and one question.
4
answer_lbs_question() — the Groq/OpenAI-compatible model request.
5
validate_answer() — reject citations outside the retrieved set.

Set up the demo

cd docs/demo
python3 -m pip install openai python-dotenv

# Create .env in the project root
GROQ_API_KEY=gsk_your_key_here
GROQ_MODEL=openai/gpt-oss-20b

python3 policy_assistant_demo.py
# Open http://127.0.0.1:8080

Keep the boundary

Put secrets only in .env.
Keep .env in .gitignore.
Restart the server after changing model configuration.

Audience lab · change one variable

Change it.
Then explain the result.

Keep the same questions and retrieval data while each group changes one part of the application.

1. Prompt contract

Rewrite SYSTEM_INSTRUCTIONS to demand a two-line answer, one citation, and a clear refusal when evidence is missing.

Observe: instructions shape this request; they do not train the model.

2. Few-shot examples

Add one grounded answer and one unsupported-question refusal to build_messages().

Observe: examples demonstrate the desired response pattern at runtime.

3. Temperature + model

Run each question five times at 0.0, 0.3, and 0.8. Then change GROQ_MODEL in .env.

Observe: stability, latency, wording, and structured-output behaviour.

4. Retrieval grounding

Ask an unsupported question. Add one approved source chunk. Ask again.

Observe: RAG extends factual scope only when relevant evidence is retrieved.

Group report: show one success, one failure, the changed code, and whether the returned citations stayed within the retrieved sources.
Shared evaluation set

Do not judge a change
from one answer.

Run the same questions before and after each modification. Record answer usefulness, citations, and the human-help flag.

Test these five questions

1
Which undergraduate programmes are offered at LBS College of Engineering?
2
Where is the college located?
3
Does the college offer a B.Tech programme in Biomedical Engineering?
4
What is the hostel fee?
5
Which support does the Career Guidance Cell provide?
Questions 3 and 4 are deliberate boundary tests. With no approved evidence, the correct result is uncertainty—not a fluent invention.
Integration mental model

Knowledge base first.
Retrieve evidence.
Code decides what is usable.

Curated LBS evidenceofficial chunks with stable IDs and URLs
→
Retrieve top chunksonly relevant evidence enters the prompt
→
Hosted LLMGroq request with controlled decoding
→
Validated answercitations, human-help flag, or safe failure
End of Day 04

Build the knowledge boundary.
Then trust the workflow.

Generative models create possibilities. Reliable applications retrieve evidence, validate citations, and make uncertainty visible.

Open demo guide →Take the project home →