Open-source LLMs: Transformers, Inference Client, Groq, LangChain

What you'll build

• A local backend that downloads a 1.1B-parameter model and runs it in your own process.
• A hosted Hugging Face backend that sends the same messages to a large model on Inference Providers.
• A Groq backend that uses Groq's native LangChain chat model.
• One chain that runs unchanged on all three.

Lesson 2 of T4.1 ran one prompt | model chain on commercial APIs. This project runs the same chain on open-weight models. It adds no new idea; it shows what the chat-model interface hides when the model moves to weights on your laptop, or to someone else's hardware.

For Larkspur this is the question behind a cost or data-residency review: whether the refund assistant can run on an open model without rewriting the chain. The call shape survives each move. What changes is where the tokens are computed, what the reply reports about itself, and which messages the backend accepts.

The two Hugging Face backends share one wrapper, and Groq needs none:

Figure 1: The three backends. ChatHuggingFace gives two completion-style classes the chat interface; ChatGroq is already a chat model.

Table 1: The project's pieces. Each optional notebook does the same job with the backend's own library, without LangChain.

Invoke the chain with the topic AI. The template produces one HumanMessage, and each backend takes it from there. We traced the paths in the installed langchain-huggingface 1.2.3 source; no model was called:

Table 2: One request through each backend. The settings are the notebook's.

The first row is the one to understand. A chat model trained on roles still reads one sequence of tokens; the roles are marked with special tokens that differ by model family. The optional Transformers notebook shows TinyLlama's format, a <|user|> line, the question, an end-of-turn token and an open <|assistant|> line for the model to continue. On the local path, ChatHuggingFace asks the tokenizer to render the messages in that format. On the hosted paths, the server does the rendering.

The optional notebooks do the same without LangChain. The Transformers notebook renders the chat template, encodes, generates and decodes by hand, then repeats it with a one-call pipeline. The Inference Client notebook calls chat_completion on a hosted model and reads token usage from the response. The Groq notebook calls Groq's own SDK and compares two Llama 4 variants on one prompt.

Table 3: Five choices and the reason for each.

• No token counts locally. The pipeline path returns the text without usage metadata. Cost and length checks that read .usage_metadata see nothing.
• The last message must be human. The local wrapper raises an error when the last message is a system or AI message, and it accepts only system, human and AI messages. Tool results (T7.1) cannot pass through it.
• Replies differ by backend. A 1.1B model on a laptop and a large hosted model give different answers to the same prompt. Re-run the golden questions before relying on any of them (the silent provider swap from lesson 2 of T4.1).

To run it, the local part needs the course's optional Hugging Face dependency group and downloads about 2 GB of weights on first use; the hosted parts need a Hugging Face token and a Groq API key in the project's environment file.

The code for this lesson is in these notebooks. Colab runs one in your browser: install the packages its first cell imports and set the API keys it reads. To run every notebook as written, clone the repository and use its uv environment.

• The same prompt | model chain runs on a local model, a hosted Hugging Face model and Groq; only the model object changes.
• ChatHuggingFace gives completion-style Hugging Face classes the chat interface. Locally it renders messages with the model's own chat template; hosted, it sends role dicts to the server.
• The interface hides real differences: the local path reports no token counts and accepts only system, human and AI messages, and every backend answers differently.