Hugging Face Models & Gradio Demos: interview questions

Production questions of the kind asked in applied AI engineering interviews.

1. Larkspur indexes each handbook page as one chunk with all-MiniLM-L6-v2. Support says "How fast does support answer a ticket?" never finds the FAQ page, though the answer is on it. What do you check?

What a strong answer covers:

• Tokenizes the page with the model's own tokenizer and compares the count with max_seq_length (256 here); the FAQ page is 291 tokens, so the support answer is past the cut
• Proves it by embedding the page and its first 1,231 characters: the two vectors are identical (cosine 1.0000), so the tail never mattered
• Fixes it at chunking time (T5.3), with chunk sizes counted in the embedding model's tokens, not characters
• Adds a check at indexing time that logs or rejects any chunk over the limit, since the library raises no error

2. A teammate's service calls pipeline("sentiment-analysis") with no model argument to tag tickets. Is that ready to ship?

What a strong answer covers:

• The task name picks a default model (distilbert-base-uncased-finetuned-sst-2-english in transformers 5.17.0) and the default can change in any release; text-generation now defaults to SmolLM3-3B, and the question-answering and translation tasks the notebook uses are gone in 5.x
• Pins model= and revision= (a commit hash) so the same weights run in every environment
• Checks the model fits the data: SST-2 is movie reviews with two labels, while tickets need categories such as billing or outage
• Measures it on labelled tickets before trusting the score, as notebook 01b does with the 1,066 Rotten Tomatoes test reviews

3. Finance asks what the ticket tagger will cost and how slow it is. How do you estimate it before launch?

What a strong answer covers:

• Counts tokens with the model's own tokenizer on a sample of real tickets; characters divided by four is close on prose and off by almost half on names and odd words (2.29 characters per token on the notebook's sentence)
• Measures latency per batch on the target hardware, with the model loaded once, not per request
• Separates the one-time costs (download, about 268 MB for the SST-2 model, and load) from the per-call cost
• Tracks token counts and latency percentiles in production, because ticket length drifts

4. The tagger runs in containers on a cluster with no internet access. The first deploy fails at startup. Why, and how do you ship models there?

What a strong answer covers:

• from_pretrained tries to download on a cache miss; with no network it fails, and every new pod would download again even with network
• Builds the cache into the image or a mounted volume (HF_HOME or HF_HUB_CACHE), at a pinned revision
• Sets HF_HUB_OFFLINE=1 so the library reads the cache and never calls the Hub
• Watches disk use with hf cache ls and removes revisions no branch points to with hf cache prune; their blobs stay on disk until then

5. The support lead's Gradio demo takes several seconds per click, though the model alone answers in a fraction of that. Where does the time go?

What a strong answer covers:

• Looks for the model being loaded inside the event function; load the pipeline once at module level and call it in the function
• Checks whether share=True routes traffic through a tunnel, and whether many users queue behind one worker
• Times each step (tokenize, model, post-process, network) instead of guessing
• Treats the demo as a demo: authentication, rate limits and monitoring belong to the real service, not to launch()