Multimodal Prompting: interview questions

Production questions of the kind asked in applied AI engineering interviews.

1. Larkspur summarises recorded export walkthroughs with GPT-4o, sampling a frame every 3 seconds. A customer's summary says the export "may not have finished", but the recording shows an "Export ready" toast. What went wrong, and how do you fix it?

What a strong answer covers:

• Lists the timestamps of the frames sent and compares them with when the toast was on screen: a 1.2 s toast and a 3 s gap can miss each other
• States the rule: only a gap shorter than the shortest event you need guarantees a frame on it; 1 s does that here, at 90 frames instead of 30
• Considers cheaper options than a dense gap everywhere: extra frames on scene changes, or relying on the transcript for events the narrator mentions
• Adds the recording to a golden set with the question "Did the export finish?" so the fix is measured

2. The billing team wants answers from the 40-page handbook PDF. One engineer sends the whole PDF to Gemini on every question; another wants to extract the text and use RAG. How do you decide?

What a strong answer covers:

• Prices the first option: 40 pages at 258 tokens is 10,320 tokens on every call, before the question
• Uploads once and reuses the file handle if the whole document is sent, instead of re-sending bytes per call
• Notes what text extraction loses (charts, tables laid out as pictures) and what it gains (citations, smaller prompts, access control per chunk)
• Proposes a measured comparison on the golden questions, and page-level retrieval (T6.5) as the middle path

3. Support calls are transcribed and summarised with Gemini audio input. How do you measure transcript quality and catch the model putting words in the wrong speaker's mouth?

What a strong answer covers:

• Builds a small set of calls with human transcripts and measures word error rate and speaker attribution separately
• Checks timestamps against the audio, since chapter titles and captions carry timecodes the prompt asked for
• Spot-checks facts the agent quoted (the 30-day refund window) against the handbook, because a wrong policy in a summary is the costly error
• Tracks cost per call from duration: 32 tokens per second is 1,920 tokens a minute

4. A customer sends a screenshot of the pricing page and asks how much outbound traffic the starter plan includes. The chart on the page has bars but no printed values. What should the assistant do?

What a strong answer covers:

• Separates reading a printed value from estimating a bar height; only the first is an answer to quote
• Tells the model to say when a value is not printed rather than guess from the picture
• Answers from the handbook text instead, where the plan lists its included traffic
• Logs the case so the pricing page gets printed labels, which makes every later screenshot answerable

5. To cut cost, a teammate proposes detail="low" for every customer screenshot sent to gpt-4o. When is that safe?

What a strong answer covers:

• Quotes the difference: 85 tokens against 1,105 for a 1280 by 720 image at high, 13 times less, and the same 85 at any size
• Notes that a 1920 by 1080 screenshot also costs 1,105 at high, because it is scaled to 1365 by 768 first, so size alone does not drive the bill
• Tests both settings on real screenshots for the task that needs small text, such as reading an error code, and keeps high only where low fails
• Redacts secrets before sending, because a base64 image carries everything on the screen to the provider