Repository navigation
feat(get-data): Spotter answer data (always on, with expired-answer reload) and iframe suppression flag - #227
Conversation
… flags Let Claude read the rows behind an unsaved Analytics Agent answer, and let callers such as TriNet's proxy strip embedded visualizations from tool output. Both are behind undocumented URL flags and off by default. - enable-spotter-answer-data=true: get_data accepts object_type SPOTTER_ANSWER with the answer_id from get_session_updates. Rows come from the answer's CSV export (session + generation number) and are parsed into the get_data shape, skipping the data-extract watermark preamble. The description keeps the iframe-first default. - suppress-iframes=true: implies the above, drops iframe_url from session updates and its schema, strips embed links and iframe tags from update text (thinking included), drops frame_url from getAnswer, and steers Claude to build visuals from get_data.
…frame-suppression # Conflicts: # src/servers/mcp-server.ts # src/servers/tool-definitions.ts # src/services/clients/get-data/spotter-answer-data.ts # test/streaming-utils.spec.ts
- Strip the watermark preamble only when it reads "Data extract…", so a single-column answer with an empty cell no longer loses rows - Coerce a cell to a number only when it round-trips exactly, keeping leading-zero IDs, >2^53 integers and formatted text as strings - Give a permission hint instead of the expiry hint on 403 - Drop the unused title parameter
…ault
- answer_id now also carries conversation_id and message_id. These outlive
the answer's session ids (about 8 hours), and existing readers ignore them
- When the CSV export fails, get_data loads the answer again through
/conversation/v2/{conv}/message/{id}/load/public (the call the Visual
Embed SDK uses to replay answers) and exports with the fresh ids.
Not on 401/403, or for answer_ids without the conversation ids
- get_data on SPOTTER_ANSWER is on by default; enable-spotter-answer-data=false
turns it off. suppress-iframes stays opt-in and forces it on
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces support for fetching the underlying data of unsaved Analytics Agent (Spotter) answers via the get_data tool, using the new SPOTTER_ANSWER object type. It adds two new URL query parameters: enable-spotter-answer-data (to control this feature) and suppress-iframes (to omit embedded visualization iframes and links from responses, relying instead on raw data visualization). To support this, the PR implements CSV parsing and watermark stripping for exported reports, adds a mechanism to reload expired answer sessions using conversation and message IDs, and updates the MCP server schemas and tool definitions dynamically based on these flags. Comprehensive unit and integration tests have been added to verify these behaviors. I have no feedback to provide as no review comments were submitted.
The answer export writes large values as "1.093225231E7", which the round-trip check left as strings. Accept plain decimals and E-notation instead, still keeping leading-zero IDs and integers beyond 2^53 as text.
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
get-data-mcp-server | 028d955 | Nov 05 2026, 01:47 AM |
Remove the enable-spotter-answer-data URL flag. get_data now always accepts SPOTTER_ANSWER, so the flag-off rejection path, prop and span attribute go too. suppress-iframes is unchanged.
mouryab
changed the title
| const enableRawSessionUpdates = | ||
| url.searchParams.get("enable-raw-session-updates") === "true"; | ||
| props.enableRawSessionUpdates = enableRawSessionUpdates; | ||
| props.suppressIframes = url.searchParams.get("suppress-iframes") === "true"; |
There was a problem hiding this comment.
Nit: make this a local var to avoid computing it twice, similar to what we did for enableRawSessionUpdates
There was a problem hiding this comment.
Done in 6c94584 — suppressIframes is now a local in both the bearer and OAuth paths, same as enableRawSessionUpdates.
| { | ||
| session_updates: messagesState.messages, | ||
| session_updates: this.ctx.props.suppressIframes | ||
| ? stripEmbeddedVisualizations(messagesState.messages) |
There was a problem hiding this comment.
Instead of removing the fields after the fact, we should just not generate the field in the first place. The iframe_url is created inside MCP Server code in src/streaming-utils.ts around line 170. Let's pass the suppressIframes flag and skip generating that field if we don't want it.
There was a problem hiding this comment.
Done. suppressIframes now flows from send_session_message → sendAgentConversationMessageStreaming → processSendAgentConversationMessageStreamingResponse, so iframe_url is never built. get_session_updates returns stored messages as they are.
One thing I kept: stripping embed links from message text (text / raw content), now done at write time alongside the iframe_url skip. In raw mode there's never an iframe_url, so text stripping is the only thing the flag does there. Do you know whether Spotter's text ever actually contains #/embed/ / tsmcp=true links? If it never does, I'll drop the stripping and just skip the field.
There was a problem hiding this comment.
Spotter's text never sends us embed links so we don't need to do any postprocessing
There was a problem hiding this comment.
Removed in 80c6179. The flag now only skips building iframe_url.
| generation_number: generationNumber, | ||
| file_format: "CSV", | ||
| }); | ||
| return file.text(); |
There was a problem hiding this comment.
Should we do file.text() outside of observeUpstreamCall wrapper? It's not really part of the upstream call - it's postprocessing
There was a problem hiding this comment.
Done — moved out of observeUpstreamCall, matching the existing getAnswerData.
Address review on #227 now that Spotter answer data is always on: - Fold SPOTTER_ANSWER into GET_DATA_SUPPORTED_TYPES and GetDataInputSchema; move the answer_id/iframe_url wording into ConversationUpdateSchema and drop the flag-specific schemas and applyUrlFlagToolOverrides. With suppress-iframes, listTools only swaps one sentence of get_data's description. - Skip building iframe_url at stream-processing time instead of stripping it in get_session_updates; embed links in message text are stripped there too. - Keep Spotter answer CSV cells as strings, read the export body outside observeUpstreamCall, and raise SpotterAnswerCsvError for an unusable CSV so it is reported directly instead of triggering an expired-answer reload. - Revert the v1 getAnswer frame_url change, use a local var for the suppress-iframes flag, drop the README URL-flags section (belongs in docs), and fix the raw get_session_updates answer_id description.
18348f7 to
f2d777d
Compare
| await appendStoredMessages( | ||
| conversationId, | ||
| suppressIframes | ||
| ? newMessages.map(stripEmbeddedVisualizationLinks) |
There was a problem hiding this comment.
This extra postprocessing is not needed, we don't get embed iframe links in the Spotter text stream
There was a problem hiding this comment.
Removed in 80c6179. The flag now only skips building iframe_url.
| ); | ||
| }; | ||
|
|
||
| // A ThoughtSpot embedded-visualization URL: the MCP embed marker or an embed route. |
There was a problem hiding this comment.
All of this below is not needed
| @@ -1,11 +1,21 @@ | |||
| // An unsaved answer from the Analytics Agent (Spotter), addressed by the | |||
| // `answer_id` that get_session_updates returns. | |||
| export const SPOTTER_ANSWER_TYPE = "SPOTTER_ANSWER"; | |||
There was a problem hiding this comment.
Let's call it ANALYTICS_AGENT_ANSWER to get ahead of whitelabeling concerns. We should not expose TS-branded names to the host agent.
There was a problem hiding this comment.
Renamed to ANALYTICS_AGENT_ANSWER everywhere the host agent sees it, in 028d955.
Spotter's text never carries #/embed/ or tsmcp=true links, so the suppress-iframes flag now only skips building iframe_url on answers.
Avoid exposing TS-branded names to the host agent ahead of whitelabeling: the get_data object_type value, its descriptions and error messages now use ANALYTICS_AGENT_ANSWER.
What
get_datacan now read the rows behind an answer from Spotter (the Analytics Agent), including answers whose session has expired. This is always on. A separate opt-in URL flag,suppress-iframes=true, removes iframes so Claude builds its own visuals from the rows. Design doc: "Spotter answer data in getData + iframe suppression".Spotter answer data (always on)
get_dataacceptsobject_type: "SPOTTER_ANSWER", withobject_idset to theanswer_idfromget_session_updates.visualization_idsis ignored.exportAnswerReportas CSV, using the answer's session id and generation number. They are parsed into the existingget_dataresult shape:1.093225231E7.total_row_countis the full row count.max_rowsworks as it does for saved Answers (default 25)."Data extract produced by …"block before the header and is recognised only by that text, so single-column answers with empty cells keep all their rows.get_datadescription keeps Claude iframe-first. Claude shows Spotter's visualization and fetches rows only when the user asks to chart, reshape, combine or build a dashboard.Expired answers
answer_idnow also carriesconversation_idandmessage_id, which last as long as the conversation does. Existing readers that only usesession_idandgen_no, such as the dashboard tools, are unaffected.get_datareloads the answer and exports again with the fresh ids. The reload isPOST /conversation/v2/{conversation_id}/message/{message_id}/load/public, the same call the Visual Embed SDK makes to replay a stored answer. It re-runs the query, so it can be slow.answer_idissued before this change.suppress-iframes=true(opt-in, undocumented)iframe_urlis removed fromget_session_updatesresults and from that tool's output schema. The schema no longer mentions iframes, embeds or visualizations.<iframe>tags are stripped from update text, including thinking text andcontentin raw mode.frame_urlis removed from V1getAnswer.get_datadescription tells Claude that answers have no visual of their own, so it should build one from the returned rows whenever that would help.suppress-iframes=trueOnly the exact value
trueturns suppression on. With it unset, iframes behave exactly as before.Why
Customers can put this server behind their own proxy and Claude client. Today Claude can show a Spotter answer but can't read its rows, and there is no way to turn iframes off. Answer data is on for everyone, for the doc's default-user story: "turn that into a Claude chart I can tweak". An earlier
enable-spotter-answer-dataoff switch was dropped. If a URL still carries it, the server ignores it.Implementation notes
suppress-iframesis parsed in both props hooks insrc/index.ts.applyUrlFlagToolOverrides(tool-definitions.ts). Text is stripped whenget_session_updatesreads the stored updates (stripEmbeddedVisualizations).answer_idis built inbuildAnswerId(streaming-utils.ts).loadSpotter3Answerinspotter-3-client.ts, recorded as theload_agent_conversation_answerupstream operation and thereload-spotter-answerspan.suppress-iframesis left out of it on purpose.Testing
SPOTTER_ANSWERcall paths: success, badanswer_id, export failure, 403;answer_id, conversation gone;loadSpotter3AnswerandgetAnswer.wrangler devagainst champagne-master-aws staging, in the default and suppress modes. Real rows came back. A simulated expired answer (a deadsession_idwith real conversation ids) was reloaded throughload/publicand returned the same rows.suppress-iframes=truefrom Claude ("which Campaign Channel drives the highest Net Sales across each age_bucket"):answer_idcarried the conversation ids, and there was noiframe_urlor embed text.get_datareturned all 42 rows, matching Spotter's text, and Claude charted them.Known gaps / follow-ups
SPOTTER_ANSWER. We rely on the tool description to keep models iframe-first when iframes are shown, which still needs a check in Claude. Whether Claude charts answers without being asked undersuppress-iframesalso needs a fresh-chat check.max_rows.skills/analyzing-data-with-thoughtspot/SKILL.md("Don't generate your own charts") conflicts with the suppressed flow.