PMLE sample questions with answers

Generate embeddings for every clause with a BigQuery ML embedding model, then train a BigQuery ML logistic regression model on those embeddings to produce the house-style summaries.

Create a BigQuery ML remote model over the base Gemini model and call AI.GENERATE_TEXT with a much longer system instruction that restates the three-sentence rule and lists the firm's risk vocabulary.

Export the clause and summary pairs to Cloud Storage as JSONL files, and start a Gemini supervised tuning job from the Google Cloud console in the Agent Platform (formerly Vertex AI) project.

Create a BigQuery ML remote model over a tunable Gemini model, with an AS SELECT clause that returns the clause as prompt and the paralegal summary as label, and call it with AI.GENERATE_TEXT.

Grant the Vertex AI Service Agent role to the connection's service account in the project where the tuned remote model is created, and then run the same CREATE MODEL statement again.

Grant the engineer's own user account the Agent Platform User role in the project, because the tuning job runs with the credentials of the person who submits the CREATE MODEL statement.

Grant the connection's service account the BigQuery Data Viewer role on the training table, because the tuning job reads the prompt and label columns through the connection's identity.

Recreate the connection in the global location, because supervised tuning of a Gemini model is only supported through a connection that uses the global endpoint for its requests.

Run a query that selects from ML.EVALUATE for the model without an input table, which returns summary statistics for every feature that the model saw during its original training run.

Run a query that selects from ML.FEATURE_IMPORTANCE for the model, which returns the importance of each feature together with its minimum, maximum and count of missing values.

Run a query that selects from ML.TRAINING_INFO for the model, which returns the loss and duration of each training iteration together with the distribution of every input feature column.

Run a query on ML.FEATURE_INFO for the model, which returns each training input column with its minimum, maximum, mean, median, standard deviation, category count and null count.

Keep the five binary classifiers but switch each of them to the maximize-au-roc objective, so that the scores produced by the five models become directly comparable with each other.

Train one AutoML tabular regression model that predicts a plan number from 1 to 5, and round each predicted value to the nearest whole number to choose the plan for each prospect.

Train one AutoML forecasting model that predicts the sequence of plan choices over time for each prospect, and take the first predicted value in the horizon as the recommended plan.

Train one AutoML tabular classification model with the chosen plan as a five-class target, which frames the task as multi-class classification and uses the log loss objective for training.

Call the processor endpoint without a version ID so the default version is used, and make a newer stable version the default only after evaluating it on labelled invoices.

Hard-code the newest stable version ID in the request URL instead, and create a new processor for each future version so that old and new versions never share an endpoint.

Point the request URL at the stable channel, so that each request is served by the newest stable version as soon as Google releases it, without any change by the team.

Point the request URL at the rc channel, so that the processor always uses the most recent release candidate and never depends on a stable version that Google can later deprecate.

Convert every report to images and send them to the Cloud Vision API TEXT_DETECTION feature, so that the text is recognized page by page before the pages are chunked for retrieval.

Process the reports with the Document AI Layout Parser, which recognizes headings, tables and lists and produces context-aware chunks, and index those chunks for the retrieval step.

Keep the plain OCR step but reduce the chunk size to 200 characters, so that each chunk is less likely to cross a heading and more of the retrieved passages stay on one topic.

Process the reports with the Document AI Form Parser, and index each key-value pair that it returns from the engineering reports as a separate retrieval chunk for the assistant to search.

Create a glossary from the terminology list with Cloud Translation - Advanced, store the glossary file in Cloud Storage, and reference the glossary in every translation request for the manuals.

Post-process every translated manual with a script that searches for incorrect renderings of each of the 600 terms and replaces them with the approved translation for the target language.

Keep using Cloud Translation - Basic but wrap the brand names in quotation marks before each request, because quoted text is never translated and the drill part will then be inferred correctly.

Train a custom NMT model on the 600 term pairs alone, because a custom model trained on the terminology list learns to apply those terms consistently throughout full manual sentences.

Use a text embedding model, embedding articles with the RETRIEVALDOCUMENT task type and customer questions with RETRIEVALQUERY, and rank articles by vector similarity to the question.

Use a smaller Gemini Flash-Lite model to rate the relevance of every article for each question, because a cheaper generative model makes the comparison of all 30,000 articles fast enough.

Use a Gemini image generation model to render every article as a thumbnail, and compare the thumbnails with a rendering of the question to find the visually closest support article.

Use the Cloud Translation API to translate every question and article into English first, and then rank the articles by the number of words that they share with the translated question.

Send each packet directly to the Document AI Bank Statement Parser processor, because it extracts the entities of the first supported document it finds and ignores all of the other pages in the file.

Send each packet to the Document AI Enterprise Document OCR processor, and split the file wherever the page text starts with a new heading so that each part can be routed afterwards.

Send each packet to a Document AI Custom Splitter processor trained on the document classes, and route every identified logical document to the matching extraction processor by its class.

Send each packet to a Document AI Custom Classifier processor, and forward the entire 60-page file to the extraction processor that matches the single class returned for the packet.

Increase the AutoML training budget in node hours, so that the architecture search explores more candidate models and has a better chance of finding one that generalizes to recent months.

Configure a manual data split that assigns 80 percent of subscriber IDs to training and 20 percent to test, so that no individual subscriber appears in both the training and test sets.

Switch the optimization objective from AUC PR to log loss, so that the predicted probabilities are better calibrated and the campaign team can rely on the scores they receive.

Configure a chronological data split on the subscriber-month timestamp column, so that the training, validation and test sets cover successive time ranges and the test set holds the most recent months.