ai_query(): Call Any LLM From SQL — Zero Python Required

By Prahlad Menon 4 min read

At DATA+AI Summit 2026, Databricks showed what might be the most practical AI feature for data teams: a SQL function that calls foundation models.

SELECT ai_query(
  'databricks-meta-llama-3-3-70b-instruct',
  'Redact all PII from this text:' || notes
) AS redacted_notes 
FROM clinical_records;

That’s it. No Python notebooks. No model deployment. No infrastructure. Just SQL that calls an LLM and returns results as a column.

What ai_query() Actually Does

ai_query() is a general-purpose AI function that lets you query any supported foundation model directly from SQL or Python. Unlike task-specific AI functions (like ai_classify() or ai_summarize()), it gives you full control over the model, prompt, and parameters.

Three things make this work:

  1. Runs everywhere — Notebooks, dashboards, Lakeflow pipelines. Anywhere you can write SQL.

  2. Scales automatically — Serverless. Handle millions of rows with zero ops. Databricks manages parallelization, retries, and scaling.

  3. Secured by design — Operates directly on governed data in Unity Catalog. Your data never leaves the platform.

The Architecture

Here’s how it all fits together — nothing leaves the platform:

┌─────────────────────────────────────────────────────────────────────┐
│                        Databricks App (Optional)                     │
│              Streamlit: structured + unstructured tables             │
└─────────────────────────────┬───────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────────┐
│                    Serverless SQL Warehouse                          │
│                      (the execution engine)                          │
└─────────────────────────────┬───────────────────────────────────────┘
                              │ invokes
                              ▼
┌─────────────────────────────────────────────────────────────────────┐
│                         AI Functions                                 │
│           ai_query() → product judge                                 │
│           ai_parse_document() → OCR                                  │
└─────────────────────────────┬───────────────────────────────────────┘
                              │ LLM call
                              ▼
┌─────────────────────────────────────────────────────────────────────┐
│                     Meta Llama 3.3 70B                               │
│                      (Databricks-Hosted)                             │
└─────────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────────┐
│     Unity Catalog: one governance layer over every table,           │
│                        volume & model                                │
├─────────────┬─────────────┬─────────────────┬───────────────────────┤
│  patients   │ medical-    │  dais_demo_docs │ pii_redaction_results │
│  Delta      │ encounters  │  Volume = PDF   │ Delta = write-back    │
│  1,000 rows │ Delta       │                 │                       │
│             │ 2,000 rows  │                 │                       │
└─────────────┴─────────────┴─────────────────┴───────────────────────┘

Run it in production with: Lakeflow retries · CI/CD · lineage

The key: everything stays inside the platform. Your data never leaves Unity Catalog. The LLM call happens serverlessly. Results write back to Delta tables.

The Healthcare PII Redaction Example

The summit demo showed PII redaction for clinical records — using a whimsical synthetic example to make the point clear.

Input (Original Clinical Note)

Patient Name: Gretel Sweettooth
DOB: 02/13/1966

Chief Complaint: Bloating, frequent urination, and fatigue over the past 3 weeks.

History of Present Illness:
Gretel Sweettooth is a 57-year-old female who presents with abdominal bloating, 
with hyperglycemia. She reports excessive consumption of sugary treats, including 
gingerbread cookies and candy canes, she has been indulging since her visit to 
the Candy Forest. She has also noticed drier skin, increased thirst, and 
occasional blurry vision.

Past Medical History: No known chronic illness prior to this episode.

Family History:
Brother Hansel Sweettooth was recently diagnosed with Type 2 Diabetes

Social History:
Lives in a gingerbread cottage. No tobacco or alcohol use. Diet high in 
refined sugars.

Output (Redacted)

Patient Name: [NAME]
DOB: [DOB]
MRN: [MRN]

Chief Complaint: Bloating, frequent urination, and fatigue over the past 3 weeks.

History of Present Illness:
[NAME] is a 57-year-old female who presents with abdominal bloating, 
with hyperglycemia. She reports excessive consumption of sugary treats, including 
gingerbread cookies and candy canes, she has been indulging since her visit to 
the Candy Forest. She has also noticed drier skin, increased thirst, and 
occasional blurry vision.

Past Medical History: No known chronic illness prior to this episode.

Family History:
Brother [NAME] was recently diagnosed with Type 2 Diabetes

Social History:
Lives in a gingerbread cottage. No tobacco or alcohol use. Diet high in 
refined sugars.

Notice what changed:

  • Patient name → [NAME] (both “Gretel Sweettooth” and “Hansel Sweettooth”)
  • DOB → [DOB]
  • MRN → [MRN] (added even though original didn’t show it)

And what stayed:

  • All clinical content: symptoms, diagnosis hints, medical history
  • Location reference (“Candy Forest”, “gingerbread cottage”) — kept because they’re part of the clinical narrative, not identifying addresses
  • The entire clinical picture remains intact

Structured Data (Column by Column)

Before:

patient_name: John Smith
ssn: 123-45-6789
diagnosis: Type 2 Diabetes
provider: Dr. Sarah Chen

After:

patient_name: [NAME]
ssn: [SSN]
diagnosis: Type 2 Diabetes
provider: [PROVIDER]

The identity goes. The clinical value stays.

Unstructured Data (Free-Text Notes)

The same query handles free-text clinical notes:

Before:

"Patient John Smith (DOB 03/15/1982, MRN 487289) presented to 
Dr. Sarah Chen at Boston General on 01/10/2026 with fatigue 
and elevated glucose..."

After:

"Patient [NAME] (DOB [DATE], MRN [MRN]) presented to [PROVIDER] 
at [FACILITY] on [DATE] with fatigue and elevated glucose..."

One SQL function. Works on structured columns and free-text blobs.

The Prompt Is the Policy

The key insight from the demo: the prompt is the policy.

For reliable PII redaction, your prompt needs three things:

1. Be explicit about what counts as PII Name it: names, dates, MRNs, SSNs, locations, providers. Don’t assume the model knows your definition.

2. Define typed replacements Use [NAME], [DATE], [MRN] — not a blanket [REDACTED]. Typed tags preserve structure and enable downstream processing.

3. Preserve clinical meaning Keep diagnoses, dosages, medical terms. The value stays, the identity goes.

A Production-Ready Prompt

Here’s the actual prompt from the demo — notice the explicit examples for each PII type:

You are a PII redaction system for healthcare data.

Replace these identifiers with tags, exactly as shown:

- Person names (patients, family members, providers) -> [NAME]
    Example: "Dr. Sarah Chen" -> "Dr. [NAME]"
- Social Security Numbers -> [SSN]
    Example: "855-17-9436" -> "[SSN]"
- Medical Record numbers and patient IDs -> [MRN]
    Example: "YKX019348245" -> "[MRN]"
- Dates of birth -> [DOB]
    Example: "03/19/1967" -> "[DOB]"
- Street addresses -> [ADDRESS]
    Example: "742 Evergreen Terrace, Springfield, IL 62704" -> "[ADDRESS]"
    Example: "1234 Oak Ave, Apt 5B, Portland, OR 97201" -> "[ADDRESS]"
- Phone numbers -> [PHONE]
    Example: "(503) 555-0147" -> "[PHONE]"
    Example: "555-867-5309" -> "[PHONE]"

Preserve all clinical content exactly: symptoms, diagnoses, medications,
dosages, vital signs, dates of clinical events, anatomical references,
facility names (e.g., "Memorial Hospital"), and any non-identifying content.
Only redact identifiers of the person.

Output only the redacted text. No explanation, no preamble.
The input label is a JSON-serialized row. Return JSON with the same shape 
and PII fields replaced.

Input:
<raw JSON>

The key patterns:

  1. Role assignment — “You are a PII redaction system”
  2. Explicit examples for each PII type — Show exactly what to match and replace
  3. Preservation rules — What to keep (clinical content, facility names)
  4. Output format — JSON in, JSON out, no explanation

Supported Models

Databricks hosts a range of foundation models optimized for batch inference:

Production-optimized models:

  • databricks-meta-llama-3-3-70b-instruct
  • databricks-claude-sonnet-4
  • databricks-gpt-5
  • databricks-gemini-3-flash

For embedding/search:

  • databricks-gte-large-en
  • databricks-qwen3-embedding-0-6b

You can also bring your own models via provisioned throughput endpoints or external model connections (OpenAI, Anthropic, etc.).

Best Practices for Production

1. Use Databricks-hosted models Models prefixed with databricks- are fully managed and scale automatically. No provisioning or configuration needed.

2. Submit your full dataset in a single query AI Functions handle parallelization internally. Manually splitting into small batches actually reduces throughput.

3. Set failOnError to false for large workloads

SELECT ai_query(
  'databricks-meta-llama-3-3-70b-instruct',
  'Redact PII: ' || notes,
  failOnError => false
) AS redacted_notes
FROM clinical_records;

Failed rows return error messages instead of killing the whole job. You keep successful results without reprocessing.

4. Use structured outputs for downstream processing

SELECT ai_query(
  'databricks-meta-llama-3-3-70b-instruct',
  'Extract entities as JSON: ' || text,
  responseFormat => 'json_object'
) AS entities
FROM documents;

Beyond PII Redaction

The same pattern works for any LLM task on your data:

Entity extraction:

SELECT ai_query(
  'databricks-meta-llama-3-3-70b-instruct',
  'Extract all medication names and dosages as JSON: ' || notes
) AS medications
FROM clinical_records;

Sentiment analysis:

SELECT ai_query(
  'databricks-meta-llama-3-3-70b-instruct',
  'Classify the sentiment (positive/negative/neutral): ' || review
) AS sentiment
FROM customer_reviews;

Summarization:

SELECT ai_query(
  'databricks-meta-llama-3-3-70b-instruct',
  'Summarize this document in 3 bullet points: ' || content
) AS summary
FROM reports;

SQL correction:

SELECT ai_query(
  'databricks-meta-llama-3-3-70b-instruct',
  'Fix the syntax errors in this SQL query. Return only the corrected SQL: ' || raw_sql
) AS corrected_sql
FROM sql_errors;

The Bigger Picture

ai_query() represents a shift in how we think about LLMs in data pipelines.

Old pattern: Export data → Run through Python script → Import results New pattern: Transform data where it lives, using SQL

This matters because:

  1. Data engineers don’t need to become ML engineers. If you can write SQL, you can call an LLM.

  2. Governance is built in. The data never leaves Unity Catalog. Lineage is tracked. Access controls apply.

  3. It scales with your data. Millions of rows? Databricks handles the parallelization. You just write the query.

The prompt is the policy. The SQL is the pipeline. The LLM is just another function.

Docs: ai_query function

More AI Functions: Enrich data using AI Functions

Presented at: DATA+AI Summit 2026