Secrets in Fabric Notebooks: Azure Key Vault Instead of Hardcoded Passwords

1. Loading secrets at runtime (code)

The vault URL goes into a parameter cell, so the same notebook runs in dev, test and prod. The actual code only references the names of the secrets:

# Parameter cell (can be overridden by the pipeline)
KEY_VAULT_URL = "https://<your-vault>.vault.azure.net/"

# Load secrets
import notebookutils

api_user = notebookutils.credentials.getSecret(KEY_VAULT_URL, "api-user")
api_key  = notebookutils.credentials.getSecret(KEY_VAULT_URL, "api-key")

# Sanity check (without leaking values)
assert api_user and api_key, "Error: at least one secret is empty!"
print("Secrets loaded successfully.")

This way the values only ever live in the memory of the Spark session.

2. Permissions: roles for managing and reading

Azure Key Vault strictly separates managing the resource (control plane) from accessing the data (data plane).

Why does the pipeline fail at night?

A classic: during the day the notebook runs fine interactively (under your own developer account with Officer/User rights). At night, in the Fabric pipeline, it fails with 403 Forbidden. Fix: The identity configured in the pipeline (for example the workspace identity or a service principal) also needs the Key Vault Secrets User role on the Key Vault.

3. Typical leaks: where secrets still slip through

Fabric redacts known secrets on direct output (print(secret)) on a best-effort basis. This is not a security boundary! As soon as the value is transformed (for example into a Base64 auth header or a substring), the protection no longer applies.

The 4 most common leak points in day-to-day work:

1. Request dumps (e.g. SOAP/REST payloads): If you print whole payloads for troubleshooting, mask auth headers or <Password> tags before logging.
2. Pipeline parameters and exit(): Never pass secrets as notebook parameters or return them with notebookutils.notebook.exit(...). They show up in plain text in the pipeline's run history. Always pass only the secret's name.
3. URL query parameters: Never append API keys to the URL (?apiKey=...). On connection errors they inevitably end up in the stack trace. Use HTTP headers instead.
4. Delta tables (audit logs): If requests are written to a Delta table for analysis, filter out passwords first. Because of Delta Lake time travel, plain-text secrets that were written by accident are hard to remove again.