Query monitoring
Polars can collect metrics while your queries run and send them to Polars Nanook. Each run shows up as a query profile in the dashboard, which shows you which part of a query is slow. That page explains what the metrics mean.
Monitoring works for queries that run locally; you do not need to move your data or your compute to Polars Nanook to use it. Your data stays on your machine. No rows or column values are transmitted, only the query plan and per-node runtime counters. The plan includes the schema, column names, source paths, and any literals you wrote into the query.
Requirements
- A Polars Nanook account. Get started at Polars Nanook.
- The
polars-nanookpackage in the same environment as Polars:
pip install 'polars-nanook>=0.13.0'
Enabling monitoring
import polars as pl
pl.Config.enable_monitoring()
The first call opens a browser window to authenticate your session if you are not already logged in. Every query you collect afterward reports its metrics to your Polars Nanook dashboard.
Use the streaming engine when monitoring. It is the only engine that reports per-node metrics, so
enabling monitoring also sets the engine affinity to "streaming".
Choosing a workspace
Metrics go to the default workspace of your account. Pass workspace to send them somewhere else,
and organization to disambiguate a workspace name that exists in more than one organization:
pl.Config.enable_monitoring(workspace="My workspace")
pl.Config.enable_monitoring(workspace="My workspace", organization="My organization")
Monitoring a single query
Monitoring usually stays on for the whole session. If you want it for one query only, Config also
works as a context manager: the previous monitoring state and the previous engine affinity are
restored on exit.
with pl.Config(enable_monitoring=True):
lf.collect()
Disabling monitoring
pl.Config.enable_monitoring(False)