Altair performance questions
Hi team, thanks for your work on this package. Apologies for the dumb question, I have been using large datasets and used altair for dynamic plotting with slider filters.
Using on jupyter:
alt.data_transformers.enable("vegafusion")
alt.renderers.enable("jupyter", offline=True), this is very fast (less than 1 sec for the update).
I tried to achieve the same filtering using marimo sliders directly on my polars dataframe and then plotting, this works but it is slower as obviously polars needs to do the filtering and run 2 cells (2-5 seconds).
For this use case, the JupyterChart (jupyter renderer) is faster, is there any plan to have it supported in marimo ? (Other renderers being too slow for large datasets)
10 Replies
can you share an example with us? you can use
alt.data_transformers.enable("vegafusion")
in marimo as wellLet me send something tomorrow
marimo | Notebook Oct 16, 2024, 7:06 AM
Explore data and build apps seamlessly with marimo, a next-generation Python notebook.
with a heavier dataset, the difference becomes noticeable
https://altair-viz.github.io/user_guide/jupyter_chart.html
I'm basically asking about support for JupyterChart at some point
Thank you for looking into this
š
I cant see any slowness - it takes a 83ms for your example to re-render for me. (although its not a large dataset). happy to debug an example if you can send one with a larger dataset.
also, I was able to render a JupyterChart as it was shown in the altair docs:
We also have a doc about better performance utilizing Data Transformers
https://docs.marimo.io/api/plotting.html#performance-and-data-transformers
Thank you for looking into this! I'll have another look over the coming days š
"The JupyterChart widget and the "jupyter" renderer are designed to work with the VegaFusion data transformer to evaluate data transformations interactively in response to selection events. This avoids the need to transfer the full dataset to the browser, and so supports interactive exploration of aggregated datasets on the order of millions of rows." I'm curious, is marimo_csv doing the same thing under the hood?
https://marimo.io/p/dev/notebook-dn5hsm-vfmcetat8211da33ux796r
are you able to run the notebook with either renderers?
jupyter renderer (works instantly in jupyter FYI) gets me an output limit
marimo_csv plot is empty for some reason
I still need to get you an example of a heavy dataset where the re-rendering time is long
marimo_csv won't work like vegafusion, but you can also just use
vegafusion
which should be performant
I can look at the jupyter data_transformer as well to see if we can bridge some capabailitiesThat would be nice ! š thank you for your help!