simaiinox•4w ago

register data source / dataframe?

Say I have a python call that returns a dataframe. without assigning it to a variable within the notebook itself, is there a way I can register it as a data source? like from a module i'm importing? Example:

def run_agent(input):
   for event in agent.run(input):
      # if is tabular
      df = pd.DataFrame(event.content)
      **mo.state.register_datasource(df, event.name + event.tool_args + "_df") # <------ saves df to var "nearest_gene_tool_chr15_88569444_df"**
      mo.output.append(df)
      ...
    ...

def run_agent(input):
   for event in agent.run(input):
      # if is tabular
      df = pd.DataFrame(event.content)
      **mo.state.register_datasource(df, event.name + event.tool_args + "_df") # <------ saves df to var "nearest_gene_tool_chr15_88569444_df"**
      mo.output.append(df)
      ...
    ...

7 Replies

Myles Scolnick•4w ago

not currently. As a workaround: you may be able to put this in a dict in mo.state({}) and then have a dropdown selected_df = mo.ui.dropdown(get_state()) and build your own "data source explorer"

simaiinoxOP•4w ago

ok! is this something you would be interested in supporting in the longer term?

Myles Scolnick•4w ago

yea, i think we could add support for this, esepcially if you are open to contributing it. we would likely need a way to handle cleanup too (maybe just weakrefs) i can see a few advanced tooling like mo.editor.register_dasource

simaiinoxOP•4w ago

yes, i can def help implement it!! is a github issue the preferred place to work out the spec?

Akshay•4w ago

Yea GitHub issue is fine. Just to clarify, what's the main reason you'd like to register as a data source? Is it to see the schema in the data sources panel?

simaiinoxOP•4w ago

the data sources panel is def one big one, helps summarize to a user what data an agent has fetched already in my use case; the other reason is to then be able to reference them using the @ syntax in the ai generated cells

simaiinoxOP•4w ago

filed: https://github.com/marimo-team/marimo/issues/2766

GitHub

Description Right now, only global scope dataframes are saved by marimo for reference. However, I'd love to be able to have them saved to marimo without explicitly defining it as a variable For...