mcburton•9mo ago

This might be more of a Pyodide question

This might be more of a Pyodide question, but I am trying to load a CSV file using Pandas read_csv in a WASM notebook. I've got the file on Github and loading it via the raw URL. In the WASM notebook I'm getting a URLError: <urlopen error unknown url type: https> and if I change to to an http url I get a URLError: <urlopen error [Errno 26] Operation in progress>. anyone know the issue here?

16 Replies

Myles Scolnick•9mo ago

Can you share your notebook? if you are able to. or send a minimal repro

mcburtonOP•9mo ago

Here is the repo: https://github.com/mcburton/you-vs-mmlu

GitHub

GitHub - mcburton/you-vs-mmlu: How do YOU fair against the MMLU ben...

How do YOU fair against the MMLU benchmark? Contribute to mcburton/you-vs-mmlu development by creating an account on GitHub.

mcburtonOP•9mo ago

warning, it is a HACK job 😉

mcburtonOP•9mo ago

WASM link : https://marimo.app/l/tvm5ru

marimo | a next-generation Python notebook

Explore data and build apps seamlessly with marimo, a next-generation Python notebook.

Myles Scolnick•9mo ago

you can do:

import pyodide
csv = pyodide.http.open_url("https://raw.githubusercontent.com/mcburton/you-vs-mmlu/main/data.csv")
questions = pd.read_csv(csv, header=None, names=columns)

import pyodide
csv = pyodide.http.open_url("https://raw.githubusercontent.com/mcburton/you-vs-mmlu/main/data.csv")
questions = pd.read_csv(csv, header=None, names=columns)

mcburtonOP•9mo ago

that worked! would that code work when not running in pyodide? or do I need some logic for the notebook to decide which approach to load the data?

Myles Scolnick•9mo ago

yea would need some logic - you can check if you are in pyodide or not by trying to import pyodide and catching the error we can add this to our docs, or maybe patch it pd.read_csv if its not too hacky

mcburtonOP•9mo ago

Is this a Marimo issue or a more general pyodide issue with Pandas...

Myles Scolnick•9mo ago

pyodide. they have particular ways to request data. and whatever pandas is using isnt compatible. but pyodide may add more compat to fetching libraries in future e.g. maybe this is what we need https://github.com/koenvo/pyodide-http it supports url and requests

mcburtonOP•9mo ago

yeah, might be good to note this in the docs. I would think fetching data from the web would be a common pattern for WASM notebooks.

Myles Scolnick•9mo ago

most definitely

mcburtonOP•9mo ago

sweet, well I got my little POC running for my noon meeting. Thanks! https://marimo.app/l/m93ysb?mode=read

marimo | a next-generation Python notebook

Explore data and build apps seamlessly with marimo, a next-generation Python notebook.

Akshay•9mo ago

@mcburton , very glad to see you trying out WASM notebooks! Thanks for being patient with these kinds of issues. Offer still stands -- I'd love to chat with you over a video call sometime. You have my email 🙂

mcburtonOP•9mo ago

Yes! It's on my todo list. I've got spring break next week so I can catch up on things. I'll be in touch soon

Mady•9mo ago

Just wanted to add on, I also had this issue and struggled to figure it out 🙂

Akshay•9mo ago

Thanks for letting us know — sorry about that! We will improve our documentation and work on finding a fix