btnaughton
btnaughton4mo ago

running marimo on modal

I actually posted this to github discussions, so apologies for double-positng. I think this is probably a better forum for this question... (Happy to delete the github one if that's helpful) I am trying to get marimo running on modal, following https://docs.marimo.io/guides/deploying/programmatically.html and https://docs.marimo.io/guides/deploying/authentication.html The following code works, up to the point of giving a 401 for /api/kernel/instantiate
import modal
from modal import asgi_app
import marimo

server = marimo.create_asgi_app().with_app(path="", root="/marimo/test.py")

app = modal.App()

@app.function(
image=modal.Image.debian_slim().pip_install("marimo==0.8.3", "fastapi"),
gpu=False,
mounts=[modal.Mount.from_local_dir("./marimo", remote_path="/marimo")],
)
@asgi_app()
def marimo_asgi():
return server.build()


if __name__ == "__main__":
modal.serve(app)
import modal
from modal import asgi_app
import marimo

server = marimo.create_asgi_app().with_app(path="", root="/marimo/test.py")

app = modal.App()

@app.function(
image=modal.Image.debian_slim().pip_install("marimo==0.8.3", "fastapi"),
gpu=False,
mounts=[modal.Mount.from_local_dir("./marimo", remote_path="/marimo")],
)
@asgi_app()
def marimo_asgi():
return server.build()


if __name__ == "__main__":
modal.serve(app)
I have tried adding middleware and I can see the requests come through. I can reject requests (return 401) but I can't figure out how to prevent a 401 for /api/kernel/instantiate
GET / -> 200 OK (duration: 117.8 ms, execution: 51.9 ms)
....
GET /android-chrome-192x192.png -> 200 OK (duration: 1.27 s, execution: 0.0 ms)
POST /api/kernel/instantiate -> 401 Unauthorized (duration: 1.17 s, execution: 13.1 ms)
CONNECT /ws -> 101 Switching Protocols (duration: 1.52 s, execution: 0.0 ms)
GET / -> 200 OK (duration: 117.8 ms, execution: 51.9 ms)
....
GET /android-chrome-192x192.png -> 200 OK (duration: 1.27 s, execution: 0.0 ms)
POST /api/kernel/instantiate -> 401 Unauthorized (duration: 1.17 s, execution: 13.1 ms)
CONNECT /ws -> 101 Switching Protocols (duration: 1.52 s, execution: 0.0 ms)
I think I am missing something fundamental about what's going on here, since I don't see where a token could be validated, or where the 401 is actually happening. I do see marimo-server-token in the header for /api/kernel/instantiate but no authorization or similar. If anyone has any ideas, I would appreciate it! Running marimo on modal would be fantastic.
22 Replies
btnaughton
btnaughtonOP4mo ago
Here is some example middleware code where I am trying to allow everything (note this is one of several attempts. There is no authorization in the header, but there is marimo-server-token . I can try turning on and off the auth_header check, edit the headers, etc. and no matter how permissive I try to make it, I get the same 401):
def auth_middleware(app):
async def middleware(scope, receive, send):
if scope['type'] == 'http':
headers = dict(scope['headers'])
auth_header = headers.get(b'authorization', b'').decode()
expected_auth = f"Bearer {TOKEN}"

if auth_header != expected_auth:
print(f"Authentication failed. Expected: {expected_auth}, Got: {auth_header}")
await send({
'type': 'http.response.start',
'status': 401,
'headers': [(b'content-type', b'text/plain')]
})
await send({
'type': 'http.response.body',
'body': b'Unauthorized\n'
})
return
await app(scope, receive, send)

return middleware
def auth_middleware(app):
async def middleware(scope, receive, send):
if scope['type'] == 'http':
headers = dict(scope['headers'])
auth_header = headers.get(b'authorization', b'').decode()
expected_auth = f"Bearer {TOKEN}"

if auth_header != expected_auth:
print(f"Authentication failed. Expected: {expected_auth}, Got: {auth_header}")
await send({
'type': 'http.response.start',
'status': 401,
'headers': [(b'content-type', b'text/plain')]
})
await send({
'type': 'http.response.body',
'body': b'Unauthorized\n'
})
return
await app(scope, receive, send)

return middleware
Akshay
Akshay4mo ago
Thanks for posting here, we monitor Discord more than GitHub discussions. We haven't tried running marimo on modal yet, but it would be very cool ...! Off the top of my head I don't know what the issue is but we can help debug
btnaughton
btnaughtonOP4mo ago
Thanks! I would be ashamed to admit how much time i've spent trying to get an AI to help figure out authentication here.....
Akshay
Akshay4mo ago
hah, no shame, we've all been there!
btnaughton
btnaughtonOP4mo ago
also you may be interested that the modal guys are "considering using marimo internally"
Akshay
Akshay4mo ago
Oh, cool! @Eric Zhang at modal was telling us a bit about features that they’re developing that could better support notebooks. Cool to hear that they’re also considering using it for their own work
Eric Zhang
Eric Zhang4mo ago
Yes I shared it with some folks in february, it probably won't happen until pyodide gets some kind of grpc support though We're most excited about developing an integrated backend with y'all so you can open Marimo notebooks locally or on the web, but plug into GPUs or other hardware resources in the cloud as desired
Eric Zhang
Eric Zhang4mo ago
Modal
Dynamic sandboxes
In addition to the function interface, Modal has a direct interface for defining containers at runtime and running arbitrary code inside them.
Myles Scolnick
Myles Scolnick4mo ago
@btnaughton - are these requests all servered from the same modal container? marimo is stateless, so if /instantiate hits a new container, it will return a 401 i just added concurrency_limit=1, - this fixes the issue you have above. but there seems to still be an issue with marimo running on modal. none of my outputs being streamed back (could be threading or some of the patches we add), we can investigate maybe its the size of the conatiner - ill try tweaking it
btnaughton
btnaughtonOP4mo ago
Ah....... thanks! I never would have thought of this. I am seeing the same thing, some kind of blockage. I do not see the POST /api/kernel/instantiate request any more for some reason so maybe it's getting stuck here. Thanks for unsticking me, I'll poke at it more now.
Myles Scolnick
Myles Scolnick4mo ago
yea, it seems that the /api/kernel/instantiate hangs. i tried to look in the modal logs, but didnt see anything. if you are more familiar with modal, you might be able to find why its hanging. we will also do our own debugging
btnaughton
btnaughtonOP4mo ago
allow_concurrent_inputs=2, seems to fix it?? ok as far as i can tell it's working 🎈 thanks!!
Myles Scolnick
Myles Scolnick4mo ago
Ah sweet! Probs because the ws is always connected
btnaughton
btnaughtonOP4mo ago
spoke a bit too soon, it dies after a few minutes, with a 401, so i assume it's changing containers or just dropping i tihnk that should be fixable with some kind of timeout or keep_warm or similar at some point modal has to disconnect or it's just a server...
Myles Scolnick
Myles Scolnick4mo ago
We don’t have a concept of resuming a session from a shutdown container. So you need to keep it alive for now.
btnaughton
btnaughtonOP4mo ago
also i don't see how to edit the code, but i think that's just basic misunderstanding of marimo!
Myles Scolnick
Myles Scolnick4mo ago
You cannot in run mode - which the asgi api for marimo is. You can while in edit - but that is through the cli
Akshay
Akshay4mo ago
Editing on a remote server is done via ssh tunneling: https://docs.marimo.io/faq.html#how-do-i-use-marimo-on-a-remote-server
btnaughton
btnaughtonOP4mo ago
got it -- good to know the asgi app is run only
Akshay
Akshay4mo ago
(I'll update the docs to clarify)
btnaughton
btnaughtonOP4mo ago
thanks again
Akshay
Akshay4mo ago
No problem! I've updated the deploying guide to distinguish between deploying an edit server and deploying notebooks as read-only web apps: https://docs.marimo.io/guides/deploying/index.html. Hope that helps, sorry for the confusion