btnaughton•3mo ago

running marimo on modal

I actually posted this to github discussions, so apologies for double-positng. I think this is probably a better forum for this question... (Happy to delete the github one if that's helpful) I am trying to get marimo running on modal, following https://docs.marimo.io/guides/deploying/programmatically.html and https://docs.marimo.io/guides/deploying/authentication.html The following code works, up to the point of giving a 401 for /api/kernel/instantiate

import modal
from modal import asgi_app
import marimo

server = marimo.create_asgi_app().with_app(path="", root="/marimo/test.py")

app = modal.App()

@app.function(
    image=modal.Image.debian_slim().pip_install("marimo==0.8.3", "fastapi"),
    gpu=False,
    mounts=[modal.Mount.from_local_dir("./marimo", remote_path="/marimo")],
)
@asgi_app()
def marimo_asgi():
    return server.build()


if __name__ == "__main__":
    modal.serve(app)

import modal
from modal import asgi_app
import marimo

server = marimo.create_asgi_app().with_app(path="", root="/marimo/test.py")

app = modal.App()

@app.function(
    image=modal.Image.debian_slim().pip_install("marimo==0.8.3", "fastapi"),
    gpu=False,
    mounts=[modal.Mount.from_local_dir("./marimo", remote_path="/marimo")],
)
@asgi_app()
def marimo_asgi():
    return server.build()


if __name__ == "__main__":
    modal.serve(app)

I have tried adding middleware and I can see the requests come through. I can reject requests (return 401) but I can't figure out how to prevent a 401 for /api/kernel/instantiate

    GET / -> 200 OK  (duration: 117.8 ms, execution: 51.9 ms)
    ....
    GET /android-chrome-192x192.png -> 200 OK  (duration: 1.27 s, execution: 0.0 ms)
   POST /api/kernel/instantiate -> 401 Unauthorized  (duration: 1.17 s, execution: 13.1 ms)
CONNECT /ws -> 101 Switching Protocols  (duration: 1.52 s, execution: 0.0 ms)

    GET / -> 200 OK  (duration: 117.8 ms, execution: 51.9 ms)
    ....
    GET /android-chrome-192x192.png -> 200 OK  (duration: 1.27 s, execution: 0.0 ms)
   POST /api/kernel/instantiate -> 401 Unauthorized  (duration: 1.17 s, execution: 13.1 ms)
CONNECT /ws -> 101 Switching Protocols  (duration: 1.52 s, execution: 0.0 ms)

I think I am missing something fundamental about what's going on here, since I don't see where a token could be validated, or where the 401 is actually happening. I do see marimo-server-token in the header for /api/kernel/instantiate but no authorization or similar. If anyone has any ideas, I would appreciate it! Running marimo on modal would be fantastic.

22 Replies

btnaughtonOP•3mo ago

Here is some example middleware code where I am trying to allow everything (note this is one of several attempts. There is no authorization in the header, but there is marimo-server-token . I can try turning on and off the auth_header check, edit the headers, etc. and no matter how permissive I try to make it, I get the same 401):

def auth_middleware(app):
    async def middleware(scope, receive, send):
        if scope['type'] == 'http':
            headers = dict(scope['headers'])
            auth_header = headers.get(b'authorization', b'').decode()
            expected_auth = f"Bearer {TOKEN}"

            if auth_header != expected_auth:
                print(f"Authentication failed. Expected: {expected_auth}, Got: {auth_header}")
                await send({
                    'type': 'http.response.start',
                    'status': 401,
                    'headers': [(b'content-type', b'text/plain')]
                })
                await send({
                    'type': 'http.response.body',
                    'body': b'Unauthorized\n'
                })
                return
        await app(scope, receive, send)

    return middleware

def auth_middleware(app):
    async def middleware(scope, receive, send):
        if scope['type'] == 'http':
            headers = dict(scope['headers'])
            auth_header = headers.get(b'authorization', b'').decode()
            expected_auth = f"Bearer {TOKEN}"

            if auth_header != expected_auth:
                print(f"Authentication failed. Expected: {expected_auth}, Got: {auth_header}")
                await send({
                    'type': 'http.response.start',
                    'status': 401,
                    'headers': [(b'content-type', b'text/plain')]
                })
                await send({
                    'type': 'http.response.body',
                    'body': b'Unauthorized\n'
                })
                return
        await app(scope, receive, send)

    return middleware

Akshay•3mo ago

Thanks for posting here, we monitor Discord more than GitHub discussions. We haven't tried running marimo on modal yet, but it would be very cool ...! Off the top of my head I don't know what the issue is but we can help debug

btnaughtonOP•3mo ago

Thanks! I would be ashamed to admit how much time i've spent trying to get an AI to help figure out authentication here.....

Akshay•3mo ago

hah, no shame, we've all been there!

btnaughtonOP•3mo ago

also you may be interested that the modal guys are "considering using marimo internally"

Akshay•3mo ago

Oh, cool! @Eric Zhang at modal was telling us a bit about features that they’re developing that could better support notebooks. Cool to hear that they’re also considering using it for their own work

Eric Zhang•3mo ago

Yes I shared it with some folks in february, it probably won't happen until pyodide gets some kind of grpc support though We're most excited about developing an integrated backend with y'all so you can open Marimo notebooks locally or on the web, but plug into GPUs or other hardware resources in the cloud as desired

Eric Zhang•3mo ago

https://modal.com/docs/guide/sandbox

Modal

Dynamic sandboxes

In addition to the function interface, Modal has a direct interface for defining containers at runtime and running arbitrary code inside them.

Myles Scolnick•3mo ago

@btnaughton - are these requests all servered from the same modal container? marimo is stateless, so if /instantiate hits a new container, it will return a 401 i just added concurrency_limit=1, - this fixes the issue you have above. but there seems to still be an issue with marimo running on modal. none of my outputs being streamed back (could be threading or some of the patches we add), we can investigate maybe its the size of the conatiner - ill try tweaking it

btnaughtonOP•3mo ago

Ah....... thanks! I never would have thought of this. I am seeing the same thing, some kind of blockage. I do not see the POST /api/kernel/instantiate request any more for some reason so maybe it's getting stuck here. Thanks for unsticking me, I'll poke at it more now.

Myles Scolnick•3mo ago

yea, it seems that the /api/kernel/instantiate hangs. i tried to look in the modal logs, but didnt see anything. if you are more familiar with modal, you might be able to find why its hanging. we will also do our own debugging

btnaughtonOP•3mo ago

allow_concurrent_inputs=2, seems to fix it?? ok as far as i can tell it's working 🎈 thanks!!

Myles Scolnick•3mo ago

Ah sweet! Probs because the ws is always connected

btnaughtonOP•3mo ago

spoke a bit too soon, it dies after a few minutes, with a 401, so i assume it's changing containers or just dropping i tihnk that should be fixable with some kind of timeout or keep_warm or similar at some point modal has to disconnect or it's just a server...

Myles Scolnick•3mo ago

We don’t have a concept of resuming a session from a shutdown container. So you need to keep it alive for now.

btnaughtonOP•3mo ago

also i don't see how to edit the code, but i think that's just basic misunderstanding of marimo!

Myles Scolnick•3mo ago

You cannot in run mode - which the asgi api for marimo is. You can while in edit - but that is through the cli

Akshay•3mo ago

Editing on a remote server is done via ssh tunneling: https://docs.marimo.io/faq.html#how-do-i-use-marimo-on-a-remote-server

btnaughtonOP•3mo ago

got it -- good to know the asgi app is run only

Akshay•3mo ago

(I'll update the docs to clarify)

btnaughtonOP•3mo ago

thanks again

Akshay•3mo ago

No problem! I've updated the deploying guide to distinguish between deploying an edit server and deploying notebooks as read-only web apps: https://docs.marimo.io/guides/deploying/index.html. Hope that helps, sorry for the confusion