Allow synchronous proxying of async JS functions by sbc100 · Pull Request #26000 · emscripten-core/emscripten

sbc100 · 2025-12-23T18:29:57Z

This new mode allows the proxied work itself to be asynchronous (i.e.
return a promise). The new behaviour is triggered by marking a
functions as both __proxy: 'sync' and also __async.

We can use this new mode to replace the bespoke proxying code that
exists for dlsync as well as the poll system call. For the poll
system call this bespoke code was added in #25523 and #25990, but can
now be completely removed.

In addition to this, because the poll syscall is now marked as
__async it automatically works under ASYNCIFY/JSPI too.

One downside of this new mode is that the proxied work requires the main
thread event loop to run. Unlike the synchronous proxied work that can
complete even when we are deeply nested.

Fixes: #25970

sbc100 · 2025-12-23T21:48:12Z

I found that combining __proxy: 'sync' and __async: true works pretty nicely. This also now means that poll() can block on the main thread when JSPI is enabled.

Non-function change split out from emscripten-core#26000.

Non-function change split out from #26000.

…H-136988) Basic support for pyrepl in Emscripten. Limitations: * requires JSPI * no signal handling implemented As followup work, it would be nice to implement a webworker variant for when JSPI is not available and proper signal handling. Because it requires JSPI, it doesn't work in Safari. Firefox requires setting an experimental flag. All the Chromiums have full support since May. Until we make it work without JSPI, let's keep the original web_example around. (cherry picked from commit c933a6b) Co-authored-by: Hood Chatham <roberthoodchatham@gmail.com> Co-authored-by: Łukasz Langa <lukasz@langa.pl> Co-authored-by: Éric <merwok@netwok.org>

sbc100 · 2026-01-21T00:01:57Z

@dschuff @kripken @tlively would you mind taking a look at this?

This is the piece that finally allows blocking calls to proxied async functions when called from threads. I'd like to land this before the next release because it actually restores the __syscall_poll JS library function which was removed in #25990. Even though this should have been an implementation detail it did manage to break python downstream. See #26132

kripken · 2026-01-21T00:56:31Z

src/jsifier.mjs

            if (oneliner) {
              body = `return ${body}`;
            }
+            let sync = 0;


What do these numbers 0,1,2 mean - where are they documented?

I added some docs in proxyToMainThread. This is internal-only so the docs only, so the docs don't need to be public in that sense.

Might be worth extracting into named const anyway, it doesn't cost anything but would make it a bit more readable.

@sbc100

I added some docs in proxyToMainThread.

Are those not pushed to the branch perhaps? I can't see them.

See proxyToMainThread changes in src/lib/libpthread.js

Here is that diff:

diff --git a/src/lib/libpthread.js b/src/lib/libpthread.js index 4ecbb2d63754e..f95fd8c8e3f0e 100644 --- a/src/lib/libpthread.js +++ b/src/lib/libpthread.js @@ -935,7 +935,7 @@ var LibraryPThread = { $proxyToMainThread__deps: ['$stackSave', '$stackRestore', '$stackAlloc', '_emscripten_run_js_on_main_thread'], $proxyToMainThread__docs: '/** @type{function(number, (number|boolean), ...number)} */', - $proxyToMainThread: (funcIndex, emAsmAddr, sync, ...callArgs) => { + $proxyToMainThread: (funcIndex, emAsmAddr, proxyMode, ...callArgs) => { // EM_ASM proxying is done by passing a pointer to the address of the EM_ASM // content as `emAsmAddr`. JS library proxying is done by passing an index // into `proxiedJSCallArgs` as `funcIndex`. If `emAsmAddr` is non-zero then @@ -944,8 +944,11 @@ var LibraryPThread = { // function arguments. // The serialization buffer contains the number of call params, and then // all the args here. - // We also pass 'sync' to C separately, since C needs to look at it. - // Allocate a buffer, which will be copied by the C code. + // + // We also pass 'proxyMode' to C separately, since C needs to look at it. + // + // Allocate a buffer (on the stack), which will be copied if necessary by + // the C code. // // First passed parameter specifies the number of arguments to the function. // When BigInt support is enabled, we must handle types in a more complex @@ -972,7 +975,7 @@ var LibraryPThread = { HEAPF64[b++] = arg; #endif } - var rtn = __emscripten_run_js_on_main_thread(funcIndex, emAsmAddr, bufSize, args, sync); + var rtn = __emscripten_run_js_on_main_thread(funcIndex, emAsmAddr, bufSize, args, proxyMode); stackRestore(sp); return rtn; },

I don't see an explanation of the values 0, 1, 2?

Sorry they were added initially to that function but now I've moved the docs in the C source code alongside __emscripten_run_js_on_main_thread which actually uses the values.

The values are not used here in the JS it turns out

system/lib/pthread/proxying.c

test/core/test_poll_blocking_asyncify.c

tlively

Unfortunately I don't really follow anything else going on here.

src/lib/libpthread.js

system/lib/pthread/proxying.c

kripken · 2026-01-21T16:48:53Z

src/jsifier.mjs

            if (oneliner) {
              body = `return ${body}`;
            }
+            let sync = 0;


@sbc100

I added some docs in proxyToMainThread.

Are those not pushed to the branch perhaps? I can't see them.

This new mode allows the proxied work itself to be asynchronous (i.e. return a promise). The new behaviour is triggered by marking a functions as both `__proxy: 'sync'` and also `__async`. We can use this new mode to replace the bespoke proxying code that exists for `dlsync` as well as the `poll` system call. For the poll system call this bespoke code was added in emscripten-core#25523 and emscripten-core#25990, but can now be completely removed. In addition to this, because the `poll` syscall is now marked as `__async` it automatically works under ASYNCIFY/JSPI too. One downside of this new mode is that the proxied work requires the main thread event loop to run. Unlike the synchronous proxied work that can complete even when we are deeply nested. Fixes: emscripten-core#25970

dschuff

Given that this is a fairly significant capability already, and this change makes it much more useful, I think we should expand the section on proxying docs/porting/pthreads.rst with more information on what proxying is, why and how you would want to use it, and maybe a couple of examples.

src/lib/libsyscall.js

kripken · 2026-01-21T21:02:11Z

src/lib/libpthread.js

-#endif
+    const callingThread = PThread.currentProxiedOperationCallerThread;
+    if (callingThread) {
+      return dlsyncThreadsAsync()


Suggested change

return dlsyncThreadsAsync()

return dlsyncThreadsAsync();

kripken · 2026-01-21T21:03:22Z

test/codesize/test_codesize_minimal_pthreads_memgrowth.json

+  "a.out.js.gz": 4056,
+  "a.out.nodebug.wasm": 19730,
+  "a.out.nodebug.wasm.gz": 9140,
+  "total": 27973,


I guess the extra size is to support the new proxying mode? edit: at least it's a small change.

sbc100 · 2026-01-21T21:22:02Z

Sorry, will follow up with docs and feedback

Non-function change split out from emscripten-core#26000.

This new mode allows the proxied work itself to be asynchronous (i.e. return a promise). The new behaviour is triggered by marking a functions as both `__proxy: 'sync'` and also `__async`. We can use this new mode to replace the bespoke proxying code that exists for `dlsync` as well as the `poll` system call. For the poll system call this bespoke code was added in emscripten-core#25523 and emscripten-core#25990, but can now be completely removed. In addition to this, because the `poll` syscall is now marked as `__async` it automatically works under ASYNCIFY/JSPI too. One downside of this new mode is that the proxied work requires the main thread event loop to run. Unlike the synchronous proxied work that can complete even when we are deeply nested. Fixes: emscripten-core#25970

sbc100 changed the title ~~Add new proxying mode for JS library functions~~ Add new async proxying mode for JS library functions Dec 23, 2025

sbc100 requested a review from kripken December 23, 2025 18:53

sbc100 force-pushed the poll_promising branch 3 times, most recently from 4c77b8f to 9ba3164 Compare December 23, 2025 21:43

sbc100 changed the title ~~Add new async proxying mode for JS library functions~~ Allow synchronous proxying of async JS functions Dec 23, 2025

sbc100 requested review from brendandahl and dschuff December 23, 2025 21:45

sbc100 force-pushed the poll_promising branch from 9ba3164 to 15b8332 Compare December 23, 2025 21:53

sbc100 requested a review from RReverser December 23, 2025 22:16

sbc100 added a commit to sbc100/emscripten that referenced this pull request Dec 23, 2025

Simplify isAsyncFunction calculation. NFC

b7b6b5b

Non-function change split out from emscripten-core#26000.

sbc100 mentioned this pull request Dec 23, 2025

Simplify isAsyncFunction calculation. NFC #26004

Merged

sbc100 added a commit that referenced this pull request Dec 23, 2025

Simplify isAsyncFunction calculation. NFC (#26004)

94e31db

Non-function change split out from #26000.

sbc100 force-pushed the poll_promising branch 2 times, most recently from d17af1e to dbc51db Compare December 23, 2025 23:20

sbc100 mentioned this pull request Jan 13, 2026

Alternative SOCKFS in Node.js environment + synchronous socket calls using JSPI #26041

Open

sbc100 force-pushed the poll_promising branch 5 times, most recently from b9db5cb to 9cb65fc Compare January 16, 2026 22:16

sbc100 enabled auto-merge (squash) January 16, 2026 22:16

sbc100 force-pushed the poll_promising branch 4 times, most recently from 89b2776 to e30484e Compare January 19, 2026 05:17

sbc100 mentioned this pull request Jan 19, 2026

[test] Consistently use require_engine test helper. NFC #26131

Merged

sbc100 force-pushed the poll_promising branch 2 times, most recently from 1f6e95c to 89a5526 Compare January 20, 2026 20:13

sbc100 force-pushed the poll_promising branch from 89a5526 to 252e00b Compare January 20, 2026 20:44

sbc100 mentioned this pull request Jan 20, 2026

__syscall_poll undefined symbol #26132

Closed

sbc100 force-pushed the poll_promising branch from 252e00b to 0e54b4a Compare January 21, 2026 00:07

kripken reviewed Jan 21, 2026

View reviewed changes

system/lib/pthread/proxying.c Outdated Show resolved Hide resolved

system/lib/pthread/proxying.c Outdated Show resolved Hide resolved

system/lib/pthread/proxying.c Show resolved Hide resolved

test/core/test_poll_blocking_asyncify.c Outdated Show resolved Hide resolved

sbc100 force-pushed the poll_promising branch 2 times, most recently from 0372b26 to e76d10a Compare January 21, 2026 01:11

tlively reviewed Jan 21, 2026

View reviewed changes

src/lib/libpthread.js Outdated Show resolved Hide resolved

kripken reviewed Jan 21, 2026

View reviewed changes

sbc100 force-pushed the poll_promising branch 2 times, most recently from 7cea7a6 to 36a6491 Compare January 21, 2026 18:26

sbc100 force-pushed the poll_promising branch from 36a6491 to 556f71f Compare January 21, 2026 18:27

dschuff reviewed Jan 21, 2026

View reviewed changes

src/lib/libsyscall.js Show resolved Hide resolved

kripken approved these changes Jan 21, 2026

View reviewed changes

sbc100 merged commit b120a17 into emscripten-core:main Jan 21, 2026
36 checks passed

sbc100 deleted the poll_promising branch January 21, 2026 21:04

sbc100 added a commit to sbc100/emscripten that referenced this pull request Jan 21, 2026

Feedback from emscripten-core#26000. NFC

3ba2aa0

sbc100 added a commit to sbc100/emscripten that referenced this pull request Jan 21, 2026

Feedback from emscripten-core#26000. NFC

d74624e

sbc100 added a commit that referenced this pull request Jan 21, 2026

Feedback from #26000. NFC (#26147)

8c3abf4

inolen pushed a commit to inolen/emscripten that referenced this pull request Feb 13, 2026

Simplify isAsyncFunction calculation. NFC (emscripten-core#26004)

3455ca3

Non-function change split out from emscripten-core#26000.

inolen pushed a commit to inolen/emscripten that referenced this pull request Feb 13, 2026

Feedback from emscripten-core#26000. NFC (emscripten-core#26147)

ba23861

sbc100 mentioned this pull request Feb 18, 2026

Calling fsync from a thread crashes #26295

Closed

Conversation

sbc100 commented Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sbc100 commented Dec 23, 2025

Uh oh!

sbc100 commented Jan 21, 2026

Uh oh!

kripken Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tlively left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dschuff left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kripken Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sbc100 commented Jan 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

sbc100 commented Dec 23, 2025 •

edited

Loading

kripken Jan 21, 2026 •

edited

Loading

kripken Jan 21, 2026 •

edited

Loading