Skip to content

Commit 68c5233

Browse files
committed
First refactor for WebIDL bindings
This commit starts the `wasm-bindgen` CLI tool down the road to being a true polyfill for WebIDL bindings. This refactor is probably the first of a few, but is hopefully the largest and most sprawling and everything will be a bit more targeted from here on out. The goal of this refactoring is to separate out the massive `crates/cli-support/src/js/mod.rs` into a number of separate pieces of functionality. It currently takes care of basically everything including: * Binding intrinsics * Handling anyref transformations * Generating all JS for imports/exports * All the logic for how to import and how to name imports * Execution and management of wasm-bindgen closures Many of these are separable concerns and most overlap with WebIDL bindings. The internal refactoring here is intended to make it more clear who's responsible for what as well as making some existing operations much more straightforward. At a high-level, the following changes are done: 1. A `src/webidl.rs` module is introduced. The purpose of this module is to take all of the raw wasm-bindgen custom sections from the module and transform them into a WebIDL bindings section. This module has a placeholder `WebidlCustomSection` which is nowhere near the actual custom section but if you squint is in theory very similar. It's hoped that this will eventually become the true WebIDL custom section, currently being developed in an external crate. Currently, however, the WebIDL bindings custom section only covers a subset of the functionality we export to wasm-bindgen users. To avoid leaving them high and dry this module also contains an auxiliary custom section named `WasmBindgenAux`. This custom section isn't intended to have a binary format, but is intended to represent a theoretical custom section necessary to couple with WebIDL bindings to achieve all our desired functionality in `wasm-bindgen`. It'll never be standardized, but it'll also never be serialized :) 2. The `src/webidl.rs` module now takes over quite a bit of functionality from `src/js/mod.rs`. Namely it handles synthesis of an `export_map` and an `import_map` mapping export/import IDs to exactly what's expected to be hooked up there. This does not include type information (as that's in the bindings section) but rather includes things like "this is the method of class A" or "this import is from module `foo`" and things like that. These could arguably be subsumed by future JS features as well, but that's for another time! 3. All handling of wasm-bindgen "descriptor functions" now happens in a dedicated `src/descriptors.rs` module. The output of this module is its own custom section (intended to be immediately consumed by the WebIDL module) which is in theory what we want to ourselves emit one day but rustc isn't capable of doing so right now. 4. Invocations and generations of imports are completely overhauled. Using the `import_map` generated in the WebIDL step all imports are now handled much more precisely in one location rather than haphazardly throughout the module. This means we have precise information about each import of the module and we only modify exactly what we're looking at. This also vastly simplifies intrinsic generation since it's all simply a codegen part of the `rust2js.rs` module now. 5. Handling of direct imports which don't have a JS shim generated is slightly different from before and is intended to be future-compatible with WebIDL bindings in its full glory, but we'll need to update it to handle cases for constructors and method calls eventually as well. 6. Intrinsic definitions now live in their own file (`src/intrinsic.rs`) and have a separated definition for their symbol name and signature. The actual implementation of each intrinsic lives in `rust2js.rs` There's a number of TODO items to finish before this merges. This includes reimplementing the anyref pass and actually implementing import maps for other targets. Those will come soon in follow-up commits, but the entire `tests/wasm/main.rs` suite is currently passing and this seems like a good checkpoint.
1 parent b11b6df commit 68c5233

File tree

18 files changed

+2826
-2209
lines changed

18 files changed

+2826
-2209
lines changed

Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -90,3 +90,4 @@ wasm-bindgen = { path = '.' }
9090
wasm-bindgen-futures = { path = 'crates/futures' }
9191
js-sys = { path = 'crates/js-sys' }
9292
web-sys = { path = 'crates/web-sys' }
93+
walrus = { git = 'https://github.com/rustwasm/walrus' }

crates/cli-support/src/descriptor.rs

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ tys! {
3838
CLAMPED
3939
}
4040

41-
#[derive(Debug)]
41+
#[derive(Debug, Clone)]
4242
pub enum Descriptor {
4343
I8,
4444
U8,
@@ -67,14 +67,14 @@ pub enum Descriptor {
6767
Clamped(Box<Descriptor>),
6868
}
6969

70-
#[derive(Debug)]
70+
#[derive(Debug, Clone)]
7171
pub struct Function {
7272
pub arguments: Vec<Descriptor>,
7373
pub shim_idx: u32,
7474
pub ret: Descriptor,
7575
}
7676

77-
#[derive(Debug)]
77+
#[derive(Debug, Clone)]
7878
pub struct Closure {
7979
pub shim_idx: u32,
8080
pub dtor_idx: u32,
@@ -146,9 +146,9 @@ impl Descriptor {
146146
}
147147
}
148148

149-
pub fn unwrap_function(&self) -> &Function {
150-
match *self {
151-
Descriptor::Function(ref f) => f,
149+
pub fn unwrap_function(self) -> Function {
150+
match self {
151+
Descriptor::Function(f) => *f,
152152
_ => panic!("not a function"),
153153
}
154154
}
@@ -199,10 +199,10 @@ impl Descriptor {
199199
}
200200
}
201201

202-
pub fn closure(&self) -> Option<&Closure> {
203-
match *self {
204-
Descriptor::Closure(ref s) => Some(s),
205-
_ => None,
202+
pub fn unwrap_closure(self) -> Closure {
203+
match self {
204+
Descriptor::Closure(s) => *s,
205+
_ => panic!("not a closure"),
206206
}
207207
}
208208

crates/cli-support/src/descriptors.rs

Lines changed: 200 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,200 @@
1+
//! Management of wasm-bindgen descriptor functions.
2+
//!
3+
//! The purpose of this module is to basically execute a pass on a raw wasm
4+
//! module that just came out of the compiler. The pass will execute all
5+
//! relevant descriptor functions contained in the module which wasm-bindgen
6+
//! uses to convey type infomation here, to the CLI.
7+
//!
8+
//! All descriptor functions are removed after this pass runs and in their stead
9+
//! a new custom section, defined in this module, is inserted into the
10+
//! `walrus::Module` which contains all the results of all the descriptor
11+
//! functions.
12+
13+
use crate::descriptor::{Closure, Descriptor};
14+
use failure::Error;
15+
use std::borrow::Cow;
16+
use std::collections::{HashMap, HashSet};
17+
use std::mem;
18+
use walrus::ImportId;
19+
use walrus::{CustomSection, FunctionId, LocalFunction, Module, TypedCustomSectionId};
20+
use wasm_bindgen_wasm_interpreter::Interpreter;
21+
22+
#[derive(Default, Debug)]
23+
pub struct WasmBindgenDescriptorsSection {
24+
pub descriptors: HashMap<String, Descriptor>,
25+
pub closure_imports: HashMap<ImportId, Closure>,
26+
}
27+
28+
pub type WasmBindgenDescriptorsSectionId = TypedCustomSectionId<WasmBindgenDescriptorsSection>;
29+
30+
/// Execute all `__wbindgen_describe_*` functions in a module, inserting a
31+
/// custom section which represents the executed value of each descriptor.
32+
///
33+
/// Afterwards this will delete all descriptor functions from the module.
34+
pub fn execute(module: &mut Module) -> Result<WasmBindgenDescriptorsSectionId, Error> {
35+
let mut section = WasmBindgenDescriptorsSection::default();
36+
let mut interpreter = Interpreter::new(module)?;
37+
38+
section.execute_exports(module, &mut interpreter)?;
39+
section.execute_closures(module, &mut interpreter)?;
40+
41+
// Delete all descriptor functions and imports from the module now that
42+
// we've executed all of them.
43+
walrus::passes::gc::run(module);
44+
45+
Ok(module.customs.add(section))
46+
}
47+
48+
impl WasmBindgenDescriptorsSection {
49+
fn execute_exports(
50+
&mut self,
51+
module: &mut Module,
52+
interpreter: &mut Interpreter,
53+
) -> Result<(), Error> {
54+
let mut to_remove = Vec::new();
55+
for export in module.exports.iter() {
56+
let prefix = "__wbindgen_describe_";
57+
if !export.name.starts_with(prefix) {
58+
continue;
59+
}
60+
let id = match export.item {
61+
walrus::ExportItem::Function(id) => id,
62+
_ => panic!("{} export not a function", export.name),
63+
};
64+
if let Some(d) = interpreter.interpret_descriptor(id, module) {
65+
let name = &export.name[prefix.len()..];
66+
let descriptor = Descriptor::decode(d);
67+
self.descriptors.insert(name.to_string(), descriptor);
68+
}
69+
to_remove.push(export.id());
70+
}
71+
72+
for id in to_remove {
73+
module.exports.delete(id);
74+
}
75+
Ok(())
76+
}
77+
78+
fn execute_closures(
79+
&mut self,
80+
module: &mut Module,
81+
interpreter: &mut Interpreter,
82+
) -> Result<(), Error> {
83+
use walrus::ir::*;
84+
85+
// If our describe closure intrinsic isn't present or wasn't linked
86+
// then there's no closures, so nothing to do!
87+
let wbindgen_describe_closure = match interpreter.describe_closure_id() {
88+
Some(i) => i,
89+
None => return Ok(()),
90+
};
91+
92+
// Find all functions which call `wbindgen_describe_closure`. These are
93+
// specially codegen'd so we know the rough structure of them. For each
94+
// one we delegate to the interpreter to figure out the actual result.
95+
let mut element_removal_list = HashSet::new();
96+
let mut func_to_descriptor = HashMap::new();
97+
for (id, local) in module.funcs.iter_local() {
98+
let entry = local.entry_block();
99+
let mut find = FindDescribeClosure {
100+
func: local,
101+
wbindgen_describe_closure,
102+
cur: entry.into(),
103+
call: None,
104+
};
105+
find.visit_block_id(&entry);
106+
if let Some(call) = find.call {
107+
let descriptor = interpreter
108+
.interpret_closure_descriptor(id, module, &mut element_removal_list)
109+
.unwrap();
110+
func_to_descriptor.insert(id, (call, Descriptor::decode(descriptor)));
111+
}
112+
}
113+
114+
// For all indirect functions that were closure descriptors, delete them
115+
// from the function table since we've executed them and they're not
116+
// necessary in the final binary.
117+
let table_id = match interpreter.function_table_id() {
118+
Some(id) => id,
119+
None => return Ok(()),
120+
};
121+
let table = module.tables.get_mut(table_id);
122+
let table = match &mut table.kind {
123+
walrus::TableKind::Function(f) => f,
124+
_ => unreachable!(),
125+
};
126+
for idx in element_removal_list {
127+
log::trace!("delete element {}", idx);
128+
assert!(table.elements[idx].is_some());
129+
table.elements[idx] = None;
130+
}
131+
132+
// And finally replace all calls of `wbindgen_describe_closure` with a
133+
// freshly manufactured import. Save off the type of this import in
134+
// ourselves, and then we're good to go.
135+
let ty = module.funcs.get(wbindgen_describe_closure).ty();
136+
for (func, (call_instr, descriptor)) in func_to_descriptor {
137+
let import_name = format!("__wbindgen_closure_wrapper{}", func.index());
138+
let id = module.add_import_func("__wbindgen_placeholder__", &import_name, ty);
139+
let import_id = module
140+
.imports
141+
.iter()
142+
.find(|i| i.name == import_name)
143+
.unwrap()
144+
.id();
145+
module.funcs.get_mut(id).name = Some(import_name);
146+
147+
let local = match &mut module.funcs.get_mut(func).kind {
148+
walrus::FunctionKind::Local(l) => l,
149+
_ => unreachable!(),
150+
};
151+
match local.get_mut(call_instr) {
152+
Expr::Call(e) => {
153+
assert_eq!(e.func, wbindgen_describe_closure);
154+
e.func = id;
155+
}
156+
_ => unreachable!(),
157+
}
158+
self.closure_imports
159+
.insert(import_id, descriptor.unwrap_closure());
160+
}
161+
return Ok(());
162+
163+
struct FindDescribeClosure<'a> {
164+
func: &'a LocalFunction,
165+
wbindgen_describe_closure: FunctionId,
166+
cur: ExprId,
167+
call: Option<ExprId>,
168+
}
169+
170+
impl<'a> Visitor<'a> for FindDescribeClosure<'a> {
171+
fn local_function(&self) -> &'a LocalFunction {
172+
self.func
173+
}
174+
175+
fn visit_expr_id(&mut self, id: &ExprId) {
176+
let prev = mem::replace(&mut self.cur, *id);
177+
id.visit(self);
178+
self.cur = prev;
179+
}
180+
181+
fn visit_call(&mut self, call: &Call) {
182+
call.visit(self);
183+
if call.func == self.wbindgen_describe_closure {
184+
assert!(self.call.is_none());
185+
self.call = Some(self.cur);
186+
}
187+
}
188+
}
189+
}
190+
}
191+
192+
impl CustomSection for WasmBindgenDescriptorsSection {
193+
fn name(&self) -> &str {
194+
"wasm-bindgen descriptors"
195+
}
196+
197+
fn data(&self, _: &walrus::IdsToIndices) -> Cow<[u8]> {
198+
panic!("shouldn't emit custom sections just yet");
199+
}
200+
}

0 commit comments

Comments
 (0)