Dissecting Deno
I had the chance to toy around with Deno recently. And with “toy around” I mean dissecting it into little pieces and see how the sausage was made. So, my view is not from a user’s perspective who wants to create and run apps with it, but rather one who has a huge interest in JavaScript runtimes, Serverless, and Rust.
Let me say that I learned a ton! And since I write down everything that I learn, I want to share my learnings with you. Disclaimer: There might be some things totally wrong. This is mostly a write-up from me browsing through the Deno repository and using the Deno crates myself. If I got something wrong, please let me know!
Also: Things might change! What you see here is more or less a snapshot in time.
A modern JavaScript runtime #
Deno markets itself as a modern runtime for JavaScript and TypeScript. Just like Node.js or the browser, its main task is to execute JavaScript. You’re able to write TypeScript and point Deno to your TypeScript files, but they get compiled in a pre-step through SWC.
Just like Node or Chrome, Deno builds upon Google’s V8 engine. The Deno team did a fantastic job in creating wonderful Rust bindings to V8 making installing and using V8 so incredibly simple. Pre-compiled V8 images for various architectures allow you to simply add a line in your Cargo.toml file.
And since Deno also builds upon V8, there are a lot of similarities between Deno and Node.js. Joyee Chung has given a fantastic talk on V8 internals at last year’s NodeConf Remote. In this talk, she explains how Node.js boots. I’m using this graphic I recreated from Joyee’s talk because the process in Node.js and Deno are very similar. But Joyee is much more of an authority than I am.
- The Node.js process starts. This boots up the V8 platform. The V8 platform is the platform-dependent bindings, so you can run V8 on all different operating systems. Initializing the process is in my experience actually the part that can take up quite some time.
- After that, Node creates a new V8 Isolate. The V8 isolate is an independent copy of the V8 runtime, including heap manager, garbage collector, etc. This runs on a single thread. Both these steps happen in native land.
- Now we enter JavaScript land. We initialize a new V8 context. A V8 Context includes the global object and JavaScript builtins. Things that make up the language, not the specific runtime. Up until this point, the browser, Node.js, and Deno are pretty much the same.
- In Node.js the runtime independent state, like the Node.js primordials are initialized. This means that all the JavaScript built-ins are cloned and frozen to be used for the runtime dependent states. So if users temper with the Object prototype or similar, this won’t affect Node.js features
- We start the event loop (Tokio in Deno, libuv in Node) and start the V8 inspector
- And finally, Node initializes the runtime dependent states. This is everything that’s related to the runtime you’re using. This means
process,require, etc. in Node.js,fetchin Deno,consoleeverywhere. - Load the main script and kick off the ol’ loop!
Let’s look at a bit of code.
Rusty V8 #
Rusty V8 contains, well, Rust bindings to V8. One of the nice things is that you don’t need to compile V8 each time, but you can rather use a prepared image because of some niceties in Rusty V8’s build.rs file. A file that’s executed the moment you install/build the crate (a package) along with your app.
Each crate from the Deno team includes a lot of very clean and easy-to-read examples that get rid of all the extras you need for running something like Deno. For example, hello_world.rs shows some of the most basic usages of V8:
// Rust!
use rusty_v8 as v8;
fn main() {
// Initialize V8.
let platform = v8::new_default_platform(0, false).make_shared();
v8::V8::initialize_platform(platform);
v8::V8::initialize();
{
// Create a new Isolate and make it the current one.
let isolate = &mut v8::Isolate::new(v8::CreateParams::default());
// Create a stack-allocated handle scope.
let handle_scope = &mut v8::HandleScope::new(isolate);
// Create a new context.
let context = v8::Context::new(handle_scope);
// Enter the context for compiling and running the hello world script.
let scope = &mut v8::ContextScope::new(handle_scope, context);
// Create a string containing the JavaScript source code.
let code = v8::String::new(scope, "'Hello' + ' World!'").unwrap();
// Compile the source code.
let script = v8::Script::compile(scope, code, None).unwrap();
// Run the script to get the result.
let result = script.run(scope).unwrap();
// Convert the result to a string and print it.
let result = result.to_string(scope).unwrap();
println!("{}", result.to_rust_string_lossy(scope));
// ...
}
unsafe {
v8::V8::dispose();
}
v8::V8::shutdown_platform();
}This couple of lines do everything V8-related: initializing the platform, creating one isolate, creating a context, and loading some basic JavaScript. A couple of remarks:
- You can have more than one isolate per platform. Think of a browser. Starting the browser you initialize the platform. Opening up a new tab creates a new isolate + context.
- If you think Serverless platforms, Cloudflare workers or Deno Deploy work very similarly. Their workers run in one V8 platform, but with each call, you can boot up a new isolate. With all the safety guarantees.
- The isolate has a global object and a context, but it lacks anything that you’re familiar with from working with Node.js, Deno, the browser. In this example, we just create a new JavaScript string that we try to get out of V8. No way to
console.log. No way to call any API that isn’t part of the language.
Booting up Deno core #
If we look at the actual JsRuntime, we see that Deno itself uses the V8 bindings a bit different (abbreviated):
// Rust!
pub fn new(mut options: RuntimeOptions) -> Self {
// Initialize the V8 platform once
let v8_platform = options.v8_platform.take();
static DENO_INIT: Once = Once::new();
DENO_INIT.call_once(move || v8_init(v8_platform));
let global_context;
// Init the Isolate + Context
let (mut isolate, maybe_snapshot_creator) = if options.will_snapshot {
// init code for an isolate that will snapshot
// snip!
(isolate, Some(creator))
} else {
// the other branch. Create a new isolate that
// might load a snapshot
// snip!
let isolate = v8::Isolate::new(params);
let mut isolate = JsRuntime::setup_isolate(isolate);
{
let scope = &mut v8::HandleScope::new(&mut isolate);
let context = if snapshot_loaded {
v8::Context::new(scope)
} else {
// If no snapshot is provided, we
// initialize the context with empty
// main source code and source maps.
bindings::initialize_context(scope)
};
global_context = v8::Global::new(scope, context);
}
(isolate, None)
};
// Attach a new insepector
let inspector =
JsRuntimeInspector::new(&mut isolate, global_context.clone());
// snip! See later
// ...
}So far, so good. A bit extra work for all the possibilities Deno offers. Then some of the interesting stuff happens. For example: attaching a module loader.
// Rust!
// Attach a module loader
let loader = options
.module_loader
.unwrap_or_else(|| Rc::new(NoopModuleLoader));The way modules are resolved is way different from Node, and handled via an extra module loader.
Copy the primordials and init the core ops #
Further down, Deno initializes the built-in extensions.
// Rust!
// Add builtins extension
options
.extensions
.insert(0, crate::ops_builtin::init_builtins());Built-ins are things like cloning the primordials.
// JavaScript
// Create copies of intrinsic objects
[
"AggregateError",
"Array",
"ArrayBuffer",
"BigInt",
"BigInt64Array",
"BigUint64Array",
"Boolean",
"DataView",
"Date",
"Error",
"EvalError",
"FinalizationRegistry",
"Float32Array",
"Float64Array",
"Function",
"Int16Array",
"Int32Array",
"Int8Array",
"Map",
"Number",
"Object",
"RangeError",
"ReferenceError",
"RegExp",
"Set",
"String",
"Symbol",
"SyntaxError",
"TypeError",
"URIError",
"Uint16Array",
"Uint32Array",
"Uint8Array",
"Uint8ClampedArray",
"WeakMap",
"WeakRef",
"WeakSet",
].forEach((name) => {
const original = globalThis[name];
primordials[name] = original;
copyPropsRenamed(original, primordials, name);
copyPrototype(original.prototype, primordials, `${name}Prototype`);
});Not only does this copy the original objects, but it also makes functions like Object.freeze available as ObjectFreeze, which is used further below:
// JavaScript
ObjectFreeze(primordials);
// Provide bootstrap namespace
globalThis.__bootstrap = { primordials };Other things include setting up the core and error behavior. The core adds functions to allow communicating between V8 and Rust using so-called “ops”. For example, this is the JavaScript side of printing something to stdout or stderr:
// JavaScript
function print(str, isErr = false) {
opSync("op_print", str, isErr);
}With opSync resolving to an opcall that has been initalized earlier on:
// Rust
// core/bidings.rs
set_func(scope, core_val, "opcall", opcall);The Rust side of print looks like that:
// Rust
/// Builtin utility to print to stdout/stderr
pub fn op_print(
_state: &mut OpState,
msg: String,
is_err: bool,
) -> Result<(), AnyError> {
if is_err {
stderr().write_all(msg.as_bytes())?;
stderr().flush().unwrap();
} else {
stdout().write_all(msg.as_bytes())?;
stdout().flush().unwrap();
}
Ok(())
}So from here on, we already have some deviation from all the other JavaScript runtimes. The moment we establish context, where we set the first bindings and where we load the core extensions.
This is the main Deno core.
Extensions that define the platform #
From here on, worker define other extensions that enable all the interesting Deno features:
// Rust
// Init extension ops
js_runtime.init_extension_ops().unwrap();
js_runtime.sync_ops_cache();
// Init async ops callback
js_runtime.init_recv_cb();
js_runtimeWhich features are loaded are defined by the workers. E.g. the main Deno worker loads this list of features:
// Rust
let extensions: Vec<Extension> = vec![
// Web APIs
deno_webidl::init(),
deno_console::init(),
deno_url::init(),
deno_web::init(options.blob_store.clone(), options.location.clone()),
deno_fetch::init::<Permissions>(
options.user_agent.clone(),
options.root_cert_store.clone(),
None,
None,
options.unsafely_ignore_certificate_errors.clone(),
None,
),
deno_websocket::init::<Permissions>(
options.user_agent.clone(),
options.root_cert_store.clone(),
options.unsafely_ignore_certificate_errors.clone(),
),
deno_webstorage::init(options.origin_storage_dir.clone()),
deno_crypto::init(options.seed),
deno_broadcast_channel::init(
options.broadcast_channel.clone(),
options.unstable,
),
deno_webgpu::init(options.unstable),
deno_timers::init::<Permissions>(),
// ffi
deno_ffi::init::<Permissions>(options.unstable),
// Metrics
metrics::init(),
// Runtime ops
ops::runtime::init(main_module.clone()),
ops::worker_host::init(options.create_web_worker_cb.clone()),
ops::fs_events::init(),
ops::fs::init(),
ops::io::init(),
ops::io::init_stdio(),
deno_tls::init(),
deno_net::init::<Permissions>(
options.root_cert_store.clone(),
options.unstable,
options.unsafely_ignore_certificate_errors.clone(),
),
ops::os::init(),
ops::permissions::init(),
ops::process::init(),
ops::signal::init(),
ops::tty::init(),
deno_http::init(),
ops::http::init(),
// Permissions ext (worker specific state)
perm_ext,
];You see a lot of features from the web here. Deno strives to be absolutely compatible with the web platform and does not want to create its own APIs. What you see here are extensions that enable Deno to have all these web platform features.
One of the important things is that the order of extensions in the vector matters. Deno is loading JavaScript after all, and you need to have e.g. console available before you can use it within the other extensions. Similarly, fetch can’t happen without having URLs.
Each extension loads a JavaScript part – an interface calling Deno ops (both sync and async), as well as a native plug-in written in Rust. The last one does the actual HTTP calls, or reads from the file system. It’s always back and forth between Deno land and native land.
After initalizing, we kick off the tokio event loop. But that’s another story, for another time.
What can you do with this? #
This happens all in the main runtime of Deno. But you can easily create your own runtime by putting together the right crates (each extension is available on its own on crates.io), and writing your own extensions. And I think this is where the real power of Deno lies: An easy way to use V8 everywhere, and form it to your needs.
// Rust
// define a couple of worker options
let options = WorkerOptions {
// ...
};
// load my main file, or a string ...
let js_path = Path::new("main.js");
let main_module = deno_core::resolve_path(&js_path.to_string_lossy())?;
// allow everything
let permissions = Permissions::allow_all();
// Initialize a runtime instance
// create a new deno worker!
let mut worker = MainWorker::from_options(
main_module.clone(),
permissions,
&options
);
let mut buf = BufferRedirect::stdout().unwrap();
worker.bootstrap(&options);
worker.execute_module(&main_module).await?;
// and let's go!!
worker.run_event_loop(false).await?;
Theoretically, you can recreate Node.js with it. It wouldn’t make a lot of sense, though. Other than that, you can provide a JavaScript runtime that e.g. console.logs to your cloud providers log engine. Or one that has a very reduced set of features to refactor response e.g. on an Edge network.
You can inject your own SDKs and access parts of your architecture that require authentication. Think of having an Edge network like Netlify or Cloudflare where you can rewrite HTTP responses and you have a ton of extra utlities available to do so.
You can have a V8 that runs serverless payloads that are tailored to their use-case. And the most important thing: Rust makes doing this tangible. Installing just parts of Deno is as easy as adding a line to Cargo.toml. This is the true power of Rust. Enabling people to do something they wouldn’t have done before.