Suspense boundaries/out of order streaming/anyhow like error handling (#2365)

* create static site generation helpers in the router crate

* work on integrating static site generation into fullstack

* move ssg into a separate crate

* integrate ssg with the launch builder

* simplify ssg example

* fix static_routes for child routes

* move CLI hot reloading websocket code into dioxus-hot-reload

* fix some unused imports

* use the same hot reloading websocket code for fullstack

* fix fullstack hot reloading

* move cli hot reloading logic into the hot reload crate

* ssg example working with dx serve

* add more examples

* fix clippy

* switch to a result for Element

* fix formatting

* fix hot reload doctest imports

* fix axum imports

* add show method to error context

* implement retaining nodes during suspense

* fix unterminated if statements

* communicate between tasks and suspense boundaries

* make suspense placeholders easier to use

* implement IntoDynNode and IntoVNode for more wrappers

* fix clippy examples

* fix rsx tests

* add streaming html utilities to the ssr package

* unify hydration and non-hydration ssr cache

* fix router with Result Element

* don't run server doc tests

* Fix hot reload websocket doc examples

* simple apps working with fullstack streaming

* fix preloading wasm

* Report errors encountered while streaming

* remove async from incremental renderer

* document new VirtualDom suspense methods

* make streaming work with incremental rendering

* fix static site generation

* document suspense structs

* create closure type; allow async event handlers in props; allow shorthand event handlers

* test forwarding event handlers with the shorthand syntax

* fix clippy

* fix imports in spawn async doctest

* fix empty rsx

* fix async result event handlers

* fix mounting router in multiple places

* Fix task dead cancel race condition

* simplify diffing before adding suspense

* fix binary size increase

* fix attribute diffing

* more diffing fixes

* create minimal fullstack feature

* smaller fullstack bundles

* allow mounting nodes that are already created and creating nodes without mounting them

* fix hot reload feature

* fix replacing components

* don't reclaim virtual nodes

* client side suspense working!

* fix CLI

* slightly smaller fullstack builds

* fix multiple suspended scopes

* fix merge errors

* yield back to tokio every few polls to fix suspending on many tasks at once

* remove logs

* document suspense boundary and update suspense example

* fix ssg

* make streaming optional

* fix some router and core tests

* fix suspense example

* fix serialization with out of order server futures

* add incremental streaming hackernews demo

* fix hackernews demo

* fix root hackernews redirect

* fix formatting

* add tests for suspense cases

* slightly smaller binaries

* slightly smaller

* improve error handling docs

* fix errors example link

* fix doc tests

* remove log file

* fix ssr cache type inference

* remove index.html

* fix ssg render template

* fix assigning ids on elements with dynamic attributes

* add desktop feature to the workspace examples

* remove router static generation example; ssg lives in the dioxus-static-generation package

* add a test for effects during suspense

* only run effects on mounted nodes

* fix multiple suspense roots

* fix node iterator

* fix closures without arguments

* fix dioxus-core readme doctest

* remove suspense logs

* fix scope stack

* fix clippy

* remove unused suspense boundary from hackernews

* assert that launch never returns for better compiler errors

* fix static generation launch function

* fix web renderer

* pass context providers into server functions

* add an example for FromContext

* clean up DioxusRouterExt

* fix server function context

* fix fullstack desktop example

* forward CLI serve settings to fullstack

* re-export serve config at the root of fullstack

* forward env directly instead of using a guard

* just set the port in the CLI for fullstack playwright tests

* fix fullstack dioxus-cli-config feature

* fix launch server merge conflicts

* fix fullstack launch context

* Merge branch 'main' into suspense-2.0

* fix fullstack html data

* remove drop virtual dom feature

* add a comment about only_write_templates binary size workaround

* remove explicit dependencies from use_server_future

* make ErrorContext and SuspenseContext more similar

* Tweak: small tweaks to tomls to make diff smaller

* only rerun components under suspense after the initial placeholders are sent to the client

* add module docs for suspense

* keep track of when suspense boundaries are resolved

* start implementing JS out of order streaming

* fix core tests

* implement the server side of suspense with js

* fix streaming ssr with nested suspense

* move streaming ssr code into fullstack

* revert minification changes

* serialize server future data as the html streams

* start loading scripts wasm immediately instead of defering the script

* very basic nested suspense example working with minimal html updates

* clean up some suspense/error docs

* fix hydrating nested pending server futures

* sort resolved boundaries by height

* Fix disconnecting clients while streaming

* fix static generation crate

* don't insert extra divs when hydrating streamed chunks

* wait to swap in the elements until they are hydrated

* remove inline streaming script

* hackernews partially working

* fix spa mode

* banish the open shadow dom

* fix removing placeholder

* set up streaming playwright test

* run web playwright tests on 9999 to avoid port conflicts with other local servers

* remove suspense nodes if the suspense boundary is replaced before the suspense resolves on the server

* ignore hydration of removed suspense boundaries

* use path based indexing to fix hydrating suspense after parent suspense with child is removed

* re-export dioxus error

* remove resolved suspense divs if the suspense boundary has been removed

* Fix client side initialized server futures

* ignore comment nodes while traversing nodes in core to avoid lists getting swapped out with suspense

* Pass initial hydration data to the client

* hide pre nodes

* don't panic if reclaiming an element fails

* fix scope stack when polling tasks

* improve deserialization out of length message

* Ok(VNode::placeholder()) -> VNode::empty()

* fix typo in rsx usage

* restore testing changes from suspense example

* clean up some logs and comments

* fix playwright tests

* clean up more changes in core

* clean up core tests

* remove anymap dependency

* clean up changes to hooks

* clean up changes in the router, rsx, and web

* revert changes to axum-hello-world

* fix use_server_future

* fix clippy in dioxus-core

* check that the next or previous node exist before checking if we should ignore them

* fix formatting

* fix suspense playwright test

* remove unused suspense code

* add more suspense playwright tests

* add more docs for error boundaries

* fix suspense core tests

* fix ErrorBoundary example

* remove a bunch of debug logging in js

* fix router failure_external_navigation

* use absolute paths in the interpreter build.rs

* strip '\r' while hashing ts files

* add a wrapper with a default error boundary and suspense boundary

* restore hot reloading

* ignore non-ts files when hashing

* sort ts files before hashing them

* fix rsx tests

* fix fullstack doc tests

* fix core tests

* fix axum auth example

* update suspense hydration diagram

* longer playwright build limit

* tiny fixes - spelling, formatting

* update diagram link

* remove comment and template nodes for suspense placeholders

* remove comment nodes as we hydrate text

* simplify hackernews example

* clean up hydrating text nodes

* switch to a separate environment variable for the base path for smaller binaries

* clean up file system html trait

* fix form data

* move streaming code into fullstack

* implement serialize and deserialize for CapturedError

* remove waits in the nested suspense playwright spec

* force sequential fullstack builds for CI

* longer nested suspense delay for CI

* fix --force-sequential flag

* wait to launch server until client build is done

---------

Co-authored-by: Jonathan Kelley <jkelleyrtp@gmail.com>

packages/html/src/web_sys_bind/events.rs

@@ -3,15 +3,13 @@ use crate::events::{
     AnimationData, CompositionData, KeyboardData, MouseData, PointerData, TouchData,
     TransitionData, WheelData,
 };
-use crate::file_data::{FileEngine, HasFileData};
-use crate::geometry::{
-    ClientPoint, ElementPoint, PagePoint, PixelsRect, PixelsSize, PixelsVector2D, ScreenPoint,
-};
+use crate::file_data::HasFileData;
+use crate::geometry::{ClientPoint, ElementPoint, PagePoint, ScreenPoint};
 use crate::input_data::{decode_key_location, decode_mouse_button_set, MouseButton};
 use crate::prelude::*;
 use keyboard_types::{Code, Key, Modifiers};
 use std::str::FromStr;
-use wasm_bindgen::{JsCast, JsValue};
+use wasm_bindgen::JsCast;
 use web_sys::{
     AnimationEvent, CompositionEvent, Event, KeyboardEvent, MouseEvent, PointerEvent, Touch,
     TouchEvent, TransitionEvent, WheelEvent,
@@ -427,30 +425,41 @@ impl From<&web_sys::Element> for MountedData {
 impl crate::RenderedElementBacking for web_sys::Element {
     fn get_scroll_offset(
         &self,
-    ) -> std::pin::Pin<Box<dyn std::future::Future<Output = crate::MountedResult<PixelsVector2D>>>>
-    {
+    ) -> std::pin::Pin<
+        Box<
+            dyn std::future::Future<Output = crate::MountedResult<crate::geometry::PixelsVector2D>>,
+        >,
+    > {
         let left = self.scroll_left();
         let top = self.scroll_top();
-        let result = Ok(PixelsVector2D::new(left as f64, top as f64));
+        let result = Ok(crate::geometry::PixelsVector2D::new(
+            left as f64,
+            top as f64,
+        ));
         Box::pin(async { result })
     }
 
     fn get_scroll_size(
         &self,
-    ) -> std::pin::Pin<Box<dyn std::future::Future<Output = crate::MountedResult<PixelsSize>>>>
-    {
+    ) -> std::pin::Pin<
+        Box<dyn std::future::Future<Output = crate::MountedResult<crate::geometry::PixelsSize>>>,
+    > {
         let width = self.scroll_width();
         let height = self.scroll_height();
-        let result = Ok(PixelsSize::new(width as f64, height as f64));
+        let result = Ok(crate::geometry::PixelsSize::new(
+            width as f64,
+            height as f64,
+        ));
         Box::pin(async { result })
     }
 
     fn get_client_rect(
         &self,
-    ) -> std::pin::Pin<Box<dyn std::future::Future<Output = crate::MountedResult<PixelsRect>>>>
-    {
+    ) -> std::pin::Pin<
+        Box<dyn std::future::Future<Output = crate::MountedResult<crate::geometry::PixelsRect>>>,
+    > {
         let rect = self.get_bounding_client_rect();
-        let result = Ok(PixelsRect::new(
+        let result = Ok(crate::geometry::PixelsRect::new(
             euclid::Point2D::new(rect.left(), rect.top()),
             euclid::Size2D::new(rect.width(), rect.height()),
         ));
@@ -481,6 +490,17 @@ impl crate::RenderedElementBacking for web_sys::Element {
         &self,
         focus: bool,
     ) -> std::pin::Pin<Box<dyn std::future::Future<Output = crate::MountedResult<()>>>> {
+        #[derive(Debug)]
+        struct FocusError(wasm_bindgen::JsValue);
+
+        impl std::fmt::Display for FocusError {
+            fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+                write!(f, "failed to focus element {:?}", self.0)
+            }
+        }
+
+        impl std::error::Error for FocusError {}
+
         let result = self
             .dyn_ref::<web_sys::HtmlElement>()
             .ok_or_else(|| crate::MountedError::OperationFailed(Box::new(FocusError(self.into()))))
@@ -492,17 +512,6 @@ impl crate::RenderedElementBacking for web_sys::Element {
     }
 }
 
-#[derive(Debug)]
-struct FocusError(JsValue);
-
-impl std::fmt::Display for FocusError {
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
-        write!(f, "failed to focus element {:?}", self.0)
-    }
-}
-
-impl std::error::Error for FocusError {}
-
 impl HasScrollData for Event {
     fn as_any(&self) -> &dyn std::any::Any {
         self
@@ -546,18 +555,15 @@ impl HasMediaData for web_sys::Event {
 }
 
 impl HasFileData for web_sys::Event {
-    fn files(&self) -> Option<std::sync::Arc<dyn FileEngine>> {
-        #[cfg(not(feature = "file_engine"))]
-        let files = None;
-        #[cfg(feature = "file_engine")]
-        let files = element
+    #[cfg(feature = "file-engine")]
+    fn files(&self) -> Option<std::sync::Arc<dyn crate::file_data::FileEngine>> {
+        let files = self
             .dyn_ref()
             .and_then(|input: &web_sys::HtmlInputElement| {
                 input.files().and_then(|files| {
                     #[allow(clippy::arc_with_non_send_sync)]
-                    crate::file_engine::WebFileEngine::new(files).map(|f| {
-                        std::sync::Arc::new(f) as std::sync::Arc<dyn dioxus_html::FileEngine>
-                    })
+                    crate::web_sys_bind::file_engine::WebFileEngine::new(files)
+                        .map(|f| std::sync::Arc::new(f) as std::sync::Arc<dyn crate::FileEngine>)
                 })
             });
 

packages/html/src/web_sys_bind/file_engine.rs

@@ -0,0 +1,133 @@
+use std::any::Any;
+
+use crate::FileEngine;
+use futures_channel::oneshot;
+use js_sys::Uint8Array;
+use wasm_bindgen::{prelude::Closure, JsCast};
+use web_sys::{File, FileList, FileReader};
+
+/// A file engine for the web platform
+pub struct WebFileEngine {
+    file_reader: FileReader,
+    file_list: FileList,
+}
+
+impl WebFileEngine {
+    /// Create a new file engine from a file list
+    pub fn new(file_list: FileList) -> Option<Self> {
+        Some(Self {
+            file_list,
+            file_reader: FileReader::new().ok()?,
+        })
+    }
+
+    fn len(&self) -> usize {
+        self.file_list.length() as usize
+    }
+
+    fn get(&self, index: usize) -> Option<File> {
+        self.file_list.item(index as u32)
+    }
+
+    fn find(&self, name: &str) -> Option<File> {
+        (0..self.len())
+            .filter_map(|i| self.get(i))
+            .find(|f| f.name() == name)
+    }
+}
+
+#[async_trait::async_trait(?Send)]
+impl FileEngine for WebFileEngine {
+    fn files(&self) -> Vec<String> {
+        (0..self.len())
+            .filter_map(|i| self.get(i).map(|f| f.name()))
+            .collect()
+    }
+
+    async fn file_size(&self, file: &str) -> Option<u64> {
+        let file = self.find(file)?;
+        Some(file.size() as u64)
+    }
+
+    // read a file to bytes
+    async fn read_file(&self, file: &str) -> Option<Vec<u8>> {
+        let file = self.find(file)?;
+
+        let file_reader = self.file_reader.clone();
+        let (rx, tx) = oneshot::channel();
+        let on_load: Closure<dyn FnMut()> = Closure::new({
+            let mut rx = Some(rx);
+            move || {
+                let result = file_reader.result();
+                let _ = rx
+                    .take()
+                    .expect("multiple files read without refreshing the channel")
+                    .send(result);
+            }
+        });
+
+        self.file_reader
+            .set_onload(Some(on_load.as_ref().unchecked_ref()));
+        on_load.forget();
+        self.file_reader.read_as_array_buffer(&file).ok()?;
+
+        if let Ok(Ok(js_val)) = tx.await {
+            let as_u8_arr = Uint8Array::new(&js_val);
+            let as_u8_vec = as_u8_arr.to_vec();
+
+            Some(as_u8_vec)
+        } else {
+            None
+        }
+    }
+
+    // read a file to string
+    async fn read_file_to_string(&self, file: &str) -> Option<String> {
+        let file = self.find(file)?;
+
+        let file_reader = self.file_reader.clone();
+        let (rx, tx) = oneshot::channel();
+        let on_load: Closure<dyn FnMut()> = Closure::new({
+            let mut rx = Some(rx);
+            move || {
+                let result = file_reader.result();
+                let _ = rx
+                    .take()
+                    .expect("multiple files read without refreshing the channel")
+                    .send(result);
+            }
+        });
+
+        self.file_reader
+            .set_onload(Some(on_load.as_ref().unchecked_ref()));
+        on_load.forget();
+        self.file_reader.read_as_text(&file).ok()?;
+
+        if let Ok(Ok(js_val)) = tx.await {
+            js_val.as_string()
+        } else {
+            None
+        }
+    }
+
+    async fn get_native_file(&self, file: &str) -> Option<Box<dyn Any>> {
+        let file = self.find(file)?;
+        Some(Box::new(file))
+    }
+}
+
+/// Helper trait for WebFileEngine
+#[async_trait::async_trait(?Send)]
+pub trait WebFileEngineExt {
+    /// returns web_sys::File
+    async fn get_web_file(&self, file: &str) -> Option<web_sys::File>;
+}
+
+#[async_trait::async_trait(?Send)]
+impl WebFileEngineExt for std::sync::Arc<dyn FileEngine> {
+    async fn get_web_file(&self, file: &str) -> Option<web_sys::File> {
+        let native_file = self.get_native_file(file).await?;
+        let ret = native_file.downcast::<web_sys::File>().ok()?;
+        Some(*ret)
+    }
+}

packages/html/src/web_sys_bind/mod.rs

@@ -1 +1,5 @@
 mod events;
+#[cfg(feature = "file-engine")]
+mod file_engine;
+#[cfg(feature = "file-engine")]
+pub use file_engine::*;