Suspense boundaries/out of order streaming/anyhow like error handling (#2365)

* create static site generation helpers in the router crate * work on integrating static site generation into fullstack * move ssg into a separate crate * integrate ssg with the launch builder * simplify ssg example * fix static_routes for child routes * move CLI hot reloading websocket code into dioxus-hot-reload * fix some unused imports * use the same hot reloading websocket code for fullstack * fix fullstack hot reloading * move cli hot reloading logic into the hot reload crate * ssg example working with dx serve * add more examples * fix clippy * switch to a result for Element * fix formatting * fix hot reload doctest imports * fix axum imports * add show method to error context * implement retaining nodes during suspense * fix unterminated if statements * communicate between tasks and suspense boundaries * make suspense placeholders easier to use * implement IntoDynNode and IntoVNode for more wrappers * fix clippy examples * fix rsx tests * add streaming html utilities to the ssr package * unify hydration and non-hydration ssr cache * fix router with Result Element * don't run server doc tests * Fix hot reload websocket doc examples * simple apps working with fullstack streaming * fix preloading wasm * Report errors encountered while streaming * remove async from incremental renderer * document new VirtualDom suspense methods * make streaming work with incremental rendering * fix static site generation * document suspense structs * create closure type; allow async event handlers in props; allow shorthand event handlers * test forwarding event handlers with the shorthand syntax * fix clippy * fix imports in spawn async doctest * fix empty rsx * fix async result event handlers * fix mounting router in multiple places * Fix task dead cancel race condition * simplify diffing before adding suspense * fix binary size increase * fix attribute diffing * more diffing fixes * create minimal fullstack feature * smaller fullstack bundles * allow mounting nodes that are already created and creating nodes without mounting them * fix hot reload feature * fix replacing components * don't reclaim virtual nodes * client side suspense working! * fix CLI * slightly smaller fullstack builds * fix multiple suspended scopes * fix merge errors * yield back to tokio every few polls to fix suspending on many tasks at once * remove logs * document suspense boundary and update suspense example * fix ssg * make streaming optional * fix some router and core tests * fix suspense example * fix serialization with out of order server futures * add incremental streaming hackernews demo * fix hackernews demo * fix root hackernews redirect * fix formatting * add tests for suspense cases * slightly smaller binaries * slightly smaller * improve error handling docs * fix errors example link * fix doc tests * remove log file * fix ssr cache type inference * remove index.html * fix ssg render template * fix assigning ids on elements with dynamic attributes * add desktop feature to the workspace examples * remove router static generation example; ssg lives in the dioxus-static-generation package * add a test for effects during suspense * only run effects on mounted nodes * fix multiple suspense roots * fix node iterator * fix closures without arguments * fix dioxus-core readme doctest * remove suspense logs * fix scope stack * fix clippy * remove unused suspense boundary from hackernews * assert that launch never returns for better compiler errors * fix static generation launch function * fix web renderer * pass context providers into server functions * add an example for FromContext * clean up DioxusRouterExt * fix server function context * fix fullstack desktop example * forward CLI serve settings to fullstack * re-export serve config at the root of fullstack * forward env directly instead of using a guard * just set the port in the CLI for fullstack playwright tests * fix fullstack dioxus-cli-config feature * fix launch server merge conflicts * fix fullstack launch context * Merge branch 'main' into suspense-2.0 * fix fullstack html data * remove drop virtual dom feature * add a comment about only_write_templates binary size workaround * remove explicit dependencies from use_server_future * make ErrorContext and SuspenseContext more similar * Tweak: small tweaks to tomls to make diff smaller * only rerun components under suspense after the initial placeholders are sent to the client * add module docs for suspense * keep track of when suspense boundaries are resolved * start implementing JS out of order streaming * fix core tests * implement the server side of suspense with js * fix streaming ssr with nested suspense * move streaming ssr code into fullstack * revert minification changes * serialize server future data as the html streams * start loading scripts wasm immediately instead of defering the script * very basic nested suspense example working with minimal html updates * clean up some suspense/error docs * fix hydrating nested pending server futures * sort resolved boundaries by height * Fix disconnecting clients while streaming * fix static generation crate * don't insert extra divs when hydrating streamed chunks * wait to swap in the elements until they are hydrated * remove inline streaming script * hackernews partially working * fix spa mode * banish the open shadow dom * fix removing placeholder * set up streaming playwright test * run web playwright tests on 9999 to avoid port conflicts with other local servers * remove suspense nodes if the suspense boundary is replaced before the suspense resolves on the server * ignore hydration of removed suspense boundaries * use path based indexing to fix hydrating suspense after parent suspense with child is removed * re-export dioxus error * remove resolved suspense divs if the suspense boundary has been removed * Fix client side initialized server futures * ignore comment nodes while traversing nodes in core to avoid lists getting swapped out with suspense * Pass initial hydration data to the client * hide pre nodes * don't panic if reclaiming an element fails * fix scope stack when polling tasks * improve deserialization out of length message * Ok(VNode::placeholder()) -> VNode::empty() * fix typo in rsx usage * restore testing changes from suspense example * clean up some logs and comments * fix playwright tests * clean up more changes in core * clean up core tests * remove anymap dependency * clean up changes to hooks * clean up changes in the router, rsx, and web * revert changes to axum-hello-world * fix use_server_future * fix clippy in dioxus-core * check that the next or previous node exist before checking if we should ignore them * fix formatting * fix suspense playwright test * remove unused suspense code * add more suspense playwright tests * add more docs for error boundaries * fix suspense core tests * fix ErrorBoundary example * remove a bunch of debug logging in js * fix router failure_external_navigation * use absolute paths in the interpreter build.rs * strip '\r' while hashing ts files * add a wrapper with a default error boundary and suspense boundary * restore hot reloading * ignore non-ts files when hashing * sort ts files before hashing them * fix rsx tests * fix fullstack doc tests * fix core tests * fix axum auth example * update suspense hydration diagram * longer playwright build limit * tiny fixes - spelling, formatting * update diagram link * remove comment and template nodes for suspense placeholders * remove comment nodes as we hydrate text * simplify hackernews example * clean up hydrating text nodes * switch to a separate environment variable for the base path for smaller binaries * clean up file system html trait * fix form data * move streaming code into fullstack * implement serialize and deserialize for CapturedError * remove waits in the nested suspense playwright spec * force sequential fullstack builds for CI * longer nested suspense delay for CI * fix --force-sequential flag * wait to launch server until client build is done --------- Co-authored-by: Jonathan Kelley <jkelleyrtp@gmail.com>
2024-07-01 20:50:36 -07:00
1 parent ffa36a67c3
commit 022e4ad203
@@ -1,7 +1,7 @@
 use crate::{
    innerlude::{ElementRef, WriteMutations},
    nodes::VNode,
-    DynamicNode, ScopeId, TemplateNode, VirtualDom,
+    DynamicNode, ScopeId, VirtualDom,
 };

 use rustc_hash::{FxHashMap, FxHashSet};
@@ -9,7 +9,7 @@ use rustc_hash::{FxHashMap, FxHashSet};
 impl VirtualDom {
    pub(crate) fn diff_non_empty_fragment(
        &mut self,
-        to: &mut impl WriteMutations,
+        to: Option<&mut impl WriteMutations>,
        old: &[VNode],
        new: &[VNode],
        parent: Option<ElementRef>,
@@ -42,7 +42,7 @@ impl VirtualDom {
    // the change list stack is in the same state when this function returns.
    fn diff_non_keyed_children(
        &mut self,
-        to: &mut impl WriteMutations,
+        mut to: Option<&mut impl WriteMutations>,
        old: &[VNode],
        new: &[VNode],
        parent: Option<ElementRef>,
@@ -54,15 +54,18 @@ impl VirtualDom {
        debug_assert!(!old.is_empty());

        match old.len().cmp(&new.len()) {
-            Ordering::Greater => self.remove_nodes(to, &old[new.len()..], None),
-            Ordering::Less => {
-                self.create_and_insert_after(to, &new[old.len()..], old.last().unwrap(), parent)
-            }
+            Ordering::Greater => self.remove_nodes(to.as_deref_mut(), &old[new.len()..], None),
+            Ordering::Less => self.create_and_insert_after(
+                to.as_deref_mut(),
+                &new[old.len()..],
+                old.last().unwrap(),
+                parent,
+            ),
            Ordering::Equal => {}
        }

        for (new, old) in new.iter().zip(old.iter()) {
-            old.diff_node(new, self, to);
+            old.diff_node(new, self, to.as_deref_mut());
        }
    }

@@ -84,7 +87,7 @@ impl VirtualDom {
    // The stack is empty upon entry.
    fn diff_keyed_children(
        &mut self,
-        to: &mut impl WriteMutations,
+        mut to: Option<&mut impl WriteMutations>,
        old: &[VNode],
        new: &[VNode],
        parent: Option<ElementRef>,
@@ -116,10 +119,11 @@ impl VirtualDom {
        //
        // `shared_prefix_count` is the count of how many nodes at the start of
        // `new` and `old` share the same keys.
-        let (left_offset, right_offset) = match self.diff_keyed_ends(to, old, new, parent) {
-            Some(count) => count,
-            None => return,
-        };
+        let (left_offset, right_offset) =
+            match self.diff_keyed_ends(to.as_deref_mut(), old, new, parent) {
+                Some(count) => count,
+                None => return,
+            };

        // Ok, we now hopefully have a smaller range of children in the middle
        // within which to re-order nodes with the same keys, remove old nodes with
@@ -164,7 +168,7 @@ impl VirtualDom {
    /// If there is no offset, then this function returns None and the diffing is complete.
    fn diff_keyed_ends(
        &mut self,
-        to: &mut impl WriteMutations,
+        mut to: Option<&mut impl WriteMutations>,
        old: &[VNode],
        new: &[VNode],
        parent: Option<ElementRef>,
@@ -176,7 +180,7 @@ impl VirtualDom {
            if old.key != new.key {
                break;
            }
-            old.diff_node(new, self, to);
+            old.diff_node(new, self, to.as_deref_mut());
            left_offset += 1;
        }

@@ -201,7 +205,7 @@ impl VirtualDom {
            if old.key != new.key {
                break;
            }
-            old.diff_node(new, self, to);
+            old.diff_node(new, self, to.as_deref_mut());
            right_offset += 1;
        }

@@ -224,7 +228,7 @@ impl VirtualDom {
    #[allow(clippy::too_many_lines)]
    fn diff_keyed_middle(
        &mut self,
-        to: &mut impl WriteMutations,
+        mut to: Option<&mut impl WriteMutations>,
        old: &[VNode],
        new: &[VNode],
        parent: Option<ElementRef>,
@@ -236,7 +240,7 @@ impl VirtualDom {
            - IE if we have ABCD becomes BACD, our sequence would be 1,0,2,3
            - if we have ABCD to ABDE, our sequence would be 0,1,3,MAX because E doesn't exist

-        now, we should have a list of integers that indicates where in the old list the new items mapto.
+        now, we should have a list of integers that indicates where in the old list the new items map to.

        4. Compute the LIS of this list
            - this indicates the longest list of new children that won't need to be moved.
@@ -257,11 +261,11 @@ impl VirtualDom {

        // 1. Map the old keys into a numerical ordering based on indices.
        // 2. Create a map of old key to its index
-        // IE if the keys were A B C, then we would have (A, 1) (B, 2) (C, 3).
+        // IE if the keys were A B C, then we would have (A, 0) (B, 1) (C, 2).
        let old_key_to_old_index = old
            .iter()
            .enumerate()
-            .map(|(i, o)| (o.key.as_ref().unwrap(), i))
+            .map(|(i, o)| (o.key.as_ref().unwrap().as_str(), i))
            .collect::<FxHashMap<_, _>>();

        let mut shared_keys = FxHashSet::default();
@@ -271,163 +275,194 @@ impl VirtualDom {
            .iter()
            .map(|node| {
                let key = node.key.as_ref().unwrap();
-                if let Some(&index) = old_key_to_old_index.get(&key) {
+                if let Some(&index) = old_key_to_old_index.get(key.as_str()) {
                    shared_keys.insert(key);
                    index
                } else {
-                    u32::MAX as usize
+                    usize::MAX
                }
            })
-            .collect::<Vec<_>>();
+            .collect::<Box<[_]>>();

        // If none of the old keys are reused by the new children, then we remove all the remaining old children and
        // create the new children afresh.
        if shared_keys.is_empty() {
-            if !old.is_empty() {
-                let m = self.create_children(to, new, parent);
-                self.remove_nodes(to, old, Some(m));
-            } else {
-                // I think this is wrong - why are we appending?
-                // only valid of the if there are no trailing elements
-                // self.create_and_append_children(new);
+            debug_assert!(
+                !old.is_empty(),
+                "we should never be appending - just creating N"
+            );
+
+            let m = self.create_children(to.as_deref_mut(), new, parent);
+            self.remove_nodes(to, old, Some(m));

-                todo!("we should never be appending - just creating N");
-            }
            return;
        }

        // remove any old children that are not shared
-        // todo: make this an iterator
-        for child in old {
-            let key = child.key.as_ref().unwrap();
-            if !shared_keys.contains(&key) {
-                child.remove_node(self, to, None, true);
-            }
+        for child_to_remove in old
+            .iter()
+            .filter(|child| !shared_keys.contains(child.key.as_ref().unwrap()))
+        {
+            child_to_remove.remove_node(self, to.as_deref_mut(), None);
        }

        // 4. Compute the LIS of this list
        let mut lis_sequence = Vec::with_capacity(new_index_to_old_index.len());

-        let mut predecessors = vec![0; new_index_to_old_index.len()];
-        let mut starts = vec![0; new_index_to_old_index.len()];
+        let mut allocation = vec![0; new_index_to_old_index.len() * 2];
+        let (predecessors, starts) = allocation.split_at_mut(new_index_to_old_index.len());

        longest_increasing_subsequence::lis_with(
            &new_index_to_old_index,
            &mut lis_sequence,
            |a, b| a < b,
-            &mut predecessors,
-            &mut starts,
+            predecessors,
+            starts,
        );

-        // the lis comes out backwards, I think. can't quite tell.
-        lis_sequence.sort_unstable();
-
        // if a new node gets u32 max and is at the end, then it might be part of our LIS (because u32 max is a valid LIS)
-        if lis_sequence.last().map(|f| new_index_to_old_index[*f]) == Some(u32::MAX as usize) {
-            lis_sequence.pop();
+        if lis_sequence.first().map(|f| new_index_to_old_index[*f]) == Some(usize::MAX) {
+            lis_sequence.remove(0);
        }

+        // Diff each nod in the LIS
        for idx in &lis_sequence {
-            old[new_index_to_old_index[*idx]].diff_node(&new[*idx], self, to);
+            old[new_index_to_old_index[*idx]].diff_node(&new[*idx], self, to.as_deref_mut());
        }

-        let mut nodes_created = 0;
+        /// Create or diff each node in a range depending on whether it is in the LIS or not
+        /// Returns the number of nodes created on the stack
+        fn create_or_diff(
+            vdom: &mut VirtualDom,
+            new: &[VNode],
+            old: &[VNode],
+            mut to: Option<&mut impl WriteMutations>,
+            parent: Option<ElementRef>,
+            new_index_to_old_index: &[usize],
+            range: std::ops::Range<usize>,
+        ) -> usize {
+            let range_start = range.start;
+            new[range]
+                .iter()
+                .enumerate()
+                .map(|(idx, new_node)| {
+                    let new_idx = range_start + idx;
+                    let old_index = new_index_to_old_index[new_idx];
+                    // If the node existed in the old list, diff it
+                    if let Some(old_node) = old.get(old_index) {
+                        old_node.diff_node(new_node, vdom, to.as_deref_mut());
+                        if let Some(to) = to.as_deref_mut() {
+                            new_node.push_all_root_nodes(vdom, to)
+                        } else {
+                            0
+                        }
+                    } else {
+                        // Otherwise, just add it to the stack
+                        new_node.create(vdom, parent, to.as_deref_mut())
+                    }
+                })
+                .sum()
+        }

-        // add mount instruction for the first items not covered by the lis
-        let last = *lis_sequence.last().unwrap();
+        // add mount instruction for the items before the LIS
+        let last = *lis_sequence.first().unwrap();
        if last < (new.len() - 1) {
-            for (idx, new_node) in new[(last + 1)..].iter().enumerate() {
-                let new_idx = idx + last + 1;
-                let old_index = new_index_to_old_index[new_idx];
-                if old_index == u32::MAX as usize {
-                    nodes_created += new_node.create(self, to, parent);
-                } else {
-                    old[old_index].diff_node(new_node, self, to);
-                    nodes_created += new_node.push_all_real_nodes(self, to);
-                }
-            }
+            let nodes_created = create_or_diff(
+                self,
+                new,
+                old,
+                to.as_deref_mut(),
+                parent,
+                &new_index_to_old_index,
+                (last + 1)..new.len(),
+            );

-            let id = new[last].find_last_element(self);
-            if nodes_created > 0 {
-                to.insert_nodes_after(id, nodes_created)
-            }
-            nodes_created = 0;
+            // Insert all the nodes that we just created after the last node in the LIS
+            self.insert_after(to.as_deref_mut(), nodes_created, &new[last]);
        }

-        // for each spacing, generate a mount instruction
-        let mut lis_iter = lis_sequence.iter().rev();
+        // For each node inside of the LIS, but not included in the LIS, generate a mount instruction
+        // We loop over the LIS in reverse order and insert any nodes we find in the gaps between indexes
+        let mut lis_iter = lis_sequence.iter();
        let mut last = *lis_iter.next().unwrap();
        for next in lis_iter {
            if last - next > 1 {
-                for (idx, new_node) in new[(next + 1)..last].iter().enumerate() {
-                    let new_idx = idx + next + 1;
-                    let old_index = new_index_to_old_index[new_idx];
-                    if old_index == u32::MAX as usize {
-                        nodes_created += new_node.create(self, to, parent);
-                    } else {
-                        old[old_index].diff_node(new_node, self, to);
-                        nodes_created += new_node.push_all_real_nodes(self, to);
-                    }
-                }
+                let nodes_created = create_or_diff(
+                    self,
+                    new,
+                    old,
+                    to.as_deref_mut(),
+                    parent,
+                    &new_index_to_old_index,
+                    (next + 1)..last,
+                );

-                let id = new[last].find_first_element(self);
-                if nodes_created > 0 {
-                    to.insert_nodes_before(id, nodes_created);
-                }
-
-                nodes_created = 0;
+                self.insert_before(to.as_deref_mut(), nodes_created, &new[last]);
            }
            last = *next;
        }

-        // add mount instruction for the last items not covered by the lis
-        let first_lis = *lis_sequence.first().unwrap();
+        // add mount instruction for the items after the LIS
+        let first_lis = *lis_sequence.last().unwrap();
        if first_lis > 0 {
-            for (idx, new_node) in new[..first_lis].iter().enumerate() {
-                let old_index = new_index_to_old_index[idx];
-                if old_index == u32::MAX as usize {
-                    nodes_created += new_node.create(self, to, parent);
-                } else {
-                    old[old_index].diff_node(new_node, self, to);
-                    nodes_created += new_node.push_all_real_nodes(self, to);
-                }
-            }
+            let nodes_created = create_or_diff(
+                self,
+                new,
+                old,
+                to.as_deref_mut(),
+                parent,
+                &new_index_to_old_index,
+                0..first_lis,
+            );

-            let id = new[first_lis].find_first_element(self);
-            if nodes_created > 0 {
-                to.insert_nodes_before(id, nodes_created);
-            }
+            self.insert_before(to, nodes_created, &new[first_lis]);
        }
    }

    fn create_and_insert_before(
        &mut self,
-        to: &mut impl WriteMutations,
+        mut to: Option<&mut impl WriteMutations>,
        new: &[VNode],
        before: &VNode,
        parent: Option<ElementRef>,
    ) {
-        let m = self.create_children(to, new, parent);
-        let id = before.find_first_element(self);
-        to.insert_nodes_before(id, m);
+        let m = self.create_children(to.as_deref_mut(), new, parent);
+        self.insert_before(to, m, before);
+    }
+
+    fn insert_before(&mut self, to: Option<&mut impl WriteMutations>, new: usize, before: &VNode) {
+        if let Some(to) = to {
+            if new > 0 {
+                let id = before.find_first_element(self);
+                to.insert_nodes_before(id, new);
+            }
+        }
    }

    fn create_and_insert_after(
        &mut self,
-        to: &mut impl WriteMutations,
+        mut to: Option<&mut impl WriteMutations>,
        new: &[VNode],
        after: &VNode,
        parent: Option<ElementRef>,
    ) {
-        let m = self.create_children(to, new, parent);
-        let id = after.find_last_element(self);
-        to.insert_nodes_after(id, m);
+        let m = self.create_children(to.as_deref_mut(), new, parent);
+        self.insert_after(to, m, after);
+    }
+
+    fn insert_after(&mut self, to: Option<&mut impl WriteMutations>, new: usize, after: &VNode) {
+        if let Some(to) = to {
+            if new > 0 {
+                let id = after.find_last_element(self);
+                to.insert_nodes_after(id, new);
+            }
+        }
    }
 }

 impl VNode {
-    /// Push all the real nodes on the stack
-    pub(crate) fn push_all_real_nodes(
+    /// Push all the root nodes on the stack
+    pub(crate) fn push_all_root_nodes(
        &self,
        dom: &VirtualDom,
        to: &mut impl WriteMutations,
@@ -440,30 +475,27 @@ impl VNode {
            .roots
            .iter()
            .enumerate()
-            .map(|(root_idx, _)| match &self.template.get().roots[root_idx] {
-                TemplateNode::Dynamic { id: idx } => match &self.dynamic_nodes[*idx] {
-                    DynamicNode::Placeholder(_) | DynamicNode::Text(_) => {
-                        to.push_root(mount.root_ids[root_idx]);
-                        1
-                    }
-                    DynamicNode::Fragment(nodes) => {
+            .map(
+                |(root_idx, _)| match self.get_dynamic_root_node_and_id(root_idx) {
+                    Some((_, DynamicNode::Fragment(nodes))) => {
                        let mut accumulated = 0;
                        for node in nodes {
-                            accumulated += node.push_all_real_nodes(dom, to);
+                            accumulated += node.push_all_root_nodes(dom, to);
                        }
                        accumulated
                    }
-                    DynamicNode::Component(_) => {
-                        let scope = ScopeId(mount.mounted_dynamic_nodes[*idx]);
+                    Some((idx, DynamicNode::Component(_))) => {
+                        let scope = ScopeId(mount.mounted_dynamic_nodes[idx]);
                        let node = dom.get_scope(scope).unwrap().root_node();
-                        node.push_all_real_nodes(dom, to)
+                        node.push_all_root_nodes(dom, to)
+                    }
+                    // This is a static root node or a single dynamic node, just push it
+                    None | Some((_, DynamicNode::Placeholder(_) | DynamicNode::Text(_))) => {
+                        to.push_root(mount.root_ids[root_idx]);
+                        1
                    }
                },
-                _ => {
-                    to.push_root(mount.root_ids[root_idx]);
-                    1
-                }
-            })
+            )
            .sum()
    }
 }