Move asset hashing into the CLI (#3988)
* Fix deduping assets * hash assets in the linker * fix extracting assets * Fix the hashed asset name * deserialize the asset back from the link section * prevent the read from being inlined * remove some extra logs * Fix liveview routing * Only look for direct dependencies on dioxus features * fix wasm split test * On wasm, create a new object file with the asset information * Fix multiple assets * add a regression test for 3467 * fix image link * fix linker out file * fix linux linker args * add windows linker command * fix linux link args * fix windows target triple * Expose an option to build assets in the linker intercept * Fix the standalone optimizer and document it in the dx help command * include the cli opt version in the hash * fix clippy * fix ssg port * add a safety note to the read_volatile call * move found assets log * fix some merge conflicts * fix build * make most env var for the linker optional * guess the target triple when dx acts as the linker intercept * automatically find the linker for normal builds * use the right linker in wasm mode * fix mac linker * look inside archive files (rlibs) * remove out.txt * fix clippy * fix typo * fix clippy * fix build * fix dx link name in docs * ignore libcompiler_builtins warnings * fix playwright * pass typocheck * fix printing sysroot * Modify final binary * fix wasm assets * remove old link section logic * fix tests * forward manganis export args * fail to evaluate relocatable address * retain all exports in the same pattern as other platforms * Fix adding assets during a wasm hot patch * less nesting and remove logs * fix clippy * log pdb files * read from pdb * print data section contents * print with object * normal windows builds working * fix hot patch windows * clean up logs * fix clippy * clean up assets * use the most recent pdb * fix doc test * fix typo * tiny nits * make bundled a reference instead of a pointer * add helpful comment to impl * match on file format * Fix asset hashing comment --------- Co-authored-by: Jonathan Kelley <jkelleyrtp@gmail.com>
This commit is contained in:
1 parent
f865e3f559
commit
58669f892d
45 files changed
+1174
-762
No files matched your search
@@ -50,7 +50,6 @@ hyper-rustls = { workspace = true }
|
||||
rustls = { workspace = true }
|
||||
rayon = { workspace = true }
|
||||
futures-channel = { workspace = true }
|
||||
target-lexicon = { version = "0.13.2", features = ["serde", "serde_support"] }
|
||||
krates = { workspace = true }
|
||||
regex = "1.11.1"
|
||||
console = "0.15.11"
|
||||
@@ -61,7 +60,12 @@ axum-server = { workspace = true, features = ["tls-rustls"] }
|
||||
axum-extra = { workspace = true, features = ["typed-header"] }
|
||||
tower-http = { workspace = true, features = ["full"] }
|
||||
proc-macro2 = { workspace = true, features = ["span-locations"] }
|
||||
syn = { workspace = true, features = ["full", "extra-traits", "visit", "visit-mut"] }
|
||||
syn = { workspace = true, features = [
|
||||
"full",
|
||||
"extra-traits",
|
||||
"visit",
|
||||
"visit-mut",
|
||||
] }
|
||||
|
||||
headers = "0.4.0"
|
||||
walkdir = "2"
|
||||
@@ -108,6 +112,8 @@ log = { version = "0.4", features = ["max_level_off", "release_max_level_off"] }
|
||||
tempfile = "3.13"
|
||||
manganis = { workspace = true }
|
||||
manganis-core = { workspace = true }
|
||||
target-lexicon = { version = "0.13.2", features = ["serde", "serde_support"] }
|
||||
wasm-encoder = "0.229.0"
|
||||
|
||||
# Extracting data from an executable
|
||||
object = { workspace = true, features = ["all"] }
|
||||
@@ -130,7 +136,6 @@ local-ip-address = "0.6.3"
|
||||
dircpy = "0.3.19"
|
||||
plist = "1.7.0"
|
||||
memoize = "0.5.1"
|
||||
wasm-encoder = "0.228.0"
|
||||
backtrace = "0.3.74"
|
||||
ar = "0.9.0"
|
||||
wasm-bindgen-externref-xform = "0.2.100"
|
||||
|
||||
@@ -0,0 +1,354 @@
|
||||
//! The dioxus asset system.
|
||||
//!
|
||||
//! This module provides functionality for extracting assets from a binary file and then writing back
|
||||
//! their asset hashes directly into the binary file. Previously, we performed asset hashing in the
|
||||
//! `asset!()` macro. The new system, implemented here, instead performs the hashing at build time,
|
||||
//! which provides more flexibility in the asset processing pipeline.
|
||||
//!
|
||||
//! We chose to implement this approach since assets might reference each other which means we minimally
|
||||
//! need to parse the asset to create a unique hash for each asset before they are used in the application.
|
||||
//! The hashes are used both for cache busting the asset in the browser and to cache the asset optimization
|
||||
//! process in the build system.
|
||||
//!
|
||||
//! We use the same lessons learned from the hot-patching engine which parses the binary file and its
|
||||
//! symbol table to find symbols that match the `__MANGANIS__` prefix. These symbols are ideally data
|
||||
//! symbols and contain the BundledAsset data type which implements ConstSerialize and ConstDeserialize.
|
||||
//!
|
||||
//! When the binary is built, the `dioxus asset!()` macro will emit its metadata into the __MANGANIS__
|
||||
//! symbols, which we process here. After reading the metadata directly from the executable, we then
|
||||
//! hash it and write the hash directly into the binary file.
|
||||
//!
|
||||
//! During development, we can skip this step for most platforms since local paths are sufficient
|
||||
//! for asset loading. However, for WASM and for production builds, we need to ensure that assets
|
||||
//! can be found relative to the current exe. Unfortunately, on android, the `current_exe` path is wrong,
|
||||
//! so the assets are resolved against the "asset root" - which is covered by the asset loader crate.
|
||||
//!
|
||||
//! Finding the __MANGANIS__ symbols is not quite straightforward when hotpatching, especially on WASM
|
||||
//! since we build and link the module as relocatable, which is not a stable WASM proposal. In this
|
||||
//! implementation, we handle both the non-PIE *and* PIC cases which are rather bespoke to our whole
|
||||
//! build system.
|
||||
|
||||
use std::{
|
||||
io::{Cursor, Read, Seek, Write},
|
||||
path::{Path, PathBuf},
|
||||
};
|
||||
|
||||
use crate::Result;
|
||||
use anyhow::Context;
|
||||
use const_serialize::{ConstVec, SerializeConst};
|
||||
use dioxus_cli_opt::AssetManifest;
|
||||
use manganis::BundledAsset;
|
||||
use object::{File, Object, ObjectSection, ObjectSymbol, ReadCache, ReadRef, Section, Symbol};
|
||||
use pdb::FallibleIterator;
|
||||
use rayon::iter::{IntoParallelRefMutIterator, ParallelIterator};
|
||||
use wasmparser::BinaryReader;
|
||||
|
||||
/// Extract all manganis symbols and their sections from the given object file.
|
||||
fn manganis_symbols<'a, 'b, R: ReadRef<'a>>(
|
||||
file: &'b File<'a, R>,
|
||||
) -> impl Iterator<Item = (Symbol<'a, 'b, R>, Section<'a, 'b, R>)> + 'b {
|
||||
file.symbols()
|
||||
.filter(|symbol| {
|
||||
if let Ok(name) = symbol.name() {
|
||||
name.contains("__MANGANIS__")
|
||||
} else {
|
||||
false
|
||||
}
|
||||
})
|
||||
.filter_map(move |symbol| {
|
||||
let section_index = symbol.section_index()?;
|
||||
let section = file.section_by_index(section_index).ok()?;
|
||||
Some((symbol, section))
|
||||
})
|
||||
}
|
||||
|
||||
/// Find the offsets of any manganis symbols in the given file.
|
||||
fn find_symbol_offsets<'a, R: ReadRef<'a>>(
|
||||
path: &Path,
|
||||
file_contents: &[u8],
|
||||
file: &File<'a, R>,
|
||||
) -> Result<Vec<u64>> {
|
||||
let pdb_file = find_pdb_file(path);
|
||||
|
||||
match file.format() {
|
||||
// We need to handle dynamic offsets in wasm files differently
|
||||
object::BinaryFormat::Wasm => find_wasm_symbol_offsets(file_contents, file),
|
||||
// Windows puts the symbol information in a PDB file alongside the executable.
|
||||
// If this is a windows PE file and we found a PDB file, we will use that to find the symbol offsets.
|
||||
object::BinaryFormat::Pe if pdb_file.is_some() => {
|
||||
find_pdb_symbol_offsets(&pdb_file.unwrap())
|
||||
}
|
||||
// Otherwise, look for manganis symbols in the object file.
|
||||
_ => find_native_symbol_offsets(file),
|
||||
}
|
||||
}
|
||||
|
||||
/// Find the pdb file matching the executable file.
|
||||
fn find_pdb_file(path: &Path) -> Option<PathBuf> {
|
||||
let mut pdb_file = path.with_extension("pdb");
|
||||
// Also try to find it in the same directory as the executable with _'s instead of -'s
|
||||
if let Some(file_name) = pdb_file.file_name() {
|
||||
let new_file_name = file_name.to_string_lossy().replace('-', "_");
|
||||
let altrnate_pdb_file = pdb_file.with_file_name(new_file_name);
|
||||
// Keep the most recent pdb file
|
||||
match (pdb_file.metadata(), altrnate_pdb_file.metadata()) {
|
||||
(Ok(pdb_metadata), Ok(alternate_metadata)) => {
|
||||
if let (Ok(pdb_modified), Ok(alternate_modified)) =
|
||||
(pdb_metadata.modified(), alternate_metadata.modified())
|
||||
{
|
||||
if pdb_modified < alternate_modified {
|
||||
pdb_file = altrnate_pdb_file;
|
||||
}
|
||||
}
|
||||
}
|
||||
(Err(_), Ok(_)) => {
|
||||
pdb_file = altrnate_pdb_file;
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
if pdb_file.exists() {
|
||||
Some(pdb_file)
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
/// Find the offsets of any manganis symbols in a pdb file.
|
||||
fn find_pdb_symbol_offsets(pdb_file: &Path) -> Result<Vec<u64>> {
|
||||
let pdb_file_handle = std::fs::File::open(pdb_file)?;
|
||||
let mut pdb_file = pdb::PDB::open(pdb_file_handle).context("Failed to open PDB file")?;
|
||||
let Ok(Some(sections)) = pdb_file.sections() else {
|
||||
tracing::error!("Failed to read sections from PDB file");
|
||||
return Ok(Vec::new());
|
||||
};
|
||||
let global_symbols = pdb_file
|
||||
.global_symbols()
|
||||
.context("Failed to read global symbols from PDB file")?;
|
||||
let address_map = pdb_file
|
||||
.address_map()
|
||||
.context("Failed to read address map from PDB file")?;
|
||||
let mut symbols = global_symbols.iter();
|
||||
let mut addresses = Vec::new();
|
||||
while let Ok(Some(symbol)) = symbols.next() {
|
||||
let Ok(pdb::SymbolData::Public(data)) = symbol.parse() else {
|
||||
continue;
|
||||
};
|
||||
let Some(rva) = data.offset.to_section_offset(&address_map) else {
|
||||
continue;
|
||||
};
|
||||
|
||||
let name = data.name.to_string();
|
||||
if name.contains("__MANGANIS__") {
|
||||
let section = sections
|
||||
.get(rva.section as usize - 1)
|
||||
.expect("Section index out of bounds");
|
||||
|
||||
addresses.push((section.pointer_to_raw_data + rva.offset) as u64);
|
||||
}
|
||||
}
|
||||
Ok(addresses)
|
||||
}
|
||||
|
||||
/// Find the offsets of any manganis symbols in a native object file.
|
||||
fn find_native_symbol_offsets<'a, R: ReadRef<'a>>(file: &File<'a, R>) -> Result<Vec<u64>> {
|
||||
let mut offsets = Vec::new();
|
||||
for (symbol, section) in manganis_symbols(file) {
|
||||
let virtual_address = symbol.address();
|
||||
|
||||
let Some((section_range_start, _)) = section.file_range() else {
|
||||
tracing::error!(
|
||||
"Found __MANGANIS__ symbol {:?} in section {}, but the section has no file range",
|
||||
symbol.name(),
|
||||
section.index()
|
||||
);
|
||||
continue;
|
||||
};
|
||||
// Translate the section_relative_address to the file offset
|
||||
let section_relative_address: u64 = (virtual_address as i128 - section.address() as i128)
|
||||
.try_into()
|
||||
.expect("Virtual address should be greater than or equal to section address");
|
||||
let file_offset = section_range_start + section_relative_address;
|
||||
offsets.push(file_offset);
|
||||
}
|
||||
|
||||
Ok(offsets)
|
||||
}
|
||||
|
||||
/// Find the offsets of any manganis symbols in the wasm file.
|
||||
fn find_wasm_symbol_offsets<'a, R: ReadRef<'a>>(
|
||||
file_contents: &[u8],
|
||||
file: &File<'a, R>,
|
||||
) -> Result<Vec<u64>> {
|
||||
// Parse the wasm file to find the globals
|
||||
let parser = wasmparser::Parser::new(0);
|
||||
|
||||
// All integer literal global values in the wasm file
|
||||
let mut global_values = Vec::new();
|
||||
for section in parser.parse_all(file_contents) {
|
||||
let Ok(wasmparser::Payload::GlobalSection(global_section)) = section else {
|
||||
continue;
|
||||
};
|
||||
|
||||
global_values = global_section
|
||||
.into_iter()
|
||||
.map(|global| {
|
||||
let global = global.ok()?;
|
||||
match global.init_expr.get_operators_reader().into_iter().next() {
|
||||
Some(Ok(wasmparser::Operator::I32Const { value })) => Some(value as u64),
|
||||
Some(Ok(wasmparser::Operator::I64Const { value })) => Some(value as u64),
|
||||
_ => None,
|
||||
}
|
||||
})
|
||||
.collect::<Vec<_>>();
|
||||
}
|
||||
let mut offsets = Vec::new();
|
||||
|
||||
for (symbol, section) in manganis_symbols(file) {
|
||||
let virtual_address = symbol.address();
|
||||
|
||||
let Some((_, section_range_end)) = section.file_range() else {
|
||||
tracing::error!(
|
||||
"Found __MANGANIS__ symbol {:?} in section {}, but the section has no file range",
|
||||
symbol.name(),
|
||||
section.index()
|
||||
);
|
||||
continue;
|
||||
};
|
||||
let section_size = section.data()?.len() as u64;
|
||||
let section_start = section_range_end - section_size;
|
||||
// Translate the section_relative_address to the file offset
|
||||
// WASM files have a section address of 0 in object, reparse the data section with wasmparser
|
||||
// to get the correct address and section start
|
||||
let reader = wasmparser::DataSectionReader::new(BinaryReader::new(
|
||||
&file_contents[section_start as usize..section_range_end as usize],
|
||||
0,
|
||||
))
|
||||
.context("Failed to create WASM data section reader")?;
|
||||
let main_memory = reader
|
||||
.into_iter()
|
||||
.next()
|
||||
.context("Failed find main memory from WASM data section")?
|
||||
.context("Failed to read main memory from WASM data section")?;
|
||||
let main_memory_offset = match main_memory.kind {
|
||||
wasmparser::DataKind::Active { offset_expr, .. } => {
|
||||
match offset_expr.get_operators_reader().into_iter().next() {
|
||||
Some(Ok(wasmparser::Operator::I32Const { value })) => -value as i128,
|
||||
Some(Ok(wasmparser::Operator::I64Const { value })) => -value as i128,
|
||||
Some(Ok(wasmparser::Operator::GlobalGet { global_index })) => {
|
||||
let Some(value) =
|
||||
global_values.get(global_index as usize).copied().flatten()
|
||||
else {
|
||||
tracing::error!(
|
||||
"Found __MANGANIS__ symbol {:?} in WASM file, but the global index {} is not found",
|
||||
symbol.name(),
|
||||
global_index
|
||||
);
|
||||
continue;
|
||||
};
|
||||
value as i128
|
||||
}
|
||||
offset_expr => {
|
||||
tracing::error!(
|
||||
"Found __MANGANIS__ symbol {:?} in WASM file, but the offset expression is not a constant is is {:?}",
|
||||
symbol.name(),
|
||||
offset_expr
|
||||
);
|
||||
continue;
|
||||
}
|
||||
}
|
||||
}
|
||||
_ => {
|
||||
tracing::error!(
|
||||
"Found __MANGANIS__ symbol {:?} in WASM file, but the data section is not active",
|
||||
symbol.name()
|
||||
);
|
||||
continue;
|
||||
}
|
||||
};
|
||||
// main_memory.data is a slice somewhere in file_contents. Find out the offset in the file
|
||||
let data_start_offset = (main_memory.data.as_ptr() as u64)
|
||||
.checked_sub(file_contents.as_ptr() as u64)
|
||||
.expect("Data section start offset should be within the file contents");
|
||||
let section_relative_address: u64 = ((virtual_address as i128) + main_memory_offset)
|
||||
.try_into()
|
||||
.expect("Virtual address should be greater than or equal to section address");
|
||||
let file_offset = data_start_offset + section_relative_address;
|
||||
offsets.push(file_offset);
|
||||
}
|
||||
|
||||
Ok(offsets)
|
||||
}
|
||||
|
||||
/// Find all assets in the given file, hash them, and write them back to the file.
|
||||
/// Then return an `AssetManifest` containing all the assets found in the file.
|
||||
pub(crate) fn extract_assets_from_file(path: impl AsRef<Path>) -> Result<AssetManifest> {
|
||||
let path = path.as_ref();
|
||||
let mut file = std::fs::File::options().write(true).read(true).open(path)?;
|
||||
let mut file_contents = Vec::new();
|
||||
file.read_to_end(&mut file_contents)?;
|
||||
let mut reader = Cursor::new(&file_contents);
|
||||
let read_cache = ReadCache::new(&mut reader);
|
||||
let object_file = object::File::parse(&read_cache)?;
|
||||
let offsets = find_symbol_offsets(path, &file_contents, &object_file)?;
|
||||
|
||||
let mut assets = Vec::new();
|
||||
|
||||
// Read each asset from the data section using the offsets
|
||||
for offset in offsets.iter().copied() {
|
||||
file.seek(std::io::SeekFrom::Start(offset))?;
|
||||
let mut data_in_range = vec![0; BundledAsset::MEMORY_LAYOUT.size()];
|
||||
file.read_exact(&mut data_in_range)?;
|
||||
|
||||
let buffer = const_serialize::ConstReadBuffer::new(&data_in_range);
|
||||
|
||||
if let Some((_, bundled_asset)) = const_serialize::deserialize_const!(BundledAsset, buffer)
|
||||
{
|
||||
assets.push(bundled_asset);
|
||||
} else {
|
||||
tracing::warn!("Found an asset at offset {offset} that could not be deserialized. This may be caused by a mismatch between your dioxus and dioxus-cli versions.");
|
||||
}
|
||||
}
|
||||
|
||||
// Add the hash to each asset in parallel
|
||||
assets
|
||||
.par_iter_mut()
|
||||
.for_each(dioxus_cli_opt::add_hash_to_asset);
|
||||
|
||||
// Write back the assets to the binary file
|
||||
for (offset, asset) in offsets.into_iter().zip(&assets) {
|
||||
let new_data = ConstVec::new();
|
||||
let new_data = const_serialize::serialize_const(asset, new_data);
|
||||
|
||||
file.seek(std::io::SeekFrom::Start(offset))?;
|
||||
// Write the modified binary data back to the file
|
||||
file.write_all(new_data.as_ref())?;
|
||||
}
|
||||
|
||||
// If the file is a macos binary, we need to re-sign the modified binary
|
||||
if object_file.format() == object::BinaryFormat::MachO {
|
||||
// Spawn the codesign command to re-sign the binary
|
||||
let output = std::process::Command::new("codesign")
|
||||
.arg("--force")
|
||||
.arg("--sign")
|
||||
.arg("-") // Sign with an empty identity
|
||||
.arg(path)
|
||||
.output()?;
|
||||
|
||||
if !output.status.success() {
|
||||
return Err(anyhow::anyhow!(
|
||||
"Failed to re-sign the binary with codesign after finalizing the assets: {}",
|
||||
String::from_utf8_lossy(&output.stderr)
|
||||
)
|
||||
.into());
|
||||
}
|
||||
}
|
||||
|
||||
// Finally, create the asset manifest
|
||||
let mut manifest = AssetManifest::default();
|
||||
for asset in assets {
|
||||
manifest.insert_asset(asset);
|
||||
}
|
||||
|
||||
Ok(manifest)
|
||||
}
|
||||
@@ -567,16 +567,15 @@ impl AppBuilder {
|
||||
let original_artifacts = self.artifacts.as_ref().unwrap();
|
||||
let asset_dir = self.build.asset_dir();
|
||||
|
||||
for (k, bundled) in res.assets.assets.iter() {
|
||||
let k = dunce::canonicalize(k)?;
|
||||
if original_artifacts.assets.assets.contains_key(k.as_path()) {
|
||||
for bundled in res.assets.assets() {
|
||||
if original_artifacts.assets.contains(bundled) {
|
||||
continue;
|
||||
}
|
||||
let from = dunce::canonicalize(PathBuf::from(bundled.absolute_source_path()))?;
|
||||
|
||||
let from = k.clone();
|
||||
let to = asset_dir.join(bundled.bundled_path());
|
||||
|
||||
tracing::debug!("Copying asset from patch: {}", k.display());
|
||||
tracing::debug!("Copying asset from patch: {}", from.display());
|
||||
if let Err(e) = dioxus_cli_opt::process_file_to(bundled.options(), &from, &to) {
|
||||
tracing::error!("Failed to copy asset: {e}");
|
||||
continue;
|
||||
@@ -584,13 +583,8 @@ impl AppBuilder {
|
||||
|
||||
// If the emulator is android, we need to copy the asset to the device with `adb push asset /data/local/tmp/dx/assets/filename.ext`
|
||||
if self.build.platform == Platform::Android {
|
||||
let changed_file = dunce::canonicalize(k).inspect_err(|e| {
|
||||
tracing::debug!("Failed to canonicalize hotreloaded asset: {e}")
|
||||
})?;
|
||||
let bundled_name = PathBuf::from(bundled.bundled_path());
|
||||
_ = self
|
||||
.copy_file_to_android_tmp(&changed_file, &bundled_name)
|
||||
.await;
|
||||
_ = self.copy_file_to_android_tmp(&from, &bundled_name).await;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -649,10 +643,13 @@ impl AppBuilder {
|
||||
/// dir that the system simulator might be providing. We know this is the case for ios simulators
|
||||
/// and haven't yet checked for android.
|
||||
///
|
||||
/// This will return the bundled name of the asset such that we can send it to the clients letting
|
||||
/// This will return the bundled name of the assets such that we can send it to the clients letting
|
||||
/// them know what to reload. It's not super important that this is robust since most clients will
|
||||
/// kick all stylsheets without necessarily checking the name.
|
||||
pub(crate) async fn hotreload_bundled_asset(&self, changed_file: &PathBuf) -> Option<PathBuf> {
|
||||
pub(crate) async fn hotreload_bundled_assets(
|
||||
&self,
|
||||
changed_file: &PathBuf,
|
||||
) -> Option<Vec<PathBuf>> {
|
||||
let artifacts = self.artifacts.as_ref()?;
|
||||
|
||||
// Use the build dir if there's no runtime asset dir as the override. For the case of ios apps,
|
||||
@@ -668,32 +665,36 @@ impl AppBuilder {
|
||||
.ok()?;
|
||||
|
||||
// The asset might've been renamed thanks to the manifest, let's attempt to reload that too
|
||||
let resource = artifacts.assets.assets.get(&changed_file)?;
|
||||
let output_path = asset_dir.join(resource.bundled_path());
|
||||
let resources = artifacts.assets.get_assets_for_source(&changed_file)?;
|
||||
let mut bundled_names = Vec::new();
|
||||
for resource in resources {
|
||||
let output_path = asset_dir.join(resource.bundled_path());
|
||||
|
||||
tracing::debug!("Hotreloading asset {changed_file:?} in target {asset_dir:?}");
|
||||
tracing::debug!("Hotreloading asset {changed_file:?} in target {asset_dir:?}");
|
||||
|
||||
// Remove the old asset if it exists
|
||||
_ = std::fs::remove_file(&output_path);
|
||||
// Remove the old asset if it exists
|
||||
_ = std::fs::remove_file(&output_path);
|
||||
|
||||
// And then process the asset with the options into the **old** asset location. If we recompiled,
|
||||
// the asset would be in a new location because the contents and hash have changed. Since we are
|
||||
// hotreloading, we need to use the old asset location it was originally written to.
|
||||
let options = *resource.options();
|
||||
let res = process_file_to(&options, &changed_file, &output_path);
|
||||
let bundled_name = PathBuf::from(resource.bundled_path());
|
||||
if let Err(e) = res {
|
||||
tracing::debug!("Failed to hotreload asset {e}");
|
||||
// And then process the asset with the options into the **old** asset location. If we recompiled,
|
||||
// the asset would be in a new location because the contents and hash have changed. Since we are
|
||||
// hotreloading, we need to use the old asset location it was originally written to.
|
||||
let options = *resource.options();
|
||||
let res = process_file_to(&options, &changed_file, &output_path);
|
||||
let bundled_name = PathBuf::from(resource.bundled_path());
|
||||
if let Err(e) = res {
|
||||
tracing::debug!("Failed to hotreload asset {e}");
|
||||
}
|
||||
|
||||
// If the emulator is android, we need to copy the asset to the device with `adb push asset /data/local/tmp/dx/assets/filename.ext`
|
||||
if self.build.platform == Platform::Android {
|
||||
_ = self
|
||||
.copy_file_to_android_tmp(&changed_file, &bundled_name)
|
||||
.await;
|
||||
}
|
||||
bundled_names.push(bundled_name);
|
||||
}
|
||||
|
||||
// If the emulator is android, we need to copy the asset to the device with `adb push asset /data/local/tmp/dx/assets/filename.ext`
|
||||
if self.build.platform == Platform::Android {
|
||||
_ = self
|
||||
.copy_file_to_android_tmp(&changed_file, &bundled_name)
|
||||
.await;
|
||||
}
|
||||
|
||||
Some(bundled_name)
|
||||
Some(bundled_names)
|
||||
}
|
||||
|
||||
/// Copy this file to the tmp folder on the android device, returning the path to the copied file
|
||||
|
||||
@@ -8,12 +8,14 @@
|
||||
//! hot-patching Rust code through binary analysis and a custom linker. The [`builder`] module contains
|
||||
//! the management of the ongoing build and methods to open the build as a running app.
|
||||
|
||||
mod assets;
|
||||
mod builder;
|
||||
mod context;
|
||||
mod patch;
|
||||
mod request;
|
||||
mod tools;
|
||||
|
||||
pub(crate) use assets::*;
|
||||
pub(crate) use builder::*;
|
||||
pub(crate) use context::*;
|
||||
pub(crate) use patch::*;
|
||||
|
||||
@@ -961,10 +961,8 @@ impl BuildRequest {
|
||||
})
|
||||
}
|
||||
|
||||
/// Traverse the target directory and collect all assets from the incremental cache
|
||||
///
|
||||
/// This uses "known paths" that have stayed relatively stable during cargo's lifetime.
|
||||
/// One day this system might break and we might need to go back to using the linker approach.
|
||||
/// Collect the assets from the final executable and modify the binary in place to point to the right
|
||||
/// hashed asset location.
|
||||
fn collect_assets(&self, exe: &Path, ctx: &BuildContext) -> Result<AssetManifest> {
|
||||
// walk every file in the incremental cache dir, reading and inserting items into the manifest.
|
||||
let mut manifest = AssetManifest::default();
|
||||
@@ -972,7 +970,7 @@ impl BuildRequest {
|
||||
// And then add from the exe directly, just in case it's LTO compiled and has no incremental cache
|
||||
if !self.skip_assets {
|
||||
ctx.status_extracting_assets();
|
||||
_ = manifest.add_from_object_path(exe);
|
||||
manifest = super::assets::extract_assets_from_file(exe)?;
|
||||
}
|
||||
|
||||
Ok(manifest)
|
||||
@@ -1109,8 +1107,7 @@ impl BuildRequest {
|
||||
|
||||
// Create a set of all the paths that new files will be bundled to
|
||||
let mut keep_bundled_output_paths: HashSet<_> = assets
|
||||
.assets
|
||||
.values()
|
||||
.assets()
|
||||
.map(|a| asset_dir.join(a.bundled_path()))
|
||||
.collect();
|
||||
|
||||
@@ -1149,8 +1146,8 @@ impl BuildRequest {
|
||||
let mut assets_to_transfer = vec![];
|
||||
|
||||
// Queue the bundled assets
|
||||
for (asset, bundled) in &assets.assets {
|
||||
let from = asset.clone();
|
||||
for bundled in assets.assets() {
|
||||
let from = PathBuf::from(bundled.absolute_source_path());
|
||||
let to = asset_dir.join(bundled.bundled_path());
|
||||
|
||||
// prefer to log using a shorter path relative to the workspace dir by trimming the workspace dir
|
||||
@@ -1381,9 +1378,7 @@ impl BuildRequest {
|
||||
}
|
||||
|
||||
// Now extract the assets from the fat binary
|
||||
artifacts
|
||||
.assets
|
||||
.add_from_object_path(&self.patch_exe(artifacts.time_start))?;
|
||||
self.collect_assets(&self.patch_exe(artifacts.time_start), ctx)?;
|
||||
|
||||
// Clean up the temps manually
|
||||
// todo: we might want to keep them around for debugging purposes
|
||||
@@ -1436,6 +1431,14 @@ impl BuildRequest {
|
||||
"--pie".to_string(),
|
||||
"--experimental-pic".to_string(),
|
||||
]);
|
||||
|
||||
// retain exports so post-processing has hooks to work with
|
||||
for (idx, arg) in original_args.iter().enumerate() {
|
||||
if *arg == "--export" {
|
||||
out_args.push(arg.to_string());
|
||||
out_args.push(original_args[idx + 1].to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// This uses "cc" and these args need to be ld compatible
|
||||
@@ -3924,7 +3927,7 @@ impl BuildRequest {
|
||||
}
|
||||
|
||||
// Inject any resources from manganis into the head
|
||||
for asset in assets.assets.values() {
|
||||
for asset in assets.assets() {
|
||||
let asset_path = asset.bundled_path();
|
||||
match asset.options() {
|
||||
AssetOptions::Css(css_options) => {
|
||||
@@ -3954,7 +3957,11 @@ impl BuildRequest {
|
||||
|
||||
// Manually inject the wasm file for preloading. WASM currently doesn't support preloading in the manganis asset system
|
||||
let wasm_source_path = self.wasm_bindgen_wasm_output_file();
|
||||
if let Some(wasm_path) = assets.assets.get(&wasm_source_path) {
|
||||
if let Some(wasm_assets) = assets.get_assets_for_source(&wasm_source_path) {
|
||||
let wasm_path = wasm_assets
|
||||
.iter()
|
||||
.next()
|
||||
.expect("There should be exactly one optimized wasm asset");
|
||||
let wasm_path = wasm_path.bundled_path();
|
||||
head_resources.push_str(&format!(
|
||||
"<link rel=\"preload\" as=\"fetch\" type=\"application/wasm\" href=\"/{{base_path}}/assets/{wasm_path}\" crossorigin>"
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
use std::{fs::create_dir_all, path::PathBuf};
|
||||
|
||||
use crate::{Result, StructuredOutput};
|
||||
use crate::{extract_assets_from_file, Result, StructuredOutput};
|
||||
use clap::Parser;
|
||||
use dioxus_cli_opt::{process_file_to, AssetManifest};
|
||||
use dioxus_cli_opt::process_file_to;
|
||||
use tracing::debug;
|
||||
|
||||
#[derive(Clone, Debug, Parser)]
|
||||
@@ -10,21 +10,17 @@ pub struct BuildAssets {
|
||||
/// The source executable to build assets for.
|
||||
pub(crate) executable: PathBuf,
|
||||
|
||||
/// The source directory for the assets.
|
||||
pub(crate) source: PathBuf,
|
||||
|
||||
/// The destination directory for the assets.
|
||||
pub(crate) destination: PathBuf,
|
||||
}
|
||||
|
||||
impl BuildAssets {
|
||||
pub async fn run(self) -> Result<StructuredOutput> {
|
||||
let mut manifest = AssetManifest::default();
|
||||
manifest.add_from_object_path(&self.executable)?;
|
||||
let manifest = extract_assets_from_file(&self.executable)?;
|
||||
|
||||
create_dir_all(&self.destination)?;
|
||||
for (path, asset) in manifest.assets.iter() {
|
||||
let source_path = self.source.join(path);
|
||||
for asset in manifest.assets() {
|
||||
let source_path = PathBuf::from(asset.absolute_source_path());
|
||||
let destination_path = self.destination.join(asset.bundled_path());
|
||||
debug!(
|
||||
"Processing asset {} --> {} {:#?}",
|
||||
|
||||
@@ -36,6 +36,9 @@ pub(crate) enum Error {
|
||||
#[error("Failed to perform hotpatch: {0}")]
|
||||
PatchingFailed(#[from] crate::build::PatchError),
|
||||
|
||||
#[error("Failed to read object file: {0}")]
|
||||
ObjectReadFailed(#[from] object::Error),
|
||||
|
||||
#[error("{0}")]
|
||||
CapturedPanic(String),
|
||||
|
||||
|
||||
@@ -152,7 +152,7 @@ impl Output {
|
||||
use std::io::IsTerminal;
|
||||
|
||||
if !stdout().is_terminal() {
|
||||
return io::Result::Err(io::Error::new(io::ErrorKind::Other, "Not a terminal"));
|
||||
return io::Result::Err(io::Error::other("Not a terminal"));
|
||||
}
|
||||
|
||||
enable_raw_mode()?;
|
||||
|
||||
@@ -309,8 +309,10 @@ impl AppServer {
|
||||
|
||||
// If it's an asset, we want to hotreload it
|
||||
// todo(jon): don't hardcode this here
|
||||
if let Some(bundled_name) = self.client.hotreload_bundled_asset(path).await {
|
||||
assets.push(PathBuf::from("/assets/").join(bundled_name));
|
||||
if let Some(bundled_names) = self.client.hotreload_bundled_assets(path).await {
|
||||
for bundled_name in bundled_names {
|
||||
assets.push(PathBuf::from("/assets/").join(bundled_name));
|
||||
}
|
||||
}
|
||||
|
||||
// If it's a rust file, we want to hotreload it using the filemap
|
||||
|
||||
Reference in new issue
Block a user