Enable runtime rust hot patching. * roll back wry to 0.45 * cleanup rustc * wip * wip: fix client/server * get the subsecond cli thing working again * back to list of builds but with accessors * migrate some methods to workspace, clean up impl a bit * pass mode through build context * use a build context, similar to cargo/rustc * merge BuildUpdate and HandleUpdate into BuilderUpdate * Move more resolution of args into runner * migrate some methods to runner * hoist out fullstack * fix build request resolver * yay request is cleaned up * fixup resolution a bit * spawn off the build! * re-wire android autodetection * re-wire android autodetection * wire back up a few more things * re-wire tooling verification * I think it's mostly in the right condition for regular app * add depinfo parser * yay okay works with regular apps again * full rebuilds are back * rewire file wtcher * wire patch! * yayyyyyy patching works and is really fast * yay android works * clean up stdout piping * yayyy memap on android so we don't need root * create a static wasm jump table * wip: global offset table for wasm * wip... thinking about making the patches relocatable * wip: customized relocation * YESSSSSS RELOCATABLE WASM * clean up impl a bit * lil bit more cleanup * lil bit more * lil bit more cleanup * lil bit more cleanup, support 32 bit platforms * sick, wasm is completely pic * hmmmm not quite working yet * woooo, patches loading from rust+wasm * integrated into the cli 😎 * condense a bit * cleaned up request a bit * bust fingerprints * Make the file diffing logic a bit more sensible * still working through a few issues, server launches but is acting weird * fix merge conflict * more merge issues * remove stuff we don't want anymore * revert name change * wip: server not working anymore :( * split apart "dioxus fullstack" into "dioxus server" and "dioxus web" * fixup a few more compile errors * grrr opening server * simultaneous frontend and backend patching * use rustc wrapper * wip: passing assets to jumptable * migrate project-like examples * patchy patchy server fns! * rollback some random changes * unwind the js files * rollback hash * no need for patch * more cleanups * lil bit more cleanup, remove some old cruft * allow patching when wasm webpage is closed * tiny bit more robust * lil bit of clean up * clean up ws code * bit more clean ups * condense * undo file moves * move back other project * move other project * migrate out harness and janky CLI * fix compile * anonymize some files, heavily document others * lots more documentation, for posterity! * ton more docs * clean up the cli a bit more * more ws cleanup * more cleanup * migrate build id to cli-config * add command file handling * random small cleanups * wip.... * wip.... * fix: use workspace for krate discovery * fix panic logging on serve by placing it *after* the logs * reorder logging to be sensible on success * bring back swc * ws cruft * small patches to relocs in loader * bump krates * hoist main funcs into the ifunc table, fixing wasm * fix tui output for long lines * add more linker logging * wow, incredible, we build fat binaries, holy heckkkkk * fix args for frontend/backend * small cleanups * properly send along build IDs to clients * fix workspace compiles * clean up jump table work a bit * clean up logging a bit * open existing * fixup some commands * open existing browser integration * wire up the new dx serve client/server syntax * fixup `dx run` command * bring back some old functionality * fix serverfn feature sets * remove toast animation and simplify its code a bit * less intrusive toast * dont change build status when patching on web * add proper cache busting to vcomponent * clean up the patch file a bit * more lints/checks cleaned up * go back to TargetArgs but a BuildTargets resolved struct * clean up more nits * use an atomicptr for jumptable * fix interaction with initial web load + suspense * don't run asset system on wasm/bindgen js in dev * reduce blast radius of PR * use profile to determine if we're in release * cleanup prod html template * cleanup profiles * fix feature resolution of fullstack * if fullstack is explicitly false, don't use it * light cleanups * drop carg config2 * pass along linker args properly * make workspace examples compile with dx again * fewer unwraps and better error loggign * bit more error handlign * small cleanup * drive-by cleanups * use queries instead of initialize for aslr stuff * fix hotpatch with higher opt level * fix aslr bug from multiple clients * fix merge conflict * fix typos * clippy * fix miscompile * fix doctest * properly rollback wry to 0.45 * fix markdown path issues, other test issues * fix test in router * fix release mode test * fix some more tests, clean up a few more items * use fnptr instead of typeid * clean up memozation via subsecond in core * use "main" as sentinel * fix imports and re-exports * get off __rust_alloc for aslr offset detection * wip * wip... fixing unoptimized web * hmmmm mmmm * close,ish, still missing the wasmbindgen dynaimc imports * aha! full wasm works now - disable externref when compilng patch modules - manually link wasm instrinsics - displace the __wbindgen_placeholder__ table with the synthesized wbg imports what an awful pain. There might be a few items missing, so will test against the docsite shortly * remove failing test * hmmmm things aren't perfect * from what I can tell, relocatable wasm just doesn't work with wasm-bindgen sad https://github.com/rustwasm/wasm-bindgen/issues/1420 * AI IS LITERALLY THE WORST * preserve wasm snippets across compiles * IT HOTPATCHES THE DOCSITE, WE ARE LIVING IN THE FUTURE * remove the macos cfg-out * fix: need to allocate 1 more page * fix url issue * properly span props * properly fill in extra symbols * light cleanups and good docs * delete cruft, better docs * More cleanups and simplifications * clippy, typos * remove harnesses * wasm-bindgen error logging * clean up deps * remove cruft * proper priv/pub * use ifuncs for all env imports * cruft * use par_iter, better output * implement a cache for the base module * delete custom sections * cache the main native module too * add par iter in a few more places * clippy, map ident * better logging and status bars * wip: some small windows support * wip more windowss * fix weird regression * implement windows jump table * small fixes for windows * better windows support * windows support pt2 * rust lld is not msvc * whole archive the rlibs * ignore windows and sys crates * logging * ohhhhhh, pop out dlls * dedupe * thANK GOD FAT LINKING WORKS ON WINDOWS * hoist caching * implement patch based on cache * fix location for pdb * pass data symbols, correct object * add all symbol for windows * add log * treat none rva as undefined * whoops * handle windows calling convention * dont use relocated arm64 stub windows * not all are text? * WINDOWS FUNCTION PATCHING WORKS but issue with statics? * use our own entry * handle windows imp symbols * add log * jump to the real symbol not the layout * whoops * hotpatching windows works completely holy heck yay * drop linker args crate * disable ssg playwright for now * light cleanups * some cleanups / preps: - bump to serverfn 0.8.2 - fix playwright test - fix clippy * use fix from #4042 * bring in other fix for liveview * hoist some configuration * use __aslr_reference instead of aslr_reference * add better errors to patcherror * fix android * panic if patching fails * clean up tomls * wip: fixing wasm compile for server * fullstack wasm again * fix cfg * fixup feature detection * fix issue with features * fix playwright! * add barebones template test harness * fix compatibility between the two hotreload engines * fix preldue of server * fix check for wasm-opt * small fixes
253 lines
11 KiB
Rust
253 lines
11 KiB
Rust
use crate::Result;
|
|
use anyhow::Context;
|
|
use serde::{Deserialize, Serialize};
|
|
use std::path::PathBuf;
|
|
use target_lexicon::Triple;
|
|
|
|
/// `dx` can act as a linker in a few scenarios. Note that we don't *actually* implement the linker logic,
|
|
/// instead just proxying to a specified linker (or not linking at all!).
|
|
///
|
|
/// This comes in two flavors:
|
|
/// --------------------------
|
|
/// - `BaseLink`: We are linking dependencies and want to dynamically select the linker from the environment.
|
|
/// This is mostly implemented for Android where the linker is selected in part by the
|
|
/// device connected over ADB which can not be determined by .cargo/Config.toml.
|
|
/// We implemented this because previous setups like cargo mobile required a hard-coded
|
|
/// linker path in your project which does not work in team-based setups.
|
|
///
|
|
/// - `NoLink`: We are not linking at all, and instead deferring our linking to the driving process,
|
|
/// usually being `dx` itself. In this case, we are just writing the linker args to a file
|
|
/// and then outputting a dummy object file to satisfy the linker. This is generally used
|
|
/// by the binary patching engine since we need to actually do "real linker logic" like
|
|
/// traversing object files and satisfying missing symbols. That process is *much* easier
|
|
/// to do in the driving host process when we have all the information available. Unfortunately,
|
|
/// rustc doesn't provide a "real" way of granularly stepping through the compile process
|
|
/// so this is basically a hack.
|
|
///
|
|
/// We use "BaseLink" when a linker is specified, and "NoLink" when it is not. Both generate a resulting
|
|
/// object file.
|
|
|
|
#[derive(Debug)]
|
|
pub struct LinkAction {
|
|
pub linker: Option<PathBuf>,
|
|
pub triple: Triple,
|
|
pub link_args_file: PathBuf,
|
|
pub link_err_file: PathBuf,
|
|
}
|
|
|
|
/// The linker flavor to use. This influences the argument style that gets passed to the linker.
|
|
/// We're imitating the rustc linker flavors here.
|
|
///
|
|
/// https://doc.rust-lang.org/beta/nightly-rustc/rustc_target/spec/enum.LinkerFlavor.html
|
|
#[derive(Debug, Serialize, Deserialize, PartialEq, Clone)]
|
|
pub enum LinkerFlavor {
|
|
Gnu,
|
|
Darwin,
|
|
WasmLld,
|
|
Unix,
|
|
Msvc,
|
|
}
|
|
|
|
impl LinkAction {
|
|
const DX_LINK_ARG: &str = "DX_LINK";
|
|
const DX_ARGS_FILE: &str = "DX_LINK_ARGS_FILE";
|
|
const DX_ERR_FILE: &str = "DX_LINK_ERR_FILE";
|
|
const DX_LINK_TRIPLE: &str = "DX_LINK_TRIPLE";
|
|
const DX_LINK_CUSTOM_LINKER: &str = "DX_LINK_CUSTOM_LINKER";
|
|
|
|
/// Should we write the input arguments to a file (aka act as a linker subprocess)?
|
|
///
|
|
/// Just check if the magic env var is set
|
|
pub(crate) fn from_env() -> Option<Self> {
|
|
if std::env::var(Self::DX_LINK_ARG).is_err() {
|
|
return None;
|
|
}
|
|
|
|
Some(Self {
|
|
linker: std::env::var(Self::DX_LINK_CUSTOM_LINKER)
|
|
.ok()
|
|
.map(PathBuf::from),
|
|
link_args_file: std::env::var(Self::DX_ARGS_FILE)
|
|
.expect("Linker args file not set")
|
|
.into(),
|
|
link_err_file: std::env::var(Self::DX_ERR_FILE)
|
|
.expect("Linker error file not set")
|
|
.into(),
|
|
triple: std::env::var(Self::DX_LINK_TRIPLE)
|
|
.expect("Linker triple not set")
|
|
.parse()
|
|
.expect("Failed to parse linker triple"),
|
|
})
|
|
}
|
|
|
|
pub(crate) fn write_env_vars(&self, env_vars: &mut Vec<(&str, String)>) -> Result<()> {
|
|
env_vars.push((Self::DX_LINK_ARG, "1".to_string()));
|
|
env_vars.push((
|
|
Self::DX_ARGS_FILE,
|
|
dunce::canonicalize(&self.link_args_file)?
|
|
.to_string_lossy()
|
|
.to_string(),
|
|
));
|
|
env_vars.push((
|
|
Self::DX_ERR_FILE,
|
|
dunce::canonicalize(&self.link_err_file)?
|
|
.to_string_lossy()
|
|
.to_string(),
|
|
));
|
|
env_vars.push((Self::DX_LINK_TRIPLE, self.triple.to_string()));
|
|
if let Some(linker) = &self.linker {
|
|
env_vars.push((
|
|
Self::DX_LINK_CUSTOM_LINKER,
|
|
dunce::canonicalize(linker)?.to_string_lossy().to_string(),
|
|
));
|
|
}
|
|
|
|
Ok(())
|
|
}
|
|
|
|
pub(crate) async fn run_link(self) {
|
|
let link_err_file = self.link_err_file.clone();
|
|
let res = self.run_link_inner().await;
|
|
|
|
if let Err(err) = res {
|
|
// If we failed to run the linker, we need to write the error to the file
|
|
// so that the main process can read it.
|
|
_ = std::fs::create_dir_all(link_err_file.parent().unwrap());
|
|
_ = std::fs::write(link_err_file, format!("Linker error: {err}"));
|
|
}
|
|
}
|
|
|
|
/// Write the incoming linker args to a file
|
|
///
|
|
/// The file will be given by the dx-magic-link-arg env var itself, so we use
|
|
/// it both for determining if we should act as a linker and the for the file name itself.
|
|
async fn run_link_inner(self) -> Result<()> {
|
|
let mut args: Vec<_> = std::env::args().collect();
|
|
if args.is_empty() {
|
|
return Ok(());
|
|
}
|
|
|
|
handle_linker_command_file(&mut args);
|
|
|
|
// Write the linker args to a file for the main process to read
|
|
// todo: we might need to encode these as escaped shell words in case newlines are passed
|
|
std::fs::write(self.link_args_file, args.join("\n"))?;
|
|
|
|
// If there's a linker specified, we use that. Otherwise, we write a dummy object file to satisfy
|
|
// any post-processing steps that rustc does.
|
|
match self.linker {
|
|
Some(linker) => {
|
|
let res = std::process::Command::new(linker)
|
|
.args(args.iter().skip(1))
|
|
.output()
|
|
.expect("Failed to run linker");
|
|
|
|
if !res.stderr.is_empty() || !res.stdout.is_empty() {
|
|
_ = std::fs::create_dir_all(self.link_err_file.parent().unwrap());
|
|
_ = std::fs::write(
|
|
self.link_err_file,
|
|
format!(
|
|
"Linker error: {}\n{}",
|
|
String::from_utf8_lossy(&res.stdout),
|
|
String::from_utf8_lossy(&res.stderr)
|
|
),
|
|
);
|
|
}
|
|
}
|
|
None => {
|
|
// Extract the out path - we're going to write a dummy object file to satisfy the linker
|
|
let out_file: PathBuf = match self.triple.operating_system {
|
|
target_lexicon::OperatingSystem::Windows => {
|
|
let out_arg = args.iter().find(|arg| arg.starts_with("/OUT")).unwrap();
|
|
out_arg.trim_start_matches("/OUT:").to_string().into()
|
|
}
|
|
_ => {
|
|
let out = args.iter().position(|arg| arg == "-o").unwrap();
|
|
args[out + 1].clone().into()
|
|
}
|
|
};
|
|
|
|
// This creates an object file that satisfies rust's use of llvm-objcopy
|
|
//
|
|
// I'd rather we *not* do this and instead generate a truly linked file (and then delete it) but
|
|
// this at least lets us delay linking until the host compiler is ready.
|
|
//
|
|
// This is because our host compiler is a stateful server and not a stateless linker.
|
|
//
|
|
// todo(jon): do we use Triple::host or the target triple? I think I ran into issues
|
|
// using the target triple, hence the use of "host" but it might not even matter?
|
|
let triple = Triple::host();
|
|
let format = match triple.binary_format {
|
|
target_lexicon::BinaryFormat::Elf => object::BinaryFormat::Elf,
|
|
target_lexicon::BinaryFormat::Coff => object::BinaryFormat::Coff,
|
|
target_lexicon::BinaryFormat::Macho => object::BinaryFormat::MachO,
|
|
target_lexicon::BinaryFormat::Wasm => object::BinaryFormat::Wasm,
|
|
target_lexicon::BinaryFormat::Xcoff => object::BinaryFormat::Xcoff,
|
|
target_lexicon::BinaryFormat::Unknown => todo!(),
|
|
_ => todo!("Binary format not supported"),
|
|
};
|
|
|
|
let arch = match triple.architecture {
|
|
target_lexicon::Architecture::Wasm32 => object::Architecture::Wasm32,
|
|
target_lexicon::Architecture::Wasm64 => object::Architecture::Wasm64,
|
|
target_lexicon::Architecture::X86_64 => object::Architecture::X86_64,
|
|
target_lexicon::Architecture::Arm(_) => object::Architecture::Arm,
|
|
target_lexicon::Architecture::Aarch64(_) => object::Architecture::Aarch64,
|
|
target_lexicon::Architecture::LoongArch64 => object::Architecture::LoongArch64,
|
|
target_lexicon::Architecture::Unknown => object::Architecture::Unknown,
|
|
_ => todo!("Architecture not supported"),
|
|
};
|
|
|
|
let endian = match triple.endianness() {
|
|
Ok(target_lexicon::Endianness::Little) => object::Endianness::Little,
|
|
Ok(target_lexicon::Endianness::Big) => object::Endianness::Big,
|
|
Err(_) => todo!("Endianness not supported"),
|
|
};
|
|
|
|
let bytes = object::write::Object::new(format, arch, endian)
|
|
.write()
|
|
.context("Failed to emit stub link file")?;
|
|
|
|
// Write a dummy object file to satisfy rust/linker since it'll run llvm-objcopy
|
|
// ... I wish it *didn't* do that but I can't tell how to disable the linker without
|
|
// using --emit=obj which is not exactly what we want since that will still pull in
|
|
// the dependencies.
|
|
std::fs::create_dir_all(out_file.parent().unwrap())?;
|
|
std::fs::write(out_file, bytes)?;
|
|
}
|
|
}
|
|
|
|
Ok(())
|
|
}
|
|
}
|
|
|
|
pub fn handle_linker_command_file(args: &mut Vec<String>) {
|
|
// Handle command files, usually a windows thing.
|
|
if let Some(command) = args.iter().find(|arg| arg.starts_with('@')).cloned() {
|
|
let path = command.trim().trim_start_matches('@');
|
|
let file_binary = std::fs::read(path).unwrap();
|
|
|
|
// This may be a utf-16le file. Let's try utf-8 first.
|
|
let content = String::from_utf8(file_binary.clone()).unwrap_or_else(|_| {
|
|
// Convert Vec<u8> to Vec<u16> to convert into a String
|
|
let binary_u16le: Vec<u16> = file_binary
|
|
.chunks_exact(2)
|
|
.map(|a| u16::from_le_bytes([a[0], a[1]]))
|
|
.collect();
|
|
|
|
String::from_utf16_lossy(&binary_u16le)
|
|
});
|
|
|
|
// Gather linker args, and reset the args to be just the linker args
|
|
*args = content
|
|
.lines()
|
|
.map(|line| {
|
|
let line_parsed = line.trim().to_string();
|
|
let line_parsed = line_parsed.trim_end_matches('"').to_string();
|
|
let line_parsed = line_parsed.trim_start_matches('"').to_string();
|
|
line_parsed
|
|
})
|
|
.collect();
|
|
}
|
|
}
|