swaits/dioxus
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Restore manganis optimizations (#3195)

Browse Source 
* Create a crate for constant serialization of config structs for manganis
* use SerializeConst for the image asset builder
* switch to a declarative macro for assets
* add serializable options for each asset type
* implement the new manganis macro
* optimize assets in the CLI
* Fix assets with dioxus formattable
* reduce fuzzing test limits
* use the new syntax in examples
* fix avif support
* Fix manganis doc tests
* cache asset optimization
* Split out asset and bundled asset
* make hash pointer width independent
* Fix manganis macro docs
* add constvec::is_empty method to fix clippy lint
* remove nasm feature
* simplify test_rsplit_once test so typos passes
* revert example change from clippy fix
* always accept unix paths
* fix windows path seperator
* fix folder asset hash
* Optimize assets in a blocking task
* Fix asset options docs
* Document Asset and BundledAsset
* simplify the linker macro a bit
* add more docs to AssetParser expansion
* add image format helpers
* Split by  both unix and windows path separators and take the smallest one
This commit is contained in:
Evan Almloff authored and GitHub committed 2024-11-26 18:12:24 +01:00
1 parent 31d72db82d
commit f9617c8d33
64 files changed
+6233 -704
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

packages/const-serialize/.gitignore
+1
View File
@@ -0,0 +1 @@
target
packages/const-serialize/Cargo.toml
+22
View File
@@ -0,0 +1,22 @@
[package]
name = "const-serialize"
version = { workspace = true }
authors = ["Evan Almloff"]
edition = "2021"
description = "A serialization framework that works in const contexts"
license = "MIT OR Apache-2.0"
repository = "https://github.com/dioxuslabs/dioxus"
homepage = "https://dioxuslabs.com/learn/0.5/getting_started"
keywords = ["const", "serialize"]
[dependencies]
const-serialize-macro = { workspace = true }
serde = { workspace = true, features = ["derive"], optional = true }
[dev-dependencies]
const-serialize = { path = "." }
rand.workspace = true
[features]
test-big-endian = []
serde = ["dep:serde"]
packages/const-serialize/README.md
+57
View File
@@ -0,0 +1,57 @@
A rust serialization library that works in const with complex(ish) types like enums, nested structs and arrays. Const rust does not have an allocator, so this library cannot work in a cross architecture environment with Vecs, slices or strings.
```rust
use const_serialize::{deserialize_const, serialize_const, serialize_eq, ConstVec, SerializeConst};
#[derive(Clone, Copy, Debug, PartialEq, SerializeConst)]
struct Struct {
a: u32,
b: u8,
c: u32,
d: Enum,
}
#[derive(Clone, Copy, Debug, PartialEq, SerializeConst)]
#[repr(C, u8)]
enum Enum {
A { one: u32, two: u16 },
B { one: u8, two: u16 } = 15,
}
const {
let data = [Struct {
a: 0x11111111,
b: 0x22,
c: 0x33333333,
d: Enum::A {
one: 0x44444444,
two: 0x5555,
},
}; 3];
let mut buf = ConstVec::new();
buf = serialize_const(&data, buf);
let buf = buf.read();
let (buf, deserialized) = match deserialize_const!([Struct; 3], buf) {
Some(data) => data,
None => panic!("data mismatch"),
};
if !serialize_eq(&data, &deserialized) {
panic!("data mismatch");
}
}
```
## How it works
`const-serialize` relies heavily on well defined layouts for the types you want to serialize. The serialization format is the linear sequence of unaligned bytes stored in the order of the fields, items or variants of the type. Numbers are stored in little endian order.
In order to support complex nested types, serialization is done using a trait. Since functions in traits cannot be const, `const-serialize` uses a macro to generate constant associated items that describe the memory layout of the type. That layout is then used to read all of the bytes in the type into the serialized buffer.
The deserialization is done in a similar way, but the layout is used to write the bytes from the serialized buffer into the type.
The rust [nomicon](https://doc.rust-lang.org/nomicon/data.html) defines the memory layout of different types. It is used as a reference for the layout of the types implemented in `const-serialize`.
## Limitations
- Only constant sized types are supported. This means that you can't serialize a type like `Vec<T>`. These types are difficult to create in const contexts in general
- Only types with a well defined memory layout are supported (see https://github.com/rust-lang/rfcs/pull/3727 and https://onevariable.com/blog/pods-from-scratch). `repr(Rust)` enums don't have a well defined layout, so they are not supported. `repr(C, u8)` enums can be used instead
- Const rust does not support mutable references or points, so this crate leans heavily on function data structures for data processing.
packages/const-serialize/src/const_buffers.rs
+33
View File
@@ -0,0 +1,33 @@
/// A buffer that can be read from at compile time. This is very similar to [Cursor](std::io::Cursor) but is
/// designed to be used in const contexts.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct ConstReadBuffer<'a> {
location: usize,
memory: &'a [u8],
}
impl<'a> ConstReadBuffer<'a> {
/// Create a new buffer from a byte slice
pub const fn new(memory: &'a [u8]) -> Self {
Self {
location: 0,
memory,
}
}
/// Get the next byte from the buffer. Returns `None` if the buffer is empty.
/// This will return the new version of the buffer with the first byte removed.
pub const fn get(mut self) -> Option<(Self, u8)> {
if self.location >= self.memory.len() {
return None;
}
let value = self.memory[self.location];
self.location += 1;
Some((self, value))
}
/// Get a reference to the underlying byte slice
pub const fn as_ref(&self) -> &[u8] {
self.memory
}
}
packages/const-serialize/src/const_vec.rs
+422
View File
@@ -0,0 +1,422 @@
#![allow(dead_code)]
use std::{fmt::Debug, hash::Hash, mem::MaybeUninit};
use crate::ConstReadBuffer;
const DEFAULT_MAX_SIZE: usize = 2usize.pow(10);
/// [`ConstVec`] is a version of [`Vec`] that is usable in const contexts. It has
/// a fixed maximum size, but it can can grow and shrink within that size limit
/// as needed.
///
/// # Example
/// ```rust
/// # use const_serialize::ConstVec;
/// const EMPTY: ConstVec<u8> = ConstVec::new();
/// // Methods that mutate the vector will return a new vector
/// const ONE: ConstVec<u8> = EMPTY.push(1);
/// const TWO: ConstVec<u8> = ONE.push(2);
/// const THREE: ConstVec<u8> = TWO.push(3);
/// const FOUR: ConstVec<u8> = THREE.push(4);
/// // If a value is also returned, that will be placed in a tuple in the return value
/// // along with the new vector
/// const POPPED: (ConstVec<u8>, Option<u8>) = FOUR.pop();
/// assert_eq!(POPPED.0, THREE);
/// assert_eq!(POPPED.1.unwrap(), 4);
/// ```
pub struct ConstVec<T, const MAX_SIZE: usize = DEFAULT_MAX_SIZE> {
memory: [MaybeUninit<T>; MAX_SIZE],
len: u32,
}
impl<T: Clone, const MAX_SIZE: usize> Clone for ConstVec<T, MAX_SIZE> {
fn clone(&self) -> Self {
let mut cloned = Self::new_with_max_size();
for i in 0..self.len as usize {
cloned = cloned.push(self.get(i).unwrap().clone());
}
cloned
}
}
impl<T: Copy, const MAX_SIZE: usize> Copy for ConstVec<T, MAX_SIZE> {}
impl<T: PartialEq, const MAX_SIZE: usize> PartialEq for ConstVec<T, MAX_SIZE> {
fn eq(&self, other: &Self) -> bool {
self.as_ref() == other.as_ref()
}
}
impl<T: Hash, const MAX_SIZE: usize> Hash for ConstVec<T, MAX_SIZE> {
fn hash<H: std::hash::Hasher>(&self, state: &mut H) {
self.as_ref().hash(state)
}
}
impl<T, const MAX_SIZE: usize> Default for ConstVec<T, MAX_SIZE> {
fn default() -> Self {
Self::new_with_max_size()
}
}
impl<T: Debug, const MAX_SIZE: usize> Debug for ConstVec<T, MAX_SIZE> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("ConstVec")
.field("len", &self.len)
.field("memory", &self.as_ref())
.finish()
}
}
impl<T> ConstVec<T> {
/// Create a new empty [`ConstVec`]
pub const fn new() -> Self {
Self::new_with_max_size()
}
}
impl<T, const MAX_SIZE: usize> ConstVec<T, MAX_SIZE> {
/// Create a new empty [`ConstVec`] with a custom maximum size
///
/// # Example
/// ```rust
/// # use const_serialize::ConstVec;
/// const EMPTY: ConstVec<u8, 10> = ConstVec::new_with_max_size();
/// ```
pub const fn new_with_max_size() -> Self {
Self {
memory: [const { MaybeUninit::uninit() }; MAX_SIZE],
len: 0,
}
}
/// Push a value onto the end of the [`ConstVec`]
///
/// # Example
/// ```rust
/// # use const_serialize::ConstVec;
/// const EMPTY: ConstVec<u8> = ConstVec::new();
/// const ONE: ConstVec<u8> = EMPTY.push(1);
/// assert_eq!(ONE.as_ref(), &[1]);
/// ```
pub const fn push(mut self, value: T) -> Self {
self.memory[self.len as usize] = MaybeUninit::new(value);
self.len += 1;
self
}
/// Extend the [`ConstVec`] with the contents of a slice
///
/// # Example
/// ```rust
/// # use const_serialize::ConstVec;
/// const EMPTY: ConstVec<u8> = ConstVec::new();
/// const ONE: ConstVec<u8> = EMPTY.extend(&[1, 2, 3]);
/// assert_eq!(ONE.as_ref(), &[1, 2, 3]);
/// ```
pub const fn extend(mut self, other: &[T]) -> Self
where
T: Copy,
{
let mut i = 0;
while i < other.len() {
self = self.push(other[i]);
i += 1;
}
self
}
/// Get a reference to the value at the given index
///
/// # Example
/// ```rust
/// # use const_serialize::ConstVec;
/// const EMPTY: ConstVec<u8> = ConstVec::new();
/// const ONE: ConstVec<u8> = EMPTY.push(1);
/// assert_eq!(ONE.get(0), Some(&1));
/// ```
pub const fn get(&self, index: usize) -> Option<&T> {
if index < self.len as usize {
Some(unsafe { &*self.memory[index].as_ptr() })
} else {
None
}
}
/// Get the length of the [`ConstVec`]
///
/// # Example
/// ```rust
/// # use const_serialize::ConstVec;
/// const EMPTY: ConstVec<u8> = ConstVec::new();
/// const ONE: ConstVec<u8> = EMPTY.push(1);
/// assert_eq!(ONE.len(), 1);
/// ```
pub const fn len(&self) -> usize {
self.len as usize
}
/// Check if the [`ConstVec`] is empty
///
/// # Example
/// ```rust
/// # use const_serialize::ConstVec;
/// const EMPTY: ConstVec<u8> = ConstVec::new();
/// assert!(EMPTY.is_empty());
/// const ONE: ConstVec<u8> = EMPTY.push(1);
/// assert!(!ONE.is_empty());
/// ```
pub const fn is_empty(&self) -> bool {
self.len == 0
}
/// Get a reference to the underlying slice
///
/// # Example
/// ```rust
/// # use const_serialize::ConstVec;
/// const EMPTY: ConstVec<u8> = ConstVec::new();
/// const ONE: ConstVec<u8> = EMPTY.push(1);
/// assert_eq!(ONE.as_ref(), &[1]);
/// ```
pub const fn as_ref(&self) -> &[T] {
unsafe {
&*(self.memory.split_at(self.len as usize).0 as *const [MaybeUninit<T>] as *const [T])
}
}
/// Swap the values at the given indices
///
/// # Example
/// ```rust
/// # use const_serialize::ConstVec;
/// const EMPTY: ConstVec<u8> = ConstVec::new();
/// const ONE: ConstVec<u8> = EMPTY.push(1);
/// const TWO: ConstVec<u8> = ONE.push(2);
/// const THREE: ConstVec<u8> = TWO.swap(0, 1);
/// assert_eq!(THREE.as_ref(), &[2, 1]);
/// ```
pub const fn swap(mut self, first: usize, second: usize) -> Self
where
T: Copy,
{
assert!(first < self.len as usize);
assert!(second < self.len as usize);
let temp = self.memory[first];
self.memory[first] = self.memory[second];
self.memory[second] = temp;
self
}
/// Pop a value off the end of the [`ConstVec`]
///
/// # Example
/// ```rust
/// # use const_serialize::ConstVec;
/// const EMPTY: ConstVec<u8> = ConstVec::new();
/// const ONE: ConstVec<u8> = EMPTY.push(1);
/// const TWO: ConstVec<u8> = ONE.push(2);
/// const THREE: ConstVec<u8> = TWO.push(3);
/// const POPPED: (ConstVec<u8>, Option<u8>) = THREE.pop();
/// assert_eq!(POPPED.0, TWO);
/// assert_eq!(POPPED.1.unwrap(), 3);
/// ```
pub const fn pop(mut self) -> (Self, Option<T>)
where
T: Copy,
{
let value = if self.len > 0 {
self.len -= 1;
let last = self.len as usize;
let last_value = unsafe { self.memory[last].assume_init() };
Some(last_value)
} else {
None
};
(self, value)
}
/// Remove the value at the given index
///
/// # Example
/// ```rust
/// # use const_serialize::ConstVec;
/// const EMPTY: ConstVec<u8> = ConstVec::new();
/// const ONE: ConstVec<u8> = EMPTY.push(1);
/// const TWO: ConstVec<u8> = ONE.push(2);
/// const THREE: ConstVec<u8> = TWO.push(3);
/// const REMOVED: (ConstVec<u8>, Option<u8>) = THREE.remove(1);
/// assert_eq!(REMOVED.0.as_ref(), &[1, 3]);
/// assert_eq!(REMOVED.1.unwrap(), 2);
/// ```
pub const fn remove(mut self, index: usize) -> (Self, Option<T>)
where
T: Copy,
{
let value = if index < self.len as usize {
let value = unsafe { self.memory[index].assume_init() };
let mut swap_index = index;
while swap_index + 1 < self.len as usize {
self.memory[swap_index] = self.memory[swap_index + 1];
swap_index += 1;
}
self.len -= 1;
Some(value)
} else {
None
};
(self, value)
}
/// Set the value at the given index
///
/// # Example
/// ```rust
/// # use const_serialize::ConstVec;
/// const EMPTY: ConstVec<u8> = ConstVec::new();
/// const ONE: ConstVec<u8> = EMPTY.push(1);
/// const TWO: ConstVec<u8> = ONE.set(0, 2);
/// assert_eq!(TWO.as_ref(), &[2]);
/// ```
pub const fn set(mut self, index: usize, value: T) -> Self {
if index >= self.len as usize {
panic!("Out of bounds")
}
self.memory[index] = MaybeUninit::new(value);
self
}
pub(crate) const fn into_parts(self) -> ([MaybeUninit<T>; MAX_SIZE], usize) {
(self.memory, self.len as usize)
}
/// Split the [`ConstVec`] into two at the given index
///
/// # Example
/// ```rust
/// # use const_serialize::ConstVec;
/// const EMPTY: ConstVec<u8> = ConstVec::new();
/// const ONE: ConstVec<u8> = EMPTY.push(1);
/// const TWO: ConstVec<u8> = ONE.push(2);
/// const THREE: ConstVec<u8> = TWO.push(3);
/// const SPLIT: (ConstVec<u8>, ConstVec<u8>) = THREE.split_at(1);
/// assert_eq!(SPLIT.0.as_ref(), &[1]);
/// assert_eq!(SPLIT.1.as_ref(), &[2, 3]);
/// ```
pub const fn split_at(&self, index: usize) -> (Self, Self)
where
T: Copy,
{
assert!(index <= self.len as usize);
let slice = self.as_ref();
let (left, right) = slice.split_at(index);
let mut left_vec = Self::new_with_max_size();
let mut i = 0;
while i < left.len() {
left_vec = left_vec.push(left[i]);
i += 1;
}
let mut right_vec = Self::new_with_max_size();
i = 0;
while i < right.len() {
right_vec = right_vec.push(right[i]);
i += 1;
}
(left_vec, right_vec)
}
}
impl<const MAX_SIZE: usize> ConstVec<u8, MAX_SIZE> {
/// Convert the [`ConstVec`] into a [`ConstReadBuffer`](crate::ConstReadBuffer)
///
/// # Example
/// ```rust
/// # use const_serialize::{ConstVec, ConstReadBuffer};
/// const EMPTY: ConstVec<u8> = ConstVec::new();
/// const ONE: ConstVec<u8> = EMPTY.push(1);
/// const TWO: ConstVec<u8> = ONE.push(2);
/// const READ: ConstReadBuffer = TWO.read();
/// ```
pub const fn read(&self) -> ConstReadBuffer<'_> {
ConstReadBuffer::new(self.as_ref())
}
}
#[test]
fn test_const_vec() {
const VEC: ConstVec<u32> = {
let mut vec = ConstVec::new();
vec = vec.push(1234);
vec = vec.push(5678);
vec
};
assert_eq!(VEC.as_ref(), &[1234, 5678]);
let vec = VEC;
let (vec, value) = vec.pop();
assert_eq!(value, Some(5678));
let (vec, value) = vec.pop();
assert_eq!(value, Some(1234));
let (vec, value) = vec.pop();
assert_eq!(value, None);
assert_eq!(vec.as_ref(), &[]);
}
#[test]
fn test_const_vec_len() {
const VEC: ConstVec<u32> = {
let mut vec = ConstVec::new();
vec = vec.push(1234);
vec = vec.push(5678);
vec
};
assert_eq!(VEC.len(), 2);
}
#[test]
fn test_const_vec_get() {
const VEC: ConstVec<u32> = {
let mut vec = ConstVec::new();
vec = vec.push(1234);
vec = vec.push(5678);
vec
};
assert_eq!(VEC.get(0), Some(&1234));
assert_eq!(VEC.get(1), Some(&5678));
assert_eq!(VEC.get(2), None);
}
#[test]
fn test_const_vec_swap() {
const VEC: ConstVec<u32> = {
let mut vec = ConstVec::new();
vec = vec.push(1234);
vec = vec.push(5678);
vec
};
let mut vec = VEC;
assert_eq!(vec.as_ref(), &[1234, 5678]);
vec = vec.swap(0, 1);
assert_eq!(vec.as_ref(), &[5678, 1234]);
vec = vec.swap(0, 1);
assert_eq!(vec.as_ref(), &[1234, 5678]);
}
#[test]
fn test_const_vec_remove() {
const VEC: ConstVec<u32> = {
let mut vec = ConstVec::new();
vec = vec.push(1234);
vec = vec.push(5678);
vec
};
let vec = VEC;
println!("{:?}", vec);
assert_eq!(vec.as_ref(), &[1234, 5678]);
let (vec, value) = vec.remove(0);
assert_eq!(value, Some(1234));
assert_eq!(vec.as_ref(), &[5678]);
let (vec, value) = vec.remove(0);
assert_eq!(value, Some(5678));
assert_eq!(vec.as_ref(), &[]);
}
packages/const-serialize/src/lib.rs
+924
View File
@@ -0,0 +1,924 @@
#![doc = include_str!("../README.md")]
#![warn(missing_docs)]
use std::{char, mem::MaybeUninit};
mod const_buffers;
mod const_vec;
pub use const_buffers::ConstReadBuffer;
pub use const_serialize_macro::SerializeConst;
pub use const_vec::ConstVec;
/// Plain old data for a field. Stores the offset of the field in the struct and the layout of the field.
#[derive(Debug, Copy, Clone)]
pub struct StructFieldLayout {
offset: usize,
layout: Layout,
}
impl StructFieldLayout {
/// Create a new struct field layout
pub const fn new(offset: usize, layout: Layout) -> Self {
Self { offset, layout }
}
}
/// Layout for a struct. The struct layout is just a list of fields with offsets
#[derive(Debug, Copy, Clone)]
pub struct StructLayout {
size: usize,
data: &'static [StructFieldLayout],
}
impl StructLayout {
/// Create a new struct layout
pub const fn new(size: usize, data: &'static [StructFieldLayout]) -> Self {
Self { size, data }
}
}
/// The layout for an enum. The enum layout is just a discriminate size and a tag layout.
#[derive(Debug, Copy, Clone)]
pub struct EnumLayout {
size: usize,
discriminant: PrimitiveLayout,
variants_offset: usize,
variants: &'static [EnumVariant],
}
impl EnumLayout {
/// Create a new enum layout
pub const fn new(
size: usize,
discriminant: PrimitiveLayout,
variants: &'static [EnumVariant],
) -> Self {
let mut max_align = 1;
let mut i = 0;
while i < variants.len() {
let EnumVariant { align, .. } = &variants[i];
if *align > max_align {
max_align = *align;
}
i += 1;
}
let variants_offset = (discriminant.size / max_align) + max_align;
Self {
size,
discriminant,
variants_offset,
variants,
}
}
}
/// The layout for an enum variant. The enum variant layout is just a struct layout with a tag and alignment.
#[derive(Debug, Copy, Clone)]
pub struct EnumVariant {
// Note: tags may not be sequential
tag: u32,
data: StructLayout,
align: usize,
}
impl EnumVariant {
/// Create a new enum variant layout
pub const fn new(tag: u32, data: StructLayout, align: usize) -> Self {
Self { tag, data, align }
}
}
/// The layout for a constant sized array. The array layout is just a length and an item layout.
#[derive(Debug, Copy, Clone)]
pub struct ListLayout {
len: usize,
item_layout: &'static Layout,
}
impl ListLayout {
/// Create a new list layout
pub const fn new(len: usize, item_layout: &'static Layout) -> Self {
Self { len, item_layout }
}
}
/// The layout for a primitive type. The bytes will be reversed if the target is big endian.
#[derive(Debug, Copy, Clone)]
pub struct PrimitiveLayout {
size: usize,
}
impl PrimitiveLayout {
/// Create a new primitive layout
pub const fn new(size: usize) -> Self {
Self { size }
}
}
/// The layout for a type. This layout defines a sequence of locations and reversed or not bytes. These bytes will be copied from during serialization and copied into during deserialization.
#[derive(Debug, Copy, Clone)]
pub enum Layout {
/// An enum layout
Enum(EnumLayout),
/// A struct layout
Struct(StructLayout),
/// A list layout
List(ListLayout),
/// A primitive layout
Primitive(PrimitiveLayout),
}
impl Layout {
/// The size of the type in bytes.
const fn size(&self) -> usize {
match self {
Layout::Enum(layout) => layout.size,
Layout::Struct(layout) => layout.size,
Layout::List(layout) => layout.len * layout.item_layout.size(),
Layout::Primitive(layout) => layout.size,
}
}
}
/// A trait for types that can be serialized and deserialized in const.
///
/// # Safety
/// The layout must accurately describe the memory layout of the type
pub unsafe trait SerializeConst: Sized {
/// The memory layout of the type. This type must have plain old data; no pointers or references.
const MEMORY_LAYOUT: Layout;
/// Assert that the memory layout of the type is the same as the size of the type
const _ASSERT: () = assert!(Self::MEMORY_LAYOUT.size() == std::mem::size_of::<Self>());
}
macro_rules! impl_serialize_const {
($type:ty) => {
unsafe impl SerializeConst for $type {
const MEMORY_LAYOUT: Layout = Layout::Primitive(PrimitiveLayout {
size: std::mem::size_of::<$type>(),
});
}
};
}
impl_serialize_const!(u8);
impl_serialize_const!(u16);
impl_serialize_const!(u32);
impl_serialize_const!(u64);
impl_serialize_const!(i8);
impl_serialize_const!(i16);
impl_serialize_const!(i32);
impl_serialize_const!(i64);
impl_serialize_const!(bool);
impl_serialize_const!(f32);
impl_serialize_const!(f64);
unsafe impl<const N: usize, T: SerializeConst> SerializeConst for [T; N] {
const MEMORY_LAYOUT: Layout = Layout::List(ListLayout {
len: N,
item_layout: &T::MEMORY_LAYOUT,
});
}
macro_rules! impl_serialize_const_tuple {
($($generic:ident: $generic_number:expr),*) => {
impl_serialize_const_tuple!(@impl ($($generic,)*) = $($generic: $generic_number),*);
};
(@impl $inner:ty = $($generic:ident: $generic_number:expr),*) => {
unsafe impl<$($generic: SerializeConst),*> SerializeConst for ($($generic,)*) {
const MEMORY_LAYOUT: Layout = {
Layout::Struct(StructLayout {
size: std::mem::size_of::<($($generic,)*)>(),
data: &[
$(
StructFieldLayout::new(std::mem::offset_of!($inner, $generic_number), $generic::MEMORY_LAYOUT),
)*
],
})
};
}
};
}
impl_serialize_const_tuple!(T1: 0);
impl_serialize_const_tuple!(T1: 0, T2: 1);
impl_serialize_const_tuple!(T1: 0, T2: 1, T3: 2);
impl_serialize_const_tuple!(T1: 0, T2: 1, T3: 2, T4: 3);
impl_serialize_const_tuple!(T1: 0, T2: 1, T3: 2, T4: 3, T5: 4);
impl_serialize_const_tuple!(T1: 0, T2: 1, T3: 2, T4: 3, T5: 4, T6: 5);
impl_serialize_const_tuple!(T1: 0, T2: 1, T3: 2, T4: 3, T5: 4, T6: 5, T7: 6);
impl_serialize_const_tuple!(T1: 0, T2: 1, T3: 2, T4: 3, T5: 4, T6: 5, T7: 6, T8: 7);
impl_serialize_const_tuple!(T1: 0, T2: 1, T3: 2, T4: 3, T5: 4, T6: 5, T7: 6, T8: 7, T9: 8);
impl_serialize_const_tuple!(T1: 0, T2: 1, T3: 2, T4: 3, T5: 4, T6: 5, T7: 6, T8: 7, T9: 8, T10: 9);
const MAX_STR_SIZE: usize = 256;
/// A string that is stored in a constant sized buffer that can be serialized and deserialized at compile time
#[derive(Debug, PartialEq, PartialOrd, Clone, Copy, Hash)]
#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))]
pub struct ConstStr {
#[cfg_attr(feature = "serde", serde(with = "serde_bytes"))]
bytes: [u8; MAX_STR_SIZE],
len: u32,
}
#[cfg(feature = "serde")]
mod serde_bytes {
use serde::{Deserialize, Serializer};
pub fn serialize<S>(bytes: &[u8], serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
serializer.serialize_bytes(bytes)
}
pub fn deserialize<'de, D>(deserializer: D) -> Result<[u8; super::MAX_STR_SIZE], D::Error>
where
D: serde::Deserializer<'de>,
{
let bytes = Vec::<u8>::deserialize(deserializer)?;
bytes
.try_into()
.map_err(|_| serde::de::Error::custom("Failed to convert bytes to a fixed size array"))
}
}
unsafe impl SerializeConst for ConstStr {
const MEMORY_LAYOUT: Layout = Layout::Struct(StructLayout {
size: std::mem::size_of::<Self>(),
data: &[
StructFieldLayout::new(
std::mem::offset_of!(Self, bytes),
Layout::List(ListLayout {
len: MAX_STR_SIZE,
item_layout: &Layout::Primitive(PrimitiveLayout {
size: std::mem::size_of::<u8>(),
}),
}),
),
StructFieldLayout::new(
std::mem::offset_of!(Self, len),
Layout::Primitive(PrimitiveLayout {
size: std::mem::size_of::<u32>(),
}),
),
],
});
}
impl ConstStr {
/// Create a new constant string
pub const fn new(s: &str) -> Self {
let str_bytes = s.as_bytes();
let mut bytes = [0; MAX_STR_SIZE];
let mut i = 0;
while i < str_bytes.len() {
bytes[i] = str_bytes[i];
i += 1;
}
Self {
bytes,
len: str_bytes.len() as u32,
}
}
/// Get a reference to the string
pub const fn as_str(&self) -> &str {
let str_bytes = self.bytes.split_at(self.len as usize).0;
match std::str::from_utf8(str_bytes) {
Ok(s) => s,
Err(_) => panic!(
"Invalid utf8; ConstStr should only ever be constructed from valid utf8 strings"
),
}
}
/// Get the length of the string
pub const fn len(&self) -> usize {
self.len as usize
}
/// Check if the string is empty
pub const fn is_empty(&self) -> bool {
self.len == 0
}
/// Push a character onto the string
pub const fn push(self, byte: char) -> Self {
assert!(byte.is_ascii(), "Only ASCII bytes are supported");
let (bytes, len) = char_to_bytes(byte);
let (str, _) = bytes.split_at(len);
let Ok(str) = std::str::from_utf8(str) else {
panic!("Invalid utf8; char_to_bytes should always return valid utf8 bytes")
};
self.push_str(str)
}
/// Push a str onto the string
pub const fn push_str(self, str: &str) -> Self {
let Self { mut bytes, len } = self;
assert!(
str.len() + len as usize <= MAX_STR_SIZE,
"String is too long"
);
let str_bytes = str.as_bytes();
let new_len = len as usize + str_bytes.len();
let mut i = 0;
while i < str_bytes.len() {
bytes[len as usize + i] = str_bytes[i];
i += 1;
}
Self {
bytes,
len: new_len as u32,
}
}
/// Split the string at a byte index. The byte index must be a char boundary
pub const fn split_at(self, index: usize) -> (Self, Self) {
let (left, right) = self.bytes.split_at(index);
let left = match std::str::from_utf8(left) {
Ok(s) => s,
Err(_) => {
panic!("Invalid utf8; you cannot split at a byte that is not a char boundary")
}
};
let right = match std::str::from_utf8(right) {
Ok(s) => s,
Err(_) => {
panic!("Invalid utf8; you cannot split at a byte that is not a char boundary")
}
};
(Self::new(left), Self::new(right))
}
/// Split the string at the last occurrence of a character
pub const fn rsplit_once(&self, char: char) -> Option<(Self, Self)> {
let str = self.as_str();
let mut index = str.len() - 1;
// First find the bytes we are searching for
let (char_bytes, len) = char_to_bytes(char);
let (char_bytes, _) = char_bytes.split_at(len);
let bytes = str.as_bytes();
// Then walk backwards from the end of the string
loop {
let byte = bytes[index];
// Look for char boundaries in the string and check if the bytes match
if let Some(char_boundary_len) = utf8_char_boundary_to_char_len(byte) {
// Split up the string into three sections: [before_char, in_char, after_char]
let (before_char, after_index) = bytes.split_at(index);
let (in_char, after_char) = after_index.split_at(char_boundary_len as usize);
if in_char.len() != char_boundary_len as usize {
panic!("in_char.len() should always be equal to char_boundary_len as usize")
}
// Check if the bytes for the current char and the target char match
let mut in_char_eq = true;
let mut i = 0;
let min_len = if in_char.len() < char_bytes.len() {
in_char.len()
} else {
char_bytes.len()
};
while i < min_len {
in_char_eq &= in_char[i] == char_bytes[i];
i += 1;
}
// If they do, convert the bytes to strings and return the split strings
if in_char_eq {
let Ok(before_char_str) = std::str::from_utf8(before_char) else {
panic!("Invalid utf8; utf8_char_boundary_to_char_len should only return Some when the byte is a character boundary")
};
let Ok(after_char_str) = std::str::from_utf8(after_char) else {
panic!("Invalid utf8; utf8_char_boundary_to_char_len should only return Some when the byte is a character boundary")
};
return Some((Self::new(before_char_str), Self::new(after_char_str)));
}
}
match index.checked_sub(1) {
Some(new_index) => index = new_index,
None => return None,
}
}
}
/// Split the string at the first occurrence of a character
pub const fn split_once(&self, char: char) -> Option<(Self, Self)> {
let str = self.as_str();
let mut index = 0;
// First find the bytes we are searching for
let (char_bytes, len) = char_to_bytes(char);
let (char_bytes, _) = char_bytes.split_at(len);
let bytes = str.as_bytes();
// Then walk forwards from the start of the string
while index < bytes.len() {
let byte = bytes[index];
// Look for char boundaries in the string and check if the bytes match
if let Some(char_boundary_len) = utf8_char_boundary_to_char_len(byte) {
// Split up the string into three sections: [before_char, in_char, after_char]
let (before_char, after_index) = bytes.split_at(index);
let (in_char, after_char) = after_index.split_at(char_boundary_len as usize);
if in_char.len() != char_boundary_len as usize {
panic!("in_char.len() should always be equal to char_boundary_len as usize")
}
// Check if the bytes for the current char and the target char match
let mut in_char_eq = true;
let mut i = 0;
let min_len = if in_char.len() < char_bytes.len() {
in_char.len()
} else {
char_bytes.len()
};
while i < min_len {
in_char_eq &= in_char[i] == char_bytes[i];
i += 1;
}
// If they do, convert the bytes to strings and return the split strings
if in_char_eq {
let Ok(before_char_str) = std::str::from_utf8(before_char) else {
panic!("Invalid utf8; utf8_char_boundary_to_char_len should only return Some when the byte is a character boundary")
};
let Ok(after_char_str) = std::str::from_utf8(after_char) else {
panic!("Invalid utf8; utf8_char_boundary_to_char_len should only return Some when the byte is a character boundary")
};
return Some((Self::new(before_char_str), Self::new(after_char_str)));
}
}
index += 1
}
None
}
}
#[test]
fn test_rsplit_once() {
let str = ConstStr::new("hello world");
assert_eq!(
str.rsplit_once(' '),
Some((ConstStr::new("hello"), ConstStr::new("world")))
);
let unicode_str = ConstStr::new("hi😀hello😀world😀world");
assert_eq!(
unicode_str.rsplit_once('😀'),
Some((ConstStr::new("hi😀hello😀world"), ConstStr::new("world")))
);
assert_eq!(unicode_str.rsplit_once('❌'), None);
for _ in 0..100 {
let random_str: String = (0..rand::random::<u8>() % 50)
.map(|_| rand::random::<char>())
.collect();
let konst = ConstStr::new(&random_str);
let mut seen_chars = std::collections::HashSet::new();
for char in random_str.chars().rev() {
let (char_bytes, len) = char_to_bytes(char);
let char_bytes = &char_bytes[..len];
assert_eq!(char_bytes, char.to_string().as_bytes());
if seen_chars.contains(&char) {
continue;
}
seen_chars.insert(char);
let (correct_left, correct_right) = random_str.rsplit_once(char).unwrap();
let (left, right) = konst.rsplit_once(char).unwrap();
println!("splitting {random_str:?} at {char:?}");
assert_eq!(left.as_str(), correct_left);
assert_eq!(right.as_str(), correct_right);
}
}
}
const CONTINUED_CHAR_MASK: u8 = 0b10000000;
const BYTE_CHAR_BOUNDARIES: [u8; 4] = [0b00000000, 0b11000000, 0b11100000, 0b11110000];
// Const version of https://doc.rust-lang.org/src/core/char/methods.rs.html#1765-1797
const fn char_to_bytes(char: char) -> ([u8; 4], usize) {
let code = char as u32;
let len = char.len_utf8();
let mut bytes = [0; 4];
match len {
1 => {
bytes[0] = code as u8;
}
2 => {
bytes[0] = (code >> 6 & 0x1F) as u8 | BYTE_CHAR_BOUNDARIES[1];
bytes[1] = (code & 0x3F) as u8 | CONTINUED_CHAR_MASK;
}
3 => {
bytes[0] = (code >> 12 & 0x0F) as u8 | BYTE_CHAR_BOUNDARIES[2];
bytes[1] = (code >> 6 & 0x3F) as u8 | CONTINUED_CHAR_MASK;
bytes[2] = (code & 0x3F) as u8 | CONTINUED_CHAR_MASK;
}
4 => {
bytes[0] = (code >> 18 & 0x07) as u8 | BYTE_CHAR_BOUNDARIES[3];
bytes[1] = (code >> 12 & 0x3F) as u8 | CONTINUED_CHAR_MASK;
bytes[2] = (code >> 6 & 0x3F) as u8 | CONTINUED_CHAR_MASK;
bytes[3] = (code & 0x3F) as u8 | CONTINUED_CHAR_MASK;
}
_ => panic!(
"encode_utf8: need more than 4 bytes to encode the unicode character, but the buffer has 4 bytes"
),
};
(bytes, len)
}
#[test]
fn fuzz_char_to_bytes() {
use std::char;
for _ in 0..100 {
let char = rand::random::<char>();
let (bytes, len) = char_to_bytes(char);
let str = std::str::from_utf8(&bytes[..len]).unwrap();
assert_eq!(char.to_string(), str);
}
}
const fn utf8_char_boundary_to_char_len(byte: u8) -> Option<u8> {
match byte {
0b00000000..=0b01111111 => Some(1),
0b11000000..=0b11011111 => Some(2),
0b11100000..=0b11101111 => Some(3),
0b11110000..=0b11111111 => Some(4),
_ => None,
}
}
#[test]
fn fuzz_utf8_byte_to_char_len() {
for _ in 0..100 {
let random_string: String = (0..rand::random::<u8>())
.map(|_| rand::random::<char>())
.collect();
let bytes = random_string.as_bytes();
let chars: std::collections::HashMap<_, _> = random_string.char_indices().collect();
for (i, byte) in bytes.iter().enumerate() {
match utf8_char_boundary_to_char_len(*byte) {
Some(char_len) => {
let char = chars
.get(&i)
.unwrap_or_else(|| panic!("{byte:b} is not a character boundary"));
assert_eq!(char.len_utf8(), char_len as usize);
}
None => {
assert!(!chars.contains_key(&i), "{byte:b} is a character boundary");
}
}
}
}
}
/// Serialize a struct that is stored at the pointer passed in
const fn serialize_const_struct(
ptr: *const (),
mut to: ConstVec<u8>,
layout: &StructLayout,
) -> ConstVec<u8> {
let mut i = 0;
while i < layout.data.len() {
// Serialize the field at the offset pointer in the struct
let StructFieldLayout { offset, layout } = &layout.data[i];
let field = unsafe { ptr.byte_add(*offset) };
to = serialize_const_ptr(field, to, layout);
i += 1;
}
to
}
/// Serialize an enum that is stored at the pointer passed in
const fn serialize_const_enum(
ptr: *const (),
mut to: ConstVec<u8>,
layout: &EnumLayout,
) -> ConstVec<u8> {
let mut discriminant = 0;
let byte_ptr = ptr as *const u8;
let mut offset = 0;
while offset < layout.discriminant.size {
// If the bytes are reversed, walk backwards from the end of the number when pushing bytes
let byte = if cfg!(target_endian = "big") {
unsafe {
byte_ptr
.byte_add(layout.discriminant.size - offset - 1)
.read()
}
} else {
unsafe { byte_ptr.byte_add(offset).read() }
};
to = to.push(byte);
discriminant |= (byte as u32) << (offset * 8);
offset += 1;
}
let mut i = 0;
while i < layout.variants.len() {
// If the variant is the discriminated one, serialize it
let EnumVariant { tag, data, .. } = &layout.variants[i];
if discriminant == *tag {
let data_ptr = unsafe { ptr.byte_add(layout.variants_offset) };
to = serialize_const_struct(data_ptr, to, data);
break;
}
i += 1;
}
to
}
/// Serialize a primitive type that is stored at the pointer passed in
const fn serialize_const_primitive(
ptr: *const (),
mut to: ConstVec<u8>,
layout: &PrimitiveLayout,
) -> ConstVec<u8> {
let ptr = ptr as *const u8;
let mut offset = 0;
while offset < layout.size {
// If the bytes are reversed, walk backwards from the end of the number when pushing bytes
if cfg!(any(target_endian = "big", feature = "test-big-endian")) {
to = to.push(unsafe { ptr.byte_add(layout.size - offset - 1).read() });
} else {
to = to.push(unsafe { ptr.byte_add(offset).read() });
}
offset += 1;
}
to
}
/// Serialize a constant sized array that is stored at the pointer passed in
const fn serialize_const_list(
ptr: *const (),
mut to: ConstVec<u8>,
layout: &ListLayout,
) -> ConstVec<u8> {
let len = layout.len;
let mut i = 0;
while i < len {
let field = unsafe { ptr.byte_add(i * layout.item_layout.size()) };
to = serialize_const_ptr(field, to, layout.item_layout);
i += 1;
}
to
}
/// Serialize a pointer to a type that is stored at the pointer passed in
const fn serialize_const_ptr(ptr: *const (), to: ConstVec<u8>, layout: &Layout) -> ConstVec<u8> {
match layout {
Layout::Enum(layout) => serialize_const_enum(ptr, to, layout),
Layout::Struct(layout) => serialize_const_struct(ptr, to, layout),
Layout::List(layout) => serialize_const_list(ptr, to, layout),
Layout::Primitive(layout) => serialize_const_primitive(ptr, to, layout),
}
}
/// Serialize a type into a buffer
///
/// # Example
///
/// ```rust
/// use const_serialize::{ConstVec, SerializeConst, serialize_const};
///
/// #[derive(Clone, Copy, Debug, PartialEq, SerializeConst)]
/// struct Struct {
/// a: u32,
/// b: u8,
/// c: u32,
/// }
///
/// let mut buffer = ConstVec::new();
/// buffer = serialize_const(&Struct {
/// a: 0x11111111,
/// b: 0x22,
/// c: 0x33333333,
/// }, buffer);
/// let buf = buffer.read();
/// assert_eq!(buf.as_ref(), &[0x11, 0x11, 0x11, 0x11, 0x22, 0x33, 0x33, 0x33, 0x33]);
/// ```
#[must_use = "The data is serialized into the returned buffer"]
pub const fn serialize_const<T: SerializeConst>(data: &T, to: ConstVec<u8>) -> ConstVec<u8> {
let ptr = data as *const T as *const ();
serialize_const_ptr(ptr, to, &T::MEMORY_LAYOUT)
}
/// Deserialize a primitive type into the out buffer at the offset passed in. Returns a new version of the buffer with the data added.
const fn deserialize_const_primitive<'a, const N: usize>(
mut from: ConstReadBuffer<'a>,
layout: &PrimitiveLayout,
out: (usize, [MaybeUninit<u8>; N]),
) -> Option<(ConstReadBuffer<'a>, [MaybeUninit<u8>; N])> {
let (start, mut out) = out;
let mut offset = 0;
while offset < layout.size {
// If the bytes are reversed, walk backwards from the end of the number when filling in bytes
let (from_new, value) = match from.get() {
Some(data) => data,
None => return None,
};
from = from_new;
if cfg!(any(target_endian = "big", feature = "test-big-endian")) {
out[start + layout.size - offset - 1] = MaybeUninit::new(value);
} else {
out[start + offset] = MaybeUninit::new(value);
}
offset += 1;
}
Some((from, out))
}
/// Deserialize a struct type into the out buffer at the offset passed in. Returns a new version of the buffer with the data added.
const fn deserialize_const_struct<'a, const N: usize>(
mut from: ConstReadBuffer<'a>,
layout: &StructLayout,
out: (usize, [MaybeUninit<u8>; N]),
) -> Option<(ConstReadBuffer<'a>, [MaybeUninit<u8>; N])> {
let (start, mut out) = out;
let mut i = 0;
while i < layout.data.len() {
// Deserialize the field at the offset pointer in the struct
let StructFieldLayout { offset, layout } = &layout.data[i];
let (new_from, new_out) = match deserialize_const_ptr(from, layout, (start + *offset, out))
{
Some(data) => data,
None => return None,
};
from = new_from;
out = new_out;
i += 1;
}
Some((from, out))
}
/// Deserialize an enum type into the out buffer at the offset passed in. Returns a new version of the buffer with the data added.
const fn deserialize_const_enum<'a, const N: usize>(
mut from: ConstReadBuffer<'a>,
layout: &EnumLayout,
out: (usize, [MaybeUninit<u8>; N]),
) -> Option<(ConstReadBuffer<'a>, [MaybeUninit<u8>; N])> {
let (start, mut out) = out;
let mut discriminant = 0;
// First, deserialize the discriminant
let mut offset = 0;
while offset < layout.discriminant.size {
// If the bytes are reversed, walk backwards from the end of the number when filling in bytes
let (from_new, value) = match from.get() {
Some(data) => data,
None => return None,
};
from = from_new;
if cfg!(target_endian = "big") {
out[start + layout.size - offset - 1] = MaybeUninit::new(value);
discriminant |= (value as u32) << ((layout.discriminant.size - offset - 1) * 8);
} else {
out[start + offset] = MaybeUninit::new(value);
discriminant |= (value as u32) << (offset * 8);
}
offset += 1;
}
// Then, deserialize the variant
let mut i = 0;
let mut matched_variant = false;
while i < layout.variants.len() {
// If the variant is the discriminated one, deserialize it
let EnumVariant { tag, data, .. } = &layout.variants[i];
if discriminant == *tag {
let offset = layout.variants_offset;
let (new_from, new_out) =
match deserialize_const_struct(from, data, (start + offset, out)) {
Some(data) => data,
None => return None,
};
from = new_from;
out = new_out;
matched_variant = true;
break;
}
i += 1;
}
if !matched_variant {
return None;
}
Some((from, out))
}
/// Deserialize a list type into the out buffer at the offset passed in. Returns a new version of the buffer with the data added.
const fn deserialize_const_list<'a, const N: usize>(
mut from: ConstReadBuffer<'a>,
layout: &ListLayout,
out: (usize, [MaybeUninit<u8>; N]),
) -> Option<(ConstReadBuffer<'a>, [MaybeUninit<u8>; N])> {
let (start, mut out) = out;
let len = layout.len;
let item_layout = layout.item_layout;
let mut i = 0;
while i < len {
let (new_from, new_out) =
match deserialize_const_ptr(from, item_layout, (start + i * item_layout.size(), out)) {
Some(data) => data,
None => return None,
};
from = new_from;
out = new_out;
i += 1;
}
Some((from, out))
}
/// Deserialize a type into the out buffer at the offset passed in. Returns a new version of the buffer with the data added.
const fn deserialize_const_ptr<'a, const N: usize>(
from: ConstReadBuffer<'a>,
layout: &Layout,
out: (usize, [MaybeUninit<u8>; N]),
) -> Option<(ConstReadBuffer<'a>, [MaybeUninit<u8>; N])> {
match layout {
Layout::Enum(layout) => deserialize_const_enum(from, layout, out),
Layout::Struct(layout) => deserialize_const_struct(from, layout, out),
Layout::List(layout) => deserialize_const_list(from, layout, out),
Layout::Primitive(layout) => deserialize_const_primitive(from, layout, out),
}
}
/// Deserialize a type into the output buffer. Accepts (Type, ConstVec<u8>) as input and returns Option<(ConstReadBuffer, Instance of type)>
///
/// # Example
/// ```rust
/// # use const_serialize::{deserialize_const, serialize_const, ConstVec, SerializeConst};
/// #[derive(Clone, Copy, Debug, PartialEq, SerializeConst)]
/// struct Struct {
/// a: u32,
/// b: u8,
/// c: u32,
/// d: u32,
/// }
///
/// let mut buffer = ConstVec::new();
/// buffer = serialize_const(&Struct {
/// a: 0x11111111,
/// b: 0x22,
/// c: 0x33333333,
/// d: 0x44444444,
/// }, buffer);
/// let buf = buffer.read();
/// assert_eq!(deserialize_const!(Struct, buf).unwrap().1, Struct {
/// a: 0x11111111,
/// b: 0x22,
/// c: 0x33333333,
/// d: 0x44444444,
/// });
/// ```
#[macro_export]
macro_rules! deserialize_const {
($type:ty, $buffer:expr) => {
unsafe {
const __SIZE: usize = std::mem::size_of::<$type>();
$crate::deserialize_const_raw::<__SIZE, $type>($buffer)
}
};
}
/// Deserialize a buffer into a type. This will return None if the buffer doesn't have enough data to fill the type.
/// # Safety
/// N must be `std::mem::size_of::<T>()`
#[must_use = "The data is deserialized from the input buffer"]
pub const unsafe fn deserialize_const_raw<const N: usize, T: SerializeConst>(
from: ConstReadBuffer,
) -> Option<(ConstReadBuffer, T)> {
// Create uninitized memory with the size of the type
let out = [MaybeUninit::uninit(); N];
// Fill in the bytes into the buffer for the type
let (from, out) = match deserialize_const_ptr(from, &T::MEMORY_LAYOUT, (0, out)) {
Some(data) => data,
None => return None,
};
// Now that the memory is filled in, transmute it into the type
Some((from, unsafe {
std::mem::transmute_copy::<[MaybeUninit<u8>; N], T>(&out)
}))
}
/// Check if the serialized representation of two items are the same
pub const fn serialize_eq<T: SerializeConst>(first: &T, second: &T) -> bool {
let first_serialized = ConstVec::<u8>::new();
let first_serialized = serialize_const(first, first_serialized);
let second_serialized = ConstVec::<u8>::new();
let second_serialized = serialize_const(second, second_serialized);
let first_buf = first_serialized.as_ref();
let second_buf = second_serialized.as_ref();
if first_buf.len() != second_buf.len() {
return false;
}
let mut i = 0;
while i < first_buf.len() {
if first_buf[i] != second_buf[i] {
return false;
}
i += 1;
}
true
}
packages/const-serialize/tests/enum.rs
+206
View File
@@ -0,0 +1,206 @@
use const_serialize::{deserialize_const, serialize_const, ConstVec, SerializeConst};
use std::mem::MaybeUninit;
#[test]
fn test_transmute_bytes_to_enum() {
#[derive(Clone, Copy, Debug, PartialEq)]
#[repr(C, u8)]
enum Enum<T> {
A { one: u32, two: u16 },
B { one: u8, two: T } = 15,
}
#[repr(C)]
#[derive(Debug, PartialEq)]
struct A {
one: u32,
two: u16,
}
#[repr(C)]
#[derive(Debug, PartialEq)]
struct B<T> {
one: u8,
two: T,
}
const SIZE: usize = std::mem::size_of::<Enum<u16>>();
let mut out = [MaybeUninit::uninit(); SIZE];
let discriminate_size = std::mem::size_of::<u8>();
let tag_align = 0;
let union_alignment = std::mem::align_of::<A>().max(std::mem::align_of::<B<u16>>());
let data_align = (discriminate_size / union_alignment) + union_alignment;
let a_one_align = std::mem::offset_of!(A, one);
let a_two_align = std::mem::offset_of!(A, two);
let b_one_align = std::mem::offset_of!(B<u16>, one);
let b_two_align = std::mem::offset_of!(B<u16>, two);
let one = 1234u32;
let two = 5678u16;
let first = Enum::A { one, two };
for (i, byte) in one.to_le_bytes().iter().enumerate() {
out[data_align + i + a_one_align] = MaybeUninit::new(*byte);
}
for (i, byte) in two.to_le_bytes().iter().enumerate() {
out[data_align + i + a_two_align] = MaybeUninit::new(*byte);
}
out[tag_align] = MaybeUninit::new(0);
let out = unsafe { std::mem::transmute_copy::<[MaybeUninit<u8>; SIZE], Enum<u16>>(&out) };
assert_eq!(out, first);
let mut out = [MaybeUninit::uninit(); SIZE];
let one = 123u8;
let two = 58u16;
let second = Enum::B { one, two };
for (i, byte) in one.to_le_bytes().iter().enumerate() {
out[data_align + i + b_one_align] = MaybeUninit::new(*byte);
}
for (i, byte) in two.to_le_bytes().iter().enumerate() {
out[data_align + i + b_two_align] = MaybeUninit::new(*byte);
}
out[tag_align] = MaybeUninit::new(15);
let out = unsafe { std::mem::transmute_copy::<[MaybeUninit<u8>; SIZE], Enum<u16>>(&out) };
assert_eq!(out, second);
}
#[test]
fn test_serialize_enum() {
#[derive(Clone, Copy, Debug, PartialEq, SerializeConst)]
#[repr(C, u8)]
enum Enum {
A { one: u32, two: u16 },
B { one: u8, two: u16 } = 15,
}
println!("{:#?}", Enum::MEMORY_LAYOUT);
let data = Enum::A {
one: 0x11111111,
two: 0x22,
};
let mut buf = ConstVec::new();
buf = serialize_const(&data, buf);
println!("{:?}", buf.as_ref());
let buf = buf.read();
assert_eq!(deserialize_const!(Enum, buf).unwrap().1, data);
let data = Enum::B {
one: 0x11,
two: 0x2233,
};
let mut buf = ConstVec::new();
buf = serialize_const(&data, buf);
println!("{:?}", buf.as_ref());
let buf = buf.read();
assert_eq!(deserialize_const!(Enum, buf).unwrap().1, data);
}
#[test]
fn test_serialize_u8_enum() {
#[derive(Clone, Copy, Debug, PartialEq, SerializeConst)]
#[repr(u8)]
enum Enum {
A,
B,
}
println!("{:#?}", Enum::MEMORY_LAYOUT);
let data = Enum::A;
let mut buf = ConstVec::new();
buf = serialize_const(&data, buf);
println!("{:?}", buf.as_ref());
let buf = buf.read();
assert_eq!(deserialize_const!(Enum, buf).unwrap().1, data);
let data = Enum::B;
let mut buf = ConstVec::new();
buf = serialize_const(&data, buf);
println!("{:?}", buf.as_ref());
let buf = buf.read();
assert_eq!(deserialize_const!(Enum, buf).unwrap().1, data);
}
#[test]
fn test_serialize_corrupted_enum() {
#[derive(Clone, Copy, Debug, PartialEq, SerializeConst)]
#[repr(C, u8)]
enum Enum {
A { one: u32, two: u16 },
}
let data = Enum::A {
one: 0x11111111,
two: 0x22,
};
let mut buf = ConstVec::new();
buf = serialize_const(&data, buf);
buf = buf.set(0, 2);
println!("{:?}", buf.as_ref());
let buf = buf.read();
assert_eq!(deserialize_const!(Enum, buf), None);
}
#[test]
fn test_serialize_nested_enum() {
#[derive(Clone, Copy, Debug, PartialEq, SerializeConst)]
#[repr(C, u8)]
enum Enum {
A { one: u32, two: u16 },
B { one: u8, two: InnerEnum } = 15,
}
#[derive(Clone, Copy, Debug, PartialEq, SerializeConst)]
#[repr(C, u16)]
enum InnerEnum {
A(u8),
B { one: u64, two: f64 } = 1000,
C { one: u32, two: u16 },
}
let data = Enum::A {
one: 0x11111111,
two: 0x22,
};
let mut buf = ConstVec::new();
buf = serialize_const(&data, buf);
println!("{:?}", buf.as_ref());
let buf = buf.read();
assert_eq!(deserialize_const!(Enum, buf).unwrap().1, data);
let data = Enum::B {
one: 0x11,
two: InnerEnum::A(0x22),
};
let mut buf = ConstVec::new();
buf = serialize_const(&data, buf);
println!("{:?}", buf.as_ref());
let buf = buf.read();
assert_eq!(deserialize_const!(Enum, buf).unwrap().1, data);
let data = Enum::B {
one: 0x11,
two: InnerEnum::B {
one: 0x2233,
two: 0.123456789,
},
};
let mut buf = ConstVec::new();
buf = serialize_const(&data, buf);
println!("{:?}", buf.as_ref());
let buf = buf.read();
assert_eq!(deserialize_const!(Enum, buf).unwrap().1, data);
let data = Enum::B {
one: 0x11,
two: InnerEnum::C {
one: 0x2233,
two: 56789,
},
};
let mut buf = ConstVec::new();
buf = serialize_const(&data, buf);
println!("{:?}", buf.as_ref());
let buf = buf.read();
assert_eq!(deserialize_const!(Enum, buf).unwrap().1, data);
}
packages/const-serialize/tests/eq.rs
+59
View File
@@ -0,0 +1,59 @@
use const_serialize::{serialize_eq, SerializeConst};
#[derive(Clone, Copy, Debug, PartialEq, SerializeConst)]
struct Struct {
a: u32,
b: u8,
c: u32,
d: Enum,
}
#[derive(Clone, Copy, Debug, PartialEq, SerializeConst)]
#[repr(C, u8)]
enum Enum {
A { one: u32, two: u16 },
B { one: u8, two: u16 } = 15,
}
#[test]
fn const_eq() {
const {
let data = [
Struct {
a: 0x11111111,
b: 0x22,
c: 0x33333333,
d: Enum::A {
one: 0x44444444,
two: 0x5555,
},
},
Struct {
a: 123,
b: 9,
c: 38,
d: Enum::B {
one: 0x44,
two: 0x555,
},
},
Struct {
a: 9,
b: 123,
c: 39,
d: Enum::B {
one: 0x46,
two: 0x555,
},
},
];
let mut other = data;
other[2].a += 1;
if serialize_eq(&data, &other) {
panic!("data should be different");
}
if !serialize_eq(&data, &data) {
panic!("data should be the same");
}
}
}
packages/const-serialize/tests/lists.rs
+34
View File
@@ -0,0 +1,34 @@
use const_serialize::{deserialize_const, serialize_const, ConstVec};
#[test]
fn test_serialize_const_layout_list() {
let mut buf = ConstVec::new();
buf = serialize_const(&[1u8, 2, 3] as &[u8; 3], buf);
println!("{:?}", buf.as_ref());
let buf = buf.read();
assert_eq!(deserialize_const!([u8; 3], buf).unwrap().1, [1, 2, 3])
}
#[test]
fn test_serialize_const_layout_nested_lists() {
let mut buf = ConstVec::new();
buf = serialize_const(
&[[1u8, 2, 3], [4u8, 5, 6], [7u8, 8, 9]] as &[[u8; 3]; 3],
buf,
);
println!("{:?}", buf.as_ref());
let buf = buf.read();
assert_eq!(
deserialize_const!([[u8; 3]; 3], buf).unwrap().1,
[[1, 2, 3], [4, 5, 6], [7, 8, 9]]
);
}
#[test]
fn test_serialize_list_too_little_data() {
let mut buf = ConstVec::new();
buf = buf.push(1);
let buf = buf.read();
assert_eq!(deserialize_const!([u64; 10], buf), None);
}
packages/const-serialize/tests/primitive.rs
+71
View File
@@ -0,0 +1,71 @@
use const_serialize::{deserialize_const, serialize_const, ConstVec};
#[test]
fn test_serialize_const_layout_primitive() {
let mut buf = ConstVec::new();
buf = serialize_const(&1234u32, buf);
if cfg!(feature = "test-big-endian") {
assert_eq!(buf.as_ref(), 1234u32.to_be_bytes());
} else {
assert_eq!(buf.as_ref(), 1234u32.to_le_bytes());
}
let buf = buf.read();
assert_eq!(deserialize_const!(u32, buf).unwrap().1, 1234u32);
let mut buf = ConstVec::new();
buf = serialize_const(&1234u64, buf);
if cfg!(feature = "test-big-endian") {
assert_eq!(buf.as_ref(), 1234u64.to_be_bytes());
} else {
assert_eq!(buf.as_ref(), 1234u64.to_le_bytes());
}
let buf = buf.read();
assert_eq!(deserialize_const!(u64, buf).unwrap().1, 1234u64);
let mut buf = ConstVec::new();
buf = serialize_const(&1234i32, buf);
if cfg!(feature = "test-big-endian") {
assert_eq!(buf.as_ref(), 1234i32.to_be_bytes());
} else {
assert_eq!(buf.as_ref(), 1234i32.to_le_bytes());
}
let buf = buf.read();
assert_eq!(deserialize_const!(i32, buf).unwrap().1, 1234i32);
let mut buf = ConstVec::new();
buf = serialize_const(&1234i64, buf);
if cfg!(feature = "test-big-endian") {
assert_eq!(buf.as_ref(), 1234i64.to_be_bytes());
} else {
assert_eq!(buf.as_ref(), 1234i64.to_le_bytes());
}
let buf = buf.read();
assert_eq!(deserialize_const!(i64, buf).unwrap().1, 1234i64);
let mut buf = ConstVec::new();
buf = serialize_const(&true, buf);
assert_eq!(buf.as_ref(), [1u8]);
let buf = buf.read();
assert!(deserialize_const!(bool, buf).unwrap().1);
let mut buf = ConstVec::new();
buf = serialize_const(&0.631f32, buf);
if cfg!(feature = "test-big-endian") {
assert_eq!(buf.as_ref(), 0.631f32.to_be_bytes());
} else {
assert_eq!(buf.as_ref(), 0.631f32.to_le_bytes());
}
let buf = buf.read();
assert_eq!(deserialize_const!(f32, buf).unwrap().1, 0.631);
}
#[test]
fn test_serialize_primitive_too_little_data() {
let mut buf = ConstVec::new();
buf = buf.push(1);
buf = buf.push(1);
buf = buf.push(1);
let buf = buf.read();
assert_eq!(deserialize_const!(u64, buf), None);
}
packages/const-serialize/tests/str.rs
+40
View File
@@ -0,0 +1,40 @@
use const_serialize::{deserialize_const, serialize_const, ConstStr, ConstVec};
#[test]
fn test_serialize_const_layout_str() {
let mut buf = ConstVec::new();
let str = ConstStr::new("hello");
buf = serialize_const(&str, buf);
println!("{:?}", buf.as_ref());
let buf = buf.read();
assert_eq!(
deserialize_const!(ConstStr, buf).unwrap().1.as_str(),
"hello"
);
}
#[test]
fn test_serialize_const_layout_nested_str() {
let mut buf = ConstVec::new();
let str = ConstStr::new("hello");
buf = serialize_const(&[str, str, str] as &[ConstStr; 3], buf);
println!("{:?}", buf.as_ref());
let buf = buf.read();
assert_eq!(
deserialize_const!([ConstStr; 3], buf).unwrap().1,
[
ConstStr::new("hello"),
ConstStr::new("hello"),
ConstStr::new("hello")
]
);
}
#[test]
fn test_serialize_str_too_little_data() {
let mut buf = ConstVec::new();
buf = buf.push(1);
let buf = buf.read();
assert_eq!(deserialize_const!(ConstStr, buf), None);
}
packages/const-serialize/tests/structs.rs
+164
View File
@@ -0,0 +1,164 @@
use const_serialize::{deserialize_const, serialize_const, ConstVec, SerializeConst};
use std::mem::MaybeUninit;
#[test]
fn test_transmute_bytes_to_struct() {
struct MyStruct {
a: u32,
b: u8,
c: u32,
d: u32,
}
const SIZE: usize = std::mem::size_of::<MyStruct>();
let mut out = [MaybeUninit::uninit(); SIZE];
let first_align = std::mem::offset_of!(MyStruct, a);
let second_align = std::mem::offset_of!(MyStruct, b);
let third_align = std::mem::offset_of!(MyStruct, c);
let fourth_align = std::mem::offset_of!(MyStruct, d);
for (i, byte) in 1234u32.to_le_bytes().iter().enumerate() {
out[i + first_align] = MaybeUninit::new(*byte);
}
for (i, byte) in 12u8.to_le_bytes().iter().enumerate() {
out[i + second_align] = MaybeUninit::new(*byte);
}
for (i, byte) in 13u32.to_le_bytes().iter().enumerate() {
out[i + third_align] = MaybeUninit::new(*byte);
}
for (i, byte) in 14u32.to_le_bytes().iter().enumerate() {
out[i + fourth_align] = MaybeUninit::new(*byte);
}
let out = unsafe { std::mem::transmute_copy::<[MaybeUninit<u8>; SIZE], MyStruct>(&out) };
assert_eq!(out.a, 1234);
assert_eq!(out.b, 12);
assert_eq!(out.c, 13);
assert_eq!(out.d, 14);
}
#[test]
fn test_serialize_const_layout_struct_list() {
#[derive(Clone, Copy, Debug, PartialEq, SerializeConst)]
struct Struct {
a: u32,
b: u8,
c: u32,
d: u32,
}
impl Struct {
#[allow(dead_code)]
const fn equal(&self, other: &Struct) -> bool {
self.a == other.a && self.b == other.b && self.c == other.c && self.d == other.d
}
}
#[derive(Clone, Copy, Debug, PartialEq, SerializeConst)]
struct OtherStruct {
a: u32,
b: u8,
c: Struct,
d: u32,
}
impl OtherStruct {
#[allow(dead_code)]
const fn equal(&self, other: &OtherStruct) -> bool {
self.a == other.a && self.b == other.b && self.c.equal(&other.c) && self.d == other.d
}
}
const INNER_DATA: Struct = Struct {
a: 0x11111111,
b: 0x22,
c: 0x33333333,
d: 0x44444444,
};
const DATA: [OtherStruct; 3] = [
OtherStruct {
a: 0x11111111,
b: 0x22,
c: INNER_DATA,
d: 0x44444444,
},
OtherStruct {
a: 0x111111,
b: 0x23,
c: INNER_DATA,
d: 0x44444444,
},
OtherStruct {
a: 0x11111111,
b: 0x11,
c: INNER_DATA,
d: 0x44441144,
},
];
const _ASSERT: () = {
let mut buf = ConstVec::new();
buf = serialize_const(&DATA, buf);
let buf = buf.read();
let [first, second, third] = match deserialize_const!([OtherStruct; 3], buf) {
Some((_, data)) => data,
None => panic!("data mismatch"),
};
if !(first.equal(&DATA[0]) && second.equal(&DATA[1]) && third.equal(&DATA[2])) {
panic!("data mismatch");
}
};
const _ASSERT_2: () = {
let mut buf = ConstVec::new();
const DATA_AGAIN: [[OtherStruct; 3]; 3] = [DATA, DATA, DATA];
buf = serialize_const(&DATA_AGAIN, buf);
let buf = buf.read();
let [first, second, third] = match deserialize_const!([[OtherStruct; 3]; 3], buf) {
Some((_, data)) => data,
None => panic!("data mismatch"),
};
if !(first[0].equal(&DATA[0]) && first[1].equal(&DATA[1]) && first[2].equal(&DATA[2])) {
panic!("data mismatch");
}
if !(second[0].equal(&DATA[0]) && second[1].equal(&DATA[1]) && second[2].equal(&DATA[2])) {
panic!("data mismatch");
}
if !(third[0].equal(&DATA[0]) && third[1].equal(&DATA[1]) && third[2].equal(&DATA[2])) {
panic!("data mismatch");
}
};
let mut buf = ConstVec::new();
buf = serialize_const(&DATA, buf);
println!("{:?}", buf.as_ref());
let buf = buf.read();
let (_, data2) = deserialize_const!([OtherStruct; 3], buf).unwrap();
assert_eq!(DATA, data2);
}
#[test]
fn test_serialize_const_layout_struct() {
#[derive(Debug, PartialEq, SerializeConst)]
struct Struct {
a: u32,
b: u8,
c: u32,
d: u32,
}
#[derive(Debug, PartialEq, SerializeConst)]
struct OtherStruct(u32, u8, Struct, u32);
println!("{:?}", OtherStruct::MEMORY_LAYOUT);
let data = Struct {
a: 0x11111111,
b: 0x22,
c: 0x33333333,
d: 0x44444444,
};
let data = OtherStruct(0x11111111, 0x22, data, 0x44444444);
let mut buf = ConstVec::new();
buf = serialize_const(&data, buf);
println!("{:?}", buf.as_ref());
let buf = buf.read();
let (_, data2) = deserialize_const!(OtherStruct, buf).unwrap();
assert_eq!(data, data2);
}
packages/const-serialize/tests/tuples.rs
+28
View File
@@ -0,0 +1,28 @@
use const_serialize::{deserialize_const, serialize_const, ConstVec};
#[test]
fn test_serialize_const_layout_tuple() {
let mut buf = ConstVec::new();
buf = serialize_const(&(1234u32, 5678u16), buf);
let buf = buf.read();
assert_eq!(
deserialize_const!((u32, u16), buf).unwrap().1,
(1234u32, 5678u16)
);
let mut buf = ConstVec::new();
buf = serialize_const(&(1234f64, 5678u16, 90u8), buf);
let buf = buf.read();
assert_eq!(
deserialize_const!((f64, u16, u8), buf).unwrap().1,
(1234f64, 5678u16, 90u8)
);
let mut buf = ConstVec::new();
buf = serialize_const(&(1234u32, 5678u16, 90u8, 1000000f64), buf);
let buf = buf.read();
assert_eq!(
deserialize_const!((u32, u16, u8, f64), buf).unwrap().1,
(1234u32, 5678u16, 90u8, 1000000f64)
);
}